Method and apparatus for generating recommendation question based on large model, electronic device and medium

Abstract

The present disclosure provides a large model-based recommendation question generation method and device, electronic equipment and medium, which relates to the technical field of artificial intelligence, and in particular to the technical fields of natural language processing, large language model and intelligent customer service. The specific implementation scheme is: receiving a current question input by a user, and generating a current answer for the current question. Then, the historical questions input by the user, the historical answers generated for the historical questions, the current question and the current answer are used as context information. Then, a large language model is used to generate a recommended question based on the context information. The diversity of the recommended question is enhanced.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the fields of natural language processing, large language models, and intelligent customer service technology. Background Art

[0002] In a multi-round conversation scenario, after an answer is generated for a user's question, the user may ask again. To facilitate the user's repeated questions, recommended questions can be provided to the user so that the user can directly select the recommended question to ask. Summary of the Invention

[0003] The present disclosure provides a method, device, electronic device, and medium for generating recommendation questions based on a large model.

[0004] A first aspect of the embodiments of the present disclosure provides a method for generating recommendation questions based on a large model, comprising:

[0005] Receive user input for the current question;

[0006] generating a current answer to the current question;

[0007] Using historical questions input by the user, historical answers generated for the historical questions, the current question, and the current answer as context information;

[0008] A large language model is used to generate recommendation questions based on the context information.

[0009] A second aspect of the embodiments of the present disclosure provides a device for generating recommendation questions based on a large model, comprising:

[0010] A receiving module is used to receive the current question input by the user;

[0011] a generating module, configured to generate a current answer to the current question received by the receiving module;

[0012] a determination module configured to use, as context information, historical questions input by a user, historical answers generated for the historical questions, the current question received by the receiving module, and the current answer generated by the generating module;

[0013] The generating module is further configured to generate a recommendation question based on the context information determined by the determining module using a large language model.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, including:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any one of the methods according to the first aspect.

[0018] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to any one of the first aspects.

[0019] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of the first aspects is implemented.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0022] Figure 1 This is a flowchart of a method for generating recommendation questions based on a large model provided by an embodiment of the present disclosure;

[0023] Figure 2 is an exemplary schematic diagram of a conversation interface provided by an embodiment of the present disclosure;

[0024] Figure 3 This is a flowchart of a method for generating recommendation questions using a large language model provided by an embodiment of the present disclosure;

[0025] Figure 4 is an exemplary schematic diagram of another dialogue interface provided by an embodiment of the present disclosure;

[0026] Figure 5 This is a flowchart of a large language model training method provided by an embodiment of the present disclosure;

[0027] Figure 6 is a flowchart of another method for generating recommendation questions based on a large model provided by an embodiment of the present disclosure;

[0028] Figure 7 is a structural diagram of a large-model-based recommendation question generation device provided by an embodiment of the present disclosure;

[0029] Figure 8is a block diagram of an electronic device for implementing a large model-based recommended question generation method according to an embodiment of the disclosure. DETAILED DESCRIPTION

[0030] Exemplary embodiments of the disclosure are described herein with reference to the accompanying drawings, which are included to provide a thorough understanding of embodiments of the disclosure and, as such, the exemplary embodiments should not be construed as limiting the scope of the disclosure. It will be understood that various changes in form and details can be made to the embodiments described herein without departing from the spirit and scope of the disclosure. As such, the exemplary embodiments should not be construed as limiting the scope of the disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0031] At present, a recommended question generation method based on a question library is generally used. The method needs to pre-construct a question library, which includes a plurality of questions pre-set by experts. When a question input by a user is received, a question similar to the question input by the user is retrieved from the question library as a recommended question based on a text similarity matching algorithm. However, this method can only select a question from the question library as a recommended question, which limits the diversity of recommended questions.

[0032] To improve the diversity of recommended questions, a large model-based recommended question generation method is provided in the embodiments of the disclosure. The method is applied to an electronic device, for example, the electronic device can be a server, a desktop computer, or a notebook computer, and the like, which has a data processing capability. As shown in Figure 1 The large model-based recommended question generation method provided by the embodiments of the disclosure includes the following steps:

[0033] S101, receiving a current question input by a user.

[0034] When the electronic device is a server, the terminal can display a dialogue interface through a browser, an application (APP), or a mini-program, and the like, and the user can input the current question in the dialogue interface of the terminal. After receiving the current question input by the user, the terminal sends the current question to the electronic device, so that the electronic device receives the current question.

[0035] Alternatively, when the electronic device is a terminal such as a desktop computer or a notebook computer, the electronic device can display a dialogue interface through a browser, an APP, or a mini-program, and the like, and the user can input the current question in the dialogue interface of the electronic device.

[0036] S102, generating a current answer to the current question.

[0037] The electronic device can input the current question into a large language model (LLM) and receive the current answer from the LLM. The LLM is a model trained by a deep learning model, such as a transformer, that has the ability to understand and generate natural language text.

[0038] When the electronic device is a server, the electronic device may also send the current answer to the terminal so that the terminal displays the current answer on the dialogue interface. When the electronic device is a terminal, the electronic device may display the current answer on the dialogue interface.

[0039] S103: Use the historical questions input by the user, the historical answers generated for the historical questions, the current question, and the current answer as context information.

[0040] The method of generating historical answers to historical questions is the same as that of generating current answers to current questions. Please refer to the relevant description of S102 and will not be repeated here.

[0041] S104: Generate recommendation questions based on context information using a large language model.

[0042] Optionally, the large language model for generating the recommended question and the large language model for generating the current answer in S102 may be the same large language model or different large language models, which is not specifically limited in the embodiments of the present disclosure.

[0043] When the electronic device is a server, the electronic device may also send a recommended question to the terminal so that the terminal displays the current answer on the dialogue interface. When the electronic device is a terminal, the electronic device may display the recommended question on the dialogue interface.

[0044] Through the above method, the disclosed embodiments can utilize a large language model to generate recommended questions based on historical questions, historical answers, current questions, and current answers. Because the disclosed embodiments can generate recommended questions in real time based on a large language model, recommended questions are no longer limited to a fixed question library. Furthermore, thanks to the large language model's powerful dialogue generation and generalization capabilities, the recommended questions generated using the large language model are more diverse.

[0045] The following is a detailed description of the method for generating recommendation questions based on a large model provided by an embodiment of the present disclosure.

[0046] In some embodiments of the present disclosure, the above S103 uses historical questions input by the user, historical answers generated for historical questions, current questions and current answers as context information, which can be implemented as follows: from the conversations included in the current session, a preset number of conversations are obtained in order of timestamps from late to early as context information.

[0047] The electronic device may search for conversations included in the current conversation based on a conversation identifier (ID) of the current conversation from a storage space for storing conversations.

[0048] Each dialog consists of a question entered by the user and an answer generated for the question. For example, Figure 2 A conversation interface provided for an electronic device includes three conversations from the current session. Conversation 1 includes question 1 and answer 1 to question 1, conversation 2 includes question 2 and answer 2 to question 2, and conversation 3 includes question 3 and answer 3 to question 3. Assuming that the conversation interface sorts the conversations in the current session from earliest to latest, conversation 3 is the current conversation, and conversation 1 and conversation 2 are each a previous conversation.

[0049] Each conversation has a corresponding timestamp, which can indicate the time when the question included in the conversation was received, or the time when the answer included in the conversation was generated, etc. The electronic device can retrieve a preset number of conversations from the conversations included in the current conversation found in descending order of timestamps as context information. The current conversation includes: a current conversation consisting of a current question and a current answer, and at least one historical conversation consisting of a historical question and a historical answer. The preset number is greater than or equal to 2, so that the preset number of conversations includes at least: the current conversation and the previous historical conversation of the current conversation.

[0050] For example, see Figure 2 , sorting the conversations of the current session in order of timestamp from latest to earliest, we get: conversation 3, conversation 2, conversation 1. Assuming the preset number is 2, we get question 3, answer 3, question 2, and answer 2 as context information. Figure 2 The “…” in the text indicates that the content is omitted to save space. In actual application scenarios, the complete answer can be displayed in the dialogue interface.

[0051] Because users' focus may vary over time, conversations within the same session generally have similar timestamps, resulting in closer focus. Therefore, the disclosed embodiment retrieves a preset number of conversations from the current session, sorted by timestamp from latest to earliest. This ensures the usefulness of contextual information and reduces the impact of useless information on recommendation generation.

[0052] It should be noted that if the current conversation only includes the current question and the current answer, and does not include any historical conversations, the electronic device may use the current question and the current answer as context information. Alternatively, in this case, the electronic device may retrieve (N-1) historical conversations from the user's historical conversations prior to the current conversation, in descending order of timestamps, where N is a preset number, and use the current question, the current answer, and the historical questions and answers included in the (N-1) historical conversations as context information.

[0053] On the other hand, the current session includes the current conversation and at least one historical conversation, but the number of conversations included in the current session is M <N的情况下,电子设备可以仅将当前会话包括的当前问题、当前回答、历史问题和历史回答,作为上下文信息。或者,在该情况下,电子设备可以从当前会话之前该用户的历史会话中,按照时间戳从晚到早的顺序,获取(N-M)个历史对话,并将当前会话包括的当前问题、当前回答、历史问题和历史回答,以及(N-M)个历史对话包括的历史问题和历史回答,作为上下文信息。

[0054] Because conversations with the same user over different periods may cover similar areas, for example, conversations with the same user in different sessions may all cover science, education, or entertainment, the conversations within different sessions of the same user can, to a certain extent, reflect the user's preferred areas and questioning habits, providing a reference for generating recommended questions. Therefore, the contextual information in the disclosed embodiments can include conversations from both the current session and past sessions, thereby increasing the relevance of recommended questions to existing conversations and improving the quality of recommended questions.

[0055] It should be noted that, when the current session only includes the current conversation but does not include historical conversations, and there is no historical conversation of the user, the electronic device may use the current question and the current answer as context information.

[0056] In some embodiments of the present disclosure, see Figure 3 The above-mentioned S104 electronic device generates a recommendation question based on context information using a large language model, including the following steps:

[0057] S301: Fill in a preset prompt template based on context information to obtain prompt information.

[0058] For example, the default prompt template is:

[0059] Please generate recommended questions based on the current conversation and historical conversations.

[0060] Current conversation:

[0061] Historical Dialogue:

[0062] ”

[0063] Combine Figure 2 Assuming that the current conversation in the context information includes question 3 and answer 3, and the historical conversation includes question 2 and answer 2, then based on the context information, fill in the preset prompt template and the resulting prompt information is:

[0064] Please generate recommended questions based on the current conversation and historical conversations.

[0065] Current conversation: "Who invented printing?" "Printing was invented by Bi Sheng. Bi Sheng was an inventor during the Northern Song Dynasty. He invented movable type printing, which was about 400 years earlier than lead movable type printing."

[0066] Historical Dialogue: "Who invented papermaking?" "The inventor of papermaking was Cai Lun. During the Eastern Han Dynasty... This type of paper is called 'Cai Hou Paper'."

[0067] ”

[0068] The prompt information may be referred to as a prompt.

[0069] S302: Input the prompt information into the large language model to obtain the recommended question output by the large language model.

[0070] Through the above method, the disclosed embodiment can fill in the prompt template using contextual information to obtain prompt information. This prompt information then guides the large language model to generate recommended questions that the user may ask based on the current and historical conversations. Furthermore, because the disclosed embodiment utilizes the large language model to generate recommended questions in real time, it avoids the situation where a user's input question does not match any of the questions in the question library, resulting in an inability to generate recommended questions.

[0071] In some embodiments of the present disclosure, users are also allowed to set constraint parameters for recommended questions, wherein the constraint parameters include at least one of the number of recommended questions, word range, type, and relevance to context information.

[0072] Among them, types include: open-ended, heuristic, and inquiring, etc. Types can also be called questioning styles.

[0073] The relevance of context information includes: strong relevance, weak relevance, and no relevance.

[0074] For example, the constraint parameters include: number of recommended questions: 3; word range: 0 to 100; type: open-ended; relevance: weak correlation.

[0075] On this basis, the above S301 electronic device fills in the preset prompt template based on the context information to obtain prompt information, which can be implemented as follows: obtaining preset constraint parameters, filling the context information and constraint parameters into the preset prompt template to obtain prompt information.

[0076] For example, the default prompt template is:

[0077] Please generate recommendation questions based on the current conversation, historical conversations, and constraint parameters.

[0078] Current conversation:

[0079] Historical Dialogue:

[0080] Constraint parameters:

[0081] ”

[0082] Combine Figure 2 Assuming that the current conversation in the context information includes question 3 and answer 3, and the historical conversation includes question 2 and answer 2, then based on the context information and constraint parameters, fill in the preset prompt template and the resulting prompt information is:

[0083] Please generate recommended questions based on the current conversation and historical conversations.

[0084] Current conversation: "Who invented printing?" "Printing was invented by Bi Sheng. Bi Sheng was an inventor during the Northern Song Dynasty. He invented movable type printing, which was about 400 years earlier than lead movable type printing."

[0085] Historical Dialogue: "Who invented papermaking?" "The inventor of papermaking was Cai Lun. During the Eastern Han Dynasty... This type of paper is called 'Cai Hou Paper'."

[0086] Constraint parameters: 2, 0-100, open, weak correlation.

[0087] ”

[0088] See also Figure 4 After inputting the above prompt information into the large language model, the large language model generates two recommended questions: "Bi Sheng's life" and "Who invented the compass?" Users can then quickly ask questions by clicking on the recommended questions.

[0089] The disclosed embodiment also supports users to set constraint parameters. On this basis, the above-mentioned method of obtaining preset constraint parameters includes the following two cases:

[0090] Case 1: If there are constraint parameters input by the user, the constraint parameters most recently input by the user are obtained as the preset constraint parameters.

[0091] Case 2: If there are no constraint parameters input by the user, the default constraint parameters are obtained as the preset constraint parameters.

[0092] As can be seen, the disclosed embodiments allow users to set constraint parameters for recommendation questions, making the generation of recommendation questions more flexible and more responsive to user needs, significantly improving the user experience. Furthermore, if the user does not set constraint parameters, default constraint parameters can be used to generate recommendation questions that meet common user needs.

[0093] Moreover, the disclosed embodiments can constrain the generated recommendation questions by constraining parameters, thereby providing a direction for generating recommendation questions for large language models, making the recommendation questions more targeted.

[0094] In the disclosed embodiment, because the context information includes multiple rounds of dialogue, the context information may be lengthy. Therefore, before filling in the preset prompt template based on the context information and obtaining the prompt information in step S301, the electronic device may further determine whether the number of words in the context information is less than or equal to a preset number of words. The preset number of words can be set based on actual requirements such as the maximum input word count of the large language model and the delay in generating the recommended question.

[0095] If so, execute S301 to fill in the preset prompt template based on the context information to obtain prompt information.

[0096] If not, some content is deleted from the historical questions and answers included in the context information to obtain updated context information. Then, based on the updated context information, a preset prompt template is filled in to obtain prompt information. The number of words in the updated context information is less than or equal to the preset number of words. Filling in the preset prompt template based on the updated context information is the same as filling in the preset prompt template based on context information described above, and reference is made to the above description and will not be repeated here.

[0097] When the context information is shorter than the preset word count, the prompt information generated based on the context information is generally not too long, and the prompt length is likely to fall within the input limit of the large language model. Furthermore, because the prompt information is shorter, the large language model uses less computational effort to generate recommendation questions based on the prompt information, ensuring efficient generation of recommendation questions.

[0098] When the number of words in the context information exceeds the preset number, the prompt information generated based on the context information may be too long. If the prompt information is generated directly based on the context information, the length of the prompt information may exceed the input limit of the large language model. Therefore, the embodiment of the present disclosure deletes some content from the historical questions and historical answers included in the context information, thereby reducing the number of words included in the context information and reducing the possibility that the length of the prompt information exceeds the input limit of the large language model. Moreover, by reducing the number of words in the prompt information, the computational complexity of the large language model in generating recommended questions based on the prompt information can also be reduced, thereby improving generation efficiency.

[0099] Moreover, compared with the current question and the current answer, the reference value of historical questions and historical answers for recommended questions is lower. Therefore, some content is deleted from the historical questions and historical answers to ensure the reference value of contextual information for recommended questions.

[0100] In some embodiments of the present disclosure, the electronic device deletes part of the content from the historical questions and historical answers included in the context information to obtain updated context information, which can be implemented as follows: intercepting a preset number of words in the context information in order of timestamps from late to early, and deleting other content to obtain updated context information.

[0101] That is, the electronic device can calculate the sum of the number of words in each conversation included in the context information and the current cumulative word count, in descending order of timestamps, and use the calculated result as the current cumulative word count, where the initial value of the current cumulative word count is zero. It then determines whether the current cumulative word count is less than a preset number of words. If so, the electronic device returns to the step of calculating the sum of the number of words in the previous conversation and the current cumulative word count. If not, the electronic device calculates the difference between the current cumulative word count and the preset number of words, deletes the words equal to the difference from the conversation, and deletes all conversations before the conversation, so that the number of words in the context information retained after the deletion is the preset number of words.

[0102] The preset word count is greater than the sum of the word counts of the current question and the current answer. That is, the preset word count is a large value so that the updated context information includes the current conversation and part of the historical conversation.

[0103] Since the earlier the timestamp, the lower the correlation between the historical conversation content and the current conversation content, and the lower the reference value for generating recommendation questions, the embodiment of the present disclosure deletes the content in historical conversations with earlier timestamps to ensure the reference value of the context information obtained after the update for generating recommendation questions.

[0104] The disclosed embodiment can also train a large language model, and the training process can also be called supervised fine-tuning (SFT). Figure 5As shown in the figure, the training process of the large language model includes the following steps:

[0105] S501: Acquire multiple pieces of sample prompt information and obtain a training label for each piece of sample prompt information. The training label represents a standard recommendation question generated based on the sample prompt information. The standard recommendation question may be manually labeled.

[0106] The electronic device can obtain sample prompt information through the following method 1 and / or method 2.

[0107] Method 1: Use each historical prompt message within a preset time period as a sample prompt message. That is, use each historical prompt message within a preset time period as a sample prompt message.

[0108] The historical prompt information refers to the prompt information generated in actual application scenarios to generate recommendation questions.

[0109] Figure 5 The training process shown can be performed once every cycle, in which case the preset time period can be the previous cycle. Alternatively, the preset time period can be set according to actual needs.

[0110] Method 2: Obtain multiple historical conversation information, where each historical conversation information includes multiple conversations in a historical session, and obtain multiple constraint parameters, and then combine each historical conversation information with a constraint parameter to form sample prompt information.

[0111] Among them, different constraint parameters have at least one parameter that is different.

[0112] The electronic device may randomly select a piece of historical conversation information and a constraint parameter each time to combine them into a sample prompt message, and obtain multiple sample prompt messages through multiple random combinations. Alternatively, the electronic device may combine each piece of historical conversation information with each constraint parameter to form a sample prompt message.

[0113] Through the above method, the disclosed embodiments can use historical prompt information obtained in actual application scenarios as sample prompt information, ensuring the authenticity of the sample prompt information and making the large language model trained with it more suitable for actual application scenarios. Furthermore, the disclosed embodiments can combine each historical conversation information with a constraint parameter into a sample prompt information, ensuring the diversity of the sample prompt information and making the large language model trained with it more generalizable.

[0114] S502: Input the sample prompt information into the large language model to obtain the predicted recommendation question output by the large language model.

[0115] In each round of iteration, a batch of sample prompt information can be selected from the sample set, and each sample prompt information in the batch can be input into the large language model respectively to obtain the prediction recommendation problem output by the large language model.

[0116] S503: Determine a loss function value based on the predicted recommendation problem and the training labels.

[0117] In each iteration, for a batch of sample prompts, the prediction recommendation questions generated by the large language model and the training labels of each sample prompt in the batch are substituted into a preset loss function to calculate the loss function value. For example, the preset loss function can be a mean squared error loss function or a cross entropy loss function.

[0118] S504. Adjust the model parameters of the large language model based on the loss function value, and return to S502 until the large language model converges to obtain an updated large language model.

[0119] The model parameters of the large language model can be adjusted based on the loss function value using a gradient descent algorithm using backpropagation. If the large language model has not converged, the next iteration is entered, i.e., step S502 is returned to select the next batch of sample prompt information from the sample set and continue training the large language model based on the selected sample prompt information. When the large language model converges, the current large language model is used as the updated large language model.

[0120] Optionally, it is possible to determine whether the number of iterations reaches a preset number; if so, it is determined that the large language model has converged; otherwise, it is determined that the large language model has not converged. Alternatively, it is possible to determine whether the currently calculated loss function value is less than a preset threshold; if so, it is determined that the large language model has converged; otherwise, it is determined that the large language model has not converged. Alternatively, it is possible to determine whether the error between the loss function values ​​calculated the most recently preset number of times is less than a preset error; if so, it is determined that the large language model has converged; otherwise, it is determined that the large language model has not converged. Alternatively, it is also possible to determine whether the large language model has converged by other means, which is not specifically limited in the embodiments of the present disclosure.

[0121] Through the above method, the embodiment of the present disclosure can supervise and fine-tune the large language model, thereby continuously improving the prediction accuracy of the large language model during the training process, and continuously improving the generation effect of the large language model on recommendation questions, thereby bringing a more immersive user experience.

[0122] See also Figure 6 The following describes the overall process of the method for generating recommendation questions based on a large model provided by an embodiment of the present disclosure:

[0123] S601: Receive a current question input by a user.

[0124] S602: Generate a current answer to the current question.

[0125] S603: Obtain a preset number of conversations from the conversations included in the current session in descending order of timestamps as context information.

[0126] S604: Fill the context information and constraint parameters into a preset prompt template to obtain prompt information.

[0127] S605: Input the prompt information into the large language model to obtain the recommended question output by the large language model.

[0128] S606: After each cycle, each historical prompt information obtained in the previous cycle is used as a sample prompt information, and a training label of the sample prompt information is obtained to obtain a sample set, wherein the sample set includes each sample prompt information and a training label of each sample prompt information.

[0129] S607: Use the sample set to perform supervised fine-tuning on the large language model to obtain an updated large language model.

[0130] Figure 6 The specific implementation methods of each step can refer to the above description and will not be repeated here.

[0131] When generating recommended questions, the disclosed embodiment comprehensively considers the current conversation and historical conversations, thereby comprehensively analyzing the user's questioning preferences, increasing the probability of the recommended questions being selected by the user, and bringing the user a better conversation experience.

[0132] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of conversation information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0133] It should be noted that the large language model in the embodiment of the present disclosure is not a large language model for a specific user and cannot reflect the personal information of a specific user. In addition, the sample set in the embodiment of the present disclosure can come from a public data set.

[0134] Based on the same inventive concept, corresponding to the above method embodiment, the embodiment of the present disclosure also provides a recommendation question generation device based on a large model, such as Figure 7 As shown, the device includes: a receiving module 701, a generating module 702 and a determining module 703;

[0135] Receiving module 701, for receiving a current question input by a user;

[0136] A generating module 702 is configured to generate a current answer to the current question received by the receiving module 701;

[0137] The determination module 703 is configured to use the historical questions input by the user, the historical answers generated for the historical questions, the current question received by the receiving module 701, and the current answer generated by the generating module 702 as context information;

[0138] The generating module 702 is further configured to generate a recommendation question using the large language model based on the context information determined by the determining module 703 .

[0139] In some embodiments of the present disclosure, the determining module 703 is specifically configured to:

[0140] From the conversations included in the current session, a preset number of conversations are obtained in descending order of timestamps as context information, where the conversations include questions input by the user and answers generated for the questions, and the preset number is greater than or equal to 2.

[0141] In some embodiments of the present disclosure, the generating module 702 is specifically configured to:

[0142] Based on the context information, fill in the preset prompt template to obtain prompt information;

[0143] Input the prompt information into the large language model and obtain the recommended question output by the large language model.

[0144] In some embodiments of the present disclosure, the generating module 702 is specifically configured to:

[0145] Obtaining preset constraint parameters, where the constraint parameters include: at least one of the number of recommended questions, word range, type, and relevance to context information;

[0146] Fill the context information and constraint parameters into the preset prompt template to obtain prompt information.

[0147] In some embodiments of the present disclosure, the generating module 702 is specifically configured to:

[0148] If there are constraint parameters input by the user, the constraint parameters most recently input by the user are obtained as the preset constraint parameters; or,

[0149] If there are no constraint parameters input by the user, the default constraint parameters are obtained as the preset constraint parameters.

[0150] In some embodiments of the present disclosure, the apparatus further comprises:

[0151] The determination module 703 is further configured to determine whether the number of words in the context information is less than or equal to a preset number of words before filling in a preset prompt template based on the context information to obtain prompt information;

[0152] The calling module is used to call the generating module 702 to execute the step of filling in a preset prompt template based on the context information to obtain prompt information if the determination result of the determining module is yes.

[0153] In some embodiments of the present disclosure, the apparatus further comprises:

[0154] a deletion module configured to, after determining whether the number of words in the context information is less than or equal to a preset number of words, delete part of the content from the historical questions and historical answers included in the context information if the determination result of the determination module is negative, thereby obtaining updated context information, wherein the number of words in the updated context information is less than or equal to the preset number of words;

[0155] The filling module is used to fill in the preset prompt template based on the updated context information to obtain prompt information.

[0156] In some embodiments of the present disclosure, the deletion module is specifically configured to:

[0157] In the order of timestamps from latest to earliest, a preset number of words is intercepted from the context information and other content is deleted to obtain updated context information. The preset number of words is greater than the total number of words in the current question and the current answer.

[0158] In some embodiments of the present disclosure, the apparatus further includes a training module, the training module being configured to:

[0159] Obtaining multiple sample prompt information and obtaining training labels for the sample prompt information, where the training labels represent standard recommendation questions generated based on the sample prompt information;

[0160] Input the sample prompt information into the large language model to obtain the prediction recommendation problem output by the large language model;

[0161] Determine the loss function value based on the predicted recommendation problem and training labels;

[0162] Based on the loss function value, the model parameters of the large language model are adjusted, and the step of inputting the sample prompt information into the large language model is returned until the large language model converges, thereby obtaining an updated large language model.

[0163] In some embodiments of the present disclosure, the training module is specifically configured to:

[0164] Each historical prompt information within a preset time period is used as sample prompt information; and / or,

[0165] Obtain multiple historical conversation information, where the historical conversation information includes multiple conversations within a historical session;

[0166] Acquiring multiple constraint parameters, the constraint parameters including at least one of the number of recommended questions, word range, type, and relevance to context information;

[0167] Each piece of historical dialogue information is combined with a constraint parameter to form sample prompt information.

[0168] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0169] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0170] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0171] Multiple components in the electronic device 800 are connected to the I / O interface 805, including an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0172] The computing unit 801 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the large-model-based recommendation question generation method. For example, in some embodiments, the large-model-based recommendation question generation method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the large-model-based recommendation question generation method described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the large model-based recommendation question generation method in any other appropriate manner (for example, by means of firmware).

[0173] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0174] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0175] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0176] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0177] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0178] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0179] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0180] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for generating recommendation questions based on a large model, comprising: Receive user input for the current question; generating a current answer to the current question; Using historical questions input by the user, historical answers generated for the historical questions, the current question, and the current answer as context information; Generate a recommendation question based on the context information using a large language model; The large language model is obtained by training with sample prompt information, and the sample prompt information is obtained by the following steps: Each historical prompt information within a preset time period is used as sample prompt information; and / or, Acquire multiple historical conversation information, wherein the historical conversation information includes multiple conversations in a historical session; Acquiring a plurality of constraint parameters, the constraint parameters including at least one of the number of recommended questions, word range, type, and relevance to context information; Each piece of historical dialogue information is combined with a constraint parameter to form sample prompt information.

2. The method according to claim 1, wherein The historical questions input by the user, the historical answers generated for the historical questions, the current question, and the current answer are used as context information, including: From the conversations included in the current session, a preset number of conversations are obtained in descending order of timestamps as the context information, wherein the conversations include questions input by the user and answers generated for the questions, and the preset number is greater than or equal to 2.

3. The method according to claim 1, wherein The step of generating a recommendation question based on the context information using a large language model includes: Based on the context information, fill in a preset prompt template to obtain prompt information; The prompt information is input into the large language model to obtain the recommended question output by the large language model.

4. The method according to claim 3, wherein: The step of filling in a preset prompt template based on the context information to obtain prompt information includes: Get the preset constraint parameters; The context information and the constraint parameters are filled into the preset prompt template to obtain the prompt information.

5. The method according to claim 4, wherein The obtaining of preset constraint parameters includes: If there are constraint parameters input by the user, the constraint parameters input by the user most recently are obtained as the preset constraint parameters; or, If there are no constraint parameters input by the user, default constraint parameters are obtained as the preset constraint parameters.

6. The method according to claim 3, before filling in a preset prompt template based on the context information to obtain prompt information, further comprising: Determining whether the number of words in the context information is less than or equal to a preset number of words; If so, the step of filling in a preset prompt template based on the context information to obtain prompt information is executed.

7. The method according to claim 6, further comprising, after determining whether the number of words in the context information is less than or equal to a preset number of words: If not, deleting part of the content from the historical questions and historical answers included in the context information to obtain updated context information, where the number of words in the updated context information is less than or equal to the preset number of words; Based on the updated context information, fill in the preset prompt template to obtain prompt information.

8. The method according to claim 7, wherein: The step of deleting some content from the historical questions and historical answers included in the context information to obtain updated context information includes: In order of timestamps from latest to earliest, the content of the preset number of words in the context information is intercepted and other content is deleted to obtain updated context information, where the preset number of words is greater than the total number of words in the current question and the current answer.

9. The method according to any one of claims 3 to 8, further comprising: Obtaining multiple pieces of sample prompt information and obtaining training labels for the sample prompt information, wherein the training labels represent standard recommendation questions generated based on the sample prompt information; Inputting the sample prompt information into the large language model to obtain a predicted recommendation question output by the large language model; Determining a loss function value based on the prediction recommendation problem and the training labels; Based on the loss function value, the model parameters of the large language model are adjusted, and the process returns to the step of inputting the sample prompt information into the large language model until the large language model converges, thereby obtaining an updated large language model.

10. A device for generating recommendation questions based on a large model, comprising: A receiving module is used to receive the current question input by the user; a generating module, configured to generate a current answer to the current question received by the receiving module; a determination module configured to use, as context information, historical questions input by a user, historical answers generated for the historical questions, the current question received by the receiving module, and the current answer generated by the generating module; The generating module is further configured to generate a recommendation question based on the context information determined by the determining module using a large language model; The large language model is obtained through training with sample prompt information, and the device further comprises: a training module; the training module is used to: Each historical prompt information within a preset time period is used as sample prompt information; and / or, Acquire multiple historical conversation information, wherein the historical conversation information includes multiple conversations in a historical session; Acquiring a plurality of constraint parameters, the constraint parameters including at least one of the number of recommended questions, word range, type, and relevance to context information; Each piece of historical dialogue information is combined with a constraint parameter to form sample prompt information.

11. The device according to claim 10, wherein The determining module is specifically configured to: From the conversations included in the current session, a preset number of conversations are obtained in descending order of timestamps as the context information, wherein the conversations include questions input by the user and answers generated for the questions, and the preset number is greater than or equal to 2.

12. The device according to claim 10, wherein The generation module is specifically used to: Based on the context information, fill in a preset prompt template to obtain prompt information; The prompt information is input into the large language model to obtain the recommended question output by the large language model.

13. The device according to claim 12, wherein The generation module is specifically used to: Get the preset constraint parameters; The context information and the constraint parameters are filled into the preset prompt template to obtain the prompt information.

14. The device according to claim 13, wherein The generation module is specifically used to: If there are constraint parameters input by the user, the constraint parameters input by the user most recently are obtained as the preset constraint parameters; or, If there are no constraint parameters input by the user, default constraint parameters are obtained as the preset constraint parameters.

15. The apparatus according to claim 12, further comprising: The determining module is further configured to determine whether the number of words in the context information is less than or equal to a preset number of words before filling in a preset prompt template based on the context information to obtain prompt information; The calling module is used to call the generating module to execute the step of filling in the preset prompt template based on the context information to obtain prompt information if the determination result of the determining module is yes.

16. The apparatus according to claim 15, further comprising: a deleting module configured to, after determining whether the number of words in the context information is less than or equal to a preset number of words, delete part of the content from the historical questions and historical answers included in the context information if the determination result of the determining module is negative, to obtain updated context information, wherein the number of words in the updated context information is less than or equal to the preset number of words; The filling module is used to fill in the preset prompt template based on the updated context information to obtain prompt information.

17. The device according to claim 16, wherein The deletion module is specifically used to: In order of timestamps from latest to earliest, the content of the preset number of words in the context information is intercepted and other content is deleted to obtain updated context information, where the preset number of words is greater than the total number of words in the current question and the current answer.

18. The apparatus according to any one of claims 12 to 17, further comprising: A training module, wherein the training module is used to: Obtaining multiple pieces of sample prompt information and obtaining training labels for the sample prompt information, wherein the training labels represent standard recommendation questions generated based on the sample prompt information; Inputting the sample prompt information into the large language model to obtain a predicted recommendation question output by the large language model; Determining a loss function value based on the prediction recommendation problem and the training labels; Based on the loss function value, the model parameters of the large language model are adjusted, and the process returns to the step of inputting the sample prompt information into the large language model until the large language model converges, thereby obtaining an updated large language model.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images