Recommendation question generation method and device based on large model, electronic equipment and medium

Abstract

The invention provides a recommendation question generation method and device based on a large model, electronic equipment and a medium, and relates to the technical field of artificial intelligence, in particular to the technical field of natural language processing, large language models and intelligent customer service. According to the specific implementation scheme, a current question input by a user is received, and a current answer is generated for the current question; and then taking a historical question input by the user, a historical answer generated for the historical question, the current question and the current answer as context information. And generating a recommendation problem based on the context information by using a large language model. The diversity of recommendation problems is enhanced.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of natural language processing, large language models, and intelligent customer service technology. Background Art

[0002] In a multi-round dialogue scenario, after an answer is generated for a question raised by a user, the user may ask the question again. In order to facilitate the user to ask the question again, recommended questions can be provided to the user so that the user can directly select the recommended question to ask. Summary of the invention

[0003] The present disclosure provides a method, device, electronic device and medium for generating recommendation questions based on a large model.

[0004] A first aspect of the embodiments of the present disclosure provides a method for generating recommendation questions based on a large model, comprising:

[0005] Receive user input for the current question;

[0006] generating a current answer to the current question;

[0007] Using the historical questions input by the user, the historical answers generated for the historical questions, the current question and the current answer as context information;

[0008] A large language model is used to generate recommendation questions based on the context information.

[0009] A second aspect of the embodiments of the present disclosure provides a device for generating recommendation questions based on a large model, including:

[0010] A receiving module, used for receiving the current question input by the user;

[0011] A generating module, configured to generate a current answer to the current question received by the receiving module;

[0012] a determination module, configured to use the historical question input by the user, the historical answers generated for the historical question, the current question received by the receiving module, and the current answer generated by the generating module as context information;

[0013] The generating module is further configured to generate a recommendation question based on the context information determined by the determining module using a large language model.

[0014] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method according to the first aspect.

[0018] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute a method according to any one of the first aspects.

[0019] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method according to any one of the first aspects is implemented.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0022] Figure 1 is a flowchart of a method for generating recommendation questions based on a large model provided by an embodiment of the present disclosure;

[0023] Figure 2 is an exemplary schematic diagram of a dialogue interface provided by an embodiment of the present disclosure;

[0024] Figure 3 is a flowchart of a method for generating recommendation questions using a large language model provided by an embodiment of the present disclosure;

[0025] Figure 4 is an exemplary schematic diagram of another dialogue interface provided by an embodiment of the present disclosure;

[0026] Figure 5 is a flow chart of a large language model training method provided by an embodiment of the present disclosure;

[0027] Figure 6 is a flowchart of another method for generating recommendation questions based on a large model provided by an embodiment of the present disclosure;

[0028] Figure 7 It is a structural schematic diagram of a recommendation question generating device based on a large model provided by an embodiment of the present disclosure;

[0029] Figure 8It is a block diagram of an electronic device used to implement the large model-based recommendation question generation method of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] At present, the recommended question generation method based on the question library is generally used. This method requires the pre-construction of a question library, which includes multiple questions pre-set by experts. When receiving a question input by a user, based on the text similarity matching algorithm, questions similar to the question input by the user are retrieved from the question library as recommended questions. However, this method can only select questions from the question library as recommended questions, which limits the diversity of recommended questions.

[0032] In order to improve the diversity of recommendation questions, the embodiment of the present disclosure provides a method for generating recommendation questions based on a large model, which is applied to electronic devices, such as servers, desktop computers, or laptop computers, which have data processing capabilities. Figure 1 As shown, the method for generating recommendation questions based on a large model provided by the embodiment of the present disclosure includes the following steps:

[0033] S101: Receive a current question input by a user.

[0034] When the electronic device is a server, the terminal can display a dialogue interface through a browser, an application (Application, APP) or a small program, and the user can enter the current question in the dialogue interface of the terminal. After receiving the current question input by the user, the terminal sends the current question to the electronic device so that the electronic device receives the current question.

[0035] Alternatively, when the electronic device is a terminal such as a desktop computer or a laptop computer, the electronic device can display a dialogue interface through a browser, an APP, or a mini-program, and the user can input the current question in the dialogue interface of the electronic device.

[0036] S102: Generate a current answer to the current question.

[0037] The electronic device can input the current question into a large language model (LLM) to obtain the current answer output by the large language model. The large language model is a model trained by a deep learning model and has the ability to understand and generate natural language text. For example, the deep learning model can be a transformer.

[0038] When the electronic device is a server, the electronic device may also send the current answer to the terminal so that the terminal displays the current answer on the dialogue interface. When the electronic device is a terminal, the electronic device may display the current answer on the dialogue interface.

[0039] S103: Use the historical questions input by the user, the historical answers generated for the historical questions, the current question and the current answer as context information.

[0040] The method of generating historical answers to historical questions is the same as that of generating current answers to current questions. Please refer to the relevant description of S102 and will not be repeated here.

[0041] S104: Generate recommendation questions based on context information using a large language model.

[0042] Optionally, the big language model for generating the recommended question and the big language model for generating the current answer in S102 may be the same big language model or different big language models, which is not specifically limited in the embodiments of the present disclosure.

[0043] When the electronic device is a server, the electronic device may also send a recommended question to the terminal so that the terminal displays the current answer on the dialogue interface. When the electronic device is a terminal, the electronic device may display the recommended question on the dialogue interface.

[0044] Through the above method, the disclosed embodiment can use the large language model to generate recommended questions based on historical questions, historical answers, current questions and current answers. Since the disclosed embodiment can generate recommended questions in real time based on the large language model, the recommended questions are no longer limited to a fixed question library, and thanks to the powerful dialogue generation and generalization capabilities of the large language model, the recommended questions generated using the large language model are more diverse.

[0045] The following is a detailed description of the method for generating recommendation questions based on a large model provided in an embodiment of the present disclosure.

[0046] In some embodiments of the present disclosure, the above S103 uses historical questions input by the user, historical answers generated for historical questions, current questions and current answers as context information, which can be implemented as follows: from the conversations included in the current session, in order of timestamps from late to early, a preset number of conversations are obtained as context information.

[0047] The electronic device can search for the conversation included in the current conversation from the storage space used to store the conversation based on the conversation identifier (ID) of the current conversation.

[0048] Each dialog consists of a question entered by the user and an answer generated for the question. Figure 2 A conversation interface is provided for an electronic device, wherein the conversation interface includes three conversations of a current conversation, wherein conversation 1 includes question 1 and answer 1 to question 1, conversation 2 includes question 2 and answer 2 to question 2, and conversation 3 includes question 3 and answer 3 to question 3. Assuming that the conversation interface sorts the conversations included in the current conversation in order from earliest to latest, conversation 3 is the current conversation, and conversation 1 and conversation 2 are each a historical conversation.

[0049] Each conversation has a corresponding timestamp, which can indicate: the time when the question included in the conversation was received, or the time when the answer included in the conversation was generated, etc. The electronic device can obtain a preset number of conversations from the conversations included in the current conversation found in the order of timestamps from the latest to the earliest, as context information. Among them, the current conversation includes: a current conversation consisting of a current question and a current answer, and at least one historical conversation consisting of a historical question and a historical answer. The preset number is greater than or equal to 2, so that the preset number of conversations at least includes: the current conversation and the previous historical conversation of the current conversation.

[0050] For example, see Figure 2 , sort the conversations of the current session in order of timestamp from latest to earliest to obtain: conversation 3, conversation 2, conversation 1. Assuming the preset number is 2, then obtain question 3, answer 3, question 2, and answer 2 as context information. Among them, Figure 2 The “…” in the text indicates content that is omitted to save space. In actual application scenarios, the complete answer can be displayed in the dialogue interface.

[0051] Since the user's focus may be different at different times, and the timestamps of the conversations in the same session are generally close, the focus is closer. Therefore, the embodiment of the disclosure obtains a preset number of conversations from the current session in the order of timestamps from late to early, which can ensure the usefulness of the context information and reduce the impact of useless information included in the context information on generating recommendation questions.

[0052] It should be noted that when the current session only includes the current conversation composed of the current question and the current answer, and does not include the historical conversation, the electronic device can use the current question and the current answer as context information. Or in this case, the electronic device can obtain (N - 1) historical conversations from the user's historical conversations before the current session in the order from the latest to the earliest timestamp, where N is a preset quantity; and use the current question, the current answer, and the historical questions and historical answers included in the (N - 1) historical conversations as context information.

[0053] On the other hand, when the current session includes the current conversation and at least one historical conversation, but the number of conversations M included in the current session < N, the electronic device can use only the current question, the current answer, the historical questions, and the historical answers included in the current session as context information. Or, in this case, the electronic device can obtain (N - M) historical conversations from the user's historical conversations before the current session in the order from the latest to the earliest timestamp, and use the current question, the current answer, the historical questions, and the historical answers included in the current session, as well as the historical questions and historical answers included in the (N - M) historical conversations as context information.

[0054] Since the conversations of the same user in different periods may cover similar fields. For example, the conversations of the same user in different sessions all cover fields such as science, education, or culture and entertainment. Therefore, the conversations within different sessions of the same user can, to a certain extent, reflect the fields preferred by the user and the questioning habits, and have certain reference value for the generation of recommended questions. Therefore, the context information in the embodiments of the present disclosure can include the conversations in the current session and the historical sessions, thereby enhancing the relevance between the recommended questions and the generated conversations and improving the quality of the recommended questions.

[0055] It should be noted that when the current session only includes the current conversation, does not include the historical conversation, and there is no historical conversation of this user, the electronic device can use the current question and the current answer as context information.

[0056] In some embodiments of the present disclosure, referring to Figure 3 , the method by which the above-mentioned S104 electronic device uses the large language model to generate recommended questions based on context information includes the following steps:

[0057] S301. Fill in a preset prompt template based on the context information to obtain prompt information.

[0058] For example, the preset prompt template is:

[0059] "Please generate recommended questions based on the current conversation and historical conversations.

[0060] Current conversation:

[0061] Historical dialogue:

[0062] ”

[0063] Combination Figure 2 , assuming that the current conversation in the context information includes question 3 and answer 3, and the historical conversation includes question 2 and answer 2, then based on the context information, fill in the preset prompt template, and the prompt information obtained is:

[0064] “Please generate recommended questions based on the current conversation and historical conversations.

[0065] Current conversation: "Who invented printing?" "Printing was invented by Bi Sheng. Bi Sheng was an inventor in the Northern Song Dynasty. He invented movable type printing, which was about 400 years earlier than lead movable type printing."

[0066] Historical dialogue: "Who invented papermaking?" "The inventor of papermaking is Cai Lun. Cai Lun was in the Eastern Han Dynasty... This kind of paper is called "Cai Hou Paper"."

[0067] ”

[0068] The prompt information may be referred to as a prompt.

[0069] S302: Input the prompt information into the large language model to obtain the recommended question output by the large language model.

[0070] Through the above method, the disclosed embodiment can fill in the prompt template through context information to obtain prompt information, and guide the large language model through the prompt information to generate recommended questions that the user may ask based on the current conversation and historical conversations. In addition, since the disclosed embodiment uses the large language model to generate recommended questions in real time, it avoids the situation where the user input question does not match the question in the question library, resulting in the failure to generate recommended questions.

[0071] In some embodiments of the present disclosure, the user is also allowed to set constraint parameters for the recommended questions, wherein the constraint parameters include at least one of the number of recommended questions, word range, type, and relevance to context information.

[0072] Among them, the types include: open-ended, heuristic, and inquiring, etc. Types can also be called questioning styles.

[0073] The relevance of context information includes: strong relevance, weak relevance, and irrelevant.

[0074] For example, the constraint parameters include: number of recommended questions: 3; word range: 0 to 100; type: open-ended; relevance: weak relevance.

[0075] On this basis, the above S301 electronic device fills in the preset prompt template based on the context information to obtain the prompt information, which can be implemented as follows: obtaining the preset constraint parameters, filling the context information and the constraint parameters into the preset prompt template to obtain the prompt information.

[0076] For example, the default prompt template is:

[0077] “Please generate recommendation questions based on the current conversation, historical conversations, and constraint parameters.

[0078] Current conversation:

[0079] Historical dialogue:

[0080] Constraint parameters:

[0081] ”

[0082] Combination Figure 2 , assuming that the current dialogue in the context information includes question 3 and answer 3, and the historical dialogue includes question 2 and answer 2, then based on the context information and constraint parameters, fill in the preset prompt template, and the prompt information obtained is:

[0083] “Please generate recommended questions based on the current conversation and historical conversations.

[0084] Current conversation: "Who invented printing?" "Printing was invented by Bi Sheng. Bi Sheng was an inventor in the Northern Song Dynasty. He invented movable type printing, which was about 400 years earlier than lead movable type printing."

[0085] Historical dialogue: "Who invented papermaking?" "The inventor of papermaking is Cai Lun. Cai Lun was in the Eastern Han Dynasty... This kind of paper is called "Cai Hou Paper"."

[0086] Constraint parameters: 2, 0-100, open, weak correlation.

[0087] ”

[0088] See also Figure 4 After inputting the above prompt information into the large language model, the two recommended questions generated by the large language model are: "Bi Sheng's life" and "Who invented the compass". Subsequent users can quickly ask questions by clicking on the recommended questions.

[0089] The disclosed embodiment also supports users to set constraint parameters. On this basis, the above-mentioned method of obtaining preset constraint parameters includes the following two cases:

[0090] Case 1: If there are constraint parameters input by the user, the constraint parameters most recently input by the user are obtained as the preset constraint parameters.

[0091] Case 2: If there are no constraint parameters input by the user, the default constraint parameters are obtained as the preset constraint parameters.

[0092] It can be seen that the embodiment of the present disclosure supports users to set constraint parameters for recommendation questions, making the generation of recommendation questions more flexible and more able to meet the needs of users, greatly improving the user experience. In addition, when the user does not set constraint parameters, the default constraint parameters can be obtained to generate recommendation questions that meet common user needs.

[0093] Moreover, the disclosed embodiments can constrain the generated recommendation questions by constraining parameters, thereby providing a direction for generating recommendation questions for large language models, making the recommendation questions more targeted.

[0094] In the disclosed embodiment, since the context information includes multiple rounds of dialogue, the context information may be long. Therefore, before the above S301 fills in the preset prompt template based on the context information and obtains the prompt information, the electronic device may also perform: determining whether the number of words in the context information is less than or equal to the preset number of words. The preset number of words may be set based on actual requirements such as the maximum number of input words of the large language model and the generation delay of the recommended question.

[0095] If so, execute S301 to fill in a preset prompt template based on the context information to obtain prompt information.

[0096] If not, some content is deleted from the historical questions and historical answers included in the context information to obtain updated context information, and then based on the updated context information, a preset prompt template is filled in to obtain prompt information. The number of words in the updated context information is less than or equal to the preset number of words. The method of filling in the preset prompt template based on the updated context information is the same as the method of filling in the preset prompt template based on the context information, which can be referred to the above description and will not be repeated here.

[0097] When the number of words in the context information is less than the preset number of words, the prompt information generated based on the context information is generally not too long, and the length of the prompt information is unlikely to exceed the input limit of the large language model. At the same time, since the prompt information is short, the large language model generates a small amount of computation for the recommendation question based on the prompt information, ensuring the efficiency of generating the recommendation question.

[0098] When the number of words in the context information is greater than the preset number of words, the prompt information generated based on the context information may be too long. If the prompt information is generated directly based on the context information, the length of the prompt information may exceed the input limit of the large language model. Therefore, the embodiment of the present disclosure deletes part of the content from the historical questions and historical answers included in the context information, thereby reducing the number of words included in the context information and reducing the possibility that the length of the prompt information exceeds the input limit of the large language model. Moreover, by reducing the number of words in the prompt information, the amount of calculation for the large language model to generate recommended questions based on the prompt information can also be reduced, thereby improving the generation efficiency.

[0099] Moreover, compared with the current question and the current answer, the historical questions and historical answers have lower reference value for recommended questions. Therefore, some content is deleted from the historical questions and historical answers to ensure the reference value of contextual information for recommended questions.

[0100] In some embodiments of the present disclosure, the electronic device deletes part of the content from the historical questions and historical answers included in the context information to obtain updated context information, which can be implemented as follows: intercepting a preset number of words in the context information in order of timestamps from late to early, and deleting other content to obtain updated context information.

[0101] That is, the electronic device can calculate the sum of the number of words in the conversation and the current cumulative number of words for each conversation included in the context information in the order of the timestamp from the latest to the earliest, and use the calculation result as the current cumulative number of words, wherein the initial value of the current cumulative number of words is zero. Then, it is determined whether the current cumulative number of words is less than the preset number of words. If so, for the previous conversation, return to the step of calculating the sum of the number of words in the conversation and the current cumulative number of words. If not, calculate the difference between the current cumulative number of words and the preset number of words, delete the difference number of words from the conversation, and delete other conversations before the conversation, so that the number of words in the context information retained after the deletion is the preset number of words.

[0102] The preset word count is greater than the sum of the word counts of the current question and the current answer. That is, the preset word count is a larger value so that the updated context information includes the current conversation and part of the historical conversation.

[0103] Since the earlier the timestamp, the lower the correlation between the historical conversation content and the current conversation content, and the lower the reference value for generating recommendation questions, the embodiment of the present disclosure deletes the content in the historical conversation with an earlier timestamp to ensure the reference value of the context information obtained after the update for generating recommendation questions.

[0104] The disclosed embodiment can also train a large language model, and the training process can also be called supervised fine-tuning (SFT). Figure 5As shown in the figure, the training process of the large language model includes the following steps:

[0105] S501, obtaining multiple pieces of sample prompt information, and obtaining a training label for each piece of sample prompt information, wherein the training label represents a standard recommendation question generated based on the sample prompt information, and the standard recommendation question may be manually annotated.

[0106] The electronic device can obtain sample prompt information through the following method one and / or method two.

[0107] Method 1: taking each piece of historical prompt information within a preset time period as sample prompt information, that is, taking each piece of historical prompt information within a preset time period as sample prompt information.

[0108] The historical prompt information refers to the prompt information generated in actual application scenarios to generate recommendation questions.

[0109] Figure 5 The training process shown can be performed once every cycle, and the preset time period can be the previous cycle. Alternatively, the preset time period can be set according to actual needs.

[0110] Method 2: obtaining multiple historical dialogue information, wherein each historical dialogue information includes multiple dialogues in a historical session, and obtaining multiple constraint parameters, and then combining each historical dialogue information with a constraint parameter to form sample prompt information.

[0111] Among them, different constraint parameters have at least one parameter that is different.

[0112] The electronic device may randomly select a piece of historical conversation information and a constraint parameter each time, and combine them into a sample prompt information, and obtain multiple sample prompt information through multiple random combinations. Alternatively, the electronic device may combine each piece of historical conversation information with each constraint parameter to form a sample prompt information.

[0113] Through the above method, the disclosed embodiment can use the historical prompt information obtained in the actual application scenario as sample prompt information, ensure the authenticity of the sample prompt information, and make the large language model trained in this way more suitable for the actual application scenario. On the other hand, the disclosed embodiment can combine each historical dialogue information with a constraint parameter into a sample prompt information, ensure the diversity of the sample prompt information, and make the large language model trained in this way more generalizable.

[0114] S502: Input the sample prompt information into the large language model to obtain the predicted recommendation question output by the large language model.

[0115] In each round of iteration, a batch of sample prompt information can be selected from the sample set, and each sample prompt information of the batch can be input into the large language model respectively to obtain the prediction recommendation problem output by the large language model.

[0116] S503: Determine a loss function value based on the predicted recommendation question and the training label.

[0117] In each round of iteration, for a batch of sample prompt information, each prediction recommendation question generated by the large language model and the training label of each sample prompt information in the batch can be substituted into a preset loss function to calculate the loss function value. For example, the preset loss function can be a mean square error loss function or a cross entropy loss function.

[0118] S504. Adjust the model parameters of the large language model based on the loss function value, and return to S502 until the large language model converges to obtain an updated large language model.

[0119] The model parameters of the large language model can be adjusted based on the loss function value by using a gradient descent algorithm in a back-propagation manner. Afterwards, when the large language model has not converged, the next round of iteration is entered, that is, returning to S502 to select the next batch of sample prompt information from the sample set, and continue to train the large language model based on the selected sample prompt information. When the large language model converges, the current large language model is used as the updated large language model.

[0120] Optionally, it can be determined whether the number of iterations reaches a preset number; if so, it is determined that the large language model has converged, otherwise, it is determined that the large language model has not converged. Alternatively, it can be determined whether the currently calculated loss function value is less than a preset threshold; if so, it is determined that the large language model has converged, otherwise, it is determined that the large language model has not converged. Alternatively, it can be determined whether the error between the loss function values ​​calculated for the most recent preset number of times is less than a preset error; if so, it is determined that the large language model has converged, otherwise, it is determined that the large language model has not converged. Alternatively, it can be determined whether the large language model has converged by other means, which is not specifically limited in the embodiments of the present disclosure.

[0121] Through the above method, the disclosed embodiment can supervise and fine-tune the large language model, thereby continuously improving the prediction accuracy of the large language model during the training process, and continuously improving the generation effect of the large language model on recommendation questions, thereby bringing a more immersive user experience.

[0122] See also Figure 6 The overall process of the method for generating recommendation questions based on a large model provided by the embodiment of the present disclosure is described below:

[0123] S601: Receive a current question input by a user.

[0124] S602: Generate a current answer to the current question.

[0125] S603: From the conversations included in the current session, a preset number of conversations are obtained in order of timestamps from the latest to the earliest, as context information.

[0126] S604: Fill the context information and constraint parameters into a preset prompt template to obtain prompt information.

[0127] S605: Input the prompt information into the large language model to obtain the recommended question output by the large language model.

[0128] S606: After each cycle, each piece of historical prompt information obtained in the previous cycle is used as a piece of sample prompt information, and a training label of the sample prompt information is obtained to obtain a sample set, wherein the sample set includes each piece of sample prompt information and a training label of each sample prompt information.

[0129] S607: Use the sample set to perform supervised fine-tuning on the large language model to obtain an updated large language model.

[0130] Figure 6 The specific implementation methods of each step in the above can refer to the above description and will not be repeated here.

[0131] When generating recommended questions, the disclosed embodiment comprehensively considers the current conversation and the historical conversation, thereby comprehensively analyzing the user's question preference, increasing the probability of the recommended question being selected by the user, and bringing the user a better conversation experience.

[0132] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of the conversation information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0133] It should be noted that the large language model in the embodiment of the present disclosure is not a large language model for a specific user and cannot reflect the personal information of a specific user. Also, the sample set in the embodiment of the present disclosure may come from a public data set.

[0134] Based on the same inventive concept, corresponding to the above method embodiment, the embodiment of the present disclosure also provides a recommendation question generating device based on a large model, such as Figure 7 As shown, the device includes: a receiving module 701, a generating module 702 and a determining module 703;

[0135] The receiving module 701 is used to receive the current question input by the user;

[0136] A generating module 702, used to generate a current answer to the current question received by the receiving module 701;

[0137] The determination module 703 is used to use the historical questions input by the user, the historical answers generated for the historical questions, the current question received by the receiving module 701, and the current answer generated by the generating module 702 as context information;

[0138] The generating module 702 is further configured to generate a recommendation question based on the context information determined by the determining module 703 using the large language model.

[0139] In some embodiments of the present disclosure, the determining module 703 is specifically configured to:

[0140] From the conversations included in the current session, a preset number of conversations are obtained in order of timestamps from late to early as context information, the conversations including questions input by the user and answers generated for the questions, and the preset number is greater than or equal to 2.

[0141] In some embodiments of the present disclosure, the generating module 702 is specifically configured to:

[0142] Based on the context information, fill in the preset prompt template to obtain prompt information;

[0143] The prompt information is input into the large language model to obtain the recommended question output by the large language model.

[0144] In some embodiments of the present disclosure, the generating module 702 is specifically configured to:

[0145] Obtaining preset constraint parameters, where the constraint parameters include: at least one of the number of recommended questions, word range, type, and relevance to context information;

[0146] Fill the context information and constraint parameters into the preset prompt template to obtain prompt information.

[0147] In some embodiments of the present disclosure, the generating module 702 is specifically configured to:

[0148] If there are constraint parameters input by the user, the constraint parameters most recently input by the user are obtained as the preset constraint parameters; or,

[0149] If there are no constraint parameters input by the user, the default constraint parameters are obtained as the preset constraint parameters.

[0150] In some embodiments of the present disclosure, the device further comprises:

[0151] The determination module 703 is further used to determine whether the number of words in the context information is less than or equal to a preset number of words before filling in a preset prompt template based on the context information to obtain prompt information;

[0152] The calling module is used to call the generating module 702 to execute the step of filling in a preset prompt template based on the context information to obtain prompt information if the determination result of the determining module is yes.

[0153] In some embodiments of the present disclosure, the device further comprises:

[0154] A deletion module, configured to, after determining whether the number of words in the context information is less than or equal to a preset number of words, delete part of the content from the historical questions and historical answers included in the context information to obtain updated context information if the determination result of the determination module is no, and the number of words in the updated context information is less than or equal to the preset number of words;

[0155] The filling module is used to fill in the preset prompt template based on the updated context information to obtain prompt information.

[0156] In some embodiments of the present disclosure, the deletion module is specifically used to:

[0157] In the order of timestamps from latest to earliest, content with a preset number of words is intercepted from the context information, and other content is deleted to obtain updated context information, where the preset number of words is greater than the sum of the number of words of the current question and the current answer.

[0158] In some embodiments of the present disclosure, the device further includes: a training module, the training module being configured to:

[0159] Obtaining multiple pieces of sample prompt information and obtaining training labels of the sample prompt information, where the training labels represent standard recommendation questions generated based on the sample prompt information;

[0160] Input the sample prompt information into the large language model to obtain the predicted recommendation question output by the large language model;

[0161] Determine the loss function value based on the predicted recommendation problem and training labels;

[0162] Based on the loss function value, the model parameters of the large language model are adjusted, and the step of inputting the sample prompt information into the large language model is returned until the large language model converges, thereby obtaining an updated large language model.

[0163] In some embodiments of the present disclosure, the training module is specifically used to:

[0164] Using each historical prompt information within a preset time period as sample prompt information; and / or,

[0165] Obtaining multiple historical conversation information, where the historical conversation information includes multiple conversations in one historical session;

[0166] Acquire multiple constraint parameters, where the constraint parameters include: at least one of the number of recommended questions, word range, type, and relevance to context information;

[0167] Each piece of historical dialogue information is respectively combined with a constraint parameter to form sample prompt information.

[0168] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0169] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0170] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0171] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0172] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a recommendation question generation method based on a large model. For example, in some embodiments, the recommendation question generation method based on a large model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the recommendation question generation method based on the large model described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the large model-based recommendation question generation method in any other appropriate manner (eg, by means of firmware).

[0173] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0174] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0175] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0176] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0177] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0178] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0179] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0180] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating recommendation questions based on a large model, comprising: Receive user input for the current question; generating a current answer to the current question; Using the historical questions input by the user, the historical answers generated for the historical questions, the current question and the current answer as context information; A large language model is used to generate recommendation questions based on the context information.

2. The method according to claim 1, wherein: The historical questions input by the user, the historical answers generated for the historical questions, the current question and the current answer are used as context information, including: From the conversations included in the current session, a preset number of conversations are obtained in order of timestamps from late to early as the context information, wherein the conversations include questions input by the user and answers generated for the questions, and the preset number is greater than or equal to 2.

3. The method according to claim 1 or 2, wherein: The step of using a large language model to generate a recommendation question based on the context information includes: Based on the context information, fill in a preset prompt template to obtain prompt information; The prompt information is input into the large language model to obtain the recommended question output by the large language model.

4. The method according to claim 3, wherein: The step of filling in a preset prompt template based on the context information to obtain prompt information includes: Obtaining preset constraint parameters, wherein the constraint parameters include: at least one of the number, word range, type, and relevance of the recommended questions to the context information; The context information and the constraint parameters are filled into the preset prompt template to obtain the prompt information.

5. The method according to claim 4, wherein: The obtaining of preset constraint parameters includes: If there are constraint parameters input by the user, the constraint parameters most recently input by the user are obtained as the preset constraint parameters; or, If there are no constraint parameters input by the user, default constraint parameters are obtained as the preset constraint parameters.

6. The method according to any one of claims 3 to 5, before filling in a preset prompt template based on the context information to obtain prompt information, further comprising: Determining whether the number of words in the context information is less than or equal to a preset number of words; If so, the step of filling in a preset prompt template based on the context information to obtain prompt information is executed.

7. The method according to claim 6, after determining whether the number of words in the context information is less than or equal to a preset number of words, further comprising: If not, deleting part of the content from the historical questions and historical answers included in the context information to obtain updated context information, wherein the number of words in the updated context information is less than or equal to the preset number of words; Based on the updated context information, fill in the preset prompt template to obtain prompt information.

8. The method according to claim 7, wherein: The step of deleting some content from the historical questions and historical answers included in the context information to obtain updated context information includes: In the order of timestamps from late to early, the content of the preset number of words in the context information is intercepted, and other content is deleted to obtain updated context information, and the preset number of words is greater than the sum of the number of words of the current question and the current answer.

9. The method according to any one of claims 3 to 8, further comprising: Acquire multiple pieces of sample prompt information, and acquire training labels of the sample prompt information, wherein the training labels represent standard recommendation questions generated based on the sample prompt information; Inputting the sample prompt information into the large language model to obtain a predicted recommendation question output by the large language model; Determining a loss function value based on the prediction recommendation problem and the training label; Based on the loss function value, the model parameters of the large language model are adjusted, and the process returns to the step of inputting the sample prompt information into the large language model until the large language model converges, thereby obtaining an updated large language model.

10. The method according to claim 9, wherein: The obtaining of multiple sample prompt information includes: Using each historical prompt information within a preset time period as sample prompt information; and / or, Acquire multiple historical conversation information, wherein the historical conversation information includes multiple conversations in a historical session; Acquire multiple constraint parameters, wherein the constraint parameters include: at least one of the number, word range, type, and relevance of the recommended questions to the context information; Each piece of historical dialogue information is respectively combined with a constraint parameter to form sample prompt information.

11. A device for generating recommendation questions based on a large model, comprising: A receiving module, used for receiving the current question input by the user; A generating module, configured to generate a current answer to the current question received by the receiving module; a determination module, configured to use the historical question input by the user, the historical answers generated for the historical question, the current question received by the receiving module, and the current answer generated by the generating module as context information; The generating module is further configured to generate a recommendation question based on the context information determined by the determining module using a large language model.

12. The device according to claim 11, wherein The determination module is specifically used for: From the conversations included in the current session, a preset number of conversations are obtained in order of timestamps from late to early as the context information, wherein the conversations include questions input by the user and answers generated for the questions, and the preset number is greater than or equal to 2.

13. The device according to claim 11 or 12, wherein: The generating module is specifically used for: Based on the context information, fill in a preset prompt template to obtain prompt information; The prompt information is input into the large language model to obtain the recommended question output by the large language model.

14. The device according to claim 13, wherein: The generating module is specifically used for: Obtaining preset constraint parameters, wherein the constraint parameters include: at least one of the number, word range, type, and relevance of the recommended questions to the context information; The context information and the constraint parameters are filled into the preset prompt template to obtain the prompt information.

15. The device according to claim 14, wherein: The generating module is specifically used for: If there are constraint parameters input by the user, the constraint parameters most recently input by the user are obtained as the preset constraint parameters; or, If there are no constraint parameters input by the user, default constraint parameters are obtained as the preset constraint parameters.

16. The device according to any one of claims 13 to 15, further comprising: The determination module is further configured to determine whether the number of words in the context information is less than or equal to a preset number of words before filling in a preset prompt template based on the context information to obtain prompt information; The calling module is used to call the generating module to execute the step of filling in a preset prompt template based on the context information to obtain prompt information if the determination result of the determining module is yes.

17. The apparatus according to claim 16, further comprising: a deleting module, configured to, after determining whether the number of words in the context information is less than or equal to a preset number of words, delete part of the content from the historical questions and historical answers included in the context information if the determination result of the determining module is no, to obtain updated context information, wherein the number of words in the updated context information is less than or equal to the preset number of words; The filling module is used to fill in the preset prompt template based on the updated context information to obtain prompt information.

18. The device according to claim 17, wherein: The deletion module is specifically used for: In the order of timestamps from late to early, the content of the preset number of words in the context information is intercepted, and other content is deleted to obtain updated context information, and the preset number of words is greater than the sum of the number of words of the current question and the current answer.

19. The device according to any one of claims 13 to 18, further comprising: A training module, wherein the training module is used to: Acquire multiple pieces of sample prompt information, and acquire training labels of the sample prompt information, wherein the training labels represent standard recommendation questions generated based on the sample prompt information; Inputting the sample prompt information into the large language model to obtain a predicted recommendation question output by the large language model; Determining a loss function value based on the prediction recommendation problem and the training label; Based on the loss function value, the model parameters of the large language model are adjusted, and the process returns to the step of inputting the sample prompt information into the large language model until the large language model converges, thereby obtaining an updated large language model.

20. The device according to claim 19, wherein The training module is specifically used for: Using each historical prompt information within a preset time period as sample prompt information; and / or, Acquire multiple historical conversation information, wherein the historical conversation information includes multiple conversations in a historical session; Acquire multiple constraint parameters, wherein the constraint parameters include: at least one of the number, word range, type, and relevance of the recommended questions to the context information; Each piece of historical dialogue information is respectively combined with a constraint parameter to form sample prompt information.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images