Recommended query information generation method and device and electronic equipment

Abstract

The invention provides a recommended query information generation method and device, electronic equipment and a readable storage medium, and relates to the technical field of artificial intelligence such as natural language processing, computer vision, deep learning and large models. The target large model training method comprises the steps of obtaining first training data; using the initial large model to obtain first prediction recommendation query information according to the first dialogue content; obtaining a first loss function value according to the first prediction recommendation query information, the preference recommendation query information and the non-preference recommendation query information; obtaining a second loss function value according to the first prediction recommendation query information and the preference recommendation query information; and adjusting parameters of the initial large model according to the first loss function value and the second loss function value to obtain a target large model. The generation method of the recommended query information comprises the steps of obtaining target dialogue content; using the target big model to obtain target recommendation query information according to the target dialogue content; and displaying the target recommendation query information.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the field of artificial intelligence technology such as natural language processing, computer vision, deep learning, and large models. A method, device, electronic device, and readable storage medium for generating recommended query information are provided. Background Art

[0002] Recommended query information is the query information that the user may enter in the next round of conversation, which is predicted by the big model based on the query information entered by the user in the previous round of conversation and the answer content generated for the query information. Therefore, how to reduce the complexity of user interaction operations during the conversation by generating more accurate recommended query information is a technical problem that needs to be solved urgently. Summary of the invention

[0003] According to a first aspect of the present disclosure, a training method for a target big model for generating recommended query information is provided, comprising: obtaining first training data, the first training data comprising a first conversation content and a corresponding recommended query pair, the first conversation content comprising a first historical query and a corresponding first answer content, the recommended query information pair comprising preference recommended query information and non-preference recommended query information; using an initial big model, based on the first conversation content, obtaining first predicted recommended query information corresponding to the first conversation content; obtaining a first loss function value based on the first predicted recommended query information, the preference recommended query information and the non-preference recommended query information; obtaining a second loss function value based on the first predicted recommended query information and the preference recommended query information; adjusting the parameters of the initial big model based on the first loss function value and the second loss function value to obtain a target big model.

[0004] According to a second aspect of the present disclosure, a method for generating recommended query information is provided, comprising: obtaining target conversation content, the target conversation content including target historical query information and its corresponding target answer content; using a target macro model to obtain target recommended query information corresponding to the target conversation content based on the target conversation content; and displaying the target recommended query information.

[0005] According to a third aspect of the present disclosure, a training device for generating a target big model of recommended query information is provided, comprising: a first acquisition unit, used to acquire first training data, the first training data comprising a first conversation content and its corresponding recommended query information pair, the first conversation content comprising a first historical query information and its corresponding first answer content, the recommended query information pair comprising a preferred recommended query information and a non-preferred recommended query information; a processing unit, used to use an initial big model to obtain, according to the first conversation content, first predicted recommended query information corresponding to the first conversation content; a first calculation unit, used to obtain a first loss function value according to the first predicted recommended query information, the preferred recommended query information and the non-preferred recommended query information; a second calculation unit, used to obtain a second loss function value according to the first predicted recommended query information and the preferred recommended query information; a first training unit, used to adjust the parameters of the initial big model according to the first loss function value and the second loss function value to obtain a target big model.

[0006] According to a fourth aspect of the present disclosure, a device for generating recommended query information is provided, comprising: a second acquisition unit, used to acquire target conversation content, wherein the target conversation content includes target historical query information and its corresponding target answer content; a generation unit, used to use a target macro model to obtain target recommended query information corresponding to the target conversation content according to the target conversation content; and a first display unit, used to display the target recommended query information.

[0007] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0008] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method as described above.

[0009] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method as described above when executed by a processor.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0013] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0017] Figure 6 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0018] Figure 7 It is a block diagram of an electronic device used to implement the training of a target large model for generating recommended query information or the method for generating recommended query information according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and mechanisms is omitted in the following description.

[0020] Figure 1 Schematic diagram of the first embodiment of the present disclosure. Figure 1 As shown, the training method of the target large model for generating recommended query information in this embodiment specifically includes the following steps:

[0021] S101, obtaining first training data, wherein the first training data includes a first conversation content and a corresponding recommended query information pair, the first conversation content includes first historical query information and a corresponding first answer content, and the recommended query information pair includes preferred recommended query information and non-preferred recommended query information;

[0022] S102: using the initial large model, according to the first conversation content, obtaining first predicted recommended query information corresponding to the first conversation content;

[0023] S103, obtaining a first loss function value according to the first predicted recommendation query information, the preference recommendation query information and the non-preference recommendation query information;

[0024] S104, obtaining a second loss function value according to the first predicted recommendation query information and the preferred recommendation query information;

[0025] S105. Adjust the parameters of the initial large model according to the first loss function value and the second loss function value to obtain a target large model.

[0026] The training method of the target big model for generating recommended query information of this embodiment, after using the initial big model to obtain the first predicted recommended query information corresponding to the first conversation content, adjusts the parameters of the initial big model according to the first loss function value obtained from the first predicted recommended query information and the preference recommended query information and the non-preference recommended query information, and the second loss function value obtained from the first predicted recommended query information and the preference recommended query information, thereby ensuring that both the generation of recommended query information (i.e., whether the recommended query information can be generated) and the accuracy of the generation of recommended query information (i.e., whether the generated recommended query information meets the user's preference) are taken into account during the training process, thereby improving the training accuracy and training efficiency of the target big model, so that the target big model can output more accurate recommended query information according to the input conversation content.

[0027] In this embodiment, a large model (LM) generally refers to a deep learning model with a relatively large number of parameters and a relatively complex structure. By processing and analyzing massive amounts of data, it can learn the complex features in the data, thereby having excellent performance in multiple tasks. The large model in this embodiment can be a large language model (LLM), a computer vision large model, a recommendation system large model, a multimodal large model, etc.

[0028] In this embodiment, the recommended query information (also referred to as the recommended query) refers to the query that the user may enter in the next round of conversation, which is predicted by the big model based on the query input by the user in the previous round of conversation and the answer content generated by the big model for the query; wherein, after the user clicks on the generated recommended query information, the big model can generate the corresponding answer content according to the recommended query information clicked by the user; it can be understood that in this embodiment, the type of answer content is related to the type of the big model, if the big model is a big language model, the answer content is text, if the big model is a computer vision big model, the answer content is an image, and if the big model is a multimodal big model, the answer content can be text or an image.

[0029] The first conversation content in the first training data obtained by executing S101 in this embodiment is composed of first historical query information and its corresponding first answer content; wherein the first answer content is text, or multimedia content such as images and videos generated by a large model (such as a source large model) according to the first historical query information.

[0030] In this embodiment, the recommended query information pair corresponding to the first conversation content in the first training data obtained by S101 is used to reflect the user's preference for the recommended query information generated by the large model (e.g., the source large model) according to the first conversation content; wherein the preferred recommended query information in the recommended query information pair is the recommended query information clicked by the user online, and the non-preferred recommended query information in the recommended query information pair is the recommended query information not clicked by the user online; the format of the recommended query information can be text.

[0031] That is, this embodiment performs direct preference optimization (DPO) training on the initial large model through recommendation query information pairs consisting of preference recommendation query information and non-preference recommendation query information.

[0032] For example, if the large model generates four candidate recommendation information based on the content of the first conversation, namely candidate recommendation information 1, candidate recommendation information 2, candidate recommendation information 3 and candidate recommendation information 4; if the user clicks on candidate recommendation information 1 as the input query for the next conversation, when executing S101, this embodiment can use candidate recommendation information 1 as the preferred candidate recommendation information in the recommended query information pair, and any one of the remaining candidate recommendation information as the non-preferred candidate recommendation information in the recommended query information pair.

[0033] When executing S101, this embodiment can obtain multiple first training data, and then train the initial large model according to the first training data set constructed by the multiple first training data, so as to obtain the target large model.

[0034] After executing S101 to obtain the first training data, this embodiment executes S102 to use the initial large model to obtain first predicted recommended query information corresponding to the first conversation content according to the first conversation content.

[0035] When executing S102 in this embodiment, the first conversation content can be directly input into the initial large model, and then the first predicted recommended query information can be obtained according to the output result of the initial large model.

[0036] In addition, when the present embodiment uses the initial large model when executing S102 to obtain the first predicted recommended query information corresponding to the first conversation content based on the first conversation content, the following implementation method can also be used: obtain the first prompt text (the prompt text is prompt), the first prompt text in the present embodiment is the prompt text corresponding to the "predicted recommended query generation task"; use the initial large model to obtain the first predicted recommended query information based on the first conversation content and the first prompt text.

[0037] In this embodiment, the initial large model may be a source large model of a specific type, or may be a large model obtained after fine-tuning the source large model of a specific type for a corresponding task.

[0038] In this embodiment, after executing S102 to obtain the first predicted recommended query information, the loss function value can be calculated according to the obtained first predicted recommended query information.

[0039] In this embodiment, S103 can be executed first to calculate the first loss function value and then S104 can be executed to calculate the second loss function value. S104 can be executed first to calculate the second loss function value and then S103 can be executed to calculate the first loss function value. The first loss function value and the second loss function value can also be calculated at the same time.

[0040] The first loss function value obtained by executing S103 in this embodiment is used to reflect the user preference of the first predicted recommended query information obtained by the initial large model.

[0041] Specifically, when this embodiment executes S103 to obtain the first loss function value according to the first predicted recommendation query information, the preference recommendation query information and the non-preference recommendation query information, the implementation method that can be adopted is: perform similarity calculation on the first predicted recommendation query information and the preference recommendation query information to obtain the first similarity; perform similarity calculation on the first predicted recommendation query information and the non-preference recommendation query information to obtain the second similarity; obtain the first loss function value according to the similarity difference between the first similarity and the second similarity.

[0042] That is to say, this embodiment obtains the first loss function value based on the similarity between the first predicted recommendation query information, the preference recommendation query information and the non-preference recommendation query information, and the first loss function value is the difference between the first similarity and the second similarity, so that the first loss function value can more accurately reflect whether the first predicted recommendation query meets the user preference, thereby improving the accuracy when adjusting the parameters of the initial large model based on the first loss function value.

[0043] It can be understood that the number of first predicted recommendation queries obtained by the initial large model in this embodiment can be one or more; if multiple first predicted recommendation queries are obtained, when executing S103, this embodiment can use the average similarity between the preference recommendation query information and multiple first predicted recommendation query information as the first similarity, or use the maximum similarity or minimum similarity between the preference recommendation query information and multiple first predicted recommendation query information as the first similarity; similarly, the second similarity is obtained in the same way.

[0044] For example, when executing S103, this embodiment can use the minimum similarity between the preference recommendation query information and multiple first predicted recommendation query information as the first similarity, and use the maximum similarity between the non-preference recommendation query information and multiple first predicted recommendation query information as the second similarity, that is, the first loss function value is obtained based on the first predicted recommendation query information that is least similar to the preference recommendation query information and the first predicted recommendation query information that is most similar to the non-preference recommendation query information, thereby improving the adjustment effect when adjusting the parameters of the initial large model based on the first loss function value.

[0045] Among them, the adjustment goal corresponding to the first loss function value in this embodiment is to gradually increase the size of the first loss function value as the training process proceeds, so that the first predicted recommendation query information output by the initial large model becomes more and more similar to the preference recommendation query information, and less and less similar to the non-preference recommendation query information.

[0046] The second loss function value obtained by executing S104 in this embodiment is used to reflect whether the first predicted recommended query information obtained by the initial large model is recommended query information.

[0047] When executing S104 in this embodiment, the similarity calculation result between the first predicted recommendation query information and the preferred recommendation query information can be directly used as the second loss function value.

[0048] In some cases, for example, the initial large model already has a good recommendation query information generation capability. In order to further improve the training effect, when executing S104 in this embodiment to obtain the second loss function value according to the first predicted recommendation query information and the preference recommendation query information, the implementation method that can be adopted is: obtain a preset weight value, and the preset weight value in this embodiment is a value less than 1; obtain the second loss function value according to the preset weight value and the similarity calculation result between the first predicted recommendation query information and the preference recommendation query information.

[0049] That is to say, this embodiment can also combine the preset weight value to obtain the second loss function value, so that the first loss function value and the second loss function value have different effects when adjusting the initial large model, so that the initial large model can generate recommended queries that are more inclined to meet user preferences through the training process.

[0050] In this embodiment, after executing S103 and S104 to obtain the first loss function value and the second loss function value respectively, S105 is executed to adjust the parameters of the initial large model according to the first loss function value and the second loss function value to obtain the target large model.

[0051] That is to say, this embodiment combines the two-part loss function to adjust the parameters of the initial large model, so that the initial large model after the parameter adjustment can generate candidate query information and ensure that the generated candidate query information meets the user's preferences.

[0052] In this embodiment, when executing S105 to adjust the parameters of the initial large model according to the first loss function value and the second loss function value, the training goal of the initial large model is to maximize the addition result between the first loss function value and the second loss function value.

[0053] In this embodiment, when executing S105, if the sum of the loss function values ​​obtained within a preset number of times converges, it is considered that the sum of the first loss function value and the second loss function value is maximized; it can also be that if the sum of the loss function values ​​obtained converges to a preset value, it is considered that the sum of the first loss function value and the second loss function value is maximized; it can also be that if the number of training times exceeds the preset number of times, it is considered that the sum of the first loss function value and the second loss function value is maximized.

[0054] In this embodiment, when executing S105, if the sum of the first loss function value and the second loss function value is maximized, the training of the initial large model is considered to be completed, thereby obtaining a target large model. Using the obtained target large model, the recommended query information for the user to click can be output according to the input conversation content.

[0055] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure. Figure 2 As shown, the method of "obtaining the initial large model" in this embodiment specifically includes the following steps:

[0056] S201, obtaining a second conversation content, where the second conversation content includes second historical query information and its corresponding second answer content;

[0057] S202, obtaining a target annotation prompt text according to the query category of the second historical query information;

[0058] S203, using the candidate large model, according to the second conversation content and the target annotation prompt text, obtaining annotation recommendation query information corresponding to the second conversation content;

[0059] S204: Train the source large model according to the second training data constructed by the second conversation content and the annotation recommendation query information to obtain the initial large model.

[0060] That is to say, on the one hand, this embodiment uses the content generation capability of the candidate big model to achieve the purpose of automatically annotating the second conversation content, thereby improving the efficiency of obtaining the second training data. On the other hand, the second training data is used to perform supervised fine-tuning (SFT) on the source big model, so that the trained initial big model has a certain ability to generate recommendation query information, and then combined with the direct preference optimization (DPO) training of the initial big model to obtain the target big model, which can further improve the training accuracy and training efficiency of the target big model.

[0061] In the second conversation content obtained by executing S201 in this embodiment, the second answer content is generated by a large model (such as a source large model) according to the second historical query information; it can be understood that the first answer content and the second answer content correspond to the same content type, such as both are text, or both are images, etc.

[0062] When executing S202, this embodiment can use the annotation prompt text corresponding to the query category of the second historical query information as the target annotation prompt text according to the preset corresponding relationship; wherein different query categories correspond to different annotation prompt texts (the promoted text is prompt), and the annotation prompt text is the prompt text corresponding to the "annotation recommendation query information generation task".

[0063] For example, the query category "knowledge question and answer" corresponds to the annotation prompt text 1, the query category "novel creation" corresponds to the annotation prompt text 2, the query category "picture creation" corresponds to the annotation prompt text 3, and so on.

[0064] In this embodiment, the initial large model is a large model of a different model type from the source large model; for example, if the source large model is large language model 1, the initial large model may be large language model 2.

[0065] When executing S203, this embodiment can input the second dialogue content and the target annotation prompt text into the candidate large model, so as to obtain the annotation recommendation query information according to the output result of the candidate large model.

[0066] In this embodiment, when executing S204, the source large model is trained according to the second training data constructed by the second conversation content and the labeled recommended query information to obtain the initial large model. The implementation method that can be adopted is: using the source large model, according to the second conversation content, obtain the second predicted recommended query information; according to the second predicted recommended query information and the labeled recommended query information, obtain the third loss function value; according to the third loss function value, adjust the parameters of the source large model to obtain the initial large model.

[0067] It can be understood that, in this embodiment, multiple second training data can be constructed according to multiple second conversation contents and their corresponding annotation recommendation query information, so as to train the source large model using the second training data set obtained from the multiple second training data.

[0068] Among them, when executing S204 to obtain the third loss function value according to the second predicted recommended query information and the labeled recommended query information, this embodiment can calculate the similarity between the second predicted recommended query information and the labeled recommended query information, and use the similarity calculation result as the third loss function value.

[0069] When this embodiment performs S204 to adjust the parameters of the source large model according to the third loss function value, the training goal of the source large model is to maximize the third loss function value, that is, to enable the source large model to output second predicted recommended query information that is increasingly similar to the labeled recommended query information.

[0070] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure. Figure 3 , a flowchart of the present embodiment when training to obtain a target large model for generating recommended query information is shown in FIG: first, the source large model is trained with SFT according to the second training data to obtain an initial large model; then, the initial large model is trained with DPO according to the first training data to obtain a target large model, thereby achieving the purpose of performing two types of training on the source large model to obtain the target large model. In the present embodiment, the source large model may be a source large language model.

[0071] Figure 4 is a schematic diagram according to the fourth embodiment of the present disclosure. Figure 4 As shown, the method for generating recommended query information in this embodiment specifically includes the following steps:

[0072] S401, obtaining target conversation content, wherein the target conversation content includes target historical query information and its corresponding target answer content;

[0073] S402, using the target macro model, according to the target conversation content, obtaining target recommendation query information corresponding to the target conversation content;

[0074] S403: Display the target recommendation query information.

[0075] That is to say, this embodiment uses the trained target large model to obtain target recommendation query information corresponding to the target conversation content, which can improve the accuracy of the obtained target recommendation query information, and display the obtained target recommendation query information. The user can click on the displayed target recommendation query information, so that the next round of conversation can be started without entering a query, which can reduce the complexity of the user's interactive operations during the conversation and simplify the user's interactive operation steps during the conversation.

[0076] The target conversation content obtained by executing S401 in this embodiment is the content of the user's conversation with the target big model in the previous round during the actual application process (or called online); among them, the target historical query information is the query information input by the user into the target big model in the previous round of conversation, and the target answer content is the answer content generated by the target big model according to the query information input by the user in the previous round, such as text, or multimedia content such as images and videos.

[0077] When executing S402 in this embodiment, the target conversation content can be input into the target macro model, and then the target recommendation query information can be obtained according to the output result of the target macro model; wherein the number of the target recommendation query information can be one or more.

[0078] In this embodiment, after executing S402 to obtain the target recommendation query information, S403 may be executed to display the obtained target recommendation query information.

[0079] After executing S403 to display the target recommendation query information, this embodiment may also include the following contents: in response to determining that the target recommendation query information is clicked, using the target macro model, obtaining current answer content corresponding to the target recommendation query information according to the clicked target recommendation query information; and displaying the obtained current answer content.

[0080] That is to say, this embodiment can realize the user's rapid interactive operation during the conversation through the generated target recommendation query information, without the user having to input query information, thereby reducing the complexity of the user's interactive operation during the conversation and simplifying the user's interactive operation steps during the conversation.

[0081] Figure 5 is a schematic diagram according to the fifth embodiment of the present disclosure. Figure 5 As shown, the training device 500 for generating a target large model of recommended query information in this embodiment includes:

[0082] A first acquisition unit 501 is used to acquire first training data, wherein the first training data includes a first conversation content and a corresponding recommended query information pair, wherein the first conversation content includes first historical query information and a corresponding first answer information, and the recommended query information pair includes preferred recommended query information and non-preferred recommended query information;

[0083] The processing unit 502 is configured to obtain first predicted recommended query information corresponding to the first conversation content according to the first conversation content using the initial large model;

[0084] A first calculation unit 503 is used to obtain a first loss function value according to the first predicted recommendation query information, the preference recommendation query information and the non-preference recommendation query information;

[0085] A second calculation unit 504 is used to obtain a second loss function value according to the first predicted recommendation query information and the preference recommendation query information;

[0086] The first training unit 505 is used to adjust the parameters of the initial large model according to the first loss function value and the second loss function value to obtain a target large model.

[0087] The first conversation content in the first training data acquired by the first acquisition unit 501 consists of first historical query information and its corresponding first answer content; wherein the first answer content is text, or multimedia content such as images and videos generated by a large model (such as a source large model) according to the first historical query information.

[0088] The recommended query information pair corresponding to the first conversation content in the first training data obtained by the first acquisition unit 501 is used to reflect the user's preference for the recommended query information generated by the large model (e.g., the source large model) according to the first conversation content; wherein the preferred recommended query information in the recommended query information pair is the recommended query information clicked by the user online, and the non-preferred recommended query information in the recommended query information pair is the recommended query information not clicked by the user online.

[0089] That is, this embodiment performs direct preference optimization (DPO) training on the initial large model through recommendation query information pairs consisting of preference recommendation query information and non-preference recommendation query information.

[0090] The first acquisition unit 501 can acquire multiple first training data, and then train the initial large model according to the first training data set constructed by the multiple first training data, so as to obtain the target large model.

[0091] In this embodiment, after the first acquisition unit 501 acquires the first training data, the processing unit 502 uses the initial large model to obtain first predicted recommended query information corresponding to the first conversation content according to the first conversation content.

[0092] The processing unit 502 may directly input the first conversation content into the initial large model, and then obtain the first predicted recommendation query information according to the output result of the initial large model.

[0093] In addition, when the processing unit 502 uses the initial large language model to obtain the first predicted recommended query information corresponding to the first conversation content according to the first conversation content, the following implementation method can be adopted: obtaining the first prompt text (the prompt text is prompt); using the initial large model to obtain the first predicted recommended query information according to the first conversation content and the first prompt text.

[0094] In this embodiment, the initial large model may be a source large model of a specific type, or may be a large model obtained after fine-tuning the source large model of a specific type for a corresponding task.

[0095] The training device 500 for generating a target big model of recommended query information in the embodiment may further include a second training unit 506, which is used to execute the following contents to obtain an initial big model: obtain a second conversation content, the second conversation content including second historical query information and its corresponding second answer content; obtain a target annotation prompt text according to the query category of the second historical query information; use a candidate big model to obtain annotated recommended query information corresponding to the second conversation content according to the second conversation content and the target annotation prompt text; train the source big model according to the second training data constructed based on the second conversation content and the annotation recommended query information to obtain an initial big model.

[0096] That is to say, on the one hand, the second training unit 506 uses the content generation capability of the candidate big model to achieve the purpose of automatically annotating the second conversation content, thereby improving the efficiency of obtaining the second training data. On the other hand, the second training data is used to perform supervised fine-tuning (SFT) training on the source big model, so that the trained initial big model has a certain ability to generate recommendation query information, and then combined with the direct preference optimization (DPO) training of the initial big model to obtain the target big model, which can further improve the training accuracy and training efficiency of the target big model.

[0097] In the second conversation content acquired by the second training unit 506, the second answer content is text, or multimedia content such as images and videos generated by the big model (such as the source big model) according to the second historical query information.

[0098] The second training unit 506 can use the annotation prompt text corresponding to the query category of the second historical query information as the target annotation prompt text according to a preset corresponding relationship; wherein different query categories correspond to different annotation prompt texts (the promoted text is prompt), and the annotation prompt text is the prompt text corresponding to the "annotation recommendation query generation task".

[0099] The second training unit 506 may input the second dialogue content and the target annotation prompt text into the candidate large model, thereby obtaining the annotation recommendation query information according to the output result of the candidate large model.

[0100] When the second training unit 506 trains the source large model according to the second training data constructed based on the second conversation content and the labeled recommended query information to obtain the initial large model, the implementation method that can be adopted is: using the source large model, according to the second conversation content, obtain the second predicted recommended query information; according to the second predicted recommended query information and the labeled recommended query information, obtain the third loss function value; according to the third loss function value, adjust the parameters of the source large model to obtain the initial large model.

[0101] It can be understood that, in this embodiment, multiple second training data can be constructed according to multiple second conversation contents and their corresponding annotation recommendation query information, so as to train the source large model using the second training data set obtained from the multiple second training data.

[0102] When the second training unit 506 obtains the third loss function value based on the second predicted recommended query information and the labeled recommended query information, it can calculate the similarity between the second predicted recommended query information and the labeled recommended query information, and use the similarity calculation result as the third loss function value.

[0103] When the second training unit 506 adjusts the parameters of the source large model according to the third loss function value, the training goal of the source large model is to maximize the third loss function value, that is, to enable the source large model to output second predicted recommended query information that is increasingly similar to the labeled recommended query information.

[0104] In this embodiment, after the processing unit 502 obtains the first predicted recommended query information, the loss function value can be calculated according to the obtained first predicted recommended query information.

[0105] In this embodiment, the first calculation unit 503 may first calculate the first loss function value, and then the second calculation unit 504 may calculate the second loss function value. The second calculation unit 504 may first calculate the second loss function value, and then the first calculation unit 503 may calculate the first loss function value. The first calculation unit 503 and the second calculation unit 504 may also calculate the first loss function value and the second loss function value at the same time.

[0106] The first loss function value obtained by the first calculation unit 503 is used to reflect the user preference of the first predicted recommended query information obtained by the initial large model.

[0107] Specifically, when the first calculation unit 503 obtains the first loss function value based on the first predicted recommendation query information, the preference recommendation query information and the non-preference recommendation query information, the implementation method that can be adopted is: performing similarity calculation on the first predicted recommendation query information and the preference recommendation query information to obtain a first similarity; performing similarity calculation on the first predicted recommendation query information and the non-preference recommendation query information to obtain a second similarity; and obtaining the first loss function value based on the similarity difference between the first similarity and the second similarity.

[0108] That is to say, the first calculation unit 503 obtains the first loss function value based on the similarity between the first predicted recommendation query information, the preference recommendation query information and the non-preference recommendation query information, and the first loss function value is the difference between the first similarity and the second similarity, so that the first loss function value can more accurately reflect whether the first predicted recommendation query meets the user preference, thereby improving the accuracy when adjusting the parameters of the initial large model based on the first loss function value.

[0109] It can be understood that the number of first predicted recommended query information obtained by the initial large model in this embodiment can be one or more; if multiple first predicted recommended query information is obtained, the first calculation unit 503 can use the average similarity between the preference recommendation query information and the multiple first predicted recommended query information as the first similarity, or use the maximum similarity or minimum similarity between the preference recommendation query information and the multiple first predicted recommended query information as the first similarity; similarly, the second similarity is obtained in the same way.

[0110] Among them, the adjustment goal corresponding to the first loss function value in this embodiment is to gradually increase the size of the first loss function value as the training process proceeds, so that the first predicted recommendation query information output by the initial large model becomes more and more similar to the preference recommendation query information, and less and less similar to the non-preference recommendation query information.

[0111] The second loss function value obtained by the second calculation unit 504 is used to reflect whether the first predicted recommended query information obtained by the initial large model is recommended query information.

[0112] The second calculation unit 504 may directly use the similarity calculation result between the first predicted recommendation query information and the preference recommendation query information as the second loss function value.

[0113] In some cases, for example, the initial large model already has a good recommendation query information generation capability. In order to further improve the training effect, when the second calculation unit 504 obtains the second loss function value based on the first predicted recommendation query information and the preference recommendation query information, the implementation method that can be adopted is: obtaining a preset weight value, the preset weight value in this embodiment is a value less than 1; obtaining the second loss function value based on the preset weight value and the similarity calculation result between the first predicted recommendation query information and the preference recommendation query information.

[0114] That is to say, the second calculation unit 504 can also combine the preset weight value to obtain the second loss function value, so that the first loss function value and the second loss function value have different effects when adjusting the initial large model, so that the initial large model can generate recommended query information that is more inclined to meet user preferences through the training process.

[0115] In this embodiment, after obtaining the first loss function value and the second loss function value, the first training unit 505 adjusts the parameters of the initial large model according to the first loss function value and the second loss function value to obtain the target large model.

[0116] That is, the first training unit 505 adjusts the parameters of the initial large model by combining the two-part loss function, so that the initial large model after parameter adjustment can generate candidate query information and ensure that the generated candidate query information meets the user's preference.

[0117] When the first training unit 505 adjusts the parameters of the initial large model according to the first loss function value and the second loss function value, the training goal of the initial large model is to maximize the addition result between the first loss function value and the second loss function value.

[0118] If the sum of the first loss function value and the second loss function value is maximized, the first training unit 505 considers that the training of the initial large model is completed, thereby obtaining the target large model. Using the obtained target large language model, the recommended query information for the user to click can be output according to the input conversation content.

[0119] Figure 6 is a schematic diagram according to the sixth embodiment of the present disclosure. Figure 6As shown, the device 600 for generating recommended query information in this embodiment includes:

[0120] The second acquisition unit 601 is used to acquire target conversation content, where the target conversation content includes target historical query information and its corresponding target answer content;

[0121] A generating unit 602 is used to obtain target recommendation query information corresponding to the target conversation content according to the target conversation content using the target macro model;

[0122] The first display unit 603 is used to display the target recommendation query information.

[0123] The target conversation content acquired by the second acquisition unit 601 is the content of the user's conversation with the target big model in the previous round during the actual application process; wherein, the target historical query information is the query input by the user to the target big model in the previous round of conversation, and the target answer content is the answer content generated by the target big model based on the query information input by the user in the previous round.

[0124] The generating unit 602 may input the target conversation content into the target macro model, and then obtain the target recommendation query information according to the output result of the target macro model; wherein the number of the target recommendation query information may be one or more.

[0125] In this embodiment, after the generating unit 602 obtains the target recommendation query information, the first display unit 603 displays the obtained target recommendation query information.

[0126] It is understandable that the target recommendation query information displayed by the first display unit 603 can be clicked by the user.

[0127] The recommended query generation device 600 of this embodiment may also include a second display unit 604, which is used to execute the following contents: in response to determining that the target recommended query information is clicked, using the target macro model, and according to the clicked target recommended query information, obtaining current answer content corresponding to the target recommended query information; and displaying the obtained current answer content.

[0128] That is to say, the second display unit 604 can realize the user's quick interactive operation during the conversation through the generated target recommendation query information, without the user having to input query information, which can reduce the complexity of the user's interactive operation during the conversation and simplify the user's interactive operation steps during the conversation.

[0129] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0130] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0131] like Figure 7 , is a block diagram of an electronic device for training a target large model for generating recommended query information or a method for generating recommended query information according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0132] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0133] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0134] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the training of a target large model for generating recommended query information or a method for generating recommended query information. For example, in some embodiments, the training of a target large model for generating recommended query information or a method for generating recommended query information may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708.

[0135] In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the training of the target large model for generating the recommended query information or the method for generating the recommended query information described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the training of the target large model for generating the recommended query information or the method for generating the recommended query information by any other appropriate means (e.g., by means of firmware).

[0136] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable device for training a target large model for generating recommended query information or generating recommended query information, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0138] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0140] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0141] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server for a distributed system, or a server combined with a blockchain.

[0142] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0143] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for a target large model for generating recommended query information, comprising: Acquire first training data, the first training data including first conversation content and its corresponding recommended query information pair, the first conversation content including first historical query information and its corresponding first answer content, the recommended query information pair including preferred recommended query information and non-preferred recommended query information; Using the initial large model, according to the first conversation content, obtain first predicted recommended query information corresponding to the first conversation content; Obtaining a first loss function value according to the first predicted recommended query information, the preferred recommended query information, and the non-preferred recommended query information; Obtaining a second loss function value according to the first predicted recommended query information and the preferred recommended query information; According to the first loss function value and the second loss function value, the parameters of the initial large model are adjusted to obtain the target large model.

2. The method according to claim 1, wherein: The obtaining of a first loss function value according to the first predicted recommendation query information, the preference recommendation query information and the non-preference recommendation query information comprises: Calculating a similarity between the first predicted recommendation query information and the preferred recommendation query information to obtain a first similarity; Calculating a similarity between the first predicted recommendation query information and the non-preferred recommendation query information to obtain a second similarity; The first loss function value is obtained according to a similarity difference between the first similarity and the second similarity.

3. The method according to claim 1, wherein: The obtaining of a second loss function value according to the first predicted recommended query information and the preferred recommended query information includes: Get the preset weight value; The second loss function value is obtained according to the preset weight value and the similarity calculation result between the first predicted recommendation query information and the preference recommendation query information.

4. The method according to claim 1, wherein: Acquiring the initial large model includes: Acquire a second conversation content, where the second conversation content includes second historical query information and a corresponding second answer content; Acquire a target annotation prompt text according to the query category of the second historical query information; Using the candidate large model, according to the second conversation content and the target annotation prompt text, obtaining annotation recommendation query information corresponding to the second conversation content; The source large model is trained according to the second training data constructed based on the second conversation content and the annotation recommendation query information to obtain the initial large model.

5. The method according to claim 4, wherein: The second training data constructed according to the second conversation content and the annotation recommendation query information is used to train the source large model to obtain the initial large model, including: Using the source big model, according to the second conversation content, obtain second predicted recommended query information; Obtaining a third loss function value according to the second predicted recommended query information and the labeled recommended query information; According to the third loss function value, the parameters of the source large model are adjusted to obtain the initial large model.

6. A method for generating recommended query information, comprising: Obtaining target conversation content, wherein the target conversation content includes target historical query information and its corresponding target answer content; Using the target macro model, according to the target conversation content, obtaining target recommendation query information corresponding to the target conversation content; Displaying the target recommendation query information; in, The target large model is trained according to any one of the methods in claims 1-5.

7. The method according to claim 6, further comprising: In response to determining that the target recommendation query information is clicked, using the target macro model, according to the target recommendation query information, obtain current answer content corresponding to the target recommendation query information; The current answer content is displayed.

8. A training device for generating a target large model of recommended query information, comprising: a first acquisition unit, configured to acquire first training data, wherein the first training data includes a first conversation content and a corresponding recommended query information pair, the first conversation content includes first historical query information and a corresponding first answer content, and the recommended query information pair includes preferred recommended query information and non-preferred recommended query information; a processing unit, configured to obtain, based on the first conversation content, first predicted recommended query information corresponding to the first conversation content using the initial large model; a first calculation unit, configured to obtain a first loss function value according to the first predicted recommendation query information, the preference recommendation query information, and the non-preference recommendation query information; a second calculation unit, configured to obtain a second loss function value according to the first predicted recommended query information and the preferred recommended query information; The first training unit is used to adjust the parameters of the initial large model according to the first loss function value and the second loss function value to obtain a target large model.

9. The device according to claim 8, wherein: When the first calculation unit obtains the first loss function value according to the first predicted recommendation query information, the preference recommendation query information and the non-preference recommendation query information, specifically performs: Calculating a similarity between the first predicted recommendation query information and the preferred recommendation query information to obtain a first similarity; Calculating a similarity between the first predicted recommendation query information and the non-preferred recommendation query information to obtain a second similarity; The first loss function value is obtained according to a similarity difference between the first similarity and the second similarity.

10. The device according to claim 8, wherein: When the second calculation unit obtains the second loss function value according to the first predicted recommendation query information and the preference recommendation query information, specifically performs: Get the preset weight value; The second loss function value is obtained according to the preset weight value and the similarity calculation result between the first predicted recommendation query information and the preference recommendation query information.

11. The apparatus according to claim 8, further comprising a second training unit, configured to execute the following steps to obtain the initial large model: Acquire a second conversation content, where the second conversation content includes second historical query information and a corresponding second answer content; Acquire a target annotation prompt text according to the query category of the second historical query information; Using the candidate large model, according to the second conversation content and the target annotation prompt text, obtaining annotation recommendation query information corresponding to the second conversation content; The source large model is trained according to the second training data constructed based on the second conversation content and the annotation recommendation query information to obtain the initial large model.

12. The device according to claim 11, wherein When the second training unit trains the source large model according to the second training data constructed according to the second conversation content and the annotation recommendation query information to obtain the initial large model, specifically: Using the source big model, according to the second conversation content, obtain second predicted recommended query information; Obtaining a third loss function value according to the second predicted recommended query information and the labeled recommended query information; According to the third loss function value, the parameters of the source large model are adjusted to obtain the initial large model.

13. A device for generating recommended query information, comprising: A second acquisition unit is used to acquire target conversation content, wherein the target conversation content includes target historical query information and its corresponding target answer content; A generating unit, configured to obtain target recommendation query information corresponding to the target conversation content according to the target conversation content using the target macro model; A first display unit, used to display the target recommendation query information; Wherein, the target large model is trained according to any one of the devices in claims 8-12.

14. The device according to claim 13, further comprising a second display unit, configured to perform: In response to determining that the target recommendation query information is clicked, using the target macro model, according to the target recommendation query information, obtain current answer content corresponding to the target recommendation query information; The current answer content is displayed.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images