Model distillation method and apparatus, electronic device, and storage medium

Abstract

The application discloses a model distillation method and device, electronic equipment and a storage medium, relates to the technical field of computers, and in particular to the field of artificial intelligence such as large models and deep learning. The specific implementation scheme is as follows: a first sample is obtained; the first sample is generated by using a plurality of teacher models, and the first sample comprises a first prompt word and first reply information corresponding to the first prompt word; the first sample is input into a student model to obtain a first probability of the first reply information output by the student model; wherein the positive evaluation index of the teacher model is greater than the positive evaluation index of the student model; and the student model is trained according to the first probability to obtain a first distillation model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, especially to the field of artificial intelligence such as large models and deep learning, and specifically relates to a model distillation method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Model distillation is a technology for transferring the knowledge of a large and complex model (as a teacher model) to a small and simple model (as a student model), aiming to improve the inference efficiency and performance of the model while maintaining similar performance to the teacher model. SUMMARY

[0003] The present application provides a model distillation method and device, an electronic device, and a storage medium.

[0004] According to an aspect of the present application, a model distillation method is provided, comprising:

[0005] obtaining a first sample; wherein the first sample is generated using a plurality of teacher models, and the first sample includes a first prompt word and first reply information corresponding to the first prompt word;

[0006] inputting the first sample into a student model to obtain a first probability of the first reply information output by the student model; wherein the forward evaluation index of the teacher model is greater than the forward evaluation index of the student model;

[0007] training the student model according to the first probability to obtain a first distillation model.

[0008] According to another aspect of the present application, a method for generating reply information is provided, comprising:

[0009] obtaining input information;

[0010] generating a prompt word according to the input information;

[0011] inputting the prompt word into a distillation model to obtain reply information output by the distillation model; wherein the distillation model is trained using the model distillation method of the above-mentioned aspect of the embodiment.

[0012] According to another aspect of the present application, a model distillation device is provided, comprising:

[0013] a first obtaining module for obtaining a first sample; wherein the first sample is generated using a plurality of teacher models, and the first sample includes a first prompt word and first reply information corresponding to the first prompt word;

[0014] The second obtaining module is configured to input the first sample into a student model to obtain a first probability of the first reply information output by the student model, wherein the forward evaluation index of the teacher model is greater than the forward evaluation index of the student model.

[0015] The first training module is configured to train the student model according to the first probability to obtain a first distillation model.

[0016] According to another aspect of the present application, a reply information generation device is provided, comprising:

[0017] The first obtaining module is configured to obtain input information.

[0018] The generating module is configured to generate a prompt word according to the input information.

[0019] The second obtaining module is configured to input the prompt word into a distillation model to obtain reply information output by the distillation model, wherein the distillation model is trained by the model distillation device in the above-mentioned another aspect embodiment.

[0020] According to another aspect of the present application, an electronic device is provided, comprising:

[0021] at least one processor; and

[0022] a memory connected with the at least one processor in communication; wherein

[0023] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the above-mentioned embodiments.

[0024] According to another aspect of the present application, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method according to the above-mentioned embodiments.

[0025] According to another aspect of the present application, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method described in the above-mentioned embodiments.

[0026] It should be understood that the contents described in this part are not intended to identify the key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings are used to better understand the present application, and do not constitute a limitation of the present application. Among them:

[0028] Figure 1 A flowchart of a model distillation method provided by an embodiment of the present application is shown in FIG. 1.

[0029] Figure 2 A flowchart of a model distillation method provided by another embodiment of the present application is shown in FIG. 2.

[0030] Figure 3 A flowchart of a model distillation method provided by another embodiment of the present application is shown in FIG. 3.

[0031] Figure 4 A flowchart of a model distillation method provided by an embodiment of the present application is shown in FIG. 4.

[0032] Figure 5 A structural diagram of a model distillation device provided by an embodiment of the present application is shown in FIG. 5.

[0033] Figure 6 A structural diagram of a reply information generation device provided by an embodiment of the present application is shown in FIG. 6.

[0034] Figure 7 A block diagram of an electronic device for implementing the model distillation method of an embodiment of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0035] Exemplary embodiments of the present application are described below with reference to the accompanying drawings, which include various details of the embodiments of the present application to assist in understanding, which should be considered in a descriptive sense only. It will thus be appreciated that various modifications and changes can be made to the embodiments described herein without departing from the scope and spirit of the application. Likewise, the description and the illustrations are not intended to limit the scope of the application to particular embodiments described herein. Features of the application are described with reference to the drawings and together with the specification, and are illustrated in the figures.

[0036] It should be noted that the acquisition, storage, use, processing, etc. of data in the technical solutions of the present application comply with relevant provisions of national laws and regulations and do not violate public order and good customs.

[0037] The model distillation method, device, electronic device and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.

[0038] In some embodiments, knowledge distillation can be based on a single teacher model. However, a single teacher model often has advantages in only some aspects, and it is difficult for a student model to learn comprehensively from a teacher model, thereby resulting in relatively low accuracy of the distilled model.

[0039] Therefore, an embodiment of the present application provides a model distillation method. Figure 1 A flowchart of a model distillation method provided by an embodiment of the present application is shown in FIG. 1.

[0040] The model distillation method of the embodiment of the present application can be executed by the model distillation device of the embodiment of the present application, which can be configured in an electronic device.

[0041] The electronic device can be any device with computing capability, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal can be a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, etc. with various operating systems, touch screens and / or display screens.

[0042] As shown in Figure 1 The model distillation method includes the following steps.

[0043] Step 101: Obtain a first sample.

[0044] The first sample can be generated by using multiple teacher models. For example, the teacher model can be a large model, or a distillation model obtained by distillation training based on multiple large models, and the forward evaluation index of the distillation model is greater than a preset threshold.

[0045] For example, the forward evaluation index can include accuracy, precision, recall, etc. If there are multiple forward evaluation indexes, each forward evaluation index can be greater than the corresponding preset threshold. For example, the accuracy of the distillation model as the teacher model is greater than a first threshold, and the precision is greater than a second threshold.

[0046] For example, the large model can be a large language model, or a text-to-image model, or an image-to-text model, etc., or other types of large models, which are not limited in the present application.

[0047] It should be noted that the types of the multiple teacher models used to generate the first sample are the same, such as both being large language models or both being text-to-image models.

[0048] In the present application, the first sample can include a first prompt word, first reply information corresponding to the first prompt word, etc.

[0049] The first reply information can be the reply information output by one of the multiple teacher models based on the first prompt word.

[0050] The modality of the first reply information can be text, image, text and image, etc., which are not limited in the present application.

[0051] It should be noted that the number of the obtained first samples can be one or more, which are not limited.

[0052] For example, if there are multiple first samples, the multiple first samples can be for different scenarios of a target field.

[0053] Step 102, input the first sample into the student model to obtain a first probability of the first reply information output by the student model.

[0054] The positive evaluation index of the teacher model can be greater than the positive evaluation index of the student model. Alternatively, the difference between the positive evaluation index of the teacher model and the positive evaluation index of the student model can be greater than a third threshold.

[0055] In the present application, the first sample can be filled into a preset prompt template to generate a prompt word for guiding the student model to output the probability of the first reply information, and the prompt word can be input into the student model to obtain the first probability of the first reply information output by the student model.

[0056] The first probability can refer to the probability of inputting the first prompt word into the student model and the student model outputting the first reply information.

[0057] Step 103, training the student model according to the first probability to obtain a first distilled model.

[0058] In the present application, the model loss can be determined according to the first probability, and the parameters of the student model are adjusted according to the model loss until the model training end condition is met to obtain the first distilled model.

[0059] For example, the negative number of the first probability can be used as the model loss, and the parameters of the student model are adjusted based on the model loss, so that the first probability output by the model becomes larger and larger until the model training end condition is met to obtain the first distilled model.

[0060] The model distillation method of the present application can be widely applied in many fields, such as the first distilled model can be used to build a more intelligent and accurate question and answer system, or applied to chat robots, virtual assistants and other applications, to generate more coherent and creative dialogue content, or applied to news, literature, report and other summary generation tasks, to generate more accurate and highly summarized automatic summaries.

[0061] For example, in a knowledge question and answer scenario, the user input question can be obtained, the corresponding prompt word can be generated according to the input question, the prompt word can be input into the first distilled model, and the reply text output by the first distilled model can be obtained. For example, the user input question is "what is the text-to-image model", and the first distilled model can output an explanation of the text-to-image model.

[0062] In the embodiment of the present application, the first sample generated based on the plurality of teacher models is input into the student model, and the student model is supervised and trained by using the probability of the first reply information output by the student model to obtain the first distillation model. Therefore, since the first sample is generated based on the plurality of teacher models, the student model is trained by using the first sample, so that the student model can learn the advantages of the plurality of teacher models, thereby realizing the comprehensive migration of the ability of the strong model to the weak model and improving the accuracy of the distillation model.

[0063] Figure 2 The flowchart of the model distillation method provided for another embodiment of the present application is shown.

[0064] As shown in Figure 2 , the model distillation method comprises:

[0065] Step 201, obtaining a first prompt word.

[0066] In the present application, the target query example can be obtained from the example library, and the query statement expansion condition can be obtained, the second prompt word for expanding the query statement can be generated according to the target query example and the query statement expansion condition, and the target query example can be expanded by using the large model based on the second prompt word to generate the second query statement, and the first query statement can be determined from the second query statement, and finally the first prompt word can be generated according to the first query statement. Therefore, based on the target query example and the query statement expansion condition, the target query example can be expanded, the query statement can be enriched, and the prompt word can be enriched.

[0067] Among them, the example library can include query statement examples in different scenarios in the target field. For example, the example library includes typical problems in different scenarios such as base stations and core networks in the communication field.

[0068] As an example, one or more target query examples can be randomly selected from the example library, or query statement examples with a character length exceeding a preset length can be selected from the example library as target query examples, or target query examples can be selected from the example library based on the similarity between each pair of query statement examples in the example library, etc.

[0069] As another example, the query statement examples in the example library can be clustered to obtain a plurality of first clusters, and target query examples can be selected from each first cluster. Among them, the target query examples selected from each first cluster can be one or more.

[0070] For example, the K-means algorithm is used to cluster the query statement examples in the example library into K clusters, n samples are randomly selected from each cluster each time, and n*K examples are composed to put into the first prompt word as reference examples. Among them, K can be an integer greater than 1, and n can be an integer greater than or equal to 1.

[0071] It should be noted that other ways can also be used to cluster the query statement examples in the example library, and no limitation is made thereto.

[0072] Thus, by selecting target query examples from the multiple first-class clusters obtained from the example library, the query statement examples of multiple scenarios are used as reference examples for the large model to generate query statements, thereby improving the richness of the generated second query statements.

[0073] The query statement expansion condition represents the requirements when expanding the query statement. For example, the query statement expansion condition can include task definition, length limit, and output requirement. For example, the task definition is to generate a query statement of the vacation type, the length limit is that the generated query statement cannot be less than 2 words and cannot exceed 20 words, and the output requirement is to output the generated query statement in the form of a Python list.

[0074] For example, the prompt template used to guide the expansion of the query statement can be filled according to the target query example, the query statement expansion condition, and the like, to obtain a second prompt word.

[0075] The second prompt word can include a task instruction, a query statement expansion condition, a target query example, and the like. For example, the task instruction can be "You are a Query expansion expert, generate 20 user query Query according to the query statement expansion condition and the target query example."

[0076] For determining the first query statement from the second query statement, for example, one or more can be randomly selected from the second query statement as the first query statement.

[0077] For example, the second query statement can also be clustered to obtain multiple second-class clusters, and a target query statement can be selected from the second-class clusters, and the target query statement selected from the multiple second-class clusters can be used as the first query statement.

[0078] For example, the K-means algorithm can be used to cluster the second query statement, or other ways can be used to cluster the second query statement, and no limitation is made thereto.

[0079] Thus, by clustering the second query statement, similar query statements can be filtered, and a typical query statement can be selected as the first query statement, thereby ensuring the diversity of the first query statement.

[0080] For example, if there are multiple first query statements, a corresponding first prompt word can be generated for each first query statement, thereby obtaining multiple first prompt words.

[0081] It should be noted that the above method can also be used in advance to extend the query statement examples in the example library using a large model to obtain a first query statement, and then generate a first prompt word based on the first query statement.

[0082] In step 202, the first prompt word is input into the teacher model to obtain second reply information output by the teacher model.

[0083] For example, the first prompt word can be input into multiple teacher models to obtain second reply information output by each teacher model.

[0084] In step 203, first attribute information of the second reply information is obtained.

[0085] The first attribute information can be in the form of a score or a confidence, and the value of the first attribute information of the second reply information can be used to represent the quality of the second reply information. The higher the value of the first attribute information, the higher the quality of the second reply information.

[0086] As a possible implementation, each second reply information has the same initial attribute information. The first matching degree between any third reply information in the second reply information and the first prompt word can be determined, and the second matching degree between other fourth reply information in the second reply information and the first prompt word can be determined. According to the comparison result between the first matching degree and the second matching degree, the initial attribute information of the third reply information and the initial attribute information of the fourth reply information are reconfigured respectively to obtain the first attribute information of the third reply information after reconfiguration and the first attribute information of the fourth reply information after reconfiguration.

[0087] The first matching degree can be used to represent the ability of the third reply information to solve the first prompt word, and the second matching degree can be used to represent the ability of the fourth reply information to solve the first prompt word. The higher the matching degree, the higher the ability to solve the first prompt word.

[0088] Taking the first attribute information in the form of a score as an example, if the first matching degree is greater than the second matching degree, the initial score of the third reply information is added by a preset score, and the initial score of the fourth reply information is subtracted by the preset score; if the first matching degree is less than the second matching degree, the initial score of the third reply information is subtracted by a preset score, and the initial score of the fourth reply information is added by the preset score; if the first matching degree is equal to the second matching degree, the initial score of the third reply information and the initial score of the fourth reply information can remain unchanged.

[0089] Exemplarily, the large model can also be used to score the ability of the third reply information and the fourth reply information to solve the first prompt word, output the size relationship between the scores of the third reply information and the fourth reply information, and update the initial scores of the third reply information and the fourth reply information based on the size relationship.

[0090] For example, two second reply information are taken from the second reply information each time, denoted as X and Y, a scoring prompt word is constructed and input into the large model, and the results of X>Y, X=Y or XY are obtained. It is assumed that the scoring prompt word is "You are a scoring expert, your task is to judge which reply information can better solve the business prompt word, and give the conclusion of X>Y, X=Y or X<Y. Input: business prompt word={**}, X={**}, Y={**}, output: "".

[0091] For example, the second reply information has Resp_1, Resp_2, Resp_3 and Resp_4. The large model can be used to score Resp_1 and Resp_2, and update the initial scores of Resp_1 and Resp_2 respectively according to the output results of the large model. The large model can be used to score Resp_1 and Resp_3, and update the current score of Resp_1 and the initial score of Resp_3 respectively according to the output results of the large model. The large model can be used to score Resp_1 and Resp_4, and update the current score of Resp_1 and the initial score of Resp_4 respectively according to the output results of the large model. The large model can be used to score Resp_2 and Resp_3, and update the current score of Resp_2 and the current score of Resp_3 respectively according to the output results of the large model. The large model can be used to score Resp_2 and Resp_4, and update the current score of Resp_2 and the current score of Resp_4 respectively according to the output results of the large model. The large model can be used to score Resp_3 and Resp_4, and update the current score of Resp_3 and the current score of Resp_4 respectively according to the output results of the large model.

[0092] Since the positions of the third reply information and the fourth reply information in the scoring prompt word may affect the scoring, the positions of the third reply information and the fourth reply information in the scoring prompt word can be exchanged, and the scoring can be performed again. The scores are updated according to the output comparison results, thereby improving the accuracy of the scores.

[0093] Thus, by comparing the abilities of each pair of reply information in the second reply information to solve the first prompt word, the attribute information of each pair of reply information is reconfigured according to the comparison result to obtain the final attribute information, which is more accurate than the attribute information obtained by separately evaluating the quality of each second reply information, thereby improving the accuracy of the quality evaluation of the second reply information.

[0094] Taking the first attribute information in the form of a score as an example, each second reply information can be input into a reward model for scoring to obtain the score of each second reply information.

[0095] In step 204, the first reply information is determined from the second reply information according to the first attribute information.

[0096] For example, the second reply information in which the value of the first attribute information is greater than the fourth threshold can be taken as the first reply information.

[0097] For example, the first reply information can include the second reply information with the maximum value of the first attribute information. For example, for each first prompt word, the second reply information with the maximum value of the first attribute information can be taken as the first reply information. Thus, the first reply information includes the second reply information with the maximum value of the first attribute information, which can enable the student model to learn from multiple teacher models at the same time, so that the student model can quickly learn the abilities of multiple teacher models, thereby improving the learning efficiency.

[0098] In step 205, the first sample is determined according to the first prompt word and the first reply information.

[0099] For example, for each first prompt word, if the first reply information is multiple, the first prompt word and each first reply information can be taken as a first sample.

[0100] For example, for each first prompt word, if the first reply information includes the reply information with the maximum score in the second reply information, the first prompt word and the first reply information can be taken as a first sample.

[0101] In step 206, the first sample is input into the student model to obtain the first probability of the first reply information output by the student model.

[0102] In step 207, the student model is trained according to the first probability to obtain the first distilled model.

[0103] In the present application, steps 206-207 adopt any implementation manner in the embodiments of the present application, and thus will not be described here.

[0104] In the embodiments of the present application, the first prompt word is input into the plurality of teacher models respectively, the second reply information output by the plurality of teacher models is obtained, the first attribute information of the second reply information is obtained, the first reply information is filtered from the second reply information based on the first attribute information of the second reply information, thereby enriching the diversity of the first reply information, and thus the first student model is trained by using the first sample obtained based on the first prompt word and the first reply information, so that the accuracy of the first distillation model can be improved.

[0105] Figure 3 A flowchart of a model distillation method provided for another embodiment of the present application is shown.

[0106] As shown in Figure 3 , the model distillation method comprises:

[0107] In step 301, the first sample and the second sample are obtained.

[0108] In the present application, the first sample can be obtained in the manner described in the above embodiments, which will not be repeated here.

[0109] In the present application, the second sample can be generated by using the above plurality of teacher models.

[0110] The second sample can include a third prompt word, fifth reply information corresponding to the third prompt word, and sixth reply information.

[0111] The value of the attribute information of the fifth reply information can be greater than the value of the attribute information of the sixth reply information. Taking the attribute information as an example in the form of a score, the score of the fifth reply information can be greater than the score of the sixth reply information, indicating that the quality of the fifth reply information is higher than that of the sixth reply information. On this basis, the difference between the score of the fifth reply information and the score of the sixth reply information can also be greater than the fifth threshold value.

[0112] The fifth reply information can be the reply information output by one of the plurality of teacher models based on the third prompt word, and the sixth reply information can also be the reply information output by one of the plurality of teacher models based on the third prompt word.

[0113] In addition, the fifth reply information and the sixth reply information can be output by the same teacher model or different teacher models, which is not limited.

[0114] Exemplarily, the third prompt word can be input into the teacher model respectively to obtain seventh reply information output by the teacher model, and second attribute information of the seventh reply information is obtained, and fifth reply information and sixth reply information are determined from the seventh reply information according to the second attribute information of the seventh reply information, and the second sample is obtained according to the third prompt word, the fifth reply information and the sixth reply information.

[0115] The method for obtaining the second attribute information of the seventh reply information is similar to the method for obtaining the first attribute information of the second reply information, and thus is not described herein again.

[0116] Therefore, by inputting the third prompt word into the plurality of teacher models respectively, obtaining seventh reply information output by the plurality of teacher models, and obtaining second attribute information of the seventh reply information, the fifth reply information and the sixth reply information are filtered from the seventh reply information based on the second attribute information of the seventh reply information, thereby enriching the second sample, and thus the accuracy of the second distillation model can be improved by training the first distillation model based on the second sample.

[0117] Exemplarily, the fifth reply information can include seventh reply information with a value of the second attribute information greater than a sixth threshold value, and the sixth reply information can include seventh reply information with a value of the second attribute information less than a seventh threshold value. The sixth threshold value is greater than the seventh threshold value.

[0118] Exemplarily, seventh reply information with a value of the second attribute information greater than the sixth threshold value in the seventh reply information can be taken as the fifth reply information, and seventh reply information with a value of the second attribute information less than the seventh threshold value in the seventh reply information can be taken as the sixth reply information.

[0119] Taking the second attribute information in the form of a score as an example, seventh reply information with a score greater than the sixth threshold value in the seventh reply information can be taken as the fifth reply information, and seventh reply information with a score less than the seventh threshold value in the seventh reply information can be taken as the sixth reply information.

[0120] Exemplarily, two reply information with a difference between scores greater than a fifth threshold value in the seventh reply information can be taken as the fifth reply information and the sixth reply information respectively.

[0121] Exemplarily, the fifth reply information can include seventh reply information with the maximum value of the second attribute information, and the sixth reply information can include seventh reply information with the minimum value of the second attribute information.

[0122] Exemplarily, for each second prompt word, seventh reply information with the maximum value of the second attribute information in the seventh reply information can be taken as the fifth reply information, and seventh reply information with the minimum value of the second attribute information in the seventh reply information can be taken as the sixth reply information.

[0123] Taking the score form of the second attribute information as an example, for each second prompt word, the seventh reply information with the maximum score in the seventh reply information can be taken as the fifth reply information, and the seventh reply information with the minimum score in the seventh reply information can be taken as the sixth reply information.

[0124] Therefore, by taking the seventh reply information with the maximum value of the second attribute information as the fifth reply information and the seventh reply information with the minimum value of the second attribute information as the fifth reply information, a preference sample can be obtained, so that model training based on the preference sample can improve the generalization ability of the distilled model.

[0125] For example, the third prompt word can be a prompt word generated based on a new query statement obtained by performing query statement expansion on a query statement example selected from the above example library by using a large model. The specific method can refer to the method of generating the first prompt word described in the above embodiments.

[0126] It should be noted that the third prompt word and the first prompt word can be the same, and the fifth reply information and the first reply information can also be the same, or the third prompt word and the first prompt word can be different, and the fifth reply information and the first reply information can also be the same, or the third prompt word and the first prompt word can be different, and the fifth reply information and the first reply information can also be different, which is not limited in the present application.

[0127] In step 302, the first sample is input into the student model to obtain a first probability of the first reply information output by the student model.

[0128] In step 303, the student model is trained according to the first probability to obtain a first distilled model.

[0129] In the present application, steps 302-303 adopt any implementation manner in the embodiments of the present application, and therefore will not be described here.

[0130] In step 304, the third prompt word and the fifth reply text in the second sample are input into the first distilled model to obtain a second probability of the fifth reply information output by the first distilled model.

[0131] For example, the third prompt word and the fifth reply information can be input into the first distilled model to obtain the second probability of the fifth reply information output by the first distilled model.

[0132] The second probability can represent the probability that the first distilled model outputs the fifth reply information when the third prompt word is input into the first distilled model.

[0133] Step 305, input the third prompt word and the sixth reply text in the second sample into the first distillation model to obtain the third probability of the sixth reply information output by the first distillation model.

[0134] For example, the third prompt word can be input into the first distillation model together with the sixth reply information to obtain the third probability of the sixth reply information output by the first distillation model.

[0135] The third probability can represent the probability of inputting the third prompt word into the first distillation model and outputting the sixth reply information by the first distillation model.

[0136] Step 306, training the first distillation model according to the second probability and the third probability to obtain the second distillation model.

[0137] In this application, the sum of the negative of the second probability and the third probability can be used as a model loss, and the first distillation model can be adjusted based on the model loss to make the second probability larger and the third probability lower until the model training end condition is met, and the second distillation model is obtained.

[0138] In the embodiments of the present application, on the basis of supervising and training the student model by using the first sample, the first distillation model obtained by the supervision training can also be trained by reinforcement learning by using high-quality reply information and low-quality reply information, so as to improve the ability of the distillation model to draw conclusions from a few cases and improve the generalization ability of the distillation model.

[0139] In order to facilitate understanding of the model distillation method of the present application, the following will be described in combination with Figure 4 , Figure 4 A model distillation process diagram provided by an embodiment of the present application.

[0140] For example, Figure 4As shown, based on the query examples in the seed query set, the large model can be used to expand the query examples to generate more similar queries, that is, to generate a query set. Based on the queries in the generated query set, the Prompt, that is, the prompt word, is input into the teacher model 1, the teacher model 2, …, the teacher model N, and N teacher models to obtain Resp1, Resp2, Resp3, …, RespN, a total of N reply information. The reward model is used to reply to the N reply information to obtain the score Score1 of Resp1, the score Score3 of Resp2, …, and the score ScoreN of RespN. Then, based on the scores of each reply information, the reply information with the maximum score is selected from the N reply information, denoted as Resp_good, forming <Prompt, Resp_good>, obtaining the instruction data set, and the reply information with the minimum score is selected, denoted as Resp_bad, forming <Prompt, Resp_good, Resp_bad>, obtaining the preference data set, and the instruction data set and the preference data set are used as the training data set.

[0141] After obtaining the training data set, the instruction data set in the training data set can be used to supervise the training of the student model to obtain the first distillation model. Then, the preference data set in the training data set is used to perform reinforcement learning training on the first distillation model to obtain the second distillation model.

[0142] Among them, the Prompt and the Resp_good can be input into the student model, and the probability of the Resp_good output by the student model is used to supervise the training of the student model.

[0143] Among them, the Prompt and the Resp_good can be input into the first distillation model to obtain the probability of the Resp_good output by the first distillation model, and the Prompt and the Resp_bad can be input into the first distillation model to obtain the probability of the Resp_bad output by the first distillation model. Then, based on the probability of the Resp_good and the probability of the Resp_bad, the first distillation model is subjected to reinforcement learning training to obtain the second distillation model.

[0144] The seed query set in the embodiment is also the example library in the above embodiment, and the query example is also the query statement example in the above embodiment.

[0145] In order to realize the above-mentioned embodiment, the embodiment of the present application also proposes a reply information generation method. The reply information generation method can include: obtaining input information, and generating a prompt word according to the input information; and inputting the prompt word into a distillation model to obtain reply information output by the distillation model.

[0146] The distillation model used can be trained using the model distillation method in the above embodiments. For example, the distillation model can be the first distillation model or the second distillation model.

[0147] The input information can be text or an image, or text and an image, and is not limited in this regard.

[0148] For example, the prompt word can be obtained by filling in the corresponding prompt word model according to the input information. For example, the prompt word is "Are you an expert in writing structured query language? Based on the user input {Query}, write a suitable structured query language statement, and output:".

[0149] In the embodiments of the present application, since the distillation model is trained based on samples obtained from multiple teacher models, the accuracy of the reply information can be improved by inputting the prompt word obtained based on the input information into the distillation model.

[0150] To implement the above embodiments, the present application also provides a model distillation device. Figure 5 A structural diagram of a model distillation device provided by an embodiment of the present application.

[0151] As shown in the structural diagram of the model distillation device 500, the model distillation device 500 includes: Figure 5 A first obtaining module 510 is configured to obtain a first sample. The first sample is generated using multiple teacher models, and the first sample includes a first prompt word and first reply information corresponding to the first prompt word.

[0152] A second obtaining module 520 is configured to input the first sample into a student model to obtain a first probability of the first reply information output by the student model. The forward evaluation index of the teacher model is greater than the forward evaluation index of the student model.

[0153] A first training module 530 is configured to train the student model according to the first probability to obtain a first distillation model.

[0154] Optionally, the first obtaining module 510 is configured to:

[0155] obtain the first prompt word;

[0156] input the first prompt word into the teacher model to obtain second reply information output by the teacher model;

[0157] obtain first attribute information of the second reply information;

[0158]

[0159] ​According to the first attribute information, the first reply information is determined from the second reply information;

[0160] According to the first prompt word and the first reply information, the first sample is determined.

[0161] Optionally, the first obtaining module 510 is configured to:

[0162] Determine a first matching degree between any third reply information in the second reply information and the first prompt word;

[0163] Determine a second matching degree between fourth reply information in the second reply information except the third reply information and the first prompt word;

[0164] According to a comparison result between the first matching degree and the second matching degree, respectively reconfigure initial attribute information of the third reply information and initial attribute information of the fourth reply information to obtain the first attribute information after reconfiguration.

[0165] Optionally, the first reply information includes the second reply information with the maximum value of the first attribute information.

[0166] Optionally, the first obtaining module 510 is configured to:

[0167] Obtain a target query sample and a query statement expansion condition;

[0168] According to the target query sample and the query statement expansion condition, a second prompt word for expanding a query statement is generated;

[0169] Based on the second prompt word, the target query sample is expanded by using a large model to generate the second query statement;

[0170] The first query statement is determined from the second query statement;

[0171] According to the first query statement, the first prompt word is generated.

[0172] Optionally, the first obtaining module 510 is configured to:

[0173] The query statement samples in the sample library are clustered to obtain a plurality of first clusters;

[0174] The target query sample is selected from the first cluster.

[0175] Optionally, the first obtaining module 510 is configured to:

[0176] The second query statement is clustered to obtain a plurality of second clusters;

[0177] selecting a target query statement from the second cluster;

[0178] determining the first query statement according to the target query statement.

[0179] Optionally, the apparatus can further include:

[0180] a third obtaining module configured to obtain a second sample, wherein the second sample is generated by using a plurality of teacher models, the second sample includes a third prompt word, fifth reply information and sixth reply information corresponding to the third prompt word, and a value of attribute information of the fifth reply information is greater than a value of attribute information of the sixth reply information;

[0181] a fourth obtaining module configured to input the third prompt word and the fifth reply information into the first distillation model to obtain a second probability of the fifth reply information output by the first distillation model;

[0182] the fourth obtaining module is configured to input the third prompt word and the sixth reply information into the first distillation model to obtain a third probability of the sixth reply information output by the first distillation model;

[0183] a second training module configured to train the first distillation model according to the second probability and the third probability to obtain a second distillation model.

[0184] Optionally, the third obtaining module is configured to:

[0185] obtain the third prompt word;

[0186] input the third prompt word into the teacher models respectively to obtain seventh reply information output by the teacher models;

[0187] obtain second attribute information of the seventh reply information;

[0188] determine the fifth reply information and the sixth reply information from the seventh reply information according to the second attribute information;

[0189] obtain the second sample according to the third prompt word, the fifth reply information and the sixth reply information.

[0190] Optionally, the fifth reply information includes the seventh reply information with the maximum value of the second attribute information;

[0191] the sixth reply information includes the seventh reply information with the minimum value of the second attribute information.

[0192] It should be noted that the foregoing model distillation method embodiment is also applicable to the model distillation device of the embodiment, and thus will not be described here.

[0193] In the embodiment of the present application, the first sample generated based on the plurality of teacher models is input into the student model, and the student model is supervised and trained by using the probability of the first reply information output by the student model, to obtain a first distillation model. Therefore, since the first sample is generated based on the plurality of teacher models, the student model is trained by using the first sample, so that the student model can learn the advantages of the plurality of teacher models, thereby realizing the comprehensive migration of the ability of the strong model to the weak model, and improving the accuracy of the distillation model.

[0194] To implement the above-mentioned embodiments, the embodiment of the present application also provides a reply information generation device. Figure 6 The structure diagram of the reply information generation device provided by an embodiment of the present application is shown.

[0195] As shown in Figure 6 The reply information generation device 600 comprises:

[0196] The first acquisition module 610 is configured to acquire input information.

[0197] The generation module 620 is configured to generate a prompt word according to the input information.

[0198] The second acquisition module 630 is configured to input the prompt word into a distillation model to acquire reply information output by the distillation model, wherein the distillation model is trained by using the model distillation device of the above-mentioned embodiment.

[0199] In the embodiment of the present application, since the distillation model is trained based on a plurality of teacher models to obtain samples, by inputting the prompt word obtained based on the input information into the distillation model, the accuracy of the reply information can be improved.

[0200] According to the embodiments of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0201] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the present application described and / or claimed in this document to the embodiments presented herein.

[0202] As shown in Figure 7 The device 700 includes a computing unit 701 that can perform various appropriate actions and processes in accordance with a computer program stored in a ROM (Read-Only Memory) 702 or a computer program loaded into a RAM (Random Access Memory) 703 from the storage unit 708. Various programs and data required for the operation of the device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An I / O (Input / Output) interface 705 is also connected to the bus 704.

[0203] Various components in the device 700 are connected to the I / O interface 705, including an input unit 706, such as a keyboard, a mouse, and the like; an output unit 707, such as various types of displays, speakers, and the like; a storage unit 708, such as a magnetic disk, a magneto-optical disk, and the like; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0204] The computing unit 701 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above, such as the model distillation method. For example, in some embodiments, the model distillation method can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the computing unit 701, one or more steps of the model distillation method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the model distillation method by any other appropriate means, such as by means of firmware.

[0205] Various implementations of the systems and techniques described above herein can be realized in digital electronic circuitry, integrated circuitry, a Field Programmable Gate Array (FPGA), an Application-Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a System on Chip (SOC), a Complex Programmable Logic Device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0206] Program code for carrying out methods of the present application can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be retrieved from a machine-readable medium or device and executed by a processor to produce a machine for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed as a stand-alone program, or in combination with other program codes, on the machine to produce a machine that implements the functions / acts specified in the flowcharts and / or block diagrams.

[0207] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include one or more lines of electrical connections, portable computer disks, hard disk drives, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory), or flash memory, fiber optics, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0208] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0209] The systems and techniques described herein can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, and a blockchain network.

[0210] The computer system can include clients and servers. The clients and the servers are generally remote from each other and typically interact through a communication network. The relationship of client and server is one of communication and distribution, with the server receiving requests from the client and transmitting data to the client. The client and the server can be implemented by using a cloud server, also known as a cloud computing server or a cloud host. The cloud server is a host product in the cloud computing service system, which solves the defects of large management difficulty and weak business scalability in traditional physical host and VPS (Virtual Private Server) service.

[0211] It should be noted that the electronic device used to implement the reply information generation method of the embodiments of the present application is similar to the above-mentioned electronic device, and therefore will not be described here.

[0212] According to the embodiments of the present application, the present application also provides a computer program product, when the processor in the computer program product executes, executes the model distillation method or the reply information generation method proposed in the above embodiments of the present application.

[0213] It should be understood that the steps shown above can be reordered, added, or deleted. For example, the steps described in the present application can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present application can be achieved, and the present application does not limit this.

[0214] The above detailed description does not limit the scope of the application. Various modifications, combinations, sub-combinations and alternatives can be made to the detailed embodiment disclosed herein without departing from the spirit and the principles of the application. Any modification, equivalent replacement or improvement made within the spirit and principles of the application shall fall within the scope of the application.

Claims

1. A model distillation method, comprising: inputting a first prompt word into a plurality of teacher models respectively to obtain second reply information output by each teacher model; obtaining first attribute information of the second reply information; determining first reply information from the second reply information according to the first attribute information; determining and obtaining a first sample according to the first prompt word and the first reply information; wherein the first reply information comprises the second reply information with the maximum value of the first attribute information; inputting the first sample into a student model to obtain a first probability of the first reply information output by the student model; wherein a forward evaluation index of the teacher model is greater than a forward evaluation index of the student model; training the student model according to the first probability to obtain a first distillation model; inputting a third prompt word into a plurality of teacher models respectively to obtain seventh reply information output by each teacher model; determining fifth reply information and sixth reply information from the seventh reply information according to second attribute information of the seventh reply information; obtaining a second sample according to the third prompt word, the fifth reply information and the sixth reply information; wherein the fifth reply information comprises the seventh reply information with the maximum value of the second attribute information, and the sixth reply information comprises the seventh reply information with the minimum value of the second attribute information; inputting the third prompt word and the fifth reply information into the first distillation model to obtain a second probability of the fifth reply information output by the first distillation model; inputting the third prompt word and the sixth reply information into the first distillation model to obtain a third probability of the sixth reply information output by the first distillation model; training the first distillation model according to the second probability and the third probability to obtain a second distillation model.

2. The method of claim 1, wherein, The obtaining of the first attribute information of the second reply information comprises: determining a first matching degree between any third reply information in the second reply information and the first prompt word; determining a second matching degree between fourth reply information other than the third reply information in the second reply information and the first prompt word; according to a comparison result between the first matching degree and the second matching degree, respectively reconfiguring initial attribute information of the third reply information and initial attribute information of the fourth reply information to obtain the first attribute information after reconfiguration.

3. The method of claim 1, wherein, The generation manner of the first prompt word comprises: obtaining a target query example and a query sentence expansion condition; generating a second prompt word for expanding a query sentence according to the target query example and the query sentence expansion condition; expanding the target query example based on the second prompt word using a large model to generate a second query sentence; determining a first query sentence from the second query sentence; generating the first prompt word according to the first query sentence.

4. The method of claim 3, wherein, The obtaining of the target query example and the query sentence expansion condition comprises: clustering query sentence examples in an example library to obtain a plurality of first clusters; selecting the target query example from the first clusters.

5. The method of claim 3, wherein, The first query statement is determined from the second query statement, including: Clustering the second query statement to obtain a plurality of second clusters; Selecting a target query statement from the second clusters; According to the target query statement, the first query statement is determined.

6. A method for generating reply information, comprising: obtaining input information; generating a prompt word according to the input information; inputting the prompt word into a distillation model to obtain reply information output by the distillation model; wherein the distillation model is a second distillation model trained by the model distillation method of any one of claims 1-5.

7. A model distillation device, comprising: a first obtaining module configured to input a first prompt word into a plurality of teacher models respectively to obtain second reply information output by each teacher model; obtaining first attribute information of the second reply information; determining first reply information from the second reply information according to the first attribute information; and determining and obtaining a first sample according to the first prompt word and the first reply information; wherein the first reply information includes the second reply information with the maximum value of the first attribute information; a second obtaining module configured to input the first sample into a student model to obtain a first probability of the first reply information output by the student model; wherein a positive evaluation index of the teacher model is greater than a positive evaluation index of the student model; a first training module configured to train the student model according to the first probability to obtain a first distillation model; a third obtaining module configured to input a third prompt word into a plurality of teacher models respectively to obtain seventh reply information output by each teacher model; determine fifth reply information and sixth reply information from the seventh reply information according to second attribute information of the seventh reply information; and obtain a second sample according to the third prompt word, the fifth reply information and the sixth reply information; wherein the fifth reply information includes the seventh reply information with the maximum value of the second attribute information, and the sixth reply information includes the seventh reply information with the minimum value of the second attribute information; a fourth obtaining module configured to input the third prompt word and the fifth reply information into the first distillation model to obtain a second probability of the fifth reply information output by the first distillation model; the fourth obtaining module is configured to input the third prompt word and the sixth reply information into the first distillation model to obtain a third probability of the sixth reply information output by the first distillation model; a second training module configured to train the first distillation model according to the second probability and the third probability to obtain a second distillation model.

8. The apparatus of claim 7, wherein, The first obtaining module is configured to: determine a first matching degree between any third reply information in the second reply information and the first prompt word; determine a second matching degree between fourth reply information in the second reply information except the third reply information and the first prompt word; According to a comparison result between the first matching degree and the second matching degree, the initial attribute information of the third reply information and the initial attribute information of the fourth reply information are reconfigured respectively to obtain the first attribute information after reconfiguration.

9. The apparatus of claim 7, wherein, The generation manner of the first prompt word comprises: Obtaining a target query example and a query sentence expansion condition; According to the target query example and the query sentence expansion condition, a second prompt word for expanding a query sentence is generated; Based on the second prompt word, the target query example is expanded by using a large model to generate a second query sentence; A first query sentence is determined from the second query sentence; According to the first query sentence, the first prompt word is generated.

10. The apparatus of claim 9, wherein, The first obtaining module is configured to: Cluster query sentence examples in a sample library to obtain a plurality of first clusters; The target query example is selected from the first clusters.

11. The apparatus of claim 9, wherein, The first obtaining module is configured to: Cluster the second query sentence to obtain a plurality of second clusters; The target query sentence is selected from the second clusters; According to the target query sentence, the first query sentence is determined.

12. A reply information generation device, comprising: A first obtaining module for obtaining input information; A generation module for generating a prompt word according to the input information; A second obtaining module for inputting the prompt word into a distillation model to obtain reply information output by the distillation model; wherein the distillation model is a second distillation model trained by the model distillation device of any one of claims 7-11.

13. An electronic device, comprising: At least one processor; And A memory connected in communication with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to make the computer execute the method according to any one of claims 1-6.

15. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of any one of claims 1-6.

Images