Model distillation method and device, electronic equipment and storage medium

Abstract

The invention discloses a model distillation method and device, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the artificial intelligence fields of large models, deep learning and the like. The specific implementation scheme is as follows: acquiring a first sample; wherein the first sample is generated by adopting a plurality of teacher models, and the first sample comprises a first cue word and first reply information corresponding to the first cue word; inputting the first sample into a student model to obtain a first probability of first reply information output by the student model; wherein the forward evaluation index of the teacher model is greater than the forward evaluation index of the student model; and training the student model according to the first probability to obtain a first distillation model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to artificial intelligence fields such as large models and deep learning, and specifically to a model distillation method, device, electronic device and storage medium. Background Art

[0002] Model distillation is a technique that transfers knowledge from a large and complex model (as the teacher model) to a small and simple model (as the student model), aiming to improve the model's inference efficiency and performance while maintaining performance close to that of the teacher model. Summary of the invention

[0003] The present application provides a model distillation method, device, electronic device and storage medium.

[0004] According to one aspect of the present application, a model distillation method is provided, comprising:

[0005] Obtain a first sample; wherein the first sample is generated by using multiple teacher models, and the first sample includes a first prompt word and first answer information corresponding to the first prompt word;

[0006] Inputting the first sample into a student model to obtain a first probability of the first reply information output by the student model; wherein the positive evaluation index of the teacher model is greater than the positive evaluation index of the student model;

[0007] The student model is trained according to the first probability to obtain a first distillation model.

[0008] According to another aspect of the present application, a method for generating reply information is provided, comprising:

[0009] Get input information;

[0010] Generate prompt words according to the input information;

[0011] The prompt word is input into a distillation model to obtain response information output by the distillation model; wherein the distillation model is trained using the model distillation method described in the above-mentioned embodiment.

[0012] According to another aspect of the present application, a model distillation apparatus is provided, comprising:

[0013] A first acquisition module is used to acquire a first sample; wherein the first sample is generated by using multiple teacher models, and the first sample includes a first prompt word and first answer information corresponding to the first prompt word;

[0014] A second acquisition module is used to input the first sample into a student model to obtain a first probability of the first reply information output by the student model; wherein the positive evaluation index of the teacher model is greater than the positive evaluation index of the student model;

[0015] The first training module is used to train the student model according to the first probability to obtain a first distillation model.

[0016] According to another aspect of the present application, a device for generating reply information is provided, comprising:

[0017] A first acquisition module, used to acquire input information;

[0018] A generating module, used for generating prompt words according to the input information;

[0019] The second acquisition module is used to input the prompt word into the distillation model to obtain the response information output by the distillation model; wherein the distillation model is trained by the model distillation device described in the above-mentioned embodiment of the other aspect.

[0020] According to another aspect of the present application, there is provided an electronic device, including:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the method described in the above embodiment.

[0024] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to the above embodiment.

[0025] According to another aspect of the present application, a computer program product is provided, including a computer program, wherein the computer program implements the steps of the method described in the above embodiment when executed by a processor.

[0026] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0028] Figure 1 A schematic diagram of a flow chart of a model distillation method provided in one embodiment of the present application;

[0029] Figure 2 A schematic diagram of a model distillation method provided in another embodiment of the present application;

[0030] Figure 3 A schematic diagram of a model distillation method provided in another embodiment of the present application;

[0031] Figure 4 A schematic diagram of a model distillation process provided in an embodiment of the present application;

[0032] Figure 5 A schematic diagram of the structure of a model distillation device provided in one embodiment of the present application;

[0033] Figure 6 A schematic diagram of the structure of a device for generating reply information provided in an embodiment of the present application;

[0034] Figure 7 It is a block diagram of an electronic device used to implement the model distillation method of an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0036] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations and do not violate public order and good morals.

[0037] The model distillation method, device, electronic device and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.

[0038] In some embodiments, knowledge distillation can be performed based on a single teacher model. However, a single teacher model is often only good in certain aspects, and it is difficult for a student model to learn comprehensive knowledge from a teacher model, resulting in a relatively low accuracy of the distillation model.

[0039] Based on this, an embodiment of the present application provides a model distillation method. Figure 1 A schematic diagram of the process of the model distillation method provided in one embodiment of the present application.

[0040] The model distillation method of the embodiment of the present application can be performed by the model distillation device of the embodiment of the present application, and the device can be configured in an electronic device.

[0041] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0042] like Figure 1 As shown, the model distillation method includes:

[0043] Step 101: Obtain a first sample.

[0044] The first sample may be generated by using multiple teacher models. Exemplarily, the teacher model may be a large model, or a distillation model obtained by distillation training based on multiple large models, and the positive evaluation index of the distillation model is greater than a preset threshold.

[0045] Exemplarily, the positive evaluation indicators may include accuracy, precision, recall, etc. If there are multiple positive evaluation indicators, each positive evaluation indicator may be greater than the corresponding preset threshold. For example, the accuracy of the distillation model serving as the teacher model is greater than the first threshold, and the precision is greater than the second threshold.

[0046] Exemplarily, the large model may be a large language model, or a text-to-graph model, or a graph-to-text model, or other types of large models, which are not limited in this application.

[0047] It should be noted that the multiple teacher models used to generate the first sample are of the same type, for example, they are all large language models, or they are all text-based graph models.

[0048] In the present application, the first sample may include a first prompt word, first reply information corresponding to the first prompt word, and the like.

[0049] The first reply information may be reply information output by one of the teacher models among the multiple teacher models based on the first prompt word.

[0050] The modality of the first reply information may be text, image, or both text and image, etc., and this application does not impose any limitation on this.

[0051] It should be noted that the number of the obtained first samples may be one or more, and there is no limitation on this.

[0052] Exemplarily, if there are multiple first samples, the multiple first samples may be for different scenarios in the target field.

[0053] Step 102: input the first sample into the student model to obtain a first probability of first reply information output by the student model.

[0054] The positive evaluation index of the teacher model may be greater than the positive evaluation index of the student model. Optionally, the difference between the positive evaluation index of the teacher model and the positive evaluation index of the student model may be greater than a third threshold.

[0055] In the present application, the first sample can be filled into a preset prompt template to generate a prompt word for guiding the student model to output the probability of the first reply information, and the prompt word is input into the student model to obtain the first probability of the first reply information output by the student model.

[0056] The first probability may refer to the probability of inputting the first prompt word into the student model and the student model outputting the first reply information.

[0057] Step 103: train the student model according to the first probability to obtain a first distillation model.

[0058] In the present application, the model loss can be determined according to the first probability, and the parameters of the student model can be adjusted according to the model loss until the model training end condition is met to obtain the first distillation model.

[0059] Exemplarily, the negative of the first probability can be used as the model loss, and the parameters of the student model can be adjusted based on the model loss so that the first probability output by the model becomes larger and larger until the end condition of the model training is met, thereby obtaining the first distillation model.

[0060] The model distillation method of the embodiment of the present application can be widely used in multiple fields. For example, the first distillation model can be used to build a more intelligent and accurate question-answering system, or to apply chatbots, virtual assistants and other applications to generate more coherent and creative dialogue content, or to apply to summary generation tasks such as news, documents, and reports to generate more accurate and comprehensive automatic summaries.

[0061] For example, in a knowledge question-answering scenario, the question input by the user can be obtained, and corresponding prompt words can be generated according to the input question, and the prompt words can be input into the first distillation model to obtain the answer text output by the first distillation model. For example, if the question input by the user is "What is the Wensheng graph model", the first distillation model can output an explanation of the Wensheng graph model.

[0062] In the embodiment of the present application, the first sample generated based on multiple teacher models is input into the student model, and the probability of the first reply information output by the student model is used to supervise the student model to obtain the first distillation model. Therefore, since the first sample is generated based on multiple teacher models, the student model is trained using the first sample, so that the student model can learn the strengths of multiple teacher models, thereby achieving the full transfer of the capabilities of the strong model to the weak model, and improving the accuracy of the distillation model.

[0063] Figure 2 A schematic flow chart of a model distillation method provided in another embodiment of the present application.

[0064] like Figure 2 As shown, the model distillation method includes:

[0065] Step 201, obtaining a first prompt word.

[0066] In the present application, a target query sample can be obtained from a sample library, and a query statement expansion condition can be obtained. According to the target query sample and the query statement expansion condition, a second prompt word for expanding the query statement can be generated. Based on the second prompt word, the target query sample can be expanded using a large model to generate a second query statement, and then the first query statement can be determined from the second query statement, and finally the first prompt word can be generated according to the first query statement. Thus, based on the target query sample and the query statement expansion condition, the target query sample can be expanded, which can enrich the query statement and further enrich the prompt word.

[0067] The sample library may include query examples in different scenarios in the target field. For example, the sample library includes typical problems in different scenarios such as base stations and core networks in the communication field.

[0068] As an example, one or more target query samples may be randomly selected from the sample library, or query statement samples whose character length exceeds a preset length may be selected from the sample library as target query samples, or target query samples may be selected from the sample library based on the similarity between any two query statement samples in the sample library.

[0069] As another example, query examples in the example library may be clustered to obtain multiple first-class clusters, and a target query example may be selected from each first-class cluster, wherein the target query example selected from each first-class cluster may be one or more.

[0070] For example, the K-means algorithm is used to cluster the query examples in the sample library into K clusters, and n samples are randomly selected from each cluster each time, forming a total of n*K samples and putting them into the first prompt word as reference examples. K can be an integer greater than 1, and n can be an integer greater than or equal to 1.

[0071] It should be noted that other methods may also be used to cluster the query statement samples in the sample library, and this is not limited.

[0072] Therefore, by selecting target query samples from multiple first clusters obtained by clustering the sample library, query statement samples of multiple scenarios are used as reference examples for query statements generated by the large model, thereby improving the richness of the generated second query statement.

[0073] The query statement extension condition indicates the requirements for extending the query statement. Exemplarily, the query statement extension condition may include task definition, length limit, and output requirement. For example, the task definition is to generate a query statement for vacation, the length limit is that the generated query statement cannot be less than 2 words and not more than 20 words, and the output requirement is to output the generated query statement in the form of a Python list.

[0074] Exemplarily, the prompt template for guiding the expansion of the query statement may be filled in according to the target query example, the query statement expansion condition, etc., to obtain the second prompt word.

[0075] The second prompt word may include task instructions, query statement expansion conditions, target query examples, etc. For example, the task instruction may be "You are a query expansion expert. According to the query statement expansion conditions and target query examples, generate 20 user-asked queries."

[0076] Regarding determining the first query statement from the second query statement, illustratively, one or more query statements may be randomly selected from the second query statements as the first query statement.

[0077] Exemplarily, the second query statement may also be clustered to obtain a plurality of second clusters, and a target query statement may be selected from the second clusters, and the target query statement selected from the plurality of second clusters may be used as the first query statement.

[0078] The second query statement may be clustered using a K-means algorithm, or may be clustered using other methods, which are not limited thereto.

[0079] Therefore, by clustering the second query statements, similar query statements can be filtered, and typical query statements can be selected as the first query statements, thereby ensuring the diversity of the first query statements.

[0080] Exemplarily, if there are multiple first query statements, a corresponding first prompt word may be generated for each first query statement, thereby obtaining multiple first prompt words.

[0081] It should be noted that the above method may also be used in advance to expand the query sentence examples in the sample library using a large model to obtain a first query sentence, and then generate a first prompt word based on the first query sentence.

[0082] Step 202: input the first prompt words into the teacher model respectively to obtain the second reply information output by the teacher model.

[0083] Exemplarily, the first prompt word may be input into a plurality of teacher models respectively to obtain the second reply information output by each teacher model.

[0084] Step 203: Obtain first attribute information of the second reply information.

[0085] The first attribute information may be in the form of a score or confidence level, and the value of the first attribute information of the second reply information may be used to characterize the quality of the second reply information. The higher the value of the first attribute information, the higher the quality of the second reply information.

[0086] As a possible implementation method, each second reply information has the same initial attribute information, and the first matching degree between any third reply information in the second reply information and the first prompt word can be determined, and the second matching degree between other fourth reply information in the second reply information except the third reply information and the first prompt word can be determined, and then based on the comparison result between the first matching degree and the second matching degree, the initial attribute information of the third reply information and the initial attribute information of the fourth reply information are reconfigured respectively to obtain the first attribute information after the third reply information is reconfigured and the first attribute information after the fourth reply information is reconfigured.

[0087] Among them, the first matching degree can be used to characterize the ability of the third reply information to solve the first prompt word, and the second matching degree can be used to characterize the ability of the fourth reply information to solve the first prompt word. The higher the matching degree, the higher the ability to solve the first prompt word.

[0088] Taking the first attribute information in the form of a score as an example, if the first matching degree is greater than the second matching degree, the initial score of the third reply information can be added to the preset score, and the initial score of the fourth reply information can be subtracted from the preset score; if the first matching degree is less than the second matching degree, the initial score of the third reply information can be subtracted from the preset score, and the initial score of the fourth reply information can be added to the preset score; if the first matching degree is equal to the second matching degree, the initial scores of the third reply information and the fourth reply information can both remain unchanged.

[0089] Exemplarily, a large model can also be used to score the capabilities of the third response information and the fourth response information to solve the first prompt, and output the magnitude relationship of the scores between the third response information and the fourth response information. Based on this magnitude relationship, the initial scores of the third response information and the fourth response information are updated.

[0090] For example, every time two pieces of the second response information are taken from the second response information, denoted as X and Y, a scoring prompt is constructed and input into the large model to obtain the result of X > Y, or X = Y, or X < Y. Suppose the scoring prompt is "You are a scoring expert, and your task is to judge which response information can better solve the business prompt, and give the conclusion of X > Y, or X = Y, or X < Y. Input: business prompt = {**}, X = {**}, Y = {**}, Output: ".

[0091] For example, if the second response information includes Resp_1, Resp_2, Resp_3, and Resp_4, the large model can be used to score Resp_1 and Resp_2, and the initial scores of Resp_1 and Resp_2 are updated respectively according to the output result of the large model. The large model is used to score Resp_1 and Resp_3, and the current score of Resp_1 and the initial score of Resp_3 are updated respectively according to the output result of the large model. The large model is used to score Resp_1 and Resp_4, and the current score of Resp_1 and the initial score of Resp_4 are updated respectively according to the output result of the large model. The large model is used to score Resp_2 and Resp_3, and the current scores of Resp_2 and Resp_3 are updated respectively according to the output result of the large model. The large model is used to score Resp_2 and Resp_4, and the current scores of Resp_2 and Resp_4 are updated respectively according to the output result of the large model. The large model is used to score Resp_3 and Resp_4, and the current scores of Resp_3 and Resp_4 are updated respectively according to the output result of the large model.

[0092] Since the positions of the third response information and the fourth response information in the scoring prompt may affect the scoring, the positions of the third response information and the fourth response information in the scoring prompt can be exchanged and scored again, and the score is updated according to the output comparison result, thereby improving the accuracy of the score.

[0093] Therefore, by comparing the ability of each pair of reply information in the second reply information to solve the first prompt word, the attribute information of each pair of reply information is reconfigured according to the comparison results to obtain the final attribute information. This is more accurate than the attribute information obtained by performing quality assessment on each second reply information separately, thereby improving the accuracy of the quality assessment of the second reply information.

[0094] Taking the first attribute information in the form of a score as an example, illustratively, each second reply information can be individually input into the reward model for scoring to obtain a score for each second reply information.

[0095] Step 204: Determine the first reply information from the second reply information based on the first attribute information.

[0096] Exemplarily, the second reply information in which the value of the first attribute information is greater than the fourth threshold may be used as the first reply information.

[0097] Exemplarily, the first reply information may include the second reply information with the largest value of the first attribute information. Exemplarily, for each first prompt word, the second reply information with the largest value of the first attribute information in the second reply information may be used as the first reply information. Thus, the first reply information includes the second reply information with the largest value of the first attribute information, which allows the student model to learn from multiple teacher models at the same time, and the student model can quickly learn the capabilities of multiple teacher models, thereby improving learning efficiency.

[0098] Step 205: Determine a first sample according to the first prompt word and the first reply information.

[0099] Exemplarily, for each first prompt word, if there are multiple first reply information, the first prompt word and each first reply information may be regarded as a first sample.

[0100] Exemplarily, for each first prompt word, if the first reply information includes reply information with the highest score among the second reply information, the first prompt word and the first reply information may be regarded as a first sample.

[0101] Step 206: input the first sample into the student model to obtain a first probability of first reply information output by the student model.

[0102] Step 207: train the student model according to the first probability to obtain a first distillation model.

[0103] In the present application, steps 206 and 207 are implemented in any of the embodiments of the present application, and therefore will not be described in detail here.

[0104] In an embodiment of the present application, the first prompt word is input into multiple teacher models respectively, the second reply information output by the multiple teacher models is obtained, and the first attribute information of the second reply information is obtained, and the first reply information is filtered out from the second reply information based on the first attribute information of the second reply information, thereby enriching the diversity of the first reply information. Therefore, the student model is trained using the first sample obtained based on the first prompt word and the first reply information, so as to improve the accuracy of the first distillation model.

[0105] Figure 3 A schematic flow chart of a model distillation method provided in another embodiment of the present application.

[0106] like Figure 3 As shown, the model distillation method includes:

[0107] Step 301: Acquire a first sample and a second sample.

[0108] In the present application, step 301 may adopt the method described in the above embodiment to obtain the first sample, which will not be described in detail here.

[0109] In the present application, the second sample may be generated using the above-mentioned multiple teacher models.

[0110] The second sample may include a third prompt word, fifth reply information corresponding to the third prompt word, and sixth reply information.

[0111] The value of the attribute information of the fifth reply information may be greater than the value of the attribute information of the sixth reply information. Taking the attribute information in the form of a score as an example, the score of the fifth reply information may be greater than the score of the sixth reply information, indicating that the quality of the fifth reply information is higher than that of the sixth reply information. On this basis, the difference between the score of the fifth reply information and the score of the sixth reply information may also be greater than the fifth threshold.

[0112] Among them, the fifth reply information can be the reply information output by one of the multiple teacher models based on the third prompt word, and the sixth reply information can also be the reply information output by one of the multiple teacher models based on the third prompt word.

[0113] In addition, the fifth reply information and the sixth reply information can be output by the same teacher model or by different teacher models, and there is no limitation on this.

[0114] Exemplarily, the third prompt word can be input into the teacher model respectively to obtain the seventh reply information output by the teacher model, and to obtain the second attribute information of the seventh reply information, and based on the second attribute information of the seventh reply information, the fifth reply information and the sixth reply information are determined from the seventh reply information, and then based on the third prompt word, the fifth reply information and the sixth reply information, the second sample is obtained.

[0115] Among them, the method of obtaining the second attribute information of the seventh reply information is similar to the method of obtaining the first attribute information of the second reply information mentioned above, so it will not be repeated here.

[0116] Therefore, by inputting the third prompt word into multiple teacher models respectively, obtaining the seventh reply information output by the multiple teacher models, and obtaining the second attribute information of the seventh reply information, based on the second attribute information of the seventh reply information, the fifth reply information and the sixth reply information are screened out from the seventh reply information, thereby enriching the second sample. Therefore, by training the first distillation model based on the second sample, the accuracy of the second distillation model can be improved.

[0117] Exemplarily, the fifth reply information may include the seventh reply information in which the value of the second attribute information is greater than the sixth threshold, and the sixth reply information may include the seventh reply information in which the value of the second attribute information is less than the seventh threshold, wherein the sixth threshold is greater than the seventh threshold.

[0118] Exemplarily, the seventh reply information in which the value of the second attribute information is greater than the sixth threshold can be used as the fifth reply information, and the seventh reply information in which the value of the second attribute information is less than the seventh threshold can be used as the sixth reply information.

[0119] Taking the second attribute information in the form of a score as an example, the seventh reply information with a score greater than the sixth threshold can be used as the fifth reply information, and the seventh reply information with a score less than the seventh threshold can be used as the sixth reply information.

[0120] Exemplarily, two reply information pieces whose difference in scores in the seventh reply information is greater than the fifth threshold value may be respectively used as the fifth reply information and the sixth reply information.

[0121] Exemplarily, the fifth reply information may include the seventh reply information having the largest value of the second attribute information, and the sixth reply information may include the seventh reply information having the smallest value of the second attribute information.

[0122] Exemplarily, for each second prompt word, the seventh reply information with the largest value of the second attribute information among the seventh reply information can be used as the fifth reply information, and the seventh reply information with the smallest value of the second attribute information among the seventh reply information can be used as the sixth reply information.

[0123] Taking the second attribute information in the form of a score as an example, for each second prompt word, the seventh reply information with the largest score among the seventh reply information can be used as the fifth reply information, and the seventh reply information with the smallest score among the seventh reply information can be used as the sixth reply information.

[0124] Therefore, by taking the seventh reply information with the largest value of the second attribute information as the fifth reply information and the seventh reply information with the smallest value of the second attribute information as the fifth reply information, preference samples can be obtained, and model training can be performed based on the preference samples to improve the generalization ability of the distillation model.

[0125] Exemplarily, the third prompt word can be a prompt word generated based on a query sentence sample selected from the above sample library, using a large model to expand the query sentence to obtain a new query sentence. The specific method can refer to the method for generating the first prompt word recorded in the above embodiment.

[0126] It should be noted that the third prompt word may be the same as the first prompt word, and the fifth reply information may be the same as the first reply information; or the third prompt word may be different from the first prompt word, and the fifth reply information may be the same as the first reply information; or the third prompt word may be different from the first prompt word, and the fifth reply information may be different from the first reply information. This application does not limit this.

[0127] Step 302: Input the first sample into the student model to obtain a first probability of first reply information output by the student model.

[0128] Step 303: train the student model according to the first probability to obtain a first distillation model.

[0129] In the present application, steps 302 to 303 are implemented in any of the embodiments of the present application, and therefore will not be described in detail here.

[0130] Step 304: input the third prompt word and the fifth reply text in the second sample into the first distillation model to obtain a second probability of the fifth reply information output by the first distillation model.

[0131] Exemplarily, the third prompt word and the fifth reply information may be input into the first distillation model together to obtain a second probability of the fifth reply information output by the first distillation model.

[0132] Among them, the second probability can represent the probability of inputting the third prompt word into the first distillation model and the first distillation model outputting the fifth reply information.

[0133] Step 305: input the third prompt word and the sixth reply text in the second sample into the first distillation model to obtain a third probability of the sixth reply information output by the first distillation model.

[0134] Exemplarily, the third prompt word and the sixth reply information may be input into the first distillation model together to obtain a third probability of the sixth reply information output by the first distillation model.

[0135] Among them, the third probability can represent the probability of inputting the third prompt word into the first distillation model and the first distillation model outputting the sixth reply information.

[0136] Step 306: Train the first distillation model according to the second probability and the third probability to obtain a second distillation model.

[0137] In the present application, the negative of the second probability and the sum of the third probability can be used as the model loss, and the parameters of the first distillation model can be adjusted based on the model loss to make the second probability larger and larger, while the third probability is getting lower and lower, until the end condition of the model training is met, and the second distillation model is obtained.

[0138] In an embodiment of the present application, on the basis of using the first sample to perform supervised training on the student model, the first distillation model obtained by the supervised training can also be subjected to reinforcement learning training by using high-quality response information and low-quality response information, thereby improving the ability of the distillation model to draw inferences from one example and improving the generalization ability of the distillation model.

[0139] In order to facilitate the understanding of the model distillation method of this application, the following Figure 4 To explain, Figure 4 A schematic diagram of a model distillation process provided in an embodiment of the present application.

[0140] like Figure 4As shown, based on the query samples in the seed query set, the query samples can be expanded using the large model to generate more similar queries, that is, to generate a query set. Based on the queries in the generated query set, Prompt, that is, the prompt word, is obtained. Prompt is input into N teacher models such as teacher model 1, teacher model 2, ..., teacher model N, etc., to obtain N reply information Resp1, Resp2, Resp3, ..., RespN, and the reward model is used to reply to the N reply information to obtain the score Score1 of Resp1, the score Score3 of Resp2, ..., and the score ScoreN of RespN. Afterwards, based on the score of each reply information, the reply information with the largest score is selected from the N reply information and recorded as Resp_good, forming<Prompt,Resp_good> , get the instruction data set, select the reply information with the smallest score and record it as Resp_bad, and form<Prompt,Resp_good,Resp_bad> , obtain the preference data set, and use the instruction data set and the preference data set as the training data set.

[0141] After obtaining the training data set, the instruction data set in the training data set can be used to perform supervised training on the student model to obtain a first distillation model. Then, the preference data set in the training data set can be used to perform reinforcement learning training on the first distillation model to obtain a second distillation model.

[0142] Among them, Prompt and Resp_good can be input into the student model together, and the student model can be supervised and trained based on the probability of Resp_good output by the student model.

[0143] Among them, Prompt and Resp_good can be input into the first distillation model together to obtain the probability of Resp_good output by the first distillation model, and Prompt and Resp_bad can be input into the first distillation model together to obtain the probability of Resp_bad output by the first distillation model, and then based on the probabilities of Resp_good and Resp_bad, the first distillation model is subjected to reinforcement learning training to obtain the second distillation model.

[0144] The seed query set in this embodiment is also the sample library in the above embodiment, and the query sample is also the query statement sample in the above embodiment.

[0145] In order to implement the above embodiment, the embodiment of the present application also proposes a method for generating reply information. The method for generating reply information may include: obtaining input information, generating prompt words according to the input information, and inputting the prompt words into a distillation model to obtain reply information output by the distillation model.

[0146] The distillation model used may be trained by the model distillation method in the above embodiment. For example, the distillation model may be the first distillation model or the second distillation model.

[0147] The input information may be text or image, or text and image, which is not limited.

[0148] For example, the corresponding prompt word model can be filled according to the input information to obtain the prompt word. For example, the prompt word is "You are an expert in writing structured query language. According to the {Query} input by the user, you write a suitable structured query language statement and output: ".

[0149] In the embodiment of the present application, since the distillation model is trained based on samples obtained from multiple teacher models, the accuracy of the reply information can be improved by inputting prompt words obtained based on the input information into the distillation model.

[0150] In order to implement the above embodiment, the embodiment of the present application also proposes a model distillation device. Figure 5 A schematic structural diagram of a model distillation apparatus provided in one embodiment of the present application.

[0151] like Figure 5 As shown, the model distillation apparatus 500 comprises:

[0152] A first acquisition module 510 is used to acquire a first sample; wherein the first sample is generated by using multiple teacher models, and the first sample includes a first prompt word and first answer information corresponding to the first prompt word;

[0153] A second acquisition module 520 is used to input the first sample into a student model to obtain a first probability of the first reply information output by the student model; wherein the positive evaluation index of the teacher model is greater than the positive evaluation index of the student model;

[0154] The first training module 530 is used to train the student model according to the first probability to obtain a first distillation model.

[0155] Optionally, the first acquisition module 510 is used to:

[0156] Obtaining the first prompt word;

[0157] Inputting the first prompt words into the teacher model respectively to obtain second answer information output by the teacher model;

[0158] Acquire first attribute information of the second reply information;

[0159] determining the first reply information from the second reply information according to the first attribute information;

[0160] The first sample is determined according to the first prompt word and the first reply information.

[0161] Optionally, the first acquisition module 510 is used to:

[0162] Determine a first matching degree between any third reply information in the second reply information and the first prompt word;

[0163] determining a second matching degree between fourth reply information other than the third reply information in the second reply information and the first prompt word;

[0164] According to the comparison result between the first matching degree and the second matching degree, the initial attribute information of the third reply information and the initial attribute information of the fourth reply information are reconfigured respectively to obtain the reconfigured first attribute information.

[0165] Optionally, the first reply information includes the second reply information having the largest value of the first attribute information.

[0166] Optionally, the first acquisition module 510 is used to:

[0167] Obtain target query samples and query statement expansion conditions;

[0168] generating a second prompt word for expanding the query statement according to the target query example and the query statement expansion condition;

[0169] Based on the second prompt word, the target query example is expanded using the large model to generate the second query statement;

[0170] Determine the first query statement from the second query statement;

[0171] The first prompt word is generated according to the first query sentence.

[0172] Optionally, the first acquisition module 510 is used to:

[0173] Clustering query statement samples in the sample library to obtain multiple first-category clusters;

[0174] The target query example is selected from the first cluster.

[0175] Optionally, the first acquisition module 510 is used to:

[0176] Clustering the second query statement to obtain a plurality of second clusters;

[0177] Selecting a target query statement from the second cluster;

[0178] The first query statement is determined according to the target query statement.

[0179] Optionally, the device may further include:

[0180] A third acquisition module is used to acquire a second sample; wherein the second sample is generated by using a plurality of the teacher models, the second sample includes a third prompt word, fifth reply information and sixth reply information corresponding to the third prompt word, and the value of the attribute information of the fifth reply information is greater than the value of the attribute information of the sixth reply information;

[0181] a fourth acquisition module, configured to input the third prompt word and the fifth reply text into the first distillation model to obtain a second probability of the fifth reply information output by the first distillation model;

[0182] The fourth acquisition module is used to input the third prompt word and the sixth reply text into the first distillation model to obtain a third probability of the sixth reply information output by the first distillation model;

[0183] The second training module is used to train the first distillation model according to the second probability and the third probability to obtain a second distillation model.

[0184] Optionally, the third acquisition module is used to:

[0185] Obtaining the third prompt word;

[0186] inputting the third prompt words into the teacher model respectively to obtain seventh answer information output by the teacher model;

[0187] Obtaining second attribute information of the seventh reply information;

[0188] determining the fifth reply information and the sixth reply information from the seventh reply information according to the second attribute information;

[0189] The second sample is obtained according to the third prompt word, the fifth reply information and the sixth reply information.

[0190] Optionally, the fifth reply information includes the seventh reply information having the largest value of the second attribute information;

[0191] The sixth reply information includes the seventh reply information having the smallest value of the second attribute information.

[0192] It should be noted that the explanation of the aforementioned model distillation method embodiment is also applicable to the model distillation device of this embodiment, so it will not be repeated here.

[0193] In the embodiment of the present application, the first sample generated based on multiple teacher models is input into the student model, and the probability of the first reply information output by the student model is used to supervise the student model to obtain the first distillation model. Therefore, since the first sample is generated based on multiple teacher models, the student model is trained using the first sample, so that the student model can learn the strengths of multiple teacher models, thereby achieving the full transfer of the capabilities of the strong model to the weak model, and improving the accuracy of the distillation model.

[0194] In order to implement the above embodiment, the embodiment of the present application also proposes a device for generating reply information. Figure 6 A schematic diagram of the structure of a device for generating reply information provided in one embodiment of the present application.

[0195] like Figure 6 As shown, the reply information generating device 600 includes:

[0196] A first acquisition module 610, used to acquire input information;

[0197] A generating module 620, used to generate a prompt word according to the input information;

[0198] The second acquisition module 630 is used to input the prompt word into the distillation model to obtain the response information output by the distillation model; wherein the distillation model is trained by the model distillation device described in the above embodiment.

[0199] In the embodiment of the present application, since the distillation model is trained based on samples obtained from multiple teacher models, the accuracy of the reply information can be improved by inputting prompt words obtained based on the input information into the distillation model.

[0200] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0201] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0202] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 702 or a computer program loaded from a storage unit 708 to a RAM (Random Access Memory) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An I / O (Input / Output) interface 705 is also connected to the bus 704.

[0203] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0204] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the model distillation method. For example, in some embodiments, the model distillation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the model distillation method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the model distillation method in any other suitable manner (eg, by means of firmware).

[0205] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0206] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0207] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0208] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0209] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0210] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0211] It should be noted that the electronic device used to implement the method for generating reply information in the embodiment of the present application is similar to the electronic device described above, so it will not be described in detail here.

[0212] According to an embodiment of the present application, the present application also provides a computer program product, which, when an instruction processor in the computer program product executes, executes the model distillation method proposed in the above-mentioned embodiment of the present application, or the method for generating reply information.

[0213] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0214] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A model distillation method, comprising: Obtain a first sample; wherein the first sample is generated by using multiple teacher models, and the first sample includes a first prompt word and first answer information corresponding to the first prompt word; Inputting the first sample into a student model to obtain a first probability of the first reply information output by the student model; wherein the positive evaluation index of the teacher model is greater than the positive evaluation index of the student model; The student model is trained according to the first probability to obtain a first distillation model.

2. The method of claim 1, wherein: The obtaining of the first sample comprises: Obtaining the first prompt word; Inputting the first prompt words into the teacher model respectively to obtain second answer information output by the teacher model; Acquire first attribute information of the second reply information; determining the first reply information from the second reply information according to the first attribute information; The first sample is determined according to the first prompt word and the first reply information.

3. The method of claim 2, wherein: The obtaining of the first attribute information of the second reply information includes: Determine a first matching degree between any third reply information in the second reply information and the first prompt word; Determine a second matching degree between the first prompt word and the other fourth reply information in the second reply information except the third reply information; According to the comparison result between the first matching degree and the second matching degree, the initial attribute information of the third reply information and the initial attribute information of the fourth reply information are reconfigured respectively to obtain the reconfigured first attribute information.

4. The method of claim 2, wherein: The first reply information includes the second reply information having the largest value of the first attribute information.

5. The method of claim 2, wherein: The obtaining of the first prompt word includes: Obtain target query samples and query statement expansion conditions; generating a second prompt word for expanding the query statement according to the target query example and the query statement expansion condition; Based on the second prompt word, the target query example is expanded using the large model to generate the second query statement; Determine the first query statement from the second query statement; The first prompt word is generated according to the first query sentence.

6. The method of claim 5, wherein: The obtaining of the target query sample and the query statement expansion condition includes: Clustering query statement samples in the sample library to obtain multiple first-category clusters; The target query example is selected from the first cluster.

7. The method of claim 5, wherein: The determining the first query statement from the second query statement includes: Clustering the second query statement to obtain a plurality of second clusters; Selecting a target query statement from the second cluster; The first query statement is determined according to the target query statement.

8. The method according to any one of claims 1 to 7, further comprising: Obtain a second sample; wherein the second sample is generated by using a plurality of the teacher models, the second sample includes a third prompt word, fifth reply information and sixth reply information corresponding to the third prompt word, and the value of the attribute information of the fifth reply information is greater than the value of the attribute information of the sixth reply information; Inputting the third prompt word and the fifth reply text into the first distillation model to obtain a second probability of the fifth reply information output by the first distillation model; Inputting the third prompt word and the sixth reply text into the first distillation model to obtain a third probability of the sixth reply information output by the first distillation model; The first distillation model is trained according to the second probability and the third probability to obtain a second distillation model.

9. The method of claim 8, wherein: The obtaining of the second sample comprises: Obtaining the third prompt word; inputting the third prompt words into the teacher model respectively to obtain seventh answer information output by the teacher model; Obtaining second attribute information of the seventh reply information; determining the fifth reply information and the sixth reply information from the seventh reply information according to the second attribute information; The second sample is obtained according to the third prompt word, the fifth reply information and the sixth reply information.

10. The method of claim 9, wherein: The fifth reply information includes the seventh reply information having the largest value of the second attribute information; The sixth reply information includes the seventh reply information having the smallest value of the second attribute information.

11. A method for generating reply information, comprising: Get input information; Generate prompt words according to the input information; The prompt word is input into a distillation model to obtain response information output by the distillation model; wherein the distillation model is trained using the model distillation method described in any one of claims 1 to 10.

12. A model distillation apparatus comprising: A first acquisition module is used to acquire a first sample; wherein the first sample is generated by using multiple teacher models, and the first sample includes a first prompt word and first answer information corresponding to the first prompt word; A second acquisition module is used to input the first sample into a student model to obtain a first probability of the first reply information output by the student model; wherein the positive evaluation index of the teacher model is greater than the positive evaluation index of the student model; The first training module is used to train the student model according to the first probability to obtain a first distillation model.

13. The device of claim 12, wherein: The first acquisition module is used to: Obtaining the first prompt word; Inputting the first prompt words into the teacher model respectively to obtain second answer information output by the teacher model; Acquire first attribute information of the second reply information; determining the first reply information from the second reply information according to the first attribute information; The first sample is determined according to the first prompt word and the first reply information.

14. The device of claim 13, wherein: The first acquisition module is used to: Determine a first matching degree between any third reply information in the second reply information and the first prompt word; determining a second matching degree between fourth reply information other than the third reply information in the second reply information and the first prompt word; According to the comparison result between the first matching degree and the second matching degree, the initial attribute information of the third reply information and the initial attribute information of the fourth reply information are reconfigured respectively to obtain the reconfigured first attribute information.

15. The device of claim 13, wherein: The first reply information includes the second reply information having the largest value of the first attribute information.

16. The device of claim 13, wherein: The first acquisition module is used to: Obtain target query samples and query statement expansion conditions; generating a second prompt word for expanding the query statement according to the target query example and the query statement expansion condition; Based on the second prompt word, the target query example is expanded using the large model to generate the second query statement; Determine the first query statement from the second query statement; The first prompt word is generated according to the first query sentence.

17. The device of claim 16, wherein: The first acquisition module is used to: Clustering query statement samples in the sample library to obtain multiple first-category clusters; The target query example is selected from the first cluster.

18. The device of claim 16, wherein: The first acquisition module is used to: Clustering the second query statement to obtain a plurality of second clusters; Selecting a target query statement from the second cluster; The first query statement is determined according to the target query statement.

19. The device according to any one of claims 12 to 18, further comprising: A third acquisition module is used to acquire a second sample; wherein the second sample is generated by using a plurality of the teacher models, the second sample includes a third prompt word, fifth reply information and sixth reply information corresponding to the third prompt word, and the value of the attribute information of the fifth reply information is greater than the value of the attribute information of the sixth reply information; a fourth acquisition module, configured to input the third prompt word and the fifth reply text into the first distillation model to obtain a second probability of the fifth reply information output by the first distillation model; The fourth acquisition module is used to input the third prompt word and the sixth reply text into the first distillation model to obtain a third probability of the sixth reply information output by the first distillation model; The second training module is used to train the first distillation model according to the second probability and the third probability to obtain a second distillation model.

20. The device of claim 19, wherein: The third acquisition module is used to: Obtaining the third prompt word; inputting the third prompt words into the teacher model respectively to obtain seventh answer information output by the teacher model; Obtaining second attribute information of the seventh reply information; determining the fifth reply information and the sixth reply information from the seventh reply information according to the second attribute information; The second sample is obtained according to the third prompt word, the fifth reply information and the sixth reply information.

21. The device of claim 20, wherein: The fifth reply information includes the seventh reply information having the largest value of the second attribute information; The sixth reply information includes the seventh reply information having the smallest value of the second attribute information.

22. A device for generating reply information, comprising: A first acquisition module, used to acquire input information; A generating module, used for generating prompt words according to the input information; The second acquisition module is used to input the prompt word into the distillation model to obtain the response information output by the distillation model; wherein the distillation model is trained using the model distillation device described in any one of claims 12-21.

23. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1 to 10, or to execute the method of claim 11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10, or to execute the method according to claim 11.

25. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 10, or implements the steps of the method according to claim 11.

Images