Large language model distillation method and device, electronic equipment and storage medium

Abstract

The invention discloses a large language model distillation method and device, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the field of artificial intelligence such as large models. According to the specific implementation scheme, the method comprises the steps of obtaining a large language model and a plurality of different first small language models; wherein the large language model serves as a teacher model, the first small language model serves as a student model, and the model scale of the large language model is larger than that of the first small language model; carrying out distillation training on the student models by utilizing the teacher model to obtain a plurality of first distillation models; obtaining a second distillation model corresponding to each first distillation model through mutual learning among the first distillation models; according to the second distillation model, training the second small language model to obtain a target distillation model; wherein the model scale of the second small language model is smaller than that of the large language model.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to the field of artificial intelligence such as large models, and specifically to a large language model distillation method, device, electronic device and storage medium. Background Art

[0002] Model distillation is a technique that transfers knowledge from a large and complex model (as the teacher model) to a small and simple model (as the student model), aiming to improve the model's inference efficiency and performance while maintaining performance close to that of the teacher model. Summary of the invention

[0003] The present application provides a large language model distillation method, device, electronic device and storage medium.

[0004] According to one aspect of the present application, a large language model distillation method is provided, comprising:

[0005] Acquire a large language model and a plurality of different first small language models; wherein the large language model is used as a teacher model, the first small language model is used as a student model, and the model scale of the large language model is larger than the model scale of the first small language model;

[0006] Using the teacher model, performing distillation training on the student model to obtain a plurality of first distillation models;

[0007] By mutual learning between the first distillation models, a second distillation model corresponding to each first distillation model is obtained;

[0008] According to the second distillation model, a second small language model is trained to obtain a target distillation model; wherein the model scale of the second small language model is smaller than the model scale of the large language model.

[0009] According to another aspect of the present application, a text generation method is provided, comprising:

[0010] Get input content;

[0011] Obtain prompt information according to the input content; wherein the prompt information is used to prompt the target distillation model to perform the target task, and the target distillation model is trained according to the above method;

[0012] The prompt information is input into the target distillation model for processing to obtain text content output by the target distillation model.

[0013] According to another aspect of the present application, a large language model distillation device is provided, comprising:

[0014] A first acquisition module is used to acquire a large language model and a plurality of different first small language models; wherein the large language model is used as a teacher model, the first small language model is used as a student model, and the model scale of the large language model is larger than the model scale of the first small language model;

[0015] A first training module, used to perform distillation training on the student model using the teacher model to obtain multiple first distillation models;

[0016] a mutual learning module, configured to obtain a second distillation model corresponding to each first distillation model by mutual learning between the first distillation models;

[0017] The second training module is used to train a second small language model according to the second distillation model to obtain a target distillation model; wherein the model scale of the second small language model is smaller than the model scale of the large language model.

[0018] According to another aspect of the present application, a text generation device is provided, comprising:

[0019] Get input content;

[0020] Obtain prompt information according to the input content; wherein the prompt information is used to prompt the target distillation model to perform the target task, and the target distillation model is trained by the above-mentioned device;

[0021] The prompt information is input into the target distillation model for processing to obtain text content output by the target distillation model.

[0022] According to another aspect of the present application, there is provided an electronic device, including:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the method described in the above embodiment.

[0026] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to the above embodiment.

[0027] According to another aspect of the present application, a computer program product is provided, including a computer program, wherein the computer program implements the steps of the method described in the above embodiment when executed by a processor.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0030] Figure 1 A schematic diagram of a large language model distillation method provided in one embodiment of the present application;

[0031] Figure 2 A schematic diagram of a large language model distillation method provided in another embodiment of the present application;

[0032] Figure 3 A schematic diagram of a large language model distillation method provided in another embodiment of the present application;

[0033] Figure 4 A flowchart of a text generation method provided in one embodiment of the present application;

[0034] Figure 5 A schematic diagram of the structure of a large language model distillation device provided in one embodiment of the present application;

[0035] Figure 6 A schematic diagram of the structure of a text generation device provided in one embodiment of the present application;

[0036] Figure 7 It is a block diagram of an electronic device used to implement the large language model distillation method of an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0038] In the technical solution of this application, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0039] The following describes the large language model distillation method, device, electronic device and storage medium of the embodiments of the present application with reference to the accompanying drawings.

[0040] Figure 1A schematic diagram of a large language model distillation method provided in accordance with an embodiment of the present application.

[0041] The large language model distillation method of the embodiment of the present application can be performed by the large language model distillation device of the embodiment of the present application, and the device can be configured in an electronic device.

[0042] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0043] like Figure 1 As shown, the large language model distillation method includes:

[0044] Step 101: obtain a large language model and a plurality of different first small language models.

[0045] The input of the large language model and the first small language model may be text, or an image, or multimodal information including text and an image, etc., and the output may be text.

[0046] The model scale of the large language model is greater than the model scale of each first small language model.

[0047] In this application, the large language model can be used as a teacher model, and the first small language model can be used as a student model.

[0048] It should be noted that the number of teacher models can be one or more. For example, multiple different large language models can be obtained, and each large language model can be used as a teacher model to obtain multiple different teacher models.

[0049] Step 102: Use the teacher model to perform distillation training on the student model to obtain multiple first distillation models.

[0050] In this application, since the teacher model has high performance and a rich knowledge base, the teacher model can be used to perform distillation training on multiple student models respectively, so as to transfer the knowledge of the teacher model to the student model and obtain the first distillation model corresponding to each student model. The number of student models is the same as the number of first distillation models.

[0051] That is to say, the teacher model is used to perform distillation training on each student model to obtain the first distillation model corresponding to each student model.

[0052] Step 103: obtaining a second distillation model corresponding to each first distillation model by mutual learning between the first distillation models.

[0053] In the present application, the first distillation models can learn from each other to obtain the second distillation model corresponding to each first distillation model.

[0054] Exemplarily, the first training sample can be input into the first distillation model to obtain the text content output by the first distillation model. The first distillation model is trained according to the text content output by the first distillation model to obtain the second distillation model.

[0055] In this application, each first distillation model can adjust its own weights and parameters based on the new knowledge learned from other first distillation models. This two-way feedback mechanism can ensure mutual learning and knowledge sharing between models, and improve the generalization ability and adaptability of each model.

[0056] Step 104: train the second small language model according to the second distillation model to obtain a target distillation model.

[0057] Among them, the model scale of the second smallest language model is smaller than the model scale of the large language model.

[0058] In the present application, a training data set can be constructed based on the input and output of multiple second distillation models, and the second small language model can be trained using the constructed training data set to obtain a target distillation model.

[0059] It can be seen that multiple student models can learn and complement each other during the distillation process. The final generated distillation model not only retains the advantages of the teacher model, but also has stronger adaptability and generalization ability.

[0060] The distillation method of mutual learning in the embodiment of the present application can be widely used in multiple fields. For example, the target distillation model of the embodiment of the present application can be used to build a more intelligent and accurate question-answering system, or to apply chat robots, virtual assistants and other applications to generate more coherent and creative dialogue content, or to summary generation tasks such as news, documents, and reports to generate more accurate and comprehensive automatic summaries.

[0061] In the embodiment of the present application, a teacher model is used to guide multiple different student models to obtain multiple first distillation models, and the first distillation models learn from each other to obtain the second distillation model corresponding to each first distillation model. According to the second distillation model, the second small language model is trained to obtain the final distillation model. Thus, through mutual learning, each first distillation model can absorb each other's strengths, make up for their respective deficiencies, and improve their respective performance. Based on the second distillation model obtained by mutual learning, the target distillation model is obtained, which can ensure that the target distillation model combines the advantages of all second distillation models, so that it can have higher efficiency and lower resource consumption at the same time, and has stronger adaptability and generalization ability.

[0062] Figure 2 A schematic diagram of a large language model distillation method provided in accordance with another embodiment of the present application.

[0063] like Figure 2 As shown, the large language model distillation method includes:

[0064] Step 201: obtain a large language model and a plurality of different first small language models.

[0065] In the present application, step 201 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0066] Step 202: Use the teacher model to perform distillation training on multiple student models to obtain multiple first distillation models.

[0067] In the present application, step 202 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0068] Exemplarily, a first training sample can be obtained and input into multiple student models respectively to obtain the fifth text content output by each student model, and for each student model, the first training sample, the fifth text content, etc. are input into the teacher model, and the teacher model is used to correct the sixth text content output by the student model to obtain the sixth text content, and then based on the fifth difference between the fifth text content and the sixth text content, the student model is distilled and trained to obtain the first distillation model.

[0069] The first training sample may be a question input by a user, and the first training sample may be in the form of text, image, or multimodal information including text and image, etc. In addition, the first training sample may be one or more, and this application does not limit this.

[0070] Exemplarily, the KL divergence between the fifth text content and the sixth text content may be calculated, and the student model may be trained using the KL divergence so that the output of the student model is as close as possible to the output of the teacher model.

[0071] Therefore, by using the teacher model to correct the text content output by the student model, the student model is trained for distillation based on the difference between the corrected content and the text content output by the student model, thereby transferring the knowledge of the teacher model to the student model and improving the performance of the student model.

[0072] Exemplarily, the first training sample has corresponding reference text content, and the reference text content can be understood as the standard text content of the first training sample. The fourth loss can be obtained based on the fifth difference between the fifth text content and the sixth text content, and the fifth loss can be obtained based on the sixth difference between the fifth text content and the reference text content corresponding to the first training sample. Then, the student model is distilled and trained based on the fourth loss and the fifth loss to obtain the first distillation model.

[0073] Among them, the fourth loss can be the KL divergence between the fifth text content and the sixth text content, and the fifth loss can be the cross entropy loss between the fifth text content and the reference text content.

[0074] Exemplarily, the fourth loss and the fifth loss may be weightedly summed, and the loss obtained by the weighted sum may be used to perform distillation training on the student model to obtain a first distillation model.

[0075] Therefore, the student model can also be distilled and trained based on the loss between the text content output by the student model and the reference text content corresponding to the first training sample, thereby further improving the performance of the model.

[0076] Exemplarily, there may be multiple teacher models, and multiple teacher models may be used to perform distillation training on multiple student models to obtain multiple first distillation models, wherein the number of teacher models may be less than the number of student models.

[0077] For example, if there are multiple teacher models, a student model can be assigned to each teacher model from multiple student models, and each teacher model can perform a distillation model on the assigned student model. The assignment can be random or in other ways, which is not limited.

[0078] Exemplarily, the number of student models corresponding to each teacher model can be determined according to the number of teacher models and the number of student models, and multiple student models can be sorted in the first order of the number of model parameters to obtain a student model sequence. Then, according to the number of student models corresponding to each teacher model and the second order of the number of model parameters of the multiple teacher models, the corresponding student model can be selected from the student model sequence for each teacher model in turn.

[0079] The number of student models corresponding to each teacher model can be obtained by dividing the number of multiple student models by the number of multiple teacher models. For example, if there are 2 teacher models and 4 student models, each teacher model can be assigned 2 student models. For another example, if the number of student models is 7 and the number of teacher models is 3, the teacher model with the largest number of model parameters can be assigned 3 student models, and the remaining 2 teacher models can be assigned 2 student models each.

[0080] The first order and the second order may be opposite. For example, if the first order is from most to least, then the second order is from least to most, or if the first order is from least to most, then the second order is from most to least.

[0081] For example, if there are 2 teacher models and 4 student models, each teacher model can be assigned 2 student models. The 4 student models can be sorted in order of the number of model parameters from most to least to get a student model sequence. Then, the first 2 student models in the student model sequence can be assigned to the teacher model with fewer model parameters, and the remaining 2 student models with fewer model parameters can be assigned to the teacher model with more model parameters.

[0082] Therefore, if there are multiple teacher models, multiple student models can be assigned to multiple teacher models, and the teacher model with a large number of model parameters can guide the student model with a small number of model parameters, and the teacher model with a small number of model parameters can guide the student model with a large number of model parameters. This can not only obtain first distillation models with similar model capabilities, avoiding the large difference in the capabilities of multiple first distillation models that affects the final distillation model, but also the guidance of multiple teacher models can enrich the knowledge of the final distillation model and improve the generalization ability of the model.

[0083] Step 203: input the first training samples into a plurality of first distillation models respectively to obtain first text content output by each first distillation model.

[0084] Since the student models are different, the first distillation models obtained based on the teacher model to guide the student model are also different. When the first training sample is input into different first distillation models, the first text contents output by multiple first distillation models may also be different.

[0085] It should be noted that the same training sample may be input into multiple first distillation models, or different training samples may be input into multiple first distillation models, and this application does not limit this.

[0086] Step 204: for the third distillation model in the first distillation model, use other first distillation models except the third distillation model to correct the first text content output by the third distillation model to obtain the second text content.

[0087] In the present application, for a third distillation model among multiple first distillation models, the first text content output by the third distillation model can be input into other first distillation models among multiple first distillation models except the third distillation model, so as to correct the first text content output by the third distillation model and obtain the second text content. Among them, the third distillation model can be any distillation model among multiple first distillation models. It can be understood that the second text content contains the knowledge of other first distillation models.

[0088] Exemplarily, a first prompt message can be generated based on the first training sample and the first text content output by the third distillation model. The first prompt message can be used to prompt other first distillation models to perform a correction task. The first prompt message is input into other first distillation models, and the other first distillation models correct the first text content according to the first training sample to obtain the second text content.

[0089] Therefore, the first prompt information is generated according to the first training sample and the text content output by the first distillation model, and other first distillation models perform a correction task on the first text content according to the first prompt information, thereby improving the accuracy of the second text content.

[0090] Step 205: training a third distillation model according to a first difference between the first text content and the second text content to obtain a second distillation model corresponding to the third distillation model.

[0091] In the present application, a first loss can be obtained based on a first difference between the first text content and the second text content, and the third distillation model can be trained based on the first loss to obtain a second distillation model corresponding to the third distillation model.

[0092] The KL divergence between the first text content and the second text content may be used as the first loss.

[0093] Among them, if there are multiple other first distillation models, the third distillation model and the other first distillation models can learn from each other at the same time, or they can learn from each other one by one, and there is no limitation to this.

[0094] For example, there are three first distillation models, namely m1, m2 and m2. Taking the first distillation model m1 as an example, the first distillation model can be trained according to the difference between the first text content and the second text content obtained by correcting the first text content with the first distillation model m2, and then the first distillation model can be further trained according to the difference between the first text content and the second text content obtained by correcting the first text content with the first distillation model m2 to obtain the second distillation model corresponding to the first distillation model m1.

[0095] Exemplarily, the first training sample has corresponding reference text content, and a second loss can be obtained according to a second difference between the first text content and the reference text. The first distillation model is trained according to the first loss and the second loss to obtain a second distillation model.

[0096] Exemplarily, the cross entropy loss between the first text content and the reference text content can be used as the second loss. Exemplarily, the first distillation model can be trained according to the weighted sum of the first loss and the second loss to obtain the second distillation model.

[0097] Therefore, the first distillation model can also be trained in combination with the loss between the first text content and the reference text content corresponding to the first training sample, thereby further improving the performance of the model.

[0098] Step 206: Train the second small language model according to the multiple second distillation models to obtain a target distillation model.

[0099] In the present application, step 206 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0100] In the embodiment of the present application, for any third distillation model among the multiple first distillation models, the text content output by the third distillation model is corrected using other first distillation models, so that the corrected text content contains the knowledge of other first distillation models, and the third distillation model is trained using the difference between the third distillation model and the text content corrected by the other first distillation models, so that the first distillation model can learn the knowledge of other first distillation models. Thus, each first distillation model can learn the knowledge of other first distillation models, so that multiple first distillation models can learn from each other's strengths, thereby improving model performance.

[0101] Figure 3 A schematic diagram of a large language model distillation method provided in accordance with another embodiment of the present application.

[0102] like Figure 3 As shown, the large language model distillation method includes:

[0103] Step 301: Obtain a large language model and a plurality of different first small language models.

[0104] In the present application, step 301 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0105] Step 302: Use the teacher model to perform distillation training on multiple student models to obtain multiple first distillation models.

[0106] In the present application, step 302 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0107] Step 303: Obtain a second distillation model corresponding to each first distillation model by mutual learning between the first distillation models.

[0108] In the present application, step 303 can be implemented in any of the embodiments of the present application, so it will not be described in detail here.

[0109] Exemplarily, the first can be respectively input into multiple first distillation models to obtain the first text content output by each first distillation model, and for the fourth distillation model in the first distillation model, the fourth distillation model can be trained according to the third difference between the first text content of the fourth distillation model and the first text content output by other first distillation models in the multiple first distillation models except the fourth distillation model, to obtain the second distillation model corresponding to the fourth distillation model. The fourth distillation model can be any distillation model in the multiple first distillation models.

[0110] Exemplarily, a third loss is obtained according to a third difference between the first text content of the fourth distillation model and the first text content output by other first distillation models, and the first distillation model is trained using the third loss.

[0111] For example, there are three first distillation models in total. For any one of the first distillation models, the first distillation model can be trained according to the sum of the third losses corresponding to the other two first distillation models to obtain a second distillation model corresponding to the first distillation model.

[0112] Therefore, based on the difference between the text content output by the fourth distillation model and the first text output by other first distillation models under the same input, the fourth distillation model is trained so that the fourth distillation model can learn the knowledge of other first distillation models, thereby improving its own performance.

[0113] Exemplarily, a second loss may be obtained based on a second difference between the first text content and the reference text content corresponding to the first training sample, and the first distillation model may be trained based on the second loss and the third loss.

[0114] Therefore, the first distillation model can also be trained in combination with the loss between the first text content and the reference text content corresponding to the first training sample, thereby further improving the performance of the model.

[0115] Step 304: input the second training samples into a plurality of second distillation models respectively to obtain third text contents output by the plurality of second distillation models.

[0116] In the present application, the first training sample can be input into each second distillation model, and the third text content output by each second distillation model can be obtained.

[0117] It should be noted that the second training sample may be the same as or different from the first training sample, and this is not limited. In addition, the second training sample may be one or more.

[0118] Step 305: Train the second small language model according to the second training samples and the third text content corresponding to the plurality of second distillation models to obtain a target distillation model.

[0119] In the present application, a training data set can be formed according to the second training samples and the third text content corresponding to each second distillation model, and the second small language model can be trained using the training data set to obtain a target distillation model. For example, a second training sample can be used as a question, and the third text content corresponding to the second training sample can be used as an answer to form a question-answer pair. Then, the training data set can include the question-answer pairs corresponding to each second distillation model.

[0120] As a possible implementation method, the third text content can be used as the reference text content corresponding to the second training sample, so that the second small language model can be supervised trained using the second training sample and the corresponding third text content to obtain the target distillation model.

[0121] Exemplarily, the second training sample may be input into the second small language model to obtain the fourth text content output by the second small language model, and the second small language model may be trained according to the fourth difference between the fourth text content and the third text content to obtain the target distillation model. For example, the cross entropy between the fourth text content and the third text content may be calculated, and the second small language model may be trained according to the cross entropy until the training end condition is met to obtain the target distillation model.

[0122] Therefore, the second small language model is supervisedly trained using the second training samples and the third text content corresponding to each second distillation model to obtain a target distillation model, so that the target distillation model integrates the advantages of each second distillation model and improves the generalization ability of the target distillation model.

[0123] Since the requirements for the final distillation model may be different, as another possible implementation, the weights of the second distillation models may be used to control the generation of target distillation models with different requirements.

[0124] Exemplarily, the weights of multiple second distillation models can be obtained. For the fifth distillation model among the multiple second distillation models, the target number of second training samples and third text content can be extracted from the second training samples and third text content corresponding to the fifth distillation model, and then the second small language model is trained according to the target number of second training samples and the corresponding third text content corresponding to the multiple second distillation models, to obtain the target distillation model. The fifth distillation model can be any distillation model among the multiple second distillation models. The target number is equal to the product of the weight of the second distillation model and the number of second training samples corresponding to the second distillation model.

[0125] For example, there are three second distillation models model1, model1, and model3, with weights of 0.7, 0.4, and 0.4 respectively. The three second distillation models have 100 question-answer pairs respectively. Then, 70 question-answer pairs can be extracted from the 100 question-answer pairs corresponding to model1, 40 question-answer pairs can be extracted from the 100 question-answer pairs corresponding to model2, and 40 question-answer pairs can be extracted from the 100 question-answer pairs corresponding to model3. Thus, the second small language model can be trained using the training data set formed by the 70 question-answer pairs corresponding to model1, the 40 question-answer pairs corresponding to model2, and the 40 question-answer pairs corresponding to model3 to obtain the target distillation model. Although the target distillation model combines the advantages of these three models, the advantage of model1 is stronger.

[0126] It should be noted that the weight of each second distillation model can be set according to actual needs, and this application does not limit this.

[0127] Therefore, the number of training samples from each second distillation model used to train the second small language model can be controlled by the weight of each second distillation model, thereby meeting different training requirements.

[0128] In an embodiment of the present application, the second small language model is trained based on the second training samples corresponding to multiple second distillation models and the output third text content, so that the target model can integrate the advantages of all second distillation models and have stronger generalization ability and higher performance.

[0129] In order to implement the above embodiment, the embodiment of the present application also proposes a text generation method. Figure 4 A flowchart of a text generation method provided in an embodiment of the present application.

[0130] like Figure 4 As shown, the text generation method includes:

[0131] Step 401, obtaining input content.

[0132] In the present application, the input content input by the user can be obtained, wherein the input content can be text, or an image, etc., which is not limited.

[0133] Step 402: Obtain prompt information according to the input content.

[0134] The prompt information can be used to instruct the target distillation model to perform the target task. For example, the target task can be a summary generation task, a picture-to-text task, or other types of text generation tasks.

[0135] Among them, the target distillation model can be trained using the above-mentioned large language model distillation method.

[0136] Exemplarily, prompt information may be generated according to input content, model output requirements, etc. The model output requirements may include word count requirements, format requirements, etc. for the output content.

[0137] For example, the prompt message is "

***

***

[0138] Step 403: input the prompt information into the target distillation model for processing to obtain the text content output by the target distillation model.

[0139] In this application, the target distillation model performs the text generation task according to the prompt information, generates text content and outputs it.

[0140] In an embodiment of the present application, the target distillation model obtained by the above-mentioned training method can be used to perform text generation tasks. Since the target distillation model combines the advantages of all the second distillation models, the text content output by the target distillation model is more accurate and richer.

[0141] In order to implement the above embodiment, the embodiment of the present application also proposes a large language model distillation device. Figure 5 A schematic diagram of the structure of a large language model distillation device provided in one embodiment of the present application.

[0142] like Figure 5 As shown, the large language model distillation device 500 includes:

[0143] A first acquisition module 510 is used to acquire a large language model and a plurality of different first small language models; wherein the large language model is used as a teacher model, the first small language models are used as student models, and the model scale of the large language model is larger than the model scale of the first small language model;

[0144] A first training module 520 is used to perform distillation training on the student model using the teacher model to obtain multiple first distillation models;

[0145] A mutual learning module 530, configured to obtain a second distillation model corresponding to each first distillation model through mutual learning between the first distillation models;

[0146] The second training module 540 is used to train a second small language model according to the second distillation model to obtain a target distillation model; wherein the model scale of the second small language model is smaller than the model scale of the large language model.

[0147] Optionally, the mutual learning module 530 is used to:

[0148] Inputting the first training samples into the first distillation models respectively to obtain first text content output by each first distillation model;

[0149] For a third distillation model among the plurality of first distillation models, using other first distillation models except the third distillation model, correct the first text content output by the third distillation model to obtain second text content;

[0150] The third distillation model is trained according to a first difference between the first text content and the second text content to obtain a second distillation model corresponding to the third distillation model.

[0151] Optionally, the mutual learning module 530 is used to:

[0152] Obtaining a first loss according to a first difference between the first text content and the second text content;

[0153] Obtaining a second loss according to a second difference between the first text content and the reference text content corresponding to the first training sample;

[0154] The third distillation model is trained according to the first loss and the second loss to obtain the second distillation model.

[0155] Optionally, the mutual learning module 530 is used to:

[0156] Generate first prompt information according to the first training sample and the first text content output by the third distillation model; wherein the first prompt information is used to prompt the other first distillation model to perform a correction task;

[0157] The first prompt information is input into the other first distillation model to modify the first text content output by the third distillation model to obtain the second text content.

[0158] Optionally, the mutual learning module 530 is used to:

[0159] Inputting the first training samples into the first distillation models respectively to obtain first text content output by each first distillation model;

[0160] For the fourth distillation model in the first distillation model, the fourth distillation model is trained according to the third difference between the first text content of the fourth distillation model and the first text content of other first distillation models except the fourth distillation model to obtain the second distillation model corresponding to the fourth distillation model.

[0161] Optionally, the mutual learning module 530 is used to:

[0162] Obtaining a second loss according to a second difference between the first text content and a reference text content corresponding to the first training sample;

[0163] Obtaining a third loss according to the third difference;

[0164] The fourth distillation model is trained according to the second loss and the third loss to obtain a second distillation model.

[0165] Optionally, the second training module 540 is used to:

[0166] Inputting the second training samples into the second distillation models respectively to obtain third text contents output by the plurality of second distillation models;

[0167] The second small language model is trained according to the second training sample and the third text content corresponding to the second distillation model to obtain the target distillation model.

[0168] Optionally, the second training module 540 is used to:

[0169] Obtaining a weight of the second distillation model;

[0170] For a fifth distillation model in the second distillation model, extracting a target number of the second training samples and the corresponding third text content from the second training samples and the third text content corresponding to the fifth distillation model according to the weight;

[0171] The second small language model is trained according to the target number of the second training samples and the corresponding third text content to obtain the target distillation model.

[0172] Optionally, the second training module 540 is used to:

[0173] Inputting the second training sample into the second small language model to obtain fourth text content output by the second small language model;

[0174] The second small language model is trained according to a fourth difference between the fourth text content and the third text content to obtain the target distillation model.

[0175] Optionally, the first training module 540 is used to:

[0176] Inputting the first training samples into the student models respectively to obtain fifth text content output by each of the student models;

[0177] Using the teacher model to correct the fifth text content output by the student model to obtain a sixth text content;

[0178] The student model is trained according to a fifth difference between the fifth text content and the sixth text content to obtain the first distillation model.

[0179] Optionally, the first training module 520 is used to:

[0180] Inputting the first training samples into the student models respectively to obtain fifth text contents output by the multiple student models;

[0181] Obtaining a fifth loss according to a sixth difference between the fifth text content and a reference text content corresponding to the first training sample;

[0182] The student model is trained according to the fourth loss and the fifth loss to obtain the first distillation model.

[0183] Optionally, there are multiple teacher models, and the first training module 520 is used to:

[0184] Determine a student model corresponding to each of the teacher models from the multiple student models;

[0185] According to each of the teacher models, distillation training is performed on the student model corresponding to each of the teacher models to obtain the first distillation model.

[0186] Optionally, the first training module 520 is used to:

[0187] Determine the number of student models corresponding to each teacher model according to the number of the teacher models and the number of the student models;

[0188] Sorting the student models according to the first order of the number of model parameters to obtain a student model sequence;

[0189] According to the number of student models corresponding to each teacher model, and in accordance with a second order of the number of model parameters of the teacher model, a corresponding student model is selected in turn from the student model sequence for each teacher model; wherein the second order is opposite to the first order.

[0190] It should be noted that the explanation of the aforementioned large language model distillation method embodiment is also applicable to the large language model distillation device of this embodiment, so it will not be repeated here.

[0191] In the embodiment of the present application, a teacher model is used to guide multiple different student models to obtain multiple first distillation models, and the first distillation models learn from each other to obtain the second distillation model corresponding to each first distillation model. According to the second distillation model, the second small language model is trained to obtain the final distillation model. Thus, through mutual learning, each first distillation model can absorb each other's strengths, make up for their respective deficiencies, and improve their respective performance. Based on the second distillation model obtained by mutual learning, the target distillation model is obtained, which can ensure that the target distillation model combines the advantages of all second distillation models, so that it can have higher efficiency and lower resource consumption at the same time, and has stronger adaptability and generalization ability.

[0192] In order to implement the above embodiment, the embodiment of the present application also proposes a text generation device. Figure 6 A schematic diagram of the structure of a text generation device provided in one embodiment of the present application.

[0193] like Figure 6 As shown, the text generation device 600 includes:

[0194] A first acquisition module 610, used to acquire input content;

[0195] A second acquisition module 620 is used to acquire prompt information according to the input content; wherein the prompt information is used to prompt the target distillation model to perform the target task, and the target distillation model is trained according to the text generation device described above;

[0196] The third acquisition module 630 is used to input the prompt information into the target distillation model for processing to obtain the text content output by the target distillation model.

[0197] It should be noted that the explanation of the aforementioned text generation method embodiment is also applicable to the text generation device of this embodiment, so it will not be repeated here.

[0198] In an embodiment of the present application, the target distillation model obtained by the above-mentioned training method can be used to perform text generation tasks. Since the target distillation model combines the advantages of all the second distillation models, the text content output by the target distillation model is more accurate and richer.

[0199] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0200] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0201] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 702 or a computer program loaded from a storage unit 708 to a RAM (Random Access Memory) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An I / O (Input / Output) interface 705 is also connected to the bus 704.

[0202] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0203] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the large language model distillation method. For example, in some embodiments, the large language model distillation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the large language model distillation method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the large language model distillation method in any other appropriate manner (for example, by means of firmware).

[0204] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0205] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0206] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0207] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0208] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0209] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0210] It should be noted that the above electronic device can also implement the text generation method described in the above embodiment.

[0211] According to an embodiment of the present application, the present application also provides a computer program product. When the instruction processor in the computer program product executes, the large language model distillation method or text generation method proposed in the above embodiments of the present application is executed.

[0212] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0213] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A large language model distillation method, comprising: Acquire a large language model and a plurality of different first small language models; wherein the large language model is used as a teacher model, the first small language model is used as a student model, and the model scale of the large language model is larger than the model scale of the first small language model; Using the teacher model, performing distillation training on the student model to obtain a plurality of first distillation models; By mutual learning between the first distillation models, a second distillation model corresponding to each of the first distillation models is obtained; According to the second distillation model, a second small language model is trained to obtain a target distillation model; wherein the model scale of the second small language model is smaller than the model scale of the large language model.

2. The method of claim 1, wherein: The step of obtaining a second distillation model corresponding to each of the first distillation models by mutual learning between the first distillation models includes: Inputting the first training samples into the first distillation models respectively to obtain first text content output by each of the first distillation models; For a third distillation model in the first distillation model, using other first distillation models except the third distillation model, correct the first text content output by the third distillation model to obtain second text content; The third distillation model is trained according to a first difference between the first text content and the second text content to obtain the second distillation model corresponding to the third distillation model.

3. The method of claim 2, wherein: The step of training the third distillation model according to the first difference between the first text content and the second text content to obtain the second distillation model corresponding to the third distillation model includes: Obtaining a first loss according to the first difference; Obtaining a second loss according to a second difference between the first text content and the reference text content corresponding to the first training sample; The third distillation model is trained according to the first loss and the second loss to obtain the second distillation model.

4. The method of claim 2, wherein: The method of modifying the first text content output by the third distillation model using other first distillation models except the third distillation model in the first distillation model to obtain the second text content includes: Generate first prompt information according to the first training sample and the first text content output by the third distillation model; wherein the first prompt information is used to prompt the other first distillation model to perform a correction task; The first prompt information is input into the other first distillation model to modify the first text content output by the third distillation model to obtain the second text content.

5. The method of claim 1, wherein: The step of obtaining a second distillation model corresponding to each of the first distillation models by mutual learning between the first distillation models includes: Inputting the first training samples into the first distillation models respectively to obtain first text content output by each first distillation model; For the fourth distillation model in the first distillation model, the fourth distillation model is trained according to the third difference between the first text content of the fourth distillation model and the first text content of other first distillation models except the fourth distillation model to obtain the second distillation model corresponding to the fourth distillation model.

6. The method of claim 5, wherein: The method of training a fourth distillation model in the first distillation models according to a third difference between the first text content of the fourth distillation model and the first text content of other first distillation models except the fourth distillation model to obtain the second distillation model corresponding to the fourth distillation model includes: Obtaining a second loss according to a second difference between the first text content and a reference text content corresponding to the first training sample; Obtaining a third loss according to the third difference; The fourth distillation model is trained according to the second loss and the third loss to obtain the second distillation model.

7. The method of claim 1, wherein: The step of training the second small language model according to the second distillation model to obtain a target distillation model includes: Inputting the second training samples into the second distillation model respectively to obtain third text content output by the second distillation model; The second small language model is trained according to the second training sample and the third text content corresponding to the second distillation model to obtain the target distillation model.

8. The method of claim 7, wherein: The step of training the second small language model according to the second training sample and the third text content corresponding to the second distillation model to obtain the target distillation model includes: Obtaining a weight of the second distillation model; For a fifth distillation model in the second distillation model, extracting a target number of the second training samples and the corresponding third text content from the second training samples and the third text content corresponding to the fifth distillation model according to the weight; The second small language model is trained according to the target number of the second training samples and the corresponding third text content to obtain the target distillation model.

9. The method of claim 7, wherein: The step of training the second small language model according to the second training sample and the third text content corresponding to the second distillation model to obtain the target distillation model includes: Inputting the second training sample into the second small language model to obtain fourth text content output by the second small language model; The second small language model is trained according to a fourth difference between the fourth text content and the third text content to obtain the target distillation model.

10. The method according to any one of claims 1 to 9, wherein: The method of using the teacher model to perform distillation training on the student model to obtain a plurality of first distillation models includes: Inputting the first training samples into the student models respectively to obtain fifth text content output by each of the student models; Using the teacher model to correct the fifth text content output by the student model to obtain a sixth text content; The student model is trained according to a fifth difference between the fifth text content and the sixth text content to obtain the first distillation model.

11. The method of claim 10, wherein: The step of training the student model according to the fifth difference between the fifth text content and the sixth text content to obtain the first distillation model comprises: According to the fifth difference, obtaining a fourth loss; Obtaining a fifth loss according to a sixth difference between the fifth text content and a reference text content corresponding to the first training sample; The student model is trained according to the fourth loss and the fifth loss to obtain the first distillation model.

12. The method according to any one of claims 1 to 9, wherein: The teacher model is multiple, and the student model is distilled and trained by using the teacher model to obtain multiple first distillation models, including: Determine a student model corresponding to each of the teacher models from the multiple student models; According to each of the teacher models, distillation training is performed on the student model corresponding to each of the teacher models to obtain the first distillation model.

13. The method of claim 12, wherein: The step of determining a student model corresponding to each teacher model from the plurality of student models comprises: Determine the number of student models corresponding to each teacher model according to the number of the teacher models and the number of the student models; Sorting the student models according to the first order of the number of model parameters to obtain a student model sequence; According to the number of student models corresponding to each of the teacher models, and in accordance with a second order of the number of model parameters of the teacher models, a corresponding student model is selected in turn from the student model sequence for each of the teacher models; wherein the second order is opposite to the first order.

14. A text generation method, comprising: Get input content; Obtain prompt information according to the input content; wherein the prompt information is used to prompt the target distillation model to perform the target task, and the target distillation model is trained by the method according to any one of claims 1-13; The prompt information is input into the target distillation model for processing to obtain text content output by the target distillation model.

15. A large language model distillation device, comprising: A first acquisition module is used to acquire a large language model and a plurality of different first small language models; wherein the large language model is used as a teacher model, the first small language models are used as student models, and the model scale of the large language model is larger than the model scale of the first small language model; A first training module, used to perform distillation training on the student model using the teacher model to obtain multiple first distillation models; a mutual learning module, configured to obtain a second distillation model corresponding to each of the first distillation models through mutual learning between the first distillation models; The second training module is used to train a second small language model according to the second distillation model to obtain a target distillation model; wherein the model scale of the second small language model is smaller than the model scale of the large language model.

16. The device of claim 15, wherein: The mutual learning module is used to: Inputting the first training samples into the first distillation models respectively to obtain first text content output by each of the first distillation models; For a third distillation model in the first distillation model, using other first distillation models except the third distillation model, correct the first text content output by the third distillation model to obtain second text content; The third distillation model is trained according to a first difference between the first text content and the second text content to obtain the second distillation model corresponding to the third distillation model.

17. The device of claim 16, wherein: The mutual learning module is used to: Obtaining a first loss according to the first difference; Obtaining a second loss according to a second difference between the first text content and the reference text content corresponding to the first training sample; The third distillation model is trained according to the first loss and the second loss to obtain the second distillation model.

18. The device of claim 16, wherein: The mutual learning module is used to: Generate first prompt information according to the first training sample and the first text content output by the third distillation model; wherein the first prompt information is used to prompt the other first distillation model to perform a correction task; The first prompt information is input into the other first distillation model to modify the first text content output by the third distillation model to obtain the second text content.

19. The device of claim 15, wherein: The mutual learning module is used to: Inputting the first training samples into the first distillation models respectively to obtain first text content output by each first distillation model; For the fourth distillation model in the first distillation model, the fourth distillation model is trained according to the third difference between the first text content of the fourth distillation model and the first text content of other first distillation models except the fourth distillation model to obtain the second distillation model corresponding to the fourth distillation model.

20. The apparatus of claim 19, wherein: The mutual learning module is used to: Obtaining a second loss according to a second difference between the first text content and a reference text content corresponding to the first training sample; Obtaining a third loss according to the third difference; The fourth distillation model is trained according to the second loss and the third loss to obtain the second distillation model.

21. The apparatus of claim 15, wherein: The second training module is used to: Inputting the second training samples into the second distillation model respectively to obtain third text content output by the second distillation model; The second small language model is trained according to the second training sample and the third text content corresponding to the second distillation model to obtain the target distillation model.

22. The device of claim 21, wherein: The second training module is used to: Obtaining a weight of the second distillation model; For a fifth distillation model in the second distillation model, extracting a target number of the second training samples and the corresponding third text content from the second training samples and the third text content corresponding to the fifth distillation model according to the weight; The second small language model is trained according to the target number of the second training samples and the corresponding third text content to obtain the target distillation model.

23. The apparatus of claim 21, wherein: The second training module is used to: Inputting the second training sample into the second small language model to obtain fourth text content output by the second small language model; The second small language model is trained according to a fourth difference between the fourth text content and the third text content to obtain the target distillation model.

24. The device according to any one of claims 15 to 23, wherein: The first training module is used to: Inputting the first training samples into the student models respectively to obtain fifth text content output by each of the student models; Using the teacher model to correct the fifth text content output by the student model to obtain a sixth text content; The student model is trained according to a fifth difference between the fifth text content and the sixth text content to obtain the first distillation model.

25. The device of claim 24, wherein: The first training module is used to: According to the fifth difference, obtaining a fourth loss; Obtaining a fifth loss according to a sixth difference between the fifth text content and a reference text content corresponding to the first training sample; The student model is trained according to the fourth loss and the fifth loss to obtain the first distillation model.

26. The device of any one of claims 15 to 23, wherein: There are multiple teacher models, and the first training module is used to: Determine a student model corresponding to each of the teacher models from the multiple student models; According to each of the teacher models, distillation training is performed on the student model corresponding to each of the teacher models to obtain the first distillation model.

27. The device of claim 26, wherein: The first training module is used to: Determine the number of student models corresponding to each teacher model according to the number of the teacher models and the number of the student models; Sorting the student models according to the first order of the number of model parameters to obtain a student model sequence; According to the number of student models corresponding to each of the teacher models, and in accordance with a second order of the number of model parameters of the teacher models, a corresponding student model is selected in turn from the student model sequence for each of the teacher models; wherein the second order is opposite to the first order.

28. A text generation device, comprising: Get input content; Acquire prompt information according to the input content; wherein the prompt information is used to prompt the target distillation model to perform the target task, and the target distillation model is trained according to the device described in any one of claims 15-27; The prompt information is input into the target distillation model for processing to obtain text content output by the target distillation model.

29. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1 to 13, or to execute the method of claim 14.

30. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 13, or to execute the method according to claim 14.

31. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 13, or implements the steps of the method according to claim 14.

Images