Distillation method and device of large language model, electronic equipment and readable storage medium

Abstract

The invention provides a large language model distillation method and device, electronic equipment and a readable storage medium, and the distillation method comprises the steps: obtaining a plurality of trained teacher large language models, and obtaining a first data set containing a plurality of training data; obtaining a subset of the first data set as a second data set, and processing each training data in the second data set based on a plurality of large language teacher models to obtain a plurality of text sequences corresponding to each training data in the second data set; and obtaining a big language student model, and training the big language student model according to the text sequence of the training data in the second data set. Therefore, the large language model can be distilled.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and in particular to a distillation method, device, electronic device and readable storage medium for a large language model. Background Art

[0002] Model distillation is a model compression technique that aims to transfer the knowledge of a complex and large model (called the teacher model) to a smaller and simpler model (called the student model), so that the student model can reduce the consumption of computing resources and storage space while retaining high performance.

[0003] Model distillation technology is of great significance for large language models (LLMs). Large language models usually contain billions or even hundreds of billions of parameters and require a lot of computing resources and storage space during the inference stage.

[0004] Therefore, how to distill large language models becomes a problem that needs to be solved urgently. Summary of the invention

[0005] The object of the present invention is to provide a large language model distillation method, device, electronic device and readable storage medium.

[0006] In order to achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides a large language model distillation method, comprising the following steps: obtaining several trained teacher large language models, obtaining a first data set containing multiple training data; obtaining a subset of the first data set as a second data set, processing each training data in the second data set based on several teacher large language models, and obtaining a text sequence output by each teacher large language model, thereby obtaining several text sequences corresponding to each training data; obtaining a student large language model, and performing the following processing on each training data in the second data set: obtaining several token vectors corresponding to the several text sequences corresponding to the training data, arranging the several token vectors into a queue, obtaining the token vectors to be trained in order from the head to the end of the queue, and training the student large language model based on the token vectors to be trained.

[0007] As a further improvement of an embodiment of the present invention, the processing of each training data in the second data set based on several teacher large language models specifically includes: processing each training data in the second data set based on several teacher large language models, and in each processing, when the output text sequence ends or the length of the output text sequence is greater than a preset threshold, stopping the processing; when a cyclically repeated character string appears at the end of the output text sequence, trimming the end of the text sequence and retaining only a cyclically repeated character string, wherein the character string is composed of several tokens.

[0008] As a further improvement of an embodiment of the present invention, the obtaining of a number of token vectors corresponding one-to-one to a number of text sequences corresponding to the training data specifically includes: obtaining a number of token vectors corresponding one-to-one to a number of text sequences corresponding to the training data, and when the number of tokens Len1 in any token vector is greater than a preset length value Len2, discarding the subsequent Len1-Len2 tokens in the token vector.

[0009] As a further improvement of an embodiment of the present invention, the training of the student large language model based on the token vector to be trained specifically includes: the number of tokens in the token vector to be trained is Len3, i=1, and the following operations are continuously performed until i=Len3, and the operations include: obtaining the first i tokens in the token vector, inputting the first i tokens into the student large language model for reasoning, calculating the probability distribution of the inferred vocabulary and the one-hot encoded vector of the i+1th token and obtaining the cross entropy, using the cross entropy as the loss function, performing loss function calculation, gradient calculation and back propagation, and then, the value of i is increased by 1.

[0010] As a further improvement of an embodiment of the present invention, the loss function is a cross loss function.

[0011] As a further improvement of an implementation manner of the present invention, Len2=16.

[0012] An embodiment of the present invention also provides a distillation device for a large language model, comprising the following modules: an information acquisition module, used to obtain a number of trained teacher large language models, and obtain a first data set containing multiple training data; a teacher module, used to obtain a subset of the first data set as a second data set, process each training data in the second data set based on the number of teacher large language models, and obtain a text sequence output by each teacher large language model, thereby obtaining a number of text sequences corresponding to each training data; a student module, used to obtain a student large language model, and perform the following processing on each training data in the second data set: obtain a number of token vectors corresponding to the several text sequences corresponding to the training data, arrange the several token vectors into a queue, obtain the token vectors to be trained in order from the head of the queue to the tail of the queue, and train the student large language model based on the token vectors to be trained.

[0013] As a further improvement of an embodiment of the present invention, the teacher module is also used to: process each training data in the second data set based on several teacher large language models, and in each processing, when the output text sequence ends or the length of the output text sequence is greater than a preset threshold, stop the processing; when a cyclically repeated string appears at the end of the output text sequence, trim the end of the text sequence and only retain one cyclically repeated string, wherein the string is composed of several tokens.

[0014] An embodiment of the present invention further provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the above-mentioned distillation method.

[0015] An embodiment of the present invention also provides a readable storage medium for storing a computer program, wherein the computer program implements the above-mentioned distillation method when executed by a processor.

[0016] Compared with the prior art, the technical effect of the present invention is that: the embodiment of the present invention provides a large language model distillation method, device, electronic device and readable storage medium, the distillation method includes: obtaining a number of trained teacher large language models, obtaining a first data set containing multiple training data; obtaining a subset of the first data set as a second data set, processing each training data in the second data set based on a number of large language teacher models, thereby obtaining a number of text sequences corresponding to each training data in the second data set; obtaining a large language student model, and training the large language student model on the text sequence of the training data in the second data set. Thus, the large language model can be distilled. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 14 is a flow chart of a distillation method for a large language model in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The present invention will be described in detail below in conjunction with the various embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0019] As used herein, terms indicating spatial relative positions such as "upper," "above," "lower," and "below" are used for the purpose of convenience to describe the relationship of one unit or feature relative to another unit or feature as shown in the accompanying drawings. Terms of spatial relative position may be intended to include different orientations of the device in use or operation in addition to the orientation shown in the drawings. For example, if the device in the figure is turned over, the units described as being "below" or "beneath" other units or features will be located "above" the other units or features. Thus, the exemplary term "below" can encompass both the above and below orientations. The device may be oriented in other ways (rotated 90 degrees or in other orientations), and the spatially relative descriptors used herein interpreted accordingly.

[0020] Furthermore, it should be understood that although the terms first, second, etc. may be used herein to describe various elements or structures, these described objects should not be limited by these terms. These terms are only used to distinguish these described objects from each other. For example, a first rack may be referred to as a second rack, and similarly, a second rack may also be referred to as a first rack, without departing from the scope of protection of this application.

[0021] Embodiment 1 of the present invention provides a distillation method for a large language model, such as Figure 1 As shown, the following steps are included: Step 101: obtaining a plurality of trained teacher large language models, and obtaining a first data set including a plurality of training data; Here, Large Language Model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question and answer, dialogue, etc. The first data set can be constructed by mixing financial field data sets and general data sets, so that the final student large language model has both general capabilities and capabilities for specific field tasks. Financial field data sets can include: web pages related to the financial field, well-known news in the financial field, company financial reports, publications of financial related companies, etc. General data sets can include: The Pile data set, C4 data set, Wikipedia data set, etc. Optionally, the first data set contains 100B Tokens corpus, of which 60B Tokens are in the financial corpus and 40B Tokens are in other categories.

[0022] Step 102: Obtain a subset of the first data set as a second data set, process each training data in the second data set based on several teacher large language models, and obtain the text sequence output by each teacher large language model, so as to obtain several text sequences corresponding to each training data.

[0023] The processing here can be understood as a kind of continuation and expansion. In simple terms, the model continues to write according to the sentences in the data. The understanding here is correct. Each teacher model will generate a sentence.

[0024] Step 103: Obtain the student language model, and perform the following processing on each training data in the second data set: obtain a number of token vectors corresponding to a number of text sequences corresponding to the training data, arrange the token vectors into a queue, obtain the token vectors to be trained in order from the head to the tail, and train the student language model based on the token vectors to be trained. Here, token can be understood as the smallest text unit, the most basic data unit or the smallest semantic unit, etc., which can be a word, number or punctuation, etc. In practice, a Tokenizer can be used to convert a text sequence into a corresponding token vector. The queue can be generated according to preset rules, such as the number of tokens in a token vector, etc.

[0025] In the experiment corresponding to the distillation method, multiple GPUs were prepared: , ,..., First, use , ,..., To train several teacher language models, multiple trained teacher language models can be obtained from step 101, and then step 102 is executed. Then, use , ,..., To execute step 103, the data preparation and training phases can be performed without interfering with each other, and can be performed in parallel, ensuring that all graphics cards are working and the graphics card vacancy rate is low, wherein N is a natural number, N≥2.

[0026] This distillation method can use multiple teachers to generate training data, enhance data diversity, and avoid overfitting the student model to one teacher. It can avoid the use of unaligned soft labels, and instead generate multiple hard labels from multiple teacher large language models, and then make the student large language model learn and fit multiple hard labels, thereby avoiding the adverse effects of vocabulary misalignment on the distillation of multiple teacher large language models.

[0027] In this embodiment, the processing of each training data in the second data set based on several teacher large language models specifically includes: processing each training data in the second data set based on several teacher large language models, and in each processing, when the output text sequence ends or the length of the output text sequence is greater than a preset threshold, stopping the processing; when a cyclically repeated character string appears at the end of the output text sequence, trimming the end of the text sequence and retaining only a cyclically repeated character string, wherein the character string is composed of several tokens.

[0028] Here, in practice, when the text sequence begins with "<eos_token> ", it means the end of the text sequence. The length of the text sequence can be understood as the number of tokens in the text sequence.

[0029] In this embodiment, obtaining a number of token vectors corresponding one-to-one to a number of text sequences corresponding to the training data specifically includes: obtaining a number of token vectors corresponding one-to-one to a number of text sequences corresponding to the training data, and when the number of tokens Len1 in any token vector is greater than a preset length value Len2, discarding the latter Len1-Len2 tokens in the token vector.

[0030] Here, the number of tokens in each token vector can be made to be at most Len2. Optionally, Len2=16.

[0031] In this embodiment, the training of the student large language model based on the token vector to be trained specifically includes: the number of tokens in the token vector to be trained is Len3, i=1, and the following operations are continuously performed until i=Len3, and the operations include: obtaining the first i tokens in the token vector, inputting the first i tokens into the student large language model for reasoning, calculating the probability distribution of the vocabulary obtained by reasoning and the one-hot encoded vector of the i+1th token and obtaining the cross entropy, using the cross entropy as the loss function, performing loss function calculation, gradient calculation and back propagation, and then, the value of i is increased by 1.

[0032] In this embodiment, the loss function is a cross loss function.

[0033] In this embodiment, Len2=16.

[0034] Embodiment 2 of the present invention provides a large language model distillation device, comprising the following modules: an information acquisition module, for acquiring a number of trained teacher large language models, and acquiring a first data set including a plurality of training data; The teacher module is used to obtain a subset of the first data set as a second data set, process each training data in the second data set based on several teacher large language models, and obtain a text sequence output by each teacher large language model, so as to obtain several text sequences corresponding to each training data; the student module is used to obtain a student large language model, and perform the following processing on each training data in the second data set: obtain several token vectors corresponding to several text sequences corresponding to the training data, arrange the several token vectors into a queue, obtain the token vectors to be trained in order from the head to the end of the queue, and train the student large language model based on the token vectors to be trained.

[0035] In this embodiment, the teacher module is also used to: process each training data in the second data set based on several teacher large language models, and stop the processing each time when the output text sequence ends or the length of the output text sequence is greater than a preset threshold; when a cyclically repeated string appears at the end of the output text sequence, trim the end of the text sequence and only retain one cyclically repeated string, wherein the string consists of several tokens.

[0036] Embodiment 3 of the present invention provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the distillation method as in Embodiment 1.

[0037] Embodiment 4 of the present invention provides a readable storage medium for storing a computer program, and when the computer program is executed by a processor, the distillation method as in Embodiment 1 is implemented.

[0038] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0039] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A distillation method for a large language model, characterized in that: The following steps are involved: Obtaining a plurality of trained teacher large language models, and obtaining a first data set including a plurality of training data; A subset of the first data set is obtained as a second data set, each training data in the second data set is processed based on a plurality of teacher large language models, and a text sequence output by each teacher large language model is obtained, thereby obtaining a plurality of text sequences corresponding to each training data; Obtain a student language model, and perform the following processing on each training data in the second data set: obtain a number of token vectors corresponding to a number of text sequences corresponding to the training data, arrange the token vectors into a queue, obtain the token vectors to be trained in order from the head to the end of the queue, and train the student language model based on the token vectors to be trained.

2. The distillation method according to claim 1, characterized in that The processing of each training data in the second data set based on the plurality of teacher large language models specifically includes: Each training data in the second data set is processed based on the plurality of teacher large language models, and each time the processing is performed, when the output text sequence ends or the length of the output text sequence is greater than a preset threshold, the processing is stopped; When a cyclically repeated string appears at the end of the output text sequence, the end of the text sequence is trimmed to retain only a cyclically repeated string, wherein the string consists of a plurality of tokens.

3. The distillation method according to claim 2, characterized in that The step of obtaining a number of token vectors corresponding to a number of text sequences corresponding to the training data specifically includes: A plurality of token vectors corresponding to a plurality of text sequences corresponding to the training data are obtained. When the number of tokens Len1 in any token vector is greater than a preset length value Len2, the latter Len1-Len2 tokens in the token vector are discarded.

4. The distillation method according to claim 3, characterized in that The training of the student language model based on the token vector to be trained specifically includes: The number of tokens in the token vector to be trained is Len3, i=1, and the following operations are continued until i=Len3, the operations including: obtaining the first i tokens in the token vector, inputting the first i tokens into the student large language model for reasoning, calculating the probability distribution of the inferred vocabulary and the one-hot encoded vector of the i+1th token and obtaining the cross entropy, using the cross entropy as the loss function, performing loss function calculation, gradient calculation and back propagation, and then increasing the value of i by 1.

5. The distillation method according to claim 4, characterized in that The loss function is a cross loss function.

6. The distillation method according to claim 4, characterized in that Len2=16.

7. A distillation device for a large language model, characterized in that: Includes the following modules: An information acquisition module, used to acquire a number of trained teacher large language models and acquire a first data set including a plurality of training data; A teacher module, used for obtaining a subset of the first data set as a second data set, processing each training data in the second data set based on a plurality of teacher large language models, and obtaining a text sequence output by each teacher large language model, thereby obtaining a plurality of text sequences corresponding to each training data; The student module is used to obtain the student large language model, and performs the following processing on each training data in the second data set: obtain a number of token vectors corresponding to a number of text sequences corresponding to the training data, arrange the token vectors into a queue, obtain the token vectors to be trained in order from the head to the end of the queue, and train the student large language model based on the token vectors to be trained.

8. The distillation apparatus according to claim 7, characterized in that The teacher module is also used to: Each training data in the second data set is processed based on the plurality of teacher large language models, and each time the processing is performed, when the output text sequence ends or the length of the output text sequence is greater than a preset threshold, the processing is stopped; When a cyclically repeated string appears at the end of the output text sequence, the end of the text sequence is trimmed to retain only a cyclically repeated string, wherein the string consists of a plurality of tokens.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the distillation method according to any one of claims 1 to 6.

10. A readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the distillation method according to any one of claims 1 to 6.

Images