Large language model training method, device, equipment and medium

Abstract

The embodiments of the present specification provide a training method, apparatus, device, computer-readable storage medium, and computer program product for a large language model. The training method includes: inputting a text corpus into the large language model to obtain a prediction feature vector output by the large language model, which is used to reflect the probability of occurrence of each text unit predicted by the large language model; then, calculating a first loss value between the prediction feature vector and a label feature vector used to reflect the actual occurrence probability of each text unit in the text corpus; then, calculating a second loss value between the prediction feature vector and a reference feature vector used to reflect the specified occurrence probability of each text unit; finally, based on the first loss value and the second loss value, training the large language model can ensure the consistency of the result output by the large language model with the text in the text corpus while improving the diversity of the output of the large language model.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to a method, apparatus, device, computer-readable storage medium, and computer program product for training a large language model. Background Art

[0002] A large language model (LLM) is a generative language model characterized by its large size, large number of parameters and training data. By using more parameters and larger data for training, it effectively improves its performance and sample efficiency on various downstream tasks.

[0003] How to apply large language models to writing scenarios is very important. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a method for training a large language model. One or more embodiments of this specification also relate to a large language model training device, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a method for training a large language model is provided, comprising: Inputting a text corpus into a large language model to obtain a prediction feature vector output by the large language model; the value in the prediction feature vector is used to reflect the probability of occurrence of each text unit predicted by the large language model; Calculating a first loss value between the predicted feature vector and the label feature vector; the value in the label feature vector is used to reflect the actual occurrence probability of each of the text units in the text corpus; Calculating a second loss value between the predicted feature vector and a reference feature vector; the value in the reference feature vector is used to reflect the specified occurrence probability of each of the text units; The large language model is trained based on the first loss value and the second loss value.

[0006] According to a second aspect of an embodiment of this specification, a large language model training device is provided, including: An input module, used to input text corpus into a large language model to obtain a prediction feature vector output by the large language model; the value in the prediction feature vector is used to reflect the probability of occurrence of each text unit predicted by the large language model; A first calculation module is used to calculate a first loss value between the predicted feature vector and the label feature vector; the value in the label feature vector is used to reflect the actual occurrence probability of each of the text units in the text corpus; A second calculation module is used to calculate a second loss value between the predicted feature vector and the reference feature vector; the value in the reference feature vector is used to reflect the specified occurrence probability of each of the text units; A training module is used to train the large language model based on the first loss value and the second loss value.

[0007] According to a third aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned large language model training method are implemented.

[0008] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned large language model training method are implemented.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned large language model training method.

[0010] A computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned large language model training method.

[0011] A computer program product includes a computer program / instruction, which implements the steps of the above-mentioned large language model training method when executed by a processor.

[0012] At least one embodiment provided in this specification can achieve the following beneficial effects: input text corpus into a large language model to obtain a predicted feature vector reflecting the probability of occurrence of each text unit predicted by the large language model; then calculate the first loss value between the predicted feature vector and the label feature vector used to reflect the actual occurrence probability of each text unit in the text corpus; in addition, calculate the second loss value between the predicted feature vector and the reference feature vector used to reflect the specified occurrence probability of each text unit; finally, train the large language model based on the first loss value and the second loss value. Since the first loss value calculates the difference between the predicted feature vector and the label feature vector of the large language model; therefore, training the large language model based on the first loss value can ensure that the trained large language model can output correct prediction results. At the same time, since the second loss value calculates the probability distribution difference between the predicted feature vector and the reference feature vector of the large language model, therefore, training the large language model based on the second loss value can ensure that the trained large language model can output other texts when outputting based on weights, thereby improving the diversity of the output of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a flowchart of a large language model training method provided by an embodiment of this specification; Figure 2 It is a structural diagram of a large language model training device provided by an embodiment of this specification; Figure 3 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0014] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0015] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0016] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0017] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0018] First, the terms involved in one or more embodiments of this specification are explained.

[0019] Large Language Model: Large Language Model (LLM), also known as large language model or large model, is a natural language processing technology based on deep learning. They can better understand natural language and generate high-quality text based on a given context. For example, models GPT-4, PaLMGalactica and LaMA. Large language models can usually contain hundreds of billions (or more) of references, which can be trained with a large amount of text corpus. Through training, large language models can learn the grammatical structure of the language and the association between words, so that they can perform a wide range of tasks.

[0020] In this specification, a training method for a large language model is provided. This specification also relates to a training device for a large language model, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0021] See also Figure 1 , Figure 1 A flowchart of a large language model training method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0022] Step 102: Input text corpus into a large language model to obtain a prediction feature vector output by the large language model; the value in the prediction feature vector is used to reflect the probability of occurrence of each text unit predicted by the large language model.

[0023] In the embodiments of this specification, the large language model can be one of the pre-trained models such as LLM (Large Language Model), GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly Optimized BERT approach), XLNet (extreme Multilabel Learning Network), T5 (Text-to-Text Transfer Transformer), etc. The trained large language model can be used to process various natural language tasks. For example, text summarization, machine translation, writing systems, etc.

[0024] The text corpus can be obtained by crawling from the Internet using a web crawler; it can also be downloaded directly from a specific conversation data website; it can also be obtained by collecting open source conversation data sets in a targeted manner and obtaining them from the conversation data sets; it can also be obtained by manual writing.

[0025] The text corpus can be used as training data for training a large language model. Each piece of training data may include input text and label output text. The label output text is the text with the highest output probability that the large language model is expected to output. For example, a text corpus may include texts in two different languages ​​for the same content, namely, text in the first language and text in the second language; wherein the input text may be the text in the first language, and the label output text may be the text in the second language. For another example, a text corpus includes an article and a summary obtained by summarizing the above article; wherein the input text may be the above article, and the label output text may be the summary obtained by summarizing the above article. For another example, a text corpus includes a paragraph. The input text unit may be the first half of the above paragraph, and the label output text unit may be the second half of the above paragraph. For example, the text corpus is "Once upon a time there was a king, and he lived in a castle"; wherein the input text is "Once upon a time there was a king,"; and the label output text is "He lived in a castle.".

[0026] In the embodiment of this specification, inputting the text corpus into the large language model may be inputting only the input text in the text corpus into the large language model. The large language model can predict multiple text units based on the input text; the text unit can be a character or a word or a short sentence.

[0027] The prediction feature vector may include multiple values. The values ​​in the prediction feature vector may reflect the output probabilities of each text unit predicted by the large language model. For example, after the large language model receives the input text "Once upon a time there was a king," the predicted text units include "it," "she," and "he," and the prediction feature vector is [0.1, 0.1, 0.8]; among them, the first 0.1 in the prediction feature vector may indicate that the probability of the output of the text unit "it" is 10%; the second 0.1 in the prediction feature vector indicates that the probability of the output of the text unit "she" is 10%; and 0.8 in the prediction feature vector indicates that the probability of the output of the text unit "he" is 80%.

[0028] Step 104: Calculate a first loss value between the predicted feature vector and the label feature vector; the value in the label feature vector is used to reflect the actual occurrence probability of each of the text units in the text corpus.

[0029] In the embodiments of this specification, the label feature vector may include multiple values. Each value in the label feature vector may reflect the actual occurrence probability of each text unit in the text corpus; the probability corresponding to the text unit actually appearing in the text corpus may be 1, and the probability of the text unit not appearing in the text corpus may be 0.

[0030] In the embodiments of this specification, the actual occurrence probability can be that each text unit actually appears in the label output text. For example, for the multiple text units "it", "she", and "he" predicted by the large language model based on "Once upon a time there was a king", since the text unit "he" is a word in the label output text, and the text unit "she" and the text unit "it" are not words in the label output text, the actual occurrence probability corresponding to the text unit "he" is 1, and the actual occurrence probability of the text unit "she" and the text unit "it" is 0. That is, the label feature vector corresponding to the text unit "it", "she", and "he" is [0,0,1].

[0031] In practical applications, the first loss value can be used to measure the difference between the actual output (prediction feature vector) and the expected output (label feature vector) of the large language model. The smaller the first loss value, the closer the prediction result of the large language model is to the actual label output text, and the more accurate the large language model is.

[0032] In the embodiment of this specification, since the first loss value only focuses on the text units that appear in the text corpus and ignores other text units, it may cause the probability of the text units that appear in the text corpus in the prediction feature vector output by the large language model to be very high, while the probability of other text units is very small. For example, the prediction feature vector output by the large language model may be [0.1, 0.1, 0.8]; when the large language model outputs text according to the weight, the output text has high repetitiveness. For example, when generating multiple different abstracts for an article, the repetitiveness in the generated multiple abstracts will be high, thus lacking diversity. Based on this, the second loss value is also provided in the embodiment of this specification.

[0033] Step 106: Calculate a second loss value between the predicted feature vector and a reference feature vector; the value in the reference feature vector is used to reflect the specified occurrence probability of each of the text units.

[0034] In the embodiment of the present specification, the second loss value can be used to improve the probability of outputting text units other than the text units appearing in the text corpus in each text unit predicted by the large language model during the training of the large language model.

[0035] In practical applications, the reference feature vector may include multiple values. Each value in the reference feature vector may reflect the specified occurrence probability of each text unit; In the embodiments of the present specification, the specified occurrence probabilities corresponding to the various text units may not be 0; or, at least one of the specified occurrence probabilities corresponding to the text units other than the text units appearing in the text corpus in the reference feature vector is not 0; or one or more of the specified occurrence probabilities corresponding to the text units other than the text units appearing in the text corpus in the reference feature vector is greater than the preset occurrence probability; the preset occurrence probability can be set according to actual needs.

[0036] The second loss value can reflect the difference in probability distribution between the predicted feature vector and the reference feature vector. The smaller the second loss value, the more similar the predicted feature vector output by the large language model is to the reference feature vector.

[0037] Step 108: Train the large language model based on the first loss value and the second loss value.

[0038] In the embodiments of the present specification, the large language model is trained based on the first loss value and the second loss value, and the comprehensive loss value composed of the first loss value and the second loss value can be made smaller and smaller as a whole. When the comprehensive loss value no longer changes or changes very little in multiple consecutive iterations or epochs (i.e., the process of completely traversing the entire training data set once), the large language model training can be stopped.

[0039] In the embodiment of the present specification, since the first loss value reflects the difference between the predicted feature vector of the large language model and the label feature vector; that is, the first loss value reflects the difference between the actual output of the large language model and the label output text in the text corpus, therefore, training the large language model based on the first loss value can ensure that the trained large language model can output a prediction result consistent with the label output text in the text corpus. At the same time, since the second loss value reflects the probability distribution difference between the predicted feature vector of the large language model and the reference feature vector, and at least one of the specified occurrence probabilities corresponding to other text units other than the text units appearing in the text corpus in the reference feature vector is not 0 or at least one is greater than the preset occurrence probability, the probability of the occurrence of other text units other than the text units appearing in the text corpus predicted by the large language model will not be too small. Therefore, training the large language model based on the second loss value can ensure that the trained large language model can also output other texts when outputting based on the weight, thereby improving the diversity of the output of the large language model.

[0040] In the embodiments of the present specification, the large language model is trained based on the first loss value and the second loss value, which can ensure that when the large language model outputs according to the weights, it is likely to output text units that appear in the text corpus, and there is a certain probability that other text units will be output; while ensuring the consistency of the results output by the large language model with the labeled output text in the text corpus, the diversity of the large language model output is improved.

[0041] In the embodiments of this specification, the training for the large language model can be fine-tuning training or enhanced training; wherein, fine-tuning training is based on a model that has undergone large-scale general language modeling (such as GPT-4) to train it to adapt it to specific tasks or fields. Enhanced training can improve the generalization ability and adaptability of the large language model.

[0042] based on Figure 1 The method, the examples of this specification also provide some specific implementation plans of the method, which are described below.

[0043] In practical applications, for ease of application, a reference feature vector may be obtained in advance.

[0044] Optionally, before calculating the second loss value between the predicted feature vector and the reference feature vector, the method may further include: The text corpus is input into a preset large language model to obtain probability values ​​corresponding to each of the text units output by the preset large language.

[0045] The reference feature vector is generated according to the probability value corresponding to each of the text units output by the preset large language model.

[0046] In the embodiment of the present specification, the specified occurrence probability corresponding to each text unit in the reference feature vector may be predicted based on the text corpus using other models.

[0047] In the embodiments of this specification, the preset large language model may be a teacher model that can guide the large language model for training; the preset large language model may be a model that has been trained; or it may be a model with more parameters and stronger learning ability that is trained together with the large language model.

[0048] After the text corpus is input into the preset large language model, the preset large language model can predict the probability of each text unit based on the text corpus. The probability values ​​of each text unit predicted by the preset large language model can be combined to obtain a reference feature vector. For example, "Once upon a time there was a king," is input into the preset large language model, and the preset large language model predicts that the output probability of the text unit "it" is 20%, the output probability of the text unit "she" is 30%, and the output probability of the text unit "he" is 50%. Then the reference feature vector can be [0.2, 0.3, 0.5].

[0049] In the embodiment of this specification, since the preset large language model uses relatively abundant samples during training and has relatively many parameters, the preset large language model can output different text units and corresponding reference feature vectors after receiving the text corpus. Therefore, the large language model is trained based on the output results of the preset large language model, which can effectively improve the diversity of the prediction results of the large language model trained based on the embodiment of this specification, and provide a basis for improving the diversity of the output results of the large language model.

[0050] In practical applications, to obtain a reference feature vector using a trained preset large language model, it is necessary to train the preset large language model, and training the preset large language model will take up more computing resources. Based on this, in order to save computing resources, the embodiments of this specification also provide another way to obtain a reference feature vector.

[0051] Optionally, before calculating the second loss value between the predicted feature vector and the reference feature vector, the method may further include: Obtain the probability distribution information of each of the text units specified by the user.

[0052] The reference feature vector is generated according to the probability distribution information of each of the text units specified by the user.

[0053] In the embodiment of this specification, the specified occurrence probability corresponding to each text unit in the reference feature vector can be set by the user. The user can be the training party of the large language model, and the user can set the probability of output of each text unit according to the experience value, that is, the specified occurrence probability of each text unit. After the setting is completed, the reference feature vector can be obtained by combining the specified occurrence probabilities of each text.

[0054] Specifically, the probability of the output of the text unit "it" can be set to 20%, the probability of the output of the text unit "she" can be set to 30%, and the probability of the output of the text unit "he" can be set to 50%, then the reference feature vector can be [0.2, 0.3, 0.5]. Alternatively, the probability of the output of the text unit "it" can be set to 50%, the probability of the output of the text unit "she" can be set to 50%, and the probability of the output of the text unit "he" can be set to 0%, then the reference feature vector can be [0.5, 0.5, 0].

[0055] In the embodiments of this specification, the user sets the output probability of each text unit by himself, avoiding the training of a preset large language model, saving computing resources, and also ensuring the diversity of the output text when the large language model outputs according to the weights.

[0056] In the embodiments of this specification, specific contents of calculating the first loss value between the predicted feature vector and the label feature vector are also provided for ease of understanding.

[0057] Optionally, the calculating a first loss value between the predicted feature vector and the label feature vector may specifically include: A cross entropy loss function is used to calculate a first loss value between the predicted feature vector and the label feature vector.

[0058] It can be understood that the first loss value between the predicted feature vector and the label feature vector calculated using the cross entropy loss function is the cross entropy loss.

[0059] Cross-Entropy Loss is a loss function widely used in machine learning, especially in classification problems. It can measure the difference between the actual output of the model and the expected output. That is, the first loss value can measure the difference between the predicted feature vector and the label feature vector.

[0060] Since the cross entropy loss function has a large penalty for misclassification, when the large language model makes an error in prediction, the value of the loss function will become very large. Therefore, using the cross entropy loss function to train the large language model can make the large language model learn and train faster, reduce the training time of the large language model, and improve the training efficiency of the large language model.

[0061] Specifically, the first loss value between the predicted feature vector and the label feature vector can be calculated based on Formula 1.

[0062] Formula 1 in, is the calculated first loss value; is the actual occurrence probability of each text unit in the label feature vector, is the predicted occurrence probability of each text unit in the predicted feature vector, and N is the number of text units. Assume that if the label feature vector is [0,0,1] and the predicted feature vector is [0.2,0.3,0.5], then the first loss value is It can be: =-[0*log(0.2)+0*log(0.3)+1*log(0.5)].

[0063] In the embodiments of this specification, for ease of understanding, specific content of calculating the second loss value between the predicted feature vector and the reference feature vector is also provided.

[0064] Optionally, the calculating a second loss value between the predicted feature vector and the reference feature vector may specifically include: A divergence loss function is used to calculate a second loss value between the predicted feature vector and the reference feature vector.

[0065] In the embodiment of this specification, the divergence loss function can be used to measure the similarity of two probability distributions. The second loss value between the predicted feature vector and the reference feature vector is calculated in combination with the divergence loss function, and the large language model is trained based on the second loss value. This can make the probability of occurrence of other text units predicted by the large language model, except for the text units appearing in the text corpus, close to the probability of occurrence of other text units in the reference feature vector, that is, the probability of occurrence of other text units will not be too small, thereby ensuring the diversity of the output of the large language model when the large language model outputs according to the weights.

[0066] In the embodiments of this specification, a specific solution for calculating the second loss value based on the divergence function is also provided.

[0067] Optionally, the adopting a divergence loss function to calculate a second loss value between the predicted feature vector and the reference feature vector may specifically include: For any text unit among the text units, a ratio of the specified occurrence probability of the text unit represented by the reference feature vector to the predicted occurrence probability of the text unit represented by the predicted feature vector is calculated.

[0068] According to the ratio and the specified occurrence probability of any text unit, a difference between the specified occurrence probability of any text unit and the predicted occurrence probability of any text unit is determined.

[0069] The difference between the specified occurrence probability and the predicted occurrence probability of each of the text units is summed to obtain a second loss value between the predicted feature vector and the reference feature vector.

[0070] In the embodiments of this specification, specifically, the second loss value between the predicted feature vector and the reference feature vector can be calculated according to Formula 2.

[0071] Formula 2 in, is the calculated second loss value; is the specified occurrence probability corresponding to each text unit in the reference feature vector; is the predicted occurrence probability corresponding to each text unit in the predicted feature vector; N is the number of text units. The difference between the specified occurrence probability of any text unit and the predicted occurrence probability of the any text unit may be represented.

[0072] Assume that the reference feature vector is [0.2, 0.3, 0.5] and the predicted feature vector is [0.1, 0.1, 0.8], then the second loss value can be: =-[0.2*log(0.2 / 0.1)+0.3*log(0.3 / 0.1)+0.5*log(0.5 / 0.8)].

[0073] In the embodiments of this specification, another specific method for calculating the second loss value is also provided.

[0074] Optionally, each of the text units may include a label text unit and a target text unit; the label text unit is a text unit among the text units that actually appears in the text corpus; the target text unit is a text unit among the text units except the label text unit; the calculating the second loss value between the predicted feature vector and the reference feature vector may specifically include: According to the specified occurrence probabilities of each of the text units reflected by the reference feature vector, the first probability of any of the target text units appearing in the set consisting of the target text units is determined, and a first feature vector consisting of the first probabilities corresponding to the target text units in the set is obtained.

[0075] According to the predicted occurrence probabilities of the text units represented by the predicted feature vector, a second probability of any target text unit appearing in the set is determined, and a second feature vector composed of the second probabilities corresponding to the target text units in the set is obtained.

[0076] A second loss value between the first eigenvector and the second eigenvector is calculated.

[0077] In the embodiments of this specification, the label text unit can be a text unit that actually appears in the text corpus among the various text units. Specifically, the label text unit can be a text unit that appears in the label output text among the various text units. The target text unit can be a text unit other than the label text unit. For example, assuming that the large language model predicts the text units including "it", "she" and "he" based on the text corpus "Once upon a time there was a king", among which the text unit "he" is a text unit that actually appears in the text corpus "He lives in a castle." Therefore, the text unit "he" is a label text unit; while the text unit "she" and the text unit "it" do not actually appear in the text corpus "He lives in a castle." Therefore, the text unit "she" and the text unit "it" are target text units.

[0078] In the embodiment of this specification, the first probability can be the proportion of the specified occurrence probability corresponding to a target text unit in the reference feature vector to the specified occurrence probability of all target text units. For example, when the reference feature vector is [0.1, 0.2], the first probability for the target text unit "it" is 0.1 / (0.1+0.2); the first probability for the target text unit "she" can be 0.2 / (0.1+0.2). The first feature vector obtained based on each target text unit is [1 / 3, 2 / 3].

[0079] In the embodiment of this specification, the second probability can be the proportion of the predicted occurrence probability corresponding to a target text unit in the prediction feature vector to the predicted occurrence probability of the entire target text. For example, in the prediction feature vector [0.3, 0.2], the second probability of the target text unit "it" is 0.3 / (0.3+0.2); the second probability of the target text unit "she" can be 0.2 / (0.3+0.2). The second feature vector obtained based on the target text unit is [3 / 5, 2 / 5].

[0080] In the embodiment of this specification, a divergence loss function may be used to calculate a second loss value between the first eigenvector and the second eigenvector.

[0081] Specifically, the second loss value between the first eigenvector [1 / 3, 2 / 3] and the second eigenvector [3 / 5, 2 / 5] =-[(1 / 3)*log((1 / 3) / (3 / 5))+2 / 3*log((2 / 3) / (2 / 5)].

[0082] In addition, a specific method for calculating the second loss value is also provided in the embodiments of this specification.

[0083] Optionally, the reference feature vector only includes the specified occurrence probabilities of the target text units among the various text units, and the sum of the specified occurrence probabilities of the target text units contained in the reference feature vector is 1; the target text unit is a text unit other than the label text unit; and the label text unit is a text unit among the various text units that actually appears in the text corpus.

[0084] In the embodiment of this specification, the specified occurrence probability of each text unit in the reference feature vector can be set by the user. In addition, since the second loss is to avoid the large language model outputting text based on the weight, the output text unit has high repetitiveness, that is, the reference feature vector is used to adjust the predicted occurrence probability of the target text unit. Therefore, when setting the specified occurrence probability of each text unit, the user can only set the specified occurrence probability of the target text unit. For example, the reference feature vector set by the user can be [3 / 5,2 / 5].

[0085] In the embodiments of the present specification, the sum of the specified occurrence probabilities of the target text units is set to 1, and training is performed on the large language model. This ensures that the probability of the output of the target text units predicted by the trained large language model will not be too small, thereby ensuring that when the large language model outputs according to the weights, the target text units have a certain probability of being output, thereby improving the diversity of the output text of the large language model.

[0086] The label text unit can be the text unit that appears in the label output text among the various text units. The target text unit is the text unit other than the label text unit. For example, among the text units "it", "she" and "he", the text unit "he" is the label text unit; while the text unit "she" and the text unit "it" are the target text units.

[0087] In the embodiment of the present specification, optionally, the designated occurrence probabilities of the target text units may be the same.

[0088] Specifically, in the embodiment of this specification, the second loss value between the predicted feature vector and the reference feature vector can be calculated according to Formula 3: Formula 3 in, is the calculated second loss value; is the specified occurrence probability corresponding to each target text unit in the reference feature vector; is the predicted occurrence probability corresponding to each target text unit in the predicted feature vector; N-1 is the number of target text units. Here, we continue to use the above example to illustrate that the value of N-1 can be 2.

[0089] In the embodiments of the present specification, setting the corresponding designated occurrence probabilities of each target text unit to the same value can effectively reduce the workload of the user in setting the designated occurrence probabilities of each target text unit and improve the efficiency of training the large language model.

[0090] In addition, in practical applications, the specified occurrence probability of the target text unit may be adjusted according to actual needs, which is not specifically limited here.

[0091] In the embodiments of this specification, specific content of training a large language model based on a first loss value and a second loss value is also provided.

[0092] Optionally, the training the large language model based on the first loss value and the second loss value may specifically include: A comprehensive loss value is calculated based on the first loss value and the second loss value.

[0093] The large language model is trained based on the comprehensive loss value.

[0094] In the embodiments of this specification, after the comprehensive loss value is calculated based on the first loss value and the second loss value, the parameters of the large language model can be updated based on the comprehensive loss value, and the training can be terminated when the comprehensive loss value of the large language model converges to a certain level and no longer decreases.

[0095] Optionally, calculating a comprehensive loss value based on the first loss value and the second loss value may specifically include: A first intermediate loss value is calculated based on the first loss value and a first weight of the first loss value.

[0096] A second intermediate loss value is calculated based on the second loss value and a second weight of the second loss value.

[0097] In the embodiment of this specification, the comprehensive loss value can be calculated based on Formula 4, optionally: Formula 4 in, is the comprehensive loss value; is the first loss value between the predicted feature vector and the label feature vector calculated in the previous article; is the first weight of the first loss value; is the second loss value between the predicted feature vector and the reference feature vector calculated in the previous text; is the second weight of the second loss value.

[0098] In the embodiment of this specification, the first weight of the first loss value is usually greater than the second weight of the second loss value to ensure that the large language model can output the label text unit. Specifically, the sum of the first weight and the second weight can be 1, wherein the second weight can be 0.1 or other values.

[0099] Corresponding to the above method embodiment, this specification also provides a large language model training device embodiment, Figure 2 FIG. 1 is a schematic diagram showing a structure of a large language model training device provided by an embodiment of the present specification. Figure 2 As shown, the device comprises: An input module 202 is used to input text corpus into a large language model to obtain a prediction feature vector output by the large language model; the value in the prediction feature vector is used to reflect the probability of occurrence of each text unit predicted by the large language model; A first calculation module 204 is used to calculate a first loss value between the predicted feature vector and the label feature vector; the value in the label feature vector is used to reflect the actual occurrence probability of each of the text units in the text corpus; A second calculation module 206 is used to calculate a second loss value between the predicted feature vector and the reference feature vector; the value in the reference feature vector is used to reflect the specified occurrence probability of each of the text units; The training module 208 is used to train the large language model based on the first loss value and the second loss value.

[0100] Optional, Figure 2 The device shown may also include: The first acquisition module is used to input the text corpus into a preset large language model to obtain the probability value corresponding to each of the text units output by the preset large language.

[0101] The first generating module is used to generate the reference feature vector according to the probability value corresponding to each of the text units output by the preset large language model.

[0102] Optional, Figure 2 The device shown may also include: A second acquisition module, used to acquire probability distribution information of each of the text units specified by a user; The second generating module is used to generate the reference feature vector according to the probability distribution information of each text unit specified by the user.

[0103] Optionally, the first calculation module 204 may specifically include: The first calculation unit is used to calculate a first loss value between the predicted feature vector and the label feature vector by using a cross entropy loss function.

[0104] Optionally, the second calculation module 206 may specifically include: The second calculation unit is used to calculate a second loss value between the predicted feature vector and the reference feature vector by using a divergence loss function.

[0105] Optionally, the second computing unit may be specifically configured to: For any text unit among the text units, a ratio of the specified occurrence probability of the text unit represented by the reference feature vector to the predicted occurrence probability of the text unit represented by the predicted feature vector is calculated.

[0106] According to the ratio and the specified occurrence probability of any text unit, a difference between the specified occurrence probability of any text unit and the predicted occurrence probability of any text unit is determined.

[0107] The difference between the specified occurrence probability and the predicted occurrence probability of each of the text units is summed to obtain a second loss value between the predicted feature vector and the reference feature vector.

[0108] Optionally, each of the text units includes a label text unit and a target text unit; the label text unit is a text unit among the text units that actually appears in the text corpus; the target text unit is a text unit among the text units except the label text unit; the second calculation module 206 may specifically include: A first feature vector acquisition unit is used to determine the first probability of any target text unit appearing in a set consisting of the target text units based on the specified occurrence probabilities of the respective text units reflected by the reference feature vector, and obtain a first feature vector consisting of the first probabilities corresponding to the respective target text units in the set.

[0109] The second feature vector acquisition unit is used to determine the second probability of any target text unit appearing in the set based on the predicted occurrence probability of each text unit represented by the predicted feature vector, and obtain a second feature vector composed of the second probabilities corresponding to each target text unit in the set.

[0110] The third calculation unit is used to calculate a second loss value between the first eigenvector and the second eigenvector.

[0111] Optionally, the third computing unit may be specifically used for: A divergence loss function is used to calculate a second loss value between the first eigenvector and the second eigenvector.

[0112] Optionally, the reference feature vector only includes the specified occurrence probabilities of the target text units among the various text units, and the sum of the specified occurrence probabilities of the target text units contained in the reference feature vector is 1; the target text unit is a text unit other than the label text unit; and the label text unit is a text unit among the various text units that actually appears in the text corpus.

[0113] Optionally, the designated occurrence probabilities of the target text units are the same.

[0114] Optionally, the training module 208 may specifically include: The fourth calculation unit is used to calculate a comprehensive loss value based on the first loss value and the second loss value.

[0115] A training unit is used to train the large language model based on the comprehensive loss value.

[0116] Optionally, the fourth computing unit may be specifically configured to: A first intermediate loss value is calculated based on the first loss value and a first weight of the first loss value.

[0117] A second intermediate loss value is calculated based on the second loss value and a second weight of the second loss value.

[0118] A sum operation is performed on the first intermediate loss value and the second intermediate loss value to obtain the comprehensive loss value.

[0119] Optionally, the first weight is greater than the second weight.

[0120] The above is a schematic scheme of a large language model training device of this embodiment. It should be noted that the technical scheme of the large language model training device and the technical scheme of the large language model training method described above belong to the same concept, and the details of the technical scheme of the large language model training device that are not described in detail can be found in the description of the technical scheme of the large language model training method described above.

[0121] Figure 3 The block diagram of a computing device 300 according to an embodiment of the present specification is shown. The components of the computing device 300 include but are not limited to a memory 310 and a processor 320. The processor 320 is connected to the memory 310 via a bus 330, and the database 350 is used to store data.

[0122] The computing device 300 also includes an access device 340 that enables the computing device 300 to communicate via one or more networks 360. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0123] In one embodiment of the present specification, the above components of the computing device 300 and Figure 3 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 3 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0124] The computing device 300 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 300 may also be a mobile or stationary server.

[0125] The processor 320 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned large language model training method.

[0126] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned large language model training method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned large language model training method.

[0127] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned large language model training method.

[0128] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the training method of the large language model described above belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the training method of the large language model described above.

[0129] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned large language model training method.

[0130] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the training method of the large language model described above belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be found in the description of the technical scheme of the training method of the large language model described above.

[0131] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0132] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0133] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0134] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0135] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A method for training a large language model, characterized in that: The method comprises: Inputting a text corpus into a large language model to obtain a prediction feature vector output by the large language model; the value in the prediction feature vector is used to reflect the probability of occurrence of each text unit predicted by the large language model; Calculating a first loss value between the predicted feature vector and the label feature vector; the value in the label feature vector is used to reflect the actual occurrence probability of each of the text units in the text corpus; Calculating a second loss value between the predicted feature vector and a reference feature vector; the value in the reference feature vector is used to reflect the specified occurrence probability of each of the text units; The large language model is trained based on the first loss value and the second loss value.

2. The method according to claim 1, characterized in that Before calculating the second loss value between the predicted feature vector and the reference feature vector, the method further includes: Inputting the text corpus into a preset large language model to obtain probability values ​​corresponding to each of the text units output by the preset large language; The reference feature vector is generated according to the probability value corresponding to each of the text units output by the preset large language model.

3. The method according to claim 1, characterized in that Before calculating the second loss value between the predicted feature vector and the reference feature vector, the method further includes: Obtaining probability distribution information of each of the text units specified by a user; The reference feature vector is generated according to the probability distribution information of each of the text units specified by the user.

4. The method according to claim 1, characterized in that: The calculating a first loss value between the predicted feature vector and the label feature vector specifically includes: A cross entropy loss function is used to calculate a first loss value between the predicted feature vector and the label feature vector.

5. The method according to claim 1, characterized in that The calculating a second loss value between the predicted feature vector and the reference feature vector specifically includes: A divergence loss function is used to calculate a second loss value between the predicted feature vector and the reference feature vector.

6. The method according to claim 5, characterized in that The adopting of the divergence loss function to calculate the second loss value between the predicted feature vector and the reference feature vector specifically includes: For any text unit among the text units, calculating a ratio of a specified occurrence probability of the text unit represented by the reference feature vector to a predicted occurrence probability of the text unit represented by the predicted feature vector; Determining, based on the ratio and the specified occurrence probability of the any text unit, a difference between the specified occurrence probability of the any text unit and the predicted occurrence probability of the any text unit; The difference between the specified occurrence probability and the predicted occurrence probability of each of the text units is summed to obtain a second loss value between the predicted feature vector and the reference feature vector.

7. The method according to claim 1, characterized in that The respective text units include a label text unit and a target text unit; the label text unit is a text unit among the respective text units that actually appears in the text corpus; the target text unit is a text unit among the respective text units except the label text unit; the calculating the second loss value between the predicted feature vector and the reference feature vector specifically includes: Determine, based on the specified occurrence probabilities of the respective text units reflected by the reference feature vector, a first probability of any of the target text units appearing in a set consisting of the respective target text units, and obtain a first feature vector consisting of the first probabilities corresponding to the respective target text units in the set; Determine a second probability of any of the target text units appearing in the set according to the predicted occurrence probabilities of each of the text units represented by the predicted feature vector, and obtain a second feature vector composed of the second probabilities corresponding to each of the target text units in the set; A second loss value between the first eigenvector and the second eigenvector is calculated.

8. The method according to claim 7, characterized in that The calculating a second loss value between the first feature vector and the second feature vector specifically includes: A divergence loss function is used to calculate a second loss value between the first eigenvector and the second eigenvector.

9. The method according to claim 1, characterized in that: The reference feature vector only includes the specified occurrence probabilities of the target text units among the various text units, and the sum of the specified occurrence probabilities of the target text units contained in the reference feature vector is 1; the target text unit is a text unit other than the label text unit; the label text unit is a text unit among the various text units that actually appears in the text corpus.

10. The method according to claim 9, characterized in that The designated occurrence probabilities of the target text units are the same.

11. The method according to claim 1, characterized in that: The training of the large language model based on the first loss value and the second loss value specifically includes: Calculating a comprehensive loss value based on the first loss value and the second loss value; The large language model is trained based on the comprehensive loss value.

12. The method according to claim 11, characterized in that The calculating the comprehensive loss value based on the first loss value and the second loss value specifically includes: Calculate a first intermediate loss value based on the first loss value and a first weight of the first loss value; Calculate a second intermediate loss value based on the second loss value and a second weight of the second loss value; A sum operation is performed on the first intermediate loss value and the second intermediate loss value to obtain the comprehensive loss value.

13. The method according to claim 11, characterized in that The first weight is greater than the second weight.

14. A large language model training device, characterized in that: include: An input module is used to input text corpus into a large language model to obtain a predicted feature vector output by the large language model; The value in the predicted feature vector is used to reflect the probability of occurrence of each text unit predicted by the large language model; A first calculation module is used to calculate a first loss value between the predicted feature vector and the label feature vector; the value in the label feature vector is used to reflect the actual occurrence probability of each of the text units in the text corpus; A second calculation module is used to calculate a second loss value between the predicted feature vector and the reference feature vector; the value in the reference feature vector is used to reflect the specified occurrence probability of each of the text units; A training module is used to train the large language model based on the first loss value and the second loss value.

15. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the large language model training method described in any one of claims 1 to 13 are implemented.

16. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, implement the steps of the large language model training method described in any one of claims 1 to 13.

17. A computer program product, characterized in that The invention comprises a computer program / instruction, which implements the steps of the method according to any one of claims 1 to 13 when executed by a processor.

Images