Model training methods, devices, equipment, and computer program products

Abstract

This disclosure provides a model training method, apparatus, device, and computer program product. The method involves inputting multiple first labels into a large language model to be trained, obtaining multiple initial vectors, and calculating multiple loss values ​​based on these initial vectors and multiple actual results. Further, based on the initial vector corresponding to each first label, multiple probability values ​​for the prediction result corresponding to that first label are determined, and these probability values ​​are subjected to multinomial sampling to obtain the prediction result corresponding to that first label. If the prediction result corresponding to the first label matches the actual result corresponding to that first label, the loss value calculated based on the actual result and initial vectors corresponding to that first label is removed from the multiple loss values, thereby reducing the large language model's dependence on high-frequency text combinations. This effectively reduces the impact of text frequency bias on the large language model training process, improving the generalization ability and performance of the large language model.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to model training methods, apparatus, devices, and computer program products. Background Technology

[0002] With the continuous development of computer technology, artificial intelligence (AI) technology is becoming increasingly mature. For example, large language models trained on AI can be used to analyze text information to achieve preset functions, such as intelligent question answering.

[0003] However, existing technologies suffer from text frequency bias in training large language models. For example, low-frequency text combinations (such as low-frequency words) occur less frequently, while high-frequency text combinations (such as high-frequency words) occur more frequently. This causes the training process of large language models to rely more heavily on high-frequency text combinations, thus reducing the performance of the large language models. Summary of the Invention

[0004] According to one aspect of this disclosure, a model training method is provided, the method comprising:

[0005] Multiple first labels related to the original text are input into the large language model to be trained, so that the large language model to be trained outputs multiple initial vectors, and the multiple first labels and the multiple initial vectors correspond one-to-one; multiple loss values ​​are calculated based on the multiple initial vectors and the actual results corresponding to the multiple first labels respectively.

[0006] For each of the plurality of first labels, based on the initial vector corresponding to the first label, multiple probability values ​​of the prediction result corresponding to the first label are determined, and polynomial sampling is performed on the multiple probability values ​​to obtain the prediction result corresponding to the first label;

[0007] If the prediction result corresponding to the first label matches the actual result corresponding to the first label, then remove the loss value calculated based on the actual result corresponding to the first label and the initial vector corresponding to the first label from the plurality of loss values ​​to obtain the remaining loss value.

[0008] Based on the remaining loss value, a target loss value is calculated, and the parameters of the large language model to be trained are updated based on the target loss value.

[0009] According to another aspect of this disclosure, a model training apparatus is provided, the apparatus comprising:

[0010] The input module is used to input multiple first labels related to the original text into the large language model to be trained, so that the large language model to be trained outputs multiple initial vectors, and the multiple first labels and the multiple initial vectors correspond one-to-one.

[0011] The first calculation module is used to calculate multiple loss values ​​based on the multiple initial vectors and the actual results corresponding to the multiple first labels respectively;

[0012] The determination module is used to determine multiple probability values ​​of the prediction result corresponding to the first label for each of the plurality of first labels, based on the initial vector corresponding to the first label;

[0013] A multinomial sampling module is used to perform multinomial sampling on the multiple probability values ​​to obtain the prediction result corresponding to the first label;

[0014] The removal module is used to remove the loss value calculated based on the actual result corresponding to the first label and the initial vector corresponding to the first label from the plurality of loss values ​​when the prediction result corresponding to the first label matches the actual result corresponding to the first label, so as to obtain the remaining loss value.

[0015] The second calculation module is used to calculate the target loss value based on the remaining loss value;

[0016] An update module is used to update the parameters of the large language model to be trained based on the target loss value.

[0017] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0018] Processor; and

[0019] Stored program memory,

[0020] The program includes instructions that, when executed by the processor, cause the processor to perform the method according to the first aspect.

[0021] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the method according to the first aspect.

[0022] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0023] One or more technical solutions provided in the embodiments of this application can reduce the dependence of large language models on high-frequency text combinations, enabling the large language model to be trained to better learn low-frequency text combinations, especially low-frequency but important text combinations. This effectively reduces the impact of text frequency bias on the training process of large language models, improving the generalization ability and performance of large language models. Attached Figure Description

[0024] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0025] Figure 1 A flowchart of the model training method provided in this embodiment of the disclosure;

[0026] Figure 2 A schematic diagram illustrating the application scenarios to which the model training method provided in the embodiments of this disclosure is applicable;

[0027] Figure 3 A schematic diagram illustrating model training provided in an embodiment of this disclosure;

[0028] Figure 4 This is a schematic diagram of the structure of the model training device provided in the embodiments of this disclosure;

[0029] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0031] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0032] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0033] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0034] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0035] Typically, during the training of large language models, the training samples suffer from text frequency bias. For example, low-frequency words appear less frequently, while high-frequency words appear more often, causing the training process of the large language model to become more reliant on high-frequency words, thus reducing the model's performance. To address this issue, this disclosure provides a model training method, which will be described below with reference to specific embodiments.

[0036] Figure 1 This is a flowchart illustrating a model training method provided in an embodiment of this disclosure. The method can be executed by a model training device, which can be implemented in software and / or hardware. This device can be configured in an electronic device, such as a server, server cluster, or terminal. Specifically, a terminal can be a desktop computer, tablet computer, or mobile phone. A server cluster can be multiple servers working together to provide the same service, appearing to the client as a single server. A server cluster can utilize multiple computers for parallel computing to achieve higher computing speeds, or it can use multiple computers for backup, ensuring the entire system continues to operate normally even if any machine fails. In other words, in this embodiment, the model training process can be executed by a server, server cluster, or terminal. When the model training process is executed by a server or server cluster, the trained model can be stored on the server or server cluster, or deployed on a terminal. When the model training process is executed by a terminal, the trained model can be stored on that terminal, or deployed on other terminals.

[0037] For example, the model training method described in this embodiment is derived from, for example, Figure 2 The server 21 shown executes the process. Terminal 22 can communicate with server 21; specifically, terminal 22 can provide server 21 with information required for model training, such as raw text. It is understood that server 21 is not limited to obtaining raw text from terminal 22. For example, in other embodiments, server 21 can obtain raw text from other servers or other terminals. Alternatively, server 21 may have the raw text pre-stored locally. Furthermore, this disclosure does not limit the specific form of the raw text; for example, the raw text can be Chinese text, English text, etc. It is understood that during model training, multiple iterations are typically required until the model parameters converge, or the number of iterations is greater than or equal to a preset number. In each iteration, the raw text can be at least one sentence, at least one paragraph, or at least one article.

[0038] The following is combined Figure 2 A detailed introduction to model training methods, such as Figure 1 As shown, the specific steps of this method are as follows:

[0039] S101. Input multiple first labels related to the original text into the large language model to be trained, so that the large language model to be trained outputs multiple initial vectors, and the multiple first labels and the multiple initial vectors correspond one-to-one.

[0040] For example Figure 2 As shown, terminal 22 sends the original text to server 21. Server 21 first converts the original text into multiple first tags, which can be as follows: Figure 3 The symbols Token1, Token2, ..., TokenN are shown. Additionally, a Large Language Model (LLM) to be trained is deployed on server 21. Server 21 inputs Token1, Token2, ..., TokenN into the LLM to be trained, causing the LLM to output multiple initial vectors, such as... Figure 3 The given values ​​are logits1, logits2, ..., logitsN, where each Token in Token1, Token2, ..., TokenN corresponds to an initialization vector. For example, the initialization vector for Token1 is logits1, the initialization vector for Token2 is logits2, and so on, with the initialization vector for TokenN being logitsN.

[0041] S102. Calculate multiple loss values ​​based on the multiple initial vectors and the actual results corresponding to the multiple first labels.

[0042] Understandably, during training, the predictions obtained by the large language model to be trained may be correct or incorrect. If the prediction matches the actual result, the prediction is correct. If the prediction does not match the actual result, the prediction is incorrect. Therefore, to determine whether the predictions obtained by the large language model to be trained are correct, it is necessary to pre-define the actual result corresponding to each first label. For example, the actual result corresponding to Token1 is denoted as actual result 1, the actual result corresponding to Token2 is denoted as actual result 2, and so on, with the actual result corresponding to TokenN denoted as actual result N. Further, based on logits1 and actual result 1, the loss value Loss1 can be calculated. Loss1 can be the difference between the vector representations of logits1 and actual result 1; for example, the dimension of logits1 and the dimension of the vector representation of actual result 1 are the same. Based on logits2 and actual result 2, the loss value Loss2 can be calculated. Loss2 can be the difference between the vector representations of logits2 and actual result 2; for example, the dimension of logits2 and the dimension of the vector representation of actual result 2 are the same. Similarly, based on logitsN and the actual result N, the loss value LossN can be calculated. LossN can be the difference between the vector representations of logitsN and the actual result N; for example, the dimensions of logitsN and the vector representation of the actual result N may be the same. Loss1, Loss2, ..., LossN are specifically as follows: Figure 3 As shown.

[0043] S103. For each of the plurality of first labels, based on the initial vector corresponding to the first label, determine multiple probability values ​​of the prediction result corresponding to the first label, and perform polynomial sampling on the multiple probability values ​​to obtain the prediction result corresponding to the first label.

[0044] like Figure 3As shown, in this embodiment of the present disclosure, each of logits1, logits2, ..., logitsN can be normalized using the softmax function to obtain multiple normalized vectors, such as prob1, prob2, ..., probN. The normalized vector obtained after normalizing logits1 is prob1, the normalized vector obtained after normalizing logits2 is prob2, and so on, until the normalized vector obtained after normalizing logitsN is probN. Further, each of prob1, prob2, ..., probN is subjected to polynomial sampling to obtain multiple second labels, such as predictToken1, predictToken2, ..., predictTokenN. Each second label can also be referred to as a prediction label. Specifically, polynomial sampling of prob1 yields predictToken1, polynomial sampling of prob2 yields predictToken2, and so on, until polynomial sampling of probN yields predictTokenN. Furthermore, prediction result 1 is obtained based on predict Token1, prediction result 2 is obtained based on predict Token2, and so on, until prediction result N is obtained based on predict TokenN. Thus, each first label corresponds to an initial vector, a normalized vector, a second label, a prediction result, and an actual result. Specifically, prob1 includes multiple probability values ​​for prediction result 1 corresponding to Token1, prob2 includes multiple probability values ​​for prediction result 2 corresponding to Token2, and so on, until probN includes multiple probability values ​​for prediction result N corresponding to TokenN.

[0045] S104. If the prediction result corresponding to the first label matches the actual result corresponding to the first label, then remove the loss value calculated based on the actual result corresponding to the first label and the initial vector corresponding to the first label from the plurality of loss values ​​to obtain the remaining loss value.

[0046] like Figure 3As shown, for each Token in Token1, Token2, ..., TokenN, it can be determined whether the predicted result corresponding to that Token matches the actual result corresponding to that Token. For example, compare predicted result 1 with actual result 1, compare predicted result 2 with actual result 2, and so on, comparing predicted result N with actual result N. If the predicted result corresponding to any Token matches the actual result corresponding to that Token, for example, they are the same, or the similarity is greater than or equal to a preset similarity, then the loss values ​​calculated based on the actual result and the initial vector corresponding to that Token in Loss1, Loss2, ..., LossN are removed. For example, if the predicted result 2 corresponding to Token2 matches the actual result 2 corresponding to Token2, then Loss2 in Loss1, Loss2, ..., LossN is removed. It is understood that this is only an illustrative example of removing Loss2; in some embodiments, more than one loss value may be removed. Therefore, after removing one or more loss values ​​from Loss1, Loss2, ..., LossN, the remaining loss values ​​can be obtained.

[0047] S105. Calculate the target loss value based on the remaining loss value, and update the parameters of the large language model to be trained based on the target loss value.

[0048] For example, after obtaining the remaining loss values, the average of these remaining loss values ​​is calculated to obtain the target loss value. Alternatively, the remaining loss values ​​can be weighted and summed according to their respective weights to obtain the target loss value. Alternatively, the target loss value can be calculated based on the remaining loss values ​​using other calculation methods. Specifically, the target loss value can be as follows: Figure 3 The loss shown is the target loss value, which can be recorded as the final loss value. Furthermore, the parameters of the large language model to be trained are updated based on this final loss value.

[0049] This embodiment of the disclosure inputs multiple first labels related to the original text into a large language model to be trained, causing the large language model to output multiple initial vectors. Multiple loss values ​​are calculated based on these initial vectors and the actual results corresponding to the multiple first labels. Further, based on the initial vector corresponding to each first label, multiple probability values ​​of the prediction result corresponding to that first label are determined, and multinomial sampling is performed on these probability values ​​to obtain the prediction result corresponding to that first label. If the prediction result corresponding to the first label matches the actual result corresponding to that first label, then that first label typically corresponds to a high-frequency text combination. Therefore, by removing the loss value calculated based on the actual result and initial vector corresponding to the first label from the multiple loss values, the dependence of the large language model on high-frequency text combinations can be reduced. Further, based on the remaining loss values, a target loss value is calculated, and the parameters of the large language model to be trained are updated based on the target loss value, enabling the large language model to better learn low-frequency text combinations, especially low-frequency but important text combinations. This effectively reduces the impact of text frequency bias on the training process of large language models, and improves the generalization ability and performance of large language models.

[0050] It is understandable that S101-S105 as described in the above embodiment represents one round of iterative training for the large language model. Multiple rounds of iterative training are typically required, and the same first label may appear multiple times. For example, the same first label may appear as input to the large language model in different iterations. Since the parameters of the large language model are different in different iterations, the initial vector corresponding to the same first label is different in different iterations; that is, the initial vector corresponding to the same first label changes dynamically in different iterations. Furthermore, although the same first label may appear in multiple iterations, it may not appear in consecutive iterations, but rather appear in one iteration and then reappear after several iterations. In this case, since the large language model does not learn the first label in these iterations, it may be forgotten. When the first label reappears, the normalized vector corresponding to the first label will change; that is, the normalized vector corresponding to the same first label changes adaptively in different iterations. Therefore, removing some of the loss values ​​from the multiple loss values ​​mentioned above is dynamic and adaptively changing in different iterations.

[0051] Based on the above embodiments, the first label corresponds to at least one text unit in the original text, and the first label represents the position information of the at least one text unit in the preset table.

[0052] In this embodiment, the original text may be at least one sentence, at least one paragraph of text, or at least one article. The original text is composed of multiple text units, and each text unit may be a Chinese character, an English word, a vocabulary, etc. Hereinafter, an example will be given for illustrative purposes where the original text is a sentence and the text unit is a Chinese character. For example, the original text is "The sun along the mountain bows", for any Token such as Figure 3 shown in Token1, Token2, …, TokenN, this Token may correspond to at least one Chinese character in "The sun along the mountain bows", or this Token may not correspond to any Chinese character in "The sun along the mountain bows". If this Token corresponds to multiple Chinese characters in "The sun along the mountain bows", then these multiple Chinese characters may be consecutive multiple Chinese characters in "The sun along the mountain bows". For example, Token1 corresponds to "The sun" in "The sun along the mountain bows". In addition, the at least one Chinese character corresponding to Token1, Token2, …, TokenN respectively may be consecutive in "The sun along the mountain bows". For example, taking each Token in Token1, Token2, …, TokenN corresponding to one Chinese character as an example, at this time, N = 5, Token1 corresponds to "白" (white), Token2 corresponds to "日" (sun), Token3 corresponds to "依" (rely on), Token4 corresponds to "山" (mountain), and Token5 corresponds to "尽" (set).

[0053] In addition, each Token in Token1, Token2, …, TokenN shown in Figure 3 represents the position information of the at least one text unit corresponding to this Token, such as the position information of at least one Chinese character in a preset table. Specifically, this position information may be the index of the at least one Chinese character in the preset table. For example, Token1 represents the index of "白" (white) in the preset table, Token2 represents the index of "日" (sun) in the preset table, Token3 represents the index of "依" (rely on) in the preset table, Token4 represents the index of "山" (mountain) in the preset table, and Token5 represents the index of "尽" (set) in the preset table. This preset table may be the preset table required when the server 21 converts "The sun along the mountain bows" into Token1, Token2, …, Token5. This preset table includes multiple text units. Specifically, the text units included in this preset table are not limited to the text units included in the original text. For example, the number of text units included in this preset table may be more than the number of text units included in the original text, and the text units included in the original text may be a subset of the text units included in this preset table. Optionally, the predicted result corresponding to the first label is the predicted result for the next set of text units of the at least one text unit, and the actual result corresponding to the first label is the next set of text units.

[0054] As Figure 3As shown, each of Token1, Token2, …, TokenN corresponds to a prediction result. Any Token corresponds to at least one Chinese character, and the prediction result corresponding to any Token is the prediction result for the next set of text units of the at least one Chinese character. The next set of text units may include at least one text unit.

[0055] For example, taking Token1 corresponding to "白", Token2 corresponding to "日", Token3 corresponding to "依", Token4 corresponding to "山", and Token5 corresponding to "尽" as an example, the server 21 inputs Token1, Token2, …, Token5 into the large language model to be trained. According to Figure 3 the process shown, 5 prediction results can be obtained. For example, the prediction result 1 corresponding to Token1 is the prediction result for the next Chinese character of "白", which is "日". Due to the certain inaccuracy of the prediction result, the prediction result 1 may be "日", or it may also be other Chinese characters other than "日".

[0056] Similarly, the prediction result 2 corresponding to Token2 is the prediction result for the next Chinese character of "日", which is "依". Due to the certain inaccuracy of the prediction result, the prediction result 2 may be "依", or it may also be other Chinese characters other than "依".

[0057] In addition, in order to determine whether each prediction result obtained through the large language model to be trained is correct, the actual results corresponding to each first label need to be set in advance. For example, since the prediction result 1 corresponding to Token1 is the prediction result for the next Chinese character of "白", which is "日", the actual result 1 corresponding to Token1 is "日", that is, the actual result 1 corresponding to Token1 is the next Chinese character of "白". The prediction result 2 corresponding to Token2 is the prediction result for the next Chinese character of "日", which is "依", so the actual result 2 corresponding to Token2 is "依", that is, the actual result 2 corresponding to Token2 is the next Chinese character of "日", and so on.

[0058] Optionally, determining multiple probability values of the prediction result corresponding to the first label according to the initial vector corresponding to the first label includes: determining the probability value that the prediction result corresponding to the first label is each text unit in the preset table according to the initial vector corresponding to the first label.

[0059] For example, the preset table includes V text units, and each text unit corresponds to an index in the preset table. As Figure 3As shown, the softmax function is used to normalize each of logits1, logits2, ..., logitsN, resulting in multiple normalized vectors, such as prob1, prob2, ..., probN. Here, logits1 corresponds to prob1, logits2 corresponds to prob2, and so on, with logitsN corresponding to probN. prob1 includes multiple probability values ​​for prediction result 1 corresponding to Token1, prob2 includes multiple probability values ​​for prediction result 2 corresponding to Token2, and so on, with probN including multiple probability values ​​for prediction result N corresponding to TokenN. The V elements in the normalized vector corresponding to each first label represent the probability values ​​of the prediction results for each text unit in the preset table. For example, the first element in prob1 represents the probability value that prediction result 1 is the first text unit in the preset table, the second element in prob1 represents the probability value that prediction result 1 is the second text unit in the preset table, and so on, with the Vth element in prob1 representing the probability value that prediction result 1 is the Vth text unit in the preset table.

[0060] Optionally, the dimension of the initial vector corresponding to the first label is V, where V represents the number of text units included in the preset table; based on the initial vector corresponding to the first label, the prediction result corresponding to the first label is determined to be the probability value of each text unit in the preset table, including: normalizing the initial vector corresponding to the first label to obtain a normalized vector, where the i-th element in the normalized vector represents the probability value of the i-th text unit in the preset table, where 1≤i≤V.

[0061] For example, the dimension of each initial vector in logits1, logits2, ..., logitsN is V, and the dimension of each normalized vector in prob1, prob2, ..., probN is V, where V is the number of text units included in the preset table.

[0062] The following uses logits1 and prob1 as examples to illustrate the transformation process from logits1 to prob1. Specifically, logits1 is a 1*V vector, and prob1 is also a 1*V vector. The V elements in logits1 correspond one-to-one with the V elements in prob1. For example, the i-th element in logits1 is denoted as x. i , 1≤i≤V, the i-th element in prob1 is denoted as softmax(x i ), softmax(x) i It can be calculated using the following formula (1):

[0063]

[0064] Where, x j This represents the j-th element in logits1, where j is used to iterate through each element in logits1. ∑ j exp(x j The expression () indicates that each element in logits1 is processed by the exponential function exp and then summed. In other words, by using formula (1) as described above, logits1, logits2, ..., logitsN can be normalized respectively, so that each element in any normalized vector is a probability value between 0 and 1, and the sum of the V elements in any normalized vector is 1. Furthermore, the V elements in the normalized vector corresponding to each first label represent the probability values ​​of the prediction results corresponding to that first label for each text unit in the preset table. For example, the first element in prob1 represents the probability value that prediction result 1 is the first text unit in the preset table, the second element in prob1 represents the probability value that prediction result 1 is the second text unit in the preset table, and so on, with the Vth element in prob1 representing the probability value that prediction result 1 is the Vth text unit in the preset table.

[0065] Optionally, based on the plurality of initial vectors and the actual results corresponding to the plurality of first labels, a plurality of loss values ​​are calculated, including: for each of the plurality of first labels, calculating the cross-entropy loss value between the initial vector corresponding to the first label and the vector representation of the actual result corresponding to the first label, wherein the cross-entropy loss value is an unreduced loss value, the j-th element in the vector representation is 1 and the remaining elements are 0, and the actual result is the j-th text unit in the preset table.

[0066] For example, Table 1 below shows a schematic diagram of the structure of a preset table, which includes 6 text units and an index for each text unit. Since this preset table includes 6 text units, V, as mentioned above, is equal to 6 here.

[0067] Table 1

[0068] index Text unit 1 Mountain 2 Exhausted 3 white 4 north 5 day 6 according to

[0069] In this embodiment, as Figure 3The specific Loss1 shown is the cross - entropy loss value between the vector representations of logits1 and the actual result1. The specific Loss2 is the cross - entropy loss value between the vector representations of logits2 and the actual result2. The specific LossN is the cross - entropy loss value between the vector representations of logitsN and the actual resultN. Since the dimension of each initial vector in logits1, logits2, …, logitsN is V, the dimension of the vector representation of each actual result in the actual result1, actual result2, …, actual resultN is also V.

[0070] For example, the actual result1 corresponding to Token1 is “日”. The index of “日” in the preset table is 5, that is, “日” is the 5th text unit in the preset table. Therefore, the vector representation of the actual result1 is denoted as [0, 0, 0, 0, 1, 0], that is, the 5th element in the vector representation of the actual result1 is 1, and the remaining elements are 0. Similarly, the vector representations corresponding to the actual result2, …, actual resultN can be determined.

[0071] For example, the loss value Loss1 can be calculated based on the vector representations of logits1 and the actual result1. Loss1 can be the cross - entropy loss value between the vector representations of logits1 and the actual result1. The loss value Loss2 can be calculated based on the vector representations of logits2 and the actual result2. Loss2 can be the cross - entropy loss value between the vector representations of logits2 and the actual result2. And so on, the loss value LossN can be calculated based on the vector representations of logitsN and the actual resultN. LossN can be the cross - entropy loss value between the vector representations of logitsN and the actual resultN. These cross - entropy loss values are all unreduced loss values. The calculation process of these cross - entropy loss values can be represented by the following formula (2):

[0072] L red_none = LossFunction(y pred , y true , reduction = 'none') (2)

[0073] where, y pred represents each initial vector output by the large - language model, and y true represents the vector representation of each actual result. For example, y pred includes logits1, logits2, …, logitsN. y trueThis includes the vector representations of actual result 1, actual result 2, ..., actual result N. `reduction` represents the reduction operator. In formula (2), the `reduction` parameter of the loss value is set to `none` to preserve the original form of the loss value and avoid immediate reduction. L red_none This represents the array of unreduced loss values, i.e., L red_none It includes N elements, which are Loss1, Loss2, ..., LossN as described above, and retains the original form of each of Loss1, Loss2, ..., LossN. No reduction operations are performed on Loss1, Loss2, ..., LossN, such as merging calculations or weighted summations. LossFunction represents the cross-entropy loss function, which is a commonly used cross-entropy loss function in this field.

[0074] Optionally, performing multinomial sampling on the plurality of probability values ​​to obtain the prediction result corresponding to the first label includes: performing multinomial sampling on the plurality of probability values ​​to obtain a second label, wherein the second label represents the position information of the prediction result corresponding to the first label in a preset table; and obtaining the prediction result corresponding to the first label from the preset table based on the second label.

[0075] For example, prob1 includes V probability values. Multinomial sampling is performed on these V probability values ​​to obtain the second label, predictToken1. predictToken1 includes the position information of prediction result 1 in a preset table, i.e., its index. For example, prediction result 1 is "day," and "day" is the fifth text unit in the preset table shown in Table 1, meaning the index of "day" in Table 1 is 5. Therefore, predictToken1 = 5. Furthermore, based on predictToken1, "day" can be retrieved from the preset table and used as prediction result 1. Similarly, prediction result 2, ..., prediction result N can be determined.

[0076] Optionally, calculating a target loss value based on the remaining loss values ​​includes: calculating the average of the remaining loss values ​​and using the average as the target loss value.

[0077] For example, after removing some loss values ​​from Loss1, Loss2, ..., LossN, the set of remaining loss values ​​is denoted as L. consider For the remaining loss values, this embodiment applies a reduction operation of reduction = mean to calculate their average value, which is used as the target loss value, i.e., the final loss value L. final The final loss value can be represented by the following formula (3):

[0078]

[0079] Where m represents the number of loss values in L, that is, after removing some loss values from Loss1, Loss2, … LossN, the number of remaining loss values. consider l represents the j-th loss value in L, where 1 ≤ j ≤ m. Further, the parameters of the large language model to be trained are updated according to the final loss value. j represents L consider in.

[0080] In the embodiment of the present disclosure, by removing some loss values, the number of loss values is reduced, thereby improving the calculation efficiency of the final loss value and the training efficiency. In this embodiment, by improving the calculation method of the final loss value, the text frequency bias problem is effectively solved, thereby improving the training efficiency and the final performance of the large language model. In addition, the model training method described in this embodiment is simple and easy to implement, and is easy to be implemented in the existing large language model training framework, for example, it can be applied to the pre-training and fine-tuning stages of the large language model.

[0081] It can be understood that the training of the large language model with the original text "The sun along the mountain bows" as described above can be recorded as an iterative training. Subsequently, the original text can be continuously changed, and the large language model can be continuously iteratively trained in a similar process as above until the parameters of the large language model tend to converge, or the number of iterations is greater than or equal to the preset number. After the training of the large language model is completed, the trained large language model can be retained in the server 21, or can be deployed in the terminal 22 or other servers. The trained large language model can be applied to the scenarios where the large language model is usually applicable, such as intelligent question answering.

[0082] Figure 4 is a schematic structural diagram of the model training device provided by the embodiment of the present disclosure. The model training device provided by the embodiment of the present disclosure can execute the processing flow provided by the method embodiment. As Figure 4 shown, the model training device 40 includes: an input module 41, a first calculation module 42, a determination module 43, a polynomial sampling module 44, a removal module 45, a second calculation module 46, and an update module 47.

[0083] The input module 41 is configured to input a plurality of first tags related to the original text into the large language model to be trained, so that the large language model to be trained outputs a plurality of initial vectors, and the plurality of first tags and the plurality of initial vectors correspond one-to-one;

[0084] The first calculation module 42 is configured to calculate a plurality of loss values according to the plurality of initial vectors and the actual results respectively corresponding to the plurality of first tags;

[0085] The determining module 43 is used to determine multiple probability values ​​of the prediction result corresponding to the first label for each of the plurality of first labels, based on the initial vector corresponding to the first label;

[0086] The multinomial sampling module 44 is used to perform multinomial sampling on the multiple probability values ​​to obtain the prediction result corresponding to the first label;

[0087] The removal module 45 is used to remove the loss value calculated based on the actual result corresponding to the first label and the initial vector corresponding to the first label from the plurality of loss values ​​when the prediction result corresponding to the first label matches the actual result corresponding to the first label, so as to obtain the remaining loss value.

[0088] The second calculation module 46 is used to calculate the target loss value based on the remaining loss value;

[0089] The update module 47 is used to update the parameters of the large language model to be trained based on the target loss value.

[0090] Optionally, when determining multiple probability values ​​of the prediction result corresponding to the first label based on the initial vector corresponding to the first label, the determining module 43 is specifically used to: determine the probability value of each text unit in the preset table based on the initial vector corresponding to the first label.

[0091] Optionally, the dimension of the initial vector corresponding to the first label is V, where V represents the number of text units included in the preset table; when the determining module 43 determines that the prediction result corresponding to the first label is the probability value of each text unit in the preset table based on the initial vector corresponding to the first label, it is specifically used to: normalize the initial vector corresponding to the first label to obtain a normalized vector, wherein the i-th element in the normalized vector represents the probability value of the i-th text unit in the preset table, where 1≤i≤V, the prediction result corresponding to the first label is the probability value of the i-th text unit in the preset table.

[0092] Optionally, when the first calculation module 42 calculates multiple loss values ​​based on the multiple initial vectors and the actual results corresponding to the multiple first labels, it is specifically used to: calculate the cross-entropy loss value between the vector representation of the initial vector corresponding to the first label and the vector representation of the actual result corresponding to the first label for each of the multiple first labels, wherein the cross-entropy loss value is an unreduced loss value, the j-th element in the vector representation is 1 and the remaining elements are 0, and the actual result is the j-th text unit in the preset table.

[0093] Optionally, when the multinomial sampling module 44 performs multinomial sampling on the plurality of probability values ​​to obtain the prediction result corresponding to the first label, it is specifically used to: perform multinomial sampling on the plurality of probability values ​​to obtain a second label, wherein the second label represents the position information of the prediction result corresponding to the first label in a preset table; and obtain the prediction result corresponding to the first label from the preset table according to the second label.

[0094] Optionally, the first label corresponds to at least one text unit in the original text, and the first label indicates the position information of the at least one text unit in a preset table.

[0095] Optionally, the prediction result corresponding to the first label is the prediction result for the next group of text units for the at least one text unit, and the actual result corresponding to the first label is the next group of text units.

[0096] Optionally, when the second calculation module 44 calculates the target loss value based on the remaining loss value, it is specifically used to: calculate the average value of the remaining loss value and use the average value as the target loss value.

[0097] Figure 4 The model training apparatus of the illustrated embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0098] Exemplary embodiments of this disclosure also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of this disclosure.

[0099] Exemplary embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to embodiments of this disclosure.

[0100] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein, when executed by a processor of a computer, the computer program is used to cause the computer to perform a method according to an embodiment of this disclosure.

[0101] refer to Figure 5The present invention describes a structural block diagram of an electronic device 50 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0102] like Figure 5 As shown, the electronic device 50 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the device 50. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0103] Multiple components in electronic device 50 are connected to I / O interface 505, including: input unit 506, output unit 507, storage unit 508, and communication unit 509. Input unit 506 can be any type of device capable of inputting information to electronic device 50. Input unit 506 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 507 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 504 may include, but is not limited to, disks and optical discs. Communication unit 509 allows electronic device 50 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0104] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the methods described in the above method embodiments can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 50 via ROM 502 and / or communication unit 509. In some embodiments, the computing unit 501 can be configured to perform the methods described in the above method embodiments by any other suitable means (e.g., by means of firmware).

[0105] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0106] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0107] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0109] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0110] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A model training method, the method comprising: inputting a plurality of first labels related to an original text into a large language model to be trained, so that the large language model to be trained outputs a plurality of initial vectors, the plurality of first labels and the plurality of initial vectors corresponding one-to-one, the first label corresponding to at least one text unit in the original text, the first label representing position information of the at least one text unit in a preset table, the preset table comprising a plurality of text units and position information of each text unit; calculating a plurality of loss values according to the plurality of initial vectors and actual results corresponding to the plurality of first labels respectively; for each first label in the plurality of first labels, determining a plurality of probability values of a predicted result corresponding to the first label according to an initial vector corresponding to the first label, and performing multinomial sampling on the plurality of probability values to obtain the predicted result corresponding to the first label; if the predicted result corresponding to the first label matches an actual result corresponding to the first label, removing a loss value calculated according to the actual result corresponding to the first label and the initial vector corresponding to the first label from the plurality of loss values to obtain remaining loss values; calculating a target loss value according to the remaining loss values, and updating parameters of the large language model to be trained according to the target loss value; wherein the determining of the plurality of probability values of the predicted result corresponding to the first label according to the initial vector corresponding to the first label comprises: determining that the predicted result corresponding to the first label is a probability value of each text unit in the preset table according to the initial vector corresponding to the first label; the calculating of the plurality of loss values according to the plurality of initial vectors and the actual results corresponding to the plurality of first labels respectively comprises: for each first label in the plurality of first labels, calculating a cross-entropy loss value between a vector representation of the initial vector corresponding to the first label and the actual result corresponding to the first label, the cross-entropy loss value being an unreduced loss value, a jth element in the vector representation being 1 and the remaining elements being 0, the actual result being a jth text unit in the preset table.

2. The method of claim 1, wherein, a dimension of the initial vector corresponding to the first label is V, V representing a number of text units included in the preset table; the determining of the predicted result corresponding to the first label being the probability value of each text unit in the preset table according to the initial vector corresponding to the first label comprises: normalizing the initial vector corresponding to the first label to obtain a normalized vector, an i-th element in the normalized vector representing a probability value that the prediction result corresponding to the first label is an i-th text unit in the preset table, i V.

3. The method of claim 1, wherein, the multinomial sampling of the plurality of probability values to obtain the predicted result corresponding to the first label comprises: performing multinomial sampling on the plurality of probability values to obtain a second label, the second label representing position information of the predicted result corresponding to the first label in the preset table; obtaining the predicted result corresponding to the first label from the preset table according to the second label.

4. The method of claim 1, wherein, the predicted result corresponding to the first label is a predicted result of a next group of text units for the at least one text unit, and the actual result corresponding to the first label is the next group of text units.

5. The method of claim 1, wherein, According to the remaining loss values, a target loss value is calculated, comprising: Calculating the average value of the remaining loss values, and taking the average value as the target loss value.

6. A model training apparatus, the apparatus comprising: An input module configured to input a plurality of first labels associated with an original text into a large language model to be trained, so that the large language model to be trained outputs a plurality of initial vectors, the plurality of first labels and the plurality of initial vectors corresponding one-to-one, the first label corresponding to at least one text unit in the original text, the first label representing position information of the at least one text unit in a preset table, the preset table comprising a plurality of text units and position information of each text unit; A first calculation module configured to calculate a plurality of loss values according to the plurality of initial vectors and actual results corresponding to the plurality of first labels respectively; A determination module configured to determine, for each first label in the plurality of first labels, a plurality of probability values of a predicted result corresponding to the first label according to an initial vector corresponding to the first label; A multinomial sampling module configured to perform multinomial sampling on the plurality of probability values to obtain the predicted result corresponding to the first label; A removal module configured to remove, when the predicted result corresponding to the first label matches an actual result corresponding to the first label, a loss value calculated according to the actual result corresponding to the first label and the initial vector corresponding to the first label from the plurality of loss values to obtain remaining loss values; A second calculation module configured to calculate a target loss value according to the remaining loss values; An update module configured to update parameters of the large language model to be trained according to the target loss value; Wherein, the determination of the plurality of probability values of the predicted result corresponding to the first label according to the initial vector corresponding to the first label comprises: Determining, according to the initial vector corresponding to the first label, that the predicted result corresponding to the first label is a probability value of each text unit in a preset table; The calculation of the plurality of loss values according to the plurality of initial vectors and the actual results corresponding to the plurality of first labels respectively comprises: For each first label in the plurality of first labels, calculating a cross-entropy loss value between a vector representation of the initial vector corresponding to the first label and the actual result corresponding to the first label, the cross-entropy loss value being an unreduced loss value, a jth element in the vector representation being 1 and the remaining elements being 0, the actual result being a jth text unit in a preset table.

7. An electronic device comprising: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the method of any one of claims 1-5. The computer program, when executed by a processor, implements the method of any one of claims 1-5.

8. A computer program product comprising a computer program, wherein, ​

Images