Large language model training method and device, computer equipment and storage medium

Abstract

The embodiment of the invention discloses a big language model training method and device, computer equipment and a storage medium, and belongs to the technical field of computers. According to the method, a first abstract text and a second abstract text are introduced in the training process of a large language model, more learnable information is provided for model training, meanwhile, due to the fact that the first abstract text and the second abstract text contain different information amounts, and the first abstract text contains correct statements and error statements, the learnable information is more learnable. Compared with the prior art, the two different abstract texts of the same sample text are subjected to comparative learning, and correct statements and wrong statements in the first abstract text are learned in a differentiated manner, so that the problems of model overfitting, inaccurate generation and the like caused by single abstract text are avoided, the generalization performance of the model is improved, and the accuracy of the model is also improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a training method, apparatus, computer equipment and storage medium for a large language model. Background Art

[0002] The text summarization task is a basic task in the field of natural language processing. Given a text, a summary text is extracted from the text through a large language model. The summary text is a general description of the text, summarizing the content of the text in a smaller space. Generally, a large language model can summarize many types of texts. For example, news texts, script texts, and meeting records.

[0003] At present, the model parameters of the large language model are usually updated according to the difference between the predicted summary text output by the large language model and the summary text used as a reference, so that the predicted summary text output by the large language model gradually approaches the summary text used as a reference. However, since the summary text used as a reference may contain erroneous sentences, the accuracy of the large language model obtained by the above training method is relatively low. Summary of the invention

[0004] The embodiment of the present application provides a training method, device, computer equipment and storage medium for a large language model, which compares and studies two different summary texts of the same sample text and distinguishes correct sentences and incorrect sentences in the first summary text, thereby avoiding problems such as model overfitting and inaccurate generation due to a single summary text, thereby improving not only the generalization performance of the model but also the accuracy of the model. The technical solution is as follows:

[0005] In one aspect, a method for training a large language model is provided, the method comprising:

[0006] Inputting the sample text into the large language model to obtain prediction summary information, wherein the prediction summary information includes the probability of each of the plurality of characters at each character position in the prediction summary text of the sample text;

[0007] Obtaining a first summary text of the sample text and a second summary text of the sample text, wherein the first summary text contains more information than the second summary text, and the first summary text includes a correct sentence and an incorrect sentence, wherein the correct sentence is a sentence that correctly reflects the sample text, and the incorrect sentence is a sentence that incorrectly reflects the sample text;

[0008] Determine a loss function value based on the predicted summary information, correct sentences in the first summary text, incorrect sentences in the first summary text, the second summary text, and predicted latent vectors corresponding to the multiple characters;

[0009] The large language model is trained based on the loss function value.

[0010] On the other hand, a large language model training device is provided, the device comprising:

[0011] A prediction module, used for inputting a sample text into a large language model to obtain prediction summary information, wherein the prediction summary information includes the probability of each of a plurality of characters at each character position in the prediction summary text of the sample text;

[0012] A first acquisition module is used to acquire a first summary text of the sample text and a second summary text of the sample text, wherein the first summary text contains more information than the second summary text, and the first summary text includes correct sentences and incorrect sentences, wherein the correct sentences are sentences that correctly reflect the sample text, and the incorrect sentences are sentences that incorrectly reflect the sample text;

[0013] A first determination module, configured to determine a loss function value based on the predicted summary information, correct sentences in the first summary text, incorrect sentences in the first summary text, the second summary text, and predicted latent vectors corresponding to the multiple characters;

[0014] A training module is used to train the large language model based on the loss function value.

[0015] In a possible implementation, the first determination module is used to determine an average loss function value of correct sentences in the first summary text and an average loss function value of incorrect sentences in the first summary text based on the predicted summary information, correct sentences in the first summary text, and incorrect sentences in the first summary text; obtain a first loss function value based on the average loss function value of correct sentences in the first summary text and the average loss function value of incorrect sentences in the first summary text, the first loss function value being a weighted sum of the average loss function value of correct sentences in the first summary text and the average loss function value of incorrect sentences in the first summary text; determine a second loss function value based on the first summary text, the second summary text, and the predicted latent vectors corresponding to the multiple characters, the second loss function value being used to indicate the similarity between the predicted latent vector of the first summary text and the predicted latent vector of the second summary text.

[0016] In a possible implementation, the first determination module is further used to determine a loss function value of each sentence in the first summary text based on the predicted summary information, the loss function value of any sentence being the average of cross entropy losses of multiple characters in the sentence; based on the loss function value of each sentence in the first summary text, determine an average loss function value of correct sentences in the first summary text and an average loss function value of incorrect sentences in the first summary text, the average loss function value of correct sentences in the first summary text being the average of loss function values ​​of multiple correct sentences in the first summary text, and the average loss function value of incorrect sentences in the first summary text being the average of loss function values ​​of multiple incorrect sentences in the first summary text.

[0017] In a possible implementation manner, the first summary text further includes an inaccurate sentence, where the inaccurate sentence is a sentence that inaccurately reflects the sample text.

[0018] In a possible implementation, the first determination module is configured to determine a first intermediate ratio based on the loss function value of each sentence in a first summary text of multiple sample texts in a current training batch, where the first intermediate ratio is the ratio of the sum of the loss function values ​​of multiple correct sentences in the first summary text of the multiple sample texts to the sum of the loss function values ​​of multiple sentences in the first summary text of the multiple sample texts; determine a weight coefficient of a correct sentence in the first summary text based on the first intermediate ratio, where the weight coefficient of the correct sentence in the first summary text is a preset power of the first intermediate ratio; and determine a weight coefficient of an incorrect sentence in the first summary text based on the weight coefficient of the correct sentence in the first summary text, where the weight coefficient of the incorrect sentence in the first summary text is the difference between the weight coefficient of the correct sentence in the first summary text and 1.

[0019] In a possible implementation, the first determination module is further configured to determine a second intermediate ratio and a third intermediate ratio based on the loss function value of each sentence in the first summary text of the plurality of sample texts in the current training batch, wherein the second intermediate ratio is a ratio of the sum of the loss function values ​​of the plurality of correct sentences in the first summary text of the plurality of sample texts to the sum of the loss function values ​​of the plurality of sentences in the first summary text of the plurality of sample texts, and the third intermediate ratio is a ratio of the sum of the loss function values ​​of the plurality of inaccurate sentences in the first summary text of the plurality of sample texts to the sum of the loss function values ​​of the plurality of sentences in the first summary text of the plurality of sample texts; based on the second intermediate ratio and the third intermediate ratio , determining weight coefficients of correct sentences in the first summary text and weight coefficients of inaccurate sentences in the first summary text, the weight coefficients of correct sentences in the first summary text being a preset power of the second intermediate ratio, and the weight coefficients of inaccurate sentences in the first summary text being a preset power of the third intermediate ratio; based on the weight coefficients of correct sentences in the first summary text and the weight coefficients of inaccurate sentences in the first summary text, determining weight coefficients of erroneous sentences in the first summary text, the weight coefficients of erroneous sentences in the first summary text being a difference between the sum of the weight coefficients of correct sentences in the first summary text and the weight coefficients of inaccurate sentences in the first summary text and 1.

[0020] In a possible implementation, the first determination module is further used to determine a predicted latent vector of the first summary text and a predicted latent vector of the second summary text based on the predicted latent vectors corresponding to the multiple characters, multiple characters in the first summary text, and multiple characters in the second summary text; pool the predicted latent vector of the first summary text and the predicted latent vector of the second summary text respectively to obtain a first intermediate vector and a second intermediate vector, and the dimensions of the first intermediate vector and the second intermediate vector are the same; based on the first intermediate vector and the second intermediate vector, determine the second loss function value, and the second loss function value is the distance between the first intermediate vector and the second intermediate vector.

[0021] In a possible implementation, the training module is used to perform a weighted summation of the first loss function value and the second loss function value to obtain a total loss function value; and train the large language model based on the total loss function value.

[0022] In one possible implementation, the first acquisition module is used to input the sample text and the first prompt text into the first additional model to obtain the first summary text, wherein the first prompt text is used to instruct the first additional model to generate a summary text containing supplementary information based on the sample text; and input the sample text and the second prompt text into the first additional model to obtain the second summary text, wherein the second prompt text is used to instruct the first additional model to generate a summary text based on the sample text.

[0023] In one possible implementation, the first acquisition module is further used to input the first summary text into the second additional model to obtain a label for each sentence in the first summary text, where the label of each sentence is used to indicate the accuracy of the sentence in reflecting the sample text; and input the second summary text into the second additional model to obtain a label for each sentence in the second summary text.

[0024] In a possible implementation, the device further includes:

[0025] a second determination module, configured to determine a third loss function value based on the predicted summary information, the first summary text, the second summary text, and reference summary information, wherein the reference summary information is output information of a reference language model having the same structure as the large language model, and the third loss function value is used to indicate a capability gap between the large language model and the reference language model in distinguishing the first summary text from the second summary text;

[0026] The training module is further used to train the large language model based on the loss function value and the third loss function value.

[0027] In a possible implementation, the second determination module is configured to determine a first probability group and a second probability group based on the predicted summary information, the first summary text, the second summary text, and the reference summary information, the first probability group including a first probability and a second probability, the first probability being the probability of the first summary text predicted by the large language model, the second probability being the probability of the second summary text predicted by the large language model, and the second probability group including a third probability and a fourth probability, the third probability being the probability of the first summary text predicted by the reference language model, and the fourth probability being the probability of the second summary text predicted by the reference language model; based on the first probability group and the second probability group, determine a first intermediate difference and a second intermediate difference, the first intermediate difference being the difference between the logarithm of the first probability and the logarithm of the second probability, and the second intermediate difference being the difference between the logarithm of the third probability and the logarithm of the fourth probability; based on the difference between the first intermediate difference and the second intermediate difference, determine the third loss function value, the third loss function value being the negative logarithm of the difference.

[0028] In a possible implementation, the second determination module is further configured to obtain, for any summary text of the first summary text and the second summary text, the probability of a character at any character position in the summary text in the predicted summary information; determine, based on the probability corresponding to the character position and the weight coefficient of the character position, an intermediate product value corresponding to the character position, the intermediate product value corresponding to the character position being the product value of the probability corresponding to the character position and the weight coefficient of the character position; determine, based on the intermediate product values ​​corresponding to multiple character positions in the summary text, the probability of the summary text predicted by the large language model, the probability of the summary text predicted by the large language model being the sum of the intermediate product values ​​corresponding to multiple character positions in the summary text.

[0029] In a possible implementation, the second determination module is further configured to obtain, for any summary text of the first summary text and the second summary text, a probability corresponding to a character at any character position in the summary text in the reference summary information; and determine, based on the probabilities corresponding to multiple character positions in the summary text, a probability of the summary text predicted by the reference language model, the probability of the summary text predicted by the reference language model being the sum of the probabilities corresponding to multiple character positions in the summary text.

[0030] In one possible implementation, the second determination module is further used to obtain a matching parameter corresponding to the character position, the matching parameter corresponding to the character position being used to indicate a degree of matching between the character at the character position in the summary text and another summary text in the first summary text and the second summary text; and determine a weight coefficient of the character position based on a difference between the matching parameter corresponding to the character position and 1, the weight coefficient of the character position being a target power of the difference.

[0031] In one possible implementation, the second determination module is further used to determine the weight coefficient of the character position based on the label associated with the character position, the weight coefficient of the character position being a preset weight coefficient corresponding to the label of the character position, the label being a first label or a second label, and the preset weight coefficient corresponding to the first label being greater than the preset weight coefficient corresponding to the second label.

[0032] In a possible implementation, the second determination module is further used to obtain a matching parameter corresponding to the character position, where the matching parameter corresponding to the character position is used to indicate a degree of matching between the character at the character position in the summary text and the other summary text in the first summary text and the second summary text; when the matching parameter corresponding to the character position is less than a matching parameter threshold, determining that the label associated with the character position is the first label; and when the matching parameter corresponding to the character position is not less than the matching parameter threshold, determining that the label associated with the character position is the second label.

[0033] In a possible implementation, the second determination module is further used to sort the matching parameters corresponding to multiple character positions in the summary text in descending order of the matching parameters; based on the matching parameters at each ranking after sorting, determine the matching parameter threshold, and the matching parameter threshold is the matching parameter at a preset ranking after sorting.

[0034] In a possible implementation, the second determination module is further configured to determine, for the sentence at the character position in the summary text, a matching sentence of the sentence, in the other summary text, wherein the similarity between the matching sentence and the sentence is greater than the similarity between other sentences in the other summary text and the sentence; and to determine a matching parameter corresponding to the character position based on a matching character in the matching sentence and the character at the character position, wherein the similarity between the matching character and the character at the character position is greater than the similarity between other characters in the matching sentence and the character at the character position, and the matching parameter corresponding to the character position is the similarity between the character at the character position and the matching character.

[0035] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the large language model training method as described in the above aspects.

[0036] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the large language model training method as described in the above aspects.

[0037] On the other hand, a computer program product is provided, including a computer program, wherein the computer program is loaded and executed by a processor to implement the large language model training method as described in the above aspects.

[0038] An embodiment of the present application provides a training method for a large language model. In the method, a first summary text and a second summary text are introduced in the training process of the large language model, so that more learnable information is provided for model training. At the same time, since the first summary text and the second summary text contain different amounts of information, and the first summary text contains correct sentences and incorrect sentences, by comparing and learning two different summary texts of the same sample text, and distinguishing and learning the correct sentences and incorrect sentences in the first summary text, problems such as model overfitting and inaccurate generation due to a single summary text are avoided, thereby improving not only the generalization performance of the model, but also the accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0041] Figure 2 It is a flowchart of a large language model training method provided in an embodiment of the present application;

[0042] Figure 3 is a flowchart of another large language model training method provided in an embodiment of the present application;

[0043] Figure 4 is a schematic diagram of determining a loss function value provided by an embodiment of the present application;

[0044] Figure 5 This is an architecture diagram for training a large language model provided in an embodiment of the present application;

[0045] Figure 6 It is another architecture diagram for training a large language model provided in an embodiment of the present application;

[0046] Figure 7 is another schematic diagram for determining the loss function value provided in an embodiment of the present application;

[0047] Figure 8 It is a flowchart of another large language model training method provided in an embodiment of the present application;

[0048] Fig. 9 It is another schematic diagram of determining the loss function value provided in an embodiment of the present application;

[0049] Fig.10 This is another architecture diagram for training a large language model provided in an embodiment of the present application;

[0050] Fig.11 It is a schematic diagram of a model application scenario provided in an embodiment of the present application;

[0051] Fig.12 It is a structural schematic diagram of a large language model training device provided in an embodiment of the present application;

[0052] Fig.13 It is a structural schematic diagram of another large language model training device provided in an embodiment of the present application;

[0053] Fig.14 is a schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0054] Fig.15 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0056] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with basically the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on quantity and execution order.

[0057] In the present application, the term "at least one" means one or more, "plurality" means two or more, "each" means each of the corresponding plurality, and "any" means any one of the plurality.

[0058] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the sample text and summary text involved in this application are all obtained with full authorization.

[0059] Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. Figure 1 The implementation environment includes: a terminal 101 and a server 102. The terminal 101 and the server 102 can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0060] In one possible implementation, the terminal 101 is a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart watch, an intelligent voice interaction device, a smart home appliance, etc., but is not limited thereto. The terminal 101 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, and other names.

[0061] In one possible implementation, server 102 is an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0062] In one possible implementation, the server 102 undertakes the main computing work and the terminal 101 undertakes the secondary computing work; or, the server 102 undertakes the secondary computing work and the terminal 101 undertakes the main computing work; or, the server 102 and the terminal 101 adopt a distributed computing architecture to perform collaborative computing.

[0063] Those skilled in the art can appreciate that the computer program involved in the present application can be deployed and executed on one computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed in multiple locations and interconnected by a communication network, without limitation.

[0064] The training method of the large language model provided in the embodiment of the present application is performed by a computer device. Optionally, the computer device is a terminal 101 or a server 102. Take the computer device as a server 102 as an example for explanation. The terminal 101 is used to obtain a training data set. The terminal 101 sends a training data set to the server 102 through a network connection with the server 102. The server 102 receives and stores the training data set sent by the terminal 101, and trains the large language model based on the training data set. When the large language model training is completed, the large language model is deployed in the server 102. The terminal 101 obtains a given text, and sends the text to the server 102 through a network connection with the server 102. The server 102 predicts the received text through the large language model to obtain a corresponding summary text. The server 102 sends the summary text to the terminal through a network connection with the terminal 101. The terminal 101 displays the summary text for the user to view.

[0065] It should be noted that the above implementation environment is only an example, and the method provided in the embodiments of the present application can also be implemented in other environments, and the embodiments of the present application are not limited to this.

[0066] Figure 2 is a flow chart of a large language model training method provided in an embodiment of the present application, such as Figure 2 As shown, the embodiment of the present application is executed by a computer device. The method includes the following steps:

[0067] 201. The computer device inputs the sample text into the large language model to obtain prediction summary information, where the prediction summary information includes the probability of each character in a plurality of characters at each character position in the prediction summary text of the sample text.

[0068] In the embodiment of the present application, when the large language model generates the predicted summary information of the sample text, the large language model can also generate the predicted summary text of the sample text. Both the predicted summary information and the predicted summary text can reflect the description effect of the large language model on the sample text.

[0069] The embodiment of the present application does not limit the model structure of the large language model. The large language model is any network model that can perform text summarization tasks. For example, the large language model is an open source language model such as LLaMA (Large Language Model Meta AI, a large language model).

[0070] The embodiment of the present application does not limit the form of the prediction summary information. For example, the prediction summary information is in the form of a matrix, the rows of the matrix correspond to the positions of the characters, and the columns of the matrix correspond to the multiple characters. At this time, an element in the matrix is ​​used to represent the probability of the character indicated by the column where the element is located appearing at the character position indicated by the row where the element is located. Among them, the multiple characters are characters in the dictionary of the large language model, and the dictionary of the large language model contains as many characters as possible. In the embodiment of the present application, the characters contained in any text belong to the dictionary of the large language model, which is convenient for parsing the given text and generating a summary text with good description effect.

[0071] The sample text is the text given to the large language model during the training process. The sample text is any type of text. For example, the sample text is a script, which consists of sentences indicating the time, place, characters, and dialogues between characters; or the sample text is a press release, which includes sentences indicating the background of the event, the development of the event, and the outcome of the event; or the sample text is an academic paper, meeting minutes, etc., which will not be repeated here.

[0072] The predicted summary text is a text that gives a general description of the sample text. Each character position of the predicted summary text is used to carry each character contained in the predicted summary text, that is, the predicted summary text can be obtained by filling the corresponding characters at each character position of the predicted summary text. The number of character positions in the predicted summary text is equal to the number of characters contained in the predicted summary text. It should be noted that the number of character positions in the predicted summary text can be a preset number or a non-fixed number. For example, the sample text and the prompt text are input into the large language model, and the prompt text is used to instruct the large language model to generate a predicted summary text containing a preset number of characters; alternatively, the sample text and another prompt text are input into the large language model, and the other prompt text indicates that there is no limit on the number of characters contained in the predicted summary text.

[0073] 202. The computer device obtains a first summary text of the sample text and a second summary text of the sample text, wherein the first summary text contains more information than the second summary text, and the first summary text includes correct sentences and incorrect sentences, wherein the correct sentences are sentences that correctly reflect the sample text, and the incorrect sentences are sentences that incorrectly reflect the sample text.

[0074] In an embodiment of the present application, the same sample text may correspond to multiple summary texts. Different summary texts may differ in terms of the number of characters, semantic coherence, and description accuracy. Generally, the first summary text has a better description effect on the sample text than the second summary text. Therefore, in the subsequent training process of the large language model, the description accuracy of the large language model is improved by making the large language model focus on learning to generate the first summary text based on the sample text and using the second summary text to assist in training.

[0075] Among them, the same summary text includes multiple sentences, and the multiple sentences may differ in the number of characters, vocabulary coherence, and description accuracy. According to the accuracy of the description of the sample text by the sentences in the summary text, the multiple sentences in the summary text can be divided into correct sentences and incorrect sentences. For example, the character relationship, event causal relationship, correspondence between characters and events, etc. described by the correct sentence are consistent with the sample text; the character relationship, event causal relationship, correspondence between characters and events, etc. described by the incorrect sentence are different from the sample text. It should be noted that according to the accuracy of the description of the sample text by the sentences in the summary text, the multiple sentences in the summary text can be divided into more types, which will not be repeated here.

[0076] Optionally, the descriptive effect of the sample text is determined by one or more of the amount of information, the accuracy of description, and the semantic coherence. The amount of information is used to indicate the amount of effective content or details contained in the summary text, that is, the degree of detail of the sample text. For example, the first summary text indicates the time, place, cause, process, and result of the event in the sample text, and the second summary text indicates the cause and result of the event in the sample text. At this time, the amount of information contained in the first summary text is greater than that of the second summary text. The accuracy of description is used to indicate the accuracy of multiple sentences in the summary text in describing the sample text. When the proportion of sentences in the summary text that incorrectly or inaccurately describe the sample text is high, it means that the description accuracy of the summary text is poor; when the proportion of sentences in the summary text that incorrectly or inaccurately describe the sample text is low, it means that the description accuracy of the summary text is good. Semantic coherence is used to indicate whether multiple sentences in the summary text and multiple words in each sentence are coherent.

[0077] Optionally, a first summary text and a second summary text corresponding to the same sample are obtained by manual correction or model generation. For example, when a summary text of a sample text is obtained, the information content of the summary text is increased or decreased or incorrect expressions are corrected by manual correction to obtain another summary text of the sample text; or, for the same sample text, an additional model is used to generate multiple summary texts with different description effects based on the sample text and different prompt texts.

[0078] In the case where there are multiple summary texts corresponding to the same sample, the first summary text and the second summary text may be determined from the multiple summary texts by manual annotation or model processing. For example, the multiple summary texts may be compared manually and the first summary text and the second summary text may be marked respectively; or the sample text, the multiple summary texts and the prompt text may be input into the additional model, and the additional model may be used to compare the description effects of the multiple summary texts, so as to determine the first summary text and the second summary text.

[0079] 203. The computer device determines a loss function value based on the predicted summary information, the correct sentences in the first summary text, the incorrect sentences in the first summary text, the second summary text, and the predicted latent vectors corresponding to the multiple characters.

[0080] In an embodiment of the present application, the predicted latent vectors corresponding to multiple characters are representations in a high-dimensional semantic space obtained by the intermediate layer of a large language model, and the predicted latent vectors corresponding to multiple characters are used to indicate implicit information in multiple characters. The embodiment of the present application does not limit the form of the predicted latent vectors corresponding to multiple characters. For example, the predicted latent vectors corresponding to multiple characters are in the form of a matrix, the rows of the matrix correspond to each semantic dimension, and the columns of the matrix correspond to multiple characters in the dictionary of the large language model. At this time, each column in the matrix is ​​used to represent the predicted latent vector corresponding to the character indicated by the column.

[0081] Since the first summary text and the second summary text have different description effects on the sample text, and the predicted latent vectors corresponding to multiple characters can accurately reflect the difference in the ability of the large language model to predict the two summary texts, the large language model can obtain richer information during the training process, which improves the description accuracy of the large language model. At the same time, since the first summary text includes correct sentences and incorrect sentences, through the distinction and learning of correct sentences and incorrect sentences, the large language model is more inclined to generate sentences that correctly reflect the input text, which improves the accuracy of the model.

[0082] 204. The computer device trains the large language model based on the loss function value.

[0083] In an embodiment of the present application, the computer device calculates the gradient of the model parameters according to the loss function value through the back propagation algorithm, and the gradient of each model parameter indicates the degree of influence of the model parameter on the loss function value. The computer device updates the model parameters according to the gradient to reduce the loss function value. The computer device continuously updates the model parameters during multiple rounds of training to achieve training of a large language model.

[0084] An embodiment of the present application provides a training method for a large language model. In the method, a first summary text and a second summary text are introduced in the training process of the large language model, so that more learnable information is provided for model training. At the same time, since the first summary text and the second summary text contain different amounts of information, and the first summary text contains correct sentences and incorrect sentences, by comparing and learning two different summary texts of the same sample text, and distinguishing and learning the correct sentences and incorrect sentences in the first summary text, problems such as model overfitting and inaccurate generation due to a single summary text are avoided, thereby improving not only the generalization performance of the model, but also the accuracy of the model.

[0085] The above embodiment introduces the process of training a large language model based on the total loss function value corresponding to a single sample text. Based on the above embodiment, the following introduces the process of training a large language model using multiple sample texts.

[0086] First, the overall training process of the large language model is introduced: the model parameters of the initialized large language model; setting the learning parameters of the large language model. When setting the learning parameters, all parameters can be fine-tuned, that is, all parameters of the large language model can be fine-tuned. In the process of training the large language model, multiple training data are obtained. The number of training data is Q, and each bs training data is regarded as a training batch. There are a total of P = Q / bs training batches. Each completion of P training batches indicates the completion of one round of iteration. The large language model is trained through multiple rounds of iterations until the mean of the loss function values ​​of multiple training batches in each round of iteration no longer decreases; or, when the mean of the loss function values ​​of multiple training batches in each round of iteration is less than the threshold, it means that the current large language model is accurate enough, and the training of the large language model is stopped.

[0087] It should be noted that when computing resources are insufficient, LoRA (Low-Rank Adaptation) fine-tuning can be used. That is, by introducing a trainable low-rank matrix in the Transformer layer of the large language model to achieve model fine-tuning. The above method significantly reduces the number of training parameters, thereby reducing the demand for computing resources.

[0088] Secondly, take any training batch in a round of iteration as an example to illustrate: determine the total loss function value corresponding to each sample text in the training batch in the manner of the above embodiment; then determine the sum of the total loss function values ​​corresponding to multiple sample texts in the training batch as the loss function value corresponding to the training batch. Based on the loss function value corresponding to the training batch, determine the model parameter gradient of the large language model. Based on the model parameter gradient, update the model parameters of the large language model.

[0089] The large language model in this application is briefly introduced below.

[0090] An exemplary structure of a large language model is as follows: the large language model is composed of multiple layers of Transformer (a neural network model) stacked together. For example, the main structure of the large language model includes 1 Embedding_tokens (embedded keywords) module, 32 Layers (layer) modules and 1 LM_Head (Language Model Head) module. Among them, the Embedding_tokens module is composed of 1 encoding (Embedding) layer, and the Embedding_tokens module is used to encode the input text; the Layers module is composed of a Decoder Layer (decoding layer), and the Layers module is used to decode the intermediate vectors in the large language model; the LM_Head module is composed of a fully connected layer (fully connected layer), and the LM_Head module is used to predict the probability of multiple characters appearing in each character position to be generated. The Layers module includes an Input_Layer Norm (Input_Layer Normalization) submodule, a Self_Attn (Self Attention) submodule, a Post_Attention_LayerNorm (Post_Attention_Layer Normalization) submodule, and an MLP (Multilayer Perceptron) submodule. The Input_LayerNorm submodule is composed of the RMSNorm (Root Mean Square Layer Normalization) layer, the Self_Attn submodule is composed of the Attention layer, the Post_Attention_LayerNorm submodule is composed of the RMSNorm layer, and the MLP submodule is composed of the MLP layer.

[0091] The processing flow of the large language model is as follows: After the sample text is input into the large language model, the large language model converts the sample text into a high-dimensional semantic space through the encoder to facilitate subsequent processing. In the large language model, for any character position to be generated, the probability of multiple characters appearing at the character position is obtained through the middle module of the large language model and the head module of the large language model, and the sequence number of the character at the character position is determined to be the sequence number of the character with the highest probability. After predicting all the character positions to be generated, the predicted summary text is determined based on the sequence number of the character at each character position; the predicted summary information is determined based on the probability of multiple characters appearing at each character position; the head module of the large language model obtains the predicted latent vectors corresponding to multiple characters.

[0092] An application scenario of the present application is given below. In this application scenario, the trained large language model is used to obtain the summary text of the script. As the capabilities of the large language model are improved, the technology of understanding the script based on the large language model has been developed. In the script understanding task, the script generally records the contents of multiple scenes in the film and television works, and the content of each scene includes information such as time, characters, and dialogues. By obtaining the story, character relationships, and character development in the script, the script is scored in multiple dimensions to facilitate the determination of the shooting value of the script and provide a basis for the selection of the script. However, the number of words in the script is usually large, such as hundreds of thousands of words, and the script contains many scenes, such as tens of thousands of scenes. It takes a long time to use manual reading. Therefore, script understanding based on a large language model will greatly improve the efficiency of script reading. By obtaining the summary text of the script based on the large language model, the general content of each scene in the script can be quickly obtained, thereby improving the processing efficiency of the script.

[0093] In the scenario of script understanding, due to the diversity of text descriptions, a certain summary text output by the large language model may be quite different from another summary text, but the actual meanings of the two are the same; or, the difference between the two summary texts is small, but the meanings are inconsistent. The above situation may exist in a sentence of the summary text, or it may exist in multiple sentences of the summary text. However, in the process of obtaining summary text by the traditional large language model, the model input is a paragraph of text, the model output is a paragraph of summary text, and a certain pre-made summary text is used as supervision information to calculate the loss of the output. The above method cannot handle the above-mentioned text differences, content differences, and errors in specific sentences, resulting in the large language model being limited to learning a single supervisory information when generating summary text, thereby overfitting the single supervisory information, and then failing to generate rich language descriptions; at the same time, there may be errors in the single supervisory information, which will lead to a high possibility of content errors in the generated summary text.

[0094] In view of the above problems, the above embodiment proposes a training method for a large language model, which can effectively solve the above problems. However, the above embodiment only briefly describes the training process of the large language model. On the basis of the above embodiment, the embodiment of the present application also provides another training method for a large language model to describe in detail the way to determine the loss function value in the training process of the large language model. In the embodiment of the present application, the loss function value corresponding to each sample text includes a first loss function value and a second loss function value.

[0095] like Figure 3 As shown, Figure 3 1 is a flowchart of another large language model training method provided in an embodiment of the present application. The present application embodiment is executed by a computer device, and the sample text is a script as an example for explanation in the present application embodiment. The method comprises the following steps:

[0096] 301. The computer device inputs the sample text into the large language model to obtain prediction summary information, where the prediction summary information includes the probability of each character in a plurality of characters at each character position in the prediction summary text of the sample text.

[0097] In the embodiment of the present application, step 301 is the same as the above-mentioned step 201 and will not be described again here.

[0098] 302. The computer device obtains a first summary text of the sample text and a second summary text of the sample text. The first summary text contains more information than the second summary text. The first summary text includes correct sentences and incorrect sentences. The correct sentences are sentences that correctly reflect the sample text, and the incorrect sentences are sentences that incorrectly reflect the sample text.

[0099] In the embodiment of the present application, step 302 is the same as the above-mentioned step 202 and will not be described again here.

[0100] In a possible implementation, the computer device obtains the first summary text and the second summary text through the first additional model. Accordingly, the computer device inputs the sample text and the first prompt text into the first additional model to obtain the first summary text, and the first prompt text is used to instruct the first additional model to generate a summary text containing supplementary information based on the sample text; the computer device inputs the sample text and the second prompt text into the first additional model to obtain the second summary text, and the second prompt text is used to instruct the first additional model to generate a summary text based on the sample text.

[0101] For example, the first additional model is an open source language model such as the GPT model (Generative Pre-trained Transformer). The first prompt text is "Given the script of a scene in a play, please give a plot summary of the scene. It is required to describe the specific events and time, place, cause, process, and result of the scene. Call the characters by their names, do not guess, and describe the specific events in the dialogue rather than abstract descriptions. The script should be within 100 words. The script is (xxx)"; the second prompt text is "Given the script of a scene in a play, please give a plot summary of the scene. It is required to describe what happened in the scene. Do not guess or make abstract descriptions. The script should be within 50 words. The script is (xxx)". Among them, "xxx" is the sample text. "Specific events and time, place, cause, process, and result" is the supplementary information contained in the first summary text; "what happened in the scene" is the basic information contained in the second summary text.

[0102] It should be noted that the first prompt text and the second prompt text shown above are only exemplary descriptions. Since the open source language model can accept any text input, other prompt texts can be used when the computer device obtains the summary text of the sample text through the open source language model. Optionally, compared with the second prompt text, the first prompt text introduces more information to the first summary text by setting a larger word limit and introducing elements such as time and place, so that the first summary text has a better description effect on the sample text than the second summary text, which improves the richness of the training data and facilitates the subsequent improvement of the description accuracy of the large language model.

[0103] Optionally, after the computer device obtains the summary text of the sample text through the first additional model, the summary text is manually adjusted to correct errors in the summary text, such as plot description errors, factual errors, etc., and the semantic coherence is optimized to correct grammatical errors in the summary text. The adjusted summary text and sample text are used as training data for the large language model.

[0104] For example, the training data is in the form of text pairs, and the training data of the large language model includes multiple text pairs. The form of each text pair is (sample text, summary text summ1, summary text summ2, summary text summx with the best description effect). Among them, summary text summ1 is the summary text corresponding to the first prompt text, that is, the first summary text; summary text summ2 is the summary text corresponding to the second prompt text, that is, the second summary text. Since the description effect of the first summary text is better than that of the second summary text in this method, the summary text summx is the same as the summary text summ1.

[0105] In the above method, by providing different prompt texts for the first additional model, the first additional model can generate a second summary text containing basic information and a first summary text containing supplementary information. The second summary information is more concise, and the first summary text is more comprehensive and accurate. The above method not only improves the diversity of training data, enables the large language model to learn diverse text representations, improves generalization performance, but also provides a basis for comparative learning for the large language model, which is conducive to improving the accuracy of the model.

[0106] In a possible implementation, when two summary texts of the sample text already exist, the computer device determines the first summary text and the second summary text through the evaluation model. For example, the evaluation model is an open source language model. The prompt text of the evaluation model is "Given the script (xxx) and two corresponding summaries - Summary 1: (xx1), Summary 2: (xx2), please compare the two summaries, and judge which one is better based on whether the content is correct and the description is complete, and return the better summary. The answer style is: {'good summary': 'summary 1'}". Among them, "xx1" and "xx2" are two summary texts of the sample text respectively; the summary text with better description effect is determined as the "good summary", that is, the first summary text; the summary text with poor description effect is determined as the second summary text, that is, another summary text other than the "good summary".

[0107] In a possible implementation, two summary texts of the sample text are obtained by combining the first prompt text and the second prompt text through the first additional model. At this time, the computer device does not directly use the summary text obtained from the first prompt text as the first summary text and the summary text obtained from the second prompt text as the second summary text, but only saves the data in the form of (sample text, two summary texts). Then, the computer device determines the summary text with better description effect from the two summary texts as the first summary text, and determines the summary text with worse description effect as the second summary text through the evaluation model, thereby obtaining training data.

[0108] For example, the training data is in the form of text pairs, and the training data of the large language model includes multiple text pairs. The form of each text pair is (sample text, summary text summ1, summary text summ2, summary text summx with the best description effect). Among them, summary text summ1 is the summary text corresponding to the first prompt text; summary text summ2 is the summary text corresponding to the second prompt text. Since the description effects of summary text summ1 and summary text summ2 need to be judged by the evaluation model, summary text summx may be summary text summ1 or summary text summ2, which is related to the judgment result of the evaluation model.

[0109] 303. The computer device determines an average loss function value of the correct sentences in the first summary text and an average loss function value of the incorrect sentences in the first summary text based on the predicted summary information, the correct sentences in the first summary text, and the incorrect sentences in the first summary text.

[0110] In the embodiment of the present application, the first summary text is divided according to the period to obtain multiple sentences in the first summary text. According to the accuracy of each sentence in describing the sample text, the multiple sentences in the first summary text can be divided into correct sentences and incorrect sentences.

[0111] In a possible implementation, the computer device determines the average loss function value through the loss function value of each sentence. Accordingly, the computer device determines the loss function value of each sentence in the first summary text based on the predicted summary information, and the loss function value of any sentence is the average of the cross entropy losses of multiple characters in the sentence; the computer device determines the average loss function value of the correct sentences in the first summary text and the average loss function value of the incorrect sentences in the first summary text based on the loss function value of each sentence in the first summary text, and the average loss function value of the correct sentences in the first summary text is the average of the loss function values ​​of multiple correct sentences in the first summary text, and the average loss function value of the incorrect sentences in the first summary text is the average of the loss function values ​​of multiple incorrect sentences in the first summary text.

[0112] In the above manner, the computer device implements classification and processing of the loss function values ​​of multiple sentences in the first summary through the above manner, thereby distinguishing correct sentences and incorrect sentences in the first summary text, which is beneficial to improving the accuracy of model generation.

[0113] In a possible implementation, the first summary text further includes inaccurate sentences, which are sentences that inaccurately reflect the sample text. That is, according to the accuracy of each sentence in describing the sample text, multiple sentences in the first summary text can be divided into correct sentences, wrong sentences, and inaccurate sentences. Accordingly, the average loss function value of the inaccurate sentences in the first summary text is the average of the loss function values ​​of the multiple inaccurate sentences in the first summary text.

[0114] For the convenience of description, multiple sentences in the first summary text are divided according to the accuracy of each sentence in describing the sample text. Take the following first summary text as an example, "At the banquet, character a drinks alone, and guests A and B gossip about her past and current identity. Character a walks towards character b, and the two clink glasses. Character b hints that character a has humiliated him. Character a sits on character b's thigh, and character b smiles triumphantly. Character c pours red wine on character a's head, and character a fights back. Character c pours red wine angrily, and character d blocks and puts his coat on character a. Character b asks character a to send character a off, and character d offers to send character a off, and character a is shocked."

[0115] The first summary text is divided according to the period to obtain five sentences in the summary text. In the case where multiple sentences in the first summary text are divided into two or three categories, the types of the five sentences are shown in (i) and (ii) below.

[0116] (I) When multiple sentences in the first summary text are divided into two categories, namely, correct sentences and incorrect sentences, the five sentences are [correct sentence, correct sentence, correct sentence, correct sentence, incorrect sentence]. Among them, the sample text describes that character a is shocked that character d blocks the red wine and puts on a coat, while the fifth sentence in the first summary text "character d takes the initiative to offer to send character a, and character a is shocked" means that character a is shocked that character d takes the initiative to offer to send character a, so the fifth sentence in the first summary text has a causal description error and is an incorrect sentence.

[0117] (II) In the case where multiple sentences in the first summary text are divided into three categories, namely correct sentences, incorrect sentences and inaccurate sentences, the five sentences are [inaccurate sentences, correct sentences, correct sentences, correct sentences, incorrect sentences]. Among them, the reason why the fifth sentence in the first summary text is an incorrect sentence is the same as in (I). For the first sentence in the first summary text, since the sample text describes the specific content of the guests' gossip, and the first sentence "guest A and guest B gossip about her past and current identity" is an abstract expression, and does not provide a general description of the specific content discussed by the guests, the first sentence in the first summary text has an inaccurate description, which is an inaccurate sentence. An exemplary accurate description is "In the small banquet hall of the club, character a drinks alone and is discussed by guests A and guest B about his past entanglement with character b."

[0118] In a possible implementation, the computer device determines the type of each sentence through the second additional model. Accordingly, the computer device inputs the first summary text into the second additional model to obtain a label for each sentence in the first summary text, where the label of each sentence is used to indicate the accuracy of the sentence reflecting the sample text; the computer device inputs the second summary text into the second additional model to obtain a label for each sentence in the second summary text.

[0119] For example, the second additional model is an open source language model. According to dividing the multiple sentences in the first summary text into two or three categories, the prompt text of the second additional model can be shown in the following (1) and (2).

[0120] (1) In the case where multiple sentences in the first summary text are divided into two categories, namely, correct sentences and incorrect sentences, the prompt text of the second additional model is: For the script text in brackets (A), there is the following plot summary (B). Please evaluate each sentence in the summary and judge whether the sentence description is correct. Give two possible situations, correct and incorrect, and give reasons for incorrect sentences. Output in the following json format: {"judgment result": [{"sentence":"x1","judgment":"incorrect","reason":"r1"},……,]}.

[0121] (2) In the case where multiple sentences in the first summary text are divided into three categories: correct sentences, wrong sentences, and inaccurate sentences, the prompt text of the second additional model is: For the script text in brackets (A), there is the following plot summary (B). Please evaluate each sentence in the summary and judge whether the sentence description is correct. Give three possible situations: correct, wrong, and inaccurate, and give reasons for wrong and inaccurate sentences. Output in the following json format: {"judgment result": [{"sentence":"x1","judgment":"wrong","reason":"r1"},...,]}.

[0122] It should be noted that the prompt text of the second additional model shown above is only an exemplary description and is not intended to limit this. Wherein, A is the sample text, B is the first summary text, x1 is sentence 1 in the first summary text, and r1 is the reason for return. According to the prompt text of the second additional model, the second additional model determines the accuracy of each sentence in the first summary text in describing the sample text, and the output result of the second additional model is (A, B, [x1, x2, ...], [a1, a2, ...], [r1, r2, ...]). Wherein, A is the sample text, B is the first summary text, x1, x2, etc. represent multiple sentences in the first summary text, a1, a2, etc. represent the judgment results of each sentence, and r1, r1, etc. represent the reasons for the judgment of each sentence.

[0123] In the above method, since the second additional model can determine the accuracy of the sentence reflecting the sample text from multiple aspects such as whether the sentence in the summary text accurately summarizes the information contained in the sample text, whether it contains redundant or irrelevant information, and whether there are description errors, etc. Therefore, by determining the label of each sentence, it is convenient for the large language model to focus on learning specific types of sentences during the training process, thereby improving the training efficiency.

[0124] In the above manner, the computer device divides the sentences in the first summary text according to the accuracy of describing the sample text, so that during the training process, the large language model can identify and focus on sentences that are wrong or inaccurately reflect the sample text, thereby reducing or avoiding the generation of sentences that are wrong or inaccurately reflect the sample text, and generating more sentences that correctly reflect the sample text, thereby improving the accuracy of model generation.

[0125] 304. The computer device obtains a first loss function value based on an average loss function value of correct sentences in the first summary text and an average loss function value of incorrect sentences in the first summary text.

[0126] In the embodiment of the present application, the first loss function value is obtained by weighted summing the average loss function value of the correct sentence and the average loss function value of the incorrect sentence. For ease of description, the first loss function value can also be referred to as the basic cross entropy loss with right and wrong distinction processing.

[0127] In a possible implementation, the first summary text also includes inaccurate sentences. Accordingly, the computer device obtains the first loss function value by weighted summing the average loss function value of the correct sentences, the average loss function value of the wrong sentences, and the average loss function value of the inaccurate sentences.

[0128] It should be noted that, when the multiple sentences in the first summary text are divided into two categories or three categories, the calculation methods of the first loss function value are respectively shown in the following method 1 and method 2.

[0129] Method 1: When multiple sentences in the first summary text are divided into two categories, namely, correct sentences and incorrect sentences, the calculation method of the first loss function value is as shown in the following formula 1.

[0130] Formula 1:

[0131]

[0132] Among them, loss1 is the first loss function value. i is the loss function value of the i-th wrong sentence in the first summary text, n1 is the number of wrong sentences in the first summary text, The part represents the average loss function value of the wrong sentences in the first summary text. j is the loss function value of the jth correct sentence in the first summary text, n2 is the number of correct sentences in the first summary text, The part represents the average loss function value of the correct sentences in the first summary text.

[0133] Wherein, w1 is the weight coefficient of the wrong sentence, and w2 is the weight coefficient of the correct sentence. Optionally, the sum of w1 and w2 is equal to 1, that is, the average loss function values ​​of the two types of sentences, namely, the correct sentence and the wrong sentence, are weighted and summed in a smooth weighted manner. For example, w1 is 0.9 and w2 is 0.1.

[0134] Method 2: When multiple sentences in the first summary text are classified into three categories: correct sentences, incorrect sentences, and inaccurate sentences, the calculation method of the first loss function value is as shown in the following formula 2.

[0135] Formula 2:

[0136]

[0137] Among them, loss1 is the first loss function value. i is the loss function value of the i-th wrong sentence in the first summary text, n1 is the number of wrong sentences in the first summary text, The part represents the average loss function value of the wrong sentences in the first summary text. j is the loss function value of the jth correct sentence in the first summary text, n2 is the number of correct sentences in the first summary text, The part represents the average loss function value of the correct sentences in the first summary text. k is the loss function value of the kth inaccurate sentence in the first summary text, n3 is the number of inaccurate sentences in the first summary text, The part represents the average loss function value of inaccurate sentences in the first summary text.

[0138] Wherein, w1 is the weight coefficient of the wrong statement, w2 is the weight coefficient of the correct statement, and w3 is the weight coefficient of the inaccurate statement. Optionally, the sum of w1, w2 and w3 is equal to 1, that is, the average loss function values ​​of the three types of statements, namely, correct statements, wrong statements and inaccurate statements, are weighted and summed in a smooth weighted manner. For example, w1 is 0.8, w2 is 0.1, and w3 is 0.1.

[0139] In the above method, the computer device determines the average loss function value of each sentence in the first summary text, and then performs weighted summation to obtain the first loss function value. Compared with the traditional method of summing the loss function value of each character in the first summary text, since different weights are set according to the importance of different types of sentences in the above method, different types of sentences have different degrees of influence on the overall loss function value.

[0140] Since there are usually fewer incorrect sentences in summary texts, if the loss function value is calculated with the same weight for each character in the traditional way, since the number of characters contained in the incorrect sentence is less than the correct sentence, the large language model does not get more emphasis on the learning of the incorrect sentence, and a large number of correct sentences or sentences with slightly different description wording constitute the main learning target, so that the large language model learns a large amount of content that no longer needs to be focused on. Through the above method, during the training process, the large language model can focus on sentences with larger weights, avoiding a large amount of processing of information that does not need to be focused on, and improving the training efficiency of the model and the accuracy of model generation.

[0141] The above steps 303 to 304 are an exemplary method for determining the first loss function value based on the prediction summary information and the first summary text provided in an embodiment of the present application.

[0142] 305. The computer device determines a predicted latent vector of the first summary text and a predicted latent vector of the second summary text based on the predicted latent vectors corresponding to the multiple characters, the multiple characters in the first summary text, and the multiple characters in the second summary text.

[0143] In an embodiment of the present application, the predicted latent vectors corresponding to multiple character positions are intermediate data obtained by the large language model head module (LM_Head). Since the predicted latent vectors corresponding to multiple characters are in matrix form, the rows of the matrix correspond to each semantic dimension, and the columns of the matrix correspond to multiple characters in the dictionary of the large language model. Therefore, the predicted latent vector of the first summary text is composed of columns corresponding to multiple characters in the first summary text, and the predicted latent vector of the second summary text is composed of columns corresponding to multiple characters in the second summary text. The predicted latent vector of the first summary text and the predicted latent vector of the second summary text are similar in the semantic space. In the above manner, the computer device obtains the predicted latent vectors of the two summary texts respectively according to the predicted latent vectors corresponding to the multiple characters, that is, the representations of the two summary texts at the semantic level are obtained respectively.

[0144] For example, when the dictionary of the large language model includes 6400 characters and multiple semantic dimensions are 4096, the predicted latent vectors corresponding to multiple character positions are represented as a matrix of 4096x6400. When the first summary text contains 100 characters and the second summary text contains 120 characters, the predicted latent vector of the first summary text is represented as a matrix of 100x4096, and the predicted latent vector of the second summary text is represented as a matrix of 120x4096.

[0145] 306. The computer device performs pooling on the predicted latent vector of the first summary text and the predicted latent vector of the second summary text respectively to obtain a first intermediate vector and a second intermediate vector, wherein the first intermediate vector and the second intermediate vector have the same dimension.

[0146] In the embodiment of the present application, the pooling operation is used to process the first summary text and the second summary text into vectors of the same dimension, namely, the first intermediate vector and the second intermediate vector, for example, the dimensions of both are 1x4096. Optionally, when performing the pooling operation, the computer device may select maximum pooling or mean pooling, which is not limited here.

[0147] 307. The computer device determines a second loss function value based on the first intermediate vector and the second intermediate vector, where the second loss function value is the distance between the first intermediate vector and the second intermediate vector.

[0148] In the embodiment of the present application, in addition to expressing the second loss function value by the Euclidean distance between the two intermediate vectors, the computer device can also use the cosine distance to express the second loss function value after normalizing the two intermediate vectors, which will not be repeated here. For ease of description, the second loss function value can be called self-supervised contrast loss. The calculation method of the second loss function is shown in the following formula three.

[0149] Formula 3:

[0150] loss2 = d(pool(a1), pool(a2))

[0151] Where loss2 is the second loss function value. pool represents the pooling operation, and d represents the distance calculation operation. a1 is the predicted hidden vector of the first summary text, and pool(a1) is the first intermediate vector; a2 is the predicted hidden vector of the second summary text, and pool(a2) is the second intermediate vector.

[0152] In the above method, the two summary information provides additional reference for model learning to avoid overfitting to a single summary text. Since the traditional model loss value is based on the predicted probability of all character positions in the dictionary of the large language model, the probability of characters in the summary text is calculated without processing the probability of characters that do not belong to the summary text, that is, for a dictionary of size 6400 at each character position, only one word is learned by the summary text, which is inefficient. Since the second loss function value represents the gap between the two summary texts in the semantic space predicted by the large language model, it is beneficial for the large language model to learn the information in the two summary texts at the same time, which not only improves the generalization of the model, but also improves the accuracy of the model.

[0153] The above steps 305 to 307 are an exemplary method for determining the second loss function value based on the first summary text, the second summary text, and the predicted latent vectors corresponding to multiple characters provided in an embodiment of the present application.

[0154] 308. The computer device performs weighted summation of the first loss function value and the second loss function value to obtain a total loss function value.

[0155] In the embodiment of the present application, the calculation method of the total loss function value is shown in the following formula 4.

[0156] Formula 4:

[0157] Loss = loss1 + w0 * loss2

[0158] Wherein, Loss is the total loss function value, loss1 is the first loss function value, and loss2 is the second loss function value. w2 can be set to 0.1 or other empirical values, which are not limited here.

[0159] 309. The computer device trains the large language model based on the total loss function value.

[0160] In the embodiment of the present application, the total loss function value is used to determine the model parameter gradient of the large language model. The computer device updates the model parameters of the large language model based on the model parameter gradient. It should be noted that in the training process of the large language model, by comparing and learning two different summaries of the same sample text, more possible change information contained in the summary text is introduced into the large language model, thereby avoiding the problems caused by a single summary text.

[0161] The present application embodiment shows the composition of a piece of training data in the above steps, and the data format of multiple pieces of training data is introduced below. It should be noted that when multiple sentences in the first summary text are divided into two categories or three categories, the training data are organized into the following data format one or data format two respectively.

[0162] Data format 1: When multiple sentences in the first summary text are divided into two categories, namely, correct sentences and incorrect sentences, each piece of training data is organized into the following format.

[0163] {"id":"999","conversations":[{"from":"human","value":"The following is a script of a scene in a play. Please extract the plot summary of this scene and describe the specific events that happened in the scene. Do not make guesses or abstract descriptions. The script is xxx"},

[0164] {"from":"gpt","value":"xxxx"},

[0165] {"from":"gpteval","value":[0,1,0,0,0]},

[0166] {"from":"gpt2","value":"xxxx2"},]}

[0167] Where id is the serial number of the sample text, which starts from 1. gpt represents the first summary text with higher description effect, and gpt2 represents the second summary text with lower description effect. gpteval is used to represent the label of each sentence in the first summary text gpt, 0 is the label indicating the correct sentence, and 1 is the label indicating the wrong sentence.

[0168] Data format 2: When multiple sentences in the first summary text are divided into three categories: correct sentences, incorrect sentences, and inaccurate sentences, each training data is organized into the following format.

[0169] {"id":"999","conversations":[{"from":"human","value":"The following is a script of a scene in a play. Please extract the plot summary of this scene and describe the specific events that happened in the scene. Do not make guesses or abstract descriptions. The script is xxx"},

[0170] {"from":"gpt","value":"xxxx"},

[0171] {"from":"gpteval","value":[0,1,2,0,0,0]},

[0172] {"from":"gpt2","value":"xxxx2"},]}

[0173] Where id is the serial number of the sample text, which starts from 1. gpt represents the first summary text with higher description effect, and gpt2 represents the second summary text with lower description effect. gpteval is used to represent the label of each sentence in the first summary text gpt, where 0 is the label indicating a correct sentence, 1 is the label indicating an inaccurate sentence, and 2 is the label indicating an incorrect sentence.

[0174] It should be noted that, for the above data formats 1 and 2, the correspondence between the meaning indicated by the label and 0, 1, and 2 can be adjusted at will, and there is no restriction on this. The first summary text gpt is used in the process of calculating the first loss function value corresponding to each sample text, and the first summary text gpt and the second summary text gpt2 are used in the process of calculating the second loss function value corresponding to each sample.

[0175] For the convenience of describing the process introduced in the embodiment of this application, see Figure 4 As shown, Figure 4It is a schematic diagram of determining the loss function value provided by an embodiment of the present application. For the convenience of description, the summary text summ1 obtained by the first additional model based on the first prompt text and the sample text is called summary text 1; the summary text summ2 obtained by the first additional model based on the second prompt text and the sample text is called summary text 2. The computer device generates summary text 1 and summary text 2 respectively through the first additional model, as shown in step 302. At this time, summary text 1 and summary text 2 are only used to indicate the two summary texts obtained according to the sample text, and do not indicate the quality of the description effect. The computer device compares the description effects of summary text 1 and summary text 2 through the evaluation model, and determines that the first summary text is the summary text with better description effect, and determines that the second summary text is the summary text with worse description effect, as shown in step 302. For the first summary text, sentences with different accuracy levels are first distinguished, and then different types of sentences are distinguished and learned to obtain a first loss function value, as shown in steps 303 and 304; at the same time, for two different summary texts, self-supervised comparative learning is performed on the predicted latent vectors corresponding to multiple characters output in the middle of the large language model, and auxiliary alignment losses are performed on different summary texts, so that the large language model maintains a certain intermediate result, that is, different result outputs are allowed at the same time, as shown in steps 305 to 307.

[0176] To facilitate the description of the architecture of training a large language model in the embodiment of the present application, see Figure 5 As shown, Figure 5 This is an architecture diagram of a large language model training provided by an embodiment of the present application. When the large language model completes the prediction, the computer device determines the first loss function value through the first summary text and the predicted summary information, as shown in steps 303 and 304; the computer device determines the second loss function value through summary text 1, summary text 2, and the predicted latent vectors corresponding to multiple characters, as shown in steps 305 to 307.

[0177] An embodiment of the present application provides a training method for a large language model. In the method, a first summary text and a second summary text are introduced in the training process of the large language model, so that more learnable information is provided for model training. At the same time, since the first summary text and the second summary text contain different amounts of information, and the first summary text contains correct sentences and incorrect sentences, by comparing and learning two different summary texts of the same sample text, and distinguishing and learning the correct sentences and incorrect sentences in the first summary text, problems such as model overfitting and inaccurate generation due to a single summary text are avoided, thereby improving not only the generalization performance of the model, but also the accuracy of the model.

[0178] In this process, the embodiments of the present application improve the accuracy of the model from the following two aspects.

[0179] First, the summary text usually contains more sentences, and usually only a few sentences have errors. In the traditional method, each character is calculated with the same weight coefficient for loss. Since there are relatively few wrong sentences and relatively many correct sentences, the model's learning of wrong sentences cannot be emphasized. At this time, a large amount of correct or slightly different descriptions of the content constitute the main learning target, so that the model cannot learn the key points. That is, in the traditional method, the same weight coefficient is used for all characters, and it is impossible to distinguish between summary contents of different importance. This method solves the problem that the model learns that there are errors in the content of the summary text by introducing a mechanism to ignore erroneous summary texts. By distinguishing the correct and erroneous sentences in the summary text, it can avoid the problem of too many meaningless or correct sentences bringing a large amount of information that does not need to be focused on, thereby avoiding the problem of insufficient attention to the loss of erroneous sentences, thereby improving the accuracy of the model.

[0180] Secondly, the embodiment of the present application introduces an auxiliary method of aligning the prediction latent vectors of different summary texts, so that the model not only outputs a single supervisory information, thereby avoiding overfitting of the single supervisory information. That is, this method adopts a multi-answer joint comparative learning method to perform self-supervised comparative learning on different summary texts, thereby introducing richer prediction information, avoiding model overfitting, and allowing the model to generate diverse answers, thereby improving the generalization performance and description accuracy of the model.

[0181] In the script understanding scenario, since the summary generation task requires the output of multiple paragraphs of text, the more outputs there are, the easier it is to produce incorrect descriptions, so the model output often contains some erroneous information; on the other hand, due to the diversity of text descriptions, there is no unique standard for the summary text used as supervisory information, that is, there are often better descriptions for the current summary text used as supervisory information, and the upper limit of the model's capabilities depends on the model's correct or incorrect understanding of the current summary text, and thereby indirectly reaches the upper limit of the current data expression.

[0182] The above embodiment can solve this problem, but the above embodiment uses both the first loss function value and the second loss function value to train the large language model. With the goal of solving the above problem, only using the first loss function value can also achieve this goal.

[0183] The process of training a large language model using only the first loss function value is described below. The process of a computer device training a large language model using only the first loss function value is shown below (1)-(6). It should be noted that in the process of determining the first loss function value in this application, the various methods described below can be applied.

[0184] (1) The computer device inputs the sample text into the large language model to obtain the predicted summary information.

[0185] (2) The computer device obtains a first summary text of the sample text. Compared with the initial summary text, the first summary text is a summary text that introduces new knowledge, that is, a summary text that is generated and expanded. Optionally, the initial summary text is the second summary text in the above embodiment. Optionally, the first summary text and the second summary text are determined in the manner in the above embodiment. Alternatively, the first summary text is obtained in the following manner.

[0186] See also Figure 7 As shown, Figure 7 It is another schematic diagram of determining the loss function value provided by an embodiment of the present application. The initial training data set is referred to as training data set 1, and each training data in training data set 1 includes a sample text and a summary text of the sample text; the training data set after generation and expansion is referred to as training data set 2, and each training data in training data set 2 includes a sample text and a summary text after generation and expansion of the sample text, and the summary text after generation and expansion is the first summary text. For training data set 1, the sentences of the summary text in training data set 1 are classified by the second additional model, and the type of each sentence of the summary text in training data set 1 is determined. The first summary text is generated based on the sample text in training data set 1 by the generation model to obtain training data set 2. For training data set 2, the sentences of the first summary text in training data set 2 are classified by the second additional model, and the type of each sentence of the first summary text in training data set 2 is determined. Optionally, the generation model can be the first additional model shown in step 302, and the generation can also be a large language model trained by training data set 1, which is not limited here.

[0187] Optionally, when the sentences in the summary text are divided into two categories, the type of each sentence is used to indicate that the sentence is one of a correct sentence and an incorrect sentence. When the sentences in the summary text are divided into three categories, the type of each sentence is used to indicate that the sentence is one of a correct sentence, an incorrect sentence and an inaccurate sentence.

[0188] (3) The computer device determines the average loss function value of each type of sentence in the first summary text based on the predicted summary information and the first summary text. As shown in the above step 303, the average loss function value of each type of sentence is the average of the loss function values ​​of the sentence of that type. The loss function value of each sentence is the average of the cross entropy losses of multiple characters in the sentence. The calculation method of the loss function value of any sentence is shown in the following formula 5.

[0189] Formula 5:

[0190]

[0191] Among them, Lm is the loss function value of the mth sentence in the first summary text, and y[m,n] is the one-hot label of the nth character position in the sentence. N is the number of characters contained in the sentence. p[m,n] is the probability of predicting the character. It should be noted that the one-hot label of each character position is a one-dimensional vector, and different elements in the vector correspond to different characters in the dictionary of the large language model. When the character corresponding to an element in the vector is the character at the corresponding character position, the element is 1; when the character corresponding to an element in the vector is not the character at the corresponding character position, the element is 0.

[0192] For the cross entropy loss, the introduction is as follows:

[0193] When calculating the cross entropy loss in a traditional large language model, an example is taken in which the summary text contains 30 character positions and the dictionary of the large language model contains 6400 characters. For each character position to be generated, the probability of all characters in the dictionary of the large language model is first predicted, and then the sequence number of the character with the maximum predicted probability is taken at each character position, and the predicted probability is formed by the sequence numbers of the 30 character positions to be generated. The dimension of the predicted probability information output by the model is 30x6400, that is, the probability that 6400 characters can be selected for each character position in the 30 character positions, and the predicted probability information is calculated with the summary text as the supervisory information. Accordingly, for each character in the summary text, the multi-classification loss of 6400 classes is calculated separately. Each character in the dictionary of the large language model is a class. The calculation method of the model loss is shown in the following formula 6.

[0194] Formula 6:

[0195]

[0196] Where y represents the one-hot label of the character; y[i,j] represents the character at the jth position in the summary text corresponding to the i-th sample text in the training batch. p represents the probability of predicting each character. bs is the number of sample texts in each training batch. Ni is the number of words in the summary text corresponding to the i-th sample text in the training batch.

[0197] (4) The computer device determines the weight coefficient of each type of statement. The weight coefficient of each type of statement can be determined by any of the following methods A, B, and C.

[0198] Method A: The weight coefficient of each type of sentence is preset manually. For example, when multiple sentences in the first summary text are divided into two categories, the weight coefficient of the wrong sentence is set to 0.9, and the weight coefficient of the correct sentence is set to 0.1; or, the weight coefficient of the wrong sentence is set to 0.8, and the weight coefficient of the correct sentence is set to 0.2. When multiple sentences in the first summary text are divided into three categories, the weight coefficient of the wrong sentence is set to 0.8, the weight coefficient of the correct sentence is set to 0.1, and the weight coefficient of the inaccurate sentence is set to 0.1; or, when multiple sentences in the first summary text are divided into three categories, the weight coefficient of the wrong sentence is set to 0.7, the weight coefficient of the correct sentence is set to 0.1, and the weight coefficient of the inaccurate sentence is set to 0.2.

[0199] Method B: The computer device dynamically adjusts the weight coefficient of each type of sentence according to the data representation of the first summary text and the second summary text in the current training batch.

[0200] When multiple sentences in the first summary text are divided into two categories, the computer device determines the weight coefficient of the correct sentence; based on the weight coefficient of the correct sentence, the weight coefficient of the wrong sentence is determined, and the weight coefficient of the wrong sentence is the difference between the weight coefficient of the correct sentence and 1, that is, 1 minus the weight coefficient of the correct sentence.

[0201] Wherein, the current training batch includes multiple training text pairs, each training text pair includes a sample text, a corresponding first summary text and a corresponding second summary text. Each summary text contains multiple sentences. The first summary text and the second summary text contained in the above multiple training text pairs are collectively referred to as all summary texts. The weight coefficient of the correct sentence is obtained based on all the summary texts in the current training batch. The weight coefficient of the correct sentence is the ratio of the sum of the loss function values ​​of the correct sentences in all the summary texts of the current training batch to the sum of the loss function values ​​of the sentences in all the summary texts of the current training batch. Wherein, the sum of the loss function values ​​of the sentences of all the summary texts is the sum of the loss function values ​​of the correct sentences of all the summary texts and the loss function values ​​of the incorrect sentences of all the summary texts.

[0202] The calculation method of the weight coefficient of the correct sentence is shown in the following formula 7.

[0203] Formula 7:

[0204]

[0205] Among them, n11 and n12 respectively represent the number of incorrect sentences in all the first summary texts and all the second summary texts of the current training batch; n21 and n22 respectively represent the number of correct sentences in all the first summary texts and all the second summary texts of the current training batch. represents the sum of the loss function values ​​of the wrong sentences in all the first summary texts and all the second summary texts of the current training batch; It represents the sum of the loss function values ​​of the correct sentences in all the first summary texts and all the second summary texts in the current training batch. q is the adjustment index.

[0206] When multiple sentences in the first summary text are divided into three categories, the computer device determines the weight coefficients of the correct sentences and the weight coefficients of the inaccurate sentences; based on the weight coefficients of the correct sentences and the weight coefficients of the inaccurate sentences, the weight coefficients of the wrong sentences are determined, and the weight coefficients of the wrong sentences are the difference between the sum of the weight coefficients of the correct sentences and the weight coefficients of the inaccurate sentences and 1, that is, 1 minus the sum of the weight coefficients of the correct sentences and the weight coefficients of the inaccurate sentences.

[0207] The weight coefficient of the inaccurate sentence is the ratio of the sum of the loss function values ​​of the inaccurate sentences in all summary texts of the current training batch to the sum of the loss function values ​​of the sentences in all summary texts of the current training batch. The sum of the loss function values ​​of the sentences in all summary texts is the sum of the loss function values ​​of the correct sentences in all summary texts, the loss function values ​​of the wrong sentences in all summary texts, and the loss function values ​​of the inaccurate sentences in all summary texts.

[0208] The calculation methods of the weight coefficient of the correct sentence and the weight coefficient of the inaccurate sentence are respectively referred to the following formula 8 and formula 9.

[0209] Formula 8:

[0210]

[0211] Formula 9:

[0212]

[0213] Among them, n31 and n32 are the number of inaccurate sentences in all the first summary texts and all the second summary texts in the current training batch. Represents the sum of the loss function values ​​of inaccurate sentences in all first summary texts and all second summary texts in the current training batch.

[0214] Method C: The computer device dynamically adjusts the weight coefficient of each type of sentence through the data representation of the first summary text in the current training batch. Since the weight coefficient of each type of sentence is used to weight the average loss function value of each type in the first summary text, the weight coefficient of each type can be determined only by all the first summary texts in the current training batch, without involving all the second summary texts in the current training batch.

[0215] In the case of classifying multiple sentences in a first summary text into two categories, the computer device determines a first intermediate ratio based on the loss function value of each sentence in the first summary text of multiple sample texts in a current training batch, the first intermediate ratio being the ratio of the sum of the loss function values ​​of multiple correct sentences in the first summary text of the multiple sample texts to the sum of the loss function values ​​of multiple sentences in the first summary text of the multiple sample texts; determines weight coefficients of correct sentences in the first summary text based on the first intermediate ratio, the weight coefficients of correct sentences in the first summary text being a preset power of the first intermediate ratio; determines weight coefficients of incorrect sentences in the first summary text based on the weight coefficients of correct sentences in the first summary text, the weight coefficients of incorrect sentences in the first summary text being the difference between the weight coefficients of correct sentences in the first summary text and 1.

[0216] Among them, the current training batch includes multiple training text pairs, and each training text pair includes a sample text, a corresponding first summary text and a corresponding second summary text. Each summary text contains multiple sentences. Therefore, the multiple erroneous sentences in the first summary text of the multiple sample texts refer to all the erroneous sentences in all the first summary texts contained in the multiple sample text pairs of the current training batch, that is, it includes every erroneous sentence in each first summary text in the current training batch. The multiple sentences in the first summary text of the multiple sample texts refer to all the sentences in all the first summary texts contained in the multiple sample text pairs of the current training batch, that is, it includes every sentence in each first summary text in the current training batch.

[0217] The calculation method of the weight coefficient of the correct sentence is shown in the following formula 10.

[0218] Formula 10:

[0219]

[0220] Among them, m11 represents the number of incorrect sentences in the first summary text with sequence number 1 in the current training batch, m12 represents the number of incorrect sentences in the first summary text with sequence number 2 in the current training batch, and so on; m21 represents the number of correct sentences in the first summary text with sequence number 1 in the current training batch, m22 represents the number of correct sentences in the first summary text with sequence number 2 in the current training batch, and so on. Represents the sum of the loss function values ​​of all wrong sentences in all the first summary texts of the current training batch; It represents the sum of the loss function values ​​of all correct sentences in all the first summary texts of the current training batch. q is the adjustment index.

[0221] In the case where multiple sentences in the first summary text are classified into three categories, the computer device determines a second intermediate ratio and a third intermediate ratio based on the loss function value of each sentence in the first summary text of multiple sample texts in the current training batch, the second intermediate ratio being the ratio of the sum of the loss function values ​​of multiple correct sentences in the first summary text of the multiple sample texts to the sum of the loss function values ​​of multiple sentences in the first summary text of the multiple sample texts, and the third intermediate ratio being the ratio of the sum of the loss function values ​​of multiple inaccurate sentences in the first summary text of the multiple sample texts to the sum of the loss function values ​​of multiple sentences in the first summary text of the multiple sample texts; based on the second intermediate ratio and The third intermediate ratio is used to determine the weight coefficients of the correct sentences in the first summary text and the weight coefficients of the inaccurate sentences in the first summary text, the weight coefficients of the correct sentences in the first summary text are the preset power of the second intermediate ratio, and the weight coefficients of the inaccurate sentences in the first summary text are the preset power of the third intermediate ratio; based on the weight coefficients of the correct sentences in the first summary text and the weight coefficients of the inaccurate sentences in the first summary text, the weight coefficients of the erroneous sentences in the first summary text are determined, and the weight coefficients of the erroneous sentences in the first summary text are the difference between the sum of the weight coefficients of the correct sentences in the first summary text and the weight coefficients of the inaccurate sentences in the first summary text and 1.

[0222] The multiple inaccurate sentences in the first summary texts of the multiple sample texts refer to all the inaccurate sentences in all the first summary texts included in the multiple sample text pairs of the current training batch, that is, every inaccurate sentence in each first summary text in the current training batch.

[0223] The calculation methods of the weight coefficients of the correct sentences and the inaccurate sentences are as shown in the following formula 11 and formula 12.

[0224] Formula 11:

[0225]

[0226] Formula 12:

[0227]

[0228] Among them, m31 represents the number of inaccurate sentences in the first summary text with sequence number 1 in the current training batch, m32 represents the number of inaccurate sentences in the first summary text with sequence number 2 in the current training batch, and so on. Represents the sum of the loss function values ​​of all incorrect sentences in all the first summary texts of the current training batch.

[0229] For the adjustment index q in the above-mentioned method B and method C, a positive integer not less than 3 is usually taken, such as the adjustment index q is 5 or 6. It should be noted that since the values ​​of the first intermediate value, the second intermediate value, the intermediate value 1, and the intermediate value 2 are less than 1, and most of the first summary text and the second summary text are correct sentences, the above values ​​are close to 1. Therefore, after adjustment by adjusting the index q, the weight coefficient of the correct sentence will become smaller, that is, the influence of the correct sentence on the overall loss function value will become smaller, which is conducive to the large language model learning other types of sentences.

[0230] In the above method, the computer device dynamically adjusts the weight coefficient of each type of sentence by using the loss function value of different types of sentences in the training batch and the exponent of the power operation. In the above method, the large language model can focus on the wrong sentences to be optimized during the training process, thereby improving the training efficiency and model accuracy.

[0231] (5) The computer device performs weighted summation on average loss function values ​​of multiple sentences in the first summary text based on the weight coefficient of each type of sentence to obtain a first loss function value.

[0232] (6) The computer device trains the large language model based on the first loss function value. The first loss function value is used to determine the model parameter gradient of the large language model. The computer device updates the model parameters of the large language model based on the model parameter gradient.

[0233] It should be noted that the first loss function value can be used only in a certain stage of training the large language model. For example, in the first 10 iterations, the cross entropy loss is used to train the model parameters for all training batches of data. This training stage is used for general information learning; then in the next 10 iterations, the first loss function value is used, that is, the classification loss learning method is used, so that the large language model is fine-tuned to produce better results.

[0234] It should be noted that, since only the first loss function value is used to train the large language model at this time, the multiple training data can be organized into the following data format: Taking the example of dividing multiple sentences in the first summary text into three categories for explanation.

[0235] {"id":"999","conversations":[{"from":"human","value":"The following is a script of a scene in a play. Please extract the plot summary of this scene and describe the specific events that happened in the scene. Do not make guesses or abstract descriptions. The script is xxx"},

[0236] {"from":"gpt","value":"xxxx"},

[0237] {"from":"gpteval","value":[0,1,2,0,0,0]},]}

[0238] Wherein, id is the serial number of the sample text, which starts from 1. gpt represents the first summary text obtained after generation and expansion. gpteval is used to represent the label of each sentence in the first summary text gpt, where 0 is the label indicating a correct sentence, 1 is the label indicating an inaccurate sentence, and 2 is the label indicating an incorrect sentence.

[0239] For the sake of convenience, see the architecture of training a large language model using the first loss function value. Figure 6 As shown, Figure 6 This is another architecture diagram for training a large language model provided by an embodiment of the present application. When the large language model completes the prediction, the computer device determines the average loss function value of each type of sentence through the first summary text and the predicted summary information; the computer device determines the first loss function value based on the weight coefficient of each type of sentence and the average loss function value of multiple types of sentences in the first summary text.

[0240] It should be noted that the above method of training the large language model using only the first loss function value can also be called a new knowledge mining training data expansion method based on generative information expansion and evaluation. In this method, new knowledge is introduced through generative mining to increase the data upper limit, providing richer and more informative training data than existing data; and a method of distinguishing correct and incorrect descriptions of summary texts is used to improve the model's focus on learning the current data when calculating the loss, that is, to make the model more focused in learning at different stages, thereby improving the accuracy of the overall description of the model.

[0241] The above embodiment introduces that the computer device trains the large language model based on the first loss function value and the second loss function value. On this basis, the computer device also uses the third loss function value to train the large language model. Figure 8 , Figure 8 is a flowchart of another large language model training method provided in an embodiment of the present application. The embodiment of the present application is executed by a computer device, and the method includes:

[0242] 801. A computer device inputs a sample text into a large language model to obtain prediction summary information, where the prediction summary information includes the probability of each character in a plurality of characters at each character position in the prediction summary text of the sample text.

[0243] In the embodiment of the present application, step 801 is the same as the above-mentioned step 201 and will not be described again here.

[0244] 802. The computer device obtains a first summary text of the sample text and a second summary text of the sample text. The first summary text contains more information than the second summary text. The first summary text includes correct sentences and incorrect sentences. The correct sentences are sentences that correctly reflect the sample text, and the incorrect sentences are sentences that incorrectly reflect the sample text.

[0245] In the embodiment of the present application, step 802 is the same as the above step 302, that is, the process in which the computer device obtains the first summary text and the second summary text through the first additional model is the same as the above step 302.

[0246] 803. The computer device determines a first loss function value and a second loss function value based on the predicted summary information, correct sentences in the first summary text, incorrect sentences in the first summary text, the second summary text, and predicted latent vectors corresponding to multiple characters.

[0247] In the embodiment of the present application, step 803 is the same as the above step 203.

[0248] That is, the computer device determines the average loss function value of each type of sentence in the first summary text based on the predicted summary information and the first summary text, as shown in step 303; determines the first loss function value based on the average loss function value of multiple types of sentences in the first summary text, as shown in step 304. And, based on the predicted latent vectors corresponding to multiple characters, multiple characters in the first summary text, and multiple characters in the second summary text, the predicted latent vector of the first summary text and the predicted latent vector of the second summary text are determined, as shown in step 305; the predicted latent vector of the first summary text and the predicted latent vector of the second summary text are pooled respectively to obtain a first intermediate vector and a second intermediate vector, and the dimensions of the first intermediate vector and the second intermediate vector are the same, as shown in step 306; based on the first intermediate vector and the second intermediate vector, a second loss function value is determined, and the second loss function value is the distance between the first intermediate vector and the second intermediate vector, as shown in step 307.

[0249] 804. For any summary text in the first summary text and the second summary text, the computer device obtains a matching parameter corresponding to any character position in the summary text, where the matching parameter corresponding to the character position is used to indicate the degree of matching between the character at the character position in the summary text and the other summary text in the first summary text and the second summary text.

[0250] In the embodiment of the present application, the larger the matching parameter corresponding to the character position, the higher the matching degree between the character at the character position and another summary text; the smaller the matching parameter corresponding to the character position, the lower the matching degree between the character at the character position and another summary text. That is, the matching parameter and the matching degree are positively correlated.

[0251] In a possible implementation, the computer device determines a matching parameter based on the similarity between characters. Accordingly, for the sentence where the character position is located in the summary text, the computer device determines a matching sentence of the sentence in another summary text; the computer device determines a matching parameter corresponding to the character position based on the matching character in the matching sentence and the character at the character position, and the matching parameter corresponding to the character position is the similarity between the character at the character position and the matching character.

[0252] For any sentence, the similarity between the matching sentences is greater than the similarity between other sentences in another summary text. For any character position, the similarity between the matching character in the matching sentence and the character at the character position is greater than the similarity between other characters in the matching sentence and the character at the character position. Optionally, the similarity between sentences is represented by Euclidean distance or cosine similarity; the similarity between characters is represented by Euclidean distance or cosine similarity.

[0253] In a possible implementation, a computer device determines a matching sentence by using the Euclidean distance between sentences as an example for explanation. Accordingly, for any sentence, the computer device determines the semantic vector of the sentence and the semantic vector of each sentence in another summary text; respectively determines the Euclidean distance between the semantic vector of the sentence and each semantic vector in another summary text; and determines the matching sentence based on the above multiple Euclidean distances. Among them, the first summary text and the second summary text are another summary text except the summary text where the sentence is located. The matching sentence is the sentence with the smallest Euclidean distance with the sentence among multiple sentences in another summary text, that is, the Euclidean distance between the matching sentence and the sentence is less than the Euclidean distance between the other sentences in the other summary text and the sentence.

[0254] The embodiment of the present application does not limit the method of obtaining the semantic vector of a sentence. For example, a BERT (Bidirectional Encoder Representations from Transformers, a bidirectional pre-trained language model) model based on a transformer network structure is used to extract the semantic vector of each sentence.

[0255] In a possible implementation, taking the example of a computer device determining matching characters through the cosine similarity between characters for illustration. Accordingly, for any character, the computer device determines the predicted hidden vector of the character and the predicted hidden vectors of each character in the matching statement based on the predicted hidden vectors corresponding to multiple characters; respectively determines the cosine similarity between the predicted hidden vector of the character and the predicted hidden vectors of each character; and determines the matching character based on the above-mentioned multiple cosine similarities. Among them, the matching character is the character with the largest cosine similarity between the character and the multiple characters in the matching statement, that is, the cosine similarity between the matching character and the character is greater than the cosine similarities between the other characters in the matching statement and the character.

[0256] Taking the example of determining the matching parameter for the character position where the character "饭" is located for illustration. The character "饭" is at a certain character position in the second sentence of the first abstract text, and the second sentence is "他吃过早饭" (He has had breakfast). First, the computer device determines the semantic vector of the second sentence and the semantic vectors of multiple sentences in the second abstract text. When the Euclidean distance between the second sentence and the third sentence "他吃过午餐" (He has had lunch) in the second abstract text is the smallest, the matching sentence is determined to be the third sentence of the second abstract text. Among the predicted hidden vectors of multiple characters output by the large language model's head module, the predicted hidden vector corresponding to the character "饭" is determined. For example, if the dictionary of the large language model contains 6000 characters and the semantic dimension is 4096, then the dimension of the predicted hidden vectors of multiple characters is 4096x6400. If the serial number corresponding to the character "饭" in the dictionary of the large language model is 6000, then the 1x4096 vector at serial number 6000 in the predicted hidden vectors of multiple characters is determined as the predicted hidden vector corresponding to the character "饭". The predicted hidden vectors corresponding to each character in the matching sentence are determined in the same way. When the cosine similarity between the character "饭" and the character "餐" in the matching sentence is the largest, the matching character is determined to be the character "餐".

[0257] In the above method, by determining the matching sentence with the highest similarity in another abstract text and determining the matching character with the highest similarity in the matching sentence, it is ensured that the matching character is highly relevant to the character at the given character position, improving the accuracy of character matching. Since the matching parameter is the similarity between the character at the character position and the matching character, the reliability of the matching parameter is improved.

[0258] 805. The computer device determines the weight coefficient of the character position based on the matching parameter corresponding to the character position.

[0259] In the embodiment of the present application, the weight coefficient of the character position is used to perform a weighted summation of the probabilities corresponding to the character positions in the subsequent model prediction of the probability of the sentence, thereby obtaining the value of the third loss function. The larger the weight coefficient of the character position, the greater the influence of the character position on the third loss function; the smaller the weight coefficient of the character position, the lower the influence of the character position on the third loss function.

[0260] It should be noted that the computer device determines the weight coefficient of the character position in any one of the following methods A and B.

[0261] Method A: The computer device first determines the label of the character position, and then determines the weight coefficient of the character position based on the label.

[0262] In a possible implementation, the computer device determines the label of the character position through the relationship between the matching parameter corresponding to the character position and the matching parameter threshold. Accordingly, when the matching parameter corresponding to the character position is less than the matching parameter threshold, the computer device determines the label of the character position associated with the character position as the first label; when the matching parameter corresponding to the character position is not less than the matching parameter threshold, the computer device determines the label associated with the character position as the second label.

[0263] Among them, when the matching parameter corresponding to the character position is smaller, it means that the large language model has a poorer learning effect on the character at this character position, and the large language model needs to focus on learning the character at this character position; when the matching parameter corresponding to the character position is larger, it means that the large language model has a better learning effect on the character at this character position, and the large language model does not need to learn the character at this character position additionally.

[0264] Therefore, when the matching parameter corresponding to a character position is less than the matching parameter threshold, the first label associated with the character position is used to indicate that the degree of match with another summary text is low, and the character at the character position corresponding to the first label has more learning value; when the matching parameter corresponding to the character position is not less than the matching parameter threshold, the second label associated with the character position is used to indicate that the degree of match with another summary text is high, and the character at the character position corresponding to the second label has low learning value.

[0265] In the above method, the label associated with the character position is determined based on the relationship between the matching parameter corresponding to the character position and the matching parameter threshold, thereby achieving differentiated processing of characters at different character positions. Through the above method, the large language model can focus on characters with high learning value during training, thereby improving training efficiency and model accuracy.

[0266] In one possible implementation, the computer device determines the weight coefficient of the character position as the preset weight coefficient corresponding to the label based on the label associated with the character position. The label includes a first label and a second label, and the preset weight coefficient corresponding to the first label is greater than the preset weight coefficient corresponding to the second label. That is, the character corresponding to the first label has a greater influence on the value of the third loss function than the second label, and the learning value of the character corresponding to the first label is higher than that of the second label. For example, the preset weight coefficient corresponding to the first label is set to 0.9, and the preset weight coefficient corresponding to the second label is set to 0.1.

[0267] In the above method, the computer device determines the weight coefficient of the character position through the label associated with each character position, which not only reduces the data processing steps and improves the training efficiency, but also facilitates the large language model to focus on the characters at the character positions with large weight coefficients during the training process, thereby improving the accuracy of the model.

[0268] In a possible implementation, the computer device determines a matching parameter threshold based on matching parameters of multiple characters in the sentence. Accordingly, the computer device sorts the matching parameters corresponding to multiple character positions in the summary text in the order of matching parameters from large to small; the computer device determines the matching parameter threshold based on the matching parameters at each ranking after sorting, and the matching parameter threshold is the matching parameter at a preset ranking after sorting. For example, in the case of sorting five matching parameters from large to small, the matching parameter threshold is the matching parameter at the second or third position after sorting; or the matching parameter threshold is the average of the matching parameters at the second and third positions after sorting.

[0269] In the above manner, the computer device determines the matching parameter threshold based on the sorted matching parameters instead of using a fixed preset threshold, thereby improving the flexibility of processing and facilitating the computer device to re-determine the matching parameter threshold when the matching parameters change, thereby re-determining the label associated with each character position, thereby improving the accuracy and applicability of classification. It should be noted that the above manner is only an exemplary description of determining the matching parameter threshold, and the matching parameter threshold can also be determined manually or by other means, without limitation.

[0270] Mode B: The computer device determines the weight coefficient of the character position based on the matching parameter corresponding to the character position.

[0271] In one possible implementation, the computer device determines a weight coefficient of the character position based on a difference between a matching parameter corresponding to the character position and 1, and the weight coefficient of the character position is a target power of the difference. That is, the weight coefficient of the character position is 1 minus the matching parameter of the character position. The target power corresponds to a preset exponent for scale stretching. The preset exponent can be adjusted by itself, such as setting the preset exponent to 0.5 or 2.

[0272] In the above method, since the weight coefficient of the character position is dynamically determined based on the matching parameters corresponding to the character position, the weight coefficient can more accurately reflect the importance and learning value of the characters at different character positions, making it easier for the large language model to focus on characters with higher learning value during training, thereby improving the accuracy of the model.

[0273] 806. The computer device determines an intermediate product value corresponding to the character position based on the probability corresponding to the character position in the prediction summary information and the weight coefficient of the character position, where the intermediate product value corresponding to the character position is the product value of the probability corresponding to the character position and the weight coefficient of the character position.

[0274] In an embodiment of the present application, the prediction summary information includes the probability of each character in the multiple characters at each character position. The probability of each character in the multiple characters appearing at a certain character position can be obtained by predicting the summary information, and then the probability of the current character appearing at the character position can be determined, that is, the probability corresponding to the character position. The probabilities corresponding to different character positions are adjusted by the weight coefficient of each character position, so that the characters at different character positions have different degrees of influence on the third loss function, which facilitates the large language model to focus on learning the characters at the positions with larger weight coefficients, thereby improving the accuracy of the model.

[0275] For example, the dictionary of a large language model contains 6000 characters, there are 30 character positions, and the prediction summary information is a matrix with a dimension of 30x6400. The prediction summary information can be used to obtain a matrix with a dimension of 1x6400 corresponding to a certain character position. The matrix is ​​used to represent the probability of each of the 6400 characters appearing at the character position. From this, it is determined that the sequence number corresponding to the character at the current character position is 6000. Then, the probability corresponding to the sequence number 6000 in the 1x6400 matrix is ​​the probability corresponding to the character position.

[0276] 807. The computer device determines the probability of the large language model predicting the summary text based on the intermediate product values ​​corresponding to the multiple character positions in the summary text, and obtains a first probability group, where the first probability group includes a first probability and a second probability, the first probability is the probability of the large language model predicting the first summary text, and the second probability is the probability of the large language model predicting the second summary text.

[0277] In the embodiment of the present application, the probability of the large language model predicting the summary text is the sum of the intermediate product values ​​corresponding to multiple character positions in the summary text. The calculation method of the probability of the large language model predicting the summary text is shown in the following formula 13.

[0278] Formula 13:

[0279]

[0280] Where summ represents the summary text, t represents a character position in the summary text, N1 represents the number of character positions contained in the summary text, id_t represents the sequence number of the character at the character position in the dictionary of the large language model, and s t Indicates the matching parameter of the character position, p is the preset index. (1-s t ) p Indicates the weight coefficient of the character position, l id_t Indicates the probability corresponding to the character position.

[0281] In the above method, the computer device predicts the probability of the character at each character position and the weight coefficient of the character position through the large language model, and determines the probability of the large language model predicting the summary text. Since characters at different character positions may correspond to different importance or information, by setting a weight coefficient for each character position, the large language model can focus on and learn characters with high weight coefficients during training, thereby improving training efficiency and model accuracy.

[0282] The above steps 804 to 807 are an exemplary manner in which the computer device determines the first probability group based on the predicted summary information, the first summary text, and the second summary text.

[0283] 808. The computer device determines the probability of the reference language model predicting the summary text based on the corresponding probabilities of multiple character positions in the summary text in the reference summary information, and obtains a second probability group, where the second probability group includes a third probability and a fourth probability, the third probability is the probability of the reference language model predicting the first summary text, and the fourth probability is the probability of the reference language model predicting the second summary text.

[0284] In the embodiment of the present application, the reference summary information is the output information of the reference language model having the same structure as the large language model. The probability of the reference language model predicting the summary text is the sum of the probabilities corresponding to multiple character positions in the summary text. Since the format of the reference summary information and the preset summary information are the same, the way in which the computer device determines the probability corresponding to each character position in the summary text in the reference information in step 808 is the same as the way in which the computer device determines the probability corresponding to each character position in the summary text in the preset reference information in step 806, and will not be repeated here.

[0285] In the above method, since the model parameters of the reference language model do not need to be trained and adjusted, the computer device does not need to set weight coefficients for each character position when determining the probability of the reference language model predicting the summary text, so that the large language model can be focused on learning. The above method reduces the complexity of calculation and improves training efficiency.

[0286] The above step 808 is an exemplary manner in which the computer device determines the second probability group based on the reference summary information, the first summary text, and the second summary text.

[0287] 809. The computer device determines a third loss function value based on the first probability group and the second probability group, where the third loss function value is used to indicate the ability gap between the large language model and the reference language model to distinguish the first summary text and the second summary text.

[0288] In the embodiment of the present application, the calculation method of the third loss function value is shown in the following formula fourteen.

[0289] Formula 14:

[0290]

[0291] Among them, y w is the first summary text, that is, summ1; y l is the second summary text, that is, summ2. θ represents a large language model, π ref represents the reference model. logπ θ (y w |x) represents the first probability, π θ (y l |x) represents the second probability, π ref (y w |x) represents the third probability, π ref (y l |x) represents the fourth probability. β is an adjustment parameter, such as being set to 0.1.

[0292] In a possible implementation, the computer device determines the third loss function value through multiple probability values. Accordingly, the computer device determines the first intermediate difference and the second intermediate difference based on the first probability group and the second probability group, the first intermediate difference being the difference between the logarithm of the first probability and the logarithm of the second probability, and the second intermediate difference being the difference between the logarithm of the third probability and the logarithm of the fourth probability; the computer device determines the third loss function value based on the difference between the first intermediate difference and the second intermediate difference, the third loss function value being the negative logarithm of the difference.

[0293] The first intermediate difference can represent the ability gap between the large language model and the second summary text, and the second intermediate difference can represent the ability gap between the reference language model and the first summary text. The third loss function value is the difference between the first intermediate difference and the second intermediate difference, that is, the third loss function value can represent the ability gap between the large language model and the reference language model to predict the summary text.

[0294] The calculation formula for the third loss function value can be divided into the following two formulas: Formula 15 and Formula 16.

[0295] Formula 15:

[0296] L dpo = -logσ(β*logits)

[0297] Formula 16:

[0298]

[0299] Among them, logits is used to represent the difference between the first middle difference and the second middle difference.

[0300] In the above method, since the first intermediate difference and the second intermediate difference respectively reflect the ability of the two models to predict different summary texts, the third loss function value determined based on the above two intermediate differences can reflect the ability gap between the two models to predict different summary texts. In other words, the computer device minimizes the third loss function value so that logits gradually become larger, that is, the gain of the first summary text to the model training in the large language model is greater than the gain of the first summary text to the model training in the reference language model, so that the large language model gradually predicts the first summary text with a greater probability than the reference language model. Since the first summary text has a better description effect on the sample text than the second summary text, the large language model can perform reinforcement learning on the better first summary text through the above method, thereby improving the description accuracy of the large language model.

[0301] It should be noted that the above method simultaneously determines the matching parameters of the character positions in the first summary text and the second summary text. Since the training goal of the large language model is to focus on learning the first summary text, only the matching parameters of each character position in the first summary text can be determined, and then weighted summation can be performed; for the probability corresponding to each character position in the second summary text, it is sufficient to directly sum it up without determining the matching parameters of each character position. Through the above method, the amount of calculation is reduced, computing resources are saved, and training efficiency is improved.

[0302] 810. The computer device trains the large language model based on the first loss function value, the second loss function value, and the third loss function value.

[0303] In an embodiment of the present application, the computer device performs weighted summation on the first loss function value, the second loss function value, and the third loss function value to obtain the sum of the loss function values. The gradient of the model parameters in the large language model is determined based on the sum of the loss function values. Based on the gradient of the model parameters, the model parameters of the large language model are updated.

[0304] Since the computer device can improve the ability of the large language model to distinguish different summary texts by minimizing the third loss function during the training process, the large language model can focus on learning a more comprehensive and accurate first summary text during this process. By using the loss function value and the third loss function value during the retraining process, the large language model can achieve different training goals and improve the accuracy of the model.

[0305] In the process of training a large language model using multiple sample texts, multiple training data are organized into the following data format.

[0306] {"prompt":"The following is a script of a scene in a play. Please extract the plot summary of this scene and describe the specific events that happened in the scene. Do not make guesses or abstract descriptions. The script is xxx","prompt_id":"0001",

[0307] "chosen":[{"content":"The following is a script of a scene in a play. Please extract the plot summary of this scene and describe the specific events that happened in the scene. Do not make guesses or abstract descriptions. The script is xxx","role":"user"},{"content":A,"role":"assistant"}],

[0308] "rejected":[{"content":"The following is a script of a scene in a play. Please extract the plot summary of this scene and describe the specific events that happened in the scene. Do not make guesses or abstract descriptions. The script is xxx","role":"user"},{"content":"B","role":"assistant"}],

[0309] "messages":[{"content":"The following is a script of a scene in a play. Please extract the plot summary of this scene and describe the specific events that happened in the scene. Do not make guesses or make abstract descriptions. The script is xxx","role":"user"},{"content":"A","role":"assistant"}]}

[0310] Among them, "Script is xxx" represents the sample text, prompt_id represents the serial number of the training data. A represents the first summary text yw, and B represents the second summary text yl. role user represents inputting the text into the corresponding language model. After the above training data is input into the large language model, the computer device determines the third loss function value through the predicted summary information or reference summary information output by the language model.

[0311] Since the reference language model is used to assist the large language model in training in the embodiment of the present application, the training process of the large language model in the embodiment of the present application is as follows: the model parameters of the network model that has been subjected to SFT (Supervised Fine-Tuning) are used, and the model parameters of the initialized large language model and the reference language model are used, and the model structures of the network model, the large language model and the reference language model are the same; the learning parameters of the large language model are set, and full parameter fine-tuning can be used when setting the learning parameters, that is, all parameters of the large language model are fine-tuned. In the process of training the large language model, multiple training data are obtained, and the number of training data is Q, and each bs training data is used as a training batch, then there are a total of P = Q / bs training batches, and each completion of P training batches indicates the completion of a round of iteration. The large language model is trained through multiple rounds of iterations until the mean of the third loss function values ​​of multiple training batches in each round of iteration no longer decreases; or, when the mean of the third loss function values ​​of multiple training batches in each round of iteration is less than the threshold, it means that the current third loss function value meets the requirements. If the mean of the loss function values ​​also meets the requirements at this time, stop training the large language model; if the mean of the loss function values ​​still does not meet the requirements at this time, stop using the third loss function value to train the large language model, and continue to use the loss function value to train the large language model. In each round of iteration, for any training batch, the sum of the third loss function values ​​corresponding to multiple sample texts in the training batch is determined in accordance with the above steps 804 to 809 as the third loss function value corresponding to the training batch. Based on the third loss function value corresponding to the training batch, the model parameter gradient of the large language model is determined, and based on the model parameter gradient, the model parameters of the large language model are updated. At this time, a model parameter update of the large language model is completed based on the third function value of the training batch.

[0312] To facilitate the description of the method for determining the third loss function value in the embodiment of the present application, see Fig. 9 As shown, Fig. 9 It is another schematic diagram of determining the loss function value provided by an embodiment of the present application. The computer device predicts the sample text through the large language model and the reference language model, respectively, and obtains the predicted summary information and the reference summary information, respectively. The computer device determines the label of each character position based on the first summary text, the second summary text and the predicted summary information, as shown in step 805. The computer device determines the probability of the large language model and the reference language model predicting the summary text, respectively, as shown in steps 804 to 808. The computer device determines the third loss function value based on the above probability, as shown in step 809.

[0313] To facilitate the description of the architecture of training a large language model in the embodiments of the present application, see Fig.10 As shown, Fig.10 It is another architecture diagram of training a large language model provided by an embodiment of the present application. When the large language model completes the prediction, the computer device determines the first probability and the second probability through the first summary text, the second summary text and the predicted probability information, as shown in steps 804 to 807. In this process, the computer device determines the label of each character position based on the first summary text, the second summary text and the predicted summary information, as shown in step 805. The computer device determines the third probability and the fourth probability based on the first summary text, the second summary text and the reference summary information, as shown in step 808. The computer device determines the third loss function value based on the above probabilities, as shown in step 809.

[0314] The embodiment of the present application provides a method for training a large language model. By introducing the first summary text and the second summary text in the training process of the large language model, more learnable information is provided for model training, which facilitates comparative learning. At the same time, by performing multiple character classifications on the first summary text with better description effect, that is, determining the characters with greater training gain for the current large language model from the first summary text and the second summary text, and setting different weight coefficients, the training process of the large language model is more targeted, and the accuracy of the model is improved.

[0315] In this process, the embodiment of the present application is based on the dynamic prediction performance in the model learning process. In two summary texts used for relative comparison or semantically misaligned, vocabulary mining is performed on the summary text with better description effect to determine the vocabulary that is more valuable to the current model, that is, to determine the characters corresponding to the first label. Then, by focusing on learning valuable vocabulary and suppressing common words that have been learned well, that is, suppressing the characters corresponding to the second label, the emphasis on learning valuable vocabulary is achieved, thereby optimizing the existing reinforcement learning strategy. In the above manner, the model's learning between the two contrasting summary texts is concentrated on valuable vocabulary, rather than a large number of common words that have been learned well, so that the model learning is more targeted, so that there are more valuable vocabulary in the summary prediction results output by the model, which improves the knowledge correctness of the summary prediction results and improves the accuracy of the model description.

[0316] It should be noted that the large language model obtained by the training method of this application can be applied in a variety of scenarios, and the example of using a large language model to perform a script understanding task is used for illustration. The following introduces two application scenarios of the large language model.

[0317] The trained large language model can be applied in the scenario of script summary. The summary texts of multiple scenes in the script text are connected in series to obtain the plot direction corresponding to the scene. Optionally, the plot direction is expressed as an event context, or the plot direction is expressed as an event context and emotional ups and downs prompts accompanying the event context, thereby providing a basis for quickly understanding the main plot of the script.

[0318] See also Fig.11 As shown, Fig.11 is a schematic diagram of a model application scenario provided in an embodiment of the present application. Fig.11 As shown in Figure (a), in response to the input operation of the script text, the computer device splits the content of each scene in the script text to obtain the script text of each scene. The computer device generates a summary text corresponding to the script text of each scene through the trained large language model. The computer device identifies the emotions of the main characters (male protagonist and female protagonist) in the script text of each scene, and obtains the emotional score form corresponding to the scene, where positive emotions are set as positive scores and negative emotions are set as negative scores. According to the emotional scores, the scene curve is determined and the summary text of each scene is marked to obtain the summarized summary data, thereby obtaining the summary data combining emotions and plots. See Fig.11 As shown in the middle (b) figure, the scene curve chart shows the event context line, the scene summary text and the emotional fluctuation prompt.

[0319] The trained large language model can be applied to content planning scenarios. In the planning stage of movies, TV series or other media projects, the trained large language model can generate summary text of the script for each scene, thus providing a basis for planners to quickly understand the main content of a large number of scripts and make effective screening and decisions. For example, it can assist planners in selecting important scenes to retain or shoot.

[0320] See also Fig.11 As shown in Figure (c), the computer device responds to the input operation of the script text, obtains the content of each scene in the script text, and obtains the script text of each scene. The computer device generates a summary text corresponding to the script text of each scene through the trained large language model, so as to understand the plot and characters. In response to the confirmation and screening operation, the computer device screens the plot characters and scenes, so as to determine the scenes that meet the screening requirements.

[0321] Fig.12 Schematic diagram of a large language model training device provided in an embodiment of the present application. Fig.12 The device includes: a prediction module 1201, a first acquisition module 1202, a first determination module 1203 and a training module 1204.

[0322] Prediction module 1201, used for inputting the sample text into the large language model to obtain prediction summary information, the prediction summary information including the probability of each character in the plurality of characters at each character position in the prediction summary text of the sample text;

[0323] A first acquisition module 1202 is used to acquire a first summary text of the sample text and a second summary text of the sample text, wherein the first summary text contains more information than the second summary text, and the first summary text includes correct sentences and incorrect sentences, wherein the correct sentences are sentences that correctly reflect the sample text, and the incorrect sentences are sentences that incorrectly reflect the sample text;

[0324] A first determination module 1203 is used to determine a loss function value based on the prediction summary information, the correct sentences in the first summary text, the wrong sentences in the first summary text, the second summary text, and the prediction latent vectors corresponding to the multiple characters;

[0325] The training module 1204 is used to train the large language model based on the loss function value.

[0326] In one possible implementation, the first determination module 1203 is used to determine an average loss function value of correct sentences in the first summary text and an average loss function value of incorrect sentences in the first summary text based on the predicted summary information, correct sentences in the first summary text, and incorrect sentences in the first summary text; obtain a first loss function value based on the average loss function value of the correct sentences in the first summary text and the average loss function value of the incorrect sentences in the first summary text, the first loss function value being a weighted sum of the average loss function value of the correct sentences in the first summary text and the average loss function value of the incorrect sentences in the first summary text; determine a second loss function value based on the first summary text, the second summary text, and the predicted latent vectors corresponding to multiple characters, the second loss function value being used to indicate the similarity between the predicted latent vector of the first summary text and the predicted latent vector of the second summary text.

[0327] In a possible implementation, the first determination module 1203 is further used to determine the loss function value of each sentence in the first summary text based on the predicted summary information, the loss function value of any sentence being the average of the cross entropy losses of multiple characters in the sentence; based on the loss function value of each sentence in the first summary text, determine the average loss function value of the correct sentences in the first summary text and the average loss function value of the incorrect sentences in the first summary text, the average loss function value of the correct sentences in the first summary text being the average of the loss function values ​​of multiple correct sentences in the first summary text, and the average loss function value of the incorrect sentences in the first summary text being the average of the loss function values ​​of multiple incorrect sentences in the first summary text.

[0328] In a possible implementation, the first summary text further includes an inaccurate sentence, where the inaccurate sentence is a sentence that inaccurately reflects the sample text.

[0329] In one possible implementation, the first determination module 1203 is configured to determine a first intermediate ratio based on the loss function value of each sentence in the first summary text of multiple sample texts in the current training batch, where the first intermediate ratio is the ratio of the sum of the loss function values ​​of multiple correct sentences in the first summary text of the multiple sample texts to the sum of the loss function values ​​of multiple sentences in the first summary text of the multiple sample texts; determine a weight coefficient of the correct sentence in the first summary text based on the first intermediate ratio, where the weight coefficient of the correct sentence in the first summary text is a preset power of the first intermediate ratio; determine a weight coefficient of an incorrect sentence in the first summary text based on the weight coefficient of the correct sentence in the first summary text, where the weight coefficient of the incorrect sentence in the first summary text is the difference between the weight coefficient of the correct sentence in the first summary text and 1.

[0330] In a possible implementation, the first determining module 1203 is further used to determine a second intermediate ratio and a third intermediate ratio based on the loss function value of each sentence in the first summary text of the multiple sample texts in the current training batch, the second intermediate ratio being the ratio of the sum of the loss function values ​​of the multiple correct sentences in the first summary text of the multiple sample texts to the sum of the loss function values ​​of the multiple sentences in the first summary text of the multiple sample texts, and the third intermediate ratio being the ratio of the sum of the loss function values ​​of the multiple inaccurate sentences in the first summary text of the multiple sample texts to the sum of the loss function values ​​of the multiple sentences in the first summary text of the multiple sample texts; based on the second intermediate ratio and the third intermediate ratio three intermediate ratios, determine the weight coefficients of correct sentences in the first summary text and the weight coefficients of inaccurate sentences in the first summary text, the weight coefficients of correct sentences in the first summary text are the preset power of the second intermediate ratio, and the weight coefficients of inaccurate sentences in the first summary text are the preset power of the third intermediate ratio; based on the weight coefficients of correct sentences in the first summary text and the weight coefficients of inaccurate sentences in the first summary text, determine the weight coefficients of erroneous sentences in the first summary text, the weight coefficients of erroneous sentences in the first summary text are the difference between the sum of the weight coefficients of correct sentences in the first summary text and the weight coefficients of inaccurate sentences in the first summary text and 1.

[0331] In a possible implementation, the first determination module 1203 is further used to determine the predicted latent vector of the first summary text and the predicted latent vector of the second summary text based on the predicted latent vectors corresponding to multiple characters, multiple characters in the first summary text, and multiple characters in the second summary text; pool the predicted latent vector of the first summary text and the predicted latent vector of the second summary text respectively to obtain a first intermediate vector and a second intermediate vector, and the dimensions of the first intermediate vector and the second intermediate vector are the same; based on the first intermediate vector and the second intermediate vector, determine a second loss function value, and the second loss function value is the distance between the first intermediate vector and the second intermediate vector.

[0332] In one possible implementation, the training module 1204 is used to perform a weighted summation of the first loss function value and the second loss function value to obtain a total loss function value; and train the large language model based on the total loss function value.

[0333] In one possible implementation, the first acquisition module 1202 is used to input the sample text and the first prompt text into the first additional model to obtain a first summary text, wherein the first prompt text is used to instruct the first additional model to generate a summary text containing supplementary information based on the sample text; and input the sample text and the second prompt text into the first additional model to obtain a second summary text, wherein the second prompt text is used to instruct the first additional model to generate a summary text based on the sample text.

[0334] In one possible implementation, the first acquisition module 1202 is further used to input the first summary text into the second additional model to obtain a label for each sentence in the first summary text, where the label of each sentence is used to indicate the accuracy of the sentence in reflecting the sample text; and input the second summary text into the second additional model to obtain a label for each sentence in the second summary text.

[0335] In another possible implementation, Fig.13 is a structural diagram of another large language model training device provided in an embodiment of the present application. Fig.13 , the device further comprises:

[0336] A second determination module 1205 is used to determine a third loss function value based on the predicted summary information, the first summary text, the second summary text, and the reference summary information, where the reference summary information is output information of a reference language model having the same structure as the large language model, and the third loss function value is used to indicate a capability gap between the large language model and the reference language model in distinguishing the first summary text from the second summary text;

[0337] The training module 1204 is further used to train the large language model based on the loss function value and the third loss function value.

[0338] In a possible implementation, the second determination module 1205 is configured to determine a first probability group and a second probability group based on the predicted summary information, the first summary text, the second summary text, and the reference summary information, the first probability group including a first probability and a second probability, the first probability being the probability of the large language model predicting the first summary text, the second probability being the probability of the large language model predicting the second summary text, the second probability group including a third probability and a fourth probability, the third probability being the probability of the reference language model predicting the first summary text, and the fourth probability being the probability of the reference language model predicting the second summary text; based on the first probability group and the second probability group, determine a first intermediate difference and a second intermediate difference, the first intermediate difference being the difference between the logarithm of the first probability and the logarithm of the second probability, the second intermediate difference being the difference between the logarithm of the third probability and the logarithm of the fourth probability; based on the difference between the first intermediate difference and the second intermediate difference, determine a third loss function value, the third loss function value being the negative logarithm of the difference.

[0339] In one possible implementation, the second determination module 1205 is further used to obtain, for any summary text in the first summary text and the second summary text, the probability of a character at any character position in the summary text in the predicted summary information; determine an intermediate product value corresponding to the character position based on the probability corresponding to the character position and the weight coefficient of the character position, the intermediate product value corresponding to the character position being the product value of the probability corresponding to the character position and the weight coefficient of the character position; determine the probability of the large language model predicting the summary text based on the intermediate product values ​​corresponding to multiple character positions in the summary text, the probability of the large language model predicting the summary text being the sum of the intermediate product values ​​corresponding to multiple character positions in the summary text.

[0340] In one possible implementation, the second determination module 1205 is further used to obtain, for any summary text in the first summary text and the second summary text, the probability of a character at any character position in the summary text corresponding to the reference summary information; based on the probabilities corresponding to multiple character positions in the summary text, determine the probability of the summary text predicted by the reference language model, and the probability of the summary text predicted by the reference language model is the sum of the probabilities corresponding to multiple character positions in the summary text.

[0341] In one possible implementation, the second determination module 1205 is further used to obtain a matching parameter corresponding to the character position, the matching parameter corresponding to the character position is used to indicate the degree of matching between the character at the character position in the summary text and another summary text in the first summary text and the second summary text; based on the difference between the matching parameter corresponding to the character position and 1, determine the weight coefficient of the character position, the weight coefficient of the character position being the target power of the difference.

[0342] In one possible implementation, the second determination module 1205 is also used to determine the weight coefficient of the character position based on the label associated with the character position, where the weight coefficient of the character position is a preset weight coefficient corresponding to the label of the character position, the label is a first label or a second label, and the preset weight coefficient corresponding to the first label is greater than the preset weight coefficient corresponding to the second label.

[0343] In one possible implementation, the second determination module 1205 is further used to obtain a matching parameter corresponding to the character position, where the matching parameter corresponding to the character position is used to indicate the degree of matching between the character at the character position in the summary text and another summary text in the first summary text and the second summary text; when the matching parameter corresponding to the character position is less than a matching parameter threshold, the label associated with the character position is determined to be the first label; when the matching parameter corresponding to the character position is not less than the matching parameter threshold, the label associated with the character position is determined to be the second label.

[0344] In one possible implementation, the second determination module 1205 is further used to sort the matching parameters corresponding to multiple character positions in the summary text in descending order of the matching parameters; based on the matching parameters at each ranking after sorting, determine the matching parameter threshold, and the matching parameter threshold is the matching parameter at a preset ranking after sorting.

[0345] In one possible implementation, the second determination module 1205 is further used to determine, for the sentence at the character position in the summary text, a matching sentence of the sentence in another summary text, the similarity between the matching sentence and the sentence being greater than the similarity between other sentences in the other summary text; determine a matching parameter corresponding to the character position based on the matching character in the matching sentence and the character at the character position, the similarity between the matching character and the character at the character position being greater than the similarity between other characters in the matching sentence and the character at the character position, and the matching parameter corresponding to the character position is the similarity between the character at the character position and the matching character.

[0346] The embodiment of the present application provides a training device for a large language model. By introducing a first summary text and a second summary text in the training process of the large language model, more learnable information is provided for model training. At the same time, since the first summary text and the second summary text contain different amounts of information, and the first summary text contains correct sentences and incorrect sentences, by comparing and learning two different summary texts of the same sample text, and distinguishing and learning the correct sentences and incorrect sentences in the first summary text, problems such as model overfitting and inaccurate generation due to a single summary text are avoided, which not only improves the generalization performance of the model, but also improves the accuracy of the model.

[0347] It should be noted that the training device for a large language model provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal is divided into different functional modules to complete all or part of the functions described above. In addition, the training device for a large language model and the training method for a large language model provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0348] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the large language model training method provided in the above embodiment.

[0349] Optionally, the computer device is provided as a terminal. Fig.14It is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. Generally, the terminal 1400 includes: a processor 1401 and a memory 1402 .

[0350] The processor 1401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In one possible implementation, the processor 1401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1401 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0351] Memory 1402 may include one or more computer-readable storage media, which may be non-transitory. Memory 1402 may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In one possible implementation, the non-transitory computer-readable storage medium in memory 1402 is used to store at least one computer program, which is used by processor 1401 to implement the training method of the large language model provided in the method embodiment of the present application.

[0352] In a possible implementation, the terminal 1400 may further optionally include: a peripheral device interface 1403 and at least one peripheral device. The processor 1401, the memory 1402 and the peripheral device interface 1403 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1403 via a bus, a signal line or a circuit board. Optionally, the peripheral device includes: at least one of a radio frequency circuit 1404, a display screen 1405, a camera assembly 1406, an audio circuit 1407 and a power supply 1408.

[0353] The peripheral device interface 1403 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1401 and the memory 1402. In one possible implementation, the processor 1401, the memory 1402, and the peripheral device interface 1403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1401, the memory 1402, and the peripheral device interface 1403 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0354] The radio frequency circuit 1404 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1404 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1404 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1404 can communicate with other devices through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In one possible implementation, the radio frequency circuit 1404 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0355] The display screen 1405 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1405 is a touch display screen, the display screen 1405 also has the ability to collect touch signals on the surface or above the surface of the display screen 1405. The touch signal can be input to the processor 1401 as a control signal for processing. At this time, the display screen 1405 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In one possible implementation, the display screen 1405 can be one, arranged on the front panel of the terminal 1400; in other embodiments, the display screen 1405 can be at least two, respectively arranged on different surfaces of the terminal 1400 or in a folding design; in other embodiments, the display screen 1405 can be a flexible display screen, arranged on a curved surface or a folding surface of the terminal 1400. Even, the display screen 1405 can also be arranged as a non-rectangular irregular figure, that is, a special-shaped screen. The display screen 1405 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0356] The camera component 1406 is used to capture images or videos. Optionally, the camera component 1406 includes a front camera and a rear camera. The front camera is arranged on the front panel of the terminal 1400, and the rear camera is arranged on the back of the terminal 1400. In one possible implementation, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In one possible implementation, the camera component 1406 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0357] The audio circuit 1407 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1401 for processing, or input them into the radio frequency circuit 1404 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 1400. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1401 or the radio frequency circuit 1404 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In one possible implementation, the audio circuit 1407 may also include a headphone jack.

[0358] The power supply 1408 is used to power various components in the terminal 1400. The power supply 1408 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1408 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0359] In a possible implementation, the terminal 1400 further includes one or more sensors 1409 . The one or more sensors 1409 include, but are not limited to: an acceleration sensor 1410 , a gyroscope sensor 1411 , a pressure sensor 1412 , an optical sensor 1413 , and a proximity sensor 1414 .

[0360] The acceleration sensor 1410 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 1400. For example, the acceleration sensor 1410 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1401 can control the display screen 1405 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1410. The acceleration sensor 1410 can also be used for collecting game or user motion data.

[0361] The gyro sensor 1411 can detect the body direction and rotation angle of the terminal 1400, and the gyro sensor 1411 can cooperate with the acceleration sensor 1410 to collect the user's 3D actions on the terminal 1400. The processor 1401 can implement the following functions based on the data collected by the gyro sensor 1411: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0362] The pressure sensor 1412 may be provided at the side frame of the terminal 1400 and / or the lower layer of the display screen 1405. When the pressure sensor 1412 is provided at the side frame of the terminal 1400, it may detect the user's holding signal of the terminal 1400, and the processor 1401 may perform left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 1412. When the pressure sensor 1412 is provided at the lower layer of the display screen 1405, the processor 1401 may control the operability control on the UI interface according to the user's pressure operation on the display screen 1405. The operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0363] The optical sensor 1413 is used to collect the ambient light intensity. In one embodiment, the processor 1401 can control the display brightness of the display screen 1405 according to the ambient light intensity collected by the optical sensor 1413. Optionally, when the ambient light intensity is high, the display brightness of the display screen 1405 is increased; when the ambient light intensity is low, the display brightness of the display screen 1405 is decreased. In another embodiment, the processor 1401 can also dynamically adjust the shooting parameters of the camera assembly 1406 according to the ambient light intensity collected by the optical sensor 1413.

[0364] The proximity sensor 1414, also called a distance sensor, is disposed on the front panel of the terminal 1400. The proximity sensor 1414 is used to collect the distance between the user and the front of the terminal 1400. In one embodiment, when the proximity sensor 1414 detects that the distance between the user and the front of the terminal 1400 is gradually decreasing, the processor 1401 controls the display screen 1405 to switch from the screen-on state to the screen-off state; when the proximity sensor 1414 detects that the distance between the user and the front of the terminal 1400 is gradually increasing, the processor 1401 controls the display screen 1405 to switch from the screen-off state to the screen-on state.

[0365] Those skilled in the art will understand that Fig.14 The structure shown in the figure does not constitute a limitation on the terminal 1400, and the terminal 1400 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0366] Optionally, the computer device is provided as a server. Fig.15It is a structural diagram of a server provided in an embodiment of the present application. The server 1500 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1501 and one or more memories 1502, wherein the memory 1502 stores at least one computer program, and the at least one computer program is loaded and executed by the processor 1501 to implement the training method of the large language model provided in the above-mentioned various method embodiments, or to implement the training method of the large language model provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for implementing device functions, which will not be repeated here.

[0367] The embodiment of the present application also provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is loaded and executed by a processor to implement the training method of the large language model in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0368] An embodiment of the present application also provides a computer program product, including a computer program, which is loaded and executed by a processor to implement the large language model training method provided in the above embodiment.

[0369] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program. The above program may be stored in a computer-readable storage medium, and the above storage medium may be a read-only memory, a disk or an optical disk, etc.

[0370] The above description is only an optional embodiment of the embodiments of the present application and is not intended to limit the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A method for training a large language model, characterized in that: The method comprises: Inputting the sample text into the large language model to obtain prediction summary information, wherein the prediction summary information includes the probability of each of the plurality of characters at each character position in the prediction summary text of the sample text; Obtaining a first summary text of the sample text and a second summary text of the sample text, wherein the first summary text contains more information than the second summary text, and the first summary text includes a correct sentence and an incorrect sentence, wherein the correct sentence is a sentence that correctly reflects the sample text, and the incorrect sentence is a sentence that incorrectly reflects the sample text; Determine a loss function value based on the predicted summary information, correct sentences in the first summary text, incorrect sentences in the first summary text, the second summary text, and predicted latent vectors corresponding to the multiple characters; The large language model is trained based on the loss function value.

2. The method according to claim 1, characterized in that The determining of the loss function value based on the predicted summary information, the correct sentences in the first summary text, the incorrect sentences in the first summary text, the second summary text, and the predicted latent vectors corresponding to the multiple characters includes: Determine, based on the predicted summary information, correct sentences in the first summary text, and incorrect sentences in the first summary text, an average loss function value of the correct sentences in the first summary text and an average loss function value of the incorrect sentences in the first summary text; Based on an average loss function value of correct sentences in the first summary text and an average loss function value of incorrect sentences in the first summary text, a first loss function value is obtained, where the first loss function value is a weighted sum of the average loss function value of correct sentences in the first summary text and the average loss function value of incorrect sentences in the first summary text; Based on the first summary text, the second summary text, and the predicted latent vectors corresponding to the multiple characters, a second loss function value is determined, where the second loss function value is used to indicate the similarity between the predicted latent vector of the first summary text and the predicted latent vector of the second summary text.

3. The method according to claim 2, characterized in that The determining, based on the predicted summary information, the correct sentences in the first summary text, and the erroneous sentences in the first summary text, an average loss function value of the correct sentences in the first summary text and an average loss function value of the erroneous sentences in the first summary text comprises: Based on the predicted summary information, determining a loss function value of each sentence in the first summary text, wherein the loss function value of any sentence is an average of cross entropy losses of multiple characters in the sentence; Based on the loss function value of each sentence in the first summary text, an average loss function value of correct sentences in the first summary text and an average loss function value of incorrect sentences in the first summary text are determined, the average loss function value of correct sentences in the first summary text being an average of loss function values ​​of multiple correct sentences in the first summary text, and the average loss function value of incorrect sentences in the first summary text being an average of loss function values ​​of multiple incorrect sentences in the first summary text.

4. The method according to claim 1, characterized in that: The first summary text also includes inaccurate sentences, where the inaccurate sentences are sentences that inaccurately reflect the sample text.

5. The method according to claim 1, characterized in that The method further comprises: Based on the loss function value of each sentence in the first summary text of the plurality of sample texts in the current training batch, determining a first intermediate ratio, the first intermediate ratio being a ratio of the sum of the loss function values ​​of the plurality of correct sentences in the first summary text of the plurality of sample texts to the sum of the loss function values ​​of the plurality of sentences in the first summary text of the plurality of sample texts; Based on the first intermediate ratio, determining a weight coefficient of a correct sentence in the first summary text, wherein the weight coefficient of the correct sentence in the first summary text is a preset power of the first intermediate ratio; Based on the weight coefficient of the correct sentence in the first summary text, the weight coefficient of the incorrect sentence in the first summary text is determined, and the weight coefficient of the incorrect sentence in the first summary text is the difference between the weight coefficient of the correct sentence in the first summary text and 1.

6. The method according to claim 4, characterized in that The method further comprises: Based on the loss function value of each sentence in the first summary text of the multiple sample texts in the current training batch, determine a second intermediate ratio and a third intermediate ratio, the second intermediate ratio being the ratio of the sum of the loss function values ​​of the multiple correct sentences in the first summary text of the multiple sample texts to the sum of the loss function values ​​of the multiple sentences in the first summary text of the multiple sample texts, and the third intermediate ratio being the ratio of the sum of the loss function values ​​of the multiple inaccurate sentences in the first summary text of the multiple sample texts to the sum of the loss function values ​​of the multiple sentences in the first summary text of the multiple sample texts; Based on the second intermediate ratio and the third intermediate ratio, determining a weight coefficient of a correct sentence in the first summary text and a weight coefficient of an inaccurate sentence in the first summary text, wherein the weight coefficient of the correct sentence in the first summary text is a preset power of the second intermediate ratio, and the weight coefficient of the inaccurate sentence in the first summary text is a preset power of the third intermediate ratio; Based on the weight coefficients of the correct sentences in the first summary text and the weight coefficients of the inaccurate sentences in the first summary text, a weight coefficient of the erroneous sentence in the first summary text is determined, the weight coefficient of the erroneous sentence in the first summary text being the difference between 1 and the sum of the weight coefficients of the correct sentences in the first summary text and the weight coefficients of the inaccurate sentences in the first summary text.

7. The method according to claim 2, characterized in that The determining a second loss function value based on the first summary text, the second summary text, and the predicted latent vectors corresponding to the plurality of characters includes: Determine a predicted latent vector of the first summary text and a predicted latent vector of the second summary text based on the predicted latent vectors corresponding to the multiple characters, multiple characters in the first summary text, and multiple characters in the second summary text; Pooling the predicted latent vector of the first summary text and the predicted latent vector of the second summary text respectively to obtain a first intermediate vector and a second intermediate vector, wherein the first intermediate vector and the second intermediate vector have the same dimension; Based on the first intermediate vector and the second intermediate vector, a second loss function value is determined, where the second loss function value is the distance between the first intermediate vector and the second intermediate vector.

8. The method according to claim 1, characterized in that The training of the large language model based on the loss function value includes: Taking a weighted sum of the first loss function value and the second loss function value to obtain a total loss function value; The large language model is trained based on the total loss function value.

9. The method according to claim 1, characterized in that: The step of obtaining a first summary text of the sample text and a second summary text of the sample text includes: Inputting the sample text and the first prompt text into the first additional model to obtain the first summary text, wherein the first prompt text is used to instruct the first additional model to generate a summary text containing supplementary information based on the sample text; The sample text and the second prompt text are input into the first additional model to obtain the second summary text, where the second prompt text is used to instruct the first additional model to generate a summary text based on the sample text.

10. The method according to claim 1, characterized in that The method further comprises: determining a third loss function value based on the predicted summary information, the first summary text, the second summary text, and reference summary information, wherein the reference summary information is output information of a reference language model having the same structure as the large language model, and the third loss function value is used to indicate a capability gap between the large language model and the reference language model in distinguishing the first summary text from the second summary text; The training of the large language model based on the loss function value includes: The large language model is trained based on the loss function value and the third loss function value.

11. The method according to claim 10, characterized in that The determining a third loss function value based on the prediction summary information, the first summary text, the second summary text, and the reference summary information includes: Determine a first probability group and a second probability group based on the predicted summary information, the first summary text, the second summary text, and the reference summary information, wherein the first probability group includes a first probability and a second probability, the first probability being the probability that the large language model predicts the first summary text, the second probability being the probability that the large language model predicts the second summary text, and the second probability group includes a third probability and a fourth probability, the third probability being the probability that the reference language model predicts the first summary text, and the fourth probability being the probability that the reference language model predicts the second summary text; Determine a first intermediate difference and a second intermediate difference based on the first probability group and the second probability group, the first intermediate difference being the difference between the logarithm of the first probability and the logarithm of the second probability, and the second intermediate difference being the difference between the logarithm of the third probability and the logarithm of the fourth probability; Based on the difference between the first intermediate difference and the second intermediate difference, the third loss function value is determined, and the third loss function value is the negative logarithm of the difference.

12. The method according to claim 11, characterized in that The step of determining the first probability group based on the predicted summary information, the first summary text, the second summary text and the reference summary information includes: For any summary text of the first summary text and the second summary text, obtaining a probability of a character at any character position in the summary text corresponding to the predicted summary information; Based on the probability corresponding to the character position and the weight coefficient of the character position, determine the intermediate product value corresponding to the character position, the intermediate product value corresponding to the character position being the product value of the probability corresponding to the character position and the weight coefficient of the character position; Based on the intermediate product values ​​corresponding to the multiple character positions in the summary text, the probability of the large language model predicting the summary text is determined, and the probability of the large language model predicting the summary text is the sum of the intermediate product values ​​corresponding to the multiple character positions in the summary text.

13. The method according to claim 11, characterized in that The step of determining the second probability group based on the predicted summary information, the first summary text, the second summary text and the reference summary information includes: For any summary text of the first summary text and the second summary text, obtaining a probability of a character at any character position in the summary text corresponding to the reference summary information; Based on the probabilities corresponding to multiple character positions in the summary text, the probability of the summary text predicted by the reference language model is determined, and the probability of the summary text predicted by the reference language model is the sum of the probabilities corresponding to multiple character positions in the summary text.

14. The method according to claim 12, characterized in that The method further comprises: Acquire a matching parameter corresponding to the character position, where the matching parameter corresponding to the character position is used to indicate a matching degree between the character at the character position in the summary text and the other summary text of the first summary text and the second summary text; Based on the difference between the matching parameter corresponding to the character position and 1, a weight coefficient of the character position is determined, and the weight coefficient of the character position is a target power of the difference.

15. The method according to claim 12, characterized in that The method further comprises: Based on the label of the character position, determine the weight coefficient of the character position, the weight coefficient of the character position is a preset weight coefficient corresponding to the label of the character position, the label is a first label or a second label, and the preset weight coefficient corresponding to the first label is greater than the preset weight coefficient corresponding to the second label.

16. The method according to claim 15, characterized in that The method further comprises: Acquire a matching parameter corresponding to the character position, where the matching parameter corresponding to the character position is used to indicate a matching degree between the character at the character position in the summary text and the other summary text of the first summary text and the second summary text; When the matching parameter corresponding to the character position is less than the matching parameter threshold, determining the label of the character position as the first label; When the matching parameter corresponding to the character position is not less than the matching parameter threshold, the label of the character position is determined to be the second label.

17. The method according to any one of claims 14 to 16, characterized in that: The obtaining of the matching parameter corresponding to the character position includes: For the sentence in the summary text where the character position is located, determining a matching sentence of the sentence in the other summary text, wherein the similarity between the matching sentence and the sentence is greater than the similarity between other sentences in the other summary text and the sentence; Based on the matching character in the matching statement and the character at the character position, a matching parameter corresponding to the character position is determined, the similarity between the matching character and the character at the character position is greater than the similarity between other characters in the matching statement and the character at the character position, and the matching parameter corresponding to the character position is the similarity between the character at the character position and the matching character.

18. A large language model training device, characterized in that: The device comprises: A prediction module, used for inputting a sample text into a large language model to obtain prediction summary information, wherein the prediction summary information includes the probability of each of a plurality of characters at each character position in the prediction summary text of the sample text; A first acquisition module is used to acquire a first summary text of the sample text and a second summary text of the sample text, wherein the first summary text contains more information than the second summary text, and the first summary text includes correct sentences and incorrect sentences, wherein the correct sentences are sentences that correctly reflect the sample text, and the incorrect sentences are sentences that incorrectly reflect the sample text; A first determination module, configured to determine a loss function value based on the predicted summary information, correct sentences in the first summary text, incorrect sentences in the first summary text, the second summary text, and predicted latent vectors corresponding to the multiple characters; A training module is used to train the large language model based on the loss function value.

19. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor and executes the large language model training method described in any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store at least one computer program, and the at least one computer program is used to execute the large language model training method described in any one of claims 1 to 17.

Images