A language generation model improvement method, system, device and storage medium

Abstract

The application discloses a language generation model improvement method, system, device and storage medium. When the method is executed, training text is acquired; the training text is input into a language generation model to determine output text; the output text is input into an evaluation model to determine evaluation information; and the language generation model is trained based on the output text and the evaluation information until the model converges and meets a preset performance requirement. On the basis of traditional standard answer supervision training, a preference evaluation model is introduced to evaluate the text generated by the language generation model. The final model training is iteratively completed based on the output text and the evaluation information of the text. The language generation model is enabled to realize self-iterative training, the model is encouraged to generate content from various angles, and the model is not only dependent on input training samples, so that the model has better generation quality and generalization.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a language generation model improvement method, system, device and storage medium. BACKGROUND

[0002] The traditional generation model generates the corresponding output text according to the input model training text. In the supervised training mode of the traditional generation model, the model calculates the loss function of the training text character by character according to the pre-set standard answer. Such a training process relies on limited training data and cannot select what word in the actual scenario is consistent with the context according to the actual situation, which may cause the output content to be inconsistent with the current context, so that the generalization and generation quality of the traditional generation model are poor.

[0003] Therefore, how to improve the generation quality of the generation model is a technical problem to be solved by those skilled in the art. SUMMARY

[0004] Based on the above problems, the present application provides a language generation model improvement method, system, device and storage medium to improve the generation quality of the generation model.

[0005] To solve the above problems, the technical scheme provided by the embodiments of the present application is as follows:

[0006] The first aspect of the present application provides a language generation model improvement method, comprising:

[0007] obtaining training text;

[0008] inputting the training text into a language generation model to determine an output text;

[0009] inputting the output text into an evaluation model to determine evaluation information;

[0010] training the language generation model based on the output text and the evaluation information until the model converges and meets the preset performance requirements.

[0011] Optionally, the output text is input into the evaluation model to determine the evaluation information.

[0012] segmenting the output text to obtain a plurality of words;

[0013] obtaining character-level evaluation information corresponding to each segmented word and sentence-level evaluation information of the output text.

[0014] Optionally, the training of the language generation model based on the output text and the evaluation information until the model converges and meets the preset performance requirements comprises:

[0015] obtaining a target text, the target text comprising words corresponding to the evaluation information meeting the preset condition;

[0016] mapping the obtained target text to obtain a mapping result corresponding to the output text;

[0017] determining a loss value of the training text based on the mapping result and the standard sample;

[0018] iteratively training the language generation model based on the calculated loss value until the model converges and meets a preset performance requirement.

[0019] Optionally, the mapping result is a word sequence corresponding to the target text, and the determining of the loss value of the training text based on the mapping result and the standard sample comprises:

[0020] obtaining a probability distribution corresponding to each word in the target text;

[0021] calculating the loss value corresponding to the training text based on the obtained probability distribution and the standard sample.

[0022] Optionally, the inputting of the training text into the language generation model to determine the output text comprises:

[0023] mapping each word included in the training text to obtain a word sequence corresponding to the training text;

[0024] predicting each word included in the training text based on the word sequence corresponding to the training text to obtain an output text corresponding to the word sequence.

[0025] The second aspect of the present application provides a language generation model improvement system, comprising:

[0026] a training text acquisition unit configured to acquire a training text;

[0027] an output text determination unit configured to input the training text into a language generation model to determine an output text;

[0028] an evaluation information determination unit configured to input the output text into an evaluation model to determine evaluation information;

[0029] a training unit configured to train the language generation model based on the output text and the evaluation information until the model converges and meets a preset performance requirement.

[0030] Optionally, the evaluation information determination unit is specifically configured to split the output text to obtain a plurality of words, and obtain character-level evaluation information corresponding to each split word and sentence-level evaluation information of the output text.

[0031] Optionally, the training unit comprises:

[0032] a target text acquisition unit, configured to acquire a target text, the target text comprising words corresponding to evaluation information meeting a preset condition;

[0033] a mapping unit, configured to map the acquired target text to obtain a mapping result corresponding to the output text;

[0034] a loss value determination unit, configured to determine a loss value of the training text based on the mapping result and a standard sample;

[0035] an iteration unit, configured to iteratively train the language generation model based on the calculated loss value until the model converges and meets a preset performance requirement.

[0036] The third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the language generation model improvement method of any one of the first aspect is implemented.

[0037] The fourth aspect of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores instructions, when the instructions are executed on a terminal device, the terminal device executes the language generation model improvement method of any one of the first aspect.

[0038] Compared with the prior art, the present application has the following beneficial effects:

[0039] The present application acquires a training text; inputs the training text into a language generation model to determine an output text; inputs the output text into an evaluation model to determine evaluation information; trains the language generation model based on the output text and the evaluation information until the model converges and meets a preset performance requirement. On the basis of traditional standard answer supervised training, a preference evaluation model is introduced to evaluate the text generated by the language generation model. The final model training is iteratively completed based on the output text and the evaluation information of the text. The language generation model is iteratively trained, the model is encouraged to generate content from multiple angles, and the model is not only dependent on the input training sample, so that the model has better generation quality and generalization. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to make the technical solutions in the embodiments or the prior art clearer, the accompanying drawings needed in the description of the embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings in the description are only some embodiments of the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative work based on the accompanying drawings also belong to the protection scope of the present application.

[0041] Figure 1 A flow chart of a language generation model improvement method provided by an embodiment of the present application;

[0042] Figure 2 An application schematic diagram of an evaluation model provided by an embodiment of the present application;

[0043] Figure 3 A model training schematic diagram provided by an embodiment of the present application;

[0044] Figure 4 A structure diagram of a language generation model improvement system provided by an embodiment of the present application;

[0045] Figure 5 A structure schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the technical solutions in the embodiments or the prior art clearer, the accompanying drawings needed in the description of the embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings in the description are only some embodiments of the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative work based on the accompanying drawings also belong to the protection scope of the present application.

[0047] In order to make the technical solutions in the embodiments or the prior art clearer, the accompanying drawings needed in the description of the embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings in the description are only some embodiments of the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative work based on the accompanying drawings also belong to the protection scope of the present application.

[0048] As described above, the traditional supervised training method of the generation model is to calculate the loss function loss according to the "standard answer" character by character, but the effective guidance of human preference is missing, and the training relies on limited training data. The supervised training method relies on labeled training data, and the data utilization is relatively low. The supervised training method cannot enable the model to iterate itself. The supervised training method makes the model learn to memorize the answer, that is, "rote memorization", which leads to poor generalization of the final model. The existing supervised training method lacks local attention, that is, it cannot effectively guide the model to learn what words are good and what words are bad in the current context. Adding a character-level evaluation model to guide the model and giving a character-level preference loss when training the generation model can effectively encourage the model to generate text that meets human expectations and avoid generating text that does not meet human expectations.

[0049] To solve the above technical problems, the embodiment of the present application provides a language generation model improvement method, which comprises: obtaining training text; inputting the training text into a language generation model to determine output text; inputting the output text into an evaluation model to determine evaluation information; training the language generation model based on the output text and the evaluation information until the model converges and meets the preset performance requirement. On the basis of the traditional standard answer supervised training, a preference evaluation model is introduced to evaluate the text generated by the language generation model. The final model training is iteratively completed based on the output text and the evaluation information of the text. The language generation model is enabled to realize self-iterative training, and the model is encouraged to generate content from multiple perspectives, not just relying on the input training samples, so that the model can have better generation quality and generalization.

[0050] In addition, the language generation model improvement method of the embodiment of the present application can not be limited to the execution subject, for example, the language generation model improvement method of the embodiment of the present application can be applied to a terminal device or a server and the like data processing device. The terminal device can be a smart phone, a computer, a personal digital assistant (PDA), a tablet computer and the like electronic device. The server can be a stand-alone server, a cloud server or a cluster server composed of multiple servers.

[0051] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0052] The following embodiment illustrates an improved language generation model method provided in this application. See also... Figure 1 The figure is a flowchart of a language generation model improvement method provided in an embodiment of this application. The execution subject of this method is a server, and further, the subject can be a model application system in the server. The method includes:

[0053] S101: Obtain training text.

[0054] The training text can be user-defined or historical text information from a historical database. For example, the training text could be "Will the weather be nice tomorrow?"

[0055] S102: Input the training text into the language generation model and determine the output text.

[0056] The training text obtained in the above process is input into the language generation model to obtain the output text generated by the language generation model. For text generation tasks, the probability distribution predicted by the model usually refers to the predicted probability distribution for the next word or character. In practical applications, the model predicts the next possible word or character based on the input text and assigns a probability value to each candidate. For example, given an input sentence "Today's weather", the model might predict that the next word is "very" with a probability of 0.4, "good" with a probability of 0.3, and "sunny" with a probability of 0.2. These probability values ​​are then normalized to obtain a probability distribution. This probability distribution can be used to select the next word or character to be generated, choosing the candidate with the highest probability as the generation result, or a sampling method can be used to randomly select the next word or character.

[0057] In one possible implementation, inputting the trained text into the language generation model and determining the output text includes:

[0058] Step A1: Map each word in the training text to obtain the word sequence corresponding to the training text.

[0059] In practical applications, the input data can first be preprocessed. This may include steps such as word segmentation, tokenization, and vocabulary construction. The text is broken down into tokens (e.g., words or sub-words) so that it can be processed by the model.

[0060] Step A2: Based on the word sequence corresponding to the training text, predict each word included in the training text to obtain the output text corresponding to the word sequence.

[0061] In an implementation scenario, taking a GPT model as a language generation model for example, the GPT (Generative Pre-trained Transformer) model is a language generation model based on the Transformer architecture. In the training of the GPT model, the language model is used, that is, the probability distribution of the text data is learned by predicting the next word, the input text is divided into a series of tokens (for example, words or subwords), and each token is mapped to a unique integer ID. In this way, the training text is represented as a token sequence. The training goal of the GPT model is to predict the probability distribution of each token, that is, given the previous token sequence, to predict the possibility of the next token. Therefore, the input is a token sequence, and the output is the prediction of the next token at each position in the token sequence. Based on the predicted output words corresponding to each word in the word sequence corresponding to the training text, the output text is composed in order, that is, the output text corresponding to the word sequence.

[0062] S103: inputting the output text into an evaluation model to determine evaluation information.

[0063] The evaluation model is trained using data with character-level and sentence-level labels, which can be constructed manually or using open-source error correction data, harmlessness judgment data, and the like. After training, the evaluation model can score each character in the output sentence and score the entire sentence. The evaluation model can be regarded as the C (critic) in the AC framework of reinforcement learning, which gives a score to the output (sampling) of A (actor, i.e., the generation model), and calculates the loss to complete the training of the generation model.

[0064] In a possible implementation, the step of inputting the output text into the evaluation model to determine the evaluation information includes steps B1-B2:

[0065] Step B1: splitting the output text to obtain a plurality of words.

[0066] For example, the output text "What did you just say" is split into six words tokens. In actual application scenarios, the text can also be split according to actual needs, for example, "just" and "just" are not split as a token "just" in the splitting process.

[0067] Step B2: obtaining character-level evaluation information corresponding to each split word and sentence-level evaluation information of the output text.

[0068] When the training text is "Will the weather be good tomorrow?", the output text is "What did you just say", which can be seen in Figure 2 ,Figure 2 An evaluation model application diagram is provided for the embodiments of the present application. Wherein, 1 or 0 represents the evaluation information, which can be composed of sentence-level preference evaluation and / or character-level preference evaluation.

[0069] S104: training the language generation model based on the output text and the evaluation information until the model converges and meets the preset performance requirements.

[0070] Based on the evaluation information of the evaluation model and the generated text probability, a loss function (such as cross-entropy loss) is calculated to optimize the parameters of the generation model. In addition, the score of the evaluation model may need to be properly mapped and adjusted to match the loss calculation of the generation model. The goal of this training method is to improve the quality and consistency of the generation model through reinforcement learning. By taking the evaluation model as a guide, the generation model can gradually learn better generation strategies. Through iterative training, the model will continuously adjust the parameters to minimize the loss function, thereby improving the quality of the generated content. Multiple rounds of iterative training are usually required until the model converges and meets the preset performance requirements. For the implementation process, please refer to Figure 3 , Figure 3 A model training diagram is provided for the embodiments of the present application.

[0071] In one possible implementation, the training of the language generation model based on the output text and the evaluation information until the model converges and meets the preset performance requirements includes steps C1-C4:

[0072] Step C1: obtaining a target text.

[0073] The target text includes words corresponding to the evaluation information that meets the preset conditions. For example, the words marked as "0" in the evaluation information can be obtained as the target text. That is, during the training process, the loss of the language generation model is focused on the part that does not meet the human expectation, and the part that meets the human expectation is not punished.

[0074] Step C2: mapping the obtained target text to obtain a mapping result corresponding to the output text.

[0075] The target text composed of several words obtained in the above process is mapped to obtain a mapping result corresponding to the target text, which is also the text corresponding to the current output text that needs to be calculated for subsequent loss. The mapping result can be a word sequence corresponding to the target text.

[0076] Step C3: determining the loss value of the training text based on the mapping result and the standard sample.

[0077] The standard sample is a standard answer in the current training process, which can be obtained by the user according to the output text and the user's own preference setting, or automatically generated by the system according to the user's preference.

[0078] Step C4: iteratively training the language generation model based on the calculated loss value until the model converges and meets the preset performance requirement.

[0079] The evaluation model can be regarded as the C (critic) of the AC framework in reinforcement learning, which gives a score to the output (sampling) of A (actor, i.e. the generation model). After calculating the loss, the training of the generation model is completed. The overall training can be performed in the reinforcement learning manner, such as the AC framework. First, the input of the training data is input into the generation model, and the generation model outputs the response text. The output text of the generation model is input into the trained evaluation model, and the evaluation model gives a score at the character level and the sentence segment level. After mapping the score, the loss is calculated with the probability of each character in the output text of the generation model. In actual application scenarios, the merged sentence segment evaluation information and the loss calculated above can also be combined to train the generation model.

[0080] In a possible implementation, the mapping result is a word sequence corresponding to the target text, and the loss value of the training text is determined based on the mapping result and the standard sample, including steps D1-D2:

[0081] Step D1: obtaining the probability distribution corresponding to each word in the target text.

[0082] In a possible implementation scenario, the text output by the generation model is converted into a probability distribution. The original score of each character output by the generation model can be converted into a probability using a softmax function.

[0083] Step D2: calculating the loss value corresponding to the training text based on the obtained probability distribution and the standard sample.

[0084] In one possible practical scenario, the loss value can be calculated using a cross-entropy loss function, which is a commonly used method to measure the difference between probability distributions, suitable for classification problems. In natural language processing, the cross-entropy loss function can be used to calculate the difference between the model's predicted probability distribution and the true next token. First, the true next token is one-hot encoded, represented as a vector. Assuming there are n possible tokens, the true next token can be represented as an n-dimensional vector, with only one position being 1 and the rest being 0, indicating the index position of the true token. Then, the model outputs a probability distribution vector, representing the predicted probability for each possible token. Assuming the model outputs a probability distribution vector p, which is also an n-dimensional vector. Next, the cross-entropy loss function is used to calculate the difference between the predicted result and the true next token. The cross-entropy loss function can be defined as:

[0085] loss = -sum(y×log(p)) (1)

[0086] In equation (1), y is the one-hot encoded vector of the true next token, log is the natural logarithm operation, * represents element multiplication, and sum represents summation. Finally, the differences at all positions are summed up as the final loss value. The model optimizes the predicted probability distribution by minimizing the cross-entropy loss function, so that the model can more accurately predict the next token.

[0087] In one possible practical scenario, a common method for calculating loss is to use the cross-entropy loss function. Given a word's probability distribution and the corresponding target word, the cross-entropy loss function can be used to measure the difference between the predicted word distribution and the target word distribution. Assuming there is a word probability distribution P, where P(i) represents the probability of the i-th word in the vocabulary, and the target word distribution Q, where Q(j) represents the probability of the target word being the j-th word. The cross-entropy loss function can be represented as:

[0088] L = -∑(Q(j)×log(P(j))) (2)

[0089] In equation (2), j represents each word in the vocabulary. For each training sample, the probability distribution P of each word can be calculated, and the target distribution Q can be obtained according to the target word. Then, P and Q are substituted into the above formula to calculate the loss L.

[0090] In practical applications, batch processing of multiple samples is usually used. For a batch of size N, the loss of each sample can be calculated, and the average value is taken as the final loss value.

[0091] It should be noted that when calculating the cross-entropy loss, there may be special characters (such as padding characters) in the generated model output. In order to avoid the influence of these special characters on the loss, a mask can be used to ignore the loss contribution of these characters.

[0092] The above are some specific implementation manners of the language generation model improvement method provided by the embodiments of the present application. Based on this, the present application also provides a corresponding system for improving the language generation model. The system provided by the embodiments of the present application will be introduced from the perspective of functional modularization. Figure 4 A language generation model improvement system structure diagram provided by an embodiment of the present application.

[0093] The system comprises:

[0094] The training text acquisition unit 110 is configured to acquire a training text.

[0095] The output text determination unit 111 is configured to input the training text into a language generation model and determine an output text.

[0096] The evaluation information determination unit 112 is configured to input the output text into an evaluation model and determine evaluation information.

[0097] The training unit 113 is configured to train the language generation model based on the output text and the evaluation information until the model converges and meets a preset performance requirement.

[0098] Optionally, the evaluation information determination unit is specifically configured to split the output text to obtain a plurality of words, and acquire character-level evaluation information corresponding to each split word and sentence-level evaluation information of the output text.

[0099] Optionally, the training unit comprises:

[0100] The target text acquisition unit acquires a target text, wherein the target text comprises words corresponding to evaluation information meeting a preset condition.

[0101] The mapping unit is configured to map the acquired target text to obtain a mapping result corresponding to the output text.

[0102] The loss value determination unit is configured to determine a loss value of the training text based on the mapping result and a standard sample.

[0103] The iteration unit is configured to iteratively train the language generation model based on the calculated loss value until the model converges and meets a preset performance requirement.

[0104] Optionally, the iteration unit comprises:

[0105] a probability distribution obtaining unit, configured to obtain a probability distribution corresponding to each word in the target text;

[0106] a loss value calculating unit, configured to calculate a loss value corresponding to the training text based on the obtained probability distribution and the standard sample.

[0107] Optionally, the output text determining unit is specifically configured to map each word included in the training text to obtain a word sequence corresponding to the training text; and predict each word included in the training text based on the word sequence corresponding to the training text to obtain an output text corresponding to the word sequence.

[0108] Embodiments of the present application also provide a corresponding device and a computer storage medium for implementing the scheme provided by the embodiments of the present application.

[0109] As shown in Figure 5 , the computer device 01 is in the form of a general-purpose computing device. The components of the computer device 01 can include but are not limited to one or more processors or processor units 03, a system memory 08, and a bus 04 connecting different system components including the system memory 08 and the processor unit 03.

[0110] The bus 04 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to an industry standard architecture (ISA) bus, a microchannel architecture (MAC) bus, an enhanced ISA bus, a video electronics standards association (VESA) local bus, and a peripheral component interconnect (PCI) bus.

[0111] The computer device 01 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 01 and includes both volatile and nonvolatile media, removable and non-removable media.

[0112] The system memory 08 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 09 and / or cache memory 10. The computer device 01 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 11 can be used for reading from and writing to non-removable, non-volatile magnetic media (e.g., a "hard drive"). Figure 3 (not shown), commonly referred to as a "hard disk drive", for reading from and writing to non-removable, non-volatile magnetic media (e.g., a "hard drive"). Although not specifically shown, such computer system can further include other removable / non-removable, volatile / non-volatile computer system storage media including, but not limited to, magnetic cassette, memory cards or sticks, DVD, CDs, digital tape, etc. Figure 3A disk drive, a floppy disk drive, and / or other mass storage device can be provided for in the example environment, which can be used to read from or write to a removable, nonvolatile magnetic media (e.g., a 'floppy disk'), and / or solid state media. Such media can store data and / or instructions for use by an appropriate device. In some embodiments, the system memory 08 can include one or more program products, which are configured to carry out the functions of embodiments of the application. The system memory 08, can include program modules that are configured to carry out the functions of embodiments of the application. The system memory 08, can include program modules that are configured to carry out the functions of embodiments of the application.

[0113] Program / utility 12, having a set (at least one) of program modules 13, can be stored in system memory 08 by way of example, and can include an operating system, one or more application programs, other program modules, and program data.

[0114] Computer device 01 can also communicate with one or more external devices 02 such as a keyboard or pointing device, a display 07, etc.; one or more devices that enable a user to interact with computer device 01; and / or one or more devices that enable computer device 01 to communicate with one or more other computing devices. Such communication can be via input / output (I / O) interfaces 06. Similarly, such Figure 5 communication can be enabled by a network adapter 05. Note that while computer device 01 appears to have only one bus 04 in the figure, there can be one or more buses Figure 5 coupling the central processing unit 03 to some of the other components in computer device 01. The bus 04 can be implemented using any suitable bus structure, including a

[0115] The processor unit 03 performs functions of various embodiments of the application by executing program code stored in the system memory 08. The program code can include one or more programs, which can be configured to carry out the functions of embodiments of the application.

[0116] Those skilled in the art can clearly understand that all or part of the steps of the above-mentioned method embodiments can be implemented by means of software plus a general hardware platform through the description of the above embodiments. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of the present application.

[0117] It should be noted that, in this document, the terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article or device that includes the element.

[0118] It should also be noted that each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for the device and apparatus embodiments, since they are basically similar to the method embodiments, they are described more simply, and the relevant parts can be referred to the part of the description of the method embodiments. The above-described device and apparatus embodiments are only schematic, and the units described as separate components can or can not be physically separated, and the components indicated as units can or can not be physical units, i.e. they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0119] The above is only one specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application can be easily thought of by those skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An improved method for language generation models, characterized in that, include: Obtain the training text; The training text is input into the language generation model to determine the output text; The output text is input into the evaluation model to determine the evaluation information; The language generation model is trained based on the output text and the evaluation information until the model converges and reaches the preset performance requirements. The output text is then input into the evaluation model to determine the evaluation information. The output text is segmented to obtain several words; Obtain character-level evaluation information for each segmented word and sentence-level evaluation information for the output text; The step of training the language generation model based on the output text and the evaluation information until the model converges and reaches the preset performance requirements includes: Obtain target text, which includes words corresponding to evaluation information that meet preset conditions; The acquired target text is mapped to obtain the mapping result corresponding to the output text; The loss value of the training text is determined based on the mapping results and standard samples; The language generation model is iteratively trained based on the calculated loss value until the model converges and reaches the preset performance requirements.

2. The method according to claim 1, characterized in that, The mapping result is the word sequence corresponding to the target text, and the step of determining the loss value of the training text based on the mapping result and standard samples includes: Obtain the probability distribution corresponding to each word in the target text; The loss value corresponding to the training text is calculated based on the obtained probability distribution and the standard samples.

3. The method according to claim 1, characterized in that, The step of inputting the training text into the language generation model and determining the output text includes: Map each word in the training text to obtain the word sequence corresponding to the training text; Based on the word sequence corresponding to the training text, predict each word included in the training text to obtain the output text corresponding to the word sequence.

4. A language generation model improvement system, characterized in that, The system includes: The training text acquisition unit is used to acquire training text; The output text determination unit is used to input the training text into the language generation model and determine the output text. An evaluation information determination unit is used to input the output text into the evaluation model and determine the evaluation information. The training unit is used to train the language generation model based on the output text and the evaluation information until the model converges and reaches the preset performance requirements. The evaluation information determination unit is specifically used to segment the output text into several words; obtain character-level evaluation information corresponding to each segmented word and sentence-level evaluation information of the output text; The training unit includes: The target text acquisition unit acquires target text, which includes words corresponding to evaluation information that meet preset conditions; The mapping unit is used to map the acquired target text to obtain the mapping result corresponding to the output text; The loss value determination unit is used to determine the loss value of the training text based on the mapping result and the standard samples; An iterative unit is used to iteratively train the language generation model based on the calculated loss value until the model converges and reaches the preset performance requirements.

5. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the language generation model improvement method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the language generation model improvement method as described in any one of claims 1-3.

Images