Generative language model training method and device, equipment, medium and product

Abstract

The invention discloses a training method and device of a generative language model, equipment, a medium and a product, and belongs to the field of natural language processing. The method comprises the steps that a training sample is obtained, the training sample comprises an input text, a target text and a sample label, and the sample label is used for indicating the text quality level of the target text; obtaining at least two text quality instructions, wherein the text quality instructions are used for indicating expected text quality levels; for each text quality instruction, calculating a text matching score and a prediction loss of the training sample for the text quality instruction; and training the generative language model based on the text matching scores and the prediction losses corresponding to the training samples for the at least two text quality instructions and the sample labels of the training samples. According to the method and the device, the generative language model is trained from multiple dimensions, so that the text evaluation capability of the obtained generative language model is higher.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a method, apparatus, device, medium, and product for training a generative language model. Background Art

[0002] Machine translation refers to the use of computers and other computing devices to translate text from one natural language (generally referred to as the source language) into another natural language (generally referred to as the target language). Big language models are AI models that have emerged in recent years, designed to understand and generate human language. With their development, they are increasingly being applied across various industries, such as machine translation.

[0003] Tests have shown that directly applying large language models to machine translation yields suboptimal results. Therefore, adjustments to large language models are necessary to make them suitable for machine translation. Related technologies typically employ instruction fine-tuning, enabling large language models to learn how to distinguish high-quality translations, thereby ensuring that the resulting translations meet translation evaluation criteria.

[0004] However, large language models generated using fine-tuning methods in related technologies are typically trained solely on prediction loss, resulting in limited translation quality assessment. Therefore, fine-tuning large language models to better assess the quality of generated translations is an urgent problem that needs to be addressed. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and product for training a generative language model. The technical solution is as follows:

[0006] According to one aspect of the present application, a method for training a generative language model is provided, the method comprising:

[0007] Acquiring a training sample, the training sample comprising an input text, a target text, and a sample label, the sample label being used to indicate a text quality level of the target text; and acquiring at least two text quality instructions, the text quality instructions being used to indicate a desired text quality level;

[0008] For each of the text quality instructions, calculating a text matching score and a prediction loss of the training sample for the text quality instruction, wherein the text matching score is used to indicate a probability of the generative language model predicting the target text based on the text quality instruction and the input text, and the prediction loss is used to indicate a loss of the generative language model predicting the target text based on the text quality instruction and the input text;

[0009] The generative language model is trained based on the text matching scores and prediction losses corresponding to the at least two text quality instructions of the training samples and the sample labels of the training samples.

[0010] According to one aspect of the present application, a training apparatus for a generative language model is provided, the apparatus comprising:

[0011] an acquisition module, configured to acquire a training sample, the training sample comprising an input text, a target text, and a sample label, the sample label being used to indicate a text quality level of the target text; and to acquire at least two text quality instructions, the text quality instructions being used to indicate a desired text quality level;

[0012] a calculation module configured to calculate, for each text quality instruction, a text matching score and a prediction loss of the training sample with respect to the text quality instruction, wherein the text matching score is used to indicate a probability of the generative language model predicting the target text based on the text quality instruction and the input text, and the prediction loss is used to indicate a loss of the generative language model predicting the target text based on the text quality instruction and the input text;

[0013] A training module is configured to train the generative language model based on the text matching scores and prediction losses corresponding to the at least two text quality instructions of the training samples and the sample labels of the training samples.

[0014] According to one aspect of the present application, a computer device is provided, comprising: a processor and a memory, wherein the memory stores at least one program; and the processor is configured to execute the at least one program in the memory to implement the above-mentioned generative language model training method.

[0015] According to one aspect of the present application, a computer-readable storage medium is provided, in which executable instructions are stored. The executable instructions are loaded and executed by a processor to implement the above-mentioned generative language model training method.

[0016] According to one aspect of the present application, a computer program product is provided, comprising computer instructions stored in a computer-readable storage medium, and a processor reading and executing the computer instructions from the computer-readable storage medium to implement the above-mentioned generative language model training method.

[0017] The beneficial effects of the technical solution provided by this application include at least:

[0018] A training sample with a sample label and at least two text quality instructions is used as a training sample. The text matching score and prediction loss of the training sample for each text quality instruction are calculated, and the generative language model is trained based on the calculated text matching score and prediction loss of the training sample for at least two text quality instructions. Since the training sample has a sample label, the sample label is used to indicate the text quality level of the target text of the training sample, and the text quality instruction can be used to indicate the text quality level of the text obtained by translating the input text of the training sample by the generative language model. When training the generative language model, it is only necessary to train the generative language model according to the sample label and the text quality level indicated by the text quality instruction. The model is trained in a targeted manner so that the text matching score calculated by the generative language model for the training samples corresponding to the text quality instructions corresponding to the sample labels is as high as possible and the prediction loss is as low as possible; the text matching score calculated for the text quality instructions that do not correspond to the sample labels for the training samples is as low as possible and the prediction loss is as high as possible. This enables the trained generative language model to have the ability to generate text of different text quality levels, that is, to enable the generative language model to learn text evaluation capabilities and be able to translate based on this text evaluation capability. At the same time, if the text quality levels are divided based on human evaluation standards, the text evaluation capabilities of the trained generative language model can also be aligned with human text evaluation standards. Simultaneously training the generative language model from multiple dimensions gives it a higher text evaluation capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 An architectural diagram of a computer system provided by an exemplary embodiment of the present application is shown;

[0021] Figure 2 A flowchart of a method for training a generative language model provided by an exemplary embodiment of the present application is shown;

[0022] Figure 3 A flowchart of a method for training a generative language model provided by an exemplary embodiment of the present application is shown;

[0023] Figure 4 A flowchart of a method for training a generative language model provided by an exemplary embodiment of the present application is shown;

[0024] Figure 5 A flowchart of a method for training a generative language model provided by an exemplary embodiment of the present application is shown;

[0025] Figure 6 A flowchart of a method for training a generative language model provided by an exemplary embodiment of the present application is shown;

[0026] Figure 7 A flowchart of a method for training a generative language model provided by an exemplary embodiment of the present application is shown;

[0027] Figure 8 A structural block diagram of a training device for a generative language model provided by an exemplary embodiment of the present application is shown;

[0028] Figure 9 A schematic structural diagram of a computer device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0030] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0031] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0032] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, storage, and display, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the information such as the settings operations involved in this application is obtained with full authorization.

[0033] It should be understood that although the terms first, second, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter without departing from the scope of this disclosure. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0034] First, let’s introduce the relevant terms involved in this application:

[0035] Semantics: The meaning of a symbol. This symbol can be a Chinese character, an English word, a pictogram, or other symbols or graphics. Semantics can also be understood as the meaning of data. In machine translation, semantics can be understood as the relationship between a linguistic symbol and the object it represents—that is, the content expressed by a word, phrase, sentence, or text. Simply put, semantics is the study of the meaning of language. In machine translation, ensuring the semantic accuracy of the translation results is crucial. This means that the translation system needs to accurately understand the meaning of the source language text and accurately convey it in the target language.

[0036] Pragmatics: The meaning conveyed by symbols when they are used. Specifically, it examines the meaning conveyed by users of linguistic symbols in a specific context and how those symbols are understood and used by listeners. Pragmatics primarily studies how language is used in real life, but also examines the information revealed in conversations beyond the meaning of sentences, and even examines differences in the meaning and use of language across cultures.

[0037] Translation quality: the degree of match between the target language sentence and the source language sentence in terms of semantics, pragmatics, etc., or the degree of match between the translation and the original text in terms of semantics, pragmatics, etc.

[0038] Generative language model: A large-scale pre-trained language model designed to understand and generate human language, including natural language and programming language. For example, models such as Bloomz, Llama, and GPT-3.5 can be used for text generation.

[0039] Figure 1 FIG1 shows an architecture diagram of a computer system provided by an exemplary embodiment of the present application. The computer system includes: a computer device 110 .

[0040] Optionally, the computer device is a server or a terminal.

[0041] Optionally, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides cloud computing services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms. Optionally, the terminal can be an electronic device such as a mobile phone, tablet computer, vehicle-mounted terminal (vehicle computer), wearable device, PC (Personal Computer), unmanned reservation terminal, etc.

[0042] The computer device 110 is configured to obtain a translation sample, the translation sample comprising a source language text, a target language text, and a sample label, the sample label being used to indicate a translation quality level of the target language text; and obtain at least two translation quality instructions, the translation quality instructions being used to indicate a desired translation quality level. The computer device 110 is configured to calculate a text matching score and prediction loss for the translation sample for each translation quality instruction. The computer device 110 is further configured to train a generative language model based on the text matching scores and prediction losses of the translation sample for the at least two translation quality instructions, as well as the sample label of the translation sample.

[0043] Exemplarily, the training method of the generative language model is illustrated by taking its application in the field of machine translation as an example. The translation sample 10 includes a source language text 11, a target language text 12, and a sample label 13. Among them, the source language text 11 is “The power of belief can overcome any obstacle in life”, the target language text 12 is “信念的力量能够战胜生活中的重重困难”, and the sample label 13 is “medium quality”. The target language text 12 can be divided into 7 target sub-word units 14 through sub-word segmentation. The translation quality levels include high quality, medium quality, and low quality, and the translation quality instructions 15 accordingly include high-quality instructions, medium-quality instructions, and low-quality instructions. During the training process of the generative language model 16, the source language text 11, the target language text 12, and the three translation quality instructions 15 in the translation sample 10 are respectively input into the generative language model 16 to calculate the text matching score and prediction loss of the translation sample 10 for each translation quality instruction. For example, the source language text 11, the target language text 12, and the high-quality instruction in the translation sample 10 are input into the generative language model 16 to calculate the text matching score and prediction loss of the translation sample 10 for the high-quality instruction; the source language text 11, the target language text 12, and the medium-quality instruction in the translation sample 10 are input into the generative language model 16 to calculate the text matching score and prediction loss of the translation sample 10 for the medium-quality instruction; the source language text 11, the target language text 12, and the low-quality instruction in the translation sample 10 are input into the generative language model 16 to calculate the text matching score and prediction loss of the translation sample 10 for the low-quality instruction. During the calculation process of the text matching score, it is necessary to first calculate at least one sub-word matching score of the translation sample for each translation quality instruction 15. For example, the 3rd target sub-word is “力量”. The generative language model calculates the 3rd sub-word matching score 17 based on each translation quality instruction. The 3rd sub-word matching score 17 is used to indicate the probability that the generative language model 16 predicts the 3rd target sub-word unit on the premise of predicting the first 2 target sub-word units of the translation sample 10 based on each translation quality instruction. As shown in the probability table 18, it is the probability that the generative language model 16 predicts different sub-word units on the premise of predicting the first 2 target sub-word units of the translation sample 10 based on the high-quality instruction. The 3rd target sub-word unit is “力量” and its corresponding probability is 0.5; as shown in the probability table 19, it is the probability that the generative language model 16 predicts different sub-word units on the premise of predicting the first 2 target sub-word units of the translation sample 10 based on the low-quality instruction. The 3rd target sub-word unit is “力量” and its corresponding probability is 0.3.Based on the text matching scores of the translation samples 10 for different translation quality indicators 15, the generative language model 16 constructs a corresponding incentive function. The incentive function is used to encourage the generative language model to generate target language text that has a higher text matching score with the labeled indicator than with the unlabeled quality indicator. The generative language model 16 then constructs a loss function based on the incentive function and the prediction loss to train the generative language model 16.

[0044] Figure 2 A flowchart of a method for training a generative language model provided by an exemplary embodiment of the present application is shown. The method is executed by a computer device, which may be the above-mentioned Figure 1 The computer device shown in. The method includes:

[0045] Step 210: Obtain training samples, which include input text, target text, and sample labels, where the sample labels are used to indicate the text quality level of the target text; and obtain at least two text quality instructions, where the text quality instructions are used to indicate the expected text quality level.

[0046] Among them, the training samples used for the training method of the generative language model include one or more. The embodiment of the present application uses one of the training samples as an example for illustration, but is not limited to this.

[0047] Optionally, the input text and the target text in the training sample are sentences, paragraphs, or articles.

[0048] Optionally, the technician divides the text quality levels of the training samples according to the text quality of the target text in the training samples; or, the generative language model divides the text quality levels of the training samples according to the text quality of the target text in the training samples. The score interval of text quality is [0, 100]; the target text with a text quality score of [0, 85) is divided into low-quality text; the target text with a text quality score of [85, 90] is divided into medium-quality text; and the target text with a text quality score of (90, 100] is divided into high-quality text. The score interval of text quality can also be [0, 1.0], [0, 10], etc. Based on different text quality score intervals and different text quality level evaluation criteria, the text quality levels corresponding to the training samples are different. It should be noted that, with respect to the division of text quality levels, the embodiment of the present application uses the text quality levels including high-quality text, medium-quality text and low-quality text as examples for illustration, but is not limited to this.

[0049] Optionally, the text quality instruction includes an instruction for indicating a desired text quality level, for example, the text quality instruction is "generate high-quality text"; or, the text quality instruction is "generate medium-quality text"; or, the text quality instruction is "generate low-quality text", etc. Alternatively, the text quality instruction includes a prompt word for indicating a desired text quality level, for example, if the prompt word is "high quality", the text quality instruction may be "generate high-quality text" or "make high-quality prediction based on input", etc.; or, if the prompt word is "medium quality", the text quality instruction may be "generate medium-quality text" or "make medium-quality prediction based on input", etc.; or, if the prompt word is "low quality", the text quality instruction may be "generate low-quality text" or "make low-quality prediction based on input", etc.

[0050] Optionally, multiple text quality instructions may be included for the same text quality level. Multiple text quality instructions corresponding to the same text quality level are referred to as one text quality instruction. For example, for the text quality level of "high-quality text," the corresponding text quality instructions include "generate high-quality text," "output based on input as an expert," and "make high-quality predictions based on input." These text quality instructions can be collectively referred to as one text quality instruction.

[0051] In some embodiments, the target text in the training sample is obtained by predicting the input text using a text prediction model. The text prediction model includes the generative language model to be trained in this application, and text generation may also include other text prediction models that have been put into use or are being trained. The embodiments of this application do not limit the type, principle, etc. of the text prediction model. The target text in the training sample is obtained by inputting the input text in the training sample into the text prediction model. The target text in the multiple prediction samples used in the training process of the generative language model can be obtained by predicting the input text based on multiple text prediction models.

[0052] Step 220: For each text quality instruction, calculate the text matching score and prediction loss of the training sample for the text quality instruction. The text matching score is used to indicate the probability of the generative language model predicting the target text based on the text quality instruction and the input text. The prediction loss is used to indicate the loss of the generative language model predicting the target text based on the text quality instruction and the input text.

[0053] Optionally, the generative language model concatenates the training sample with each text quality indicator and calculates the text matching score and prediction loss of the training sample for at least two text quality indicators. Alternatively, the training sample and each text quality indicator are input into the generative language model, which then calculates the text matching score and prediction loss of the training sample for each text quality indicator. Alternatively, for at least two text quality indicators, the generative language model calculates the text matching score and prediction loss of the training sample for each text quality indicator.

[0054] Optionally, the text matching score is used to indicate the probability that the generative language model predicts the target text according to the text quality instruction. Alternatively, the text matching score is used to indicate the probability that the generative language model predicts the input text in the training sample to be the target text in the training sample according to the text quality instruction.

[0055] Optionally, the prediction loss is used to indicate the loss of the target text predicted by the generative language model according to the text quality instruction. Alternatively, the prediction loss is used to indicate the degree of difference between the text predicted by the generative language model according to the text quality instruction and the target text. Alternatively, the prediction loss is used to indicate the difficulty of the generative language model in predicting the target text according to the text quality instruction.

[0056] Exemplarily, there are n types of text quality instructions; the text matching score and prediction loss of the training sample for the first text quality instruction, the text matching score and prediction loss of the training sample for the second text quality instruction,..., the text matching score of the training sample for the n-1th text quality instruction, and the text matching score of the training sample for the nth text quality instruction are calculated respectively.

[0057] The text matching scores and prediction losses of the above training samples for at least two text quality instructions are calculated simultaneously by the generative language model; or, are calculated one by one in sequence by the generative language model; or, are calculated in batches by the generative language model.

[0058] In an optional embodiment, at least two text quality instructions include the i-th text quality instruction, the target text includes at least one target unit, and i is a positive integer; the target text is a sentence or a paragraph or an article; the target unit is a character or a sub-word or a word. A character refers to the basic component in the text, such as each Chinese character and punctuation mark in the Chinese text, etc.; a sub-word refers to the result obtained after dividing a long word or an unknown word or an unknown phrase in the text. For example, when the word "natural" is divided into sub-words, it may be divided into sub-word units "natura" and sub-word unit "l"; a word refers to the lexical unit with meaning in the text. For example, in the sentence "I have a cat", "a" is usually divided into a word. According to different configurations of the generative language model, the target text can be divided into different types of target units. For example, if there is a sub-word division module configured in the generative language model, the target text is divided into at least one target unit; or, according to different configurations in the server, the target text is divided into different types of target units. For example, if there is a word division module configured in the server, the target text is divided into at least one target word.

[0059] In some embodiments, for each text quality instruction, calculate the text matching score of the training sample for the text quality instruction; equivalently, input the training sample and the i-th text quality instruction in at least two text quality instructions into the generative language model, and calculate the text matching score of the training sample for the i-th text quality instruction; update i to i + 1, and continue to execute the above step until the text matching scores of the training sample for at least two text quality instructions are calculated. For each text quality instruction, calculate the prediction loss of the training sample; equivalently, input the training sample and the i-th text quality instruction in at least two text quality instructions into the generative language model, and calculate the prediction loss of the training sample for the i-th text quality instruction; update i to i + 1, and continue to execute the above step until the prediction losses of the training sample for at least two text quality instructions are calculated. Wherein, i is a positive integer.

[0060] In some embodiments, for each text quality instruction, calculating the text matching score of the training sample for the text quality instruction can be implemented by: calculating at least one unit matching score of the training sample for the i-th text quality instruction; calculating the text matching score of the training sample for the i-th text quality instruction, where the text matching score is the ratio of the third cumulative sum to the text length of the target text, and the third cumulative sum is the cumulative sum of at least one unit matching score. The unit matching score is used to indicate the probability of the generative language model predicting the target unit based on the i-th text quality instruction for the training sample; or, the unit matching score is used to indicate the probability of the generative language model translating the input text in the training sample based on the i-th text quality instruction to obtain the target unit; or, the unit matching score is used to indicate the probability of the generative language model translating the source language text in the training sample based on the i-th text quality instruction to obtain the target unit located at the corresponding position in the target text in the training sample.

[0061] In some embodiments, calculating at least one unit matching score of a training sample for the i-th text quality instruction can be implemented as follows: for the i-th text quality instruction, calculating the unit matching score of the j-th target unit in the target text, the j-th unit matching score is used to indicate the probability that the generative language model predicts the j-th target unit based on the i-th text quality instruction for the training sample, where j is a positive integer; updating j to j+1, and continuing to perform one of the above steps until at least one unit matching score of the training sample for the i-th text quality instruction is calculated. The j-th unit matching score is used to indicate the probability that the generative language model predicts the j-th target unit based on the i-th text quality instruction for the training sample; or, the j-th unit matching score is used to indicate the probability that the generative language model predicts the j-th target unit based on the premise that the generative language model predicts the first j-1 target units based on the i-th text quality instruction for the training sample. That is, the unit matching score of the j-th target unit can be recorded as P(y j kc,x,y ＜j ), where y j represents the jth target unit, c represents the text quality instruction, x represents the input text in the training sample, y ＜j represents the 1st to j-1th target units of the target text in the training sample; the formula P(y j |c,x,y ＜j ) can be understood as the probability that the jth target unit generated by the generative language model is the jth target unit in the target text based on the generation of the 1st to j-1th target units for the input text x under the instruction of the text quality instruction c.

[0062] Step 230: Train the generative language model based on the text matching scores and prediction losses corresponding to the at least two text quality instructions of the training samples and the sample labels of the training samples.

[0063] Optionally, a generative language model is trained based on the text matching scores and prediction losses of the training samples for at least two text quality instructions and the sample labels of the training samples. The training goal is to make the text matching scores for the text quality instructions corresponding to the sample labels calculated by the generative language model for the training samples as high as possible and the prediction losses as low as possible; and to make the text matching scores for the text quality instructions that do not correspond to the sample labels calculated by the generative language model for the training samples as low as possible and the prediction losses as high as possible.

[0064] Optionally, a backpropagation algorithm is used to train the generative language model based on the text matching scores and prediction losses of the training samples for at least two text quality instructions, as well as the sample labels of the training samples. Specifically, a loss function group is constructed based on the text matching scores and prediction losses of the training samples for at least two text quality instructions, as well as the sample labels of the training samples. The gradient of the loss value of the training samples with respect to the model parameters of the generative language model is calculated. The gradient is used to indicate the direction and rate of change of the loss value with the model parameters. The model parameters of the generative language model are continuously updated based on the gradient optimization algorithm. When the number of updates reaches a threshold or the model parameters converge, the trained generative language model is obtained. It should be noted that the model parameters updated vary depending on the training method used. For example, in the pre-training stage of the generative language model, the updated model parameters are all the model parameters of the generative language model; in the fine-tuning stage of the generative language model, such as using instruction fine-tuning or LoRA (Low-Rank Adaptation), the updated model parameters are only some of the model parameters of the generative language model. LoRA is a method that reduces the model parameters that need to be updated during model training by introducing a low-rank matrix during model training.

[0065] In some embodiments, the text matching scores and prediction losses of the training samples for the at least two text quality instructions include at least two text matching scores and at least two prediction losses; training the generative language model based on the text matching scores and prediction losses corresponding to the at least two text quality instructions of the training samples and the sample labels of the training samples can be implemented as follows: for each text quality instruction, based on the text matching scores of the training samples for the at least two text quality instructions and the sample labels of the training samples, constructing an excitation function for the training samples for the text quality instruction, the excitation function being used to instruct the generative language model to distinguish between the text quality level of the text quality instruction and the text quality level of the sample label; training the generative language model based on the excitation function and prediction loss of the training samples for the at least two text quality instructions. Among them, the activation function is used to instruct the generative language model to distinguish the text quality level of the text quality instruction and the text quality level of the sample label; or, the activation function is used to instruct the generative language model to generate a target text with a high degree of matching with the label instruction and a low degree of matching with the non-label instruction, the label instruction is a text quality instruction that matches the text quality level of the sample label, and the non-label instruction is a text quality instruction that does not match the text quality level of the sample label; or, the activation function is used to incentivize the generative language model to generate a target text with a higher text matching score with the label instruction than with the non-label quality.

[0066] In some embodiments, for each text quality instruction, an excitation function of the training sample for the text quality instruction is constructed based on the text matching scores of the training sample for at least two text quality instructions and the sample label of the training sample. This can be implemented as follows: based on the text matching scores of the training sample for at least two text quality instructions, a prediction index of the training sample for at least two text quality instructions is calculated, where the prediction index is an exponent with a natural constant as the base and the text matching score as the power; the prediction index of the label instruction corresponding to the sample label of the training sample is obtained, where the label instruction is a text quality instruction in at least two text quality instructions whose text quality level is the same as the text quality level of the sample label; and at least two excitation functions are constructed based on the prediction index of the label instruction and the prediction index of at least two text quality instructions.

[0067] In some embodiments, at least two excitation functions are constructed based on the prediction index of the label instruction and the prediction index of at least two text quality instructions, which can be implemented as follows: for the i-th text quality instruction among the at least two text quality instructions, a third intermediate formula is constructed, and the third intermediate formula is a cumulative formula of the prediction index of the at least two text quality instructions; an excitation function corresponding to the i-th text quality instruction is constructed, one side of the excitation function is the excitation value, and the other side of the excitation function is the negative of the ratio of the prediction index of the label instruction to the third intermediate formula, and the excitation function is a function with the prediction index of the i-th text quality instruction as the independent variable and the excitation value as the dependent variable; i is updated to i+1, and the above two steps are continued until at least two excitation functions are constructed.

[0068] In summary, the method provided in the embodiment of the present application takes a training sample with a sample label and at least two text quality instructions as a training sample, calculates the text matching score and prediction loss of the training sample for each text quality instruction, and trains the generative language model based on the calculated text matching score and prediction loss of the training sample for at least two text quality instructions. Since the training sample has a sample label, the sample label is used to indicate the text quality level of the target text of the training sample, and the text quality instruction can be used to indicate the text quality level of the text obtained by translating the input text of the training sample by the generative language model. When training the generative language model, it is only necessary to train the generative language model according to the text quality indicated by the sample label and the text quality instruction. The generative language model is trained in a targeted manner based on the quality level. This ensures that the text matching score calculated by the generative language model for the training samples corresponding to the sample labels is as high as possible, and the prediction loss is as low as possible. The text matching score calculated for the training samples not corresponding to the sample labels is as low as possible, and the prediction loss is as high as possible. This enables the trained generative language model to generate texts of different text quality levels. In other words, the generative language model learns text evaluation capabilities and can translate based on these text evaluation capabilities. At the same time, if the text quality levels are divided based on human evaluation standards, the text evaluation capabilities of the trained generative language model can also be aligned with human text evaluation standards. Simultaneously training the generative language model from multiple dimensions enhances its text evaluation capabilities.

[0069] The above-mentioned generative language model training method can be applied to different fields, such as machine translation, text generation, search, question and answer, etc. The following will exemplify the specific process of applying the generative language model training method to the fields of machine translation and text generation. It should be noted that the embodiments of this application only list some application fields of the generative language model training method. For the remaining application fields, the embodiments of this application will not be listed one by one, but the scope of protection of this application is not limited to them.

[0070] 1. Machine translation

[0071] Figure 3 A flowchart of a generative language model training method provided by an exemplary embodiment of the present application is shown. When the generative language model training method is applied to machine translation, the training samples include translation samples, the input text includes source language text, the target text includes target language text, and the text quality level includes the translation quality level. Step 210 can be implemented as step 310, step 220 can be implemented as step 320, and step 230 can be implemented as step 330.

[0072] Step 310: Obtain a translation sample, which includes a source language text, a target language text, and a sample label, where the sample label is used to indicate a translation quality level of the target language text; and obtain at least two translation quality instructions, where the translation quality instructions are used to indicate a desired translation quality level.

[0073] Among them, the translation samples used for the training method of the generative language model include one or more. The embodiment of the present application uses one of the translation samples as an example for illustration, but is not limited to this.

[0074] Optionally, the source language text and the target language text in the translation sample are sentences, paragraphs, or articles.

[0075] Optionally, the technician divides the translation samples into translation quality levels based on the translation quality of the target language text in the translation samples; or, the generative language model divides the translation samples into translation quality levels based on the translation quality of the target language text in the translation samples. The translation quality score interval is [0, 100]; the target language text with a translation quality score in [0, 85) is divided into low-quality translations; the target language text with a translation quality score in [85, 90] is divided into medium-quality translations; and the target language text with a translation quality score in (90, 100) is divided into high-quality translations. The translation quality score interval can also be [0, 1.0], [0, 10], etc. Based on different translation quality score intervals and different translation quality level evaluation criteria, the translation quality levels corresponding to the translation samples are different. It should be noted that, with respect to the division of translation quality levels, the embodiment of the present application uses the translation quality levels including high-quality translations, medium-quality translations and low-quality translations as examples, but is not limited to this.

[0076] Optionally, the translation quality instruction includes an instruction for indicating a desired translation quality level, for example, the translation quality instruction is "generate a high-quality translation"; or the translation quality instruction is "generate a medium-quality translation"; or the translation quality instruction is "generate a low-quality translation", etc. Alternatively, the translation quality instruction includes a prompt word for indicating a desired translation quality level, for example, if the prompt word is "high quality", the translation quality instruction may be "generate a high-quality translation" or "perform a high-quality translation of the following source text", etc.; or, if the prompt word is "medium quality", the translation quality instruction may be "generate a medium-quality translation" or "perform a medium-quality translation of the following source text", etc.; or, if the prompt word is "low quality", the translation quality instruction may be "generate a low-quality translation" or "perform a low-quality translation of the following source text", etc.

[0077] Optionally, multiple translation quality instructions may be included for the same translation quality level. These multiple translation quality instructions corresponding to the same translation quality level are referred to as one translation quality instruction. For example, for the translation quality level of "high-quality translation," the corresponding translation quality instructions include "generate a high-quality translation," "translate as an expert," and "produce a high-quality translation of the following source text." These translation quality instructions can be collectively referred to as one translation quality instruction.

[0078] Step 320: For each translation quality instruction, calculate the text matching score and prediction loss of the translation sample for the translation quality instruction. The text matching score is used to indicate the probability of the generative language model predicting the target language text based on the translation quality instruction and the source language text. The prediction loss is used to indicate the loss of the generative language model predicting the target language text based on the translation quality instruction and the source language text.

[0079] Optionally, the generative language model concatenates the translation sample with each translation quality specification and calculates the text matching score and prediction loss of the translation sample for at least two translation quality specifications. Alternatively, the translation sample and each translation quality specification are input into the generative language model, which then calculates the text matching score and prediction loss of the translation sample for each translation quality specification. Alternatively, for at least two translation quality specifications, the generative language model calculates the text matching score and prediction loss of the translation sample for each translation quality specification.

[0080] Optionally, the text match score indicates the probability that the generative language model predicts the target language text based on the translation quality directive. Alternatively, the text match score indicates the probability that the generative language model predicts the source language text in the translation sample to be the target language text in the translation sample based on the translation quality directive.

[0081] Optionally, the prediction loss is used to indicate the loss of the target language text predicted by the generative language model based on the translation quality instruction. Alternatively, the prediction loss is used to indicate the degree of difference between the translation predicted by the generative language model based on the translation quality instruction and the target language text. Alternatively, the prediction loss is used to indicate the difficulty of the generative language model in predicting the target language text based on the translation quality instruction.

[0082] Exemplarily, there are n types of translation quality instructions; the text matching score and prediction loss of the translation sample for the first translation quality instruction, the text matching score and prediction loss of the translation sample for the second translation quality instruction, ..., the text matching score of the translation sample for the n-1th translation quality instruction, and the text matching score of the translation sample for the nth translation quality instruction are calculated respectively.

[0083] The text matching scores and prediction losses of the above translation samples for at least two translation quality instructions are calculated simultaneously by the generative language model; or, are calculated one by one sequentially by the generative language model; or, are calculated in batches by the generative language model.

[0084] Step 330: Train a generative language model based on the text matching scores and prediction losses corresponding to the translation samples for at least two translation quality indicators and the sample labels of the translation samples.

[0085] Optionally, a generative language model is trained based on the text matching scores and prediction losses of the translation samples for at least two translation quality instructions and the sample labels of the translation samples. The training goal is to make the text matching scores for the translation quality instructions corresponding to the sample labels calculated by the generative language model for the translation samples as high as possible and the prediction losses as low as possible; and to make the text matching scores for the translation quality instructions that do not correspond to the sample labels calculated by the generative language model for the translation samples as low as possible and the prediction losses as high as possible.

[0086] Optionally, a sampling backpropagation algorithm is used to train the generative language model based on the text matching scores and prediction losses of the translation samples for at least two translation quality instructions, as well as the sample labels of the translation samples. Specifically, a loss function group is constructed based on the text matching scores and prediction losses of the translation samples for at least two translation quality instructions, as well as the sample labels of the translation samples. The gradient of the loss value of the translation sample with respect to the model parameters of the generative language model is calculated. The gradient is used to indicate the direction and rate at which the loss value changes with the model parameters. The model parameters of the generative language model are continuously updated based on the gradient optimization algorithm. When the number of updates reaches a threshold or the model parameters converge, the trained generative language model is obtained. It should be noted that the model parameters updated vary depending on the training method used. For example, in the pre-training stage of the generative language model, the updated model parameters are all the model parameters of the generative language model; in the fine-tuning stage of the generative language model, such as using instruction fine-tuning, LoRA, etc., the updated model parameters are only some of the model parameters in the generative language model. LoRA is a method that reduces the number of model parameters that need to be updated during model training by introducing a low-rank matrix during model training.

[0087] In summary, the method provided in the embodiment of the present application uses translation samples with sample labels and at least two translation quality instructions as training samples, calculates the text matching score and prediction loss of the translation samples for each translation quality instruction, and trains the generative language model based on the calculated text matching score and prediction loss of the translation samples for at least two translation quality instructions. Since the translation samples have sample labels, the sample labels are used to indicate the translation quality level of the target language text of the translation samples, and the translation quality instructions can be used to indicate the translation quality level of the translation obtained by the generative language model when translating the source language text of the translation samples. When training the generative language model, it is only necessary to train the translation according to the sample labels and the translation quality instructions. The generative language model is trained specifically for quality levels, ensuring that the text matching scores calculated for translation samples for translation quality instructions corresponding to sample labels are as high as possible and the prediction loss is as low as possible. Furthermore, the text matching scores calculated for translation quality instructions that do not correspond to sample labels are as low as possible and the prediction loss is as high as possible. This enables the trained generative language model to generate translations of varying quality levels. In other words, the generative language model learns translation evaluation capabilities and can translate based on these capabilities. Furthermore, if translation quality levels are based on human evaluation criteria, the translation evaluation capabilities of the trained generative language model can also be aligned with human evaluation standards. Simultaneously training the generative language model from multiple dimensions enhances its translation evaluation capabilities.

[0088] The text matching score is used to indicate the probability of the generative language model predicting the target language text according to the translation quality instruction. The calculation of the text matching score is introduced below. Figure 3 In an optional embodiment, step 320 includes two sub-steps, namely sub-step 1 and sub-step 2. Sub-step 1 is to calculate the text matching score of the translation sample against the translation quality instruction for each translation quality instruction; sub-step 2 is to calculate the prediction loss of the translation sample against the translation quality instruction for each translation quality instruction. Sub-step 1 in step 320 can be alternatively implemented as steps 410 and 420. Figure 4 A flowchart of a method for training a generative language model provided by an exemplary embodiment of the present application is shown. The at least two translation quality instructions include an i-th translation quality instruction, the target language text includes at least one target subword unit, and i is a positive integer.

[0089] Optionally, the source language text and the target language text in the translation sample are sentences, paragraphs, or articles. The target language text is a sentence, and the generative language model divides the sentence into subwords to obtain at least one target subword unit; or, the target language text is a paragraph, and the generative language model divides the paragraph into subwords to obtain at least one target subword unit; or, the target language text is a paragraph, and the generative language model divides the paragraph into sentences to obtain at least one sentence, and divides the at least one sentence into subwords to obtain at least one target subword unit; or, the target language text is an article, and the generative language model divides the article into subwords to obtain at least one target subword unit; or, the target language text is an article, and the generative language model divides the article into paragraphs to obtain at least one paragraph, and divides the paragraph into subwords to obtain at least one target subword unit; or, the target language text is an article, and the generative language model divides the article into paragraphs to obtain at least one paragraph, and divides the paragraph into sentences to obtain at least one sentence, and divides the at least one sentence into subwords to obtain at least one target subword unit. This embodiment of the present application is not limited to this.

[0090] For example, the target language text is "I have a cat.", and the target subword units are {"I", "have", "a", "cat", "."}; or, the target language text is "I have a cat.", and the target subword units are {"I", "have", "a", "cat", "."}.

[0091] Optionally, for each translation quality instruction, a text matching score of the translation sample with respect to the translation quality instruction is calculated; this is equivalent to inputting the translation sample and the i-th translation quality instruction from at least two translation quality instructions into a generative language model, and calculating the text matching score of the translation sample with respect to the i-th translation quality instruction; updating i to i+1 and continuing with the above step until the text matching score of the translation sample with respect to the at least two translation quality instructions is calculated. For each translation quality instruction, a prediction loss of the translation sample is calculated; this is equivalent to inputting the translation sample and the i-th translation quality instruction from at least two translation quality instructions into a generative language model, and calculating the prediction loss of the translation sample with respect to the i-th translation quality instruction; updating i to i+1 and continuing with the above step until the prediction loss of the translation sample with respect to the at least two translation quality instructions is calculated. Where i is a positive integer.

[0092] Step 410: Calculate at least one subword matching score of the translation sample for the i-th translation quality instruction.

[0093] Among them, the subword matching score is used to indicate the probability of the generative language model predicting the translation sample based on the i-th translation quality instruction to obtain the target subword unit; or, the subword matching score is used to indicate the probability of the generative language model translating the source language text in the translation sample based on the i-th translation quality instruction to obtain the target subword unit; or, the subword matching score is used to indicate the probability of the generative language model translating the source language text in the translation sample based on the i-th translation quality instruction to obtain the target subword unit located at the corresponding position of the target language text in the translation sample.

[0094] Optionally, at least one target subword unit and the i-th translation quality instruction are input into a generative language model, and the generative language model calculates at least one subword matching score of the translation sample for the i-th translation quality instruction; or, the translation sample and the i-th translation quality instruction are input into the generative language model, and the generative language model performs subword division on the target language text of the translation sample to obtain at least one target subword unit, and calculates at least one subword matching score of the translation sample for the i-th translation quality instruction.

[0095] The at least one subword matching score of the translation sample for the i-th translation quality instruction is calculated simultaneously by the generative language model; or, is calculated one by one in sequence by the generative language model; or, is calculated in batches by the generative language model.

[0096] In an alternative embodiment, Figure 4 As shown, step 410 can be implemented alternatively as steps 411 and 412 .

[0097] Step 411: For the i-th translation quality instruction, calculate the subword matching score of the j-th target subword unit in the target language text, where the j-th subword matching score is used to indicate the probability of the generative language model predicting the translation sample based on the i-th translation quality instruction to obtain the j-th target subword unit, where j is a positive integer.

[0098] Among them, the j-th subword matching score is used to indicate the probability of the generative language model predicting the j-th target subword unit based on the i-th translation quality instruction for the translation sample; or, the j-th subword matching score is used to indicate the probability of the generative language model predicting the j-th target subword unit based on the premise that the generative language model predicts the first j-1 target subword units for the translation sample based on the i-th translation quality instruction. That is, the subword matching score of the j-th target subword unit can be recorded as P(y j |c,x,y ＜j ), where y j represents the jth target subword unit, c represents the translation quality instruction, x represents the source language text in the translation sample, y ＜jrepresents the 1st to j-1th target subword units of the target language text in the translation sample; the formula P(y j |c,x,y ＜j ) can be understood as the probability that the jth target subword unit generated by the generative language model is the jth target subword unit in the target language text based on the generation of the 1st to j-1th target subword units for the source language text x under the instruction of the translation quality instruction c.

[0099] Optionally, the generative language model stores a model lexicon, which is a collection of subwords that the translation task relies on. The prediction process, or translation process, of the generative language model involves selecting subword units from the model lexicon. During this process, the generative language model selects subword units based on their probabilities, or subword matching scores.

[0100] Optionally, for the i-th translation quality instruction, the generative language model calculates the subword matching scores of all subword units in the model vocabulary for the translation quality instruction, and queries the subword matching score corresponding to the j-th target subword unit; or, for the i-th translation quality instruction, the generative language model calculates the subword matching score of the j-th target subword unit.

[0101] Step 412: Update j to j+1, and continue to execute the above step until at least one subword matching score of the translation sample for the i-th translation quality instruction is calculated.

[0102] For example, the source language text is "The power of belief can overcome any obstacle in life", the target language text is "A firm belief can overcome the many difficulties in life", and at least one target sub-word unit is {"firm", "of", "belief", "can", "overcome", "life", "in", "of", "heavy", "difficulties"}. The first target sub-word unit is "firm", and the sub-word matching score of the first target sub-word unit is The generative language model predicts the source language text in the translation sample based on the i-th translation quality instruction and obtains the probability that the first target sub-word unit is "firm"; the second target sub-word unit is "of", and the sub-word matching score of the second target sub-word unit is The generative language model predicts the source language text in the translation sample based on the i-th translation quality, and on the basis of obtaining the first target sub-word unit as "firm", predicts the probability that the second target sub-word unit is "of"; the third target sub-word unit is "belief", and the sub-word matching score of the third target sub-word unit is The generative language model predicts the source language text in the translation sample based on the i-th translation quality, and on the basis of obtaining the first target sub-word unit as "firm", predicts the probability that the second target sub-word unit is "of"; the third target sub-word unit is "belief", and the sub-word matching score of the third target sub-word unit is Based on the i-th translation quality instruction, the source language text in the translation sample is predicted. Based on the first two target sub-word units being {"firm", "of"}, the probability of the third target sub-word unit being "belief" is predicted;...; the tenth target sub-word unit is "difficult", and the sub-word matching score of the tenth target sub-word unit is The generative language model predicts the source language text in the translation sample based on the i-th translation quality instruction. Based on the first nine target sub-word units being {"firm", "of", "belief", "able", "overcome", "life", "in", "of", "heavy"}, the probability of the tenth target sub-word unit being "difficult" is predicted.

[0103] Step 420: Calculate a text matching score of the translation sample for the i-th translation quality instruction. The text matching score is the ratio of the first cumulative sum to the text length of the target language text. The first cumulative sum is the cumulative sum of at least one subword matching score.

[0104] Optionally, a text matching score of the translation sample for the i-th translation quality instruction is calculated based on the at least one subword matching score.

[0105] Exemplarily, the calculation formula for the text matching score is shown below.

[0106]

[0107] Where n is the length of the target language text; c is the translation quality instruction; x is the source language text in the translation sample; y is the translation quality instruction. j represents the jth target subword unit, the starting value of j is 1; y ＜jrepresents the 1st to j-1th target subword units of the target language sentence in the translation sample; P(y j |c,x,y ＜j ) can be understood as the probability that the jth target subword unit generated by the generative language model is the jth target subword unit in the target language text based on the generation of the 1st to j-1th target subword units for the source language text x under the instruction of the translation quality instruction c.

[0108] In summary, the method provided in the embodiment of the present application calculates at least one subword matching score of the translation sample for the i-th translation quality instruction for at least one target subword unit in the target language text, and then calculates the text matching score of the translation sample for the i-th translation quality instruction based on the at least one subword matching score. This is because when the generative language model makes a prediction, it is usually selected based on the model vocabulary, and the model vocabulary stores multiple subword units that are manually entered or stored by the generative language model. The text matching score is obtained based on the at least one subword matching score calculation, which can better adapt to the training process of the generative language model, and thus realize the training of the generative language model with translation evaluation capability.

[0109] Furthermore, an embodiment of the present application also shows a calculation method for at least one subword matching score. The jth subword matching score is used to indicate the probability that the generative language model predicts the jth target subword unit based on the i-th translation quality instruction for the translation sample. It can also be said that the jth subword matching score is used to indicate the probability that the generative language model predicts the jth target subword unit based on the premise that the first j-1 target subword units are predicted for the translation sample based on the i-th translation quality instruction. When calculating the jth subword matching score, it is only necessary to know the first j-1 target subword units of the target language text, and the generative language model is enabled to predict the probability of the jth target subword unit based on the i-th translation quality instruction based on the premise that the first j-1 target subword units are known. The jth subword matching score does not need to know other subword matching scores when calculating, so that the generative language model can calculate at least one subword matching score in batches, thereby improving the training efficiency of the generative language model.

[0110] Next, a specific process of training a generative language model based on text matching scores and prediction losses of translation samples for at least two translation quality indicators and sample labels of the translation samples is shown.

[0111] Based on Figure 3 In an optional embodiment, step 330 can be replaced by step 331 and step 332. Figure 5 A flowchart of a method for training a generative language model provided by an example embodiment of the present application is shown.

[0112] Step 331: For each translation quality instruction, based on the text matching scores of the translation sample for at least two translation quality instructions and the sample label of the translation sample, an incentive function for the translation sample for the translation quality instruction is constructed. The incentive function is used to instruct the generative language model to distinguish between the translation quality level of the translation quality instruction and the translation quality level of the sample label.

[0113] Optionally, the incentive function is used to instruct the generative language model to distinguish between the translation quality level of the translation quality instruction and the translation quality level of the sample label; or, the incentive function is used to instruct the generative language model to generate a target language text with a high degree of match with the label instruction and a low degree of match with the non-label instruction, the label instruction is a translation quality instruction that matches the translation quality level of the sample label, and the non-label instruction is a translation quality instruction that does not match the translation quality level of the sample label; or, the incentive function is used to incentivize the generative language model to generate a target language text with a higher text matching score with the label instruction than with the non-label quality text.

[0114] Optionally, the at least two translation quality instructions include an i-th translation quality instruction. For the i-th translation quality instruction, an incentive function for the translation sample with respect to the i-th translation quality instruction is constructed based on the text matching score of the translation sample with respect to the at least two translation quality instructions and the sample label of the translation sample. The incentive function is a function with the text matching score of the i-th translation quality instruction as an independent variable and the incentive value as a dependent variable.

[0115] Step 332: Train a generative language model based on the activation function and prediction loss of the translation samples for at least two translation quality indicators.

[0116] Optionally, based on the activation functions and prediction losses of the translation sample for at least two translation quality indicators, at least two loss functions for the translation sample for the at least two translation quality indicators are constructed, and the generative language model is trained based on the loss function set. The training objectives are to ensure that the generative language model achieves as high a text matching score as possible for the translation quality indicator corresponding to the sample label and as low a prediction loss as possible, and to ensure that the generative language model achieves as low a text matching score as possible for the translation quality indicator not corresponding to the sample label and as high a prediction loss as possible.

[0117] Optionally, based on the excitation function and prediction loss of the translation sample for the i-th translation quality instruction, a loss function of the translation sample for the i-th translation quality instruction is constructed, where the loss function is the sum of the excitation function and the prediction loss.

[0118] Optionally, the prediction loss of the translation sample for at least two translation quality indicators is a cross-entropy loss. The prediction loss is calculated as shown below. The prediction loss has a loss value on one side and a first ratio on the other side. The first ratio is the negative of the ratio of the probability product and the length of the target text, where the probability product is the product of the true probability and the predicted probability.

[0119]

[0120] Where n is the length of the target language text; c is the translation quality instruction; x is the source language text in the translation sample; y is the translation quality instruction. j represents the jth target subword unit, the starting value of j is 1; y ＜j represents the 1st to j-1th target subword units of the target language sentence in the translation sample; P(y j |c,x,y ＜j ) can be understood as the probability that the jth target subword unit generated by the generative language model is the jth target subword unit in the target language text based on the generation of the 1st to j-1th target subword units for the source language text x under the instruction of the translation quality instruction c, that is, P(y j |c,x,y ＜j ) can be called the predicted probability.

[0121] Optionally, the incentive function of the translation sample for the i-th translation quality instruction can be recorded as reward exp The incentive function is a function that takes the translation index of the i-th translation quality instruction as the independent variable and the incentive value as the dependent variable. Alternatively, the incentive function is a function that takes the text matching score of the i-th translation quality instruction as the independent variable and the incentive value as the dependent variable.

[0122] Therefore, the loss function of the generative language model is shown below.

[0123] loss = L1 + reward exp

[0124] Optionally, during the training process of the generative language model, when training is performed based on the loss function of the i-th translation quality instruction, the text matching score and prediction probability of the i-th translation quality instruction are used as independent variables, that is, according to the adjustment of the model parameters of the generative language model, the text matching score and prediction probability of the i-th translation quality instruction will change, and the text matching scores and prediction probabilities of other translation quality instructions are fixed, that is, according to the adjustment of the model parameters of the generative language model, the text matching scores and prediction probabilities of other translation quality instructions will not change.

[0125] The training goal of a generative language model is to reduce the loss for labeled instructions and increase the loss for unlabeled instructions. Alternatively, the goal is to increase the difference between the loss for labeled and unlabeled instructions. Alternatively, the goal is to make the text matching score and prediction probability for labeled instructions higher than those for unlabeled instructions. The i-th translation quality instruction can be either a labeled instruction or an unlabeled instruction.

[0126] In summary, the method provided in the embodiment of the present application shows the construction process of the loss function of the generative language model. The loss function of the generative language model includes two parts, one is the cross-entropy loss, and the other is the activation function. The cross-entropy loss is used to indicate the difference between the true probability and the predicted probability, and the activation function is used to indicate the incentive method of the translation sample for different translation quality instructions. The multi-task learning method is adopted to enable the generative language model to explicitly learn the translation evaluation ability by comparison and to translate based on the translation evaluation ability. At the same time, if the translation quality level is divided based on human evaluation standards, then the translation evaluation ability of the trained generative language model can also be aligned with the human translation evaluation standard.

[0127] Next, a method for constructing the activation function is shown. In an optional embodiment, Figure 6 A flowchart of a method for training a generative language model according to an exemplary embodiment of the present application is shown. Step 331 can be replaced by steps 510 to 530.

[0128] Step 510: Calculate a translation index of the translation sample for the at least two translation quality instructions based on the text matching scores of the translation sample for the at least two translation quality instructions. The translation index is an index raised to the power of the text matching score with a natural constant as the base.

[0129] For example, the text matching score of the i-th translation quality instruction among the at least two translation quality instructions is recorded as s i , then the translation index of the i-th translation quality instruction is exp(e i ), exp(·) represents the exponential function with the natural constant e as the base, exp(s i ) is equivalent to

[0130] Step 520: Obtain a translation index of a label instruction corresponding to a sample label of the translation sample, where the label instruction is a translation quality instruction of at least two translation quality instructions having a translation quality level that is the same as the translation quality level of the sample label.

[0131] Optionally, the label instruction is a translation quality instruction in which the translation quality level of the at least two translation quality instructions is the same as the translation quality level of the sample label. The at least two translation quality instructions can be divided into label instructions and non-label instructions. The non-label instruction is a translation quality instruction in which the translation quality level of the at least two translation quality instructions is different from the translation quality level of the sample label.

[0132] Step 530: Construct at least two activation functions based on the translation index of the label instruction and the translation indexes of at least two translation quality instructions.

[0133] In an optional embodiment, step 530 may be implemented as steps 531 to 533 .

[0134] Step 531 : For the i-th translation quality instruction among the at least two translation quality instructions, construct a first intermediate formula, where the first intermediate formula is a cumulative formula of the translation indices of the at least two translation quality instructions.

[0135] Optionally, the first intermediate formula is a formula with the translation index of the i-th translation quality instruction as an independent variable. Alternatively, the first intermediate formula is a formula with the text matching score of the i-th translation quality instruction as an independent variable.

[0136] Exemplarily, the at least two translation quality instructions are a first translation quality instruction, a second translation quality instruction, and a third translation quality instruction. The translation index of the first translation quality instruction is denoted as exp(s1), the translation index of the second translation quality instruction is denoted as exp(s2), and the translation index of the third translation quality instruction is denoted as exp(s3). When the i-th translation quality instruction is the first translation quality instruction, a first intermediate formula (exp(s1)+exp(s2)+exp(s3)) is constructed. The first intermediate formula is a formula with the translation index exp(s1) of the i-th translation quality instruction as the independent variable. In other words, the first intermediate formula is a formula with the text matching score s1 of the i-th translation quality instruction as the independent variable.

[0137] Step 532: Construct an excitation function corresponding to the i-th translation quality instruction, where one side of the excitation function is the excitation value, and the other side of the excitation function is the negative of the ratio of the translation index of the label instruction to the first intermediate formula. The excitation function is a function with the translation index of the i-th translation quality instruction as the independent variable and the excitation value as the dependent variable.

[0138] Optionally, the incentive function corresponding to the i-th translation quality instruction is a function with the translation index of the i-th translation quality instruction as the independent variable and the incentive value as the dependent variable. Alternatively, the incentive function is a function with the text matching score of the i-th translation quality instruction as the independent variable and the incentive value as the dependent variable.

[0139] Reward functionexp The expression is shown below.

[0140]

[0141] Where s exp Indicates the text matching score of the tag instruction; s1, s2, ..., s n represents the text matching score corresponding to at least two translation quality instructions; exp(·) represents the exponential function with the natural constant e as the base, exp(s i ) is equivalent to

[0142] Step 533: Update i to i+1, and continue to execute the above two steps until at least two activation functions are constructed.

[0143] Optionally, i is updated to i+1, and the above steps 531 and 532 are continued to be performed until at least two activation functions are constructed.

[0144] In summary, in the embodiment of the present application, when constructing the incentive function, it is first necessary to convert the text matching score into a translation index. The translation index is an index with the natural constant e as the base and the text matching score as the power. Since the text matching scores are all positive numbers, the translation indexes are all positive numbers greater than 1. Converting the text matching score into a text translation index can ensure the monotonicity of the incentive function, so that the generative language model can be correctly trained based on the incentive function. At the same time, the translation index of the label instruction is obtained, and then based on the translation index of the label instruction and the translation index of at least two translation quality instructions, an incentive function is constructed so that the incentive function can encourage the generative language model to learn the difference between the label instruction and the non-label instruction, so that the generative language model can explicitly learn the translation evaluation ability.

[0145] Furthermore, the method for constructing an incentive function is also shown. The incentive function corresponding to the i-th translation quality instruction is a function with the translation index of the i-th translation quality instruction as the independent variable and the incentive value as the dependent variable. Alternatively, the incentive function is a function with the text matching score of the i-th translation quality instruction as the independent variable and the incentive value as the dependent variable. When the i-th translation quality instruction is a label instruction, the incentive value decreases as the text matching score corresponding to the i-th translation quality instruction increases. Since the loss value of the generative language model is positively correlated with the incentive value, the loss value also decreases, which means that the generative language model is encouraged to obtain a higher text matching score when translating for label instructions. When the i-th translation quality instruction is a non-label instruction, the incentive value increases as the text matching score corresponding to the i-th translation quality instruction decreases. Since the loss value of the generative language model is positively correlated with the incentive value, the loss value also increases, which means that the generative language model is encouraged to obtain a lower text matching score when translating for non-label instructions, enabling the generative language model to learn translation evaluation capabilities.

[0146] In an optional embodiment, the target language text in the translation sample is obtained by translating the source language text using a machine translation model. The machine translation model includes the generative language model to be trained in this application, and the machine translation model may also include other machine translation models that have been put into use or are being trained. The embodiment of this application does not limit the type, principle, etc. of the machine translation model. The target language text in the translation sample is obtained by inputting the source language text in the translation sample into the machine translation model. The target language text in the multiple translation samples used in the training process of the generative language model can be obtained by translating the source language text based on multiple machine translation models. Since the error distribution of the machine translation model and the artificially synthesized translation is usually different, obtaining the target language text based on the machine translation model and then using the translation sample including the target language text to train the generative language model can enable the generative language model to learn more specifically the impact of errors that are difficult to detect in artificially synthesized translations on the translation quality, thereby improving the generative language model.

[0147] 2. Text Generation

[0148] In addition to its application in the aforementioned machine translation field, generative language model training methods can also be applied to text generation. This includes the ability to generate a variety of texts, such as blog posts, long articles, short stories, summaries, scripts, questionnaires, surveys, and social media posts, based on user prompts.

[0149] Figure 7 A flow chart of a method for training a generative language model provided by an exemplary embodiment of the present application is shown. The method can be executed by a computer device, which can be Figure 1The computer device shown. The method includes:

[0150] Step 610: Obtaining a generated sample, where generating a sample includes generating an input text, generating a target text, and a sample label, where the sample label is used to indicate the content quality level of the generated target text; and obtaining at least two generation quality instructions, where the generation quality instructions are used to indicate the expected content quality level.

[0151] Among them, the generated samples used for the generative language model include one or more. The embodiment of the present application uses one of the generated samples as an example for illustration, but is not limited to this.

[0152] Optionally, the technician divides the generated samples into content quality levels according to the content quality of the generated target text in the generated samples; or, the generative language model divides the generated samples into content quality levels according to the content quality of the generated target text in the generated samples. The content quality score interval is [0, 100]; the generated target text with a content quality score in [0, 85) is divided into low-quality text; the generated target text with a content quality score in [85, 90] is divided into medium-quality text; and the generated target text with a content quality score in (90, 100] is divided into high-quality text. The content quality score interval can also be [0, 1.0], [0, 10], etc. Based on different content quality score intervals and different content quality level evaluation criteria, the content quality levels corresponding to the generated samples are different. It should be noted that, with respect to the division of content quality levels, the embodiment of the present application uses the content quality levels including high-quality text, medium-quality text and low-quality text as an example, but is not limited to this.

[0153] Optionally, the generation quality instruction includes an instruction for indicating the desired content quality level, for example, the generation quality instruction is "generate high-quality text content"; or, the generation quality instruction is "generate medium-quality text content"; or, the generation quality instruction is "generate low-quality text content", etc. Alternatively, the generation quality instruction includes a prompt word for indicating the desired content quality level, for example, if the prompt word is "high quality", the generation quality instruction may be "generate high-quality text content" or "generate high-quality content based on input", etc.; or, if the prompt word is "medium quality", the generation quality instruction may be "generate medium-quality text content" or "generate medium-quality content based on input", etc.; or, if the prompt word is "low quality", the generation quality instruction may be "generate low-quality text content" or "generate low-quality content based on input", etc.

[0154] Optionally, multiple quality generation instructions may be included for the same content quality level. Multiple quality generation instructions corresponding to the same content quality level are referred to as one quality generation instruction. For example, for the content quality level of "high-quality text," the corresponding quality generation instructions include "generate high-quality text," "answer as an expert," and "generate high-quality text based on prompts." These quality generation instructions can be collectively referred to as one quality generation instruction.

[0155] In some embodiments, the generated target text in the generated sample is obtained by predicting the generated input text using a text generation model. The text generation model includes the generative language model to be trained in this application, and text generation may also include other text generation models that have been put into use or are being trained. The embodiments of this application do not limit the type, principle, etc. of the text generation model. The generated target text in the generated sample is obtained by inputting the generated input text in the generated sample into the text generation model. The generated target texts in the multiple generated samples used in the training process of the generative language model can be obtained by predicting the generated input text based on multiple text generation models.

[0156] Step 620: For each generation quality instruction, calculate the text matching score and prediction loss of the generated sample for the generation quality instruction. The text matching score is used to indicate the probability of the generative language model predicting the generated target text based on the generation quality instruction and the generated input text. The prediction loss is used to indicate the loss of the generative language model predicting the generated target text based on the generation quality instruction and the generated input text.

[0157] Optionally, the generative language model concatenates the generated sample with each generation quality instruction and calculates the text matching score and prediction loss of the generated sample for at least two generation quality instructions. Alternatively, the generated sample and each generation quality instruction are input into the generative language model, and the generative language model calculates the text matching score and prediction loss of the generated sample for each generation quality instruction. Alternatively, for at least two generation quality instructions, the generative language model calculates the text matching score and prediction loss of the generated sample for each generation quality instruction.

[0158] Optionally, the text matching score is used to indicate the probability that the generative language model predicts the generated target text according to the generation quality instruction. Alternatively, the text matching score is used to indicate the probability that the generative language model predicts the generated input text in the generated sample to become the generated target text in the generated sample according to the generation quality instruction.

[0159] Optionally, the prediction loss is used to indicate the loss of the generative language model in predicting the target text according to the generation quality instruction. Alternatively, the prediction loss is used to indicate the degree of difference between the text predicted by the generative language model according to the generation quality instruction and the generated target text. Alternatively, the prediction loss is used to indicate the difficulty of the generative language model in predicting the target text according to the generation quality instruction.

[0160] Exemplarily, there are n types of generation quality instructions; the text matching score and prediction loss of the generation sample for the first type of generation quality instruction, the text matching score and prediction loss of the generation sample for the second type of generation quality instruction,…, the text matching score of the generation sample for the n-1th type of generation quality instruction, and the text matching score of the generation sample for the nth type of generation quality instruction are calculated respectively.

[0161] The text matching scores and prediction losses of the above-mentioned generated samples for at least two generation quality instructions are calculated simultaneously by the generative language model; or, are calculated one by one in sequence by the generative language model; or, are calculated in batches by the generative language model.

[0162] In an optional embodiment, the at least two generation quality instructions include an i-th generation quality instruction, the generated target text includes at least one target unit, i being a positive integer; the generated target text is a sentence, paragraph, or article; and the target unit is a character, subword, or word. A character refers to a basic component of a text, such as each character and punctuation mark in a Chinese text; a subword refers to the result of segmenting a long word, an unknown word, or an unknown phrase in a text, such as the word "natural" may be segmented into the subword unit "natura" and the subword unit "l" during subword segmentation; and a word refers to a meaningful lexical unit in a text, such as the word "a cat" in the sentence "I have a cat" is typically segmented into a single word. Depending on the configuration of the generative language model, the generated target text can be segmented into different types of target units. For example, if the generative language model is configured with a subword segmentation module, the generated target text is segmented into at least one target unit. Alternatively, depending on the configuration of the server, the generated target text can be segmented into different types of target units. For example, if the server is configured with a word segmentation module, the generated target text is segmented into at least one target word.

[0163] In some embodiments, for each quality generation instruction, a text matching score of the generated sample against the quality generation instruction is calculated; this is equivalent to inputting the generated sample and the i-th quality generation instruction from at least two quality generation instructions into a generative language model, and calculating the text matching score of the generated sample against the i-th quality generation instruction; updating i to i+1, and continuing to perform one of the above steps until the text matching score of the generated sample against the at least two quality generation instructions is calculated. For each quality generation instruction, a prediction loss of the generated sample is calculated; this is equivalent to inputting the generated sample and the i-th quality generation instruction from at least two quality generation instructions into a generative language model, and calculating the prediction loss of the generated sample against the i-th quality generation instruction; updating i to i+1, and continuing to perform one of the above steps until the prediction loss of the generated sample against the at least two quality generation instructions is calculated. Where i is a positive integer.

[0164] In some embodiments, for each generation quality instruction, calculating the text matching score of the generation sample for the generation quality instruction can be implemented as follows: calculating at least one unit matching score of the generation sample for the i-th generation quality instruction; calculating the text matching score of the generation sample for the i-th generation quality instruction, the text matching score being the ratio of the second cumulative sum to the text length of the generated target text, and the second cumulative sum being the cumulative sum of at least one unit matching score.

[0165] In some embodiments, calculating at least one unit matching score of the generated sample for the i-th generation quality instruction can be implemented as follows: for the i-th generation quality instruction, calculating the unit matching score of the j-th target unit in the generated target text, where the j-th unit matching score is used to indicate the probability of the generative language model predicting the j-th target unit for the generated sample based on the i-th generation quality instruction, where j is a positive integer; updating j to j+1, and continuing to perform one of the above steps until at least one unit matching score of the generated sample for the i-th generation quality instruction is calculated.

[0166] Step 630: Train the generative language model based on the text matching scores and prediction losses corresponding to the generated samples for the at least two generation quality instructions and the sample labels of the generated samples.

[0167] Optionally, the generative language model is trained based on the text matching scores and prediction losses of the generated samples for at least two types of generation quality instructions and the sample labels of the generated samples. The training goal is to make the text matching scores of the generation quality instructions corresponding to the sample labels calculated by the generative language model for the generated samples as high as possible and the prediction losses as low as possible; and to make the text matching scores of the generation quality instructions that do not correspond to the sample labels calculated by the generative language model for the generated samples as low as possible and the prediction losses as high as possible.

[0168] Optionally, a sampling backpropagation algorithm is used to train the generative language model based on the text matching scores and prediction losses of the generated samples for at least two quality generation instructions, as well as the sample labels of the generated samples. Specifically, a loss function set is constructed based on the text matching scores and prediction losses of the generated samples for at least two quality generation instructions, as well as the sample labels of the generated samples. The gradient of the loss value of the generated samples with respect to the model parameters of the generative language model is calculated. The gradient indicates the direction and rate at which the loss value changes with the model parameters. The model parameters of the generative language model are continuously updated based on the gradient optimization algorithm. When the number of updates reaches a threshold or the model parameters converge, the trained generative language model is obtained. It should be noted that the model parameters updated vary depending on the training method used. For example, in the pre-training phase of the generative language model, the updated model parameters are all model parameters of the generative language model; in the fine-tuning phase of the generative language model, such as using instruction fine-tuning or LoRA, the updated model parameters are only a portion of the model parameters of the generative language model. LoRA introduces a low-rank matrix during model training to reduce the number of model parameters that need to be updated during model training.

[0169] In some embodiments, training a generative language model based on the text matching scores and prediction losses corresponding to at least two generation quality instructions of the generated samples and the sample labels of the generated samples can be implemented as follows: for each generation quality instruction, constructing an excitation function for the generated sample for the generation quality instruction based on the text matching scores of the generated sample for at least two generation quality instructions and the sample labels of the generated samples, the excitation function being used to instruct the generative language model to distinguish between the content quality level of the generation quality instruction and the content quality level of the sample label; and training the generative language model based on the excitation function and prediction loss of the generated sample for at least two generation quality instructions.

[0170] In some embodiments, for each generation quality instruction, an incentive function for the generation sample for the generation quality instruction is constructed based on the text matching score of the generation sample for at least two generation quality instructions and the sample label of the generation sample. This can be implemented as follows: based on the text matching score of the generation sample for at least two generation quality instructions, a generation index of the generation sample for at least two generation quality instructions is calculated, where the generation index is an exponent with a natural constant as the base and the text matching score as the power; the generation index of the label instruction corresponding to the sample label of the generation sample is obtained, where the label instruction is a generation quality instruction with the same content quality level as the content quality level of the sample label among the at least two generation quality instructions; and at least two incentive functions are constructed based on the generation index of the label instruction and the generation index of at least two generation quality instructions.

[0171] In some embodiments, at least two excitation functions are constructed based on the generation index of the label instruction and the generation index of at least two generation quality instructions, which can be implemented as follows: for the i-th generation quality instruction among the at least two generation quality instructions, a second intermediate formula is constructed, and the second intermediate formula is a cumulative formula of the generation index of the at least two generation quality instructions; an excitation function corresponding to the i-th generation quality instruction is constructed, one side of the excitation function is the excitation value, and the other side of the excitation function is the negative of the ratio of the generation index of the label instruction to the first intermediate formula, and the excitation function is a function with the generation index of the i-th generation quality instruction as the independent variable and the excitation value as the dependent variable; i is updated to i+1, and the above two steps are continued until at least two excitation functions are constructed.

[0172] In summary, the method provided in the embodiment of the present application takes a generated sample with a sample label and at least two generated quality instructions as a generated sample, calculates the text matching score and prediction loss of the generated sample for each generated quality instruction, and trains the generative language model based on the calculated text matching score and prediction loss of the generated sample for at least two generated quality instructions. Since the generated sample has a sample label, the sample label is used to indicate the content quality level of the generated target text of the generated sample, and the generated quality instruction can be used to indicate the content quality level of the text predicted by the generated language model for the generated input text of the generated sample; when training the generative language model, it is only necessary to train the generated language model based on the content quality indicated by the sample label and the generated quality instruction. The generative language model is trained specifically based on the quality level, ensuring that the text matching score calculated for the generated samples for the generated quality instructions corresponding to the sample labels is as high as possible and the prediction loss is as low as possible. The text matching score calculated for the generated samples for the generated quality instructions that do not correspond to the sample labels is as low as possible and the prediction loss is as high as possible. This enables the trained generative language model to generate text with different content quality levels. In other words, the generative language model learns text evaluation capabilities and can make predictions based on these text evaluation capabilities. Furthermore, if the content quality levels are divided based on human evaluation criteria, the text evaluation capabilities of the trained generative language model can also be aligned with human text evaluation standards. Simultaneously training the generative language model from multiple dimensions enhances its text evaluation capabilities.

[0173] In addition to the fields of machine translation and text generation, the embodiments of the present application can also be applied to the search field, question-answering / dialogue field, etc. The specific implementation methods are similar to those in the text generation field and will not be repeated here.

[0174] Next, examples are given to illustrate that the translation quality levels include high-quality translation, medium-quality translation and low-quality translation, the source language text and the target language text are the source language sentence and the target language sentence respectively, and the translation quality instructions include high-quality instruction, medium-quality instruction and low-quality instruction.

[0175] Step 1: Get translation samples.

[0176] Translation samples include source language sentences and target language sentences.

[0177] Optionally, the target language sentence is generated by a machine translation model based on the source language sentence. The machine translation model may include the generative language model in this application, or may include other generative language models or machine translation models.

[0178] Optionally, the target language sentences in the translation sample are divided into translation quality levels. The translation quality level division is based on the translation quality of the target language sentences. For example, the translation quality score interval is [0, 100]; the target language sentences with translation quality scores in [0, 85] are divided into low-quality translations; the target language sentences with translation quality scores in [85, 90] are divided into medium-quality translations; and the target language sentences with translation quality scores in (90, 100] are divided into high-quality translations. The translation quality score interval can also be [0, 1.0], [0, 10], etc. Based on different translation quality score intervals and different translation quality level evaluation criteria, the translation quality levels corresponding to the translation samples are different.

[0179] Optionally, based on the classification level of the target language sentence, the sample label of the translation sample corresponding to the target language sentence is determined. That is, when the target language sentence is a high-quality translation, the corresponding translation data is marked with a "high-quality" sample label; when the target language sentence is a medium-quality translation, the corresponding translation data is marked with a "medium-quality" sample label; when the target language sentence is a low-quality translation, the corresponding translation data is marked with a "low-quality" sample label. In order to facilitate storage, the sample labels can also be in English form, such as the sample label of "high quality" can be expressed as "high" or "H"; the sample label of "medium quality" can be expressed as "mid" or "M"; the sample label of "low quality" can be expressed as "low" or "L", etc.

[0180] Step 2: Calculate the matching scores between the translation sample and each translation quality instruction.

[0181] Optionally, the translation quality instruction includes an instruction for indicating a desired translation quality level, for example, a translation quality instruction of "generate a high-quality translation"; or a translation quality instruction of "generate a medium-quality translation"; or a translation quality instruction of "generate a low-quality translation", etc. Alternatively, the translation quality instruction includes a prompt word for indicating a desired translation quality level, for example, a translation quality instruction of "generate a high-quality translation" with a prompt word of "high quality"; or a translation quality instruction of "generate a medium-quality translation" with a prompt word of "medium quality"; or a translation quality instruction of "generate a low-quality translation" with a prompt word of "low quality", etc. In other words, the translation quality instruction includes at least one of a high-quality instruction, a medium-quality instruction, and a low-quality instruction.

[0182] The target language sentence in the translation sample includes at least one target sub-word unit. For example, the target language sentence is "I have a cat."; or the target sub-word unit is {"I", "have", "a", "cat", "."}. For the target language sentence "I have a cat.", the target sub-word unit is {"I", "have", "a", "cat", "."}.

[0183] First, the probability of the generative language model predicting the tth subword unit based on the translation quality instruction is calculated, where t is a positive integer. Alternatively, the probability of the generative language model predicting the tth target subword unit is calculated based on the translation quality instruction.

[0184] The t-th subword unit is the t-th target subword unit in the target language sentence in the translation sample. The probability of predicting the t-th subword unit is recorded as P(y t |c,x,y ＜t ), where y t represents the t-th subword unit, c represents the translation quality instruction, x represents the source language sentence in the translation sample, y ＜t represents the 0th to t-1th target subword units of the target language sentence in the translation sample; the formula P(y t |c,x,y ＜t ) can be understood as the probability that the tth subword unit generated is the tth target subword unit in the target language sentence, based on the fact that the generative language model has generated the 0th to t-1th target subword units for the source language sentence x under the instruction of the translation quality instruction c.

[0185] Based on the calculated generative language model's translation prediction of the source language sentence in the translation sample based on the translation quality directive, the probability of at least one target subword unit in the target language sentence in the translation sample is calculated. The matching score between the translation sample and the translation quality directive is calculated. This matching score can also be referred to as the text matching score of the translation sample for the translation quality directive. The matching score calculation formula is shown below.

[0186]

[0187] Where n is the length of the target language sentence; c is the translation quality instruction; x is the source language sentence in the translation sample; y is the translation quality instruction. ＜t represents the 0th to t-1th target subword units of the target language sentence in the translation sample; P(y t |c,x,y ＜t ) can be understood as the probability that the tth subword unit generated is the tth target subword unit in the target language sentence, based on the fact that the generative language model has generated the 0th to t-1th target subword units for the source language sentence x under the instruction of the translation quality instruction c.

[0188] During training, the goal is to maximize the matching scores between a translation sample and each translation quality indicator, corresponding to the translation sample's label. For example, if a translation sample is labeled "high quality," the goal is to maximize the matching scores between the translation sample and the high-quality indicator, the translation sample and the medium-quality indicator, and the translation sample and the low-quality indicator. This expectation allows us to construct the incentive function used in generative language model training.

[0189] Step 3: construct an incentive function based on the calculated text matching scores of the translation samples for each translation quality instruction.

[0190] Optionally, the incentive function is used to instruct the generative language model to distinguish between the translation quality level of the translation quality instruction and the translation quality level of the sample label; or, the incentive function is used to instruct the generative language model to generate a target language sentence with a high degree of match with the label instruction and a low degree of match with the non-label instruction, the label instruction is a translation quality instruction that matches the translation quality level of the sample label, and the non-label instruction is a translation quality instruction that does not match the translation quality level of the sample label; or, the incentive function is used to incentivize the generative language model to generate a target language sentence with a higher text matching score with the label instruction than with the non-label quality.

[0191] Exemplarily, each translation quality instruction includes a high-quality instruction, a medium-quality instruction, and a low-quality instruction. The incentive function is constructed based on a first matching score, a second matching score, and a third matching score. The first matching score is the text matching score of the translation sample for the high-quality instruction, the second matching score is the text matching score of the translation sample for the medium-quality instruction, and the third matching score is the text matching score of the translation sample for the low-quality instruction. One side of the incentive function is the calculated incentive value, and the other side is the negative of the ratio of the label index to the first sum value. The label index is calculated based on the text matching score of the translation quality instruction corresponding to the sample label. The first sum value is the sum of the first index, the second index, and the third index. The label index is an index with a natural number e as the base and the text matching score of the label instruction as a power. The first index is an index with e as the base and the first matching score as a power, the second index is an index with e as the base and the second matching score as a power, and the third index is an index with e as the base and the third matching score as a power. The incentive function is shown in the following formula.

[0192]

[0193] In the formula, reward exp is the excitation value; exp(·) represents the exponential function with e as the base; s exp is the instruction matching score corresponding to the label instruction; s high is the first matching score, i.e., the text matching score of the translation sample against the high-quality instruction; mid is the second matching score, i.e., the text matching score of the translation sample against the medium-quality instruction; low is the third matching score, i.e., the text matching score of the translation sample against the low-quality instruction.

[0194] Step 4: Calculate the cross entropy loss between the translation sample and each translation quality indicator.

[0195] The calculation formula of cross entropy loss is as follows.

[0196]

[0197] Where n is the length of the target language sentence; c is the translation quality instruction; x is the source language sentence in the translation sample; y is the translation quality instruction. ＜t represents the 0th to t-1th target subword units of the target language sentence in the translation sample; P(y t |c,x,y ＜t ) can be understood as the probability that the tth subword unit generated is the tth target subword unit in the target language sentence, based on the fact that the generative language model has generated the 0th to t-1th target subword units for the source language sentence x under the instruction of the translation quality instruction c.

[0198] In step 5, the generative language model is trained based on the cross entropy loss and the activation function of the translation samples and each translation quality indicator.

[0199] The loss function of the generative language model is shown below.

[0200] loss=K1+reward exp

[0201] Optionally, during the training process of the generative language model, when training is performed based on the loss function of the i-th translation quality instruction, the text matching score and prediction probability of the i-th translation quality instruction are used as independent variables, that is, according to the adjustment of the model parameters of the generative language model, the text matching score and prediction probability of the i-th translation quality instruction will change, and the text matching scores and prediction probabilities of other translation quality instructions are fixed, that is, according to the adjustment of the model parameters of the generative language model, the text matching scores and prediction probabilities of other translation quality instructions will not change.

[0202] The training goal of a generative language model is to reduce the loss for labeled instructions and increase the loss for unlabeled instructions. Alternatively, the goal is to increase the difference between the loss for labeled and unlabeled instructions. Alternatively, the goal is to make the text matching score and prediction probability for labeled instructions higher than those for unlabeled instructions. The i-th translation quality instruction can be either a labeled instruction or an unlabeled instruction.

[0203] In some embodiments, during the use of the generative language model, by inputting a translation quality instruction and a source language text into the generative language model, the generative language model translates the source language text based on the translation quality instruction to obtain the target language text. For example:

[0204] <Example 1>

[0205] Input: Please provide a high-quality translation of the following English sentence into the corresponding Chinese: “The power of belief can overcome any obstacle in life.”

[0206] Output: The power of belief is extremely strong and can help us break through the obstacles in life.

[0207] <Example 2>

[0208] Input: Please provide a medium-quality translation of the following English sentence to obtain the corresponding Chinese sentence: "The power of belief can overcome any obstacle in life."

[0209] Output: Strong belief can overcome many difficulties in life.

[0210] <Example 3>

[0211] Input: Please perform a low-quality translation of the following English sentence to obtain a slightly incorrect Chinese equivalent: "The power of belief can overcome any obstacle in life."

[0212] Output: The power of belief can easily defeat all difficulties in life.

[0213] In some embodiments, during the use of the generative language model, by inputting a content quality instruction and a generated input text into the generative language model, the generative language model predicts and generates a target text based on the content quality instruction and the generated input text. For example:

[0214] <Example 4>

[0215] Input: Please aim for high quality and write a sentence describing the arrival of spring.

[0216] Output: With the gentle breeze, spring comes quietly with a sea of ​​colorful flowers and a vibrant atmosphere. The earth seems to be gently awakened by the warm sunshine, blooming with endless vitality and vigor.

[0217] <Example 5>

[0218] Input: Please write a sentence describing the arrival of spring, aiming for medium quality.

[0219] Output: Spring is here, everything is reviving, the earth is dressed in new clothes, the air is filled with the fragrance of flowers and soil, making people feel the power of life and the arrival of hope.

[0220] <Example 6>

[0221] Input: Please write a sentence describing the arrival of spring, with low quality as the goal.

[0222] Output: Spring is here, the flowers are blooming, the weather is getting warmer, and everything is fine.

[0223] Please refer to Figure 8 , which shows a structural block diagram of a training device for a generative language model provided by an exemplary embodiment of the present application. The device has the function of implementing the above-mentioned training method example of the generative language model, and the function can be implemented by hardware or by hardware executing corresponding software. The device can be the encoding device described above, or it can be set in the encoding device. Figure 8 As shown, the device may include: an acquisition module 710 , a calculation module 720 , and a training module 730 .

[0224] An acquisition module 710 is configured to acquire a training sample, wherein the training sample includes an input text, a target text, and a sample label, wherein the sample label is used to indicate a text quality level of the target text; and acquire at least two text quality instructions, wherein the text quality instructions are used to indicate a desired text quality level;

[0225] a calculation module 720 configured to calculate, for each text quality instruction, a text matching score and a prediction loss of the training sample with respect to the text quality instruction, wherein the text matching score indicates a probability of the generative language model predicting the target text based on the text quality instruction and the input text, and the prediction loss indicates a loss of the generative language model predicting the target text based on the text quality instruction and the input text;

[0226] The training module 730 is configured to train the generative language model based on the text matching scores and prediction losses corresponding to the at least two text quality instructions of the training samples and the sample labels of the training samples.

[0227] In some embodiments, the at least two text qualities include an i-th text quality instruction, the target text includes at least one target unit, and i is a positive integer; the calculation module 720 includes a first calculation submodule and a second calculation submodule.

[0228] A first calculation submodule is configured to calculate at least one unit matching score of a training sample for an i-th text quality instruction;

[0229] The second calculation submodule is used to calculate the text matching score of the training sample for the i-th text quality instruction, where the text matching score is the ratio of the third cumulative sum to the text length of the target text, and the third cumulative sum is the cumulative sum of at least one unit matching score.

[0230] In some embodiments, the first calculation submodule includes a first calculation unit and a first circulation unit.

[0231] A first calculation unit is configured to calculate a unit matching score of a j-th target unit in a target text for an i-th text quality instruction, wherein the j-th unit matching score is used to indicate a probability of the generative language model predicting the j-th target unit based on the training sample based on the i-th text quality instruction, where j is a positive integer;

[0232] The first loop unit is used to update j to j+1 and continue to perform the above step until at least one unit matching score of the training sample for the i-th text quality instruction is calculated.

[0233] In some embodiments, the text matching scores and prediction losses of the training samples for the at least two text quality instructions include at least two text matching scores and at least two prediction losses; the training module 730 includes a construction submodule and a training submodule.

[0234] A construction submodule is configured to construct, for each text quality instruction, an activation function for the training sample with respect to the text quality instruction based on the text matching scores of the training sample with respect to at least two text quality instructions and the sample label of the training sample, wherein the activation function is used to instruct the generative language model to distinguish between the text quality level of the text quality instruction and the text quality level of the sample label;

[0235] A training submodule is used to train a generative language model based on an activation function and a prediction loss for at least two text quality instructions of the training samples.

[0236] In some embodiments, the first construction submodule includes a second calculation unit, an acquisition unit, and a construction unit.

[0237] a second calculation unit, configured to calculate a prediction index of the training sample for the at least two text quality instructions based on the text matching scores of the training sample for the at least two text quality instructions, the prediction index being an exponent raised to a power of the text matching score with a natural constant as a base;

[0238] an acquiring unit, configured to acquire a prediction index of a label instruction corresponding to a sample label of a training sample, wherein the label instruction is a text quality instruction having the same text quality level as the sample label among at least two text quality instructions;

[0239] The construction unit is configured to construct at least two activation functions based on a prediction index of a label instruction and prediction indexes of at least two text quality instructions.

[0240] In some embodiments, the construction unit is further used to construct a third intermediate formula for the i-th text quality instruction among the at least two text quality instructions, where the third intermediate formula is a cumulative formula of the prediction indexes of the at least two text quality instructions; construct an excitation function corresponding to the i-th text quality instruction, where one side of the excitation function is the excitation value, and the other side of the excitation function is the negative of the ratio of the prediction index of the label instruction to the third intermediate formula, and the excitation function is a function with the prediction index of the i-th text quality instruction as the independent variable and the excitation value as the dependent variable; update i to i+1, and continue to execute the above two steps until at least two excitation functions are constructed.

[0241] In some embodiments, the target text in the training sample is obtained by predicting the input text using a text prediction model.

[0242] In an optional embodiment, the training samples include translation samples, the input text includes source language text, the target text includes target language text, the text quality level includes translation quality level, and the text quality instruction includes translation quality instruction.

[0243] The acquisition module 710 is further configured to acquire the translation sample, the translation sample including the source language text, the target language text, and the sample label, the sample label being used to indicate a translation quality level of the target language text; and acquire at least two translation quality instructions, the translation quality instructions being used to indicate a desired translation quality level;

[0244] The calculation module 720 is further configured to calculate, for each translation quality instruction, a text matching score and a prediction loss of the translation sample with respect to the translation quality instruction, wherein the text matching score indicates a probability of the generative language model predicting the target language text based on the translation quality instruction and the source language text, and the prediction loss indicates a loss of the generative language model predicting the target language text based on the translation quality instruction and the source language text.

[0245] The training module 730 is further configured to train the generative language model based on the text matching scores and prediction losses of the translation samples corresponding to the at least two translation quality instructions, and the sample labels of the translation samples.

[0246] In some embodiments, the at least two translation quality instructions include an i-th translation quality instruction, the target language text includes at least one target sub-word unit, and i is a positive integer.

[0247] The first calculation submodule is further configured to calculate at least one subword matching score of the translation sample for the i-th translation quality instruction;

[0248] The second calculation submodule is further configured to calculate a text matching score of the translation sample for the i-th translation quality instruction, where the text matching score is a ratio of the first cumulative sum to the text length of the target language text, and the first cumulative sum is a cumulative sum of the at least one subword matching score.

[0249] In some embodiments, a first calculation unit is configured to calculate, for the i-th translation quality instruction, a subword matching score of the j-th target subword unit in the target language text, where the j-th subword matching score is used to indicate the probability of the generative language model predicting the translation sample based on the i-th translation quality instruction to obtain the j-th target subword unit, where j is a positive integer; and a first loop unit is configured to update j to j+1 and continue to perform one of the above steps until at least one subword matching score of the translation sample for the i-th translation quality instruction is calculated.

[0250] In some embodiments, the text matching scores and prediction losses of the translation sample for the at least two translation quality instructions include at least two text matching scores and at least two prediction losses.

[0251] The construction submodule is further configured to construct, for each translation quality instruction, an incentive function for the translation sample with respect to the translation quality instruction based on the text matching score of the translation sample with respect to the at least two translation quality instructions and the sample label of the translation sample, wherein the incentive function is configured to instruct the generative language model to distinguish between the translation quality level of the translation quality instruction and the translation quality level of the sample label;

[0252] The training submodule is further configured to train the generative language model based on the activation function and prediction loss of the translation sample for the at least two translation quality instructions.

[0253] In some embodiments, the second calculation unit is further used to calculate a translation index of the translation sample for the at least two translation quality instructions based on the text matching scores of the translation sample for the at least two translation quality instructions, where the translation index is an exponent with a natural constant as the base and the text matching score as the power. The acquisition unit is further used to obtain the translation index of a label instruction corresponding to a sample label of the translation sample, where the label instruction is a translation quality instruction in the at least two translation quality instructions whose translation quality level is the same as the translation quality level of the sample label. The construction unit is further used to construct at least two of the incentive functions based on the translation index of the label instruction and the translation indices of the at least two translation quality instructions.

[0254] In some embodiments, the construction unit is further used to construct a first intermediate formula for the i-th translation quality instruction among the at least two translation quality instructions, where the first intermediate formula is a cumulative formula of the translation indexes of the at least two translation quality instructions; construct an excitation function corresponding to the i-th translation quality instruction, where one side of the excitation function is an excitation value, and the other side of the excitation function is the negative of the ratio of the translation index of the label instruction to the first intermediate formula, and the excitation function is a function with the translation index of the i-th translation quality instruction as an independent variable and the excitation value as a dependent variable; update i to i+1, and continue to perform the above two steps until the at least two excitation functions are constructed.

[0255] In some embodiments, the target language text in the translation sample is obtained by translating the source language text using a machine translation model.

[0256] In an optional embodiment, the training sample includes a generated sample, the input text includes generating an input text, the target text includes generating a target text, the text quality level includes a content quality level, and the text quality instruction includes generating a quality instruction;

[0257] The acquisition module 710 is further configured to acquire the generated sample, wherein the generated sample includes the generated input text, the generated target text, and the sample label, wherein the sample label is used to indicate the content quality level of the generated target text; and acquire at least two of the generated quality instructions, wherein the generated quality instructions are used to indicate the expected content quality level;

[0258] The calculation module 720 is further configured to calculate, for each of the generation quality instructions, a text matching score and a prediction loss of the generated sample with respect to the generation quality instruction, wherein the text matching score is used to indicate the probability of the generative language model predicting the generated target text based on the generation quality instruction and the generated input text, and the prediction loss is used to indicate the loss of the generative language model predicting the generated target text based on the generation quality instruction and the generated input text;

[0259] The training module 730 is further configured to train the generative language model based on the text matching scores and prediction losses of the generated samples corresponding to the at least two generation quality instructions and the sample labels of the generated samples.

[0260] In some embodiments, the at least two translation quality instructions include an i-th translation quality instruction, the target language text includes at least one target sub-word unit, and i is a positive integer.

[0261] The first calculation submodule is further configured to calculate at least one unit matching score of the generated sample for the i-th generation quality instruction;

[0262] The second calculation submodule is also used to calculate the text matching score of the generated sample for the i-th generation quality instruction, the text matching score is the ratio of the second cumulative sum to the text length of the generated target text, and the second cumulative sum is the cumulative sum of at least one unit matching score.

[0263] In some embodiments, the first calculation unit is used to calculate the unit matching score of the jth target unit in the generated target text for the i-th generation quality instruction, where the j-th unit matching score is used to indicate the probability of the generative language model predicting the j-th target unit for the generated sample based on the i-th generation quality instruction, where j is a positive integer; the first loop unit is used to update j to j+1 and continue to execute one of the above steps until at least one unit matching score of the generated sample for the i-th generation quality instruction is calculated.

[0264] In some embodiments, generating text matching scores and prediction losses of the sample for the at least two generation quality instructions includes at least two text matching scores and at least two prediction losses.

[0265] The construction submodule is further configured to construct, for each generation quality instruction, an incentive function for the generated sample with respect to the generation quality instruction based on the text matching score of the generated sample with respect to at least two generation quality instructions and the sample label of the generated sample, wherein the incentive function is configured to instruct the generative language model to distinguish between the content quality level of the generation quality instruction and the content quality level of the sample label;

[0266] The training submodule is further configured to train the generative language model based on the activation function and prediction loss of the generated samples for at least two generation quality instructions.

[0267] In some embodiments, the second calculation unit is further used to calculate the generation index of the generated sample for at least two generation quality instructions based on the text matching score of the generated sample for at least two generation quality instructions, where the generation index is an exponent with a natural constant as the base and the text matching score as the power; the acquisition unit is further used to obtain the generation index of the label instruction corresponding to the sample label of the generated sample, where the label instruction is a generation quality instruction with the same content quality level as the content quality level of the sample label among the at least two generation quality instructions; the construction unit is further used to construct at least two excitation functions based on the generation index of the label instruction and the generation index of at least two generation quality instructions.

[0268] In some embodiments, the construction unit is further used to construct a second intermediate formula for the i-th generation quality instruction among the at least two generation quality instructions, where the second intermediate formula is a cumulative formula of the generation indexes of the at least two generation quality instructions; construct an excitation function corresponding to the i-th generation quality instruction, where one side of the excitation function is the excitation value, and the other side of the excitation function is the negative of the ratio of the generation index of the label instruction to the first intermediate formula, and the excitation function is a function with the generation index of the i-th generation quality instruction as the independent variable and the excitation value as the dependent variable; update i to i+1, and continue to execute the above two steps until at least two excitation functions are constructed.

[0269] In some embodiments, the generated target text in the generated sample is obtained by predicting the generated input text using a text generation model.

[0270] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the devices and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0271] Figure 9 A structural block diagram of a computer device provided by an exemplary embodiment of the present application is shown.

[0272] The computer device 800 includes a central processing unit (CPU) 801, a system memory 804 including a random access memory (RAM) 802 and a read-only memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the CPU 801. The computer device 800 also includes a basic input / output system (I / O system) 806 that facilitates information transmission between various components within the computer device, and a mass storage device 807 for storing an operating system 813, application programs 814, and other program modules 815.

[0273] The basic input / output system 806 includes a display 808 for displaying information and an input device 809 such as a mouse and keyboard for user input. The display 808 and the input device 809 are both connected to the central processing unit 801 via an input / output controller 810 connected to the system bus 805. The basic input / output system 806 may also include an input / output controller 810 for receiving and processing input from a variety of other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 810 also provides output to a display screen, printer, or other types of output devices.

[0274] The mass storage device 807 is connected to the central processing unit 801 via a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer-readable storage medium provide non-volatile storage for the computer device 800. In other words, the mass storage device 807 may include a computer-readable storage medium (not shown) such as a hard disk or a CD-ROM drive.

[0275] Without loss of generality, the computer-readable storage medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable storage instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electronically Erasable Programmable Read-Only Memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, tape cassettes, magnetic tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media are not limited to the above-mentioned ones. The above-mentioned system memory 804 and mass storage device 807 can be collectively referred to as memory.

[0276] The memory stores one or more programs, and the one or more programs are configured to be executed by one or more central processing units 801. The one or more programs contain instructions for implementing the above-mentioned method embodiments. The central processing unit 801 executes the one or more programs to implement the methods provided by the above-mentioned method embodiments.

[0277] According to various embodiments of the present application, the computer device 800 may also be connected to a remote computer device on a network such as the Internet for operation. That is, the computer device 800 may be connected to a network 812 via a network interface unit 811 connected to the system bus 805. Alternatively, the network interface unit 811 may be used to connect to other types of networks or remote computer device systems (not shown).

[0278] The memory also includes one or more programs, which are stored in the memory and include steps executed by a computer device in the method provided in the embodiment of the present application.

[0279] In an exemplary embodiment, a non-transitory computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the above-mentioned training method of the generative language model.

[0280] In an exemplary embodiment, a computer program product is further provided. When the computer program product is executed by a processor, it is used to implement the above-mentioned training method of the generative language model.

[0281] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application do not limit this.

[0282] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for training a generative language model, characterized in that: The method comprises: Acquiring a training sample, the training sample comprising an input text, a target text, and a sample label, the sample label being used to indicate a text quality level of the target text; and acquiring at least two text quality instructions, the text quality instructions being used to indicate a desired text quality level; For each of the text quality instructions, calculating a text matching score and a prediction loss of the training sample for the text quality instruction, wherein the text matching score is used to indicate a probability of the generative language model predicting the target text based on the text quality instruction and the input text, and the prediction loss is used to indicate a loss of the generative language model predicting the target text based on the text quality instruction and the input text; The generative language model is trained based on the text matching scores and prediction losses corresponding to the at least two text quality instructions of the training samples, and the sample labels of the training samples.

2. The method according to claim 1, characterized in that The training samples include translation samples, the input text includes source language text, the target text includes target language text, the text quality level includes translation quality level, and the text quality instruction includes translation quality instruction; The training sample is obtained, wherein the training sample includes an input text, a target text, and a sample label, wherein the sample label is used to indicate a text quality level of the target text; and obtaining at least two text quality instructions, wherein the text quality instructions are used to indicate a desired text quality level, including: Obtaining the translation sample, wherein the translation sample includes the source language text, the target language text, and the sample label, wherein the sample label is used to indicate a translation quality level of the target language text; and obtaining at least two translation quality instructions, wherein the translation quality instructions are used to indicate a desired translation quality level; For each of the text quality instructions, calculating a text matching score and a prediction loss of the training sample for the text quality instruction, wherein the text matching score is used to indicate a probability of the generative language model predicting the target text based on the text quality instruction and the input text, and the prediction loss is used to indicate a loss of the generative language model predicting the target text based on the text quality instruction and the input text, including: For each translation quality instruction, calculating a text matching score and a prediction loss of the translation sample with respect to the translation quality instruction, wherein the text matching score is used to indicate a probability of the generative language model predicting the target language text based on the translation quality instruction and the source language text, and the prediction loss is used to indicate a loss of the generative language model predicting the target language text based on the translation quality instruction and the source language text; The training of the generative language model based on the text matching scores and prediction losses corresponding to the at least two text quality instructions of the training samples and the sample labels of the training samples includes: The generative language model is trained based on the text matching scores and prediction losses of the translation samples corresponding to the at least two translation quality instructions and the sample labels of the translation samples.

3. The method according to claim 2, characterized in that The at least two translation quality instructions include an i-th translation quality instruction, the target language text includes at least one target subword unit, and i is a positive integer; For each of the translation quality instructions, calculating a text matching score of the translation sample against the translation quality instruction includes: Calculating at least one subword matching score of the translation sample for the i-th translation quality instruction; A text matching score of the translation sample for the i-th translation quality instruction is calculated, where the text matching score is a ratio of a first cumulative sum to a text length of the target language text, where the first cumulative sum is a cumulative sum of the at least one subword matching score.

4. The method according to claim 3, characterized in that The calculating of at least one subword matching score of the translation sample for the i-th translation quality instruction includes: For the i-th translation quality instruction, calculating a subword matching score of the j-th target subword unit in the target language text, wherein the j-th subword matching score is used to indicate a probability of the generative language model predicting the translation sample based on the i-th translation quality instruction to obtain the j-th target subword unit, where j is a positive integer; Update j to j+1, and continue to perform the above step until at least one subword matching score of the translation sample for the i-th translation quality instruction is calculated.

5. The method according to any one of claims 2 to 4, characterized in that: The text matching scores and prediction losses corresponding to the at least two translation quality instructions of the translation sample include at least two text matching scores and at least two prediction losses; The training of the generative language model based on the text matching scores and prediction losses corresponding to the at least two translation quality instructions of the translation sample and the sample labels of the translation sample includes: For each translation quality instruction, constructing an incentive function for the translation sample with respect to the translation quality instruction based on the text matching score of the translation sample with respect to the at least two translation quality instructions and the sample label of the translation sample, wherein the incentive function is used to instruct the generative language model to distinguish between the translation quality level of the translation quality instruction and the translation quality level of the sample label; The generative language model is trained based on the activation function and prediction loss of the translation samples for the at least two translation quality indicators.

6. The method according to claim 5, characterized in that For each of the translation quality instructions, constructing an incentive function for the translation sample with respect to the translation quality instruction based on the text matching score of the translation sample with respect to the at least two translation quality instructions and the sample label of the translation sample, including: Calculating a translation index of the translation sample for the at least two translation quality instructions based on the text matching scores of the translation sample for the at least two translation quality instructions, wherein the translation index is an exponent raised to a power of the text matching score with a natural constant as a base; Obtaining a translation index of a label instruction corresponding to a sample label of the translation sample, wherein the label instruction is a translation quality instruction having a translation quality level that is the same as the translation quality level of the sample label among the at least two translation quality instructions; At least two of the incentive functions are constructed based on the translation index of the label instruction and the translation indexes of the at least two translation quality instructions.

7. The method according to claim 6, characterized in that The constructing of at least two incentive functions based on the translation index of the label instruction and the translation indexes of the at least two translation quality instructions comprises: For the i-th translation quality instruction among the at least two translation quality instructions, constructing a first intermediate formula, where the first intermediate formula is a cumulative formula of the translation indices of the at least two translation quality instructions; Constructing an incentive function corresponding to the i-th translation quality instruction, where one side of the incentive function is an incentive value, and the other side of the incentive function is a negative number of the ratio of the translation index of the label instruction to the first intermediate formula, and the incentive function is a function with the translation index of the i-th translation quality instruction as an independent variable and the incentive value as a dependent variable; Update i to i+1, and continue to perform the above two steps until the at least two activation functions are constructed.

8. The method according to any one of claims 2 to 7, characterized in that: The target language text in the translation sample is obtained by translating the source language text using a machine translation model.

9. The method according to claim 1, characterized in that The training sample includes a generated sample, the input text includes a generated input text, the target text includes a generated target text, the text quality level includes a content quality level, and the text quality instruction includes a generated quality instruction; The training sample is obtained, wherein the training sample includes an input text, a target text, and a sample label, wherein the sample label is used to indicate a text quality level of the target text; and obtaining at least two text quality instructions, wherein the text quality instructions are used to indicate a desired text quality level, including: Acquire the generated sample, wherein the generated sample includes the generated input text, the generated target text, and the sample label, wherein the sample label is used to indicate the content quality level of the generated target text; and acquire at least two of the generated quality instructions, wherein the generated quality instructions are used to indicate the expected content quality level; For each of the text quality instructions, calculating a text matching score and a prediction loss of the training sample for the text quality instruction, wherein the text matching score is used to indicate a probability of the generative language model predicting the target text based on the text quality instruction and the input text, and the prediction loss is used to indicate a loss of the generative language model predicting the target text based on the text quality instruction and the input text, including: For each of the generation quality instructions, calculating a text matching score and a prediction loss of the generated sample for the generation quality instruction, wherein the text matching score is used to indicate a probability of the generative language model predicting the generated target text based on the generation quality instruction and the generated input text, and the prediction loss is used to indicate a loss of the generative language model predicting the generated target text based on the generation quality instruction and the generated input text; The training of the generative language model based on the text matching scores and prediction losses corresponding to the at least two text quality instructions of the training samples and the sample labels of the training samples includes: The generative language model is trained based on the text matching scores and prediction losses of the generated samples corresponding to the at least two generation quality instructions and the sample labels of the generated samples.

10. A training device for a generative language model, characterized in that: The device comprises: an acquisition module, configured to acquire a training sample, the training sample comprising an input text, a target text, and a sample label, the sample label being used to indicate a text quality level of the target text; and to acquire at least two text quality instructions, the text quality instructions being used to indicate a desired text quality level; a calculation module configured to calculate, for each text quality instruction, a text matching score and a prediction loss of the training sample with respect to the text quality instruction, wherein the text matching score is used to indicate a probability of the generative language model predicting the target text based on the text quality instruction and the input text, and the prediction loss is used to indicate a loss of the generative language model predicting the target text based on the text quality instruction and the input text; A training module is configured to train the generative language model based on the text matching scores and prediction losses corresponding to the at least two text quality instructions of the training samples and the sample labels of the training samples.

11. The device according to claim 10, characterized in that The device further comprises: The acquisition module is further configured to acquire a translation sample, the translation sample comprising a source language text, a target language text, and a sample label, the sample label being used to indicate a translation quality level of the target language text; and to acquire at least two translation quality instructions, the translation quality instructions being used to indicate a desired translation quality level; The calculation module is further configured to calculate, for each translation quality instruction, a text matching score and a prediction loss of the translation sample with respect to the translation quality instruction, wherein the text matching score is used to indicate a probability of the generative language model predicting the target language text according to the translation quality instruction, and the prediction loss is used to indicate a loss of the generative language model predicting the target language text according to the translation quality instruction; The training module is further configured to train the generative language model based on the text matching scores and prediction losses of the translation samples corresponding to the at least two translation quality instructions, respectively, and the sample labels of the translation samples.

12. The device according to claim 10, characterized in that The device further comprises: The acquisition module is further configured to acquire a training sample, wherein the training sample includes a prompt text, a target text, and a sample label, wherein the sample label is used to indicate a text quality level of the target text; and acquire at least two generation quality instructions, wherein the generation quality instructions are used to indicate a desired text quality level; The calculation module is further configured to calculate, for each of the quality generation instructions, a text matching score and a prediction loss of the training sample with respect to the quality generation instruction, wherein the text matching score is used to indicate a probability of the generative language model predicting the target text based on the quality generation instruction and the prompt text, and the prediction loss is used to indicate a loss of the generative language model predicting the target text based on the quality generation instruction and the prompt text; The training module is further configured to train the generative language model based on the text matching scores and prediction losses corresponding to the at least two generation quality instructions of the training samples and the sample labels of the training samples.

13. A computer device, characterized in that: The computer device includes: a processor and a memory, wherein the memory stores at least one program; the processor is used to execute the at least one program in the memory to implement the generative language model training method described in any one of claims 1 to 9.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores executable instructions, which are loaded and executed by a processor to implement the training method of the generative language model as described in any one of claims 1 to 9.

15. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. The processor reads and executes the computer instructions from the computer-readable storage medium to implement the training method of the generative language model according to any one of claims 1 to 9.

Images