Migration training method and device of text generation model based on sorting constraint

Abstract

The invention provides a migration training method and device of a text generation model based on sorting constraint, and is applied to the technical field of natural language processing. The method comprises the steps of obtaining training data; inputting the training data into a first text generation model to obtain first prediction distribution output of the first text generation model on each prediction position of the reply text; inputting the training data into a second text generation model to obtain second prediction distribution output of the second text generation model on each prediction position of the reply text; calculating a sorting loss and a distillation loss between the first predicted distribution output and the second predicted distribution output; fusing the sorting loss and the distillation loss to obtain mixing loss, and taking a mean value of the mixing loss at all prediction positions as total loss; and performing back propagation based on the total loss to update model parameters of the second text generation model.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a migration training method and device for a text generation model based on sorting constraints. Background Art

[0002] Large language models have a powerful ability to solve text generation tasks, which has ushered in a huge revolution in the field of artificial intelligence. However, the large number of parameters in large language models brings a huge computational burden, resulting in high training and reasoning costs. Therefore, it is of great significance to explore how to reduce the size and computational cost of the model while maintaining its text generation performance as much as possible.

[0003] The knowledge distillation technology of language models is a type of model compression technology. It is the process of transferring the knowledge of a large language model (called the teacher model) to a model with a smaller parameter scale (called the student model). Through this process, the student model can learn the implicit knowledge of the teacher model, that is, the complex probability distribution of the teacher model on the input data, rather than just learning the surface content of the input data. After knowledge distillation, the student model can have a smaller parameter scale and computing resource requirements while maintaining high text generation performance, making it more suitable for deployment and use in resource-constrained environments.

[0004] However, since there may be multiple correct predicted words for the same input, the prediction distribution of the language model is often lengthy and multi-peaked. The multi-peak phenomenon of the language model distribution makes it difficult for the student model to learn the distribution. The existing distillation objectives show low efficiency in the process of learning multi-peak distribution, and the effect of knowledge distillation is poor. Summary of the invention

[0005] The present invention provides a migration training method and device for a text generation model based on sorting constraints, which are used to solve the problem that the existing distillation target shows low efficiency in the process of learning multi-peak distribution and the effect of knowledge distillation is poor.

[0006] The present invention provides a migration training method for a text generation model based on sorting constraints, comprising: acquiring training data, each training data comprising an input text and a reply text; inputting the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model at each prediction position of the reply text; inputting the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model at each prediction position of the reply text; calculating a sorting loss and a distillation loss between the first prediction distribution output and the second prediction distribution output; fusing the sorting loss and the distillation loss to obtain a mixed loss, and taking the average of the mixed losses at all prediction positions as the total loss; performing back propagation based on the total loss to update the model parameters of the second text generation model; wherein the parameter scale of the first text generation model is greater than the parameter scale of the second text generation model.

[0007] According to a migration training method for a text generation model based on sorting constraints provided by the present invention, the calculation of the sorting loss and the distillation loss between the first prediction distribution output and the second prediction distribution output includes: respectively obtaining the head prediction sequence of the first text generation model and the second text generation model at each prediction position; determining the union of the head prediction sequence of the first text generation model and the head prediction sequence of the second text generation model; respectively obtaining the prediction probability value sequence of the categories in the union in the original prediction distribution of the first text generation model and the second text generation model; and calculating the sorting loss based on the Spearman rank correlation coefficient according to the prediction probability value sequence.

[0008] According to a migration training method for a text generation model based on sorting constraints provided by the present invention, the objective function of the sorting loss is: ; in, represents the sorting loss, represents the Spearman rank correlation coefficient between the prediction distributions of the first text generation model and the second text generation model, represents the predicted probability value sequence of the first text generation model, represents the predicted probability value sequence of the second text generation model, express and The covariance of and express and The standard deviation of .

[0009] According to a migration training method for a text generation model based on sorting constraints provided by the present invention, the fusion of the sorting loss and the distillation loss to obtain a mixed loss includes: fusing the sorting loss and the distillation loss in a fixed ratio to obtain the mixed loss.

[0010] According to a migration training method of a text generation model based on sorting constraints provided by the present invention, after obtaining the training data, the method also includes: performing word segmentation processing on the input text and the reply text, and converting the input text and the reply text into a data sequence based on a preset vocabulary.

[0011] The present invention also provides a migration training device for a text generation model based on sorting constraints, comprising the following modules: an acquisition module and a processing module; the acquisition module is used to acquire training data, each training data includes an input text and a reply text; the processing module is used to input the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model for each prediction position of the reply text; the training data is input into a second text generation model to obtain a second prediction distribution output of the second text generation model for each prediction position of the reply text; the sorting loss and the distillation loss between the first prediction distribution output and the second prediction distribution output are calculated; the sorting loss and the distillation loss are merged to obtain a mixed loss, and the average of the mixed losses at all prediction positions is taken as the total loss; back propagation is performed based on the total loss to update the model parameters of the second text generation model; wherein the parameter scale of the first text generation model is larger than the parameter scale of the second text generation model.

[0012] According to a migration training device for a text generation model based on sorting constraints provided by the present invention, the processing module is used to respectively obtain the head prediction sequence of the first text generation model and the second text generation model at each prediction position; determine the union of the head prediction sequence of the first text generation model and the head prediction sequence of the second text generation model; respectively obtain the prediction probability value sequence of the categories in the union in the original prediction distribution of the first text generation model and the second text generation model; and calculate the sorting loss based on the Spearman rank correlation coefficient according to the prediction probability value sequence.

[0013] According to a migration training device for a text generation model based on sorting constraints provided by the present invention, the objective function of the sorting loss is: ; in, represents the sorting loss, represents the Spearman rank correlation coefficient between the prediction distributions of the first text generation model and the second text generation model, represents the predicted probability value sequence of the first text generation model, represents the predicted probability value sequence of the second text generation model, express and The covariance of and express and The standard deviation of .

[0014] According to a migration training device for a text generation model based on sorting constraints provided by the present invention, the processing module is used to fuse the sorting loss and the distillation loss in a fixed ratio to obtain the mixed loss.

[0015] According to a migration training device for a text generation model based on sorting constraints provided by the present invention, the processing module is used to perform word segmentation processing on the input text and the reply text, and convert the input text and the reply text into a data sequence based on a preset vocabulary.

[0016] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a migration training method for a text generation model based on sorting constraints as described in any one of the above is implemented.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a migration training method for a text generation model based on sorting constraints as described in any one of the above.

[0018] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned migration training methods for the text generation model based on sorting constraints.

[0019] The transfer training method and device of the text generation model based on the sorting constraint provided by the present invention can obtain training data, each training data includes input text and reply text; input the training data into the first text generation model to obtain the first prediction distribution output of the first text generation model at each prediction position of the reply text; input the training data into the second text generation model to obtain the second prediction distribution output of the second text generation model at each prediction position of the reply text; calculate the sorting loss and distillation loss between the first prediction distribution output and the second prediction distribution output; merge the sorting loss and the distillation loss to obtain the mixed loss, and take the average of the mixed losses at all prediction positions as the total loss; perform back propagation based on the total loss to update the model parameters of the second text generation model; wherein the parameter scale of the first text generation model is greater than the parameter scale of the second text generation model. Through this scheme, since the sorting loss between the first prediction distribution output and the second prediction distribution output can be calculated, and the total loss obtained by combining the distillation loss is used to update the model parameters of the second text generation model, the consistency of the peak predictions of the two models can be calculated through the word-level sorting loss, thereby realizing the efficient alignment of the multi-peak prediction distributions of the first text generation model and the second text generation model. In this way, not only can the fine-grained information between the two predicted distribution peak categories be more fully utilized, but also good compatibility with existing distillation objectives can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 It is a flow chart of the migration training method of the text generation model based on sorting constraints provided by the present invention; Figure 2 It is a structural schematic diagram of a migration training device for a text generation model based on sorting constraints provided by the present invention; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0024] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0025] In order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and order of execution.

[0026] The embodiments of the present application describe some exemplary embodiments for the purpose of explanation. It should be understood that the present application can be implemented in other ways that are not specifically shown in the drawings.

[0027] like Figure 1As shown, the embodiment of the present application provides a migration training method for a text generation model based on sorting constraints, and the migration training method for a text generation model based on sorting constraints can be applied to a migration training device for a text generation model based on sorting constraints. The migration training method for a text generation model based on sorting constraints may include S101-S106: S101. A transfer training device for a text generation model based on sorting constraints obtains training data.

[0028] Each training data includes an input text and a response text. For example, if the training data is a primary school math question and answer, the input text may be "Please calculate the answer to the following math question", and the response text may be "The answer to this math question is 78".

[0029] Optionally, after acquiring the training data, the transfer training device of the text generation model based on the sorting constraint may first perform word segmentation processing on the input text and the reply text, and convert the input text and the reply text into a data sequence based on a preset vocabulary.

[0030] Specifically, the transfer training device of the text generation model based on sorting constraints can perform word segmentation on the input text and the response text of each training data, and convert them into serial numbers according to a preset vocabulary, and then splice the serialized data to obtain a data sequence.

[0031] S102, the transfer training device of the text generation model based on the sorting constraint inputs the training data into the first text generation model to obtain the first prediction distribution output of the first text generation model for each prediction position of the reply text.

[0032] Optionally, the transfer training device of the text generation model based on sorting constraints can input the above data sequence into the first text generation model to obtain the first prediction distribution output of the first text generation model for each prediction position of the reply text.

[0033] It should be noted that the above-mentioned first text generation model is a "teacher model" with a relatively large parameter scale.

[0034] S103, the transfer training device of the text generation model based on the sorting constraint inputs the training data into the second text generation model to obtain the second prediction distribution output of the second text generation model for each prediction position of the reply text.

[0035] Among them, the parameter scale of the first text generation model is greater than the parameter scale of the second text generation model.

[0036] Optionally, the transfer training device of the text generation model based on the sorting constraint can input the above data sequence into the second text generation model to obtain the second prediction distribution output of the second text generation model for each prediction position of the reply text.

[0037] It should be noted that the above-mentioned second text generation model is a "student model" with a smaller parameter scale.

[0038] S104. The transfer training device of the text generation model based on the sorting constraint calculates the sorting loss and the distillation loss between the first prediction distribution output and the second prediction distribution output.

[0039] Optionally, a transfer training device for a text generation model based on a sorting constraint calculates a sorting loss and a distillation loss between the first prediction distribution output and the second prediction distribution output, including: calculating the distillation loss between the prediction distributions of a teacher model and a student model at each prediction position using KL (Kullback-Leible) divergence, reverse KL divergence, JS (Jensen–Shannon) divergence or total variational distance as an objective function.

[0040] Optionally, the objective function of the above KL divergence is: ; in, represents the distillation loss, represents the KL divergence between the current prediction distributions of the first text generation model and the second text generation model, represents the predicted category, Represents the first text generation model predicting the category The predicted probability of Represents the second text generation model predicting the category The predicted probability of .

[0041] Optionally, a migration training device for a text generation model based on a ranking constraint calculates a ranking loss and a distillation loss between the first prediction distribution output and the second prediction distribution output, including: respectively obtaining a head prediction sequence of the first text generation model and the second text generation model at each prediction position; determining a union of the head prediction sequence of the first text generation model and the head prediction sequence of the second text generation model; respectively obtaining a sequence of predicted probability values ​​of categories in the union in the original prediction distribution of the first text generation model and the second text generation model; and calculating a ranking loss based on the Spearman rank correlation coefficient according to the sequence of predicted probability values.

[0042] Specifically, for each prediction position, the transfer training device of the text generation model based on the sorting constraint can obtain the head prediction sequence of the first text generation model and the second text generation model at the current position. For example, the head prediction sequence can take the first k prediction categories in each category after the prediction probability is sorted from large to small; then, for each prediction position, the transfer training device of the text generation model based on the sorting constraint can take the union of the two head prediction sequences, and respectively obtain the prediction probability value sequence of the category in the union in the original prediction distribution of the first text generation model and the second text generation model; finally, at each prediction position, the transfer training device of the text generation model based on the sorting constraint can calculate the sorting loss based on the Spearman rank correlation coefficient for the prediction probability value sequence.

[0043] Optionally, the objective function of the sorting loss is: ; in, represents the sorting loss, represents the Spearman rank correlation coefficient between the prediction distributions of the first text generation model and the second text generation model, represents the predicted probability value sequence of the first text generation model, represents the predicted probability value sequence of the second text generation model, express and The covariance of and express and The standard deviation of .

[0044] S105. The transfer training device of the text generation model based on the sorting constraint fuses the sorting loss and the distillation loss to obtain a mixed loss, and takes the average of the mixed losses at all prediction positions as the total loss.

[0045] Optionally, the transfer training device of the text generation model based on the sorting constraint fuses the sorting loss and the distillation loss to obtain a mixed loss, including: fusing the sorting loss and the distillation loss in a fixed ratio to obtain the mixed loss.

[0046] Specifically, the objective function of the transfer training device of the text generation model based on the sorting constraint that integrates the sorting loss and the distillation loss can be: ; in, Indicates mixed loss.

[0047] Optionally, the transfer training device of the text generation model based on the sorting constraint fuses the sorting loss and the distillation loss to obtain a mixed loss, including: adopting a dynamic loss fusion strategy to fuse the sorting loss and the distillation loss to obtain the mixed loss.

[0048] Specifically, the dynamic loss fusion strategy uses the coverage of peak elements in the prediction distributions of the two models as the allocation indicator, and its objective function is: ; in, Represents the number of categories in the intersection of the first and second text generation models’ top k predictions.

[0049] Optionally, the transfer training device of the text generation model based on the sorting constraint can take the mean of the mixed losses at all prediction positions as the total loss, and its objective function is: ; in, represents the total loss, and n represents the total length of the training data prediction sequence.

[0050] S106. The transfer training device of the text generation model based on the sorting constraint performs back propagation based on the total loss to update the model parameters of the second text generation model.

[0051] Specifically, the migration training device of the text generation model based on the sorting constraint can freeze the model parameters of the first text generation model, and update the model parameters of the second text generation model by back propagation based on the total loss.

[0052] In the embodiment of the present application, since the ranking loss between the first prediction distribution output and the second prediction distribution output can be calculated, and the total loss obtained by combining the distillation loss is used to update the model parameters of the second text generation model, the consistency of the peak predictions of the two models can be calculated by the word-level ranking loss, thereby achieving efficient alignment of the multi-peak prediction distributions of the first text generation model and the second text generation model. In this way, not only can the fine-grained information between the peak categories of the two prediction distributions be more fully utilized, but also good compatibility with existing distillation targets can be guaranteed.

[0053] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0054] It should be noted that the device in the embodiment of the present application includes a virtual device and a physical device. The virtual device can be a transfer training device for a text generation model based on sorting constraints, and the physical device can include an electronic device, a computer storage medium, and a computer program product.

[0055] The transfer training method of the text generation model based on sorting constraints provided in the embodiment of the present application can be performed by a transfer training device for the text generation model based on sorting constraints, or a control module for transfer training of the text generation model based on sorting constraints in the transfer training device for the text generation model based on sorting constraints. In the embodiment of the present application, the transfer training method of the text generation model based on sorting constraints performed by the transfer training device for the text generation model based on sorting constraints is taken as an example to illustrate the transfer training device for the text generation model based on sorting constraints provided in the embodiment of the present application.

[0056] It should be noted that the embodiment of the present application can divide the functional modules of the migration training device of the text generation model based on the sorting constraint according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. Optionally, the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0057] like Figure 2As shown, the embodiment of the present application provides a migration training device 200 for a text generation model based on a sorting constraint. The migration training device 200 for a text generation model based on a sorting constraint includes: an acquisition module 201 and a processing module 202. The acquisition module 201 is used to acquire training data, each training data includes an input text and a reply text; the processing module 202 is used to input the training data into the first text generation model to obtain the first prediction distribution output of the first text generation model for each prediction position of the reply text; input the training data into the second text generation model to obtain the second prediction distribution output of the second text generation model for each prediction position of the reply text; calculate the sorting loss and distillation loss between the first prediction distribution output and the second prediction distribution output; merge the sorting loss and the distillation loss to obtain a mixed loss, and take the average of the mixed losses at all prediction positions as the total loss; perform back propagation based on the total loss to update the model parameters of the second text generation model; wherein the parameter scale of the first text generation model is larger than the parameter scale of the second text generation model.

[0058] Optionally, the processing module 202 is used to respectively obtain the head prediction sequence of the first text generation model and the second text generation model at each prediction position; determine the union of the head prediction sequence of the first text generation model and the head prediction sequence of the second text generation model; respectively obtain the prediction probability value sequence of the categories in the union in the original prediction distribution of the first text generation model and the second text generation model; and calculate the sorting loss based on the Spearman rank correlation coefficient according to the prediction probability value sequence.

[0059] Optionally, the objective function of the sorting loss is: ; in, represents the sorting loss, represents the Spearman rank correlation coefficient between the prediction distributions of the first text generation model and the second text generation model, represents the predicted probability value sequence of the first text generation model, represents the predicted probability value sequence of the second text generation model, express and The covariance of and express and The standard deviation of .

[0060] Optionally, the processing module 202 is used to fuse the sorting loss and the distillation loss in a fixed ratio to obtain the mixed loss.

[0061] Optionally, the processing module 202 is used to perform word segmentation processing on the input text and the reply text, and convert the input text and the reply text into a data sequence based on a preset vocabulary.

[0062] In the embodiment of the present application, since the ranking loss between the first prediction distribution output and the second prediction distribution output can be calculated, and the total loss obtained by combining the distillation loss is used to update the model parameters of the second text generation model, the consistency of the peak predictions of the two models can be calculated by the word-level ranking loss, thereby achieving efficient alignment of the multi-peak prediction distributions of the first text generation model and the second text generation model. In this way, not only can the fine-grained information between the peak categories of the two prediction distributions be more fully utilized, but also good compatibility with existing distillation targets can be guaranteed.

[0063] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310 , a communications interface 320 , a memory 330 and a communication bus 340 , wherein the processor 310 , the communications interface 320 , and the memory 330 communicate with each other via the communication bus 340 . The processor 310 can call the logic instructions in the memory 330 to execute the migration training method of the text generation model based on the sorting constraint, which method includes: obtaining training data, each training data includes an input text and a reply text; inputting the training data into the first text generation model to obtain the first prediction distribution output of the first text generation model for each prediction position of the reply text; inputting the training data into the second text generation model to obtain the second prediction distribution output of the second text generation model for each prediction position of the reply text; calculating the sorting loss and the distillation loss between the first prediction distribution output and the second prediction distribution output; fusing the sorting loss and the distillation loss to obtain a mixed loss, and taking the average of the mixed losses at all prediction positions as the total loss; performing back propagation based on the total loss to update the model parameters of the second text generation model; wherein the parameter scale of the first text generation model is larger than the parameter scale of the second text generation model.

[0064] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0065] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the migration training method of the text generation model based on sorting constraints provided by the above-mentioned methods, which method includes: obtaining training data, each training data includes input text and reply text; inputting the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model for each prediction position of the reply text; inputting the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model for each prediction position of the reply text; calculating the sorting loss and distillation loss between the first prediction distribution output and the second prediction distribution output; fusing the sorting loss and the distillation loss to obtain a mixed loss, and taking the average of the mixed losses at all prediction positions as the total loss; performing back propagation based on the total loss to update the model parameters of the second text generation model; wherein the parameter scale of the first text generation model is larger than the parameter scale of the second text generation model.

[0066] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the migration training method of the text generation model based on sorting constraints provided by the above-mentioned methods, the method comprising: obtaining training data, each training data comprising an input text and a reply text; inputting the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model for each prediction position of the reply text; inputting the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model for each prediction position of the reply text; calculating the sorting loss and the distillation loss between the first prediction distribution output and the second prediction distribution output; fusing the sorting loss and the distillation loss to obtain a mixed loss, and taking the average of the mixed losses at all prediction positions as the total loss; performing back propagation based on the total loss to update the model parameters of the second text generation model; wherein the parameter scale of the first text generation model is larger than the parameter scale of the second text generation model.

[0067] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0068] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0069] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A migration training method for a text generation model based on sorting constraints, characterized in that: include: Acquire training data, each training data includes input text and response text; Inputting the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model at each prediction position of the reply text; Inputting the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model for each prediction position of the reply text; Calculating a ranking loss and a distillation loss between the first predicted distribution output and the second predicted distribution output; The ranking loss and the distillation loss are combined to obtain a mixed loss, and the average of the mixed losses at all predicted positions is taken as the total loss; Performing back propagation based on the total loss to update model parameters of the second text generation model; Among them, the parameter scale of the first text generation model is greater than the parameter scale of the second text generation model.

2. The migration training method of the text generation model based on sorting constraints according to claim 1 is characterized in that: The calculating the ranking loss and the distillation loss between the first predicted distribution output and the second predicted distribution output includes: Respectively obtain the head prediction sequence of the first text generation model and the second text generation model at each prediction position; Determine a union of a head prediction sequence of the first text generation model and a head prediction sequence of the second text generation model; Respectively obtaining a sequence of predicted probability values ​​of the categories in the union in the original predicted distributions of the first text generation model and the second text generation model; A ranking loss based on the Spearman rank correlation coefficient is calculated based on the predicted probability value sequence.

3. The migration training method of the text generation model based on sorting constraints according to claim 2 is characterized in that: The objective function of the sorting loss is: ； in, represents the sorting loss, represents the Spearman rank correlation coefficient between the prediction distributions of the first text generation model and the second text generation model, represents the predicted probability value sequence of the first text generation model, represents the predicted probability value sequence of the second text generation model, express and The covariance of and express and The standard deviation of .

4. The migration training method of the text generation model based on sorting constraints according to claim 1 is characterized in that: The step of fusing the sorting loss and the distillation loss to obtain a mixed loss includes: The sorting loss and the distillation loss are fused in a fixed ratio to obtain the mixed loss.

5. The migration training method of the text generation model based on sorting constraints according to claim 1 is characterized in that: After obtaining the training data, the method further includes: The input text and the reply text are word segmented, and the input text and the reply text are converted into data sequences based on a preset vocabulary.

6. A migration training device for a text generation model based on sorting constraints, characterized in that: include: Acquisition module and processing module; The acquisition module is used to acquire training data, each training data includes input text and response text; The processing module is used to input the training data into the first text generation model to obtain the first prediction distribution output of the first text generation model for each prediction position of the reply text; input the training data into the second text generation model to obtain the second prediction distribution output of the second text generation model for each prediction position of the reply text; calculate the sorting loss and the distillation loss between the first prediction distribution output and the second prediction distribution output; fuse the sorting loss and the distillation loss to obtain the mixed loss, and take the average of the mixed losses at all prediction positions as the total loss; perform back propagation based on the total loss to update the model parameters of the second text generation model; Among them, the parameter scale of the first text generation model is greater than the parameter scale of the second text generation model.

7. The transfer training device for the text generation model based on sorting constraints according to claim 6 is characterized in that: The processing module is used to respectively obtain the head prediction sequence of the first text generation model and the second text generation model at each prediction position; determine the union of the head prediction sequence of the first text generation model and the head prediction sequence of the second text generation model; respectively obtain the prediction probability value sequence of the categories in the union in the original prediction distribution of the first text generation model and the second text generation model; and calculate the sorting loss based on the Spearman rank correlation coefficient according to the prediction probability value sequence.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the transfer training method of the text generation model based on sorting constraints as described in any one of claims 1 to 5 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the transfer training method of the text generation model based on sorting constraints as described in any one of claims 1 to 5 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the transfer training method of the text generation model based on sorting constraints as described in any one of claims 1 to 5 is implemented.

Images