Method and apparatus for transfer training of text generation model based on ranking constraints

Abstract

The application provides a migration training method and device of a text generation model based on ranking constraints, and applies to the technical field of natural language processing. The method comprises the following steps: obtaining training data; inputting the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model on each predicted position of a reply text; inputting the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model on each predicted position of the reply text; calculating a ranking loss and a distillation loss between the first prediction distribution output and the second prediction distribution output; fusing the ranking loss and the distillation loss to obtain a hybrid loss, and taking the mean value of the hybrid loss on all predicted positions as a total loss; and performing back propagation based on the total loss to update the model parameters of the second text generation model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a migration training method and device of a text generation model based on ranking constraints. BACKGROUND

[0002] Large language models have strong capabilities to solve text generation tasks, opening up great changes in the field of artificial intelligence. However, the huge parameter quantity of large language models brings huge computational burden, causing high use cost of training and inference. Therefore, it is of great significance to explore how to reduce the size and computational cost of the model while maintaining its text generation performance as much as possible.

[0003] The knowledge distillation technology of language models is a kind of model compression technology, which is the process of migrating the knowledge of a large language model (called a teacher model) to a smaller parameter model (called a student model). Through this process, the student model can learn the implicit knowledge of the teacher model, that is, the complex probability distribution of the teacher model on the input data, rather than just learning the surface content of the input data. After knowledge distillation, the student model can maintain high text generation performance while having smaller parameter size and computational resource demand, making it more suitable for deployment and use in resource-constrained environments.

[0004] However, since there can be multiple correct prediction words for the same input, the prediction distribution of the language model is often long and multi-peak. The multi-peak phenomenon of the language model distribution makes it difficult for the student model to learn the distribution, and the existing distillation target shows low efficiency in learning the multi-peak distribution, and the effect of knowledge distillation is poor. SUMMARY

[0005] The present application provides a migration training method and device of a text generation model based on ranking constraints, to solve the problem that the existing distillation target shows low efficiency in learning the multi-peak distribution, and the effect of knowledge distillation is poor.

[0006] The application provides a migration training method of a text generation model based on ranking constraints, comprising: obtaining training data, each training data comprising input text and reply text; inputting the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model on each predicted position of the reply text; inputting the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model on each predicted position of the reply text; calculating a ranking loss and a distillation loss between the first prediction distribution output and the second prediction distribution output; fusing the ranking loss and the distillation loss to obtain a hybrid loss, and taking the mean value of the hybrid loss on all predicted positions as a total loss; performing back propagation based on the total loss to update model parameters of the second text generation model; wherein the parameter size of the first text generation model is greater than the parameter size of the second text generation model.

[0007] According to the migration training method of the text generation model based on ranking constraints provided by the application, the calculation of the ranking loss and the distillation loss between the first prediction distribution output and the second prediction distribution output comprises: obtaining a head prediction sequence of the first text generation model and the second text generation model on each predicted position respectively; determining the union set of the head prediction sequence of the first text generation model and the head prediction sequence of the second text generation model; obtaining a prediction probability value sequence of the category in the original prediction distribution of the first text generation model and the second text generation model in the union set respectively; and calculating the ranking loss based on the Spearman rank correlation coefficient according to the prediction probability value sequence.

[0008] According to the migration training method of the text generation model based on ranking constraints provided by the application, the target function of the ranking loss is:

[0009] ;

[0010] Among them, the ranking loss, the Spearman rank correlation coefficient between the prediction distribution of the first text generation model and the second text generation model, the prediction probability value sequence of the first text generation model, the prediction probability value sequence of the second text generation model, the covariance of and and the standard deviation of and

[0011] ​​According to the migration training method of the text generation model based on the sorting constraint provided by the application, the sorting loss and the distillation loss are fused to obtain a hybrid loss, which includes: adopting a fixed proportion to fuse the sorting loss and the distillation loss to obtain the hybrid loss.

[0012] According to the migration training method of the text generation model based on the sorting constraint provided by the application, after the training data is obtained, the method further includes: performing word segmentation processing on the input text and the reply text, and converting the input text and the reply text into data sequences based on a preset word table.

[0013] The application also provides a migration training device of a text generation model based on a sorting constraint, which includes the following modules: an acquisition module and a processing module; the acquisition module is used to acquire training data, and each training data includes an input text and a reply text; the processing module is used to input the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model on each predicted position of the reply text; input the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model on each predicted position of the reply text; calculate a sorting loss and a distillation loss between the first prediction distribution output and the second prediction distribution output; fuse the sorting loss and the distillation loss to obtain a hybrid loss, and take the average of the hybrid losses on all predicted positions as a total loss; perform back propagation based on the total loss to update model parameters of the second text generation model; wherein the parameter size of the first text generation model is greater than the parameter size of the second text generation model.

[0014] According to the migration training device of the text generation model based on the sorting constraint provided by the application, the processing module is used to respectively acquire head prediction sequences of the first text generation model and the second text generation model on each predicted position; determine the union set of the head prediction sequence of the first text generation model and the head prediction sequence of the second text generation model; respectively acquire prediction probability value sequences of categories in the original prediction distribution of the first text generation model and the second text generation model in the union set; and calculate a sorting loss based on a Spearman rank correlation coefficient according to the prediction probability value sequences.

[0015] According to the migration training device of the text generation model based on the sorting constraint provided by the application, the target function of the sorting loss is:

[0016] ;

[0017] Wherein, The sorting loss is represented as LSS, a Spearman rank correlation coefficient between the first text generation model and the second text generation model, a sequence of predicted probability values of the first text generation model, a sequence of predicted probability values of the second text generation model, a covariance of and and a standard deviation of and

[0018] According to the application, a migration training device for a text generation model based on ranking constraints is provided, and the processing module is used to fuse the ranking loss and the distillation loss by using a fixed proportion to obtain the mixed loss.

[0019] According to the application, a migration training device for a text generation model based on ranking constraints is provided, and the processing module is used to perform word segmentation processing on the input text and the reply text, and convert the input text and the reply text into data sequences based on a preset word table.

[0020] The application further provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the migration training method of the text generation model based on ranking constraints according to any one of the above when executing the program.

[0021] The application further provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the migration training method of the text generation model based on ranking constraints according to any one of the above.

[0022] The application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the migration training method of the text generation model based on ranking constraints according to any one of the above.

[0023] ​​The application provides a migration training method and device of a text generation model based on ranking constraints, which can obtain training data, each of which comprises input text and reply text; the training data is input into a first text generation model to obtain a first prediction distribution output of the first text generation model on each predicted position of the reply text; the training data is input into a second text generation model to obtain a second prediction distribution output of the second text generation model on each predicted position of the reply text; a ranking loss and a distillation loss between the first prediction distribution output and the second prediction distribution output are calculated; a mixed loss is obtained by fusing the ranking loss and the distillation loss, and a mean value of the mixed loss on all predicted positions is taken as a total loss; the total loss is used for back propagation to update model parameters of the second text generation model; and the parameter scale of the first text generation model is greater than that of the second text generation model. According to the scheme, the consistency degree of peak prediction of two models can be calculated through the ranking loss at the word level, so that efficient alignment of the multi-peak prediction distribution of the first text generation model and the second text generation model is realized. In this way, not only can the fine-grained information between the peak categories of two prediction distributions be more fully utilized, but also good compatibility with existing distillation targets can be ensured. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0025] Figure 1 FIG. 1 is a flowchart of a migration training method of a text generation model based on ranking constraints provided by the application;

[0026] Figure 2 FIG. 2 is a structural schematic diagram of a migration training device of a text generation model based on ranking constraints provided by the application;

[0027] Figure 3 FIG. 3 is a structural schematic diagram of an electronic device provided by the application. DETAILED DESCRIPTION

[0028] In the following, the technical solutions of the present application will be described clearly and completely in connection with the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of them. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of the present application.

[0029] It should be noted that the words "exemplary" and "for example" are used herein to mean "an example of" rather than "an ideal example" or "an ideal implementation". Any implementation described herein as "exemplary" or "for example" is not necessarily to be construed as preferred or advantageous over other implementations.

[0030] It should be noted that the terms "comprising", "including", or any other variant are intended to cover non-exclusive inclusions, so that processes, methods, articles, or devices that comprise a list of elements not only include those elements, but also include other elements not expressly listed, or inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or device that includes the element. In addition, it should be noted that the scope of the methods and devices in the present application is not limited to performing functions in the order shown or discussed, but can also include performing functions in a substantially simultaneous manner or in reverse order, for example, the described methods can be performed in an order different from that described, and various steps can be added, omitted, or combined. In addition, features described with reference to certain examples can be combined in other examples.

[0031] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, "first", "second", etc. are used to distinguish the same or similar items with basically the same function and role, and those skilled in the art can understand that "first", "second", etc. are not limited in number and execution order.

[0032] The embodiments of the present application describe some exemplary embodiments for the purpose of illustration, and it should be understood that the present application can be implemented in other ways not specifically shown in the drawings.

[0033] As Figure 1As shown, the embodiment of the present application provides a migration training method of a ranking constraint based text generation model, which can be applied to a migration training device of the ranking constraint based text generation model. The migration training method of the ranking constraint based text generation model can include S101-S106:

[0034] S101, the migration training device of the ranking constraint based text generation model acquires training data.

[0035] Each training data includes an input text and a reply text. For example, if the training data is a primary school mathematics question and answer, the input text can be "please calculate the answer to the following mathematics problem", and the reply text can be "the answer to the mathematics problem is 78".

[0036] Optionally, after acquiring the training data, the migration training device of the ranking constraint based text generation model can first perform word segmentation processing on the input text and the reply text, and convert the input text and the reply text into data sequences based on a preset word table.

[0037] Specifically, the migration training device of the ranking constraint based text generation model can perform word segmentation processing on the input text and the reply text of each training data, and convert them into serial numbers according to the preset word table, and then splice the serialized data to obtain a data sequence.

[0038] S102, the migration training device of the ranking constraint based text generation model inputs the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model on each predicted position of the reply text.

[0039] Optionally, the migration training device of the ranking constraint based text generation model can input the above data sequence into the first text generation model to obtain a first prediction distribution output of the first text generation model on each predicted position of the reply text.

[0040] It should be noted that the above first text generation model is a "teacher model" with a large parameter scale.

[0041] S103, the migration training device of the ranking constraint based text generation model inputs the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model on each predicted position of the reply text.

[0042] Wherein, the parameter scale of the first text generation model is greater than the parameter scale of the second text generation model.

[0043] Optionally, the migration training apparatus of the text generation model based on ranking constraints can input the above data sequence into a second text generation model to obtain a second prediction distribution output of the second text generation model at each predicted position of the reply text.

[0044] It should be noted that the second text generation model is a "student model" with a smaller parameter scale.

[0045] S104, the migration training apparatus of the text generation model based on ranking constraints calculates a ranking loss and a distillation loss between the first prediction distribution output and the second prediction distribution output.

[0046] Optionally, the migration training apparatus of the text generation model based on ranking constraints calculates a ranking loss and a distillation loss between the first prediction distribution output and the second prediction distribution output, including: calculating the distillation loss between the prediction distribution of the teacher model and the prediction distribution of the student model at each predicted position as a target function of KL (Kullback-Leible) divergence, reverse KL divergence, JS (Jensen-Shannon) divergence or total variation distance.

[0047] Optionally, the target function of the KL divergence is:

[0048] ;

[0049] wherein, the distillation loss is represented by D, the KL divergence between the current prediction distribution of the first text generation model and the second text generation model is represented by KL, the predicted category is represented by y, the prediction probability of the first text generation model for the predicted category y is represented by p, the prediction probability of the second text generation model for the predicted category y is represented by q.

[0050] Optionally, the migration training apparatus of the text generation model based on ranking constraints calculates a ranking loss and a distillation loss between the first prediction distribution output and the second prediction distribution output, including: respectively obtaining the head prediction sequence of the first text generation model and the second text generation model at each predicted position; determining the union of the head prediction sequence of the first text generation model and the head prediction sequence of the second text generation model; respectively obtaining the prediction probability value sequence of the category in the original prediction distribution of the first text generation model and the second text generation model in the union; and calculating the ranking loss based on the Spearman rank correlation coefficient according to the prediction probability value sequence.

[0051] ​​Specifically, for each prediction position, the migration training device of the text generation model based on the ranking constraint can obtain the head prediction sequence of the first text generation model and the second text generation model at the current position, for example, the head prediction sequence can take the top k prediction categories in each category in descending order of prediction probability; then, for each prediction position, the migration training device of the text generation model based on the ranking constraint can take the union of the two head prediction sequences, and obtain the prediction probability value sequence of the categories in the union in the original prediction distribution of the first text generation model and the second text generation model respectively; finally, at each prediction position, the migration training device of the text generation model based on the ranking constraint can calculate the ranking loss based on the Spearman rank correlation coefficient on the prediction probability value sequence.

[0052] Optionally, the target function of the ranking loss is:

[0053] ;

[0054] wherein, denotes the ranking loss, denotes the Spearman rank correlation coefficient between the prediction distribution of the first text generation model and the second text generation model, denotes the prediction probability value sequence of the first text generation model, denotes the prediction probability value sequence of the second text generation model, denotes and the covariance of and denotes and the standard deviation of

[0055] S105, the migration training device of the text generation model based on the ranking constraint fuses the ranking loss and the distillation loss to obtain a mixed loss, and takes the average of the mixed loss at all prediction positions as the total loss.

[0056] Optionally, the migration training device of the text generation model based on the ranking constraint fuses the ranking loss and the distillation loss to obtain a mixed loss, comprising: adopting a fixed ratio to fuse the ranking loss and the distillation loss to obtain the mixed loss.

[0057] Specifically, the target function of the migration training device of the text generation model based on the ranking constraint for fusing the ranking loss and the distillation loss can be:

[0058] ;

[0059] wherein, denotes the mixed loss.

[0060] Optionally, the migration training apparatus of the text generation model based on the ranking constraint fuses the ranking loss and the distillation loss to obtain a hybrid loss, including: adopting a dynamic loss fusion strategy to fuse and process the ranking loss and the distillation loss to obtain the hybrid loss.

[0061] Specifically, the dynamic loss fusion strategy takes the coverage rate of the peak elements in the two model prediction distributions as the allocation index, and the objective function is:

[0062] ;

[0063] wherein, represents the number of categories in the intersection of the first k predictions of the first text generation model and the second text generation model.

[0064] Optionally, the migration training apparatus of the text generation model based on the ranking constraint can take the mean value of the hybrid loss at all prediction positions as the total loss, and the objective function is:

[0065] ;

[0066] wherein, represents the total loss, and n represents the total length of the predicted sequence of the training data.

[0067] S106, the migration training apparatus of the text generation model based on the ranking constraint performs back propagation based on the total loss to update the model parameters of the second text generation model.

[0068] Specifically, the migration training apparatus of the text generation model based on the ranking constraint can freeze the model parameters of the first text generation model, and perform back propagation based on the total loss to update the model parameters of the second text generation model.

[0069] In the embodiments of the present application, since the ranking loss between the first prediction distribution output and the second prediction distribution output can be calculated, and the model parameters of the second text generation model are updated based on the total loss obtained by combining the distillation loss, the consistency of the peak predictions of the two models can be calculated through the word-level ranking loss, thereby realizing efficient alignment of the multi-peak prediction distribution of the first text generation model and the second text generation model. In this way, not only can the fine-grained information between the peak categories of the two prediction distributions be more fully utilized, but also good compatibility with existing distillation targets can be ensured.

[0070] The above mainly introduces the scheme provided by the embodiments of the application from the method aspect. To implement the above functions, the corresponding hardware structure and / or software module for executing each function are included. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed in the present application, the embodiments of the application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is implemented in hardware or computer software driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0071] It should be noted that the device in the embodiments of the present application includes a virtual device and a physical device. The virtual device can be a migration training device for a text generation model based on a ranking constraint, and the physical device can include an electronic device, a computer storage medium, and a computer program product.

[0072] The migration training method for the text generation model based on the ranking constraint provided by the embodiments of the present application can be executed by the migration training device for the text generation model based on the ranking constraint, or a control module for migration training of the text generation model based on the ranking constraint in the migration training device for the text generation model based on the ranking constraint. In the embodiments of the present application, the migration training method for the text generation model based on the ranking constraint is executed by the migration training device for the text generation model based on the ranking constraint, which is taken as an example to illustrate the migration training device for the text generation model based on the ranking constraint provided by the embodiments of the present application.

[0073] It should be noted that the migration training device for the text generation model based on the ranking constraint can be divided into functional modules according to the above method examples, for example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or software functional module. Optionally, the division of the modules in the embodiments of the present application is illustrative, and is only a logical functional division. In actual implementation, another division mode can be used.

[0074] As Figure 2As shown, the embodiment of the present application provides a migration training device 200 of a ranking constraint based text generation model. The migration training device 200 of the ranking constraint based text generation model comprises an acquisition module 201 and a processing module 202. The acquisition module 201 is configured to acquire training data, each of which comprises input text and reply text; the processing module 202 is configured to input the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model on each predicted position of the reply text; input the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model on each predicted position of the reply text; calculate a ranking loss and a distillation loss between the first prediction distribution output and the second prediction distribution output; fuse the ranking loss and the distillation loss to obtain a hybrid loss, and take the mean value of the hybrid loss on all predicted positions as a total loss; perform back propagation based on the total loss to update model parameters of the second text generation model; wherein the parameter size of the first text generation model is greater than the parameter size of the second text generation model.

[0075] Optionally, the processing module 202 is configured to acquire a head prediction sequence of the first text generation model and a head prediction sequence of the second text generation model on each predicted position respectively; determine a union set of the head prediction sequence of the first text generation model and the head prediction sequence of the second text generation model; acquire a prediction probability value sequence of the category in the original prediction distribution of the first text generation model and the second text generation model in the union set respectively; and calculate a ranking loss based on a Spearman rank correlation coefficient according to the prediction probability value sequence.

[0076] Optionally, the target function of the ranking loss is:

[0077] ;

[0078] wherein, represents the ranking loss, represents a Spearman rank correlation coefficient between the prediction distribution of the first text generation model and the second text generation model, represents a prediction probability value sequence of the first text generation model, represents a prediction probability value sequence of the second text generation model, represents and covariance of and represents and standard deviation of

[0079] Optionally, the processing module 202 is configured to fuse the sorting loss and the distillation loss in a fixed proportion to obtain the mixed loss.

[0080] Optionally, the processing module 202 is configured to perform word segmentation processing on the input text and the reply text, and convert the input text and the reply text into data sequences based on a preset word table.

[0081] In the embodiments of the present application, the sorting loss between the first prediction distribution output and the second prediction distribution output can be calculated, and the model parameters of the second text generation model can be updated based on the total loss obtained by combining the distillation loss, so that the consistency of the peak prediction of the two models can be calculated through the word-level sorting loss, thereby realizing efficient alignment of the multi-peak prediction distribution of the first text generation model and the second text generation model. In this way, not only can the fine-grained information between the peak categories of the two prediction distributions be more fully utilized, but also good compatibility with existing distillation targets can be ensured.

[0082] Figure 3 An example of an entity structure diagram of an electronic device is shown in Figure 3 As shown, the electronic device can include a processor 310, a communications interface 320, a memory 330, and a communications bus 340, wherein the processor 310, the communications interface 320, and the memory 330 can communicate with each other through the communications bus 340. The processor 310 can invoke the logical instructions in the memory 330 to execute the migration training method of the text generation model based on the sorting constraint, which includes: obtaining training data, each training data including an input text and a reply text; inputting the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model on each predicted position of the reply text; inputting the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model on each predicted position of the reply text; calculating a sorting loss and a distillation loss between the first prediction distribution output and the second prediction distribution output; fusing the sorting loss and the distillation loss to obtain a mixed loss, and taking the average of the mixed loss on all predicted positions as a total loss; performing back propagation based on the total loss to update the model parameters of the second text generation model; wherein the parameter size of the first text generation model is greater than the parameter size of the second text generation model.

[0083] In addition, the logical instructions in the memory 330 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0084] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor, so that the computer can execute the migration training method of the text generation model based on the ranking constraint provided by the above-mentioned method, the method comprises: obtaining training data, each training data comprising an input text and a reply text; inputting the training data into a first text generation model to obtain a first predicted distribution output of the reply text at each predicted position of the first text generation model; inputting the training data into a second text generation model to obtain a second predicted distribution output of the reply text at each predicted position of the second text generation model; calculating the ranking loss and the distillation loss between the first predicted distribution output and the second predicted distribution output; fusing the ranking loss and the distillation loss to obtain a hybrid loss, and taking the average of the hybrid loss at all predicted positions as a total loss; performing back propagation based on the total loss to update the model parameters of the second text generation model; wherein the parameter size of the first text generation model is greater than the parameter size of the second text generation model.

[0085] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the method for training a ranking constraint based text generation model provided by any of the above methods, the method comprising: obtaining training data, each training data comprising an input text and a reply text; inputting the training data into a first text generation model to obtain a first predicted distribution output of the first text generation model on each predicted position of the reply text; inputting the training data into a second text generation model to obtain a second predicted distribution output of the second text generation model on each predicted position of the reply text; calculating a ranking loss and a distillation loss between the first predicted distribution output and the second predicted distribution output; fusing the ranking loss and the distillation loss to obtain a hybrid loss, and taking the mean of the hybrid loss on all predicted positions as a total loss; performing back propagation based on the total loss to update model parameters of the second text generation model; wherein a parameter size of the first text generation model is greater than a parameter size of the second text generation model.

[0086] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0087] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary general hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in terms of the contribution to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0088] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A migration training method for a text generation model based on sorting constraints, characterized in that: include: Obtain training data, each training data including input text and response text; Inputting the training data into a first text generation model to obtain a first prediction distribution output of the first text generation model at each prediction position of the reply text; Inputting the training data into a second text generation model to obtain a second prediction distribution output of the second text generation model at each prediction position of the reply text; Calculating a ranking loss and a distillation loss between the first predicted distribution output and the second predicted distribution output; The ranking loss and the distillation loss are fused to obtain a mixed loss, and the average of the mixed losses at all predicted positions is taken as the total loss; Performing backpropagation based on the total loss to update model parameters of the second text generation model; Among them, the parameter scale of the first text generation model is greater than the parameter scale of the second text generation model.

2. The migration training method of the text generation model based on sorting constraints according to claim 1 is characterized in that The calculating the ranking loss and the distillation loss between the first prediction distribution output and the second prediction distribution output includes: Respectively obtain a head prediction sequence of the first text generation model and the second text generation model at each prediction position; Determine a union of a head prediction sequence of the first text generation model and a head prediction sequence of the second text generation model; Respectively obtaining a sequence of predicted probability values ​​of the categories in the union in the original prediction distributions of the first text generation model and the second text generation model; A ranking loss based on the Spearman rank correlation coefficient is calculated based on the predicted probability value sequence.

3. The migration training method of the text generation model based on sorting constraints according to claim 2 is characterized in that The objective function of the ranking loss is: ； in, represents the ranking loss, represents the Spearman rank correlation coefficient between the prediction distributions of the first and second text generation models, represents the predicted probability value sequence of the first text generation model, represents the predicted probability value sequence of the second text generation model, express and The covariance of and express and The standard deviation of .

4. The migration training method of the text generation model based on sorting constraints according to claim 1 is characterized in that The fusion of the sorting loss and the distillation loss to obtain a mixed loss includes: The sorting loss and the distillation loss are fused with each other in a fixed ratio to obtain the mixed loss.

5. The migration training method of the text generation model based on sorting constraints according to claim 1 is characterized in that After obtaining the training data, the method further includes: The input text and the reply text are segmented, and the input text and the reply text are converted into data sequences based on a preset vocabulary.

6. A migration training device for a text generation model based on sorting constraints, characterized in that: include: Acquisition module and processing module; The acquisition module is used to acquire training data, each training data includes input text and response text; The processing module is used to input the training data into the first text generation model to obtain the first prediction distribution output of the first text generation model at each prediction position of the reply text; input the training data into the second text generation model to obtain the second prediction distribution output of the second text generation model at each prediction position of the reply text; calculate the ranking loss and distillation loss between the first prediction distribution output and the second prediction distribution output; fuse the ranking loss and the distillation loss to obtain a mixed loss, and take the average of the mixed losses at all prediction positions as the total loss; perform backpropagation based on the total loss to update the model parameters of the second text generation model; Among them, the parameter scale of the first text generation model is greater than the parameter scale of the second text generation model.

7. The migration training device for the text generation model based on sorting constraints according to claim 6, characterized in that: The processing module is used to respectively obtain the head prediction sequence of the first text generation model and the second text generation model at each prediction position; determine the union of the head prediction sequence of the first text generation model and the head prediction sequence of the second text generation model; respectively obtain the prediction probability value sequence of the categories in the union in the original prediction distribution of the first text generation model and the second text generation model; and calculate the ranking loss based on the Spearman rank correlation coefficient according to the prediction probability value sequence.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the transfer training method of the text generation model based on sorting constraints as described in any one of claims 1 to 5 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the transfer training method of the text generation model based on sorting constraints as described in any one of claims 1 to 5 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the transfer training method of the text generation model based on sorting constraints as described in any one of claims 1 to 5 is implemented.

Images