Text generation model training method and device, equipment and storage medium

Abstract

The application provides a text generation model training method and device, equipment and a storage medium. The method comprises the following steps: obtaining a first text and a second text; performing vector encoding processing on the first text and the second text based on an encoder of a text generation model to obtain a first vector corresponding to the first text and a second vector corresponding to the second text; determining a first loss value according to the first vector and the second vector based on a preset contrast loss function; determining a target output text according to an input target condition vector and the first vector based on a vector processing network of the text generation model; determining a second loss value according to the target output text and the first text based on a preset training loss function; and adjusting parameters of the text generation model according to the first loss value and the second loss value to obtain a trained text generation model. The training effect of the text generation model can be improved, and the accuracy of the trained text generation model in text generation can be improved.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a training method, apparatus, device, and storage medium for a text generation model. Background Technology

[0002] In current natural language processing, text generation models are used to generate text that meets user needs. For example, conditional variational autoencoders (CVAs) are used to generate text. In medical consultation scenarios, doctors can input a small amount of information and data into the CVA to obtain medical records or prescriptions. However, most current training methods for CVAs do not consider the relevant information between training samples. As a result, when different labels are input, such as when a doctor inputs lesions and causes, the generated text may be quite similar or even identical, leading to insufficient accuracy in the text generated by the model. Summary of the Invention

[0003] This application provides a training method, apparatus, device, and storage medium for a text generation model, aiming to improve the accuracy of text generated by the text generation model through training.

[0004] Firstly, this application provides a method for training a text generation model, the method comprising the following steps:

[0005] Obtain a first text and a second text, wherein the text similarity between the first text and the second text is less than or equal to a preset text similarity threshold;

[0006] The encoder based on the text generation model performs vector encoding processing on the first text and the second text to obtain the first vector corresponding to the first text and the second vector corresponding to the second text.

[0007] Based on a preset contrast loss function, a first loss value is determined according to the first vector and the second vector;

[0008] The vector processing network based on the text generation model determines the target output text according to the input target condition vector and the first vector;

[0009] Based on a preset training loss function, a second loss value is determined according to the target output text and the first text;

[0010] The parameters of the text generation model are adjusted based on the first loss value and the second loss value to obtain a trained text generation model.

[0011] Secondly, this application also provides a training apparatus for a text generation model, the training apparatus for the text generation model comprising:

[0012] The text acquisition module is used to acquire a first text and a second text, wherein the text similarity between the first text and the second text is less than or equal to a preset text similarity threshold;

[0013] The vector encoding module is used to perform vector encoding processing on the first text and the second text based on the text generation model to obtain the first vector corresponding to the first text and the second vector corresponding to the second text.

[0014] The first loss calculation module is used to determine a first loss value based on the first vector and the second vector according to a preset contrast loss function;

[0015] The vector processing module is used to determine the target output text based on the vector processing network of the text generation model and the first vector.

[0016] The second loss calculation module is used to determine a second loss value based on the target output text and the first text, according to a preset training loss function.

[0017] The parameter adjustment module is used to adjust the parameters of the text generation model according to the first loss value and the second loss value to obtain a trained text generation model.

[0018] Thirdly, this application also provides a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the training method for the text generation model as described above.

[0019] Fourthly, this application also provides a storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the training method for the text generation model as described above.

[0020] This application provides a training method, apparatus, device, and storage medium for a text generation model. The method involves acquiring a first text and a second text, where the text similarity between the first and second texts is less than or equal to a preset text similarity threshold. Based on an encoder of the text generation model, the first and second texts are vector-encoded to obtain a first vector corresponding to the first text and a second vector corresponding to the second text. A first loss value is determined based on the first and second vectors using a preset contrastive loss function. A target output text is determined based on the input target conditional vector and the first vector using a vector processing network of the text generation model. A second loss value is determined based on the target output text and the first text using a preset training loss function. The parameters of the text generation model are adjusted based on the first and second loss values ​​to obtain a trained text generation model. By calculating the first loss value based on the first vector corresponding to the first text and the second vector corresponding to the second text, contrastive learning is achieved. During training, relevant information between samples is considered and calculated, enabling the trained model to further distinguish the differences in text generated by different tags, thereby improving the accuracy of text generation. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a training method for a text generation model provided in an embodiment of this application;

[0023] Figure 2 A scene diagram illustrating the training method for the text generation model implemented in this application;

[0024] Figure 3 A schematic block diagram of a training apparatus for a text generation model provided in an embodiment of this application;

[0025] Figure 4 This is a schematic block diagram of the structure of a computer device according to an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0028] This application provides a method, apparatus, computer device, and storage medium for training a text generation model. The training method for this text generation model can be applied to a terminal device, such as a tablet computer, laptop computer, or desktop computer. It can also be applied to a server, which can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0029] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0030] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a training method for a text generation model provided in an embodiment of this application.

[0031] like Figure 1 As shown, the training method of this text generation model includes steps S101 to S106.

[0032] Step S101: Obtain the first text and the second text, wherein the text similarity between the first text and the second text is less than or equal to a preset text similarity threshold.

[0033] For example, both the first text and the second text are training text data. The first text and the second text are dissimilar statements. Therefore, the text similarity between the first text and the second text is less than or equal to a preset text similarity threshold, so that the text generation model can receive dissimilar statements for training.

[0034] Step S102: The encoder based on the text generation model performs vector encoding processing on the first text and the second text to obtain the first vector corresponding to the first text and the second vector corresponding to the second text.

[0035] For example, the first text and the second text are input into the encoder of the text generation model so that the encoder of the text generation model performs vector encoding processing on the first text and the second text to obtain the first vector corresponding to the first text and the second vector corresponding to the second text.

[0036] For example, networks such as Long Short-Term Memory (LSTM) can be used to process the first text and the second text. It should be noted that the network used for vector encoding of the first text and the second text can be set according to actual needs, and this application does not limit it.

[0037] Step S103: Based on a preset contrast loss function, determine a first loss value according to the first vector and the second vector.

[0038] For example, after obtaining the first vector and the second vector, a first loss value is determined based on the first vector and the second vector according to a preset contrast loss function. It can be understood that the first loss value is used to indicate the contrast loss between the first text and the second text. By adjusting the model according to the first loss value, the model can better distinguish the differences between different texts, thereby improving the accuracy of the generated text.

[0039] Please see Figure 2 , Figure 2 A scene diagram illustrating the training method for implementing the text generation model in this application.

[0040] In some embodiments, the encoder based on the text generation model performs vector encoding processing on the first text and the second text to obtain a first vector corresponding to the first text and a second vector corresponding to the second text, including: performing vector encoding processing on the first text based on a first encoding network in the encoder of the text generation model to obtain a first vector; performing vector encoding processing on the first text based on a second encoding network in the encoder of the text generation model to obtain an associated vector of the first vector; wherein the node connection similarity in the first encoding network and the second encoding network is greater than or equal to a preset connection similarity threshold; determining a first loss value based on the first vector and the second vector based on a preset contrast loss function includes: when the first vector correlation degree between the associated vector and the first vector is less than or equal to a first vector correlation degree threshold, determining the first loss value based on the contrast loss function, the first vector, the second vector, and the associated vector.

[0041] For example, the encoder includes a first encoding network and a second encoding network, wherein the encoder includes multiple connected nodes, and different encoding networks are formed by making different connections to the nodes, such as adding or removing connected nodes.

[0042] For example, the first text is vector encoded using a first encoding network to obtain a first vector, and the second text is vector encoded using a second encoding network to obtain an associated vector of the first vector, so as to achieve multiple encodings of the first text to obtain two different vectors.

[0043] In the specific implementation process, the first encoding network and the second encoding network can be implemented through dropout. That is, in different vector encoding processes, different encoding networks are formed by randomly adding or removing connected nodes.

[0044] Specifically, the node connection similarity between the generated second encoding network and the first encoding network is greater than or equal to a preset connection similarity threshold, ensuring that the associated vectors of the first vector obtained from the first encoding network and the first vector obtained from the second encoding network are correlated. If the node connection similarity between the second encoding network and the first encoding network is less than the preset connection similarity threshold, the second encoding network is adjusted until the node connection similarity between the second encoding network and the first encoding network is greater than or equal to the preset connection similarity threshold.

[0045] For example, the node connection similarity between the first encoding network and the second encoding network is greater than or equal to a preset connection similarity threshold, so as to avoid the vectors generated by the two encodings being completely dissimilar, thus making it impossible to know the association between the same text.

[0046] In the specific implementation process, when the correlation degree between the correlation vector and the first vector is less than or equal to the correlation degree threshold of the first vector, the first loss value is determined based on the contrastive loss function, according to the first vector, the second vector and the correlation vector.

[0047] Understandably, when the correlation degree between the associated vector and the first vector is less than or equal to the correlation degree threshold of the first vector, it is determined that the generated associated vector is related to the first vector, which meets the requirements. Therefore, based on the contrastive loss function, the first loss value is determined according to the first vector, the second vector, and the associated vector.

[0048] It should be noted that any function that can calculate the contrast loss for the first vector, the second vector, and the associated vector can be used as the contrast loss function, and this application does not limit the contrast loss function.

[0049] In other embodiments, the method further includes: adjusting the node connections in the second encoding network when the first vector correlation degree is greater than the first vector correlation degree threshold; and performing vector encoding processing on the first text based on the adjusted second encoding network to obtain the correlation vector.

[0050] For example, when the correlation degree of the first vector is greater than the correlation degree threshold of the first vector, it is determined that the associated vector is not associated with the first vector, the node connections in the second encoding network are adjusted, and the first text is re-encoded based on the adjusted second encoding network to obtain the associated vector.

[0051] Understandably, the second encoding network is adjusted to randomly connect nodes to form a network with a different connection method than the first encoding network. The connection similarity between the adjusted second encoding network and the nodes in the first encoding network is greater than or equal to a preset connection similarity threshold.

[0052] By adjusting the second encoding network to ensure that the generated association vector is related to the first vector, while the first and second vectors are significantly different, the model can distinguish the differences between different texts and the similar features between similar texts, thereby improving the accuracy of the model when generating text.

[0053] In some embodiments, the method further includes: obtaining a third text, wherein the text similarity between the third text and the second text is greater than or equal to the preset similarity threshold, and the text similarity between the third text and the first text is less than the text similarity threshold; performing vector encoding processing on the third text based on an encoder of a text generation model to obtain a third vector corresponding to the third text; and determining the first loss value based on the contrast loss function according to the first vector, the second vector, and the association vector, including: determining the first loss value based on the contrast loss function according to the first vector, the second vector, the association vector, and the third vector.

[0054] Please continue reading. Figure 2 The third text is obtained, wherein the text similarity between the third text and the second text is greater than or equal to the text similarity threshold, and the text similarity between the third text and the first text is less than the text similarity threshold. That is, the third text and the second text are similar texts, and the third text and the first text are dissimilar. The second text and the third text are used as comparison texts that are dissimilar to the first text and are input into the text generation model for training. This allows the text generation model to know the differences between texts, thereby improving the text generation model to encode similar texts similarly and to encode dissimilar texts, thus improving the training effect and improving the text generation effect of the text generation model.

[0055] For example, based on the contrastive loss function, the model is trained using the first vector, the associated vector, the second vector, and the third vector. In this way, the similarity between the first vector and the associated vector can be reduced, while the similarity between the first vector and the second vector, and between the first vector and the third vector can be reduced. This allows the model to learn the common features between similar texts and distinguish the different features between dissimilar texts. Furthermore, contrastive learning does not require attention to the tedious details in the text, making the model training process simpler and more convenient.

[0056] In some embodiments, the method further includes: determining a second vector correlation degree between the correlation vector and the second vector or the third vector, and determining a third vector correlation degree between the second vector and the third vector; the step of determining the first loss value based on the contrastive loss function according to the first vector, the second vector and the correlation vector includes: when the correlation degree of the second vector is less than or equal to the first vector correlation degree threshold and the correlation degree of the third vector is greater than the second vector correlation degree threshold, determining the first loss value based on the contrastive loss function according to the first vector, the second vector and the correlation vector; wherein the second vector correlation degree threshold is greater than the first vector correlation degree threshold.

[0057] For example, by determining the vector correlation between vectors, it is determined whether the generated vector meets the requirements of the contrastive loss calculation. If it does not meet the requirements, the corresponding vector is regenerated. If it does meet the requirements, the first loss value is calculated based on the contrastive loss function, according to the first vector, the second vector, the correlated vector, and the third vector.

[0058] Step S104: Based on the vector processing network of the text generation model, determine the target output text according to the input target condition vector and the first vector.

[0059] For example, the text generation model also obtains the input target condition vector through a vector processing network, and determines the target output text based on the target condition vector and the first vector.

[0060] Understandably, the target condition vector is used to indicate target condition information so that the text generation model can generate the corresponding target output text based on the input target condition information.

[0061] For example, if the target condition information is the efficacy and ingredients of a drug, then the target output text will include a description of the efficacy and ingredients of the target drug, thereby shortening the time doctors spend entering information and improving the efficiency of medical consultations. Furthermore, by controlling the target condition vector, the form and content of the generated target output text can be controlled.

[0062] In some embodiments, the vector processing network based on the text generation model determines the target output text according to a preset condition vector and the first vector, including: a sampling sub-network based on the vector processing network sampling the target condition vector and the first vector; and a decoder based on the vector processing network decoding the sampled target condition vector and the sampled first vector to obtain the target output text.

[0063] For example, the vector processing network includes a sampling sub-network and a decoder. The sampling sub-network samples the target conditional vector and the first vector to avoid the target conditional vector and the first vector being too discrete, resulting in poor text generation and reduced model training effectiveness when using these generated texts for parameter adjustment. The encoder decodes the sampled target conditional vector and the sampled first vector to obtain the target output text.

[0064] By using a sampling sub-network and decoder, the text generation effect is improved, thereby enhancing the effectiveness of model training using the generated text.

[0065] In some embodiments, the method further includes: acquiring target condition information, the target condition information being used to indicate keywords in the target output text; encoding the target condition information based on the prior network of the text generation model to obtain a target condition vector corresponding to the target condition information; and inputting the target condition vector into the vector processing network of the text generation model.

[0066] Please continue reading. Figure 2 The method further includes obtaining target condition information, wherein the target condition information is determined in response to user input operations, and the target condition information is used to indicate keywords in the target output text, such as the ingredients and effects of a drug, and the target output text generated based on the target condition information includes the name, ingredients and their corresponding content of the target drug, as well as its effects and their corresponding content, thereby enabling the generation of the target output text guided by the target condition information.

[0067] In the specific implementation process, the text generation model also includes a prior network, which is used to encode the target condition information to obtain the target condition vector corresponding to the target condition information. The target condition vector is then input into the vector processing network of the text generation model so that the vector processing network can sample the target condition vector and determine the target output text based on the target condition vector and the first vector.

[0068] Understandably, by using the input target condition information and contrastive learning to determine the first loss value, there's no need to add labeled data. The input text data can utilize the input condition information to define which text is similar to and dissimilar to another, thus providing a supervisory signal for model training. Therefore, it can directly provide the final text generation effect based on the original text. For example, different texts are considered dissimilar, while the same text can be used to generate two different vectors by randomly deactivating some neural connections (using different node connection methods).

[0069] Step S105: Based on the preset training loss function, determine the second loss value according to the target output text and the first text.

[0070] For example, based on a preset training loss function, a second loss value is determined by determining the target output text and the first text. For instance, the second loss value is determined by the text similarity between the target output text and the first text. This application does not limit the training loss function.

[0071] Step S106: Adjust the parameters of the text generation model according to the first loss value and the second loss value to obtain the trained text generation model.

[0072] For example, the parameters of the text generation model are adjusted according to the first loss value and the second loss value to complete the training of the text generation model, so that the text generation model can perform text generation tasks. It can be understood that the trained text generation model can better distinguish the differences between texts, and thus can more accurately determine the generated text based on the input condition information, thereby improving the accuracy of text generation.

[0073] The training method for the text generation model provided in the above embodiments calculates a first loss value based on the first vector corresponding to the first text and the second vector corresponding to the second text, thereby achieving comparative learning. During the training process, relevant information between samples is considered and calculated, enabling the trained model to further distinguish the differences between texts generated by different labels, thereby improving the accuracy of the trained text generation model in text generation.

[0074] In specific implementation scenarios, the method of this application can be applied to the medical consultation process in smart hospitals. For example, based on the patient's description, the doctor determines the ingredients or efficacy of the corresponding medicine. For instance, if the patient is allergic to a certain ingredient, the doctor needs to avoid that ingredient when prescribing the medicine, and also determine a certain efficacy. In this case, the doctor inputs the required ingredients and efficacy into a trained text generation model. The trained text generation model determines the target condition information based on the input information. Based on the target condition information and the pre-input first text, which includes the medicine and its corresponding name, ingredients, efficacy, etc., the trained text generation model determines the target output text. The target output includes the name, ingredients, efficacy, etc., of the medicine. This eliminates the need for the doctor to memorize the specific ingredients and efficacy of the medicine, and also eliminates the need to search and select among similar medicines, thus improving the efficiency of the consultation. At the same time, since the trained text generation model can distinguish the differences in the text content based on the input target condition information, the accuracy of the medicine recommendation is improved.

[0075] Please see Figure 3 , Figure 3 A schematic block diagram of a training apparatus for a text generation model provided in an embodiment of this application.

[0076] like Figure 3 As shown, the training device for the text generation model includes: a text acquisition module 110, a vector encoding module 120, a first loss calculation module 130, a vector processing module 140, a second loss calculation module 150, and a parameter adjustment module 160.

[0077] The text acquisition module 110 is used to acquire a first text and a second text, wherein the text similarity between the first text and the second text is less than or equal to a preset text similarity threshold.

[0078] The vector encoding module 120 is used for an encoder based on a text generation model to perform vector encoding processing on the first text and the second text to obtain a first vector corresponding to the first text and a second vector corresponding to the second text.

[0079] The first loss calculation module 130 is used to determine a first loss value based on the first vector and the second vector according to a preset contrast loss function.

[0080] The vector processing module 140 is used to determine the target output text based on the vector processing network of the text generation model and the first vector.

[0081] The second loss calculation module 150 is used to determine a second loss value based on the target output text and the first text, according to a preset training loss function.

[0082] The parameter adjustment module 160 is used to adjust the parameters of the text generation model according to the first loss value and the second loss value to obtain a trained text generation model.

[0083] For example, the vector processing module 140 includes a sampling processing submodule and a decoder submodule.

[0084] The sampling processing submodule is used to perform sampling processing on the target condition vector and the first vector based on the sampling subnetwork of the vector processing network.

[0085] The decoder submodule is used to decode the target output text based on the vector processing network, according to the sampled target condition vector and the sampled first vector.

[0086] For example, the vector encoding module 120 includes a first encoding network submodule and a second encoding network submodule.

[0087] The first encoding network submodule is used to perform vector encoding processing on the first text based on the first encoding network in the encoder of the text generation model to obtain a first vector.

[0088] The second encoding network submodule is used to perform vector encoding processing on the first text based on the second encoding network in the encoder of the text generation model, and the associated vector of the first vector; wherein the node connection similarity between the first encoding network and the second encoding network is greater than or equal to a preset connection similarity threshold.

[0089] The first loss calculation module 130 is further configured to determine the first loss value based on the contrast loss function, according to the first vector, the second vector and the associated vector, when the first vector correlation degree between the associated vector and the first vector is less than or equal to the first vector correlation degree threshold.

[0090] For example, the training apparatus for the text generation model also includes an encoding network tuning module.

[0091] The encoding network adjustment module is used to adjust the node connections in the second encoding network when the first vector correlation degree is greater than the first vector correlation degree threshold.

[0092] The second encoding network submodule is further configured to perform vector encoding processing on the first text based on the adjusted second encoding network to obtain the associated vector.

[0093] For example, the text acquisition module 110 is further configured to acquire a third text, wherein the text similarity between the third text and the second text is greater than or equal to the text similarity threshold, and the text similarity between the third text and the first text is less than a preset similarity threshold.

[0094] The vector encoding module 120 is also used to perform vector encoding processing on the third text based on the text generation model to obtain the third vector corresponding to the third text.

[0095] The first loss calculation module 130 is further configured to determine the first loss value based on the contrast loss function, according to the first vector, the second vector, the correlation vector, and the third vector.

[0096] For example, the training apparatus for the text generation model also includes a correlation determination module.

[0097] The correlation degree determination module is used to determine the second vector correlation degree between the correlation vector and the second vector or the third vector, and to determine the third vector correlation degree between the second vector and the third vector.

[0098] The first loss calculation module 130 is further configured to determine the first loss value based on the comparison loss function, according to the first vector, the second vector, and the associated vector, when the second vector correlation degree is less than or equal to the first vector correlation degree threshold and the third vector correlation degree is greater than the second vector correlation degree threshold; wherein, the second vector correlation degree threshold is greater than the first vector correlation degree threshold.

[0099] For example, the training apparatus for the text generation model also includes an information acquisition module, a prior network module, and a vector input module.

[0100] The information acquisition module is used to acquire target condition information, which is used to indicate keywords in the target output text.

[0101] The prior network module is used to encode the target condition information based on the prior network of the text generation model to obtain the target condition vector corresponding to the target condition information.

[0102] The vector input module is used to input the target condition vector into the vector processing network of the text generation model.

[0103] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the training device for the text generation model described above can be referred to the corresponding process in the aforementioned training method embodiment of the text generation model, and will not be repeated here.

[0104] Please see Figure 4 , Figure 4 This is a schematic block diagram illustrating the structure of a computer device provided in an embodiment of this application. The computer device may be a server or a terminal.

[0105] like Figure 4 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and internal memory.

[0106] The storage medium may store the operating system and computer programs. The computer programs include program instructions that, when executed, cause the processor to perform a training method for any text generation model.

[0107] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0108] Internal memory provides an environment for the execution of computer programs stored in the storage medium. When these computer programs are executed by the processor, the processor can perform any training method for a text generation model.

[0109] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0110] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0111] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps:

[0112] Obtain a first text and a second text, wherein the text similarity between the first text and the second text is less than or equal to a preset text similarity threshold;

[0113] The encoder based on the text generation model performs vector encoding processing on the first text and the second text to obtain the first vector corresponding to the first text and the second vector corresponding to the second text.

[0114] Based on a preset contrast loss function, a first loss value is determined according to the first vector and the second vector;

[0115] The vector processing network based on the text generation model determines the target output text according to the input target condition vector and the first vector;

[0116] Based on a preset training loss function, a second loss value is determined according to the target output text and the first text;

[0117] The parameters of the text generation model are adjusted based on the first loss value and the second loss value to obtain a trained text generation model.

[0118] In one embodiment, when the processor implements a vector processing network based on the text generation model to determine the target output text according to a preset condition vector and the first vector, it is used to:

[0119] Based on the sampling sub-network of the vector processing network, the target condition vector and the first vector are sampled and processed.

[0120] The decoder based on the vector processing network performs decoding processing based on the sampled target condition vector and the sampled first vector to obtain the target output text.

[0121] In one embodiment, when the processor implements an encoder based on a text generation model to perform vector encoding processing on the first text and the second text to obtain a first vector corresponding to the first text and a second vector corresponding to the second text, it is used to implement:

[0122] Based on the first encoding network in the encoder of the text generation model, the first text is vector encoded to obtain a first vector;

[0123] Based on the second encoding network in the encoder of the text generation model, the first text is vector encoded to obtain the associated vector of the first vector; wherein, the node connection similarity between the first encoding network and the second encoding network is greater than or equal to a preset connection similarity threshold.

[0124] When the processor implements a preset contrastive loss function to determine a first loss value based on the first vector and the second vector, it is used to:

[0125] When the correlation degree between the correlation vector and the first vector is less than or equal to the first vector correlation degree threshold, the first loss value is determined based on the contrast loss function, according to the first vector, the second vector and the correlation vector.

[0126] In one embodiment, the processor, when implementing the training method for the text generation model, is used to implement:

[0127] When the correlation degree of the first vector is greater than the correlation threshold of the first vector, the node connections in the second encoding network are adjusted;

[0128] Based on the adjusted second encoding network, the first text is vector encoded to obtain the associated vector.

[0129] In one embodiment, the processor, when implementing the training method for the text generation model, is used to implement:

[0130] Obtain a third text, wherein the text similarity between the third text and the second text is greater than or equal to the text similarity threshold, and the text similarity between the third text and the first text is less than a preset similarity threshold;

[0131] An encoder based on a text generation model performs vector encoding on the third text to obtain a third vector corresponding to the third text;

[0132] When the processor determines the first loss value based on the contrastive loss function, according to the first vector, the second vector, and the correlation vector, it is also configured to:

[0133] Based on the contrast loss function, the first loss value is determined according to the first vector, the second vector, the correlation vector, and the third vector.

[0134] In one embodiment, the processor, when implementing the training method for the text generation model, is used to implement:

[0135] Determine the second vector correlation degree between the correlation vector and the second vector or the third vector, and determine the third vector correlation degree between the second vector and the third vector;

[0136] When the processor determines the first loss value based on the contrastive loss function, according to the first vector, the second vector, and the correlation vector, it is also configured to:

[0137] When the correlation degree of the second vector is less than or equal to the correlation degree threshold of the first vector, and the correlation degree of the third vector is greater than the correlation degree threshold of the second vector, the first loss value is determined based on the contrast loss function, according to the first vector, the second vector, and the correlation vector.

[0138] Wherein, the second vector correlation threshold is greater than the first vector correlation threshold.

[0139] In one embodiment, the processor, when implementing the training method for the text generation model, is used to implement:

[0140] Obtain target condition information, which is used to indicate keywords in the target output text;

[0141] Based on the prior network of the text generation model, the target condition information is encoded to obtain the target condition vector corresponding to the target condition information;

[0142] The target conditional vector is input into the vector processing network of the text generation model.

[0143] This application also provides a storage medium storing a computer program, the computer program including program instructions, and the method implemented when the program instructions are executed can refer to various embodiments of the training method of the text generation model of this application.

[0144] The storage medium can be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0145] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0146] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0147] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for a text generation model, characterized in that, include: Obtain a first text and a second text, wherein the text similarity between the first text and the second text is less than or equal to a preset text similarity threshold; The encoder based on the text generation model performs vector encoding processing on the first text and the second text to obtain a first vector corresponding to the first text and a second vector corresponding to the second text. This includes: performing vector encoding processing on the first text based on a first encoding network in the encoder of the text generation model to obtain a first vector; and performing vector encoding processing on the first text based on a second encoding network in the encoder of the text generation model to obtain an associated vector of the first vector; wherein the node connection similarity between the first encoding network and the second encoding network is greater than or equal to a preset connection similarity threshold. Determining the first loss value based on the first vector and the second vector according to the preset contrast loss function includes: when the first vector correlation degree between the correlation vector and the first vector is less than or equal to the first vector correlation degree threshold, determining the first loss value based on the first vector, the second vector and the correlation vector according to the contrast loss function; The vector processing network based on the text generation model determines the target output text according to the input target condition vector and the first vector; Based on a preset training loss function, a second loss value is determined according to the target output text and the first text; The parameters of the text generation model are adjusted based on the first loss value and the second loss value to obtain a trained text generation model.

2. The training method for the text generation model as described in claim 1, characterized in that, The vector processing network based on the text generation model determines the target output text according to a preset condition vector and the first vector, including: Based on the sampling sub-network of the vector processing network, the target condition vector and the first vector are sampled and processed. The decoder based on the vector processing network performs decoding processing based on the sampled target condition vector and the sampled first vector to obtain the target output text.

3. The training method for the text generation model as described in claim 1, characterized in that, The method further includes: When the correlation degree of the first vector is greater than the correlation threshold of the first vector, the node connections in the second encoding network are adjusted; Based on the adjusted second encoding network, the first text is vector encoded to obtain the associated vector.

4. The training method for the text generation model as described in claim 1, characterized in that, The method further includes: Obtain a third text, wherein the text similarity between the third text and the second text is greater than or equal to the text similarity threshold, and the text similarity between the third text and the first text is less than a preset similarity threshold; An encoder based on a text generation model performs vector encoding on the third text to obtain a third vector corresponding to the third text; The step of determining the first loss value based on the contrast loss function, according to the first vector, the second vector, and the correlation vector, includes: Based on the contrast loss function, the first loss value is determined according to the first vector, the second vector, the correlation vector, and the third vector.

5. The training method for the text generation model as described in claim 4, characterized in that, The method further includes: Determine the second vector correlation degree between the correlation vector and the second vector or the third vector, and determine the third vector correlation degree between the second vector and the third vector; The step of determining the first loss value based on the contrast loss function, according to the first vector, the second vector, and the correlation vector, includes: When the correlation degree of the second vector is less than or equal to the correlation degree threshold of the first vector, and the correlation degree of the third vector is greater than the correlation degree threshold of the second vector, the first loss value is determined based on the contrast loss function, according to the first vector, the second vector, and the correlation vector. Wherein, the second vector correlation threshold is greater than the first vector correlation threshold.

6. The training method for the text generation model as described in claim 1 or 2, characterized in that, The method further includes: Obtain target condition information, which is used to indicate keywords in the target output text; Based on the prior network of the text generation model, the target condition information is encoded to obtain the target condition vector corresponding to the target condition information; The target conditional vector is input into the vector processing network of the text generation model.

7. A training device for a text generation model, characterized in that, The training device for the text generation model includes: The text acquisition module is used to acquire a first text and a second text, wherein the text similarity between the first text and the second text is less than or equal to a preset text similarity threshold; The vector encoding module is used to perform vector encoding processing on the first text and the second text based on the encoder of the text generation model to obtain a first vector corresponding to the first text and a second vector corresponding to the second text. This includes: performing vector encoding processing on the first text based on a first encoding network in the encoder of the text generation model to obtain a first vector; and performing vector encoding processing on the first text based on a second encoding network in the encoder of the text generation model to obtain an associated vector of the first vector; wherein the node connection similarity between the first encoding network and the second encoding network is greater than or equal to a preset connection similarity threshold. The first loss calculation module is used to determine a first loss value based on a preset contrast loss function, according to the first vector and the second vector, including: when the first vector correlation degree between the correlation vector and the first vector is less than or equal to the first vector correlation degree threshold, the first loss value is determined based on the contrast loss function, according to the first vector, the second vector and the correlation vector. The vector processing module is used to determine the target output text based on the vector processing network of the text generation model and the first vector. The second loss calculation module is used to determine a second loss value based on the target output text and the first text, according to a preset training loss function. The parameter adjustment module is used to adjust the parameters of the text generation model according to the first loss value and the second loss value to obtain a trained text generation model.

8. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, it implements the steps of the training method for the text generation model as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the training method for the text generation model as described in any one of claims 1 to 6.

Images