Dialogue model training, dialogue generation method, system, computer and storage medium

Abstract

The present application relates to a dialogue model training, dialogue generation method, system, computer and storage medium, wherein the dialogue model includes a generator and a discriminator, and the dialogue model training method includes: a data set acquisition step, obtaining a training data set including sample sentences and their real response sentences; a discriminator training step, inputting the sample sentences into the generator to generate fake response sentences, combining them with real response sentences to construct a discriminator training set, and then inputting the discriminator to determine whether they are real; a generator training optimization step, obtaining the discrimination result of the discriminator training step, suppressing or amplifying the loss value of the sample based on the discrimination result of the discriminator until the loss function of the generator no longer decreases; an alternating training step, alternatingly executing the generator training optimization step or the discriminator training step until the loss functions of both the discriminator and the generator no longer decrease. Through this application, the robustness and accuracy of the dialogue model are improved, and the accuracy of dialogue generation is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a conversation model training and conversation generation method, system, computer device, and computer-readable storage medium based on a generative adversarial network. Background Art

[0002] A dialogue system, or dialogue agent, is a computer system designed to engage in conversations with humans. Dialogue systems communicate across input and output channels using one or more of text, voice, graphics, touch, gestures, and other modalities. Dialogue generation is a core algorithm and functional module within a dialogue system.

[0003] Currently, there are two main approaches to generative dialogue: pipeline-based methods and deep learning network-based methods. Pipeline-based dialogue generation methods primarily include three components: natural language understanding, dialogue state management, and natural language generation. Because the overall performance of pipeline-based models is constrained by all of these components, the model's generalization ability is poor. Multi-turn dialogue generation methods based on deep learning networks are primarily limited by the storage and utilization of information from previous rounds. As the number of dialogue rounds increases, so does the amount of background information. Furthermore, fundamental information such as the dialogue method and sequence length are not controlled, placing a high demand on the dialogue system's storage capacity. Summary of the Invention

[0004] The embodiments of the present application provide a method, system, computer device, and computer-readable storage medium for conversation model training and conversation generation based on a generative adversarial network, which improve the robustness and accuracy of the conversation model and the accuracy of conversation generation.

[0005] In a first aspect, an embodiment of the present application provides a method for training a dialogue model, wherein the dialogue model includes a generator and a discriminator, and the method includes:

[0006] A data set acquisition step of acquiring a training data set, wherein the training data set includes a plurality of real sentence pairs consisting of sample sentences and their real response sentences;

[0007] a discriminator training step, inputting the sample sentence into the generator through a sample selector to generate a fake reply sentence, constructing a discriminator training set based on the fake reply sentence and the real reply sentence, and inputting the sample sentence into the discriminator to determine whether the sentence is real;

[0008] A generator training optimization step, obtaining the discrimination result of the discriminator training step, and suppressing or amplifying the loss value of the sample corresponding to the discrimination result based on the discrimination result of the discriminator until the loss function of the generator no longer decreases;

[0009] The alternating training steps are performed, with the loss function of the discriminator or the loss function of the generator no longer decreasing as the alternating condition, and the generator training optimization step or the discriminator training step is performed alternately until the loss functions of the discriminator and the generator no longer decrease.

[0010] Through the above steps, this application uses a generative adversarial network to train a dialogue generation model, increasing the robustness and accuracy of the dialogue model from the perspective of natural language understanding. Compared with the ordinary single model training method, the difficulty of model training is increased, allowing the model to further fit the language characteristics of the real response, and further making the dialogue generated by the model authentic.

[0011] In some embodiments, the discriminator training step further comprises:

[0012] In the step of obtaining a fake reply, the sample sentence is extracted by the sample selector and input into the generator, and a fake reply sentence is generated by the generator. It should be noted that the generator at this time is used in the initial stage and has not been optimized and trained.

[0013] a discriminator training set acquisition step, extracting a set proportion of the real response sentences, and constructing a discriminator training set using the real response sentences and the forged response sentences;

[0014] A discriminator training step, wherein the discriminator training set is input to the discriminator, and the discriminator determines whether the sentence in the discriminator training set is true or false; that is, whether it is a true reply sentence from the training data set or a false reply sentence output by the generator;

[0015] The discriminator training optimization step is to repeatedly execute the forged reply acquisition step, the discriminator training set acquisition step and the discriminator training step until the loss function of the discriminator no longer decreases.

[0016] Through the above steps, the purpose of the embodiment of the present application is to help the dialogue model learn the characteristics of real reply sentences by detecting the difference between fake reply sentences and real reply sentences.

[0017] In some embodiments, in the generator training optimization step, suppressing or amplifying the loss value of the sample corresponding to the discrimination result further includes:

[0018] If the forged reply statement is judged to be forged, the loss value of the sample corresponding to the forged reply statement is multiplied by an amplification parameter α;

[0019] If the judgment result of the forged reply statement is true, the loss value of the sample corresponding to the forged reply statement is multiplied by a suppression parameter β, where α>1 and β<1.

[0020] Through the above steps, the embodiment of the present application suppresses or amplifies the loss value based on the discrimination result of the discriminator, thereby giving rewards or penalties to the generator, so that the fake reply sentences of the optimized generator are closer to the real reply sentences; the discriminator model is used to evaluate the quality of the generated fake reply sentences from a semantic perspective, avoiding the situation where fixed indicators do not have enough depth in language text mining, and improving the accuracy of dialogue generation.

[0021] In some embodiments, in the generator training optimization step, suppressing or amplifying the loss value of the sample corresponding to the discrimination result further includes:

[0022] If the forged reply statement is judged to be forged n times, the loss value of the sample corresponding to the forged reply statement is multiplied by the amplification parameter α n ;

[0023] If the number of times the forged reply statement is judged to be true is n, the loss value of the sample corresponding to the forged reply statement is multiplied by the suppression parameter β n , where α>1, β<1, n>1.

[0024] In a second aspect, an embodiment of the present application provides a method for generating a dialogue, including:

[0025] User query acquisition step, acquiring the query statement input by the user;

[0026] a reply statement generating step, using a generator as a dialogue generation model, inputting the query statement into the generator, and obtaining a reply statement corresponding to the query statement;

[0027] Wherein, the generator is trained using the dialogue model training method described in the first aspect above.

[0028] In a third aspect, an embodiment of the present application provides a conversation model training system, wherein the conversation model includes a generator and a discriminator, and the system includes:

[0029] A data set acquisition module is used to acquire a training data set, wherein the training data set includes a plurality of real sentence pairs consisting of sample sentences and their real response sentences;

[0030] A discriminator training module is configured to input the sample sentence into the generator through a sample selector to generate a fake reply sentence, construct a discriminator training set based on the fake reply sentence and the real reply sentence, and input the sample sentence into the discriminator to determine whether the sentence is real or not;

[0031] The generator training optimization module is used to obtain the discrimination result of the discriminator training module, and suppress or amplify the loss value of the sample corresponding to the discrimination result based on the discrimination result of the discriminator until the loss function of the generator no longer decreases.

[0032] An alternating training module is used to alternately execute the generator training optimization module or the discriminator training module, taking the loss function of the discriminator or the loss function of the generator no longer decreasing as an alternating condition, until the loss functions of the discriminator and the generator no longer decrease.

[0033] Through the above modules, this application uses a generative adversarial network to train the dialogue generation model, increasing the robustness and accuracy of the dialogue generation model from the perspective of natural language understanding. Compared with the ordinary single model training method, the difficulty of model training is increased, allowing the model to further fit the language characteristics of the annotated responses.

[0034] In some embodiments, the discriminator training module further comprises:

[0035] The fake reply acquisition module is used to extract the sample sentence through the sample selector and input it into the generator, and generate a fake reply sentence through the generator; it should be noted that the generator at this time is used in the initial stage and has not been optimized and trained.

[0036] a discriminator training set acquisition module, configured to extract a set proportion of the real reply sentences and construct a discriminator training set using the real reply sentences and the forged reply sentences;

[0037] The discriminator training module is used to input the discriminator training set to the discriminator, and the discriminator determines whether the sentence in the discriminator training set is true or false; that is, to determine whether it is a true response sentence from the training data set or a false response sentence output by the generator.

[0038] The discriminator training optimization module is used to repeatedly execute the forged reply acquisition module, the discriminator training set acquisition module and the discriminator training module until the loss function of the discriminator no longer decreases.

[0039] Through the above modules, the purpose of the embodiment of the present application is to help the dialogue model learn the characteristics of real reply sentences by detecting the difference between fake reply sentences and real reply sentences.

[0040] In some embodiments, in the generator training optimization module, the specific principle of suppressing or amplifying the loss value of the sample corresponding to the discrimination result is:

[0041] If the forged reply statement is judged to be forged, the loss value of the sample corresponding to the forged reply statement is multiplied by an amplification parameter α;

[0042] If the judgment result of the forged reply statement is true, the loss value of the sample corresponding to the forged reply statement is multiplied by a suppression parameter β, where α>1 and β<1.

[0043] Through the above steps, the embodiment of the present application suppresses or amplifies the loss value based on the discrimination result of the discriminator, thereby giving rewards or penalties to the generator, so that the fake reply sentences of the optimized generator are closer to the real reply sentences; the discriminator model is used to evaluate the quality of the generated fake reply sentences from a semantic perspective, avoiding the situation where fixed indicators do not have enough depth in language text mining, and improving the accuracy of dialogue generation.

[0044] In some embodiments, considering that a certain sample may be extracted into the generator multiple times, in view of this situation, the specific principle of suppressing or amplifying the loss value of the sample corresponding to the discrimination result in the generator training optimization module further includes:

[0045] If the forged reply statement is judged to be forged n times, the loss value of the sample corresponding to the forged reply statement is multiplied by the amplification parameter α n ;

[0046] If the number of times the forged reply statement is judged to be true is n, the loss value of the sample corresponding to the forged reply statement is multiplied by the suppression parameter β n , where α>1, β<1, n>1.

[0047] In a fourth aspect, an embodiment of the present application provides a dialogue generation system, including:

[0048] User query acquisition module, used to obtain the query statement input by the user;

[0049] A reply statement generation module is used to use a generator as a dialogue generation model, input the query statement into the generator, and obtain a reply statement corresponding to the query statement; wherein, the generator is trained using the dialogue model training method described in the first aspect above.

[0050] In a fifth aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the processor implements the dialogue model training method described in the first aspect and / or the dialogue generation method described in the second aspect.

[0051] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the dialogue model training method described in the first aspect above and / or implements the dialogue generation method described in the second aspect above.

[0052] Compared to related technologies, the embodiments of the present application provide a conversation model training, conversation generation method, system, computer device, and computer-readable storage medium based on a generative adversarial network. Specifically, they relate to a knowledge graph technology that uses a generative adversarial network to train a conversation generation model, thereby increasing the robustness and accuracy of the conversation model from the perspective of natural language understanding. Compared to ordinary single-model training methods, the difficulty of model training is increased. Through adversarial training of the generator and the discriminator, the fake reply statements output by the generator are similar to or even identical to the real reply statements. Compared to ordinary single-model training methods, the difficulty of model training is increased, allowing the model to further fit the language features of the real reply, further making the conversation generated by the model authentic. In addition, the present application does not require the maintenance of a very large chat corpus, has good tolerance for storage capacity, and has a flexible application environment.

[0053] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0055] Figure 1 is a flowchart of a method for training a dialogue model according to an embodiment of the present application;

[0056] Figure 2 is a step-by-step flow chart of a conversation model training method according to an embodiment of the present application;

[0057] Figure 3 is a flowchart of a conversation model training method according to a preferred embodiment of the present application;

[0058] Figure 4Schematic diagram of the principle of the dialogue model training method according to the preferred embodiment of the present application;

[0059] Figure 5 is a flowchart of a method for generating a conversation according to an embodiment of the present application;

[0060] Figure 6 is a structural block diagram of a dialogue model training system according to an embodiment of the present application;

[0061] Figure 7 is a structural block diagram of a dialogue generation system according to an embodiment of the present application;

[0062] Figure 8 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present application.

[0063] in:

[0064] 1. Dataset acquisition module; 2. Discriminator training module; 3. Generator training and optimization module;

[0065] 4. Alternating training module; 201. Forged response acquisition module; 202. Discriminator training set acquisition module;

[0066] 203. Discriminator training module; 204. Discriminator training optimization module;

[0067] 501. User query acquisition module; 502. Response statement generation module. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is described and illustrated below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely used to explain this application and are not intended to limit this application. Based on the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without making any creative efforts are within the scope of protection of this application.

[0069] Obviously, the drawings described below are merely examples or embodiments of the present application. Those skilled in the art can, without inventive effort, apply the present application to other similar scenarios based on these drawings. Furthermore, it is also understood that, although the effort involved in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, changes in design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as an insufficiency of the content disclosed in this application.

[0070] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments unless there is a conflict.

[0071] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0072] This embodiment provides a dialogue model training method, where the dialogue model includes a generator and a discriminator. Optionally, but not limited to, the generator of this application is constructed based on GPT (Generative Pre-training) and the discriminator is constructed based on the Bert model. The generator can also be constructed based on the RoBerta model or other classification models, and the discriminator can also be based on a series of language models such as GPT-2 and GPT-Large. Figure 1-2 is a flow chart of a method for generating a dialogue according to an embodiment of the present application, such as Figure 1-2 As shown, the process includes the following steps:

[0073] Dataset acquisition step S1, obtains a training dataset Data Set, which includes a number of real sentence pairs consisting of sample sentences and their real response sentences.<Query,Response> ;

[0074] Discriminator training step S2: input the sample sentence into the generator through a sample selector to generate a fake reply sentence. After constructing the discriminator training set based on the fake reply sentence and the real reply sentence, the sample sentence is input into the discriminator to determine whether it is true or not.

[0075] Generator training optimization step S3 obtains the discrimination result of the discriminator training step, and suppresses or amplifies the loss value of the sample corresponding to the discrimination result based on the discrimination result of the discriminator until the loss function of the generator GPT_loss no longer decreases.

[0076] In the alternating training step S4, the loss function of the discriminator or the loss function of the generator no longer decreases is used as the alternating condition, and the generator training optimization step S3 or the discriminator training step S2 is alternately executed until the loss functions of the discriminator and the generator no longer decrease.

[0077] Through the above steps, this application uses a generative adversarial network to train a dialogue generation model, increasing the robustness and accuracy of the dialogue model from the perspective of natural language understanding. Compared with the ordinary single model training method, the difficulty of model training is increased, allowing the model to further fit the language characteristics of the real response, and further making the dialogue generated by the model authentic.

[0078] In some embodiments, the discriminator training step S2 further comprises:

[0079] In the fake response acquisition step S201, the sample sentence Query is extracted through the sample selector and input into the generator, and the fake response sentence Fake Response is generated by the generator; it should be noted that the generator at this time is used in the initial stage and has not been optimized and trained. Optionally, the sample selector is a pre-trained semantic similarity model built based on the Bert model. Based on this, the sample sentences extracted by the sample selector each time are samples with high semantic similarity, which can be understood as clustering selection of samples, making the samples in the generator and discriminator training process more targeted, and improving the training efficiency of the generator and discriminator. The specific semantic similarity threshold can be flexibly set according to actual needs, which will not be elaborated here.

[0080] In step S202 , a discriminator training set is obtained by extracting a set proportion of real response sentences Response, and constructing a discriminator training set using the real response sentences Response and the fake response sentences Fake Response;

[0081] In the discriminator training step S203, the discriminator training set is input to the discriminator, and the discriminator determines whether the sentence in the discriminator training set is true or false; that is, whether it is a true response sentence from the training data set or a false response sentence output by the generator.

[0082] Discriminator training optimization step S204, repeatedly performing the forged reply acquisition step S201, the discriminator training set acquisition step S202, and the discriminator training step S203 until the discriminator loss function Bert_loss no longer decreases;

[0083] It should be noted that when the forged reply acquisition step S201 is executed for the first time, a sample sentence is randomly selected from the training data set and input into the generator and sample selector. The sample selector scores all samples in the training data set based on the sample sentence and selects the one with the highest score as the next sample sentence to participate in the forged reply acquisition step S201.

[0084] Through the above steps, the purpose of the embodiment of the present application is to help the dialogue model learn the characteristics of real reply sentences by detecting the difference between fake reply sentences and real reply sentences.

[0085] In some embodiments, in the generator training optimization step S3, suppressing or amplifying the loss value of the sample corresponding to the discrimination result further includes:

[0086] If the judgment result of the fake response statement Fake Response is fake, the loss value of the sample corresponding to the fake response statement FakeResponse is multiplied by the amplification parameter α;

[0087] If the judgment result of the fake response statement Fake Response is true, the loss value of the sample corresponding to the fake response statement FakeResponse is multiplied by a suppression parameter β, where α>1 and β<1.

[0088] Through the above steps, the embodiment of the present application suppresses or amplifies the loss value based on the discrimination result of the discriminator, thereby giving rewards or penalties to the generator, so that the fake reply sentences of the optimized generator are closer to the real reply sentences; the discriminator model is used to evaluate the quality of the generated fake reply sentences from a semantic perspective, avoiding the situation where fixed indicators do not have enough depth in language text mining, and improving the accuracy of dialogue generation.

[0089] In some embodiments, in the generator training optimization step S3, suppressing or amplifying the loss value of the sample corresponding to the discrimination result further includes:

[0090] If the number of times the forged response statement Fake Response is judged to be forged is n, then the loss value of the sample corresponding to the forged response statement Fake Response is multiplied by the amplification parameter α n ;

[0091] If the number of times that the fake response statement Fake Response is judged to be true is n, then the loss value of the sample corresponding to the fake response statement Fake Response is multiplied by the suppression parameter β n , where α>1, β<1, n>1.

[0092] The embodiments of the present application are described and illustrated below through preferred embodiments.

[0093] Figure 3 is a flowchart of a method for generating a conversation according to a preferred embodiment of the present application. Figure 4 Schematic diagram of the principle of the dialogue model training method according to the preferred embodiment of the present application. Figure 3-4 As shown, the dialogue generation method includes the following steps:

[0094] Step S301: construct or obtain a data set Data Set, which contains multiple real sentence pairs.<Query,Response> composition.

[0095] In step S302, a sample selector is used to extract several queries from the data set and input them to the generator GPT. The generator GPT generates corresponding forged responses. At this time, the generator GPT is not optimized and trained in the initial stage. The sample selector is implemented through the Bert model, which is a trained semantic similarity model.

[0096] In step S303, a certain proportion of the responses of real sentence pairs are taken from the dataset and combined with the fake responses generated in step S302 into a training set. The discriminator Bert is trained with this training set to determine whether the sentence pair comes from the dataset or is generated by the generator GPT. If it comes from the dataset, it is marked as True, otherwise it is Fake.

[0097] Step S304: Repeat steps S302 and S303 to train the discriminator Bert until the loss function value of the discriminator Bert no longer decreases.

[0098] Step S305: train the generator GPT. When the generator GPT’s fake response is judged as Fake by the discriminator Bert, the loss value of the sample is multiplied by α, (α>1); otherwise, it is judged as True, and the loss value of the sample is multiplied by β, (β<1). Repeatedly train the generator GPT until the loss function of the generator no longer decreases. If the number of times the discriminator finds the fake response corresponding to the sample is n, the loss value of the sample is multiplied by α. n ; Otherwise, the loss value of the sample is multiplied by β n .

[0099] In step S306, the generator GPT and the discriminator Bert are alternately trained and optimized until the loss function values ​​of both no longer decrease, and the training optimization is stopped.

[0100] Step S307: After training stops, the generator GPT is used as the dialogue generation model.

[0101] It should be noted that during the first training, a sample is randomly selected from the dataset. This sample is fed into both the generator GPT and the sample selector. The sample selector then scores all other samples in the dataset based on this sample. The sample with the highest score is used as the next training sample. This process continues until the loss of the generator GPT stops decreasing.

[0102] In addition, this embodiment also provides a dialogue generation method. Figure 5 is a flow chart of a method for generating a dialogue according to an embodiment of the present application, such as Figure 5 As shown, the process includes the following steps:

[0103] User query acquisition step S501, acquiring a query statement input by the user;

[0104] In the reply statement generation step S502, a generator is used as a dialogue generation model, a query statement is input into the generator, and a reply statement corresponding to the query statement is obtained; wherein the generator is trained using the dialogue model training method as described in the above embodiment.

[0105] The above steps provide a robust and accurate dialogue generation method that can be applied to customer service systems or other dialogue models to generate responses that are consistent with real-world scenarios.

[0106] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0107] This embodiment also provides a dialogue model training system, which includes a generator and a discriminator. Optionally, but not limited to, the generator of this application is constructed based on GPT (Generative Pre-training) and the discriminator is constructed based on the Bert model. The generator can also be constructed based on the RoBerta model or other classification models, and the discriminator can also be based on a series of language models such as GPT-2 and GPT-Large. The system is used to implement the above-mentioned embodiments and preferred implementation methods, and those that have been explained will not be repeated. As used below, the terms "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that implements predetermined functions. Although the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and conceived.

[0108] Figure 6 is a structural block diagram of a dialogue model training system according to an embodiment of the present application, such as Figure 6 As shown, the system includes a data set acquisition module 1, a discriminator training module 2, a generator training optimization module 3 and an alternating training module 4, wherein:

[0109] The data set acquisition module 1 is used to obtain the training data set Data Set, which includes a number of real sentence pairs consisting of sample sentences and their real response sentences.<Query,Response> ;

[0110] The discriminator training module 2 is used to input the sample sentence into the generator through a sample selector to generate a fake reply sentence, construct a discriminator training set based on the fake reply sentence and the real reply sentence, and input it into the discriminator to judge whether it is true or not; wherein, the discriminator training module 2 further includes: a fake reply acquisition module 201, a discriminator training set acquisition module 202, a discriminator training module 203 and a discriminator training optimization module 204.

[0111] Specifically, the fake response acquisition module 201 is used to extract the sample sentence Query through the sample selector and input it into the generator, and generate the fake response sentence Fake Response through the generator; it should be noted that the generator at this time is used in the initial stage and has not been optimized and trained. Optionally, the sample selector is a pre-trained semantic similarity model built based on the Bert model. Based on this, the sample sentences extracted by the sample selector each time are samples with high semantic similarity, which can be understood as clustering selection of samples, making the samples in the generator and discriminator training process more targeted, and improving the training efficiency of the generator and discriminator. The specific semantic similarity threshold can be flexibly set according to actual needs, which will not be elaborated here. The discriminator training set acquisition module 202 is used to extract a set proportion of real response sentences, and use the real response sentences Response and the fake response sentences Fake Response to construct the discriminator training set; the discriminator training module 203 is used to input the discriminator training set to the discriminator, and the discriminator discriminates whether the sentences in the discriminator training set are real or fake; that is, to judge whether they are real response sentences from the training data set or fake response sentences output by the generator. The discriminator training optimization module 204 is used to repeatedly execute the fake response acquisition module 201, the discriminator training set acquisition module 202 and the discriminator training module 203 until the loss function Bert_loss of the discriminator no longer decreases. Based on the above modules, the purpose of the embodiment of the present application is to help the dialogue model learn the characteristics of real response sentences by detecting the difference between fake response sentences and real response sentences.

[0112] It should be noted that when the forged reply acquisition module is executed for the first time, a sample sentence is randomly selected from the training dataset and input into the generator and sample selector. The sample selector scores all samples in the training dataset based on the sample sentence and selects the one with the highest score as the next sample sentence to participate in the forged reply acquisition module.

[0113] The generator training optimization module 3 is used to obtain the discrimination result of the discriminator training module 203, and suppress or amplify the loss value of the sample corresponding to the discrimination result based on the discrimination result of the discriminator, until the loss function of the generator GPT_loss no longer decreases. The specific principle of suppressing or amplifying the loss value of the sample corresponding to the discrimination result is as follows: if the discrimination result of the fake reply sentence is fake, the loss value of the sample corresponding to the fake reply sentence is multiplied by an amplification parameter α; if the discrimination result of the fake reply sentence is true, the loss value of the sample corresponding to the fake reply sentence is multiplied by a suppression parameter β, wherein α>1, β<1. Through the above steps, the embodiment of the present application suppresses or amplifies the loss value based on the discrimination result of the discriminator, thereby giving rewards or penalties to the generator, so that the fake reply sentence of the optimized generator is closer to the real reply sentence; the discriminator model is used to evaluate the quality of the generated fake reply sentence from a semantic perspective, avoiding the situation where the fixed indicators are not deep enough in language text mining, and improving the accuracy of dialogue generation.

[0114] In addition, considering that a certain sample may be extracted into the generator multiple times to generate fake replies, in view of this situation, the specific principle of suppressing or amplifying the loss value of the sample corresponding to the discrimination result in the generator training optimization module 3 also includes: if the discrimination result of the fake reply statement is fake n times, then the loss value of the sample corresponding to the fake reply statement is multiplied by the amplification parameter α n If the number of times the forged reply statement is judged to be true is n, the loss value of the sample corresponding to the forged reply statement is multiplied by the suppression parameter β n , where α>1, β<1, n>1.

[0115] The alternating training module 4 is used to alternately execute the generator training optimization module 3 or the discriminator training module 2, taking the loss function of the discriminator or the loss function of the generator no longer decreasing as the alternating condition, until the loss functions of the discriminator and the generator no longer decrease.

[0116] Through the above modules, this application uses a generative adversarial network to train the dialogue generation model, increasing the robustness and accuracy of the dialogue generation model from the perspective of natural language understanding. Compared with the ordinary single model training method, the difficulty of model training is increased, allowing the model to further fit the language characteristics of the annotated responses.

[0117] In addition, this embodiment also provides a dialogue generation system, which is used to implement the above-mentioned embodiments and preferred embodiments. Details already described will not be repeated here. As used below, the terms "module," "unit," "subunit," etc. may refer to a combination of software and / or hardware that implements a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0118] Figure 7 is a structural block diagram of a dialogue generation system according to an embodiment of the present application, such as Figure 7 As shown, the system includes: a user query acquisition module 501 and a reply statement generation module 502; the user query acquisition module 501 is used to obtain the query statement input by the user; the reply statement generation module 502 is used to use a generator as a dialogue generation model, input the query statement into the generator, and obtain a reply statement corresponding to the query statement; wherein, the generator is trained using the dialogue model training method as in the above embodiment.

[0119] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0120] In addition, combined Figure 1-2 The dialogue model training method and / or Figure 5 The dialog generation method described in the embodiment of the present application can be implemented by a computer device. Figure 8 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present application.

[0121] The computer device may include a processor 81 and a memory 82 storing computer program instructions.

[0122] Specifically, the processor 81 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0123] Among them, the memory 82 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 82 may be inside or outside the data processing device. In a specific embodiment, the memory 82 is a non-volatile memory. In a specific embodiment, the memory 82 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0124] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .

[0125] The processor 81 implements any one of the methods in the above embodiments by reading and executing computer program instructions stored in the memory 82 .

[0126] In some embodiments, the computer device may further include a communication interface 83 and a bus 80. Figure 8 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.

[0127] The communication interface 83 is used to enable communication between the various modules, devices, units, and / or devices in the embodiments of the present application. The communication port 83 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0128] The bus 80 includes hardware, software, or both, and couples components of the computer device to each other. The bus 80 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Bus 80 may include one or more buses, where appropriate. Although embodiments herein describe and illustrate a particular bus, this application contemplates any suitable bus or interconnect.

[0129] In addition, in conjunction with the conversation model training method and / or conversation generation method in the above embodiments, embodiments of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when executed by a processor, the computer program instructions implement any of the conversation generation methods in the above embodiments.

[0130] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for training a dialogue model, characterized in that: The dialogue model includes a generator and a discriminator, and the method includes: A data set acquisition step of acquiring a training data set, wherein the training data set includes a plurality of real sentence pairs consisting of sample sentences and their real response sentences; a discriminator training step, inputting the sample sentence into the generator through a sample selector to generate a fake reply sentence, constructing a discriminator training set based on the fake reply sentence and the real reply sentence, and inputting the sample sentence into the discriminator to determine whether the sentence is real; A generator training optimization step, obtaining the discrimination result of the discriminator training step, and suppressing or amplifying the loss value of the sample corresponding to the discrimination result based on the discrimination result of the discriminator until the loss function of the generator no longer decreases; In the generator training optimization step, suppressing or amplifying the loss value of the sample corresponding to the discrimination result further includes: If the forged reply statement is judged to be forged n times, the loss value of the sample corresponding to the forged reply statement is multiplied by the amplification parameter α n ; If the number of times the forged reply statement is judged to be true is n, the loss value of the sample corresponding to the forged reply statement is multiplied by the suppression parameter β n , where α>1, β<1, n>1; The alternating training steps are performed, with the loss function of the discriminator or the loss function of the generator no longer decreasing as the alternating condition, and the generator training optimization step or the discriminator training step is performed alternately until the loss functions of the discriminator and the generator no longer decrease.

2. The method for training a dialogue model according to claim 1, wherein: The discriminator training step further comprises: a fake reply acquisition step, extracting the sample sentence through the sample selector and inputting it into the generator, and generating a fake reply sentence through the generator; a discriminator training set acquisition step, extracting a set proportion of the real response sentences, and constructing a discriminator training set using the real response sentences and the forged response sentences; a discriminator training step, inputting the discriminator training set into the discriminator, wherein the discriminator discriminates whether the sentences in the discriminator training set are true or false; The discriminator training optimization step is to repeatedly execute the forged reply acquisition step, the discriminator training set acquisition step and the discriminator training step until the loss function of the discriminator no longer decreases.

3. The dialogue model training method according to claim 2, characterized in that In the generator training optimization step, suppressing or amplifying the loss value of the sample corresponding to the discrimination result further includes: If the forged reply statement is judged to be forged, the loss value of the sample corresponding to the forged reply statement is multiplied by an amplification parameter α; If the judgment result of the forged reply statement is true, the loss value of the sample corresponding to the forged reply statement is multiplied by a suppression parameter β, where α>1 and β<1.

4. A method for generating a dialogue, characterized in that: include: User query acquisition step, acquiring the query statement input by the user; a reply statement generating step, using a generator as a dialogue generation model, inputting the query statement into the generator, and obtaining a reply statement corresponding to the query statement; Wherein, the generator is trained using the dialogue model training method described in any one of claims 1 to 3.

5. A dialogue model training system, characterized in that: The dialogue model includes a generator and a discriminator, and the system includes: A data set acquisition module is used to acquire a training data set, wherein the training data set includes a plurality of real sentence pairs consisting of sample sentences and their real response sentences; A discriminator training module is configured to input the sample sentence into the generator through a sample selector to generate a fake reply sentence, construct a discriminator training set based on the fake reply sentence and the real reply sentence, and input the sample sentence into the discriminator to determine whether the sentence is real or not; A generator training optimization module is configured to obtain the discrimination result of the discriminator training module, and suppress or amplify the loss value of the sample corresponding to the discrimination result based on the discrimination result of the discriminator until the loss function of the generator no longer decreases; suppressing or amplifying the loss value of the sample corresponding to the discrimination result in the generator training optimization module further includes: If the forged reply statement is judged to be forged n times, the loss value of the sample corresponding to the forged reply statement is multiplied by the amplification parameter α n ; If the number of times the forged reply statement is judged to be true is n, the loss value of the sample corresponding to the forged reply statement is multiplied by the suppression parameter β n , where α>1, β<1, n>1; An alternating training module is used to alternately execute the generator training optimization module or the discriminator training module, taking the loss function of the discriminator or the loss function of the generator no longer decreasing as an alternating condition, until the loss functions of the discriminator and the generator no longer decrease.

6. The dialogue model training system according to claim 5, characterized in that: The discriminator training module further includes: a forged reply acquisition module, configured to extract the sample sentence through the sample selector and input it into the generator, and generate a forged reply sentence through the generator; a discriminator training set acquisition module, configured to extract a set proportion of the real reply sentences and construct a discriminator training set using the real reply sentences and the forged reply sentences; a discriminator training module, configured to input the discriminator training set into the discriminator, and the discriminator determines whether a sentence in the discriminator training set is true or false; The discriminator training optimization module is used to repeatedly execute the forged reply acquisition module, the discriminator training set acquisition module and the discriminator training module until the loss function of the discriminator no longer decreases.

7. A dialogue generation system, characterized in that: include: User query acquisition module, used to obtain the query statement input by the user; a reply statement generation module, configured to use the generator as a dialogue generation model, input the query statement into the generator, and obtain a reply statement corresponding to the query statement; Wherein, the generator is trained using the dialogue model training method described in any one of claims 1 to 3.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the dialogue model training method according to any one of claims 1 to 3 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the dialogue model training method according to any one of claims 1 to 4 is implemented.

Images