Dialogue text generation method, system and device based on generative adversarial network

Abstract

The application discloses a dialogue text generation method, system and equipment based on an adversarial generation network, and belongs to the technical field of machine learning. The method is as follows: real-time question text of a first dialogue party is input into a target dialogue model to obtain real-time reply text of a second dialogue party. The target dialogue model is obtained by reinforcement learning training on an intermediate dialogue model. In each round of reinforcement learning training, a current dialogue model is trained based on a current training sample and a current reward sequence. The current training sample and the current reward sequence are obtained according to several rounds of dialogue interaction training of the current dialogue model and a target generator. Therefore, the application can enhance the generalization ability of the model and improve the dialogue effect.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a method, system, and device for generating dialogue text based on generative adversarial networks. Background Technology

[0002] With the rapid development of artificial intelligence technology, large language models are widely used in the field of dialogue systems. In professional dialogue scenarios such as psychological counseling, how to train dialogue models to possess professional conversational capabilities is currently a hot research topic.

[0003] Currently, dialogue models are typically trained using supervised learning, which involves collecting a large amount of real-world consultation dialogue data, extracting dialogue history and consultation responses as training samples, and then fine-tuning the large language model in a supervised manner to enable the model to learn consultation response patterns.

[0004] In psychological counseling dialogue scenarios, clients often exhibit resistance (such as silence, avoidance, and justification). However, supervised learning methods can only learn fixed mapping relationships in training samples, making it difficult for dialogue models to effectively identify and respond to diverse resistance questions, resulting in poor dialogue performance. Summary of the Invention

[0005] This invention provides a dialogue text generation method, system, and device based on generative adversarial networks, which can solve the problem in existing dialogue model training methods that, due to the use of supervised learning, can only learn fixed mapping relationships in training samples, making it difficult for the model to effectively identify and respond to diverse impedance question texts, resulting in poor dialogue performance.

[0006] This invention provides a dialogue text generation method based on generative adversarial networks, comprising:

[0007] Obtain the real-time question text from the first party in the conversation;

[0008] The real-time question text is input into the target dialogue model so that the target dialogue model can parse the real-time question text and output the real-time response text of the second dialogue party.

[0009] The target dialogue model is obtained by training the intermediate dialogue model through reinforcement learning. In each round of reinforcement learning training, the current dialogue model is trained based on the current training samples and the current reward sequence. The current training samples and the current reward sequence are obtained through several rounds of dialogue interaction training between the current dialogue model and the target generator. The intermediate dialogue model is obtained by training it using supervised learning techniques based on historical impedance dialogue data. The target generator and the target discriminator are obtained by adversarially training the intermediate generator and the intermediate discriminator using first historical dialogue data from the historical impedance dialogue data. The intermediate generator and the intermediate discriminator are obtained by training them using supervised learning techniques based on second historical dialogue data from the historical impedance dialogue data.

[0010] The target generator is used for the impedance speech of the first dialogue party.

[0011] This invention constructs a three-stage progressive model training system consisting of supervised learning pre-training, adversarial training optimization, and reinforcement learning enhancement. Through multi-level utilization of historical impedance dialogue data and interactive training of the target generator, a high-quality dialogue model for impedance dialogue scenarios is established, achieving a synergistic improvement in dialogue understanding accuracy, response generation quality, and dynamic interaction capabilities. This significantly enhances the response accuracy and adaptability of the dialogue model in complex interaction scenarios.

[0012] Furthermore, the target generator and target discriminator are obtained by adversarial training of the intermediate generator and intermediate discriminator based on the first historical dialogue data in the historical impedance dialogue data, specifically as follows:

[0013] In each batch of adversarial training, the first historical dialogue text in the first historical dialogue data is input into the first generator in the current adversarial training process to obtain the first generated question text of the first dialogue party.

[0014] The first historical dialogue text and the first generated question text are input into the first discriminator in the current adversarial training process to obtain the generated evaluation value and the generated classification value.

[0015] The first historical dialogue text and the first actual question text of the first dialogue party corresponding to the first historical dialogue text are input into the first discriminator to obtain the actual evaluation value and the actual classification value.

[0016] The generated evaluation value is calculated according to a preset first loss function to obtain a target generation loss value, and the model parameters of the first generator are updated according to the target generation loss value to obtain the next generator;

[0017] The generated classification value, the actual evaluation value, and the actual classification value are calculated according to the preset second loss function to obtain the target discrimination loss value. The parameters of the first discriminator are updated according to the target discrimination loss value to obtain the next discriminator.

[0018] The next generator and the next discriminator are iteratively trained until the first preset stopping condition is met, at which point the iteration stops and the target generator and the target discriminator are obtained.

[0019] This complete process, involving historical sample construction, generator text synthesis, discriminator dual-sample evaluation, dual loss function optimization, and iterative convergence determination, establishes an adversarial training mechanism for the co-evolution of the generator and discriminator. This achieves continuous improvement in the quality of generated text and simultaneous enhancement of discriminative evaluation capabilities, significantly improving the authenticity of the target generator's dialogue text and the evaluation accuracy of the target discriminator.

[0020] Further, the step of calculating the generated evaluation value according to a preset first loss function to obtain the target generation loss value specifically involves:

[0021] In the first preset layer of the first discriminator, the generated feature vector corresponding to the first generated question text and the actual feature vector corresponding to the first actual question text are obtained, and the Euclidean distance is calculated based on the generated feature vector and the actual feature vector.

[0022] In the second preset layer of the first generator, the generated semantic feature vector corresponding to the first generated question text is obtained, and the cosine similarity is calculated based on all the generated semantic feature vectors in each batch.

[0023] Calculate the negative of the average of all generated evaluation values ​​within each batch to obtain the first generation loss value;

[0024] Calculate the average of all the Euclidean distances within each batch to obtain the second generation loss value;

[0025] The third generation loss value is obtained by calculating the negative of the average value of all the cosine similarities within each batch;

[0026] The second generated loss value and the third generated loss value are weighted according to the first preset weighting coefficient, and then summed with the first generated loss value to obtain the target generated loss value.

[0027] By fusing multi-dimensional losses such as discriminator feature distance, generator semantic similarity, and adversarial scoring, a refined optimization target system for the generator was established. This enabled simultaneous optimization of the generated text at three levels: discriminant recognition, feature distribution, and semantic connotation, significantly improving the authenticity, semantic accuracy, and consistency with the target distribution of the generator's output text.

[0028] Further, the step of calculating the generated classification value, the actual evaluation value, and the actual classification value according to a preset second loss function to obtain the target discrimination loss value specifically involves:

[0029] The first discrimination loss value is obtained by calculating the negative of the average of all actual evaluation values ​​in each batch and the difference between it and the first generated loss value.

[0030] Random sampling is performed based on the first generated question text and the first actual question text to obtain interpolation samples. The interpolation samples are then input into the first discriminator to obtain a second discriminant loss value corresponding to the gradient constraint.

[0031] Based on the consistency between the generated classification value and the first impedance type label corresponding to the first actual question text, the classification cross-entropy loss is calculated to obtain the third discriminant loss value;

[0032] The second and third discriminant loss values ​​are weighted according to the second preset weighting coefficient, and then summed with the first discriminant loss value to obtain the target discriminant loss value.

[0033] By weighted fusion of adversarial loss, gradient constraint, and classification loss, a comprehensive optimization objective system for the discriminator is established. This achieves simultaneous enhancement of the discriminator in three dimensions: real / generated discrimination capability, training process stability, and impedance type recognition accuracy. This significantly improves the reliability of discriminator evaluation, the smoothness of training convergence, and the accuracy of type judgment.

[0034] Furthermore, the target dialogue model is obtained by training the intermediate dialogue model through reinforcement learning, specifically as follows:

[0035] In each round of reinforcement learning training, based on a preset optimization function, a weighted sum is calculated on a preset discount factor and each reward value in the current reward sequence to obtain the current cumulative reward value corresponding to the current training sample.

[0036] The current cumulative reward value is calculated based on a preset advantage estimation function to obtain the current advantage estimate.

[0037] Based on the current advantage estimate, the current dialogue model is updated by maximizing the preset pruning agent objective function based on the PPO algorithm to obtain the next dialogue model;

[0038] The next dialogue model is iteratively trained until the second preset stopping condition is met, at which point the iteration stops and the target dialogue model is obtained.

[0039] This complete process of cumulative discount reward calculation, advantage estimation, and PPO pruning optimization establishes a stable and efficient reinforcement learning training mechanism. It achieves synergistic optimization of the dialogue model in terms of maximizing long-term rewards, policy update stability, and convergence reliability, significantly improving the decision-making quality and convergence efficiency of the target dialogue model in dynamic interaction scenarios.

[0040] Furthermore, the current training samples and the current reward sequence are obtained based on several rounds of dialogue interaction training between the current dialogue model and the target generator, specifically:

[0041] In each round of dialogue interaction, the current dialogue sample is input into the target generator to obtain the current generated text of the first dialogue party, and the current generated text is input into the target discriminator to obtain the first reward value;

[0042] The currently generated text and the current dialogue sample are input into the current dialogue model to obtain the current response text of the second dialogue party. The current response text is then input into a preset evaluator to obtain a second reward value that characterizes the dialogue completion of the current response text.

[0043] The first reward value and the second reward value are weighted and summed according to the third preset weighting coefficient to obtain the current reward value;

[0044] The currently generated text and the currently replied text are added to the current dialogue sample in sequence to obtain the next dialogue sample. The update is stopped when the third preset stopping condition is met, and the current training sample is obtained. The current reward values ​​obtained from multiple rounds of dialogue interaction are arranged in chronological order to obtain the current reward sequence.

[0045] This complete interactive loop of target generator asking questions, target discriminator evaluating, current model responding, evaluator scoring, reward fusion, and sample updating establishes a high-quality dialogue simulation training environment. It realizes multi-source fusion of reward signals and dynamic accumulation of dialogue context, significantly improving the authenticity of reinforcement learning training samples and the reliability of reward signals, and providing high-quality training data support for the optimization of target dialogue models.

[0046] Furthermore, the intermediate generator and the intermediate discriminator are trained using supervised learning techniques based on the second historical dialogue data in the historical impedance dialogue data, specifically:

[0047] In each round of training, the second historical dialogue text in the second historical dialogue data, the second actual question text corresponding to the second historical dialogue text, and the second impedance type label corresponding to the second actual question text are input into the second generator in the current iteration process to obtain the second generated question text. Supervised learning training is performed with the optimization objective of minimizing the difference between the second generated question text and the second actual question text until the fourth preset stopping condition is met, at which point the iteration stops and the intermediate generator is obtained.

[0048] Simultaneously, the second generated question text, the second historical dialogue text, and the second actual question text are input into the second discriminator in the current iteration process. The second actual question text is used as a positive sample, and the second generated question text is used as a negative sample for supervised learning training until the fifth preset stopping condition is met, at which point the iteration stops and the intermediate discriminator is obtained.

[0049] By inputting labeled historical data, reconstructing and optimizing the generator, and training the discriminator in binary classification, a basic capability training mechanism for the generator and discriminator was established. This enabled the generator to initially fit the distribution of real questions and the discriminator to initially distinguish between real and fake samples, providing a good initial parameter foundation and capability starting point for subsequent adversarial training.

[0050] Furthermore, the method also includes:

[0051] Acquire raw impedance dialogue data and a data annotation model based on a large language model;

[0052] The original impedance dialogue data is input into the data labeling model. Each original historical dialogue text in the original impedance dialogue data is processed through multiple sampling or multiple prompting strategies to obtain multiple original impedance type labels corresponding to the original historical dialogue text.

[0053] The consistency evaluation value or confidence level among the multiple original impedance type labels is calculated, and the original historical dialogue text with the consistency evaluation value greater than or equal to a preset evaluation threshold, or the confidence level greater than or equal to a preset confidence threshold, is used as intermediate impedance dialogue data.

[0054] The intermediate impedance dialogue data is encoded according to a preset impedance classification system to obtain the historical impedance dialogue data.

[0055] This complete process of multi-strategy annotation, consistency screening, and standardized coding for large models establishes an intelligent construction mechanism for high-quality impedance dialogue data. It realizes the automated transformation from raw impedance dialogue data to structured training data, significantly improving the efficiency, consistency, and accuracy of data annotation, and providing a high-quality, large-scale, and standardized data foundation for model training.

[0056] Another embodiment of the present invention provides a dialogue text generation system based on generative adversarial networks, including: an acquisition module and a response module;

[0057] The acquisition module is used to acquire the real-time question text of the first dialogue party;

[0058] The response module is used to input the real-time question text into the target dialogue model, so that the target dialogue model can parse the real-time question text and output the real-time response text of the second dialogue party.

[0059] The target dialogue model is obtained by training the intermediate dialogue model through reinforcement learning. In each round of reinforcement learning training, the current dialogue model is trained based on the current training samples and the current reward sequence. The current training samples and the current reward sequence are obtained through several rounds of dialogue interaction training between the current dialogue model and the target generator. The intermediate dialogue model is obtained by training it using supervised learning techniques based on historical impedance dialogue data. The target generator and the target discriminator are obtained by adversarially training the intermediate generator and the intermediate discriminator using first historical dialogue data from the historical impedance dialogue data. The intermediate generator and the intermediate discriminator are obtained by training them using supervised learning techniques based on second historical dialogue data from the historical impedance dialogue data.

[0060] The target generator is used for the impedance speech of the first dialogue party.

[0061] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the steps of the dialogue text generation method based on adversarial generative networks of the present invention. Attached Figure Description

[0062] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0063] Figure 1This is a flowchart illustrating a dialogue text generation method based on generative adversarial networks provided in an embodiment of the present invention.

[0064] Figure 2 This is a schematic diagram of the structure of a dialogue text generation system based on an adversarial generative network provided in an embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0067] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0068] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0069] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0070] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0071] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0072] See Figure 1 To address the problem that existing supervised learning methods can only learn fixed mapping relationships in training samples, leading to poor model recognition and response to diverse impedance question texts and poor dialogue performance, an embodiment of this invention provides a dialogue text generation method based on generative adversarial networks, comprising:

[0073] Step 101: Obtain the real-time question text from the first dialogue party.

[0074] In this step, the dialogue text input by the user is collected in real time through a terminal device or application, which serves as the real-time question text of the first dialogue party. The real-time question text can be text input, speech-to-text, or interactive content obtained from the dialogue interface.

[0075] Step 102: Input the real-time question text into the target dialogue model so that the target dialogue model can parse the real-time question text and output the real-time response text of the second dialogue party.

[0076] The target dialogue model is obtained by training the intermediate dialogue model through reinforcement learning. In each round of reinforcement learning training, the current dialogue model is trained based on the current training samples and the current reward sequence. The current training samples and the current reward sequence are obtained through several rounds of dialogue interaction training between the current dialogue model and the target generator. The intermediate dialogue model is obtained by training it using supervised learning techniques based on historical impedance dialogue data. The target generator and the target discriminator are obtained by adversarially training the intermediate generator and the intermediate discriminator using first historical dialogue data from the historical impedance dialogue data. The intermediate generator and the intermediate discriminator are obtained by training them using supervised learning techniques based on second historical dialogue data from the historical impedance dialogue data.

[0077] The target generator is used for the impedance speech of the first dialogue party.

[0078] In this embodiment, the target dialogue model is obtained through the following training process:

[0079] Pre-training phase: Extract second historical dialogue data (including dialogue history and visitor resistance statements) from historical impedance dialogue data, and pre-train the initial generator and initial discriminator using supervised learning techniques to obtain intermediate generator and intermediate discriminator; at the same time, based on historical impedance dialogue data (including dialogue history and counselor effective responses), pre-train the initial dialogue model using supervised learning techniques to obtain intermediate dialogue model.

[0080] Adversarial training phase: Based on the first historical dialogue data, the intermediate generator and intermediate discriminator are trained adversarially to enable the generator to learn to generate realistic and diverse impedance speech, thus obtaining the target generator and target discriminator.

[0081] Reinforcement learning training phase: In each round of training, the current dialogue model and the target generator engage in multiple rounds of dialogue interaction. The target generator generates impedance speech based on the dialogue history, and the current dialogue model generates the consultant's response. The reward value for each round is calculated according to the preset reward function, forming the current training sample and the current reward sequence. The parameters of the current dialogue model are updated based on the reinforcement learning algorithm.

[0082] Repeat the reinforcement learning training process described above until the model converges to obtain the target dialogue model. Input the real-time question text obtained in step 101 into the target dialogue model. The model performs semantic parsing and contextual understanding on the input text and outputs the corresponding real-time response text from the second dialogue party, thus completing the dialogue generation.

[0083] As an example of an embodiment of the present invention, the target generator and target discriminator are obtained by adversarial training of the intermediate generator and intermediate discriminator based on the first historical dialogue data in the historical impedance dialogue data, specifically:

[0084] In each batch of adversarial training, the first historical dialogue text in the first historical dialogue data is input into the first generator in the current adversarial training process to obtain the first generated question text of the first dialogue party.

[0085] The first historical dialogue text and the first generated question text are input into the first discriminator in the current adversarial training process to obtain the generated evaluation value and the generated classification value.

[0086] The first historical dialogue text and the first actual question text of the first dialogue party corresponding to the first historical dialogue text are input into the first discriminator to obtain the actual evaluation value and the actual classification value.

[0087] The generated evaluation value is calculated according to a preset first loss function to obtain a target generation loss value, and the model parameters of the first generator are updated according to the target generation loss value to obtain the next generator;

[0088] The generated classification value, the actual evaluation value, and the actual classification value are calculated according to the preset second loss function to obtain the target discrimination loss value. The parameters of the first discriminator are updated according to the target discrimination loss value to obtain the next discriminator.

[0089] The next generator and the next discriminator are iteratively trained until the first preset stopping condition is met, at which point the iteration stops and the target generator and the target discriminator are obtained.

[0090] In this embodiment, a batch of training samples is sampled from the first historical dialogue data. Each sample contains the first historical dialogue text and its corresponding first actual question text. The first historical dialogue text is input into the current first generator to generate the first generated question text. The first historical dialogue text and the first generated question text are input into the first discriminator to obtain a generation evaluation value (authenticity score) and a generation classification value (impedance type probability distribution). The first historical dialogue text and the first actual question text are input into the first discriminator to obtain an actual evaluation value and an actual classification value. A first loss function is calculated based on the generation evaluation value to obtain the target generation loss value. The parameters of the first generator are updated through backpropagation to obtain the next generator. The first loss function aims to make the generated speech as high as possible in terms of discriminator score, while being close to the real sample in the feature space and maintaining the diversity among the generated samples. The second loss function is calculated based on the generated classification value, the actual evaluation value, and the actual classification value to obtain the target discriminant loss value. The parameters of the first discriminator are updated through backpropagation to obtain the next discriminator. The second loss function ensures that the discriminator scores real samples higher than generated samples, while also imposing gradient smoothing constraints on the discriminator and ensuring its classification output matches the true impedance type label. The next generator and the next discriminator are used as the first generator and the first discriminator in the current adversarial training process. Steps two through five are repeated until the model converges or reaches the preset training epochs, resulting in the target generator and the target discriminator.

[0091] As an example of an embodiment of the present invention, the step of calculating the generated evaluation value according to a preset first loss function to obtain the target generation loss value specifically involves:

[0092] In the first preset layer of the first discriminator, the generated feature vector corresponding to the first generated question text and the actual feature vector corresponding to the first actual question text are obtained, and the Euclidean distance is calculated based on the generated feature vector and the actual feature vector.

[0093] In the second preset layer of the first generator, the generated semantic feature vector corresponding to the first generated question text is obtained, and the cosine similarity is calculated based on all the generated semantic feature vectors in each batch.

[0094] Calculate the negative of the average of all generated evaluation values ​​within each batch to obtain the first generation loss value;

[0095] Calculate the average of all the Euclidean distances within each batch to obtain the second generation loss value;

[0096] The third generation loss value is obtained by calculating the negative of the average value of all the cosine similarities within each batch;

[0097] The second generated loss value and the third generated loss value are weighted according to the first preset weighting coefficient, and then summed with the first generated loss value to obtain the target generated loss value.

[0098] In this embodiment, the first generated question text and the first actual question text are respectively input into the first discriminator. The corresponding generated feature vector and actual feature vector are extracted in the first preset layer (e.g., the penultimate layer) of the first discriminator. The Euclidean distance between the two feature vectors is calculated as a feature matching metric. The smaller the distance, the closer the generated sample is to the real sample in the discriminator's deep feature space. The first generated question text is input into the first generator, and a generated semantic feature vector is extracted in the generator's second preset layer (e.g., the last hidden state). The cosine similarity between the two generated semantic feature vectors is calculated as a semantic similarity metric. The higher the similarity, the closer the generated sample is to the real sample semantically. The average of all generated evaluation values ​​in the current batch is calculated, and its negative value is taken to obtain the first generation loss value. The smaller this loss value, the higher the discriminator's score for the authenticity of the generated sample. The average of all Euclidean distances in the current batch is calculated to obtain the second generation loss value. The smaller this loss value, the closer the generated sample is to the real sample in the discriminator's deep feature space. The average cosine similarity of all samples in the current batch is calculated, and its negative value is taken to obtain the third generation loss value. The larger this loss value, the greater the semantic difference and the better the diversity among the generated samples in the current batch. The second and third generation loss values ​​are weighted and summed according to the first preset weight coefficient, and then added to the first generation loss value to obtain the target generation loss value. The gradient is calculated through backpropagation based on the target generation loss value, and the parameters of the first generator are updated so that the generator can generate more realistic and diverse impedance speech in subsequent training.

[0099] Specifically, the loss function of the first generator G is expressed as:

[0100] ;

[0101] Among them, the first generation loss term ( Adversarial loss, or adversarial loss, uses the discriminator's score of the generated fragment's authenticity as a sequence-level adversarial reward. real It is a real sample, that is, the first actual question text, r gen H represents the generated sample, i.e., the first generated question text, and H represents the dialogue history, i.e., the first historical dialogue text. The authenticity score output by the first discriminator D, i.e., the generation evaluation value, represents the generated text r under the dialogue history H. gen The probability of speaking a true impedance;

[0102] The second generation loss term, namely feature matching loss. , represents the feature output of a certain layer in the middle of the first discriminator D (e.g., the penultimate layer). The feature matching loss forces the generator to learn the deep feature distribution of the real sample by calculating the L2 distance between the real sample and the generated sample in the feature space.

[0103] The third generation loss term is the diversity loss. Used to prevent generator collapse, it is represented as:

[0104] ;

[0105] Where B is the batch size. For the first The semantic vector of each generated sample (which can be the mean of the last hidden state of the language model) is used to encourage the generated samples to be distributed in a dispersed manner in the semantic space. λ1 and λ2 are the balance coefficients, which are also the first weight coefficients.

[0106] As an example of an embodiment of the present invention, the step of calculating the generated classification value, the actual evaluation value, and the actual classification value according to a preset second loss function to obtain the target discrimination loss value specifically involves:

[0107] The first discrimination loss value is obtained by calculating the negative of the average of all actual evaluation values ​​in each batch and the difference between it and the first generated loss value.

[0108] Random sampling is performed based on the first generated question text and the first actual question text to obtain interpolation samples. The interpolation samples are then input into the first discriminator to obtain a second discriminant loss value corresponding to the gradient constraint.

[0109] Based on the consistency between the generated classification value and the first impedance type label corresponding to the first actual question text, the classification cross-entropy loss is calculated to obtain the third discriminant loss value;

[0110] The second and third discriminant loss values ​​are weighted according to the second preset weighting coefficient, and then summed with the first discriminant loss value to obtain the target discriminant loss value.

[0111] In this embodiment, the loss function of the first discriminator D adds a classification auxiliary task to the realism discrimination and incorporates a gradient penalty term, as follows:

[0112] ;

[0113] The first discriminant loss term, namely the authenticity loss, uses the difference between the real sample score and the generated sample score as the adversarial component; the second discriminant loss term, namely the gradient penalty... Is it a real sample r? real With the generated sample r gen Samples obtained by random interpolation on the connection line. It is an interpolated sample The probability distribution, specifically, refers to the probability distribution of the real sample r. realWith the generated sample r gen The distribution formed by all possible samples obtained through random interpolation on the connection line is used. Gradient penalty forces the discriminator to satisfy the 1-Lipschitz constraint on this interpolation space to stabilize training; the third discriminant loss term, i.e., the classification loss L... cls The type classification cross-entropy is calculated only for real samples to encourage the discriminator to extract features related to impedance type; λ3 and λ4 are balance coefficients, also known as the second weight coefficients.

[0114] Repeat the above steps until the generator's authenticity score, class consistency index, and diversity index on the validation set reach the preset threshold, or until the specified number of training rounds are reached, then stop the iteration.

[0115] As an example of an embodiment of the present invention, the target dialogue model is obtained by training an intermediate dialogue model through reinforcement learning, specifically as follows:

[0116] In each round of reinforcement learning training, based on a preset optimization function, a weighted sum is calculated on a preset discount factor and each reward value in the current reward sequence to obtain the current cumulative reward value corresponding to the current training sample.

[0117] The current cumulative reward value is calculated based on a preset advantage estimation function to obtain the current advantage estimate.

[0118] Based on the current advantage estimate, the current dialogue model is updated by maximizing the preset pruning agent objective function based on the PPO algorithm to obtain the next dialogue model;

[0119] The next dialogue model is iteratively trained until the second preset stopping condition is met, at which point the iteration stops and the target dialogue model is obtained.

[0120] In this embodiment, during each round of reinforcement learning training, the current reward sequence generated by the interaction between the current dialogue model and the target generator is obtained. R 1, R 2,…, R LThe reward sequence contains the immediate reward value for each round of interaction. Based on a preset discount factor γ, each reward value in the reward sequence is weighted and summed to obtain the cumulative discounted reward value G for each dialogue trajectory. This cumulative discounted reward value reflects the overall performance of the entire dialogue trajectory, and the discount factor γ is used to balance the importance of current rewards and future rewards. The cumulative discounted reward value G is input into a preset dominance estimation function (e.g., using GAE), and combined with the state value estimate output by the value network of the current dialogue model at sampling time, a dominance estimate is calculated. The dominance estimate measures the superiority or inferiority of the current action relative to the average level: a positive value indicates that the action is better than the average level, and a negative value indicates that it is worse than the average level. Based on the dominance estimate, and combined with the old policy probability and the current policy probability of the current dialogue model at sampling time, an importance sampling weight is calculated. The importance sampling weight is multiplied by the dominance estimate to obtain the first optimization objective; the importance sampling weight is clipped to the interval [1-ε, 1+ε] and then multiplied by the dominance estimate to obtain the second optimization objective, where ε is a preset clipping threshold. The smaller value between the first and second optimization objectives is taken as the pruning proxy objective function. The parameters of the current dialogue model are updated using gradient ascent to maximize this pruning proxy objective function, resulting in the next dialogue model. Specifically, the gradient of the pruning proxy objective function with respect to the model parameters is calculated, and the parameters are updated along the gradient direction, making the model more likely to generate actions that yield higher advantage estimates in subsequent interactions. The next dialogue model is used as the current dialogue model, and steps one through four are repeated for multiple rounds of iterative training. In each iteration, the current dialogue model interacts with the target generator again to generate new training samples and reward sequences, and the model parameters are updated accordingly. Iteration stops when the model performance converges or reaches a preset number of training rounds, yielding the target dialogue model.

[0121] Specifically, the trajectory of a complete round of dialogue τ =( a 1, r 2, a 3, r 4,…, a L ) and its corresponding reward sequence R 1, R 2,…, R L As training samples, the termination conditions of the complete dialogue trajectory include at least one of the following: reaching a preset maximum number of rounds; impedance strength falling below a threshold; the consultation process evaluator determining the task is complete; and the dialogue entering a natural ending state. After the trajectory ends, the states, actions, and rewards of all time steps are summarized to constitute reinforcement learning training samples. In this invention, the reinforcement learning strategy corresponds to the LLM psychological counseling model T, and its parameterized strategy π θ (a t |Ht ) indicates in the dialogue history H t Generate consultation response under the condition a t The probability distribution of H, where the state is the dialogue history. t The action is a response generated by the counselor. t The parameters of the LLM psychological counseling model T are updated using the PPO algorithm. The advantage function estimation is defined. If t, then the pruning proxy target of PPO is:

[0122] ;

[0123] in, Importance sampling weights; The pruning threshold is usually set to 0.2; the overall objective function may include a value function error term and an entropy reward term, which are omitted here.

[0124] During the policy update process, the pruning mechanism in the PPO algorithm is used to limit the deviation between the old and new policies, and the importance sampling ratio is constrained. In the interval Internally, this pruning mechanism prevents drastic changes in the strategy during a single update, thus avoiding the generation of unstable or out-of-context responses. This mechanism ensures the smoothness of strategy updates, allowing the model to maintain consistency and safety in its consulting behavior while learning new strategies. The optimization objective is to maximize the cumulative discount reward.

[0125] ;

[0126] Where γ∈[0,1] is the discount factor.

[0127] Iterate through the training until the performance of model T converges or reaches the preset number of training rounds.

[0128] As an example of an embodiment of the present invention, the current training sample and the current reward sequence are obtained based on several rounds of dialogue interaction training performed between the current dialogue model and the target generator, specifically:

[0129] In each round of dialogue interaction, the current dialogue sample is input into the target generator to obtain the current generated text of the first dialogue party, and the current generated text is input into the target discriminator to obtain the first reward value;

[0130] The currently generated text and the current dialogue sample are input into the current dialogue model to obtain the current response text of the second dialogue party. The current response text is then input into a preset evaluator to obtain a second reward value that characterizes the dialogue completion of the current response text.

[0131] The first reward value and the second reward value are weighted and summed according to the third preset weighting coefficient to obtain the current reward value;

[0132] The currently generated text and the currently replied text are added to the current dialogue sample in sequence to obtain the next dialogue sample. The update is stopped when the third preset stopping condition is met, and the current training sample is obtained. The current reward values ​​obtained from multiple rounds of dialogue interaction are arranged in chronological order to obtain the current reward sequence.

[0133] In this embodiment, an initial dialogue sample H0 is set, which may include the counselor's opening remarks or an empty dialogue history. In each round of dialogue interaction, the current dialogue sample is input into the target generator G. The target generator generates the current generated text of the first dialogue party (the client) based on the current dialogue context, simulating the client's resistive statements. This generated text is input into the target discriminator D, which outputs a realism score as the first reward value, used to measure the realism of the generated resistive statements. The current generated text is concatenated with the current dialogue sample to form an updated dialogue context. The updated dialogue context is input into the current dialogue model, which generates the current response text of the second dialogue party (the counselor) based on the context. This response text is input into a preset consultation process evaluator, which outputs a task success reward value, used to characterize the contribution of the response text to achieving the preset consultation effectiveness indicators, as the second reward value. The first reward value and the second reward value are weighted and summed according to a third preset weighting coefficient to obtain the immediate reward value for the current round. This immediate reward considers both the realism of the generated resistive statements and the effectiveness of the consultation response. The generated text and the current response text are added sequentially to the current dialogue sample to obtain the next round of dialogue samples. This process is repeated for multiple rounds of dialogue interaction, with each round generating new statements and updating the dialogue history, until a preset stopping condition is met, such as reaching the maximum number of dialogue rounds, the impedance strength falling below a threshold, or the evaluator determining the task is complete. After stopping the interaction, the dialogue history, all generated texts, all response texts, and their corresponding immediate reward values ​​generated throughout the entire dialogue interaction process are summarized to form the current training sample. Simultaneously, the immediate reward values ​​obtained from multiple rounds of dialogue interaction are arranged in chronological order to form the current reward sequence. This training sample and reward sequence will be used for subsequent reinforcement learning training to update the parameters of the dialogue model.

[0134] Specifically, using a pre-trained generator G as a dynamic environment, the dialogue strategy of the LLM counseling model T is optimized through a reinforcement learning algorithm, enabling it to learn effective coping mechanisms in interactions with increasingly resistant clients. A proximal policy optimization (PPO) algorithm is employed, incorporating a multi-dimensional reward function. The pre-trained LLM counseling model T is loaded, and the pre-trained generator G and discriminator D are fixed. The dialogue training samples and reward sequences are obtained through several rounds of dialogue interaction training between the intermediate dialogue model and the target generator. A dialogue is initialized, such as the counselor's opening remarks a1. For each round t: the current dialogue history H is... t Input generator G, generate the visitor's impedance response r t+1 =G(H t ). (H) t ,r t+1 Input the discriminator D to obtain the authenticity score D. t =D real (H t ,r t+1 As part of the survival reward, r t+1 Joining the history of dialogue, forming a new history H t+1 =H t ∪{r t+1 The LLM psychological counseling model T is based on H. t+1 Generate the next consultation response a t+2 =T(H t+1 ).Will at+2 Add to the dialogue history and proceed to the next round. Continue interaction until the preset maximum number of rounds is reached or the dialogue terminates naturally. The reward sequence is calculated based on the evaluation value output by the target discriminator according to a preset reward function; for each round of LLM response a... t The task success reward is calculated using an independent consultation process evaluator. This evaluator can share structure or parameters with the data filtering model, or be trained using the same feature set. The evaluator can be a pre-trained regression model, with the dialogue history H as input. t−1 And LLM response a t That is, H in the previous iteration t+1 and a t+2 Output a scalar R task ∈[0,1], indicating the effectiveness of the response in advancing counseling goals (such as establishing a therapeutic alliance, exploring the root causes of resistance, and alleviating resistance intensity). The training data for the evaluator can be composed of expert scores on a large number of dialogue segments. For each round t, the counselor responds with a. t The instant reward received is:

[0135] R t =α*R task+β*D t ;

[0136] Among them, D t This refers to the survival reward, where α and β are the third weighting coefficients. The immediate reward R... t This will serve as the foundational feedback signal in reinforcement learning policy optimization, used for subsequent advantage function estimation and policy gradient calculation. The reward sequence and the corresponding dialogue trajectory together constitute the reinforcement learning training samples.

[0137] As an example of an embodiment of the present invention, the intermediate generator and the intermediate discriminator are trained using supervised learning techniques based on the second historical dialogue data in the historical impedance dialogue data, specifically:

[0138] In each round of training, the second historical dialogue text in the second historical dialogue data, the second actual question text corresponding to the second historical dialogue text, and the second impedance type label corresponding to the second actual question text are input into the second generator in the current iteration process to obtain the second generated question text. Supervised learning training is performed with the optimization objective of minimizing the difference between the second generated question text and the second actual question text until the fourth preset stopping condition is met, at which point the iteration stops and the intermediate generator is obtained.

[0139] Simultaneously, the second generated question text, the second historical dialogue text, and the second actual question text are input into the second discriminator in the current iteration process. The second actual question text is used as a positive sample, and the second generated question text is used as a negative sample for supervised learning training until the fifth preset stopping condition is met, at which point the iteration stops and the intermediate discriminator is obtained.

[0140] In this embodiment, during each round of training, a batch of training samples are sampled from the second historical dialogue data. Each sample contains the second historical dialogue text H, the second actual question text r (the visitor's actual impedance statement) corresponding to the historical dialogue text, and the second impedance type label f ​​corresponding to the actual question text.

[0141] The second historical dialogue text H is input into the second generator G in the current iteration. The generator generates the second generated question text r' based on the dialogue history. The difference between the generated text r' and the actual text r is calculated. Minimizing this difference is the optimization objective. The second generator is trained using a negative log-likelihood loss function to learn the mapping relationship from the dialogue history to the actual impedance speech.

[0142] The parameters of the second generator are updated by backpropagation. The above process is repeated until the model converges or meets the fourth preset stopping condition. The iteration is then stopped, and an intermediate generator that can generate realistic impedance speech based on the dialogue history is obtained.

[0143] While training the generator, or alternately, a batch of training samples is sampled from the second historical dialogue data. The second historical dialogue text H and the second actual question text r are input into the second discriminator D in the current iteration as positive samples, enabling the discriminator to learn the characteristics of real impedance speech. Simultaneously, the second historical dialogue text H and the second generated question text r' generated by the current second generator are input into the second discriminator D as negative samples.

[0144] With the goal of distinguishing between positive and negative samples, a binary classification cross-entropy loss function is used to supervise the training of the second discriminator, enabling it to learn whether the input speech is a true impedance or a generated impedance. Specifically, the discriminator's output for positive samples should be close to 1, and its output for negative samples should be close to 0.

[0145] The parameters of the second discriminator are updated by backpropagation. The above process is repeated until the model converges or meets the fifth preset stopping condition. The iteration is then stopped, and an intermediate discriminator that can distinguish between the real impedance and the generated impedance is obtained.

[0146] The pre-training of the generator and discriminator can be performed independently or alternately. The discriminator parameters are fixed during generator training, and the generator parameters are fixed during discriminator training. Through supervised learning, the generator initially masters the ability to generate impedance signals, and the discriminator initially masters the ability to distinguish between true and false impedances, laying the foundation for subsequent adversarial training.

[0147] Specifically, the model is fine-tuned in a supervised manner using an open-source general-purpose large language model as the initial weights. For the sample (H) t−1 ,r t ,f t ), will the dialogue history H t−1 and impedance type label f t The input is concatenation, and the target output is the actual impedance statement r. t The training objective is to minimize the negative log-likelihood loss, expressed as:

[0148] ;

[0149] in, It is the distribution of real samples in historical impedance dialogue data. It is the probability that generator G generates real text given the dialogue history H and the impedance type label f.

[0150] This step enables generator G to initially grasp the ability to generate realistic impedance responses based on context and specified impedance types. The generator can employ a Transformer-based decoder-style language model, such as Qwen, GLM, or other models with autoregressive generation capabilities; the number of layers, hidden dimensions, and attention heads of the model can be configured according to computing power and corpus size, and are not limited to fixed values.

[0151] Based on encoder architectures (such as BERT), a multi-task discriminant model is constructed. The discriminator D contains two output heads: a truth discriminant head, which outputs a scalar D. real (H,r)∈[0,1], representing the probability that the input segment is the true impedance; a classification auxiliary head, used to output a probability distribution D with dimension K. cls (H,r), where K is the total number of categories in the current CR / WR classification system. Initially, K=12, and it dynamically expands based on the discovery of new impedances during the closed-loop evolution process. Real impedance fragments are used as positive samples, and impedance fragments generated by the pre-trained generator G are used as negative samples to train the discriminator D. The loss function consists of two parts, expressed as:

[0152] ;

[0153] in, The binary cross-entropy for authenticity determination, where (H,·) in the formula refers to the cross-entropy from the true data distribution R. real Take a positive sample (H, r) from the output, but only use the dialogue history H, ignoring the real text r, and use the negative sample output by the generator. Generate text in Perform calculations; γ is the type classification cross-entropy on real samples; γ is the balance coefficient. This step enables the discriminator D not only to distinguish between true and false signals, but also to learn fine-grained impedance features.

[0154] The impedance scenario discriminator D adopts a Transformer-based encoder architecture, specifically:

[0155] Base model: Initialized using a bidirectional encoder model such as BERT or RoBERTa;

[0156] Input layer: The dialogue history Ht−1 and the speech to be evaluated rt are concatenated into an input sequence, and a vector representation is obtained through a word embedding layer;

[0157] Position encoding: Absolute position encoding or relative position encoding is used;

[0158] Transformer encoder layer: Stacked L=24 encoder blocks, each layer has a structure similar to the generator decoder block (but without causal masking, and uses bidirectional attention).

[0159] Output layer: The output vector at position [CLS] is taken as the aggregate representation of the entire input sequence;

[0160] Authenticity Discrimination Head: Input the [CLS] vector into a linear layer + sigmoid, and output the authenticity score D. real ;

[0161] Classification auxiliary head: Input the [CLS] vector into another linear layer + softmax, and output a K-dimensional impedance type probability distribution D. cls .

[0162] As an example of an embodiment of the present invention, the method further includes:

[0163] Acquire raw impedance dialogue data and a data annotation model based on a large language model;

[0164] The original impedance dialogue data is input into the data labeling model. Each original historical dialogue text in the original impedance dialogue data is processed through multiple sampling or multiple prompting strategies to obtain multiple original impedance type labels corresponding to the original historical dialogue text.

[0165] The consistency evaluation value or confidence level among the multiple original impedance type labels is calculated, and the original historical dialogue text with the consistency evaluation value greater than or equal to a preset evaluation threshold, or the confidence level greater than or equal to a preset confidence threshold, is used as intermediate impedance dialogue data.

[0166] The intermediate impedance dialogue data is encoded according to a preset impedance classification system to obtain the historical impedance dialogue data.

[0167] In this embodiment, raw impedance dialogue data is acquired, including dialogue texts collected from anonymized real psychological counseling records, professional psychological counseling textbook cases, and clinical supervision cases. Simultaneously, a data annotation model based on a large language model is acquired. This data annotation model possesses the ability to identify and classify impedance behavior through prompting engineering or instruction fine-tuning. Each raw historical dialogue text from the raw impedance dialogue data is input into the data annotation model. Multiple independent judgments are performed on the same input (e.g., using multiple sampling or multi-prompt strategies), generating multiple raw impedance type labels corresponding to that raw historical dialogue text. For example, for the same client's speech, the model generates multiple candidate labels under different random seeds or different prompts. A consistency evaluation value (e.g., label similarity ratio or voting score) is calculated among the multiple raw impedance type labels, or a confidence score is obtained for each raw impedance type label. Raw historical dialogue texts with a consistency evaluation value greater than or equal to a preset evaluation threshold (e.g., 80%), or a confidence score greater than or equal to a preset confidence threshold (e.g., 0.9), are determined to be reliably labeled samples and retained as intermediate impedance dialogue data. Samples with low consistency or insufficient confidence are discarded or submitted for manual review. The filtered intermediate impedance dialogue data are encoded according to a pre-defined impedance classification system. This system includes at least two main categories: adversarial impedance (CR) and withdrawn impedance (WR), with each category containing multiple fine-grained impedance subtypes (such as refusal to intervene, complaining about the counselor, minimalist responses, and superficial compliance). Each impedance statement is mapped to a corresponding category code, forming a one-hot encoded impedance type label vector. The encoded data is then organized into historical impedance dialogue data according to a pre-defined format. Each sample contains the dialogue history H, the impedance statement r, and the impedance type label f, used for subsequent pre-training and adversarial training of the generator and discriminator.

[0168] It should be noted that the CR / WR classification system shown in Table 1 is an initial classification system, which is divided into Confrontation Rupture (CR) and Withdrawal Rupture (WR) according to the impedance performance. In the initial state, it includes the following 12 fine-grained categories, and will be dynamically expanded in subsequent steps according to the emergence of new impedances:

[0169] Table 1 Initial classification system

[0170]

[0171] The impedance type label f tThe generation is accomplished through an automatic classification model, which is built upon a large language model and combined with a predefined CR / WR classification system. The model determines the category of the input segment through cue constraints or classification output. The annotation result is a one-hot encoding ft∈{0,1}K of one of the above categories, where K=12 is the initial total number of categories.

[0172] To improve classification stability, multiple classification inferences are performed on the same sample, and the final category label is determined based on the consistency of probability distribution or voting mechanism. The corresponding confidence score can be output for subsequent screening and training weighting.

[0173] Collect valid response fragments generated by professional consultants to form training corpora for supervised fine-tuning of the consultation model. Each valid response sample includes at least one complete dialogue context and the corresponding consultant response, where: H t For the dialogue context that includes the client's and counselor's historical interactions; a t+1 This refers to an effective response from the counselor in this context. The effective response is automatically identified through a data filtering model, which is a scoring model built based on supervised learning or a pre-trained language model, used to quantitatively evaluate candidate counselor responses across multiple dimensions. Specifically, for each candidate sample, the data filtering model generates a corresponding score based on at least one of the following computable features:

[0174] (1) Dialogue progress score: Based on semantic similarity change or information increment index, calculate the semantic gain of the dialogue state before and after the counselor's response, for example, by quantifying the difference of context embedding vector (such as cosine distance) or information entropy change;

[0175] (2) Impedance relief score: Based on the impedance identification model, the impedance intensity of the visitor's speech before and after the response is predicted, and the decrease in impedance intensity is calculated as the score index.

[0176] (3) Treatment Alliance Score: Based on the pre-trained evaluation model, the empathic expression and supportive language features in the response are detected, such as by using emotion classification probability, empathic label classification results or keyword matching features for quantification;

[0177] (4) Language quality score: including fluency, coherence and grammatical correctness, which can be obtained through language model perplexity or discriminant model output.

[0178] The data filtering model inputs the above multidimensional features into a weighted scoring function to obtain a comprehensive score, expressed as:

[0179] ;

[0180] in, For the first Each feature score These are the corresponding weight parameters.

[0181] When the overall score S is greater than the preset threshold τ, the corresponding counselor response at+1 is judged as a valid response and included in the counselor dialogue history database Hcounselor.

[0182] Each sample is a single-turn or multi-turn dialogue segment, used for subsequent supervised fine-tuning of the basic dialogue capabilities of the LLM to be optimized.

[0183] like Figure 2 As shown, based on the above method embodiments, an embodiment of the present invention provides a dialogue text generation system 200 based on generative adversarial networks, including: an acquisition module 201 and a response module 202;

[0184] The acquisition module 201 is used to acquire the real-time question text of the first dialogue party;

[0185] The response module 202 is used to input the real-time question text into the target dialogue model so that the target dialogue model can parse the real-time question text and output the real-time response text of the second dialogue party.

[0186] The target dialogue model is obtained by training the intermediate dialogue model through reinforcement learning. In each round of reinforcement learning training, the current dialogue model is trained based on the current training samples and the current reward sequence. The current training samples and the current reward sequence are obtained through several rounds of dialogue interaction training between the current dialogue model and the target generator. The intermediate dialogue model is obtained by training it using supervised learning techniques based on historical impedance dialogue data. The target generator and the target discriminator are obtained by adversarially training the intermediate generator and the intermediate discriminator using first historical dialogue data from the historical impedance dialogue data. The intermediate generator and the intermediate discriminator are obtained by training them using supervised learning techniques based on second historical dialogue data from the historical impedance dialogue data.

[0187] The target generator is used for the impedance speech of the first dialogue party.

[0188] It is understood that the above system embodiments correspond to the method embodiments of the present invention, and can implement the dialogue text generation method based on adversarial generative networks provided by any of the above method embodiments of the present invention.

[0189] It should be noted that the system embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0190] For ease of description and brevity, the system embodiments of the present invention include all the implementation methods described in the above embodiments of the dialogue text generation method based on adversarial generative networks, and will not be repeated here.

[0191] Based on the above embodiments of the dialogue text generation method based on adversarial generative networks, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the dialogue text generation method based on adversarial generative networks of any embodiment of the present invention.

[0192] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0193] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0194] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0195] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the dialogue text generation method based on adversarial generative networks described in any of the above-described method embodiments of the present invention.

[0196] Based on the above-described method embodiments, this invention also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of any of the above-described method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0197] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0198] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A dialogue text generation method based on generative adversarial networks, characterized in that, include: Obtain the real-time question text from the first party in the conversation; The real-time question text is input into the target dialogue model so that the target dialogue model can parse the real-time question text and output the real-time response text of the second dialogue party. The target dialogue model is obtained by training the intermediate dialogue model through reinforcement learning. In each round of reinforcement learning training, the current dialogue model is trained based on the current training samples and the current reward sequence. The current training samples and the current reward sequence are obtained based on several rounds of dialogue interaction training between the current dialogue model and the target generator. The intermediate dialogue model is trained using supervised learning techniques based on historical impedance dialogue data; the target generator and target discriminator are obtained by adversarial training of the intermediate generator and intermediate discriminator based on the first historical dialogue data in the historical impedance dialogue data; the intermediate generator and the intermediate discriminator are trained using supervised learning techniques based on the second historical dialogue data in the historical impedance dialogue data. The target generator is used for the impedance speech of the first dialogue party.

2. The dialogue text generation method based on generative adversarial networks as described in claim 1, characterized in that, The target generator and target discriminator are obtained by adversarial training of the intermediate generator and intermediate discriminator based on the first historical dialogue data in the historical impedance dialogue data, specifically as follows: In each batch of adversarial training, the first historical dialogue text in the first historical dialogue data is input into the first generator in the current adversarial training process to obtain the first generated question text of the first dialogue party. The first historical dialogue text and the first generated question text are input into the first discriminator in the current adversarial training process to obtain the generated evaluation value and the generated classification value. The first historical dialogue text and the first actual question text of the first dialogue party corresponding to the first historical dialogue text are input into the first discriminator to obtain the actual evaluation value and the actual classification value. The generated evaluation value is calculated according to a preset first loss function to obtain a target generation loss value, and the model parameters of the first generator are updated according to the target generation loss value to obtain the next generator; The generated classification value, the actual evaluation value, and the actual classification value are calculated according to the preset second loss function to obtain the target discrimination loss value. The parameters of the first discriminator are updated according to the target discrimination loss value to obtain the next discriminator. The next generator and the next discriminator are iteratively trained until the first preset stopping condition is met, at which point the iteration stops and the target generator and the target discriminator are obtained.

3. The dialogue text generation method based on generative adversarial networks as described in claim 2, characterized in that, The step of calculating the generated evaluation value according to the preset first loss function to obtain the target generation loss value is as follows: In the first preset layer of the first discriminator, the generated feature vector corresponding to the first generated question text and the actual feature vector corresponding to the first actual question text are obtained, and the Euclidean distance is calculated based on the generated feature vector and the actual feature vector. In the second preset layer of the first generator, the generated semantic feature vector corresponding to the first generated question text is obtained, and the cosine similarity is calculated based on all the generated semantic feature vectors in each batch. Calculate the negative of the average of all generated evaluation values ​​within each batch to obtain the first generation loss value; Calculate the average of all the Euclidean distances within each batch to obtain the second generation loss value; The third generation loss value is obtained by calculating the negative of the average value of all the cosine similarities within each batch; The second generated loss value and the third generated loss value are weighted according to the first preset weighting coefficient, and then summed with the first generated loss value to obtain the target generated loss value.

4. The dialogue text generation method based on generative adversarial networks as described in claim 3, characterized in that, The step of calculating the target discrimination loss value based on the generated classification value, the actual evaluation value, and the actual classification value according to the preset second loss function is as follows: The first discrimination loss value is obtained by calculating the negative of the average of all actual evaluation values ​​in each batch and the difference between it and the first generated loss value. Random sampling is performed based on the first generated question text and the first actual question text to obtain interpolation samples. The interpolation samples are then input into the first discriminator to obtain a second discriminant loss value corresponding to the gradient constraint. Based on the consistency between the generated classification value and the first impedance type label corresponding to the first actual question text, the classification cross-entropy loss is calculated to obtain the third discriminant loss value; The second and third discriminant loss values ​​are weighted according to the second preset weighting coefficient, and then summed with the first discriminant loss value to obtain the target discriminant loss value.

5. The dialogue text generation method based on generative adversarial networks as described in claim 1, characterized in that, The target dialogue model is obtained by training the intermediate dialogue model through reinforcement learning, specifically: In each round of reinforcement learning training, based on a preset optimization function, a weighted sum is calculated on a preset discount factor and each reward value in the current reward sequence to obtain the current cumulative reward value corresponding to the current training sample. The current cumulative reward value is calculated based on a preset advantage estimation function to obtain the current advantage estimate. Based on the current advantage estimate, the current dialogue model is updated by maximizing the preset pruning agent objective function based on the PPO algorithm to obtain the next dialogue model; The next dialogue model is iteratively trained until the second preset stopping condition is met, at which point the iteration stops and the target dialogue model is obtained.

6. The dialogue text generation method based on generative adversarial networks as described in claim 1, characterized in that, The current training samples and the current reward sequence are obtained based on several rounds of dialogue interaction training between the current dialogue model and the target generator, specifically: In each round of dialogue interaction, the current dialogue sample is input into the target generator to obtain the current generated text of the first dialogue party, and the current generated text is input into the target discriminator to obtain the first reward value; The currently generated text and the current dialogue sample are input into the current dialogue model to obtain the current response text of the second dialogue party. The current response text is then input into a preset evaluator to obtain a second reward value that characterizes the dialogue completion of the current response text. The first reward value and the second reward value are weighted and summed according to the third preset weighting coefficient to obtain the current reward value; The currently generated text and the currently replied text are added to the current dialogue sample in sequence to obtain the next dialogue sample. The update is stopped when the third preset stopping condition is met, and the current training sample is obtained. The current reward values ​​obtained from multiple rounds of dialogue interaction are arranged in chronological order to obtain the current reward sequence.

7. The dialogue text generation method based on generative adversarial networks as described in claim 2, characterized in that, The intermediate generator and the intermediate discriminator are trained using supervised learning techniques based on the second historical dialogue data in the historical impedance dialogue data, specifically: In each round of training, the second historical dialogue text in the second historical dialogue data, the second actual question text corresponding to the second historical dialogue text, and the second impedance type label corresponding to the second actual question text are input into the second generator in the current iteration process to obtain the second generated question text. Supervised learning training is performed with the optimization objective of minimizing the difference between the second generated question text and the second actual question text until the fourth preset stopping condition is met, at which point the iteration stops and the intermediate generator is obtained. Simultaneously, the second generated question text, the second historical dialogue text, and the second actual question text are input into the second discriminator in the current iteration process. The second actual question text is used as a positive sample, and the second generated question text is used as a negative sample for supervised learning training until the fifth preset stopping condition is met, at which point the iteration stops and the intermediate discriminator is obtained.

8. The dialogue text generation method based on generative adversarial networks as described in claim 1, characterized in that, The method further includes: Acquire raw impedance dialogue data and a data annotation model based on a large language model; The original impedance dialogue data is input into the data labeling model. Each original historical dialogue text in the original impedance dialogue data is processed through multiple sampling or multiple prompting strategies to obtain multiple original impedance type labels corresponding to the original historical dialogue text. The consistency evaluation value or confidence level among the multiple original impedance type labels is calculated, and the original historical dialogue text with the consistency evaluation value greater than or equal to a preset evaluation threshold, or the confidence level greater than or equal to a preset confidence threshold, is used as intermediate impedance dialogue data. The intermediate impedance dialogue data is encoded according to a preset impedance classification system to obtain the historical impedance dialogue data.

9. A dialogue text generation system based on generative adversarial networks, characterized in that, include: Get module and reply module; The acquisition module is used to acquire the real-time question text of the first dialogue party; The response module is used to input the real-time question text into the target dialogue model, so that the target dialogue model can parse the real-time question text and output the real-time response text of the second dialogue party. The target dialogue model is obtained by training the intermediate dialogue model through reinforcement learning. In each round of reinforcement learning training, the current dialogue model is trained based on the current training samples and the current reward sequence. The current training samples and the current reward sequence are obtained based on several rounds of dialogue interaction training between the current dialogue model and the target generator. The intermediate dialogue model is trained using supervised learning techniques based on historical impedance dialogue data; the target generator and target discriminator are obtained by adversarial training of the intermediate generator and intermediate discriminator based on the first historical dialogue data in the historical impedance dialogue data; the intermediate generator and the intermediate discriminator are trained using supervised learning techniques based on the second historical dialogue data in the historical impedance dialogue data. The target generator is used for the impedance speech of the first dialogue party.

10. A terminal device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the dialogue text generation method based on any one of claims 1-8.

Images