Dialogue information acquisition methods, devices, equipment and storage media

Abstract

This application relates to a method, apparatus, device, and storage medium for acquiring dialogue information, and pertains to the field of artificial intelligence technology. The method includes: acquiring first dialogue information; processing data using a first dialogue model based on the first dialogue information to obtain second dialogue information; processing data using a first discriminator based on the first and second dialogue information to obtain a first confidence score; and updating the parameters of the first dialogue model based on the first confidence score. In this scheme, the trained discriminator can detect the matching degree between the first dialogue information input to the first dialogue model and the second dialogue information output by the first dialogue model, and uses the matching score as a reward value. Furthermore, each round of dialogue can obtain a reward value corresponding to that round, increasing the density of reward value generation and thus improving the training efficiency of the first dialogue model obtained through reinforcement learning.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for acquiring dialogue information. Background Technology

[0002] Intelligent dialogue is a series of related technologies that receive text information input by users and process it through artificial intelligence in order to respond to the text information input by users.

[0003] In related technologies, intelligent dialogue systems include task-oriented dialogue systems and non-task-oriented dialogue systems (such as chatbots). Task-oriented dialogue systems are typically trained using reinforcement learning. During the training process of a task-oriented dialogue system using reinforcement learning, after each dialogue, a reward function is determined based on the number of dialogue rounds and whether the dialogue was successful, and the model parameters are updated based on the reward function.

[0004] In the above technical solution, when training a task-oriented dialogue system using reinforcement learning, the reward for each dialogue can only be obtained according to the reward function after each dialogue is completed, resulting in sparse rewards obtained during training and low training efficiency. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for acquiring dialogue information, which can improve the training efficiency of dialogue models. The technical solution is as follows:

[0006] On the one hand, a method for obtaining dialogue information is provided, the method comprising:

[0007] Obtain the first dialogue information;

[0008] Based on the first dialogue information, data processing is performed through the first dialogue model to obtain the second dialogue information; the first dialogue information and the second dialogue information constitute a round of dialogue.

[0009] Based on the first dialogue information and the second dialogue information, a first discriminator performs data processing to obtain a first confidence score; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first discriminator is obtained by training on at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator through adversarial learning.

[0010] Based on the first confidence score, the parameters of the first dialogue model are updated;

[0011] The trained first dialogue model is used to obtain fourth dialogue information based on the input third dialogue information.

[0012] On another front, a method for obtaining dialogue information is provided, the method comprising:

[0013] Obtain third-party dialogue information;

[0014] Based on the third dialogue information, data processing is performed through the first dialogue model to obtain the fourth dialogue information output by the first dialogue model.

[0015] Wherein, the first dialogue model is obtained by updating parameters based on a first confidence score; the first confidence score is obtained by data processing through a first discriminator based on first dialogue information and second dialogue information; the first dialogue information and the second dialogue information constitute a round of dialogue; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first discriminator is obtained by training at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator through adversarial learning.

[0016] Furthermore, a dialogue information acquisition device is provided, the device comprising:

[0017] The first dialogue acquisition module is used to acquire first dialogue information;

[0018] The second dialogue acquisition module is used to process data through the first dialogue model based on the first dialogue information to obtain the second dialogue information; the first dialogue information and the second dialogue information constitute a round of dialogue.

[0019] The confidence score acquisition module is used to process data through a first discriminator based on the first dialogue information and the second dialogue information to obtain a first confidence score; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first discriminator is obtained by training on at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator through adversarial learning.

[0020] The model parameter update module is used to update the parameters of the first dialogue model based on the first confidence score.

[0021] The trained first dialogue model is used to obtain fourth dialogue information based on the input third dialogue information.

[0022] In one possible implementation, the device further includes:

[0023] The first vector acquisition module is used to obtain a first feature vector corresponding to the first dialogue information based on the first dialogue information.

[0024] The second dialogue acquisition module includes:

[0025] The first vector processing unit is used to process the first feature vector through the first dialogue model to obtain the second feature vector;

[0026] The second dialogue acquisition unit is used to obtain the second dialogue information based on the second feature vector;

[0027] The confidence score acquisition module is also used for,

[0028] The first discriminator processes the first feature vector and the second feature vector to obtain the first confidence score.

[0029] In one possible implementation, the device further includes:

[0030] The probability distribution acquisition module is used to acquire a first probability distribution corresponding to the first feature vector based on the first dialogue information; the first probability distribution is used to indicate the confidence level of the information corresponding to each vector value in the first feature vector.

[0031] The first vector processing unit is further configured to,

[0032] Based on the first probability distribution, the first feature vector is processed by the first dialogue model to obtain the second feature vector.

[0033] In one possible implementation, the model parameter update module includes:

[0034] The value function value acquisition unit is used to acquire the value function value corresponding to the first dialogue model based on the first confidence score.

[0035] The model parameter update unit is used to update the parameters of the first dialogue model based on the value function value corresponding to the first dialogue model.

[0036] In one possible implementation, the first confidence score is any one of the confidence scores in the set of confidence scores;

[0037] The value function value acquisition unit is also used for,

[0038] Based on each confidence score in the set of confidence scores, the value function value corresponding to the first dialogue model is obtained.

[0039] In one possible implementation, the device further includes:

[0040] The first training set acquisition module is used to acquire the first training set; the first training set contains at least one set of sample dialogue information.

[0041] A simulated dialogue acquisition module is used to acquire at least one set of simulated dialogue information generated by the first generator;

[0042] The confidence data acquisition module is used to acquire first confidence data through a first discriminator based on the at least one set of sample dialogue information and the at least one set of simulated dialogue information; the first confidence data includes the confidence level corresponding to the at least one set of sample dialogue information and the confidence level corresponding to the at least one set of simulated dialogue information.

[0043] The discriminator update module is used to update the parameters of the first discriminator based on the first confidence data;

[0044] The generator update module is used to update the parameters of the first generator based on the confidence levels corresponding to at least one set of simulated dialogue information.

[0045] In one possible implementation, the device further includes:

[0046] The second training set acquisition module is used to acquire a second training set; the second training set contains at least one set of sample dialogue information.

[0047] The first generator training module is used to train the first generator based on at least one set of sample dialogue information in the second training set.

[0048] In one possible implementation, at least one set of the sample dialogue information in the second training set includes the first sample input information;

[0049] The first generator training module is also used for,

[0050] The first sample input information is input into the first encoding network in the first generator to obtain the first input feature value;

[0051] Based on the first input feature value, the first decoding network in the first generator is input to obtain the first predicted input information;

[0052] The first generator is updated based on the first sample input information and the first prediction input information.

[0053] In one possible implementation, at least one set of the sample dialogue information in the second training set includes the first sample output information;

[0054] The first generator training module is also used for,

[0055] The first sample output information is input into the first encoding network in the first generator to obtain the first output feature value;

[0056] Based on the first output feature value, input to the first decoding network in the first generator to obtain the first predicted output information;

[0057] The first generator is updated based on the first sample output information and the first prediction output information.

[0058] In another aspect, a dialogue information acquisition device is provided, the device comprising:

[0059] The third dialogue acquisition module is used to acquire third dialogue information.

[0060] The fourth dialogue acquisition module is used to process data through the first dialogue model based on the third dialogue information to obtain the fourth dialogue information output by the first dialogue model.

[0061] Wherein, the first dialogue model is obtained by updating parameters based on a first confidence score; the first confidence score is obtained by data processing through a first discriminator based on first dialogue information and second dialogue information; the first dialogue information and the second dialogue information constitute a round of dialogue; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first discriminator is obtained by training at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator through adversarial learning.

[0062] In another aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the above-described dialogue information acquisition method.

[0063] In another aspect, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-described dialogue information acquisition method.

[0064] In another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned dialogue information acquisition method.

[0065] The beneficial effects of the technical solutions provided in this application include at least the following:

[0066] By simultaneously training the generator and discriminator through adversarial learning, the discriminator gains a high ability to recognize the matching degree between a set of dialogue information. The trained discriminator can detect the matching degree between the first dialogue information input to the first dialogue model and the second dialogue information output by the first dialogue model, and use the matching degree between the first dialogue information and the second dialogue information as the reward value in reinforcement learning to update the parameters of the first dialogue model. That is, each round of dialogue can obtain the reward value corresponding to that round of dialogue, which increases the density of reward value generation and thus improves the training efficiency of the first dialogue model obtained through reinforcement learning.

[0067] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0068] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0069] Figure 1 This is a schematic diagram illustrating the structure of a dialogue information acquisition system according to an exemplary embodiment;

[0070] Figure 2 This is a flowchart illustrating a method for obtaining dialogue information according to an exemplary embodiment;

[0071] Figure 3 This is a flowchart illustrating a method for obtaining dialogue information according to an exemplary embodiment;

[0072] Figure 4 This is a flowchart illustrating a method for obtaining dialogue information according to an exemplary embodiment;

[0073] Figure 5 It shows Figure 4 The illustrated embodiment is a schematic diagram of a first generator structure.

[0074] Figure 6 It shows Figure 4The illustrated embodiment is a schematic diagram of an adversarial learning model.

[0075] Figure 7 This is a flowchart illustrating a method for obtaining dialogue information according to an exemplary embodiment;

[0076] Figure 8 This is a structural block diagram of a dialogue information acquisition device according to an exemplary embodiment;

[0077] Figure 9 This is a structural block diagram of a dialogue information acquisition device according to an exemplary embodiment;

[0078] Figure 10 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment. Detailed Implementation

[0079] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0080] Before describing the various embodiments shown in this application, several concepts involved in this application will be introduced first:

[0081] 1) AI (Artificial Intelligence)

[0082] AI (Artificial Intelligence) is the theory, methods, technology, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, artificial intelligence is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. Artificial intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities. Artificial intelligence technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0083] 2) NLP (Nature Language Processing)

[0084] NLP is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. Natural Language Processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close connection with linguistic research. Natural Language Processing techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0085] 3) ML (Machine Learning)

[0086] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0087] 4) RL (Reinforcement Learning)

[0088] Reinforcement learning, also known as reward learning, evaluation learning, or enhancement learning, is a paradigm and methodology in machine learning used to describe and solve problems where an agent learns policies to maximize rewards or achieve specific goals during interactions with its environment. A common reinforcement learning model is the standard Markov Decision Process (MDP). Depending on the conditions, reinforcement learning can be categorized into model-based reinforcement learning (RL) and model-free reinforcement learning (RL), as well as active reinforcement learning (RL) and passive reinforcement learning (RL). Variations of reinforcement learning include inverse reinforcement learning, hierarchical reinforcement learning, and reinforcement learning for partially observable systems. Algorithms used to solve reinforcement learning problems can be divided into policy search algorithms and value function algorithms. Deep learning models can be used in reinforcement learning, forming deep reinforcement learning.

[0089] 5) Adversarial learning

[0090] Adversarial learning generally refers to generative adversarial networks (GANs). Adversarial networks simulate a data probability distribution generator, making the probability distribution consistent with or as close as possible to the statistical probability distribution of the observed data. This process incorporates game-theoretic mechanisms from Nash equilibrium. Specifically, during training, the discriminative model extracts features that increase accuracy, while the generative model "falsifies" results to make the discriminative model believe they are genuine. Further, adversarial learning includes conditional generative adversarial networks, deep convolutional adversarial networks, etc., to address issues such as low stability, poor training data fidelity, and low training data quality.

[0091] Figure 1 This is a schematic diagram illustrating the structure of a dialogue information acquisition system according to an exemplary embodiment. The system includes a server 120 and a user terminal 140.

[0092] Server 120 may be a single server, or may include several servers, or may be a virtualization platform, or a cloud computing service center, etc. This application does not impose any restrictions.

[0093] User terminal 140 can be a terminal device with data processing capabilities, such as a mobile phone, tablet computer, laptop computer, or desktop computer. The number of user terminals 140 is not limited.

[0094] The user terminal 140 may have a client installed, which may be an office client, instant messaging client, browser client, educational client, etc., and may have voice input or text input functions. This application embodiment does not limit the software type of the client.

[0095] Optionally, the user can input text information through text input software in the client of the user terminal 140.

[0096] Optionally, users can also input voice information via the voice acquisition component in user terminal 140. User terminal 140 then calls the corresponding voice recognition function to convert the user's voice information into text information.

[0097] Optionally, the voice recognition function can be implemented by the user terminal 140 calling the voice recognition module in the user terminal 140.

[0098] Optionally, the speech recognition function can be implemented by the user terminal 140 calling the speech recognition module in the server corresponding to the client.

[0099] User terminal 140 and server 120 are connected via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0100] Optionally, the system may also include a management device ( Figure 1 (Not shown), the management device is connected to the server 120 via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0101] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to LAN (Local Area Network), MAN (Metropolitan Area Network), WAN (Wide Area Network), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including HTML (Hypertext Markup Language), XML (Extensible Markup Language), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as SSL (Secure Socket Layer), TLS (Transport Layer Security), VPN (Virtual Private Network), and IPsec (Internet Protocol Security) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0102] Optionally, the system involved in the embodiments of this application can be a distributed system formed by connecting a client and multiple nodes (any form of computing device in the network, such as a server or terminal) through network communication.

[0103] Optionally, the distributed system can be a blockchain system.

[0104] Optionally, as disclosed in this application, the data involved in the dialogue information acquisition method can be stored on the aforementioned blockchain system.

[0105] Figure 2This is a flowchart illustrating a method for obtaining dialogue information according to an exemplary embodiment. The method can be executed by a computer device, which can be a server, wherein the server can be the one described above. Figure 1 Server 120 in the illustrated embodiment. For example... Figure 2 As shown, the method for obtaining dialogue information may include the following steps:

[0106] Step 21: Obtain the first dialogue information.

[0107] Optionally, the initial dialogue information is pre-stored in a data storage component on the server.

[0108] Optionally, the first dialogue information is text data uploaded by the terminal device corresponding to the server.

[0109] Optionally, the first dialogue information is language text information directly uploaded by the terminal device corresponding to the server.

[0110] In one possible implementation, the server obtains the first voice information uploaded by the terminal device, recognizes the first voice information through a voice recognition module, and obtains the first dialogue information corresponding to the first voice information.

[0111] Step 22: Based on the first dialogue information, perform data processing through the first dialogue model to obtain the second dialogue information.

[0112] The first dialogue information and the second dialogue information constitute a round of dialogue.

[0113] Optionally, the first dialogue information and the second dialogue information constitute one round of dialogue in a multi-round dialogue.

[0114] In a task-oriented dialogue system, after receiving input dialogue information, the dialogue model can generate feedback information (e.g., second dialogue information) based on the input dialogue information (e.g., first dialogue information) and return it. At this point, the first and second dialogue information constitute one round of dialogue. However, in a task-oriented dialogue system, one round of dialogue is usually insufficient to help the user complete a specific task. Therefore, the dialogue model may receive new first dialogue information after completing this round. The dialogue model then generates new second dialogue information based on the new first dialogue information and returns it. This new first and second dialogue information also constitutes one round of dialogue. In other words, a round of dialogue consists of input dialogue information and dialogue information generated by the dialogue model based on that input dialogue information.

[0115] Optionally, the first dialogue model is a neural network model used to generate second dialogue information corresponding to the first dialogue information based on the input first dialogue information.

[0116] Optionally, the first dialogue model can be used to generate multi-turn dialogues based on the input dialogue information. The earlier turn in the multi-turn dialogue can influence the later turn; correspondingly, in the later turn, the dialogue information generated by the dialogue model after receiving the dialogue information can be influenced by the earlier turn.

[0117] Step 23: Based on the first dialogue information and the second dialogue information, data processing is performed by the first discriminator to obtain the first confidence score.

[0118] The first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first discriminator is obtained by training on at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator through adversarial learning.

[0119] In one possible implementation, the first discriminator can be trained together with the first generator through adversarial learning.

[0120] Optionally, the first discriminator can be used to indicate the confidence level of the input first dialogue information and the second dialogue information. Since the second dialogue information is generated by the first dialogue model after receiving the first dialogue information, the corresponding feedback information of the second dialogue information indicates the degree of matching between the second dialogue information and the first dialogue information.

[0121] Step 24: Update the parameters of the first dialogue model based on the first confidence score.

[0122] The trained first dialogue model is used to obtain fourth dialogue information based on the input third dialogue information.

[0123] During the training process of the aforementioned first dialogue model, for each input dialogue information, when the first dialogue model generates output dialogue information based on the input dialogue information, it generates a confidence score between the input dialogue information and the output dialogue information based on the first discriminator, and updates the parameters of the first dialogue model according to the confidence score. That is to say, in a multi-turn dialogue scenario, the first dialogue model generates a corresponding confidence score for each turn of dialogue. Therefore, the parameter update process of the first dialogue model considers the matching degree of each turn of dialogue in a multi-turn dialogue, improves the impact of the correctness of each turn of dialogue on the update of the first dialogue model, and improves the accuracy of dialogue information acquisition of the updated first dialogue model.

[0124] Furthermore, in multi-turn dialogue scenarios, since dialogues of different numbers may involve dialogue content from different domains, during reinforcement learning, when updating the model after completing a multi-turn dialogue, the differences between dialogues from different domains cannot be taken into account. However, with the solution shown in the embodiments of this application, since each round of dialogue generates a corresponding confidence score to update the first dialogue model, the differences in dialogue information from different domains can be taken into account during the update process of the first dialogue model, thereby improving the accuracy of dialogue information acquisition of the updated first dialogue model in multi-turn dialogue scenarios.

[0125] In summary, in the scheme shown in the embodiments of this application, the generator and discriminator are trained simultaneously through adversarial learning, enabling the discriminator to have a high ability to recognize the matching degree between a set of dialogue information. The trained discriminator can detect the matching degree between the first dialogue information input to the first dialogue model and the second dialogue information output by the first dialogue model, and use the matching degree between the first dialogue information and the second dialogue information as the reward value in reinforcement learning to update the parameters of the first dialogue model. That is, each round of dialogue can obtain the reward value corresponding to that round of dialogue, which increases the density of reward value generation and thus improves the training efficiency of the first dialogue model obtained through reinforcement learning.

[0126] Figure 3 This is a flowchart illustrating a method for obtaining dialogue information according to an exemplary embodiment. The method can be executed by a computer device, which can be a server, wherein the server can be the one described above. Figure 1 Server 120 in the illustrated embodiment. For example... Figure 3 As shown, the method for obtaining dialogue information may include the following steps:

[0127] Step 31: Obtain the third dialogue information.

[0128] Optionally, the third dialogue information is text data uploaded by the terminal device corresponding to the server.

[0129] Optionally, the third dialogue information is language text information directly uploaded by the terminal device corresponding to the server.

[0130] In one possible implementation, the server obtains the second voice information uploaded by the terminal device, recognizes the second voice information through a voice recognition module, and obtains the third dialogue information corresponding to the second voice information.

[0131] Step 32: Based on the third dialogue information, data processing is performed through the first dialogue model to obtain the fourth dialogue information output by the first dialogue model.

[0132] The third dialogue information and the fourth dialogue information constitute a round of dialogue.

[0133] Optionally, the third dialogue information and the fourth dialogue information constitute one round of dialogue in a multi-round dialogue.

[0134] In a task-oriented dialogue system, after receiving input dialogue information, the dialogue model can generate feedback information (e.g., a fourth dialogue message) based on the input dialogue information (e.g., a third dialogue message) and return it. At this point, the third and fourth dialogue messages constitute one round of dialogue. However, in a task-oriented dialogue system, one round of dialogue is usually insufficient to help the user complete a specific task. Therefore, the dialogue model may receive new third dialogue information after completing this round. The dialogue model then generates new fourth dialogue information based on this new third dialogue information and returns it. In this case, the new first and fourth dialogue messages also constitute one round of dialogue. That is, a round of dialogue consists of the input dialogue information and the dialogue information generated by the dialogue model based on that input dialogue information.

[0135] The first dialogue model is obtained by updating parameters based on a first confidence score; the first confidence score is obtained by data processing through a first discriminator based on first dialogue information and second dialogue information; the first dialogue information and the second dialogue information constitute a round of dialogue; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first discriminator is obtained by training at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator through adversarial learning.

[0136] In summary, in the technical solution provided in this application embodiment, the generator and discriminator are trained simultaneously through adversarial learning, enabling the discriminator to have a high ability to recognize the matching degree between a set of dialogue information. The trained discriminator can detect the matching degree between the first dialogue information input to the first dialogue model and the second dialogue information output by the first dialogue model, and use the matching degree between the first dialogue information and the second dialogue information as the reward value in reinforcement learning to update the parameters of the first dialogue model. That is, each round of dialogue can obtain the reward value corresponding to that round of dialogue, which increases the density of reward value generation, thereby improving the training efficiency of the first dialogue model obtained through reinforcement learning.

[0137] Figure 4 This is a flowchart illustrating a method for obtaining dialogue information according to an exemplary embodiment. The method can be executed by a computer device, which can be a server, wherein the server can be the one described above. Figure 1 Server 120 in the illustrated embodiment. For example... Figure 4As shown, the process of this dialogue information acquisition method may include the following steps:

[0138] Step 401: Obtain the second training set.

[0139] The second training set contains at least one set of dialogue information from that sample.

[0140] Optionally, the dialogue sample information in the second training set includes sample input information and sample output information. The sample input information is the information input by the user in a round of dialogue; the sample output information may be the feedback information corresponding to the sample input information.

[0141] Step 402: Update the parameters of the first generator based on at least one set of sample dialogue information in the second training set.

[0142] In one possible implementation, the first generator includes a first encoding network and a first decoding network.

[0143] The first encoding network is used to generate a corresponding feature vector based on the input sample dialogue information; the first decoding network is used to output the corresponding predicted dialogue information based on the feature vector.

[0144] In one possible implementation, both the first encoding network and the first decoding network are multi-layer neural networks.

[0145] In one possible implementation, at least one set of sample dialogue information in the second training set includes first sample input information; the first sample input information is input into the first encoding network in the first generator to obtain a first input feature value; based on the first input feature value, it is input into the first decoding network in the first generator to obtain first predicted input information; and the first generator is updated based on the first sample input information and the first predicted input information.

[0146] When at least one set of sample dialogue information in the second training set contains the first sample input information, the first sample input information can be input into the first encoding network to obtain the first input feature value. Then, the first decoding network decodes the first input feature value into the corresponding first predicted input information. The first generator is then updated based on the difference between the first predicted input information and the first sample input information. The first decoding network in the updated first generator can generate simulated input information by simulating feature values ​​similar to the first input feature value.

[0147] In one possible implementation, a first generation loss function value is obtained based on the first sample input information and the first prediction input information; the first generator is updated based on the first generation loss function value.

[0148] For example, when the first predicted input information corresponding to the first sample input information is obtained, the first generation loss function value can be determined by using a loss function (such as the cross-entropy loss function) based on the feature value of the first sample input information and the feature value of the first predicted input information, and the neural network parameters in the first generator can be updated by using a backpropagation algorithm or other methods.

[0149] In one possible implementation, at least one set of sample dialogue information in the second training set includes first sample output information; the first sample input information is input into the first encoding network in the first generator to obtain a first output feature value; based on the first output feature value, it is input into the first decoding network in the first generator to obtain first prediction output information; and the first generator is updated based on the first sample output information and the first prediction output information.

[0150] When at least one set of sample dialogue information in the second training set contains the first sample output information, the first sample output information can be input into the first encoding network to obtain the first output feature value. Then, the first decoding network decodes the first input feature value into the corresponding first predicted output information. The first generator is then updated based on the difference between the first predicted output information and the first sample output information. The first decoding network in the updated first generator can generate simulated output information by simulating feature values ​​similar to the first output feature value.

[0151] In one possible implementation, a second generation loss function value is obtained based on the first sample output information and the first prediction output information; the first generator is then updated based on the second generation loss function.

[0152] For example, when the first predicted output information corresponding to the first sample information is obtained, the second generation loss function value can be determined by the loss function based on the feature value of the first sample output information and the feature value of the first predicted output information, and the neural network parameters in the first generator can be updated by means of backpropagation algorithm, etc.

[0153] Figure 5 A schematic diagram of a first generator structure according to an embodiment of this application is shown. Figure 5As shown in section 501, the first generator includes a first encoding network and a first decoding network. When dialogue information is input into the first encoding network of the first generator, the first encoding network can encode the dialogue information to obtain the feature vector corresponding to the dialogue information. Then, the first decoding network in the first generator performs a decoding operation on the feature vector to obtain the predicted dialogue output by the first generator. At this time, the parameters of the first generator can be updated by comparing the predicted dialogue with the initial dialogue information.

[0154] After the first generator is updated, as follows Figure 5 As shown in section 502, a simulated vector of the same type as the feature vector corresponding to the dialogue information can be randomly generated, and the simulated vector is input into the first decoding network in the first generator to obtain the simulated dialogue corresponding to the simulated vector, thereby realizing the process of generating simulated dialogue.

[0155] In one possible implementation, the first generator could be a VAE (Variational Autoencoder) model. A VAE model is a generative model that incorporates latent variables. It utilizes a neural network trained to obtain two functions (also known as an inference network (i.e., the first encoding network) and a generator network (i.e., the first decoding network)) to generate data not present in the input data.

[0156] During VAE model training, the loss function is shown in the following formula.

[0157]

[0158] The loss function consists of two main parts: the first is the reassembly loss, which is the loss from reassembling from the decoder (first decoding network); the second is the KL (Kullback-Leibler Divergence, relative entropy) divergence loss. The VAE model is essentially a generative model; its input is source data, and the intermediate layers reparameterize it, causing it to project onto a normal Gaussian function. The purpose of KL divergence is to reduce the difference between the intermediate variables and the Gaussian function. The formula is as follows, where the intermediate variables are represented by... Let's represent it this way: the first half of the loss function is the common loss in reinforcement learning, represented by the log-likelihood of this reconstruction; the second half of the loss function is... The divergence loss consists of two main parts: the first part is the Gaussian distribution obtained through s resampling, and the second part is the normal Gaussian distribution.

[0159] The calculation formula is shown below. and These represent two different discrete distributions.

[0160]

[0161] Step 403: Obtain the first training set.

[0162] The first training set contains at least one set of dialogue information from that sample.

[0163] Optionally, at least one set of the sample dialogue information in the first training set includes second sample input information and second sample output information.

[0164] Optionally, the second sample input information and the second sample output information constitute one round of dialogue in a multi-round dialogue.

[0165] The second sample output information is used to indicate the annotation feedback information corresponding to the second sample input information in each sample output information. In at least one set of sample dialogue information in the first training set, the second sample output information can be used as the annotation information corresponding to the second sample input information.

[0166] Step 404: Obtain at least one set of simulated dialogue information generated by the first generator.

[0167] In one possible implementation, simulated feature values ​​are obtained; the simulated feature values ​​are then processed by a first decoding network of the first generator to obtain at least one set of simulated dialogue information.

[0168] The simulated feature value may be randomly generated and be of the same type as at least one of the first input feature value and the first output feature value.

[0169] Before generating at least one set of simulated dialogue information through the first generator, the first generator can be trained with at least one set of sample dialogue information from the second training set, so that the first decoding network in the trained first generator can generate simulated dialogue information corresponding to the simulated feature values ​​based on the input simulated feature values.

[0170] Optionally, the simulated dialogue information includes simulated input information and simulated output information.

[0171] In one possible implementation, a first simulated feature value of the same type as the first input feature value is randomly generated, and the first simulated input feature value is input into the first decoding network to obtain the first simulated input information; a first simulated output feature value of the same type as the first output feature value is randomly generated, and the first simulated output feature value is output into the first decoding network to obtain the first simulated output information.

[0172] Step 405: Based on at least one set of sample dialogue information and at least one set of simulated dialogue information in the first training set, first confidence data is obtained through the first discriminator.

[0173] The first confidence data includes at least one set of confidence scores corresponding to the sample dialogue information and at least one set of confidence scores corresponding to the simulated dialogue information.

[0174] In this embodiment, adversarial learning can be used to jointly train the first generator and the first discriminator. The first discriminator can be used as a classifier, and the goal of training it is to accurately distinguish between real and simulated input data. The first generator, acting as a generator of simulated data, is trained to generate simulated data that the discriminator cannot distinguish as real or simulated. Therefore, in this adversarial learning process—the adversarial process between the first discriminator and the first generator—the first discriminator can learn more accurately the feature information in real data, thereby improving its ability to distinguish between real and simulated data. Similarly, the first generator can learn more accurately the feature information in real data, thereby improving the similarity between the simulated data it generates and real data. Thus, during adversarial learning, both the generation capability of the first generator and the discrimination capability of the first discriminator are continuously improved.

[0175] During the training process of the first discriminator, at least one set of sample dialogue information in the first training set can be used as real data in the training process of the first discriminator. At this time, the at least one set of sample dialogue information in the first training set is input into the first discriminator to obtain the confidence level corresponding to the at least one set of sample dialogue information in the first training set. The confidence level corresponding to the at least one set of sample dialogue information in the first training set is used to indicate the probability that the at least one set of sample dialogue information is real data, as output by the first discriminator.

[0176] During the training process of the first discriminator, the simulated dialogue information generated by the first generator can be used as virtual data in the adversarial learning training process. At this time, at least one set of simulated dialogue information generated by the first generator is input into the first discriminator to obtain the confidence level corresponding to the at least one set of simulated dialogue information. The confidence level corresponding to the at least one set of simulated dialogue information is used to indicate the probability that the at least one set of simulated dialogue information output by the first discriminator is real data.

[0177] In one possible implementation, the simulated dialogue information generated by the first generator is subjected to natural language processing to obtain a first generated feature vector corresponding to the first generator; and at least one set of sample dialogue information in the first training set is subjected to natural language processing to obtain a first sample feature vector corresponding to the first training set; the first generated feature vector is input into the first discriminator to obtain the confidence level of at least one set of simulated dialogue information output by the first discriminator; the first sample feature vector is input into the first discriminator to obtain the confidence level of at least one set of simulated dialogue information in the first training set output by the first discriminator. That is, the input to the first discriminator can be vector data obtained after natural language processing of each dialogue information.

[0178] Figure 6 A schematic diagram of an adversarial learning model according to an embodiment of this application is shown. Figure 6 As shown, this model consists of two parts: a discriminator (601) and a generator (602) within a VAE model structure. From the discriminator's perspective, its primary task is to distinguish whether the data comes from real or fictitious samples. Real samples come from the state-action pairs in the dataset, while fictitious samples come from state-action combinations generated by the generator. In the real sample data, we need to process the state into continuous variables, as the previous discrete variables are difficult for the discriminator to learn. We use VAE to learn this process. The generator produces two parts: s_sim and a_sim. The first part mimics the distribution of the state, and the second part mimics the distribution of the action data. Correspondingly, s and a come from the real data, where a represents actions that are not processed, while s undergoes encoding operations by the VAE model. The generator and discriminator are both constructed using MLP networks, and the discriminator has only one numerical output. In the output layer of the discriminator network, there is a sigmoid layer to control its output between 0 and 1. The formula for Sigmoid is as follows, where x represents the variable input to the sigmoid layer.

[0179]

[0180] Step 406: Based on the first confidence data, update the parameters of the first discriminator.

[0181] The first confidence data includes at least one set of confidence scores corresponding to the sample dialogue information and at least one set of confidence scores corresponding to the simulated dialogue information. Therefore, the first discriminator can update itself based on the confidence scores corresponding to the at least one set of sample dialogue information, the annotations (e.g., 1) corresponding to the at least one set of sample dialogue information, and the annotations (e.g., 0) corresponding to the at least one set of simulated dialogue information.

[0182] For example, for a discriminator, the loss function is the GAN loss function. This loss function has two inputs, as shown in the formula below. The first part of the formula consists of spoofed inputs. The expected distribution corresponding to a false input. For cross-entropy loss, the second part consists of real inputs. The expected distribution corresponding to the true input. The cross-entropy loss corresponds to the actual input.

[0183]

[0184] Step 407: Update the parameters of the first generator based on the confidence scores corresponding to at least one set of simulated dialogue information.

[0185] Once the discriminator obtains the confidence level corresponding to the simulated dialogue information, the first generator can update its parameters based on the confidence level.

[0186] For example, when the generator is a VAE (Variable Object Evaluator), the VAE samples based on regular Gaussian noise and outputs a fake state-action, which is then given to the discriminator for scoring. The generator then uses this score for training. In the joint training of the generator and discriminator, we set a ratio of training rounds, for example, training the discriminator for ten rounds followed by one round of generator training. In practice, the generator needs to be trained more often because capturing the distribution of the original samples is quite difficult. The generator's loss function can be shown below.

[0187]

[0188] Among them, steps 403 to 407 can be executed repeatedly until the first discriminator and the first generator are trained to convergence.

[0189] In the adversarial learning process described above, it is important to note that the first discriminator and the first generator are updated sequentially during training. When updating the first generator, the parameters of the first discriminator can be kept unchanged until the first generator converges; similarly, when updating the first discriminator, the parameters of the first generator can be kept unchanged until the first discriminator converges. Training of the first discriminator and the first generator stops when the training reaches the termination condition (e.g., a certain number of training rounds, or the loss function value is less than a certain threshold).

[0190] Step 408: Obtain the first dialogue information.

[0191] In one possible implementation, the process of jointly training the first generator and the first discriminator through adversarial learning can be conducted offline. That is, before the first discriminator judges a round of dialogue generated by the first dialogue network, it can first be trained through adversarial learning using a first training set and a first generator pre-trained based on a second training set. The trained first discriminator is then used to process the first dialogue model. At this point, the trained first discriminator, having undergone offline adversarial learning, already has a certain ability to discriminate the authenticity of the dialogue, thus minimizing the possibility of model collapse that might occur during adversarial learning.

[0192] Step 409: Based on the first dialogue information, obtain the first feature vector corresponding to the first dialogue information.

[0193] In one possible implementation, natural language processing is performed on the first dialogue information to obtain the first feature vector corresponding to the first dialogue information.

[0194] In one possible implementation, a first probability distribution corresponding to the first feature vector is obtained; the first probability distribution is used to indicate the confidence level of the information corresponding to each vector value in the first feature vector.

[0195] When the first feature vector corresponding to the first dialogue information is obtained through natural language processing, each vector value in the first feature vector can correspond to each word in the first dialogue information. When generating the first feature vector, the first probability distribution corresponding to the first feature vector can also be obtained. Each probability value in the first probability distribution is used to indicate the confidence level of the information corresponding to each vector value in the first feature vector.

[0196] In one possible implementation, the probability values ​​corresponding to the vector value in the first probability distribution are used to indicate the importance of the vector value in the first feature vector.

[0197] Step 410: Using the first dialogue model, the first feature vector is processed to obtain the second feature vector.

[0198] In one possible implementation, based on the first probability distribution, the first feature vector is processed by the first dialogue model to obtain the second feature vector.

[0199] In one possible implementation, based on the first probability distribution and the first feature vector, a first hotspot vector value corresponding to the first feature vector is obtained, and the first dialogue model is used to process the first hotspot vector value to obtain a second feature vector.

[0200] The first hotspot vector value is the feature vector value corresponding to the largest probability value in the first probability distribution.

[0201] In each round of dialogue, each part can actually be represented by a feature vector, which can summarize the current dialogue information and make a reasonable embedded expression of the current dialogue. It is mainly represented by two parts. The first part is the belief states (i.e., the first probability distribution), which is mainly the current system's confidence prediction for each module, containing the probability values ​​corresponding to each slot value (i.e., each vector value). The second part is the one-hot encoding composed of system and user behaviors. Since the number of user and system behaviors is fixed, it is easy to construct this encoding form. We use states to represent this input feature vector.

[0202] Step 411: Based on the second feature vector, obtain the second dialogue information.

[0203] In one possible implementation, the first dialogue model contains pre-set dialogue information, and in response to the first dialogue model generating a second feature vector, the second dialogue information corresponding to the second feature vector is obtained from the dialogue information.

[0204] At this point, the first dialogue model selects the dialogue information with the highest matching degree from the pre-set dialogue information based on the input first dialogue information, and outputs it as the second dialogue information.

[0205] Step 412: The first discriminator processes the first feature vector and the second feature vector to obtain the first confidence score.

[0206] The second feature vector is generated by the first dialogue model based on the first feature vector. That is, the first feature vector is the feature vector corresponding to the first dialogue information, and the second feature vector is the feature vector corresponding to the second dialogue information. The first feature vector and the second feature vector are input into the first discriminator, and the first confidence score output by the first discriminator can be used to indicate the degree of matching between the first dialogue information and the second dialogue information.

[0207] Step 413: Update the parameters of the first dialogue model based on the first confidence score.

[0208] In one possible implementation, the first dialogue model could be an LSTM (Long Short-Term Memory) network model.

[0209] When training this LSTM model using reinforcement learning, the input dialogue information serves as the LSTM's state. The LSTM model generates output dialogue information as its actions based on the input dialogue information, and updates the LSTM's environment based on these actions. After updating the environment, the LSTM receives the state information again and generates subsequent actions. When the LSTM's dialogue reaches certain conditions, a reward function is triggered, providing feedback on the previously completed actions, and the LSTM is updated based on the reward given by the reward function.

[0210] In one possible implementation, the first confidence score is any one of the confidence scores in the set of confidence scores; based on each confidence score in the set of confidence scores, the value function value corresponding to the first dialogue model is obtained; based on the value function value corresponding to the first dialogue model, the first dialogue model is updated.

[0211] At this point, when the first dialogue model generates the second dialogue information based on the first dialogue information, it can input the first feature vector corresponding to the first dialogue information and the second feature vector corresponding to the second dialogue information into the first discriminator, obtain the confidence score generated by the first discriminator, and generate the reward function corresponding to the input of the first dialogue information based on the confidence score, thereby generating the reward value corresponding to the first dialogue information. At this point, each dialogue information input during the training process of the first dialogue model can generate a corresponding reward value, providing feedback to the first dialogue model on its behavior of outputting dialogue information based on the input dialogue information.

[0212] In one possible implementation, the value function value corresponding to the first dialogue model is obtained based on the dialogue order corresponding to each confidence score.

[0213] When the first confidence score is any one of the confidence scores in the set of confidence scores, that is, during the training process of the first dialogue model, the first discriminator has discriminated multiple sets of dialogue information and obtained the confidence scores corresponding to each set of dialogue information. When updating based on the confidence scores corresponding to the multiple sets of dialogue information, the discount rate of the reward value of each set of dialogue information can be determined according to the time sequence of the multiple sets of dialogue information. That is, the reward value of the dialogue information that is earlier in the time sequence is smaller, which makes it easier for the first dialogue model to take into account the influence of subsequent multiple rounds of dialogue and improves the model training effect in multi-round dialogue scenarios.

[0214] In one possible implementation, third dialogue information is obtained; based on the third dialogue information, data processing is performed through a first dialogue model to obtain fourth dialogue information output by the first dialogue model.

[0215] Once the first dialogue model is trained, the third dialogue information input by the user can be input into the first dialogue model to obtain the fourth dialogue information output by the first dialogue model and return it to the user. In this way, the dialogue information corresponding to the third dialogue information input by the user can be obtained through the first dialogue model.

[0216] By comparing the traditional approach with the approach proposed in this application, two metrics will be used to measure the maturity of the model, as shown in Table 1. The first metric is the final success rate, and the second metric is the number of data points required for the model to converge.

[0217] Table 1

[0218]

[0219] As can be seen from the experimental results in Table 1, the scheme shown in the embodiments of this application can make the model converge faster and improve the final effect of the model, which is very meaningful in practical application.

[0220] Success rate testing was conducted using a simulated user approach, which is both time- and resource-efficient. Furthermore, this training method is relatively effective. The testing platform used was convlab, developed by the Tsinghua University dialogue system team and the Amazon dialogue system team.

[0221] Currently, task-oriented dialogue is becoming increasingly popular, and it can be used in telephone customer service, mobile customer service, and mobile assistants to complete basic tasks such as booking flights and hotels, greatly reducing the use of human resources. In task-oriented dialogue, the industry mainly adopts a modular system, which divides the system into five common parts: input statement preprocessing, natural language understanding, dialogue state tracking, dialogue management, and dialogue response.

[0222] This application focuses on dialogue management. Mainstream models utilize reinforcement learning, but efficiently learning from data remains a challenge. Reinforcement learning primarily consists of three parts: environmental variables, a reward function, and the system's actions. However, traditional methods, with manually controlled reward functions, are highly inefficient when learning from large amounts of sample data.

[0223] However, reinforcement learning generally presents several training problems: 1. Slow convergence speed. 2. Local optima, not global optima, because reinforcement learning only makes one action based on the current state. 3. Vast exploration space; for dialogue systems, there are over three hundred possible actions, making the computational cost of exploration too high for reinforcement learning applications in dialogue systems, leading to slow convergence.

[0224] This application employs inverse reinforcement learning to learn the reward function of reinforcement learning, thereby enabling better and faster learning of dialogue management knowledge. While inverse reinforcement learning can be trained online through adversarial training, where the discriminator (reward estimator) engages in adversarial learning with the reinforcement learning algorithm, this method introduces adversarial learning into the reinforcement learning training, leading to the inherent training problem of GANs: model collapse. This solution uses an offline training method, pre-training the discriminator and then directly using this fixed discriminator for reward estimation, thus avoiding the model collapse problem.

[0225] The solution shown in the embodiments of this application can effectively solve some problems existing in dialogue models:

[0226] To address the issue of reward sparsity, this solution aims to design a reward model that can learn the correct reward mechanism from the data and then provide a specific reward in each round of dialogue.

[0227] Regarding the issue of learning efficiency, this solution aims to address the problem of low learning effectiveness in traditional reinforcement learning and can accelerate the learning process.

[0228] For multi-domain problems, traditional reward functions cannot adequately plan for the specific domain, necessitating the resolution of this domain dependency issue. A model designed to effectively consider the domain context when outputting corresponding rewards is highly effective.

[0229] In traditional reward functions, the lack of information contained in the reward leads to poor learning results. However, by introducing a better reward function, the final training results can be improved.

[0230] In summary, in the scheme shown in the embodiments of this application, the generator and discriminator are trained simultaneously through adversarial learning, enabling the discriminator to have a high ability to recognize the matching degree between a set of dialogue information. The trained discriminator can detect the matching degree between the first dialogue information input to the first dialogue model and the second dialogue information output by the first dialogue model, and use the matching degree between the first dialogue information and the second dialogue information as the reward value in reinforcement learning to update the parameters of the first dialogue model. That is, each round of dialogue can obtain the reward value corresponding to that round of dialogue, which increases the density of reward value generation and thus improves the training efficiency of the first dialogue model obtained through reinforcement learning.

[0231] Figure 7 This is a flowchart illustrating a dialogue information acquisition method according to an exemplary embodiment. The dialogue information acquisition method can be... Figure 1 The method for obtaining dialogue information, executed by server 120, includes the following steps.

[0232] like Figure 7 As shown in section 710, during the training of the first generator, dialogue information (i.e., sample dialogue information from the second training set) is input into the first encoding network of the first generator. After encoding by the first encoding network, a feature vector corresponding to the dialogue information is generated. The encoded feature vector is then input into the first decoding network to obtain the predicted dialogue output by the first decoding network (i.e., the predicted dialogue information corresponding to the sample dialogue information). At this point, the dialogue information and the predicted dialogue are input into the loss function to obtain the loss function value, and the first generator is trained.

[0233] like Figure 7As shown in section 720, after the first generator is trained, it can be used to generate simulated dialogues (i.e., simulated dialogue information). The simulated dialogues generated by the first generator and sample dialogues from the first training set (i.e., sample dialogue information) are then input into the first discriminator to obtain confidence data output by the first discriminator. This confidence data includes confidence scores for both the simulated dialogue information and the sample dialogue information. The first discriminator can then be updated based on these confidence scores; alternatively, the first generator can be updated based on these confidence scores. When the adversarial learning process meets specified conditions, the trained first discriminator is obtained, at which point it can determine the credibility of the input dialogue information.

[0234] like Figure 7 As shown in section 730, after the first discriminator is trained, it can be used as the reward function of the first dialogue model to obtain the reward value corresponding to each action of the first dialogue model. For example, when the first dialogue information is input as the state into the first dialogue model, the first dialogue model performs an action, i.e., generates the corresponding second dialogue information. At this time, the first dialogue information and the second dialogue information are input as a set of dialogue information into the first discriminator. The first discriminator determines the confidence score of the first dialogue information and the second dialogue information, i.e., the matching degree of the first dialogue information and the second dialogue information. The confidence score of the first dialogue information and the confidence score of the second dialogue information are used as the reward for the first dialogue model to generate the second dialogue information (action) to train the first dialogue model.

[0235] like Figure 7 As shown in section 740, once the first dialogue model is trained, the third dialogue information input by the user can be input into the first dialogue model so that the first dialogue model can process the third dialogue information and obtain the corresponding fourth dialogue information to return to the user.

[0236] Figure 8 This is a structural block diagram illustrating a dialogue information acquisition device according to an exemplary embodiment. This face image data updating device can realize the following: Figure 2 , Figure 3 and Figure 4 The method provided in any of the embodiments shown in the figure includes all or part of the steps, and the dialogue information acquisition device includes:

[0237] First dialogue acquisition module 801 is used to acquire first dialogue information;

[0238] The second dialogue acquisition module 802 is used to obtain second dialogue information by performing data processing through the first dialogue model based on the first dialogue information; the first dialogue information and the second dialogue information constitute a round of dialogue.

[0239] The confidence score acquisition module 803 is used to obtain a first confidence score by performing data processing through a first discriminator based on the first dialogue information and the second dialogue information; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first discriminator is obtained by training it through adversarial learning based on at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator.

[0240] The model parameter update module 804 is used to update the parameters of the first dialogue model based on the first confidence score.

[0241] The trained first dialogue model is used to obtain fourth dialogue information based on the input third dialogue information.

[0242] In one possible implementation, the device further includes:

[0243] The first vector acquisition module is used to obtain a first feature vector corresponding to the first dialogue information based on the first dialogue information.

[0244] The second dialogue acquisition module 802 includes:

[0245] The first vector processing unit is used to process the first feature vector through the first dialogue model to obtain the second feature vector;

[0246] The second dialogue acquisition unit is used to obtain the second dialogue information based on the second feature vector;

[0247] The confidence score acquisition module 803 is also used for,

[0248] The first discriminator processes the first feature vector and the second feature vector to obtain the first confidence score.

[0249] In one possible implementation, the device further includes:

[0250] The probability distribution acquisition module is used to acquire a first probability distribution corresponding to the first feature vector based on the first dialogue information; the first probability distribution is used to indicate the confidence level of the information corresponding to each vector value in the first feature vector.

[0251] The first vector processing unit is further configured to,

[0252] Based on the first probability distribution, the first feature vector is processed by the first dialogue model to obtain the second feature vector.

[0253] In one possible implementation, the model parameter update module 804 includes:

[0254] The value function value acquisition unit is used to acquire the value function value corresponding to the first dialogue model based on the first confidence score.

[0255] The model parameter update unit is used to update the parameters of the first dialogue model based on the value function value corresponding to the first dialogue model.

[0256] In one possible implementation, the first confidence score is any one of the confidence scores in the set of confidence scores;

[0257] The value function value acquisition unit is also used for,

[0258] Based on each confidence score in the set of confidence scores, the value function value corresponding to the first dialogue model is obtained.

[0259] In one possible implementation, the device further includes:

[0260] The first training set acquisition module is used to acquire the first training set; the first training set contains at least one set of sample dialogue information.

[0261] A simulated dialogue acquisition module is used to acquire at least one set of simulated dialogue information generated by the first generator;

[0262] The confidence data acquisition module is used to acquire first confidence data through a first discriminator based on the at least one set of sample dialogue information and the at least one set of simulated dialogue information; the first confidence data includes the confidence level corresponding to the at least one set of sample dialogue information and the confidence level corresponding to the at least one set of simulated dialogue information.

[0263] The discriminator update module is used to update the parameters of the first discriminator based on the first confidence data;

[0264] The generator update module is used to update the parameters of the first generator based on the confidence levels corresponding to at least one set of simulated dialogue information.

[0265] In one possible implementation, the device further includes:

[0266] The second training set acquisition module is used to acquire a second training set; the second training set contains at least one set of sample dialogue information.

[0267] The first generator training module is used to train the first generator based on at least one set of sample dialogue information in the second training set.

[0268] In one possible implementation, at least one set of the sample dialogue information in the second training set includes the first sample input information;

[0269] The first generator training module is also used for,

[0270] The first sample input information is input into the first encoding network in the first generator to obtain the first input feature value;

[0271] Based on the first input feature value, the first decoding network in the first generator is input to obtain the first predicted input information;

[0272] The first generator is updated based on the first sample input information and the first prediction input information.

[0273] In one possible implementation, at least one set of the sample dialogue information in the second training set includes the first sample output information;

[0274] The first generator training module is also used for,

[0275] The first sample output information is input into the first encoding network in the first generator to obtain the first output feature value;

[0276] Based on the first output feature value, input to the first decoding network in the first generator to obtain the first predicted output information;

[0277] The first generator is updated based on the first sample output information and the first prediction output information.

[0278] In summary, in the scheme shown in the embodiments of this application, the generator and discriminator are trained simultaneously through adversarial learning, enabling the discriminator to have a high ability to recognize the matching degree between a set of dialogue information. The trained discriminator can detect the matching degree between the first dialogue information input to the first dialogue model and the second dialogue information output by the first dialogue model, and use the matching degree between the first dialogue information and the second dialogue information as the reward value in reinforcement learning to update the parameters of the first dialogue model. That is, each round of dialogue can obtain the reward value corresponding to that round of dialogue, which increases the density of reward value generation and thus improves the training efficiency of the first dialogue model obtained through reinforcement learning.

[0279] Figure 9This is a structural block diagram illustrating a dialogue information acquisition device according to an exemplary embodiment. This face image data updating device can realize the following: Figure 2 , Figure 3 and Figure 4 The method provided in any of the embodiments shown in the figure includes all or part of the steps, and the dialogue information acquisition device includes:

[0280] The third dialogue acquisition module 901 is used to acquire third dialogue information;

[0281] The fourth dialogue acquisition module 902 is used to process data through the first dialogue model based on the third dialogue information to obtain the fourth dialogue information output by the first dialogue model.

[0282] Wherein, the first dialogue model is obtained by updating parameters based on a first confidence score; the first confidence score is obtained by data processing through a first discriminator based on first dialogue information and second dialogue information; the first dialogue information and the second dialogue information constitute a round of dialogue; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first discriminator is obtained by training at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator through adversarial learning.

[0283] In summary, in the scheme shown in the embodiments of this application, the generator and discriminator are trained simultaneously through adversarial learning, enabling the discriminator to have a high ability to recognize the matching degree between a set of dialogue information. The trained discriminator can detect the matching degree between the first dialogue information input to the first dialogue model and the second dialogue information output by the first dialogue model, and use the matching degree between the first dialogue information and the second dialogue information as the reward value in reinforcement learning to update the parameters of the first dialogue model. That is, each round of dialogue can obtain the reward value corresponding to that round of dialogue, which increases the density of reward value generation and thus improves the training efficiency of the first dialogue model obtained through reinforcement learning.

[0284] Figure 10This is a schematic diagram illustrating the structure of a computer device according to an exemplary embodiment. The computer device can be implemented as a model training device and / or signal processing device in the various method embodiments described above. The computer device 1000 includes a central processing unit (CPU) 1001, a system memory 1004 including random access memory (RAM) 1002 and read-only memory (ROM) 1003, and a system bus 1005 connecting the system memory 1004 and the central processing unit 1001. The computer device 1000 also includes a basic input / output system 1006 to facilitate information transfer between various devices within the computer, and a mass storage device 1007 for storing the operating system 1013, application programs 1014, and other program modules 1015.

[0285] The mass storage device 1007 is connected to the central processing unit 1001 via a mass storage controller (not shown) connected to the system bus 1005. The mass storage device 1007 and its associated computer-readable media provide non-volatile storage for the computer device 1000. That is, the mass storage device 1007 may include computer-readable media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0286] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, flash memory or other solid-state storage technologies, CD-ROM, or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1004 and the mass storage device 1007 described above can be collectively referred to as memory.

[0287] Computer device 1000 can be connected to the Internet or other network devices via network interface unit 1011 connected to the system bus 1005.

[0288] The memory also includes one or more programs, which are stored in the memory, and the central processing unit 1001 implements these programs by executing them. Figure 2 , Figure 3 or Figure 4 All or part of the steps of any of the methods shown.

[0289] In exemplary embodiments, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including a computer program (instructions) that can be executed by a processor of a computer device to perform the methods shown in the various embodiments of this application. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0290] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods shown in the various embodiments described above.

[0291] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0292] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for acquiring dialogue information, characterized in that, The method includes: Obtain the first dialogue information; Based on the first dialogue information, obtain the first feature vector corresponding to the first dialogue information; Based on the first dialogue information, a first probability distribution corresponding to the first feature vector is obtained; the first probability distribution is used to indicate the confidence level of the information corresponding to each vector value in the first feature vector. Based on the first probability distribution, the first feature vector is processed by the first dialogue model to obtain the second feature vector; based on the second feature vector, the second dialogue information is obtained; the first dialogue information and the second dialogue information constitute a round of dialogue. The first discriminator processes the first feature vector and the second feature vector to obtain a first confidence score; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first discriminator is trained by adversarial learning based on at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator. Based on the first confidence score, the parameters of the first dialogue model are updated; The trained first dialogue model is used to obtain fourth dialogue information based on the input third dialogue information.

2. The method according to claim 1, characterized in that, The first confidence score is any one of the confidence scores in the set of confidence scores; the step of updating the parameters of the first dialogue model based on the first confidence score includes: Based on each confidence score in the set of confidence scores, obtain the value function value corresponding to the first dialogue model; Based on the value function value corresponding to the first dialogue model, the parameters of the first dialogue model are updated.

3. The method according to claim 2, characterized in that, The step of obtaining the value function value corresponding to the first dialogue model based on each confidence score in the confidence score set includes: Based on the dialogue order corresponding to each confidence score, the value function value corresponding to the first dialogue model is obtained.

4. The method according to claim 1, characterized in that, The method further includes: Obtain the first training set; the first training set contains at least one set of sample dialogue information; Obtain at least one set of the simulated dialogue information generated by the first generator; Based on the at least one set of sample dialogue information and the at least one set of simulated dialogue information, a first confidence data is obtained through a first discriminator; the first confidence data includes the confidence level corresponding to the at least one set of sample dialogue information and the confidence level corresponding to the at least one set of simulated dialogue information. Based on the first confidence data, the parameters of the first discriminator are updated; The parameters of the first generator are updated based on the confidence levels corresponding to at least one set of simulated dialogue information.

5. The method according to claim 4, characterized in that, Before obtaining at least one set of simulated dialogue information generated by the first generator, the method further includes: Obtain a second training set; the second training set contains at least one set of the sample dialogue information. The first generator is trained based on at least one set of sample dialogue information from the second training set.

6. The method according to claim 5, characterized in that, At least one set of the sample dialogue information in the second training set contains the first sample input information; The first generator is trained based on at least one set of sample dialogue information from the second training set, including: The first sample input information is input into the first encoding network in the first generator to obtain the first input feature value; Based on the first input feature value, the first decoding network in the first generator is input to obtain the first predicted input information; The first generator is updated based on the first sample input information and the first prediction input information.

7. The method according to claim 5, characterized in that, At least one set of the sample dialogue information in the second training set includes the first sample output information; The first generator is trained based on at least one set of sample dialogue information from the second training set, including: The first sample output information is input into the first encoding network in the first generator to obtain the first output feature value; Based on the first output feature value, input to the first decoding network in the first generator to obtain the first predicted output information; The first generator is updated based on the first sample output information and the first prediction output information.

8. A method for acquiring dialogue information, characterized in that, The method includes: Obtain third-party dialogue information; Based on the third dialogue information, data processing is performed through the first dialogue model to obtain the fourth dialogue information output by the first dialogue model. Wherein, the first dialogue model is obtained by updating parameters based on a first confidence score; the first confidence score is obtained by processing the first feature vector and the second feature vector through a first discriminator; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first dialogue information and the second dialogue information constitute a round of dialogue; the first feature vector is obtained based on the first dialogue information; the second feature vector is obtained by processing the first feature vector through the first dialogue model based on the first probability distribution corresponding to the first feature vector; the first probability distribution is obtained based on the first dialogue information; the first probability distribution is used to indicate the confidence level of the information corresponding to each vector value in the first feature vector; the second dialogue information is obtained based on the second feature vector; the first discriminator is obtained by training at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator through adversarial learning.

9. A dialogue information acquisition device, characterized in that, The device includes: The first dialogue acquisition module is used to acquire first dialogue information; The first vector acquisition module is used to obtain a first feature vector corresponding to the first dialogue information based on the first dialogue information. The probability distribution acquisition module is used to acquire a first probability distribution corresponding to the first feature vector based on the first dialogue information; the first probability distribution is used to indicate the confidence level of the information corresponding to each vector value in the first feature vector. The second dialogue acquisition module is used to process the first feature vector based on the first probability distribution using the first dialogue model to obtain a second feature vector; and to obtain second dialogue information based on the second feature vector; the first dialogue information and the second dialogue information constitute a round of dialogue. The confidence score acquisition module is used to process the first feature vector and the second feature vector through a first discriminator to obtain a first confidence score; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first discriminator is obtained by training on at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator through adversarial learning. The model parameter update module is used to update the parameters of the first dialogue model based on the first confidence score; The trained first dialogue model is used to obtain fourth dialogue information based on the input third dialogue information.

10. A dialogue information acquisition device, characterized in that, The device includes: The third dialogue acquisition module is used to acquire third dialogue information. The fourth dialogue acquisition module is used to process data through the first dialogue model based on the third dialogue information to obtain the fourth dialogue information output by the first dialogue model. Wherein, the first dialogue model is obtained by updating parameters based on a first confidence score; the first confidence score is obtained by processing the first feature vector and the second feature vector through a first discriminator; the first confidence score is used to indicate the degree of matching between the second dialogue information and the first dialogue information; the first dialogue information and the second dialogue information constitute a round of dialogue; the first feature vector is obtained based on the first dialogue information; the second feature vector is obtained by processing the first feature vector through the first dialogue model based on the first probability distribution corresponding to the first feature vector; the first probability distribution is obtained based on the first dialogue information; the first probability distribution is used to indicate the confidence level of the information corresponding to each vector value in the first feature vector; the second dialogue information is obtained based on the second feature vector; the first discriminator is obtained by training at least one set of sample dialogue information in the first training set and at least one set of simulated dialogue information generated by the first generator through adversarial learning.

11. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the dialogue information acquisition method as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the dialogue information acquisition method as described in any one of claims 1 to 8.

13. A computer program product, characterized in that, The computer program product includes computer instructions that are executed by a processor of a computer device to implement the dialogue information acquisition method as described in any one of claims 1 to 8.

Images