Method, device and storage medium for data processing

Abstract

A data processing method, device and storage medium. One of the data processing methods includes receiving an input utterance from a device in conversation with a dialogue model, obtaining a plurality of candidate replies to the input utterance from the dialogue model, determining a plurality of discriminator scores of the candidate replies based on a reference-free discriminator, determining a plurality of quality scores associated with the candidate replies, and training the dialogue model based on the quality scores.

Description

Technical Field

[0001] This disclosure relates to generating dialogue models, and more specifically to training neural network-based dialogue models based on a no-reference discriminator. Background Technology

[0002] Dialogue response generation aims to generate interesting and coherent responses given a dialogue history. It has attracted increasing research interest, primarily due to growing commercial demand. Increasingly powerful neural models have been proposed, where token-level cross-entropy loss has been used to maximize the likelihood of human-annotated responses for each dialogue context. However, key errors such as generating meaningless or illogical responses are frequently observed in dialogue response generation.

[0003] Initial efforts to address this problem involved leveraging external knowledge to acquire richer features. However, these approaches assume a single, optimal response (often manually selected) for a given context, while various valid responses with different meanings can exist for each dialogue context, thus ignoring any potential error propagation. Consequently, they may face challenges from erroneous knowledge retrieval. Recently, large-scale training has been proposed as a solution, based on the assumption that these problems can be significantly alleviated when the model has seen enough (potentially billions) of instances. However, this solution is challenged by the increasingly rapid evolution of language, as it constantly introduces new topics, words, and slang. Furthermore, large-scale training is time-consuming and resource-intensive, and therefore may not be a sustainable approach. Summary of the Invention

[0004] According to some possible implementations, a method (preferably generating an open-domain dialogue model based on a neural network) may include receiving input utterances from a device engaging in a conversation with the dialogue model (wherein the device receiving the input utterances is a first device, and the dialogue model is run / executed on a second device during conversational communication with the first device); obtaining multiple candidate responses to the input utterances from the dialogue model (i.e., from the second device running the dialogue model), the multiple candidate responses including a first candidate response and a second candidate response; determining multiple discriminator scores for the first candidate response, the multiple discriminator scores being provided by multiple discriminators that evaluate the first candidate response based on information corresponding to the first candidate response and contextual information related to the history of the conversation. The process involves: assessing the quality of a first candidate response; determining multiple discriminator scores for a second candidate response, the multiple discriminator scores being provided by multiple discriminators that evaluate the quality of the second candidate response based on information corresponding to the second candidate response and contextual information corresponding to the history of the conversation; determining a first quality score associated with the first candidate response, the first quality score being based on a weighted sum of the multiple discriminator scores corresponding to the first candidate response; determining a second quality score associated with the second candidate response, the second quality score being based on a weighted sum of the multiple discriminator scores corresponding to the second candidate response; and training the dialogue model based on at least one of the first quality score and the second quality score.

[0005] According to some possible implementations, the device may include at least one memory configured to store program code and a neural network-based open-domain dialogue model; and at least one processor configured to read the program code and operate according to the instructions of the program code to execute the method according to this application. A device is also provided, comprising: a receiving unit for receiving input utterances from a device engaging in conversation with a dialogue model; an obtaining unit for obtaining a plurality of candidate responses to the input utterances from the dialogue model, the plurality of candidate responses including a first candidate response and a second candidate response; a first determining unit for determining a plurality of discriminator scores for the first candidate response, the plurality of discriminator scores being provided by a plurality of discriminators, the plurality of discriminators evaluating the quality of the first candidate response based on information corresponding to the first candidate response and contextual information corresponding to the history of the conversation; a second determining unit for determining a plurality of discriminator scores for the second candidate response, the plurality of discriminator scores being provided by the plurality of discriminators, the plurality of discriminators evaluating the quality of the second candidate response based on information corresponding to the second candidate response and contextual information corresponding to the history of the conversation; a third determining unit for determining a first quality score associated with the first candidate response, the first quality score being based on a weighted sum of the plurality of discriminator scores corresponding to the first candidate response; a fourth determining unit for determining a second quality score associated with the second candidate response, the second quality score being based on a weighted sum of the plurality of discriminator scores corresponding to the second candidate response; and a training unit for training the dialogue model based on at least one of the first quality score and the second quality score.

[0006] According to some possible implementations, a non-transitory computer-readable medium stores instructions, the instructions including: one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to perform a method according to an embodiment of this application. Attached Figure Description

[0007] The above and other aspects, features and aspects of embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0008] Figure 1 This is an overview diagram of the example implementation described in this article;

[0009] Figure 2 This is a schematic diagram of the environment in which the systems and / or methods described in this paper can be implemented;

[0010] Figure 3 yes Figure 2 Example component diagrams of one or more devices;

[0011] Figure 4This is a flowchart illustrating an example process for generating a neural network-based open-domain dialogue model. Detailed Implementation

[0012] The following detailed description of the example embodiments is with reference to the accompanying drawings. The same reference numerals in different drawings may identify the same or similar elements.

[0013] The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit implementations to the precise forms disclosed. Modifications and variations are possible based on the foregoing disclosure, or can be derived from the practice of implementation.

[0014] Clearly, the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not a limitation on the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code—it should be understood that software and hardware can be designed to implement the systems and / or methods based on the descriptions herein.

[0015] Although specific combinations of features are stated in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not specifically stated in the claims and / or disclosed in the specification. While each dependent claim listed below may be directly subordinated to only one claim, the disclosure of possible implementations includes every dependent claim in combination with each other claim in the group of claims.

[0016] Elements, actions, or instructions used herein should not be construed as critical or essential unless explicitly stated otherwise. Furthermore, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” The term “one” or similar language is used when referring to only one item. Additionally, as used herein, the terms “has,” “have,” “having,” “include,” “including,” etc., are intended to be open-ended terms. Furthermore, the phrase “based on” is intended to mean “at least partially based on” unless explicitly stated otherwise.

[0017] Figure 1 This is an overview diagram of an embodiment of a method for training a neural network-based dialogue model. Method 100 can be executed on a platform such as a server, desktop computer, handheld computing device, or a combination thereof.

[0018] like Figure 1 As shown, at operation 102, the platform can obtain utterances from a conversation. The utterances can be text strings, such as "Do you want to play a game?". The utterances can be received from external devices communicating with the platform.

[0019] At operation 104, based on the obtained utterances, the dialogue model can generate a series of candidate responses to the obtained utterances. For example... Figure 1 As shown, first to Nth candidate responses can be generated. According to an embodiment, the dialogue model can use a beam search algorithm.

[0020] At operation 106, the platform can determine the first to Nth no-reference discriminator scores. Each no-reference discriminator score can correspond to a different no-reference discriminator. No-reference discriminators can calculate their corresponding no-reference discriminator scores based on inputs of candidate responses and conversation history. According to an embodiment, the no-reference discriminators can consider only candidate responses and conversation history, without considering any external knowledge.

[0021] According to an embodiment, each of the discriminators can be based on supplementary factors. These supplementary factors may include the specificity of the candidate responses, the consistency of the candidate responses, the fluency of the candidate responses, and the relevance of the candidate responses. For example, refer to... Figure 1 The first no-reference discriminator score can indicate the specificity of the candidate response, the second no-reference discriminator can indicate the consistency of the response with the previous response, and the additional no-reference discriminator can take into account other supplementary factors or other aspects of the supplementary factors that have already been considered.

[0022] Figure 1 The no-reference discriminator score is shown only for the first candidate response. However, a no-reference discriminator can be obtained for each of the first to Nth candidate responses.

[0023] At operation 108, a weighted sum of the first candidate responses can be generated based on each of the no-reference discriminator scores corresponding to the first candidate responses. For example, a weight can be given to each no-reference discriminator score corresponding to the first candidate response based on empirical data, and then each of the weighted no-reference discriminator scores can be summed to obtain the weighted sum of the candidate responses. According to an embodiment, a weighted sum can be generated for each candidate response generated at operation 104.

[0024] At operation 110, the dialogue model can be trained based on a weighted sum corresponding to the first candidate response. For example, the training algorithm can be based on a loss function that incorporates the weighted sum of the first candidate responses. According to an embodiment, the dialogue model can be trained based on a weighted sum corresponding to multiple candidate responses generated at operation 104.

[0025] Thus, some implementations in this paper provide dialogue models that generate more meaningful, logical, and relevant responses based on less resource-intensive training.

[0026] Figure 2 This is a schematic diagram of an environment 200 in which the systems and / or methods described herein can be implemented. (See diagram for example.) Figure 2 As shown, environment 200 may include user equipment 210, platform 220, and network 230. Devices in environment 200 can be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0027] User equipment 210 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information related to platform 220. For example, user equipment 210 may include computing devices (e.g., desktop computers, laptop computers, tablet computers, handheld computers, smart speakers, servers, etc.), mobile phones (e.g., smartphones, cordless phones, etc.), wearable devices (e.g., smart glasses or smartwatches), or similar devices. In some embodiments, user equipment 210 may receive information from and / or send information to platform 220.

[0028] Platform 220 includes one or more devices capable of training dialogue models using a no-reference discriminator, as described elsewhere herein. In some embodiments, platform 220 may include a cloud server or a group of cloud servers. In some embodiments, platform 220 may be designed to be modular, allowing software components to be swapped in or out as needed. This allows platform 220 to be easily and / or quickly reconfigured for different purposes.

[0029] In some implementations, as shown in the figures, platform 220 may be hosted in a cloud computing environment 222. It is worth noting that while the implementations described herein describe platform 220 as hosted in a cloud computing environment 222, in some implementations, platform 220 may not be cloud-based (i.e., may be implemented outside of a cloud computing environment) or may be partially cloud-based.

[0030] The cloud computing environment 222 includes the environment of the hosting platform 220. The cloud computing environment 222 can provide services such as computing, software, data access, and storage, without requiring end users (e.g., user equipment 210) to know the physical location and configuration of the systems and / or devices of the hosting platform 220. As shown in the figure, the cloud computing environment 222 may include a set of computing resources 224 (collectively referred to as "computing resources 224" and individually as "computing resource 224").

[0031] Computing resource 224 includes one or more personal computers, workstations, server devices, or other types of computing and / or communication devices. In some embodiments, computing resource 224 may host platform 220. Cloud resources may include computing instances executed in computing resource 224, storage devices provided in computing resource 224, data transmission devices provided by computing resource 224, etc. In some embodiments, computing resource 224 may communicate with other computing resources 224 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0032] Further as Figure 2 As shown, computing resources 224 include a set of cloud resources, such as one or more applications (“APP”) 224-1, one or more virtual machines (“VM”) 224-2, virtualized storage (“VS”) 224-3, one or more hypervisors (“HYP”) 224-4, etc.

[0033] Application 224-1 includes one or more software applications that may be provided to, or accessed by, user device 210 and / or platform 220. Application 224-1 does not require the installation and execution of any software applications on user device 210. For example, application 224-1 may include software associated with platform 220, and / or any other software available through cloud computing environment 222. In some implementations, an application 224-1 may send / receive information to or from one or more other applications 224-1 via virtual machine 224-2.

[0034] Virtual machine 224-2 includes a software implementation of a machine (e.g., a computer) that executes programs, similar to a physical machine. Virtual machine 224-2 can be a system virtual machine or a process virtual machine, depending on the extent to which virtual machine 224-2 uses and corresponds to any real machine. A system virtual machine can provide a complete system platform that supports the execution of a complete operating system (“OS”). A process virtual machine can execute a single program and can support a single process. In some implementations, virtual machine 224-2 can execute on behalf of a user (e.g., user device 210) and can manage the infrastructure of the cloud computing environment 222, such as data management, synchronization, or long-term data transfer.

[0035] Virtualized storage 224-3 includes one or more storage systems and / or one or more devices that utilize virtualization technology within the storage systems or devices of computing resource 224. In some implementations, the type of virtualization within the context of the storage system may include block virtualization and file virtualization. Block virtualization may refer to the abstraction (or separation) of logical storage from physical storage so that the storage system can be accessed without regard to physical storage or heterogeneous architecture. Separation allows storage system administrators the flexibility to manage end-user storage. File virtualization can eliminate the dependency between data accessed at the file level and the location of physical storage files. This can optimize storage usage, server consolidation, and / or performance for non-disruptive file migration.

[0036] Hypervisor 224-4 provides hardware virtualization technology that allows multiple operating systems (e.g., "guest operating systems") to run simultaneously on a host computer such as computing resource 224. Hypervisor 224-4 can provide a virtual operating platform to the guest operating system and manage the execution of the guest operating system. Multiple instances of various operating systems can share virtualized hardware resources.

[0037] Network 230 includes one or more wired and / or wireless networks. For example, network 230 may include cellular networks (e.g., fifth-generation (5G) networks, Long-Term Evolution (LTE) networks, third-generation (3G) networks, Code Division Multiple Access (CDMA) networks, etc.), Public Land Mobile Networks (PLMNs), Local Area Networks (LANs), Wide Area Networks (WANs), Metropolitan Area Networks (MANs), telephone networks (e.g., Public Switched Telephone Networks (PSTNs)), private networks, self-organizing networks, intranets, the Internet, fiber-optic networks, etc., and / or combinations of these or other types of networks.

[0038] Figure 2 The number and arrangement of devices and networks shown are provided as an example. In reality, with... Figure 2 Compared to the devices and / or networks shown, there can be more devices and / or networks, fewer devices and / or networks, different devices and / or networks, or devices and / or networks arranged differently. Furthermore, Figure 2The two or more devices shown can be implemented within a single device, or Figure 2 The single device shown can be implemented as multiple distributed devices. Alternatively, a group of devices in environment 200 (e.g., one or more devices) can perform one or more functions described as being performed by another group of devices in environment 200.

[0039] Figure 3 This is an example component diagram of device 300. Device 300 may correspond to user device 210 and / or platform 220. (Example...) Figure 3 As shown, device 300 may include bus 310, processor 320, memory 330, storage component 340, input component 350, output component 360 and communication interface 370.

[0040] Bus 310 includes components that allow communication between components of device 300. Processor 320 is implemented in hardware, firmware, or a combination of hardware and software. Processor 320 is a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), microprocessor, microcontroller, digital signal processor (DSP), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or another type of processing component. In some embodiments, processor 320 includes one or more processors that can be programmed to perform functions. Memory 330 includes random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic storage, and / or optical storage) that stores information and / or instructions for use by processor 320.

[0041] Storage component 340 stores information and / or software related to the operation and use of device 300. For example, storage component 340 may include hard disks (e.g., magnetic disks, optical disks, magneto-optical disks, and / or solid-state disks), optical disks (CDs), digital versatile disks (DVDs), floppy disks, cassette tapes, magnetic tapes, and / or other types of non-volatile computer-readable media, and corresponding drives.

[0042] Input component 350 includes components that allow device 300 to receive information, such as a touchscreen display, keyboard, keypad, mouse, buttons, switches, and / or microphone. Alternatively, input component 350 may include sensors for sensing information (e.g., a Global Positioning System (GPS) component, accelerometer, gyroscope, and / or actuator). Output component 360 includes components that provide output information from device 300, such as a display, speaker, and / or one or more light-emitting diodes (LEDs).

[0043] The communication interface 370 includes transceiver-like components (e.g., a transceiver and / or separate receiver and transmitter) that enable the device 300 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface 370 may allow the device 300 to receive information from and / or provide information to another device. For example, the communication interface 370 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0044] Device 300 can perform one or more processes described herein. Device 300 can perform these processes in response to processor 320 executing software instructions stored in a non-volatile computer-readable medium (e.g., memory 330 and / or storage component 340). Computer-readable medium is defined herein as a non-volatile memory device. A memory device includes storage space within a single physical storage device or storage space distributed across multiple physical storage devices.

[0045] Software instructions can be read into memory 330 and / or storage component 340 from another computer-readable medium or another device via communication interface 370. When executed, the software instructions stored in memory 330 and / or storage component 340 can cause processor 320 to perform one or more processes described herein. Alternatively or additionally, hardware wiring circuitry may be used in place of or in combination with the software instructions to perform one or more processes described herein. Therefore, the embodiments described herein are not limited to any particular combination of hardware circuitry and software.

[0046] Figure 3 The number and arrangement of components shown are provided as an example. In fact, with... Figure 3 Compared to the components shown, device 300 may include more components, fewer components, different components, or components arranged differently. Alternatively, a set of components of device 300 (e.g., one or more components) may perform one or more functions described as being performed by another set of components of device 300.

[0047] Figure 4 This is a flowchart of example method 400 for generating dialogue models based on a no-reference discriminator. In some implementations, Figure 4 One or more processing boxes can be executed by platform 220. In some implementations, Figure 4 One or more process frames can be executed by another device or a group of devices (such as user equipment 210) that is separate from or includes platform 220.

[0048] like Figure 4As shown, method 400 may include receiving input utterances (box 410). The input utterances may be in the form of a text string.

[0049] For example, platform 220 can receive the input utterance “Do you want to play the game?” from a user or training set. The input utterance can be part of a dialogue or conversation.

[0050] like Figure 4 As further shown in the figure, method 400 may include determining a discriminator score for each of the candidate responses (box 420).

[0051] Platform 220 can determine the discriminator score based on a discriminator that quantifies complementary factors of good responses. Good responses can be meaningful, logical, and relevant. Discriminators can be non-referenced, meaning they do not consider external knowledge but rather candidate responses and contextual information from the current conversation or dialogue history.

[0052] Complementary factors for good responses can be selected to minimize overlap, which can minimize computational waste. For example, factors could include the specificity of the candidate response, consistency with previous output responses, fluency of the candidate response, and relevance of the candidate response. The discriminator can correspond to one of the supplementary factors for good responses.

[0053] According to the embodiment, the discriminator spec u It can be used to calculate the specificity score of candidate response u, as the average of word normalized inverse document frequency (NIDF) values.

[0054] The NIDF value of word w can be based on the inverse document frequency (IDF) of word w. The IDF of word w is defined as IDF(w) = log(|| / | w |), where |U| is the number of candidate responses in the generated candidate response set, and | w | represents the number of candidate responses containing w.

[0055] The NDIF can be determined based on the following equation 1:

[0056]

[0057] In Equation 1, min_idf represents the minimum IDF value of the candidate responses, and max_idf represents the maximum IDF value among the candidate responses.

[0058] The discriminator spec can be determined using the following Equation 2. u :

[0059]

[0060] The inclusion of the independent variable c (contextual information) is for consistency with other discriminators, but in determining the spec u Time is not considered. That is, spec u (u,c) considers candidate responses without considering contextual information (such as conversation or dialogue history).

[0061] According to the embodiment, the discriminator spec c This can be used to calculate the specificity score of candidate response u by considering contextual information c. The discriminator spec can be determined using the following Equation 3. c :

[0062]

[0063] As shown in Equation 3, spec c Only the word w, which appears in both the candidate response u and the context information c, is considered.

[0064] Discriminator spec u and spec c This can encourage dialogue models to generate detailed and engaging responses. For example, in response to the utterance "How about going bowling?", the discriminator spec... u and spec c It can prevent the dialogue model from using ordinary and meaningless candidate responses, such as "I don't know".

[0065] According to an embodiment, the discriminator `cons` can evaluate whether a candidate response contradicts a previous response output by the dialogue model during the conversation. Thus, the discriminator `cons` is based on a consistency complement factor.

[0066] The discriminator `cons` can be pre-trained using deep bidirectional transforms for language understanding. For example, the discriminator `cons` can be based on optimized BERT pre-training methods, such as the RoBRETa model. The RoBRETa model can be fine-tuned on datasets such as the Multigene Natural Language Inference (MNLI) dataset. The MNLI dataset considers the following three possible relations between each sentence pair: contradictory, necessary, and neutral.

[0067] For example, if a person in a conversation states "I love dogs," then that person cannot state "I am afraid of dogs and I usually stay away from them." Therefore, the discriminator const considers candidate responses u and contextual information c to generate a consistency discriminator score by calculating a probability that the candidate response u does not contradict previous responses of the dialogue model during the conversation or the current dialogue session.

[0068] According to an embodiment, the discriminator `flu` determines whether candidate responses are fluent and natural. Thus, the discriminator `flu` is based on fluentness as a supplementary factor.

[0069] The discriminator `flu` can be based on a tunable neural conversation generation model. For example, the discriminator `flu` can be based on a tunable gigaword-range neural network model for generating conversation responses, such as the Dialogue Generation Pre-trained Transformer `DialoGPT`.

[0070] The discriminator flu can generate a fluency discriminator score based on the perplexity of each generated round u given context information c.

[0071] According to an embodiment, the discriminator rel s The relevance of the generated responses to the dialogue model is measured. Thus, the discriminator... s Supplementary factors based on correlation.

[0072] Discriminator rel s It can focus on sentence-level semantic similarity. For example, the discriminator rel s The cosine similarity between the representation vector of candidate response u and the representation vector of context c can be used as a basis for pre-training deep bidirectional representations from unlabeled text. This cosine similarity is generated by a language representation model that pre-trains deep bidirectional representations from unlabeled text by jointly adjusting the left and right contexts across all layers. For example, the cosine similarity between the representation vector of candidate response u and the representation vector of context c can be generated by a model such as BERT (Bidirectional Encoder Representation from Transform), which is designed to pre-train deep bidirectional representations from unlabeled text by jointly adjusting the left and right contexts across all layers.

[0073] like Figure 4 As further shown in the figure, method 400 may further include determining a quality score for each of the candidate responses based on the discriminator scores (box 440). The quality score of candidate response u may be based on a weighted sum of the discriminator scores of candidate responses u. For example, platform 220 may determine the quality score r(u,c) of candidate response u in context c based on the following reward function equation [4]:

[0074]

[0075] In equation 4, d j (,c) represents the discriminator score, |d| represents the number of discriminators, and φ j This represents the weights of a given discriminator. The weights φ for a given discriminator score can be determined based on empirical evidence. For example, a specific weight φ can be given to each discriminator score, which is determined experimentally. The weight φ for each discriminator can be determined only for that discriminator. Therefore, each discriminator corresponds to a different weight φ. According to embodiments, the weights φ can be optimized for different situations.

[0076] like Figure 4 As further shown, method 400 may further include training a dialogue model based on quality scores (box 450).

[0077] For example, platform 220 can use a self-critical training algorithm to train the dialogue model. This algorithm can be a policy gradient reinforcement learning (RL) algorithm. The loss function used for training is shown in Equation 5:

[0078]

[0079] In equation 5, These are baseline candidate responses generated through greedy decoding. According to an example, It is the candidate response with the highest discriminator score. In Equation 5, u s It is based on sample discourse distributed according to the current model. According to the example, u s The candidate responses are randomly selected from the generated list of candidate responses. Therefore, r(·,·) represents the quality score based on the reward function shown in Equation 4.

[0080] In equation 5, |u s |Indicates the sampling discourse u s The length of w. As mentioned above, w represents the words in the candidate responses, and c represents contextual information such as conversation or dialogue history.

[0081] According to an embodiment, the self-critical training algorithm can be a special case of reinforcement of the baseline algorithm.

[0082] According to the embodiments, the above RL-based target l rl It can be used with the target l based on cross-entropy ce Combined to produce the final training objective shown in Equation 6:

[0083] l final =λ rl +(1-λ)l ce [Equation 6]

[0084] In Equation 6, λ can be determined empirically. According to the embodiment, λ can be set to 0.7 based on experience.

[0085] The target l based on cross-entropy ce Token-level cross-entropy loss can be determined based on human-annotated responses.

[0086] Although Figure 4 An example box for method 400 is shown, but in some implementations, method 400 may include... Figure 4The boxes depicted in the diagram may be fewer, different, or arranged differently compared to additional boxes. Alternatively, two or more boxes of method 400 may be executed in parallel.

[0087] The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit implementations to the precise forms disclosed. Modifications and variations are possible based on the foregoing disclosure, or can be derived from the practice of implementation.

[0088] As used in this article, the term "component" is intended to be interpreted broadly as hardware, firmware, or a combination of hardware and software.

[0089] Clearly, the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not a limitation on the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code—it should be understood that software and hardware can be designed to implement the systems and / or methods based on the descriptions herein.

[0090] Although specific combinations of features are stated in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not specifically stated in the claims and / or disclosed in the specification. While each dependent claim listed below may be directly subordinated to only one claim, the disclosure of possible implementations includes every dependent claim in combination with each other claim in the group of claims.

[0091] Elements, actions, or instructions used herein should not be construed as critical or necessary unless explicitly stated otherwise. Furthermore, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Additionally, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with “one or more.” The term “one” or similar language is used when referring to only one item. Furthermore, as used herein, the terms “has,” “have,” “having,” etc., are intended to be open-ended terms. Additionally, the phrase “based on” is intended to mean “at least partially based on” unless explicitly stated otherwise.

Claims

1. A data processing method, characterized in that, include: Receive input utterances from the device that is conversing with the dialogue model; Multiple candidate responses to the input utterance are obtained from the dialogue model, the multiple candidate responses including a first candidate response and a second candidate response; Determine multiple discriminator scores for the first candidate response, the multiple discriminator scores being provided by multiple discriminators, the multiple discriminators evaluating the quality of the first candidate response based on information corresponding to the first candidate response and contextual information corresponding to the history of the conversation; Determine multiple discriminator scores for the second candidate response, the multiple discriminator scores being provided by the multiple discriminators, the multiple discriminators evaluating the quality of the second candidate response based on information corresponding to the second candidate response and contextual information corresponding to the history of the conversation; A first quality score is determined to be associated with the first candidate response, the first quality score being a weighted sum of multiple discriminator scores corresponding to the first candidate response; A second quality score is determined to be associated with the second candidate response, the second quality score being a weighted sum of multiple discriminator scores corresponding to the second candidate response; as well as The dialogue model is trained based on at least one of the first quality score and the second quality score; The plurality of discriminators include: The first discriminator evaluates the specificity score of each candidate response by determining the average normalized inverse function document frequency of each word that appears in both the candidate response and the context information. The second discriminator, based on pre-training of deep bidirectional transform for language understanding, evaluates the consistency of each candidate response by calculating a probability that the candidate response contradicts a previous candidate response output by the dialogue model during the conversation. The third discriminator, based on an adjustable gigaword range neural network model, determines the perplexity of each candidate response based on the context information and evaluates the fluency of each candidate response. The fourth discriminator, based on a language representation model, evaluates whether each candidate response is relevant to the conversation by calculating the cosine similarity between the representation vector of each candidate response and the representation vector of the context information. The language representation model is designed to be pre-trained with a deep bidirectional representation from unlabeled text by jointly adjusting the left and right contexts across all layers.

2. The method according to claim 1, characterized in that, The dialogue model is trained based on the quality score of the candidate response with the highest quality score among the multiple candidate responses, and the quality score of a candidate response randomly selected from the multiple candidate responses.

3. The method according to claim 1, characterized in that, Training the dialogue model based on at least one of the first quality score and the second quality score includes: Based on at least one of the first quality score and the second quality score, a reinforcement learning-based objective is determined; The cross-entropy-based objective of the multiple candidate responses is evaluated by determining the token-level cross-entropy loss; and The final training objective is determined based on the combination of the reinforcement learning-based objective and the cross-entropy-based objective.

4. A device, characterized in that, include: At least one memory is configured to store program code and a neural network-based open-domain dialogue model; At least one processor is configured to read the program code and operate according to the instructions of the program code to perform the method as described in any one of claims 1-3.

5. A device, characterized in that, include: The receiving unit receives input utterances from the device that is conversing with the dialogue model. The obtaining unit obtains multiple candidate responses to the input utterance from the dialogue model, the multiple candidate responses including a first candidate response and a second candidate response; The first determining unit determines multiple discriminator scores for the first candidate response, the multiple discriminator scores being provided by multiple discriminators, the multiple discriminators evaluating the quality of the first candidate response based on information corresponding to the first candidate response and contextual information related to the history of the conversation; The second determining unit determines multiple discriminator scores for the second candidate response. The multiple discriminator scores are provided by the multiple discriminators, which evaluate the quality of the second candidate response based on information corresponding to the second candidate response and contextual information corresponding to the history of the conversation. The third determining unit determines a first quality score associated with the first candidate response, the first quality score being a weighted sum of multiple discriminator scores corresponding to the first candidate response; The fourth determining unit determines a second quality score associated with the second candidate response, the second quality score being a weighted sum of multiple discriminator scores corresponding to the second candidate response; as well as The training unit trains the dialogue model based on at least one of the first quality score and the second quality score; The plurality of discriminators include: The first discriminator evaluates the specificity score of each candidate response by determining the average normalized inverse function document frequency of each word that appears in both the candidate response and the context information. The second discriminator, based on pre-training of deep bidirectional transform for language understanding, evaluates the consistency of each candidate response by calculating a probability that the candidate response contradicts a previous candidate response output by the dialogue model during the conversation. The third discriminator, based on an adjustable gigaword range neural network model, determines the perplexity of each candidate response based on the context information and evaluates the fluency of each candidate response. The fourth discriminator, based on a language representation model, evaluates whether each candidate response is relevant to the conversation by calculating the cosine similarity between the representation vector of each candidate response and the representation vector of the context information. The language representation model is designed to be pre-trained with a deep bidirectional representation from unlabeled text by jointly adjusting the left and right contexts across all layers.

6. A non-transitory computer-readable medium, characterized in that, The device stores instructions, including one or more instructions that, when executed by one or more processors of the device, cause the one or more processors to perform the method as described in any one of claims 1-3.

Images