Intent recognition response method, device, equipment and storage medium
By performing multiple rounds of encoding and multi-hop query in a task-based dialogue system, combined with fine-grained decoding, the problem of insufficient intent recognition and response quality in complex scenarios is solved, and more accurate user intent recognition and response are achieved generate.
Patent Information
- Application Number
- CN202210415455.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-20
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-04-20
AI Technical Summary
When existing task-based dialogue systems deal with complex cross-task scenarios, it is difficult to accurately identify user intentions, especially when conceptual transformations are frequent, resulting in limited response quality and difficult for existing models to perform fine-grained multi-hop inference.
By obtaining the initial semantic representation of each round of dialogue, multiple rounds of encoding are constructed, external knowledge base is used to generate responses using multi-hop query and intent-aware representation, and combining fine-grained and coarse-grained decoding, the similarity between query vectors and knowledge triplets is calculated. , capture target knowledge and generate intent-aware representations.
Effectively identify user intentions, generate more accurate responses, overcome the difficulties of context dispersed and multi-hop reasoning, and improve the response quality of dialogue systems in complex scenarios.
Smart Images

Figure CN114817467B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of end-to-end intent recognition and response, and more specifically, to an intent recognition and response method, apparatus, device and storage medium. Background Art
[0002] Since the early days of artificial intelligence research, people have been committed to developing highly intelligent human-computer dialogue systems. The first generation of dialogue systems are mainly rule-based dialogue systems. Their advantages are transparent internal logic and easy analysis and debugging, but they are highly dependent on expert manual intervention, and their flexibility and scalability are poor. With the rise of big data technology, a second generation of dialogue systems based on statistical methods and data driven by reinforcement learning have emerged. It is a modular system that avoids high dependence on experts, but its disadvantages are that the model is difficult to maintain and its scalability is relatively limited. In recent years, with the major breakthroughs of deep learning in the fields of images, speech and text, the third generation of dialogue systems with deep learning as the main method have emerged. This system still continues the framework of the statistical dialogue system, but each module uses a neural network model. The acquisition of dialogue status is no longer obtained by using Bayesian posterior judgment, but by directly calculating the maximum conditional probability. In the optimization of dialogue strategies, people have also begun to use deep reinforcement learning models.
[0003] Task-based dialogues are usually designed to meet the needs of users with clear purposes, such as checking traffic, checking phone bills, ordering meals, booking tickets, consulting, and other task-based scenarios. Since user needs are relatively complex, they usually require multiple rounds of interaction. Users may also constantly modify and improve their needs during the dialogue process. Task-based robots need to help users clarify their purposes through questioning, clarification, and confirmation. As users' requirements for product experience gradually increase, actual dialogue scenarios become more complex. How to effectively respond to changes in user behavior based on existing technologies and improve the system's ability to recognize user intent and reason about deterministic knowledge are issues that need to be urgently addressed in task-based dialogue systems.
[0004] Task-based dialogue can be understood as a sequential decision-making process. During the dialogue process, the machine needs to update and maintain the internal dialogue state by understanding the user's statements, and then select the next optimal action (such as confirming requirements, asking for restrictions, providing results, etc.) according to the current dialogue state to complete the task. Task-based dialogue systems can be divided into two categories in terms of structure. One is the pipeline system, which adopts a modular structure, including a natural language understanding module, a dialogue management module, and a natural language generation module. This modular system structure is highly interpretable and easy to implement. Most practical task-based dialogue systems in the industry adopt this structure. However, its disadvantage is that it is not flexible enough. Each module is relatively independent, and it is difficult to jointly tune and adapt to changing application scenarios. In addition, since the errors between modules will accumulate layer by layer, the upgrade of a single module may also require the entire system to be adjusted together. Another implementation of the task-based dialogue system is an end-to-end system, which can directly output the system response with plain text as input. It is also a popular direction in academia in recent years. This type of structure hopes to train an overall mapping relationship from natural language input on the user side to natural language output on the machine side. It has the characteristics of strong flexibility and high scalability, reduces the labor cost in the design process, and breaks the isolation between traditional modules. In recent years, with the development of end-to-end neural generation models, an end-to-end trainable framework has been built for task-oriented dialogue systems.
[0005] Unlike traditional pipeline models, end-to-end models use one module and interact with a structured external database. In the field of dialogue systems, there are several reasons for using end-to-end systems:
[0006] ① What the dialogue needs to do is very similar to seq input seq output;
[0007] ② The dialogue system pipeline is a pipeline of four modules connected in series, so the errors of the previous modules will be amplified by the later ones, and there is no filtering or noise reduction measures. Using an end-to-end system will perform a local or global noise reduction on the entire process, which may help improve accuracy;
[0008] ③The end-to-end system is not that complicated to design, which is conducive to training and use.
[0009] However, task-based dialogue systems require strong "informativeness". They not only need to logically complete a certain service, but also need to interact with some structured information and require a large amount of data for training. As a result, the quality of responses generated by existing models is still limited.
[0010] Therefore, in order to meet the scalability requirements of task-based dialogue systems and improve the quality of responses generated by the system in practical applications, it is necessary to propose a task-based dialogue model that can perform fine-grained reasoning on deterministic knowledge, capture concept shifts in cross-task scenarios, and identify the user's true intentions. Many memory-enhanced end-to-end models that have been proposed use dialogue history and domain-specific knowledge base (KB) to merge KB information and perform knowledge-based reasoning, which can achieve better performance, but there are still two major limitations. On the one hand, the model relies heavily on the soft attention mechanism to generate responses by taking the weighted sum of the embedded memory triples (from the dialogue history and external KB) as the output representation. Since the representation obtained in this way is scattered by the context, it is difficult to model the specific conceptual tags of deterministic knowledge. On the other hand, the soft attention mechanism itself is not suitable for performing fine-grained (tag-level) multi-hop reasoning, which makes it difficult to capture the user's true intention to generate accurate responses, especially in complex cross-task scenarios where concept shifts may occur. Existing attention-based models are usually unable to perform such tag-level multi-hop reasoning, which hinders them from obtaining accurate responses. Summary of the invention
[0011] In response to at least one defect or improvement need in the prior art, the present invention provides an intent recognition and response method, apparatus, device and storage medium for capturing concept transformations involved in task-oriented conversations, with the aim of effectively identifying user intent and generating accurate responses.
[0012] To achieve the above object, according to one aspect of the present invention, there is provided an intention recognition response method, comprising the following steps:
[0013] Obtain each round of dialogue, parse each round of dialogue into an initial semantic representation of a dialogue sequence including a single question sequence and a response sequence; perform multiple rounds of encoding on the initial semantic representation to generate a contextual hidden state and a query vector for each round of dialogue sequence; wherein the query vector of the previous round of dialogue sequence acts on the encoding of the initial semantic representation of the next round of dialogue sequence;
[0014] Building an external knowledge base based on multi-domain conversation histories and integrating key user information according to the conversation histories, parsing each piece of knowledge in the external knowledge base into knowledge triples;
[0015] Using the context hidden state of the last round of dialogue sequence as the initial query vector, a multi-hop query is performed in the external knowledge base and the query vector is updated, the similarity between the query vector and each piece of knowledge in the external knowledge base is calculated, and the similarity is binary classified to generate a global pointer;
[0016] Performing knowledge screening in an external knowledge base according to the global pointer, calculating similarities between the query vector, the screened knowledge triples and the model deterministic knowledge, capturing the target knowledge and generating an intent-aware representation;
[0017] Fine-grained decoding and coarse-grained decoding are performed according to the intention-aware representation to obtain respective context-aware response sequences and their corresponding probabilities, and the response sequence with the maximum probability is taken as the output response sequence.
[0018] Furthermore, the above-mentioned intention recognition response method further includes:
[0019] Each piece of knowledge in the external knowledge base is represented as a knowledge triple containing a head entity, a relationship and a tail entity, and each of the knowledge triples is embedded in the memory network to obtain a corresponding knowledge embedding matrix.
[0020] Furthermore, in the above-mentioned intention recognition response method, the context hidden state of the last round of dialogue sequence is used as the initial query vector to perform multi-hop query in the external knowledge base and update the query vector specifically as follows:
[0021] According to the initial query vector, multiple hops are performed in the knowledge embedding matrix, and the attention weight corresponding to each hop of the knowledge embedding matrix is calculated respectively;
[0022] The memory network is read according to the weighted sum of the attention weight corresponding to the knowledge embedding matrix of the current hop and the knowledge embedding matrix of the next hop to obtain the query vector of the next hop.
[0023] Furthermore, in the above-mentioned intent recognition response method, the similarity between the query vector and each piece of knowledge in the external knowledge base is calculated as follows:
[0024] The query vector is dot-producted with the knowledge triplet to obtain the similarity between the query vector and each piece of knowledge in the external knowledge base.
[0025] Furthermore, the above-mentioned intent recognition response method calculates the similarity between the query vector, the screened knowledge triples and the model deterministic knowledge, captures the target knowledge and generates the intent-aware representation, specifically:
[0026] The similarity between the query vector, the head entity of the knowledge triple and the model deterministic knowledge is calculated, and the tail entities of the related knowledge triples are integrated according to the similarity to obtain the target knowledge and generate the intent-aware representation.
[0027] Furthermore, in the above-mentioned intent recognition response method, the coarse-grained decoding is specifically as follows:
[0028] The conversation history attention representation in the hidden state is calculated and concatenated with the conversation history attention representation to obtain a coarse-grained context-aware output representation.
[0029] Furthermore, in the above-mentioned intent recognition response method, the fine-grained decoding is specifically as follows:
[0030] The conversation history attention representation under the hidden state is calculated, the hidden state is concatenated with the conversation history attention representation and fused with the intent-aware representation to obtain a fine-grained context-aware output representation.
[0031] According to a second aspect of the present invention, there is also provided an intention recognition response device, which comprises:
[0032] A contextual dialogue history editor obtains each dialogue round and parses each dialogue round into an initial semantic representation of a dialogue sequence including a single question sequence and a response sequence; the initial semantic representation is encoded for multiple rounds to generate a contextual hidden state and a query vector for each dialogue sequence; wherein the query vector of the previous dialogue sequence acts on the encoding of the initial semantic representation of the next dialogue sequence;
[0033] An external knowledge base is constructed based on multi-domain conversation histories and incorporates user key information according to the conversation histories, and each piece of knowledge information in the external knowledge base is parsed into knowledge triples;
[0034] Using the context hidden state of the last round of dialogue sequence as the initial query vector, a multi-hop query is performed in the external knowledge base and the query vector is updated, the similarity between the query vector and each piece of knowledge in the external knowledge base is calculated, and the similarity is binary classified to generate a global pointer;
[0035] An intention reasoning module, which screens knowledge in an external knowledge base according to the global pointer, calculates the similarity between the query vector, the screened knowledge triples and the model deterministic knowledge, captures the target knowledge and generates an intention-aware representation;
[0036] The hierarchical reply decoder performs fine-grained decoding and coarse-grained decoding according to the intent-aware representation, obtains respective context-aware response sequences and their corresponding probabilities, and takes the response sequence with the maximum probability as the output response sequence.
[0037] According to the third aspect of the present invention, there is also provided an intention identification response device, which includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit performs the steps of any one of the above-mentioned methods.
[0038] According to a fourth aspect of the present invention, there is also provided a storage medium storing a computer program executable by an access authentication device, wherein when the computer program runs on the access authentication device, the access authentication device executes the steps of any one of the above methods.
[0039] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:
[0040] (1) The present invention provides an intent recognition and response method, apparatus, device, and storage medium. By calculating the similarity between the query vector, knowledge triples, and model deterministic knowledge, specific conceptual tags are modeled on the deterministic knowledge to generate intent-aware representations. The intent recognition and response method, apparatus, device, and storage medium can be used to capture the concept transformations involved in task-oriented dialogues, so that context information is not dispersed, thereby effectively identifying the user's intent and generating more accurate responses.
[0041] (2) The present invention provides an intent recognition response method, apparatus, device and storage medium, which parses each round of dialogue into a dialogue sequence including a single question sequence and a response sequence, overcoming the difficulty of modeling long dialogue texts, and obtains the contextual hidden state and query vector of each round of dialogue sequence through multi-round encoding, which can better capture two-way semantic dependencies and generate more accurate responses. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0043] Figure 1 A flow chart of an intent recognition response method provided in this embodiment;
[0044] Figure 2 A structural diagram of an intent recognition response device provided in this embodiment. DETAILED DESCRIPTION
[0045] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0046] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.
[0047] In addition, in order to avoid obscuring the understanding of the present invention by the skilled person, well-known or widely used techniques, elements, structures and processes may not be described or shown in detail. Although the drawings represent exemplary embodiments of the present invention, the drawings are not necessarily drawn to scale, and certain features may be enlarged or omitted to better illustrate and explain the present invention.
[0048] In view of the shortcomings of existing task-based dialogues, such as the scattered context, the difficulty in modeling specific conceptual tags of deterministic knowledge, the inability to perform tag-level multi-hop reasoning, and the occurrence of concept transfer in complex cross-task scenarios, this application proposes an end-to-end task-oriented dialogue solution based on an intentional reasoning network, which can effectively use dialogue history and external knowledge to accurately model the jump relationship between entities in multiple fields, and then accurately identify the user's intention and generate entity knowledge replies with intention awareness. It overcomes the defect that existing dialogue systems cannot produce effective and high-quality responses in daily dialogue scenarios due to frequent concept and intention conversion.
[0049] The following is a description of the meanings of the modules and parameters involved in this application:
[0050] (1) Y is the response sequence, Y = {y1, y2, ..., y n};
[0051] (2) The dialogue sequence for each round is (Q p ,Y p ), where Q p and Y p They represent the question sequence (with m tokens) and response sequence (with n tokens) of round p respectively;
[0052] (3) The external memory module is M, M = [X; B] = (m1, m2, ..., m l ), X is the conversation history, B is the knowledge base, and each entity in M is represented by a triple, that is, m i =(h,r,t);
[0053] (4) A representation of a dialogue sequence, where Representation of the problem sequence, a representation representing a sequence of responses;
[0054] (5) Contextual hidden state h enc =(h enc,1 ,h enc,2 ,...,h enc,i );
[0055] (6) The embeddable external matrix C = (c 1 ,...,c k+1 );
[0056] Initial query vector is the query vector of the k-th hop;
[0057] On the one hand, this embodiment provides an intent recognition response method, Figure 1 For a flowchart of an intent recognition response method provided in this embodiment, please refer to Figure 1 , the method comprising:
[0058] (1) Obtain each round of dialogue, obtain each round of dialogue, and parse each round of dialogue into an initial semantic representation of a dialogue sequence including a single question sequence and a response sequence; encode the initial semantic representation for multiple rounds to generate a contextual hidden state and a query vector for each round of dialogue sequence; wherein the query vector of the previous round of dialogue sequence acts on the encoding of the initial semantic representation of the next round of dialogue sequence;
[0059] Unlike traditional dialogue system processing, the input representation is not obtained in units of the entire dialogue sequence, but in units of a single question sequence Q p and the response sequence Y p The dialogue sequence is composed of p ,Yp), where Q p and Y p They represent the question sequence (with m tokens) and response sequence (with n tokens) in round p respectively.
[0060] The set of response sequences Y can be expressed as formula (1):
[0061] Y={y1,y2,...,y n} (1)
[0062] In this embodiment, the semantic role labeling SRL tool connects each dialogue sequence into “[CLS]Q p [SEP]Y p [SEP]", where the [CLS] tag is at the beginning of the sentence and is used to separate the input sequence.p [SEP]Y p [SEP]” is input into the BERT model to obtain the initial semantic representation of each round of dialogue sequence, and output the initial semantic representation of the dialogue sequence, as shown in formula (2).
[0063]
[0064] in, is the representation of the dialogue sequence, is the representation of the problem sequence, is the representation of the response sequence, m and n are both natural numbers;
[0065] In a specific example, the initial semantic representation of the above dialogue sequence is input into the bidirectional long short-term memory network BiSTM through multiple rounds of encoding to generate the complete context hidden state h of each round of dialogue sequence. enc,i , which can be expressed as formula (3).
[0066] h enc =(h enc,1 ,h enc,2 ,...,h enc,i ) (3)
[0067] Each round of dialogue sequence generates a query vector and acts on the initial semantic representation of the next round of dialogue sequence, and finally obtains a query vector based on the entire dialogue sequence, which can be expressed as formula (4):
[0068]
[0069] (2) constructing an external knowledge base based on multi-domain conversation histories and incorporating user key information based on the conversation histories, parsing each piece of knowledge in the external knowledge base into knowledge triples;
[0070] In a specific example, in order to perform fine-grained representation learning, each entity in the external knowledge base is parsed into a knowledge triple through the BERT network model, including a head entity, a relationship entity, and a tail entity, which can be expressed as formula (5):
[0071] m i =BERT(h,r,t) (5)
[0072] Among them, h represents the head entity, r represents the relationship entity between the head entity and the tail entity, and t represents the tail entity.
[0073] The memory module of the external knowledge base is M, which consists of knowledge triples, i.e. (h, r, t)∈M, which can be expressed as formula (66):
[0074] M=[X;B]=(m1,m2,...,ml ) (6)
[0075] Among them, X represents the conversation history and B represents the knowledge base.
[0076] In order to better encode each piece of knowledge in the external knowledge base and make it more suitable for multi-hop reasoning and vector calculation, this application embeds the knowledge triple into a word vector space with strong entity relationship and semantic displacement information to obtain the corresponding memory embedding (e h ,e r ,e t ), see formula (7), can be embedded in the external matrix see formula (8), and the corresponding knowledge embedding matrix is obtained
[0077] m i =(h,r,t);(e h ,e r ,e t ) (7)
[0078] C=(c 1 ,..., k+1 ) (8)
[0079] (3) Using the context hidden state of the last round of dialogue sequence as the initial query vector, a multi-hop query is performed in the external knowledge base and the query vector is updated, the similarity between the query vector and each piece of knowledge in the external knowledge base is calculated, and the similarity is binary classified to generate a global pointer;
[0080] Specifically, the initial query vector is circulated in the knowledge embedding matrix for multiple jumps, and the attention weight corresponding to each jump of the knowledge embedding matrix is calculated respectively;
[0081] The memory network is read according to the weighted sum of the attention weight corresponding to the knowledge embedding matrix of the current hop and the knowledge embedding matrix of the next hop to obtain the query vector of the next hop.
[0082] The contextual hidden state h of the last round of dialogue sequence enc,m+n As the initial query vector
[0083] In this embodiment, a natural number k is given, according to the query vector Perform k-hop loops in the knowledge embedding matrix and calculate the current knowledge embedding matrix The corresponding attention weight It can be expressed as formula (9),
[0084]
[0085] Embedding matrix based on the knowledge of the current jump The corresponding attention weight and the knowledge embedding matrix of the next hop Weighted and read memory network It can be expressed as formula (10),
[0086]
[0087] Based on the current query vector Memory network with read Update the next hop query vector, which can be expressed as formula (11):
[0088]
[0089] The query vector The knowledge triple vector in the constructed external knowledge base Do the dot product to get the similarity between the query vector and each piece of knowledge in the external knowledge base, and search for possible results in the constructed external knowledge base.
[0090] Based on the query vector and knowledge embedding matrix The similarity between them generates a binary external knowledge global pointer G, see formula (12) and formula (13). The external knowledge global pointer is used to filter out worthless external knowledge and select several pieces of knowledge with the highest value for subsequent further decoding.
[0091]
[0092] G=(g1,...g l ) (12)
[0093] In a specific embodiment, the Sigmoid function is used to transform the multi-label classification problem into a single-label binary classification problem to obtain a global pointer to external knowledge.
[0094] (4) screening knowledge in the external knowledge base according to the global pointer, calculating the similarity between the query vector, the screened knowledge triples and the model deterministic knowledge, capturing the target knowledge and generating an intent-aware representation;
[0095] Specifically, in order to model specific conceptual tags for deterministic knowledge, the similarity between the query vector, the head entity tags of the filtered knowledge triples and the modeled deterministic knowledge is calculated, and the tail entity tags of the related knowledge triples are integrated so that the contextual information is not dispersed. Tag-level joint reasoning and multi-hop reasoning are responsible for capturing specific target information from width and depth, respectively, to generate intent-aware representations.
[0096] (5) Fine-grained decoding and coarse-grained decoding are performed according to the intention-aware representation to obtain respective context-aware response sequences and their corresponding probabilities, and the response sequence with the maximum probability is taken as the output response sequence.
[0097] For coarse-grained decoding, the context hidden state h is calculated dec,t The dialogue history attention representation h′ dec,t , and the context hidden state h dec,t With the dialogue history attention representation h′ dec,t Connect to obtain a coarse-grained context-aware output representation It can be expressed as formula (14),
[0098]
[0099] For fine-grained decoding, the context hidden state h is calculated dec,t The dialogue history attention representation h′ dec,t , the hidden state h dec,t With the dialogue history attention representation h′ dec,t Connecting and Fusion Intent-Aware Representations I dec,t , and obtain the fine-grained context-aware output representation o c,t , which can be expressed as formula (15),
[0100] o c,t =W[h dec,t ,h′ dec,t ,I dec,t ] (15)
[0101] The intentional reasoning network IR-Net provided in this embodiment is actually a memory-enhanced Seq2Seq model, which aims to generate the most likely response sequence. The probabilities of the two decoding methods are calculated based on the response sequences obtained by fine-grained decoding and coarse-grained decoding. The probability of the response of the response sequence Y is defined as shown in formula (16),
[0102]
[0103] The response sequence corresponding to the maximum probability value obtained by the two decoding methods is used as the final output result of the decoder, that is, the final generated response sequence.
[0104] It should be noted that although the operations of the methods of the embodiments of this specification are described in a specific order in the above-mentioned embodiments, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flowchart can change the order of execution. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution.
[0105] On the other hand, this embodiment also provides an intention recognition response device, Figure 2 For a structural diagram of an intention recognition response device provided in this embodiment, please refer to Figure 2 , the device comprises:
[0106] A contextual dialogue history editor is used to obtain each round of dialogue, parse each round of dialogue into an initial semantic representation of a dialogue sequence including a single question sequence and a response sequence; encode the initial semantic representation for multiple rounds to generate a contextual hidden state and a query vector for each round of dialogue sequence; wherein the query vector of the previous round of dialogue sequence acts on the encoding of the initial semantic representation of the next round of dialogue sequence;
[0107] In a specific embodiment, the contextual dialogue history encoder encodes historical dialogues one by one in multiple rounds through a bidirectional long short-term memory network BiSTM to capture bidirectional semantic dependencies; the initial semantic representation of the input is obtained in units of dialogue sequences consisting of a single question sequence and a response sequence, overcoming the shortcomings of modeling long dialogue texts.
[0108] An external knowledge base is constructed based on multi-domain conversation histories and incorporates user key information according to the conversation histories, and each piece of knowledge information in the external knowledge base is parsed into knowledge triples;
[0109] Using the context hidden state of the last round of dialogue sequence as the initial query vector, a multi-hop query is performed in the external knowledge base and the query vector is updated, the similarity between the query vector and each piece of knowledge in the external knowledge base is calculated, and the similarity is binary classified to generate a global pointer;
[0110] The external knowledge base module is a key part of the task-oriented dialogue system as the basis for knowledge query. The external knowledge base uses a memory network (MN) to store the global cross-domain knowledge shared by the encoder and the decoder, fully utilizes the encoder and decoder, and is built based on multi-domain dialogue history. It also relies on the dialogue history to integrate the user's key information into the external knowledge base to obtain a more accurate response.
[0111] An intention reasoning module, used to screen knowledge in an external knowledge base according to the global pointer, calculate the similarity between the query vector, the screened knowledge triples and the model deterministic knowledge, capture the target knowledge and generate an intention-aware representation;
[0112] By mining potential reasoning chains in a fine-grained manner, the completeness and accuracy of the generated responses can be improved. At the same time, this module can well cope with the concept transfer that may occur in complex cross-task scenarios, and can clearly mine the entity jump relationship between multiple fields to generate intent-aware responses.
[0113] The hierarchical reply decoder is used to perform fine-grained decoding and coarse-grained decoding according to the intention-aware representation, respectively obtain respective context-aware response sequences and their corresponding probabilities, and take the response sequence with the maximum probability as the output response sequence.
[0114] This module uses a hierarchical mechanism to decode the response sequence. When decoding a dialogue sequence, a coarse-grained LSTM decoder and a fine-grained LSTM decoder are used to simultaneously calculate the probability. Finally, the sequence corresponding to the maximum probability obtained by the two decoders is taken as the final output result of the decoder, that is, the final generated response sequence.
[0115] In a specific embodiment, there is the following dialogue:
[0116] User: Please check the temperature for me today.
[0117] IR-Net:?
[0118] The specific processing process is:
[0119] (1) Input the current round of dialogue into the context history dialogue encoder module to generate the query vector and the initial semantic representation of the triple (today, temperature,?);
[0120] (2) Input the above initial semantic representation into the external knowledge base for query and selection to generate a global pointer G;
[0121] (3) Combine the global pointer G generated above with the external knowledge base to obtain the triple representation of the relevant knowledge (Monday, low_temp, 20f) and (Monday, high_temp, 30f), and combine joint reasoning and multi-hop reasoning to obtain the corresponding intent-aware representation;
[0122] (4) The above intention perception representation is input into the hierarchical reply decoder, and the final response sequence (today, temperature, 20f-30f) is obtained through coarse-grained and fine-grained analysis and selection, and the final response sequence is output, that is:
[0123] User: Please check the temperature for me today.
[0124] IR-Net: Toady's temperature is 20f-30f.
[0125] This embodiment also provides an intent recognition response device, which includes at least one processor and at least one memory, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the processor executes the steps of the intent recognition response method in Example 1. The specific steps are shown in Example 1 and will not be repeated here. In this embodiment, the types of processor and memory are not specifically limited. For example, the processor can be a microprocessor, a digital information processor, an on-chip programmable logic system, etc. The memory can be a volatile memory, a non-volatile memory, or a combination thereof.
[0126] The intention recognition response device can also communicate with one or more external devices (such as keyboards, pointing terminals, displays, etc.), and can also communicate with one or more terminals that enable users to interact with the intention recognition response device, and / or communicate with any terminal (such as a network card, a modem, etc.) that enables the intention recognition response device to communicate with one or more other computing terminals. Such communication can be carried out through an input / output (I / O) interface. In addition, the intention recognition response device can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter.
[0127] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.
[0128] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0129] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0130] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.
[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0132] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0133] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program code.
[0134] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by entering a program to instruct the relevant hardware. The program may be stored in a computer-readable memory, and the memory may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0135] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.
[0136] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0137] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. An intention recognition response method, characterized in that: The following steps are involved: Obtain each round of dialogue, parse each round of dialogue into an initial semantic representation of a dialogue sequence including a single question sequence and a response sequence; perform multiple rounds of encoding on the initial semantic representation to generate a contextual hidden state and a query vector for each round of dialogue sequence; The query vector of the previous dialogue sequence acts on the encoding of the initial semantic representation of the next dialogue sequence; Building an external knowledge base based on multi-domain conversation histories and integrating key user information according to the conversation histories, parsing each piece of knowledge in the external knowledge base into knowledge triples; Using the context hidden state of the last round of dialogue sequence as the initial query vector, a multi-hop query is performed in the external knowledge base and the query vector is updated, the similarity between the query vector and each piece of knowledge in the external knowledge base is calculated, and the similarity is binary classified to generate a global pointer; Performing knowledge screening in an external knowledge base according to the global pointer, calculating similarities between the query vector, the screened knowledge triples and the model deterministic knowledge, capturing the target knowledge and generating an intent-aware representation; Fine-grained decoding and coarse-grained decoding are performed according to the intention-aware representation to obtain respective context-aware response sequences and their corresponding probabilities, and the response sequence with the maximum probability is taken as the output response sequence.
2. The intention recognition response method according to claim 1, characterized in that: Also includes: Each piece of knowledge in the external knowledge base is represented as a knowledge triple containing a head entity, a relationship and a tail entity, and each of the knowledge triples is embedded in the memory network to obtain a corresponding knowledge embedding matrix.
3. The intention recognition response method according to claim 2, characterized in that: The method of using the context hidden state of the last round of dialogue sequence as the initial query vector to perform multi-hop query in the external knowledge base and update the query vector is specifically as follows: According to the initial query vector, multiple hops are performed in the knowledge embedding matrix, and the attention weight corresponding to each hop of the knowledge embedding matrix is calculated respectively; The memory network is read according to the weighted sum of the attention weight corresponding to the knowledge embedding matrix of the current hop and the knowledge embedding matrix of the next hop to obtain the query vector of the next hop.
4. The intention recognition response method according to claim 1, characterized in that: The specific calculation of the similarity between the query vector and each piece of knowledge in the external knowledge base is: The query vector is dot-producted with the knowledge triplet to obtain the similarity between the query vector and each piece of knowledge in the external knowledge base.
5. The intention recognition response method according to claim 1, characterized in that: The calculation of the similarity between the query vector, the filtered knowledge triples and the model deterministic knowledge captures the target knowledge and generates the intent-aware representation, specifically: The similarity between the query vector, the head entity of the knowledge triple and the model deterministic knowledge is calculated, and the tail entities of the related knowledge triples are integrated according to the similarity to obtain the target knowledge and generate the intent-aware representation.
6. The intention recognition response method according to claim 1, characterized in that: The coarse-grained decoding is specifically as follows: The conversation history attention representation in the hidden state is calculated and concatenated with the conversation history attention representation to obtain a coarse-grained context-aware output representation.
7. The intention recognition response method according to claim 1, characterized in that: The fine-grained decoding is specifically as follows: The conversation history attention representation under the hidden state is calculated, the hidden state is concatenated with the conversation history attention representation and fused with the intent-aware representation to obtain a fine-grained context-aware output representation.
8. An intention recognition response device, characterized in that: include: The contextual dialogue history editor obtains each dialogue round and parses each dialogue round into an initial semantic representation of the dialogue sequence containing a single question sequence and a response sequence; Encoding the initial semantic representation for multiple rounds to generate a context hidden state and a query vector for each round of dialogue sequence; The query vector of the previous dialogue sequence acts on the encoding of the initial semantic representation of the next dialogue sequence; An external knowledge base is constructed based on multi-domain conversation histories and incorporates user key information according to the conversation histories, and each piece of knowledge information in the external knowledge base is parsed into knowledge triples; Using the context hidden state of the last round of dialogue sequence as the initial query vector, a multi-hop query is performed in the external knowledge base and the query vector is updated, the similarity between the query vector and each piece of knowledge in the external knowledge base is calculated, and the similarity is binary classified to generate a global pointer; An intention reasoning module, which screens knowledge in an external knowledge base according to the global pointer, calculates the similarity between the query vector, the screened knowledge triples and the model deterministic knowledge, captures the target knowledge and generates an intention-aware representation; The hierarchical reply decoder performs fine-grained decoding and coarse-grained decoding according to the intent-aware representation, obtains respective context-aware response sequences and their corresponding probabilities, and takes the response sequence with the maximum probability as the output response sequence.
9. An intention recognition response device, characterized in that: The method comprises at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit executes the steps of the method according to any one of claims 1 to 7.
10. A storage medium, characterized in that: It stores a computer program executable by an access authentication device. When the computer program runs on the access authentication device, the access authentication device executes the steps of the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Multi-round dialogue method and system integrating knowledge graph and emotion supervision
CN111651609A
Dialogue generation method and device based on knowledge base, terminal and storage medium
CN112818098A
Global-to-Local Memory Pointer Networks for Task-Oriented Dialogue
US20200105272A1
Method, apparatus, device, and storage medium for training model and generating dialog
US20210342551A1
