An intelligent question-answering method, device, equipment and storage medium

Abstract

Embodiments of the present invention relate to an intelligent question-answering method, device, equipment and storage medium, which use two context-aware transformations to incorporate context information into the word representation in the dialogue scenario; store the dialogue scenario and the knowledge base semantic information separately; understand the user's current statement through a memory network, and activate the current dialogue generation task; the working memory dynamically controls the "activation" of the long-term memory, reads the content related to the current task from the long-term memory into the short-term storage, and then generates the current reply word by word based on a heuristic strategy. The present invention can store the dialogue scenario and the knowledge base semantic information separately, and utilize the information stored in these two ways through memory reasoning, can efficiently utilize the external knowledge base information, and greatly improves the performance of task-based dialogue generation.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of intelligent dialogue, and in particular, to an intelligent question-answering method, device, equipment and storage medium. Background Art

[0002] In recent years, human-computer dialogue has received extensive attention from the academic and industrial communities. Task-oriented dialogue systems, such as hotel reservation or technical support services, can help users achieve specific goals in natural language. In the task of task-oriented dialogue generation, the dialogue system can obtain the user's statement at the current moment, the interaction information between the dialogue system and the user at an earlier moment in the current dialogue, and external database information. Therefore, to build such a system, useful information needs to be aggregated into their responses, and the dialogue system must utilize external database information to provide correct information to users and complete the users' goals.

[0003] Classic dialogue systems are pipeline-designed and usually include natural language understanding, dialogue management, and dialogue generation. Such systems require human effort for internal annotation and expert knowledge to design internal state representations. Moreover, classic systems use SQL query language to operate on external knowledge bases. Therefore, the prior art is prone to ignoring knowledge base information, directly learning the mapping from the dialogue history to the dialogue response, and splicing the knowledge base information with the dialogue history, unable to efficiently utilize external knowledge for effective dialogue reasoning. Summary of the Invention

[0004] In view of this, to solve the above technical problems or some of the technical problems, embodiments of the present invention provide an intelligent question-answering method, device, equipment and storage medium.

[0005] In a first aspect, embodiments of the present invention provide an intelligent question-answering method, including:

[0006] Performing dual-channel memory encoding at the word granularity on historical dialogue information and performing context-aware transformation to obtain dialogue scenario memory encoding;

[0007] Performing dual-channel memory encoding at the triple granularity on database semantic knowledge outside the dialogue history to obtain semantic memory encoding;

[0008] Under the stimulation of the initial semantic encoding, performing information reasoning on the word granularity memory unit to obtain an encoding vector for the current dialogue task;

[0009] Based on the encoding vector of the current dialogue task, the attention controller cyclically generates a corresponding query vector for each moment;

[0010] Based on the query vector, activate the information in the episodic memory encoding and the semantic memory encoding through working memory to obtain a vocabulary probability distribution, an episodic information copy probability distribution, and a semantic knowledge copy probability distribution;

[0011] Select the word to be generated currently from the probability distributions of the three pieces of information based on the rule system to form an answer.

[0012] In a possible implementation, the method further includes:

[0013] Perform word-level memory encoding on the historical dialogue information according to context-aware transformation to obtain context-aware word-level dual-channel memory encoding.

[0014] In a possible implementation, the method further includes:

[0015] Add the dialogue person attribute information and the time information to the historical dialogue information to obtain an information set X = {x 1 , …, x n , $}, where n is the maximum time series length of the information set, and randomly initialize two word vector matrices and V is the dictionary dimension, and d is the dimension of the word vector;

[0016] x i The corresponding dual-channel vectorized encoding is and

[0017] In a possible implementation, the method further includes:

[0018] Based on the following transformation:

[0019]

[0020]

[0021] Obtain the context-aware word-level dual-channel memory encoding as and

[0022] In a possible implementation, the method further includes:

[0023] Under the stimulation of the initial semantic encoding, complete the information reasoning of the word-level memory unit through multiple rounds of iterative attention mechanism to obtain the encoding vector of the current dialogue task.

[0024] In a possible implementation, the method further includes:

[0025] Use cross-entropy supervision to train the answer model.

[0026] In a second aspect, an embodiment of the present invention provides an intelligent question-answering device, including:

[0027] An encoding module, configured to perform dual-channel memory encoding at the word granularity on historical dialogue information, and perform context-aware transformation to obtain dialogue scenario memory encoding;

[0028] The encoding module is further configured to perform dual-channel memory encoding at the triple granularity on database semantic knowledge outside the dialogue history to obtain semantic memory encoding;

[0029] An inference module, configured to perform information inference on the word granularity memory unit under the stimulation of the initial semantic encoding to obtain an encoding vector for the current dialogue task;

[0030] A query module, based on the encoding vector of the current dialogue task, cyclically generates a corresponding query vector for each moment by an attention controller;

[0031] The query module is further configured to, based on the query vector, activate information in the scenario memory encoding and the semantic memory encoding through working memory to obtain a word list probability distribution, a scenario information copy probability distribution, and a semantic knowledge copy probability distribution;

[0032] The query module is further configured to select the word to be generated currently from the probability distributions of the three pieces of information based on a rule system to form an answer.

[0033] In a possible implementation manner, the method further includes:

[0034] The encoding module is further configured to perform word granularity memory encoding on historical dialogue information according to context-aware transformation to obtain context-aware word granularity dual-channel memory encoding.

[0035] In a third aspect, an embodiment of the present invention provides a device, including: a processor and a memory, where the processor is configured to execute an intelligent question-answering program stored in the memory to implement the intelligent question-answering method according to any one of the first aspects above.

[0036] In a fourth aspect, an embodiment of the present invention provides a storage medium, where the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the intelligent question-answering method according to any one of the first aspects above.

[0037] The embodiments of the present invention combine context-aware transformation with a multi-layer memory module, which can more accurately reason about dialogue scenario information, thereby further improving the rationality and accuracy of dialogue responses. Two memory modules, namely episodic memory and working memory, are used to store dialogue history and external knowledge bases respectively, realizing the separate storage and utilization of information with different structures, effectively integrating external knowledge, and further improving the rationality and accuracy of dialogue responses. The attention controller can cyclically generate memory query vectors, and during the dialogue generation process, dynamically activate relevant information in episodic memory and working memory at each moment, thereby generating reasonable dialogue responses. Copying words from the dialogue history and external knowledge bases can generate words outside the vocabulary; and a rule system can effectively fuse the vocabulary generation probability and the probabilities of two copied vocabularies, thus effectively solving the problem of generating unseen words. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic flowchart of an intelligent question-answering method provided by an embodiment of the present invention;

[0039] Figure 2 is a schematic flowchart of information reasoning related to an embodiment of the present invention;

[0040] Figure 3 is a schematic structural diagram of an intelligent question-answering device provided by an embodiment of the present invention;

[0041] Figure 4 is a schematic structural diagram of a device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0043] For ease of understanding of the embodiments of the present invention, the following will further explain with specific embodiments in conjunction with the accompanying drawings. The embodiments do not constitute a limitation to the embodiments of the present invention.

[0044] Figure 1 is a schematic flowchart of an intelligent question-answering method with memory as the core provided by an embodiment of the present invention. As Figure 1 shown, the method specifically includes:

[0045] Step S101: Perform dual-channel memory encoding at the word granularity on the historical dialogue information, and perform context-aware transformation to obtain the dialogue scenario memory encoding.

[0046] In the embodiment of the present invention, the information of the historical dialogue is obtained through the information acquisition method, and word granularity memory encoding is performed on multiple sentences in the dialogue history. First, the present invention adds the speaker attribute and the relative time information of each word in the sentence; then it is concatenated into a long sentence X = {x 1 ,…,x n ,$}, and a special "sentinel" symbol $ is added at the end of the sentence. Wherein, n is the maximum time series length of the sentence set; then the sentence X = {x 1 ,…,x n ,$} is subjected to dual-channel word vector mapping, and the speaker attribute and the relative time information are given to the word vector in an additive manner to obtain the dual-channel word vectorized encoding of the sentence. Two word vector matrices are randomly initialized and Wherein, V is the dictionary dimension, d is the dimension of the word vector, and A and C respectively adopt a normal distribution with a standard deviation of 0.1 and a mean of 0 as the random initialization parameters. The word x in the sentence i is subjected to dual-channel word vector mapping to obtain the dual-channel vectorized encoding as and To simulate the out-of-vocabulary word phenomenon, the present invention adopts a random masking strategy to discard some words. Finally, for the dual-channel vectorized encoding, the present invention proposes two context-aware transformations to alleviate the out-of-vocabulary word encoding problem, and finally obtains the context-aware word granularity dual-channel memory encoding and The specific transformation is as follows:

[0047]

[0048]

[0049] Step S102: Perform dual-channel memory encoding at the triple granularity on the database semantic knowledge outside the dialogue history to obtain the semantic memory encoding.

[0050] In the embodiment of the present invention, based on the classification system of human memory, the historical dialogue information is divided into scenario information and semantic knowledge in the external knowledge base. Based on the word granularity dual-channel memory encoding, the scenario memory encoding corresponding to the scenario information and the semantic memory encoding corresponding to the semantic knowledge are determined, and the scenario memory encoding and the semantic memory encoding are respectively stored in the scenario module and the semantic module.

[0051] During the dialogue process, the dialogue system can not only obtain the dialogue history X = {x 1 ,…,x n, $}, and can also obtain external knowledge base information B = {b 1 , L, b l , $}, with a total of l pieces of knowledge, where b i represents a certain piece of knowledge, presented in the form of a subject-relation-object triple b i = (s i , r i , o i ). Different from the dialogue history scenario of the real experience of the dialogue system, the external knowledge base is an objective fact. Therefore, the dialogue history X = {x 1 , …, x n , $} and the external knowledge base B = {b 1 , L, b l , $} are different types of information. The present invention separates the scenario information in the dialogue process from the semantic knowledge in the external knowledge base based on the classification system of human memory, and stores them separately in the scenario and semantic memory modules.

[0052] Among them, the dialogue history X = {x 1 , …, x n , $} is stored in the scenario memory module, and the storage form refers to the context-aware word granularity dual-channel memory encoding in step S101 and Note that the network parameters in step S102 are not shared with those in step S101, and the two word vector matrices are denoted as and

[0053] Among them, the external knowledge base B = {b 1 , L, b l , $} is stored in the semantic memory module. Using the triple knowledge granularity memory encoding, the triple knowledge granularity dual-channel memory encoding of the external knowledge base is obtained. For each subject-relation-object in each triple knowledge in the external knowledge base B = {b 1 , L, b l , $}, dual-channel word vector mapping is performed, and the dual-channel word vectorized encoding of the triple knowledge granularity is obtained by the way of summation. Randomly initialize two word vector matrices and Among them, V is the dictionary dimension, d is the dimension of the word vector, A S and C S respectively use the normal distribution with a standard deviation of 0.1 and a mean of 0 as the random initialization parameters. For the triple knowledge b i = (s i , r i , o iThe subject relation objects in are respectively subjected to dual-channel word vector mapping, and the dual-channel word vectorized encoding of the triple knowledge granularity is obtained by summing them up. and To simulate the out-of-vocabulary phenomenon, the present invention adopts a random masking strategy to discard some entity words in the triple knowledge.

[0054] Step S103: Under the stimulation of the initial semantic encoding, perform information reasoning on the word granularity memory unit to obtain the encoding vector of the current dialogue task.

[0055] Step S103 includes:

[0056] Figure 2 It is a schematic flow diagram of the information reasoning involved in the embodiments of the present invention, specifically including:

[0057] Sub-step S103a: Based on the stimulation of the initial semantic encoding, use the attention mechanism to activate information in the dual-channel memory encoding of the context-aware word granularity memory unit;

[0058] In the embodiments of the present invention, the dot product attention method is used to calculate the initial semantic encoding In the attention weight of the word granularity memory unit Note See the dual-channel memory encoding of the context-aware word granularity memory unit in step S101; then, under the stimulation of the initial semantic encoding, the activation information of the dual-channel memory encoding of the word granularity memory unit is: Where

[0059] Sub-step S103b: Complete the information reasoning of the word granularity memory unit through the attention mechanism of multiple rounds of iteration to obtain the semantic encoding information of the current dialogue scenario.

[0060] In the embodiments of the present invention, K rounds of iterative information reasoning are performed on the sentence granularity memory unit to obtain the semantic encoding information of the current dialogue scenario, that is, the activation information o of the Kth round K . Among them, in the kth round of information activation,

[0061]

[0062]

[0063] q k+1 = q k + o k

[0064]

[0065]

[0066] Perform information reasoning for K rounds of iteration on the sentence-level memory unit to obtain the semantic encoding information of the current dialogue scenario, that is, the activation information o of the Kth round. K Among them, 1 ≤ k < K, and an independent word vector matrix A is adopted in the kth round of information activation. k and C k The dialogue history is vectorized, and in order to reduce the number of model parameters, the present invention adopts an adjacent weight sharing strategy, making A k +1 = C k C k Adopt a normal distribution with a standard deviation of 0.1 and a mean of 0 as the random initialization parameter.

[0067] Step S104: Based on the encoding vector of the current dialogue task, the attention controller cyclically generates a corresponding query vector for each moment.

[0068] In the embodiment of the present invention, the dialogue scenario is initialized to obtain the situational memory encoding of the dialogue scenario, and the attention controller is used to cyclically generate query vectors for each moment to query the information in the situational memory and semantic memory.

[0069] Propose an intelligent question-answering system with memory as the core. The controller cyclically generates query vectors for each moment to query the information in the situational memory and semantic memory. Specifically, the present invention instantiates the attention controller as a gated recurrent unit model (GRU), where the semantic encoding information o of the current dialogue scenario obtained in step S103 K Initializes GRU, that is, q 0 = o K . At each moment t of dialogue generation, the attention controller generates a query vector q t : where is the word generated by the network at the previous moment, represents its vector encoding, is the word vector matrix of the first layer in the situational memory.

[0070] Step S105: Based on the query vector, activate the information in the situational memory encoding and the semantic memory encoding through working memory to obtain the word table probability distribution, the situational information copy probability distribution, and the semantic knowledge copy probability distribution.

[0071] In the embodiment of the present invention, based on the query control vector, working memory activates the relevant information in the situational memory module and the semantic memory module to obtain three activated probability distributions.

[0072] At each moment t of dialogue generation, based on this query control vector q t, working memory activates relevant information from the two memory modules of episodic memory and semantic memory. Step S105 includes:

[0073] S105a: Based on this query control vector, working memory first activates a vocabulary generation probability

[0074] Calculate the query control vector using dot product attention The attention weight of the word-grained memory unit in episodic memory Attention p i = p t,i Ignore time t; Represents the memory representation of the first layer of x in episodic memory E i of See the dual-channel memory encoding of the context-aware word-grained memory unit in step S101; then under the stimulation of the initial semantic encoding, the activation information of the dual-channel memory encoding of the word-grained memory unit is: Where

[0075] Then concatenate the query control vector and the activation information and map it linearly to the vocabulary size, and finally normalize it through softmax to the vocabulary probability distribution This probability distribution determines the probability of generating a word at the current time t of Where represents the mapping matrix.

[0076] S105b: Based on this query control vector working memory activates relevant information from episodic memory, that is, the probability distribution P of copying words from the dialogue context E·ptr :

[0077] Similar to the process in step S103b, at each moment during the decoding process, the present invention makes to obtain the probability distribution P of copying words from the dialogue context E·ptr :

[0078]

[0079]

[0080]

[0081]

[0082]

[0083]

[0084] S105b: Based on this query control vector and the scenario memory activation vector Working memory activates relevant information from semantic memory, that is, copies the probability distribution P of entity words from triple semantic knowledge S·ptr :

[0085] Similar to the process of step S105b, at each moment during the decoding process, the present invention makes Copy the probability distribution P of entity words from triple semantic knowledge S·ptr :

[0086]

[0087]

[0088]

[0089]

[0090]

[0091]

[0092] Step S106: Select the word to be generated currently from the probability distributions of the three pieces of information based on the rule system to form an answer

[0093] In the embodiment of the present invention, a rule system is preset in advance, and the currently generated vocabulary is selected from the activated vector content based on the rule system. The working memory activates three distributions P E·ptr and P S·ptr . The rule system selects the currently generated vocabulary from the three activated distributions and uses cross-entropy to supervise the training model

[0094] Sub-step S106a:

[0095] The preset rules include: Rule 1: The default behavior of the model during generation is to copy, that is, copy words or entities from P E·ptr and P S·ptr . When the word in the dialogue reply appears in both the dialogue history and the knowledge base, select the word with the highest probability from P E·ptr and P S·ptr as the word generated at the current moment

[0096] Rule 2: When the word in the dialogue reply appears in the dialogue history or the knowledge base, the model selects the corresponding P E·ptr or PS·ptr Select the word with the highest probability from among them as the word generated at the current moment At the same time, let the remaining P S·ptr Or P E·ptr Point to the "sentinel" position.

[0097] Rule 3: If the word in the dialogue reply is not in the dialogue history and the knowledge base, the word Is generated from the vocabulary probability Among them, and P E·ptr And P S·ptr Point to the "sentinel" position.

[0098] Sub-step S106b:

[0099] In order to ensure that the model can learn the behavior in sub-step S106a, the present invention uses cross-entropy supervised training of the model.

[0100]

[0101] And Are the cross-entropy of generating the current word from the vocabulary and the cross-entropy of copying the current word from the dialogue history and the semantic knowledge base respectively. T is the length of the dialogue reply.

[0102] The intelligent question-answering method provided by the present invention uses two types of context-aware transformations to incorporate context information into the word representations in the dialogue scenario; stores the dialogue scenario and the semantic information of the knowledge base separately; understands the user's current statement through a memory network and activates the current dialogue generation task; the working memory dynamically controls the "activation" of the long-term memory, reads the content related to the current task from the long-term memory into the short-term storage, and then generates the current reply word by word based on a heuristic strategy. By storing the dialogue scenario and the semantic information of the knowledge base separately and using the information stored in these two ways through memory reasoning, the external knowledge base information can be efficiently utilized, greatly improving the performance of task-oriented dialogue generation.

[0103] Figure 3 Is a schematic structural diagram of an intelligent question-answering device provided by an embodiment of the present invention, as Figure 3 Shown, the device specifically includes:

[0104] An encoding module 31, configured to perform dual-channel memory encoding at the word granularity on the historical dialogue information, and perform context-aware transformation to obtain a dialogue scenario memory encoding;

[0105] The encoding module is further configured to perform dual-channel memory encoding at the triple granularity on the database semantic knowledge outside the dialogue history to obtain a semantic memory encoding;

[0106] The inference module 32 is used to perform information inference on the word-level memory unit under the stimulation of the initial semantic encoding to obtain the encoding vector of the current dialogue task;

[0107] The query module 33 is used to cyclically generate corresponding query vectors for each moment by the attention controller based on the encoding vector of the current dialogue task;

[0108] The query module 33 is further used to activate the information in the episodic memory encoding and the semantic memory encoding based on the query vector through working memory to obtain the vocabulary probability distribution, the episodic information copy probability distribution, and the semantic knowledge copy probability distribution;

[0109] The query module 33 is further used to select the word to be generated currently from the probability distributions of the three pieces of information based on the rule system to form an answer.

[0110] In a possible implementation manner, the encoding module 31 is used to perform word-level memory encoding on the historical dialogue information according to the context-aware transformation to obtain the context-aware word-level dual-channel memory encoding;

[0111] In a possible implementation manner, the encoding module 31 is used to add the dialogue participant attribute information and the moment information to the historical dialogue information to obtain an information set X = {x 1 ,…,x n ,$}, where n is the maximum time series length of the information set, and two word vector matrices are randomly initialized and V is the dictionary dimension, and d is the dimension of the word vector;

[0112] The dual-channel vectorized encoding corresponding to x i is and

[0113] In a possible implementation manner, the encoding module 31 is based on the following transformation:

[0114]

[0115]

[0116] to obtain the context-aware word-level dual-channel memory encoding as and

[0117] In a possible implementation manner, the inference module 32 is specifically used to complete the information inference of the word-level memory unit through the multi-round iterative attention mechanism under the stimulation of the initial semantic encoding to obtain the encoding vector of the current dialogue task.

[0118] The intelligent question-answering device provided in this embodiment may be a device as shown in Figure 3 and can execute all steps of the intelligent question-answering method as shown in Figure 1 to achieve the technical effects of the intelligent question-answering method as shown in Figure 1 . For specific reference, please refer to Figure 1 the relevant description. For the sake of concise description, it will not be elaborated here.

[0119] Figure 4 FIG. is a schematic structural diagram of a device provided in an embodiment of the present invention. Figure 4 The device 400 shown includes: at least one processor 401, a memory 402, at least one network interface 404, and other user interfaces 403. Each component in the device 400 is coupled together through a bus system 405. It can be understood that the bus system 405 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 405 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 4 all kinds of buses are labeled as the bus system 405.

[0120] Among them, the user interface 403 may include a display, a keyboard, or a pointing device (for example, a mouse, a trackball, a touchpad, or a touch screen, etc.).

[0121] It can be understood that the memory 402 in the embodiments of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synch link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). The memory 402 described herein is intended to include but not be limited to these and any other suitable types of memory.

[0122] In some embodiments, the memory 402 stores the following elements, executable units, or data structures, or subsets thereof, or extended sets thereof: the operating system 4021 and the application program 4022.

[0123] Among them, the operating system 4021 includes various system programs, such as the framework layer, the core library layer, the driver layer, etc., and is used to implement various basic services and process hardware-based tasks. The application program 4022 includes various application programs, such as a media player and a browser, etc., and is used to implement various application services. The program for implementing the method of the embodiments of the present invention can be included in the application program 4022.

[0124] In the embodiments of the present invention, by invoking the programs or instructions stored in the memory 402, specifically, the programs or instructions stored in the application program 4022, the processor 401 is used to execute the method steps provided in each method embodiment, for example, including:

[0125] Perform dual-channel memory encoding at the word granularity on historical dialogue information, and perform context-aware transformation to obtain dialogue scenario memory encoding;

[0126] Perform dual-channel memory encoding at the triple granularity on database semantic knowledge outside the dialogue history to obtain semantic memory encoding;

[0127] Under the stimulation of the initial semantic encoding, perform information reasoning on the word granularity memory unit to obtain the encoding vector of the current dialogue task;

[0128] Based on the encoding vector of the current dialogue task, the attention controller cyclically generates corresponding query vectors for each moment;

[0129] Based on the query vector, activate the information in the scenario memory encoding and the semantic memory encoding through working memory to obtain the vocabulary probability distribution, the scenario information copy probability distribution, and the semantic knowledge copy probability distribution;

[0130] Select the word to be generated currently from the probability distributions of the three pieces of information based on the rule system to form an answer.

[0131] In a possible implementation manner, perform word granularity memory encoding on historical dialogue information according to context-aware transformation to obtain context-aware word granularity dual-channel memory encoding.

[0132] In a possible implementation manner, add the dialogue participant attribute information and the moment information to the historical dialogue information to obtain an information set X = {x 1 ,…,x n ,$}, where n is the maximum time series length of the information set, and randomly initialize two word vector matrices and V is the dictionary dimension, and d is the dimension of the word vector;

[0133] The dual-channel vectorized encoding corresponding to x i is and

[0134] In a possible implementation manner, based on the following transformation:

[0135]

[0136]

[0137] Obtain the context-aware word granularity dual-channel memory encoding as and

[0138] In a possible implementation, under the stimulation of initial semantic encoding, the information reasoning of the word-level memory unit is completed through multiple rounds of iterative attention mechanisms to obtain the encoding vector of the current dialogue task.

[0139] In a possible implementation, the answer model is trained using cross-entropy supervision.

[0140] The method disclosed in the above embodiments of the present invention can be applied to or implemented by the processor 401. The processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method can be completed by the integrated logic circuit in the hardware of the processor 401 or instructions in the form of software. The above-mentioned processor 401 may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed by the hardware decoding processor, or executed by a combination of the hardware and software units in the decoding processor. The software unit may be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory 402, and the processor 401 reads the information in the memory 402 and combines its hardware to complete the steps of the above method.

[0141] It can be understood that the embodiments described herein can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in this application, or a combination thereof.

[0142] For software implementation, the technologies described herein can be implemented by units that execute the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented within the processor or external to the processor.

[0143] The device provided in this embodiment can be a device as shown in Figure 4 and can execute all steps of an intelligent question-answering method as shown in Figure 1 to achieve the technical effects of an intelligent question-answering method as shown in Figure 1 . For specific reference, please refer to Figure 1 the relevant description. For the sake of brevity, it will not be elaborated here.

[0144] The embodiment of the present invention also provides a storage medium (computer-readable storage medium). The storage medium stores one or more programs. Among them, the storage medium can include volatile memory, such as random access memory; the memory can also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; the memory can also include a combination of the above types of memory.

[0145] When one or more programs in the storage medium can be executed by one or more processors, the intelligent question-answering method executed on the device side as described above can be implemented.

[0146] The processor is used to execute the control program stored in the memory to implement the following steps of the intelligent question-answering method executed on the device side:

[0147] Perform two-channel memory encoding at the word granularity on the historical conversation information and perform context-aware transformation to obtain the conversation scenario memory encoding;

[0148] Perform two-channel memory encoding at the triple granularity on the database semantic knowledge outside the dialogue history to obtain semantic memory encoding;

[0149] Under the stimulation of the initial semantic encoding, perform information reasoning on the word granularity memory unit to obtain the encoding vector of the current dialogue task;

[0150] Based on the encoding vector of the current dialogue task, the attention controller cyclically generates corresponding query vectors for each moment;

[0151] Based on the query vector, activate the information in the episodic memory encoding and the semantic memory encoding through working memory to obtain the vocabulary probability distribution, the episodic information copy probability distribution, and the semantic knowledge copy probability distribution;

[0152] Select the word to be generated currently from the probability distributions of the three pieces of information based on the rule system to form an answer.

[0153] In a possible implementation, perform word granularity memory encoding on the historical dialogue information according to context-aware transformation to obtain context-aware word granularity two-channel memory encoding.

[0154] In a possible implementation, add the dialogue participant attribute information and the moment information to the historical dialogue information to obtain an information set X = {x 1 ,…,x n ,$}, where n is the maximum time series length of the information set, and randomly initialize two word vector matrices and V is the dictionary dimension, and d is the dimension of the word vector;

[0155] The two-channel vectorized encoding corresponding to x i is and

[0156] In a possible implementation, based on the following transformation:

[0157]

[0158]

[0159] Obtain the context-aware word granularity two-channel memory encoding as and

[0160] In a possible implementation, under the stimulation of the initial semantic encoding, complete the information reasoning of the word granularity memory unit through the attention mechanism of multiple rounds of iteration to obtain the encoding vector of the current dialogue task.

[0161] In a possible implementation, the answer model is trained using cross-entropy supervision.

[0162] Those skilled in the art should also be able to further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0163] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be implemented by hardware, software modules executed by a processor, or a combination of the two. The software modules can be placed in a random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0164] The specific implementation manners described above further elaborate on the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only the specific implementation manners of the present invention and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent question-answering method, characterized in that, comprising: Performing dual-channel memory encoding at the word granularity on historical dialogue information, and performing context-aware transformation to obtain a dialogue scenario memory encoding; Performing dual-channel memory encoding at the triple granularity on database semantic knowledge outside the dialogue history to obtain a semantic memory encoding; Under the stimulation of the initial semantic encoding, performing information reasoning on the word granularity memory unit to obtain an encoding vector for the current dialogue task; Based on the encoding vector of the current dialogue task, the attention controller cyclically generates a corresponding query vector for each moment; Based on the query vector, activating the information in the scenario memory encoding and the semantic memory encoding through working memory to obtain a word list probability distribution, a scenario information copy probability distribution, and a semantic knowledge copy probability distribution; Selecting the word to be generated currently from the probability distributions of the three pieces of information based on a rule system to form an answer.

2. The method according to claim 1, characterized in that, The performing dual-channel memory encoding at the word granularity on historical dialogue information, and performing context-aware transformation to obtain a dialogue scenario memory encoding includes: Performing word granularity memory encoding on historical dialogue information according to context-aware transformation to obtain a context-aware word granularity dual-channel memory encoding.

3. The method according to claim 2, characterized in that, The performing dual-channel memory encoding at the word granularity on historical dialogue information, and performing context-aware transformation to obtain a dialogue scenario memory encoding includes: Add the interlocutor attribute information and the time information to the historical dialogue information to obtain an information set , where is the symbol finally added to the information set, is the maximum time series length of the information set, and randomly initialize two word vector matrices and , is the dictionary dimension, is the dimension of the word vector, and respectively adopt the normal distribution with a standard deviation of 0.1 and a mean of 0 as the random initialization parameters; The corresponding dual-channel vectorized encoding is and , where the is a word in the information set.

4. The method according to claim 1, characterized in that, The performing information reasoning on the word granularity memory unit under the stimulation of the initial semantic encoding to obtain an encoding vector for the current dialogue task includes: Under the stimulation of the initial semantic encoding, completing the information reasoning of the word granularity memory unit through a multi-round iterative attention mechanism to obtain an encoding vector for the current dialogue task.

5. The method according to claim 1, characterized in that, The method further includes: Using cross-entropy supervision to train the answer model.

6. An intelligent question-answering device, characterized in that, comprising: An encoding module for performing dual-channel memory encoding at the word granularity on historical dialogue information, and performing context-aware transformation to obtain a dialogue scenario memory encoding; The encoding module is further configured to perform dual-channel memory encoding at the triple granularity on database semantic knowledge outside the dialogue history to obtain a semantic memory encoding; An inference module for performing information reasoning on the word granularity memory unit under the stimulation of the initial semantic encoding to obtain an encoding vector for the current dialogue task; A query module, based on the encoding vector of the current dialogue task, the attention controller cyclically generates a corresponding query vector for each moment; The query module is further configured to activate the information in the scenario memory encoding and the semantic memory encoding through working memory based on the query vector to obtain a word list probability distribution, a scenario information copy probability distribution, and a semantic knowledge copy probability distribution; The query module is further configured to select the word to be generated currently from the probability distributions of the three pieces of information based on a rule system to form an answer.

7. The device according to claim 6, characterized in that, comprising: The encoding module is further configured to perform word-level memory encoding on the historical conversation information according to the context-aware transformation, so as to obtain context-aware word-level dual-channel memory encoding.

8. An electronic device, characterized in that it includes: a processor and a memory, where the processor is configured to execute an intelligent answering program stored in the memory to implement the intelligent question-answering method according to any one of claims 1 to 5.

9. A storage medium, characterized in that the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the intelligent question-answering method according to any one of claims 1 to 5.

Images