Model apparatus and method for guiding a dialogue to a complex semantic goal based on a dynamic knowledge graph

Abstract

The application discloses a model device and method for guiding a dialogue to a complex semantic target based on a dynamic knowledge graph, the model comprising a dynamic graph construction module, a path selection module and a dialogue generation module; the dynamic graph construction module is used for dynamically constructing a dynamic knowledge graph required by a target sentence guiding task in each dialogue, comprising a vertex selection module and an edge construction module; the vertex selection module is used for selecting concept words with diversified semantic ranges and expression forms and related to a context as vertices in the dynamic knowledge graph during dynamic graph construction; the edge construction module is used for connecting the vertices obtained by the vertex selection module in the dynamic knowledge graph; the path selection module is used for finding a plurality of connected paths capable of connecting two topic sentences in the constructed dynamic knowledge graph, and screening and selecting a logical transfer path finally used for reply generation; and the dialogue generation module is used for generating a final transfer reply in combination with the selected logical transfer path and the two topic sentences.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, computer natural language processing, and dialogue systems, and specifically relates to a model and method for guiding dialogue to complex semantic goals based on dynamic knowledge graphs. Background Technology

[0002] With the development of computer technology, interactions between humans and machines are becoming increasingly frequent. Among various human-computer interaction methods, enabling machines to converse with humans in natural language (human language) is an important and indispensable approach, and also one of the key goals of artificial intelligence. To achieve this goal, it is essential to develop sufficiently intelligent human-computer dialogue systems. Based on the dialogue scenario, human-computer dialogue systems are mainly divided into three categories: open-domain dialogue systems, task-oriented dialogue systems, and goal-oriented dialogue systems. Open-domain dialogue systems can engage users in fun conversations, typically requiring fluent and natural responses, such as Microsoft's Xiaoice. Task-oriented dialogue systems focus on understanding user intent and collecting information to complete tasks. These systems are primarily implemented using slot filling, guiding users to provide input related to task information slots (such as hotel name, check-in time, and check-out time in hotel bookings) to complete corresponding dialogue tasks. This can replace humans in performing basic repetitive tasks, saving labor costs. Goal-oriented dialogue systems aim to guide conversations flexibly, naturally, and efficiently towards a preset goal through multi-turn dialogues. The goal can be a keyword, knowledge point, or product. This is widely used in e-commerce sales, dialogue recommendations, psychotherapy, and education.

[0003] In recent years, many studies have focused on how to enable chatbots to actively guide conversations toward a specific goal. As a typical simplified task in this area, keyword-guided dialogue studies how to smoothly and quickly guide conversations toward given keywords within a multi-turn dialogue setting. Carnegie Mellon University first defined the framework for this task as two stages: predicting the next round of keywords and retrieving responses based on those keywords. Researchers at Tianjin University added global planning to the keyword prediction step and applied the generation method to the response. Carnegie Mellon University also pioneered a goal-oriented task using semantically more complex sentences as targets and proposed a method for generating multi-hop knowledge paths between a pair of concepts using a language model, achieving state-of-the-art performance on this task. Heriot-Watt University studied how to achieve smooth topic transitions within a single turn of dialogue and proposed an open-source dataset for goal-oriented dialogue, which will be used in this technical approach. However, existing methods suffer from poor performance when current pre-trained models require common-sense knowledge for response generation; furthermore, path retrieval methods are limited by the incompleteness of existing knowledge graphs, resulting in poor controllability of path generation methods. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a goal-oriented human-computer dialogue model device and method that can guide dialogue to preset topic sentences. Specifically, it is a model device and method that guides dialogue to complex semantic goals based on dynamic knowledge graphs. This application combines knowledge retrieval technology and relation prediction technology to obtain a dynamic knowledge graph that matches the current dialogue context and contains common sense logic. Then, the model searches for a connecting path that can connect two topic sentences in the dynamic graph, and finally uses the path to generate a transition response.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A model device for guiding dialogue to complex semantic goals based on dynamic knowledge graphs includes a dynamic graph construction module, a path selection module, and a dialogue generation module;

[0007] The dynamic graph construction module is used to dynamically construct the dynamic knowledge graph required for the target sentence-oriented task in each dialogue, including a vertex selection module and an edge construction module. The vertex selection module is used to select concept words with diverse semantic ranges and expressions and relevant to the context as vertices in the dynamic knowledge graph during dynamic graph construction. The edge construction module is based on the link prediction technology in the knowledge graph completion research, which trains a relation prediction model and a relation discriminator to connect the vertices obtained in the vertex selection module in the dynamic knowledge graph.

[0008] The path selection module is used to search for several connecting paths that can connect two topic sentences in the constructed dynamic knowledge graph and filter them to select the logical transition path that is finally used for response generation.

[0009] The dialogue generation module is used to generate the final transition response by combining the selected logical transition path and two topic sentences.

[0010] This invention also provides a model training method based on dynamic knowledge graphs to guide dialogues toward complex semantic goals, comprising the following steps:

[0011] S1. Combine the "source bag of keywords" and "target bag of keywords" with the random walk algorithm to obtain the initial candidate space of vertices; use the path routing method to perform preliminary screening of candidate vertices, retaining the top K1 concept words. ; Use the corpus to train the vertex prediction model required for the vertex selection module. Once the vertex prediction model converges, for A second round of screening was conducted, retaining the top K2 concept terms. Ultimately, together with the key phrases in the source topic sentence and the target topic sentence, they form the vertices in the dynamic graph;

[0012] S2. Reduce and enhance the training data required for the edge construction module; train the relation prediction model and relation discriminator required for the edge construction module. After the relation prediction model converges, predict the relations between each pair of vertices in the dynamic knowledge graph, and discriminate the predicted relations. Finally, retain the edges in the dynamic knowledge graph whose discriminator judgment result is 1. Thus, the dynamic graph construction module is completed.

[0013] S3. Starting with the key phrases in the source topic sentence and ending with the key phrases in the target topic sentence, find three or fewer connected paths in the dynamic knowledge graph;

[0014] S4. Train the dialogue generation module. The dialogue generation module is based on the pre-trained language model GPT2-small. The input of this module is the concatenation of the source topic sentence, the target topic sentence, and the knowledge path. The output is the transition response. The training process is optimized by minimizing the log-likelihood loss.

[0015] This invention also provides a prediction method for guiding dialogue to complex semantic goals based on dynamic knowledge graphs, comprising:

[0016] (1) Use the vertex selection module and edge construction module to construct a dynamic knowledge graph that matches the test cases;

[0017] (2) Find connected paths in the dynamic knowledge graph and filter out the path with the lowest ppl;

[0018] (3) Send the path obtained in step (2) into the dialogue generation module to obtain the final transfer response.

[0019] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0020] 1. This invention employs a method of first acquiring common-sense logical transition paths and then using them for dialogue generation, resulting in responses containing appropriate transition logic and rich content. By integrating the nodes and relationships in the path into the input of the language model, compared to general end-to-end language models that do not incorporate knowledge, and generation methods based solely on keywords without incorporating logical relationships between keywords, the responses can contain richer information.

[0021] 2. In the dynamic graph construction module, this invention first selects vertices by reasonably filtering information in the original knowledge graph, and then dynamically constructs edges through link prediction technology. This approach combines the reliability advantages of static graphs with the flexibility advantages of dynamic generation. Compared with directly searching for knowledge from static graphs, this method is less restricted by the incompleteness of knowledge in existing knowledge graphs, and the obtained paths contain fewer intermediate interference nodes. Compared with paths generated directly using language models, the paths obtained by this method contain less context-irrelevant and low-reliability knowledge, and are more controllable.

[0022] 3. This invention targets topic sentences, which is of greater practical significance than earlier work that targeted single keywords. At the application level, it provides new technologies and models for social media opinion guidance, e-commerce dialogue recommendation, and other fields, and has good theoretical significance and commercial promotion value. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall structure of the model of the present invention that guides dialogue to complex semantic goals based on dynamic knowledge graphs.

[0024] Figure 2 This is a structural diagram of the dynamic graph building module.

[0025] Figure 3 This is a comparison chart of experimental results obtained using the method of this invention and other methods. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0027] like Figure 1 This embodiment provides a model device for guiding dialogue to complex semantic targets based on dynamic knowledge graphs, including a dynamic graph construction module, a path selection module, and a dialogue generation module.

[0028] The dynamic graph construction module is a key module of this invention, such as... Figure 2 The diagram includes a vertex selection module and an edge construction module, used to build a dynamic knowledge graph that conforms to the dialogue context and contains common-sense logic. The path selection module is used to find several multi-hop connected paths in the constructed dynamic knowledge graph and filter them according to rules. The dialogue generation module is used to generate the final transition response based on the source topic sentence, the target topic sentence, and the filtered common-sense logic transition paths.

[0029] In this embodiment, the vertex selection module in the dynamic graph construction module follows these steps:

[0030] (1) Extract the source key phrase bag and the target key phrase bag based on the source topic sentence and the target topic sentence. This includes key phrases in the topic sentence and key phrases in the knowledge graph that are adjacent to it and have the most similar semantics. This step is to expand the candidate space of vertices and ensure the diversity of vertices in the dynamic knowledge graph in terms of semantic range and expression.

[0031] (2) Path routing filtering.

[0032] ① Starting with each phrase in the source key phrase bag and ending with each phrase in the target key phrase bag, a path is found in the knowledge graph (ConceptNet). All nodes in the path constitute the initial vertex candidate space.

[0033] ② Calculate the score for each path obtained in ① The calculation method is based on the semantic similarity between the path after it is converted into natural language form and the topic sentence, using SentenceTransformer.

[0034] ③ After obtaining the path score, calculate the route score for each candidate word by averaging the scores of all paths passing through that node.

[0035]

[0036] Among them For the nodes passed through in the path obtained in ① The path. Finally, retain the top K1 nodes with the highest s(c) scores. This step aggregates path scores onto relevant nodes, ensuring the contextual relevance of candidate vertices.

[0037] (3) Vertex prediction model. Each concept representation is queried using sentence representations via attention dot products, and concept words from real responses in the corpus are used as labels for supervised training. Finally, the top K2 nodes with the highest probabilities are selected based on the prediction results of the vertex prediction model, and together with the key phrases in the topic sentence, they are used as vertices of the dynamic knowledge graph.

[0038] In this embodiment, the edge construction module in the dynamic graph construction module takes the following steps:

[0039] The edge building module includes a relation prediction model and a relation discriminator.

[0040] (1) Modify the training data of the relation prediction model: only retain the 15 relations that are most meaningful for semantic transfer out of the original 37 relations; extract concept word pairs that are not directly connected in the knowledge graph (ConceptNet) from the corpus based on co-occurrence frequency, add them to the relation prediction training data and define the relation as "DialogAct". Fine-tune the pre-trained model DistilBERT as the relation prediction model in the form of a multi-classification task.

[0041] (2) Constructing training data for the relation discriminator: Using the training data of the relation prediction model as positive samples, negative samples are constructed by replacing the relations and tail nodes in the knowledge triples of the initial training data. The pre-trained model DistilBERT is fine-tuned and trained as the relation discriminator in the form of a binary classification task.

[0042] (3) Input each pair of vertices in the dynamic knowledge graph into the relation prediction model, and connect the vertices using the predicted relations; then input the newly obtained triples into the relation discriminator, and the edges with the final discrimination result of 1 are retained in the dynamic knowledge graph.

[0043] In this embodiment, the path selection module steps are as follows:

[0044] (1) Starting with key phrases in the source topic sentence and ending with key phrases in the target topic sentence, search for several multi-hop connected paths in the dynamic knowledge graph. Then, perform the following two steps for filtering:

[0045] ① Remove paths where the overlap rate of node words is greater than a specified threshold.

[0046] ② Convert the path into natural language form, calculate its PPL index, and finally select the one with the lowest PPL for response generation.

[0047] In this embodiment, the dialogue generation module steps are as follows:

[0048] (1) The source topic sentence, the transfer path, and the target topic sentence are concatenated in the form of '[context] source topic sentence [target] target topic sentence [knowledge] knowledge path [response]'.

[0049] (2) The above spliced ​​results are sent to the generation model for response generation.

[0050] Following the implementation steps above, in the vertex selection module, K1=100 and K2=20; in the edge construction module, the Adam optimizer was used with an initial learning rate of 0.00001 and a single sample pass count of 64; in the dialogue generation module, the Adam optimizer was used with an initial learning rate of 0.00002 and a single sample pass count of 32. The trained model was tested on the publicly available OTTers dialogue dataset, and the specific experimental results are as follows. Figure 3 As shown, the dialogue has shown significant improvement in both reference-based and model-based evaluation metrics.

[0051] This invention is not limited to the embodiments described above. The above description of specific embodiments is intended to illustrate and explain the technical solutions of this invention. The specific embodiments described above are merely illustrative and not restrictive. Without departing from the spirit and scope of the claims, those skilled in the art can make many specific modifications based on the teachings of this invention, and these modifications all fall within the scope of protection of this invention.

Claims

1. A model apparatus for guiding a dialogue to a complex semantic goal based on a dynamic knowledge graph, characterized by, The system comprises a dynamic graph construction module, a path selection module and a dialogue generation module. The dynamic graph construction module is used for dynamically constructing a dynamic knowledge graph required by a target sentence-oriented task in each dialogue, and comprises a vertex selection module and an edge construction module. The vertex selection module is used for selecting concept words with diversified semantic ranges and expression forms and related to a context as vertices in the dynamic knowledge graph during dynamic graph construction. The edge construction module is used for connecting the vertices obtained by the vertex selection module in the dynamic knowledge graph.

2. A model training method for guiding a dialogue to a complex semantic target based on a dynamic knowledge graph, characterized in that, The path selection module is used for finding a plurality of connected paths connecting two topic sentences in the constructed dynamic knowledge graph and performing screening to select a logical transition path used for generating a final reply. The dialogue generation module is used for generating a final transition reply in combination with the selected logical transition path and the two topic sentences. The candidate vertices are preliminarily screened using a path routing method, and topK1 concept words are reserved ; The vertex prediction model required for training the vertex selection module using the corpus, when the vertex prediction model converges, is used to perform a second screening, and keep topK2 concept words Finally, together with the key phrases in the source topic sentence and the target topic sentence, as vertices in the dynamic graph. The system comprises the following steps: S1. An initial candidate space of vertices is obtained in combination with a "source key phrase bag", a "target key phrase bag" and a random walk algorithm. S2. Training data required by the edge construction module is reduced and enhanced. A relationship prediction model and a relationship discriminator required by the edge construction module are trained. After the relationship prediction model converges, relationships between vertices in the dynamic knowledge graph are predicted two by two, and the predicted relationships are discriminated.

3. A prediction method for guiding a dialogue to a complex semantic target based on a dynamic knowledge graph, based on the model for guiding a dialogue to a complex semantic target based on a dynamic knowledge graph according to claim 1, characterized in that, Finally, edges with a discriminator discrimination result of 1 are retained in the dynamic knowledge graph. Thus, the dynamic graph construction module is completed. S3. Three or fewer connected paths are found in the dynamic knowledge graph with key phrases in a source topic sentence as a starting point and key phrases in a target topic sentence as a terminal point. S4. The dialogue generation module is trained. The dialogue generation module is based on a pre-trained language model GPT2-small. The input of the module is a splicing of a source topic sentence, a target topic sentence and a knowledge path, and the output is a transition reply. The training process is optimized by minimizing a log-likelihood loss. The system comprises the following steps: (1) The vertex selection module and the edge construction module are used to construct a dynamic knowledge graph matched to a test case. (2) Connected paths are found in the dynamic knowledge graph, and a path with the lowest ppl is selected. (3) The path selected in step (2) is sent to the dialogue generation module to obtain a final transition reply.

Images