Question answering method, device, equipment and storage medium based on retrieval enhancement

Abstract

The present invention relates to the field of human-computer dialogue technology, and discloses a question-answering method, device, equipment and storage medium based on retrieval enhancement. The question-answering method based on retrieval enhancement includes: constructing a subgraph containing all entities and entity associations related to the original question; applying the subgraph to a graph neural network based on an attention mechanism for multiple rounds of reasoning to obtain a final representation of each node rich in context; identifying several target nodes most relevant to the original question from the final representation of all nodes in the subgraph as candidate answers, extracting the shortest path from each entity in the original question to each candidate answer from the subgraph and converting it into a path description in natural language; inputting the original question and path description into a large language model to perform a retrieval enhancement generation task, and outputting the final answer corresponding to the original question. The present invention solves the limitations of a single model in complex knowledge reasoning and natural language understanding, and improves the question-answering performance.

Description

Technical Field

[0001] The present invention relates to the field of human-computer dialogue technology, and in particular to a question-answering method, device, equipment and storage medium based on retrieval enhancement. Background Art

[0002] Knowledge graph question answering is a technology that uses knowledge graphs to answer natural language questions. This technology semantically understands and parses the user's natural language questions, and then uses the entities, relationships, and attributes in the knowledge graph to query and reason, and finally gives an accurate answer. Traditional knowledge graph question answering technologies mainly include methods based on semantic parsing and information retrieval methods. Methods based on semantic parsing may have advantages in accuracy, but they face problems such as high cost and insufficient generalization ability; while information retrieval methods are flexible and easy to implement, they are somewhat insufficient when dealing with complex logic and deep reasoning tasks. Summary of the invention

[0003] The main purpose of the present invention is to provide a question-answering method, apparatus, device and storage medium based on retrieval enhancement, aiming to solve the technical problem of how to improve the performance of knowledge graph question-answering.

[0004] A first aspect of the present invention provides a question-answering method based on retrieval enhancement, the question-answering method based on retrieval enhancement comprising:

[0005] Obtain the original question asked by the user in natural language;

[0006] Construct a subgraph containing all entities and entity associations related to the original question;

[0007] Taking the subgraph as a basic unit for graph neural network processing, an initial feature vector is assigned to each node in the subgraph;

[0008] Input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on the attention mechanism for multiple rounds of reasoning to obtain a final representation of each node rich in context;

[0009] identifying a number of target nodes that are most relevant to the original question from the final representation of all nodes of the subgraph;

[0010] Taking the entities corresponding to each target node as candidate answers to the original question, and extracting the shortest paths from each entity in the original question to each candidate answer;

[0011] Convert each extracted shortest path into a path description using natural language;

[0012] The original question and the path description are input into a large language model to perform a retrieval enhancement generation task, and a final answer corresponding to the original question is output.

[0013] Optionally, in a first implementation of the first aspect of the present invention, constructing a subgraph including all entities and entity association relationships related to the original question includes:

[0014] The original question is parsed using natural language processing technology, and several key elements of the original question are extracted from the parsed results;

[0015] Based on each of the key elements, forwardly traverse the preset knowledge graph to obtain a number of first key information directly related to the original question;

[0016] Based on each of the first key information, construct a forward subgraph including all entities and entity association relationships directly related to the original question;

[0017] Based on each entity in the forward subgraph, traverse the knowledge graph in reverse order to obtain a number of second key information directly related to each entity;

[0018] Based on each of the second key information, construct a reverse subgraph including all entities and entity association relationships indirectly related to the original question;

[0019] The forward subgraph and the reverse subgraph are structurally fused to obtain a bidirectional fused subgraph.

[0020] Optionally, in a second implementation of the first aspect of the present invention, inputting the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism for multiple rounds of reasoning to obtain a final context-rich representation of each node includes:

[0021] Input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism;

[0022] By using the graph neural network, the initial feature vector corresponding to each node in the subgraph is converted into a continuous vector space representation to obtain an embedding vector of each node and its neighboring nodes in the subgraph;

[0023] Calculate the dot product of the embedding vector between each node and its corresponding neighbor nodes respectively to obtain the original attention score between the corresponding neighbor nodes;

[0024] Normalize the attention scores of all neighbor nodes, and use the normalized attention scores to perform weighted summation on the Value vectors of all neighbor nodes to obtain the final context-rich representation of each node after update;

[0025] Each attention layer of the graph neural network receives the node representation output by the previous attention layer as input, and outputs an updated node representation.

[0026] Optionally, in a third implementation of the first aspect of the present invention, identifying a number of target nodes most relevant to the original question from the final representation of all nodes of the subgraph includes:

[0027] Using a language model to encode the keywords and entities in the original question into vector representations, and concatenating the vector representations of the keywords and entities as the final representation of the original question;

[0028] Using the attention mechanism, the first attention weight of the original question representation is calculated using a learnable first parameter matrix, and the second attention weight of each node representation is calculated using a learnable second parameter matrix;

[0029] Based on the first attention weight and the second attention weight, using a dot product attention function to respectively calculate a similarity score between the original question representation and each node representation;

[0030] According to the similarity scores corresponding to the nodes, a preset number of nodes with the highest scores are selected as target nodes most relevant to the original question.

[0031] Optionally, in a fourth implementation of the first aspect of the present invention, taking the entities corresponding to each of the target nodes as candidate answers to the original question, and extracting the shortest paths from each entity in the original question to each of the candidate answers includes:

[0032] List all nodes and all edges in the subgraph to obtain a node list and an edge list, wherein each edge represents a relationship between two nodes;

[0033] Constructing an adjacency matrix according to the node list and the edge list;

[0034] In the adjacency matrix, the entity of the original question is set as the starting point and the candidate answer is set as the end point, and a queue is initialized to store the current node and the path from the starting point to the current node;

[0035] The adjacency matrix is ​​traversed using a breadth-first search algorithm until an end point is found or all nodes are traversed, and the shortest path from each entity in the original question to each candidate answer is output.

[0036] Optionally, in a fifth implementation of the first aspect of the present invention, converting the extracted shortest paths into path descriptions in natural language includes:

[0037] Obtaining the natural language meaning corresponding to each node and each edge in the subgraph, and generating labels corresponding to each node and each edge based on the natural language meaning;

[0038] Based on the order of each node and each edge in the corresponding shortest path, the labels corresponding to each node and each edge are used to construct a corresponding natural language description as the path description corresponding to each of the shortest paths.

[0039] Optionally, in a sixth implementation manner of the first aspect of the present invention, inputting the original question and the path description into a large language model to perform a retrieval enhancement generation task, and outputting a final answer corresponding to the original question includes:

[0040] Inputting the original question and the path description into a large language model;

[0041] Using a large language model, key information in the original question is identified, and the key information is integrated with the path description to obtain nodes and edges to be retrieved;

[0042] Through the large language model, the external knowledge graph is retrieved to obtain detailed information related to the nodes and edges to be retrieved;

[0043] The retrieved detailed information is used as the context information of the original question and the path description, and a final answer corresponding to the original question is generated and output through a large language model.

[0044] The second aspect of the present invention further provides a question-answering device based on retrieval enhancement, the question-answering device based on retrieval enhancement comprising:

[0045] An acquisition module is used to acquire the original question raised by the user in natural language;

[0046] A construction module, used to construct a subgraph including all entities and entity association relationships related to the original problem;

[0047] An initialization module, used to use the subgraph as a basic unit for graph neural network processing and assign an initial feature vector to each node in the subgraph;

[0048] An inference module, used to input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism for multiple rounds of inference to obtain a final representation of each node rich in context;

[0049] A node identification module, used for identifying a number of target nodes most relevant to the original question from the final representation of all nodes of the subgraph;

[0050] A path extraction module, used to take the entities corresponding to each target node as candidate answers to the original question, and extract the shortest paths from each entity in the original question to each candidate answer;

[0051] A conversion module, used for converting each extracted shortest path into a path description in natural language;

[0052] A generation module is used to input the original question and the path description into a large language model to perform a retrieval enhancement generation task and output a final answer corresponding to the original question.

[0053] Optionally, in a first implementation of the second aspect of the present invention, the building module is specifically used to:

[0054] The original question is parsed using natural language processing technology, and several key elements of the original question are extracted from the parsed results;

[0055] Based on each of the key elements, forwardly traverse the preset knowledge graph to obtain a number of first key information directly related to the original question;

[0056] Based on each of the first key information, construct a forward subgraph including all entities and entity association relationships directly related to the original question;

[0057] Based on each entity in the forward subgraph, traverse the knowledge graph in reverse order to obtain a number of second key information directly related to each entity;

[0058] Based on each of the second key information, construct a reverse subgraph including all entities and entity association relationships indirectly related to the original question;

[0059] The forward subgraph and the reverse subgraph are structurally fused to obtain a bidirectional fused subgraph.

[0060] Optionally, in a second implementation of the second aspect of the present invention, the reasoning module is specifically used to:

[0061] Input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism;

[0062] By using the graph neural network, the initial feature vector corresponding to each node in the subgraph is converted into a continuous vector space representation to obtain an embedding vector of each node and its neighboring nodes in the subgraph;

[0063] Calculate the dot product of the embedding vector between each node and its corresponding neighbor nodes respectively to obtain the original attention score between the corresponding neighbor nodes;

[0064] Normalize the attention scores of all neighbor nodes, and use the normalized attention scores to perform weighted summation on the Value vectors of all neighbor nodes to obtain the final context-rich representation of each node after update;

[0065] Each attention layer of the graph neural network receives the node representation output by the previous attention layer as input, and outputs an updated node representation.

[0066] Optionally, in a third implementation manner of the second aspect of the present invention, the node identification module is specifically used to:

[0067] Using a language model to encode the keywords and entities in the original question into vector representations, and concatenating the vector representations of the keywords and entities as the final representation of the original question;

[0068] Using the attention mechanism, the first attention weight of the original question representation is calculated using a learnable first parameter matrix, and the second attention weight of each node representation is calculated using a learnable second parameter matrix;

[0069] Based on the first attention weight and the second attention weight, using a dot product attention function to respectively calculate a similarity score between the original question representation and each node representation;

[0070] According to the similarity scores corresponding to the nodes, a preset number of nodes with the highest scores are selected as target nodes most relevant to the original question.

[0071] Optionally, in a fourth implementation of the second aspect of the present invention, the path extraction module is specifically used to:

[0072] List all nodes and all edges in the subgraph to obtain a node list and an edge list, wherein each edge represents a relationship between two nodes;

[0073] Constructing an adjacency matrix according to the node list and the edge list;

[0074] In the adjacency matrix, the entity of the original question is set as the starting point and the candidate answer is set as the end point, and a queue is initialized to store the current node and the path from the starting point to the current node;

[0075] The adjacency matrix is ​​traversed using a breadth-first search algorithm until an end point is found or all nodes are traversed, and the shortest path from each entity in the original question to each candidate answer is output.

[0076] Optionally, in a fifth implementation of the second aspect of the present invention, the conversion module is specifically configured to:

[0077] Obtaining the natural language meaning corresponding to each node and each edge in the subgraph, and generating labels corresponding to each node and each edge based on the natural language meaning;

[0078] Based on the order of each node and each edge in the corresponding shortest path, the labels corresponding to each node and each edge are used to construct a corresponding natural language description as the path description corresponding to each of the shortest paths.

[0079] Optionally, in a sixth implementation of the second aspect of the present invention, the generating module is specifically configured to:

[0080] Inputting the original question and the path description into a large language model;

[0081] Using a large language model, key information in the original question is identified, and the key information is integrated with the path description to obtain nodes and edges to be retrieved;

[0082] Through the large language model, the external knowledge graph is retrieved to obtain detailed information related to the nodes and edges to be retrieved;

[0083] The retrieved detailed information is used as the context information of the original question and the path description, and a final answer corresponding to the original question is generated and output through a large language model.

[0084] A third aspect of the present invention provides a computer device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the computer device executes the above-mentioned retrieval-enhanced question-answering method.

[0085] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned retrieval-enhanced question-answering method.

[0086] In the technical solution provided by the present invention, a subgraph containing all entities and entity associations related to the original question is first constructed; then the subgraph is applied to the graph neural network based on the attention mechanism for multiple rounds of reasoning to obtain the final representation of each node rich in context; then several target nodes most relevant to the original question are identified from the final representation of all nodes in the subgraph and used as candidate answers to the original question, and the shortest paths from each entity in the original question to each candidate answer are extracted from the subgraph; then each extracted shortest path is converted into a path description in natural language; finally, the original question and the path description are input into a large language model to perform the retrieval enhancement generation task, and the final answer corresponding to the original question is output. The present invention combines the advanced natural language processing capabilities of large language models and the deep graph structure reasoning capabilities of graph neural networks based on the attention mechanism, and constructs a novel retrieval and generation technology framework for knowledge graph question-answering tasks. The framework solves the limitations of a single model in complex knowledge reasoning and natural language understanding by intelligently integrating the core advantages of the two models in the retrieval enhancement mechanism, especially when dealing with multi-hop relationships and multi-entity queries, achieving a huge performance leap. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 A schematic diagram of an embodiment of a question-answering method based on retrieval enhancement in an embodiment of the present invention;

[0088] Figure 2 Schematic diagram of an embodiment of a question-answering device based on retrieval enhancement in an embodiment of the present invention;

[0089] Figure 3 FIG. 1 is a schematic diagram of an embodiment of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0090] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0091] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1, an embodiment of the question-answering method based on retrieval enhancement in an embodiment of the present invention includes:

[0092] 101. Obtaining the original question raised by the user in natural language;

[0093] In this embodiment, obtaining the original question raised by the user in natural language is the first step of the knowledge graph question answering (KGQA) system and the basis of the entire process.

[0094] The system needs to provide a user interface (UI) or application programming interface (API) so that users can enter questions. This interface can be a web form, a chat window, voice input, etc.

[0095] After the user submits a question, the system first performs a series of natural language processing preprocessing steps on the question, including but not limited to:

[0096] Word segmentation: split the question text into independent words or phrases.

[0097] Part-of-speech tagging: assigning a part of speech (such as noun, verb, adjective, etc.) to each word.

[0098] Named entity recognition: Identifying entities in text (such as names of people, places, and institutions), which is very important for subsequent knowledge graph queries.

[0099] Syntactic Analysis: Analyzing the structure of a sentence, such as subject, predicate, object, etc., helps in understanding the meaning of the sentence.

[0100] 102. Construct a subgraph including all entities and entity association relationships related to the original question;

[0101] In the knowledge graph question answering (KGQA) system, building a subgraph containing all entities and entity associations related to the original question is a key step. This process aims to extract the parts closely related to the user's question from the huge knowledge graph for more efficient and accurate query and reasoning. The following are the general steps for building a subgraph:

[0102] (1) Entity recognition: Use named entity recognition (NER) technology to extract all possible entity names from the original question. These entities may be names of people, places, institutions, concepts, etc., and they usually correspond to nodes in the knowledge graph.

[0103] (2) Entity linking: Match the identified entity name with the entity in the knowledge graph to find the corresponding node. This may involve techniques such as fuzzy matching, synonym matching, and spelling correction, because the user input may not be completely consistent with the standard name in the knowledge graph.

[0104] (3) Relationship extraction: Analyze the context and semantic information in the original question and identify the relationships related to the identified entities. These relationships may be explicit (such as "born in", "located in", etc.) or implicit (need to be inferred based on the question context and common sense).

[0105] (4) Subgraph construction: Using the identified entities as nodes and the extracted relationships as edges, a subgraph is constructed in the knowledge graph. The subgraph should contain all entities and relationships directly related to the original question, as well as any indirect connections between these entities and relationships (through other entities and relationships).

[0106] It is important to note that the size of the subgraph should be moderate, containing enough information to support accurate query and reasoning, but not too large to cause inefficient processing.

[0107] In one embodiment, the above step 102 specifically constructs a subgraph in the following manner, including:

[0108] 1021. Analyze the original question using natural language processing technology, and extract several key elements of the original question from the analysis results;

[0109] First, the original question is cleaned to remove irrelevant symbols, extra spaces, special characters, etc., and then the original question sentence is segmented to divide it into a series of independent words or phrases. At the same time, a part-of-speech tag (such as noun, verb, adjective, etc.) is assigned to each word in the sentence, which helps to understand the grammatical function and semantic role of the word in the sentence.

[0110] Identify entities with specific meanings from the original question sentence, such as names of people, places, institutions, time expressions, etc. These entities are often nodes in the knowledge graph and are crucial for subsequent queries and reasoning. Analyze the dependencies between words in the sentence and construct a dependency syntax tree. This tree structure can clearly represent the grammatical and semantic relationships between the various components in the sentence. Further, the semantic information in the original question sentence can be analyzed to mark the relationship between the predicate (usually a verb or adjective) and the argument (such as agent, patient, tool, etc.) in the sentence. This helps to understand the participants and attributes of the event in the sentence. According to the structure and semantic information of the sentence, the questions are classified into different types (such as description, comparison, cause and effect, yes or no, etc.). This helps the system adopt different query and reasoning strategies to answer different types of questions.

[0111] Combining the above preprocessing results, we can extract the key elements that are crucial for query and reasoning from the original question, including:

[0112] Entity: The main object asked or described in the question (subject entity, object entity).

[0113] Relationship: A link between different entities, or a term that describes the attributes of an entity.

[0114] Other modifiers: words used to limit or describe subjects, relationships or objects, such as time, place, degree, etc.

[0115] In addition, we can further construct a query statement or logical expression for the knowledge graph based on the extracted key elements. This query should accurately reflect the intention and needs of the user's question.

[0116] 1022. Based on each of the key elements, forwardly traverse the preset knowledge graph to obtain a number of first key information directly related to the original question;

[0117] Use the parsed key elements (such as the subject entity) as the starting point of the query. This starting point is one or more nodes in the knowledge graph, representing the core entities in the user's question. Based on the relationships between key elements (such as relationship attributes), a query path is constructed from the starting point. These relationships indicate the direction and conditions for traversing the knowledge graph. It may be necessary to deal with the uncertainty or ambiguity of the relationship and expand the query path by matching similar relationships or applying semantic reasoning.

[0118] Starting from the query starting point, forward traverse is performed in the knowledge graph along the constructed query path. During the traversal, key information directly related to the original question is recorded. This information may be entity nodes, relationship edges, or a combination of these nodes and edges (such as triples).

[0119] In this embodiment, it is necessary to first build a knowledge graph, collect data from existing source documents, and then organize the collected data, remove redundant and erroneous information, and standardize it. After identifying the entities in the new data, they are associated with existing entities or newly created entities in the knowledge graph. The relationships between entities are further extracted from the data, and these relationships are added to the knowledge graph as edges, and a special graph database Neo4j is used to save the knowledge graph.

[0120] 1023. Based on each of the first key information, construct a forward subgraph including all entities and entity association relationships directly related to the original question;

[0121] Identify core entities from the first key information. These entities are usually explicitly mentioned in the user's question, or are directly related to the question obtained by parsing the question. Traverse the first key information and extract other entities directly related to the core entity and the relationships between them. These related entities and relationships constitute the main components of the subgraph. Use graph data structures (such as nodes and edges in adjacency lists, adjacency matrices, or graph databases) to build subgraphs. Use the core entity as the starting point or central node of the subgraph, the extracted related entities as the nodes of the subgraph, and connect them with the core entity and the relationship between them as edges, ensuring that each edge in the subgraph explicitly represents the relationship between entities and that these relationships are consistent with the query meaning of the original question. Figure 1 To.

[0122] In this embodiment, a preliminary query is performed in the knowledge graph based on the key elements analyzed to obtain information directly related to the question. Based on this information, a preliminary subgraph is constructed that includes all entities involved in the question and their associations.

[0123] 1024. Based on each entity in the forward subgraph, traverse the knowledge graph in reverse order to obtain a number of second key information directly related to each entity;

[0124] 1025. Based on each of the second key information, construct a reverse subgraph including all entities indirectly related to the original question and entity association relationships;

[0125] This embodiment uses a bidirectional semantic subgraph construction method to construct a subgraph containing all entities involved in the problem and their associations. Starting from the entities extracted from the preliminary subgraph, the subgraph is expanded by reverse searching for other entities and relationships related to these entities. This reverse search can discover more potential related information and help build a more comprehensive subgraph. Based on the results of the reverse search, reverse subgraphs are constructed, which contain information that may be indirectly related to the problem.

[0126] 1026. Perform graph structural fusion on the forward subgraph and the reverse subgraph to obtain a bidirectional fused subgraph.

[0127] In this embodiment, the nodes and edges of the forward subgraph and the reverse subgraph are first traversed to find the shared nodes and edges that appear in both subgraphs. These shared nodes and edges are the basis of the fused subgraph because they connect different parts of the two subgraphs. For each shared node, only one node instance is retained in the fused subgraph, and all edges from the two subgraphs are connected to this node to ensure that no duplicate edges are introduced or logical inconsistencies are generated during the fusion process. Nodes and edges unique to the forward subgraph and the reverse subgraph are added to the fused subgraph. At the same time, attention should also be paid to handling possible node or edge renaming issues to ensure that there are no duplicate identifiers in the fused subgraph. Check the structure of the fused subgraph to ensure that it is connected (if required by the business logic), and remove any unnecessary nodes or edges to reduce the complexity of the subgraph and improve query efficiency.

[0128] In this embodiment, the subgraphs constructed in the forward and reverse directions are merged to form a complete subgraph structure. This fusion ensures that the subgraph contains both directly related information and indirectly related information. The fused subgraph is optimized to remove redundant information and retain the most relevant core entities and relationships. The fused subgraph will serve as the basic unit for subsequent graph neural network processing.

[0129] 103. Using the subgraph as a basic unit for graph neural network processing, assigning an initial feature vector to each node in the subgraph;

[0130] Before allocating the initial feature vector, it is necessary to determine which categories of features are included in the feature vector, such as node type, keyword features, and other attributes.

[0131] Node type: Usually a categorical variable that indicates the role or type of a node in a knowledge graph or graph structure. For example, a node may represent a person, a place, an organization, etc. Node types are usually represented by one-hot encoding or an embedding layer. One-hot encoding assigns a binary vector to each possible category, while the embedding layer learns a denser, lower-dimensional vector representation.

[0132] Keyword features: These are keywords extracted based on the text or metadata associated with the node, used to represent the core content or theme of the node. Keywords can be obtained through text processing methods such as TF-IDF, Word2Vec, BERT, etc. If the keyword features are word vectors obtained through text processing (such as the output of Word2Vec or BERT), you can use these vectors directly, or aggregate (such as average, sum) multiple word vectors to get a fixed-length feature vector. If the number of keywords is limited and fixed, you can also use one-hot encoding or embedding layers.

[0133] Other attributes: This includes any numeric or categorical data that is descriptive of the node, such as age, gender, rating, timestamp, etc. Numeric attributes can be directly included as part of the feature vector, while categorical attributes need to be encoded like the node type.

[0134] The above encoded features are combined into a fixed-length feature vector. Therefore, it is necessary to allocate a certain amount of space (i.e., the dimension of the feature vector) for each type of feature and concatenate them.

[0135] Assume the following node types and attributes:

[0136] Node Type: Person, Location, Organization

[0137] Keyword features: word vectors obtained through some text processing methods (assuming that each node has 3 keywords, and each keyword is represented by a 100-dimensional vector)

[0138] Other attributes: age (numeric), gender (categorical)

[0139] The feature vector initialized for each node is as follows:

[0140] Node type (assuming one-hot encoding, 3 types -> 3 dimensions)

[0141] Keyword features (3 keywords, 100 dimensions each -> 300 dimensions)

[0142] Age (directly as one dimension)

[0143] Gender (using one-hot encoding, assuming there are only two genders, male and female -> 2 dimensions)

[0144] Therefore, the feature vector of each node will have 3 (node ​​type) + 300 (keyword features) + 1 (age) + 2 (gender) = 306 dimensions.

[0145] 104. Input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism for multiple rounds of reasoning to obtain a final representation of each node rich in context;

[0146] The structure of the subgraph (nodes and edges) and the initial feature vector of each node are passed as input to the GNN model. In the GNN model, each node is updated based on its own feature vector and the feature vectors of its neighboring nodes (information passed through the edges).

[0147] In each layer of GNN, the attention mechanism evaluates the importance of neighboring nodes to the current node and aggregates the information of neighboring nodes according to these importance weights. This is achieved by calculating the similarity scores (such as dot product, cosine similarity, etc.) between the feature vectors of neighboring nodes and the feature vector of the current node, and then normalizing these scores (such as using the softmax function) to obtain attention weights. These attention weights are used to weight the aggregation of information from neighboring nodes to update the representation of the current node.

[0148] GNN models usually contain multiple layers, and each time a node passes through a layer, its representation is updated based on the information of its neighbors. During multiple rounds of reasoning, the representation of the node gradually incorporates contextual information from the entire subgraph. After each round of reasoning, the node representation becomes richer in contextual information because it contains not only the initial features of the node itself, but also information from its neighbors, neighbors of neighbors, and so on. After multiple rounds of reasoning, the GNN model outputs the final feature representation of each node. The final feature representation of the node is rich in contextual information from the entire subgraph and can be used for various downstream tasks such as node classification, link prediction, graph classification, etc.

[0149] In one embodiment, the step 104 further includes:

[0150] 1041. Input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism;

[0151] Take the subgraph (including nodes and edges) and the initial feature vector of each node as input, ensuring that the data structure of the subgraph and the format of the node feature vector match the requirements of the graph neural network model.

[0152] The graph neural network Attn-GNN model based on the attention mechanism needs to be fine-tuned before use. Specifically, a large number of annotated data sets containing questions and corresponding correct answers are used to convert the questions into query subgraphs, where the entities and relationships in the questions constitute the nodes and edges of the subgraphs. Then, an optimization algorithm (RMSprop) is used to minimize the difference between the predicted answer and the true answer (such as cross entropy loss). During the fine-tuning process, the model learns how to effectively use the attention mechanism to focus on key nodes and paths, improving the accuracy of question understanding and answer generation.

[0153] 1042. Convert the initial feature vector corresponding to each node in the subgraph into a continuous vector space representation through the graph neural network to obtain an embedding vector of each node and its neighboring nodes in the subgraph;

[0154] Through the first layer (or embedding layer) of the graph neural network, the initial feature vector of each node is converted into a representation in a high-dimensional, continuous vector space. This conversion process usually involves linear transformations (such as fully connected layers) and nonlinear activation functions (such as ReLU). The embedding vectors of each node and its neighboring nodes are obtained. These vectors are now in the same continuous vector space and have initially integrated the feature information of the node itself.

[0155] 1043. Calculate the dot product of the embedding vector between each node and the neighboring node corresponding to the node respectively, and obtain the original attention score between the neighboring nodes corresponding to the node;

[0156] For each node in the subgraph, the dot product (or other similarity measure, such as cosine similarity) of the embedding vectors between each node and all neighboring nodes is calculated. This dot product result represents the raw attention score between the node and its neighbors. The higher the score, the greater the importance of the neighbor node to the current node, thus obtaining the raw attention score set between each node and all its neighboring nodes.

[0157] 1044. Normalize the attention scores of all neighbor nodes, and use the normalized attention scores to perform weighted summation on the Value vectors of all neighbor nodes to obtain the updated context-rich final representation of each node;

[0158] Each attention layer of the graph neural network receives the node representation output by the previous attention layer as input, and outputs an updated node representation.

[0159] The attention scores of all neighbor nodes are normalized, usually using a softmax function, to ensure that the sum of the attention weights of all neighbors is 1. In this way, each neighbor node will receive a normalized attention weight based on its importance to the current node. Using the normalized attention weights, the Value vector of each neighbor node (in some models, the Value vector may be the same as the embedding vector or may be different) is weighted summed. This process aggregates information from neighbor nodes and weights them according to their importance to update the representation of the current node. The final updated context-rich representation of each node is obtained. These representations not only contain the initial feature information of the node itself, but also incorporate contextual information from neighbor nodes.

[0160] 105. Identify several target nodes that are most relevant to the original problem from the final representations of all nodes of the subgraph;

[0161] First, we need to clarify how to measure the relevance of a node to the original question. This usually depends on the specific content and form of the original question. For example, if the original question is a classification problem about a specific topic or category, then the relevance of a node may be related to the similarity between its final representation and the representation of that category.

[0162] For each node, calculate the similarity between its final representation and the original problem representation (or a reference representation, such as a category center). The similarity measure can be cosine similarity, the negative of the square of the Euclidean distance, dot product, etc., depending on the nature of the representation vector and the requirements of the problem. In some cases, it may be more intuitive to calculate the distance directly (such as the Euclidean distance), but it is usually necessary to convert the distance into a similarity score (for example, by taking the inverse of the distance or applying some kind of decay function). Finally, sort all nodes according to the similarity or distance score so that the nodes most relevant to the original problem are ranked first. Select several top-ranked nodes as target nodes according to requirements. The number of selections depends on the specific application scenario and performance requirements.

[0163] In one embodiment, the step 105 further includes:

[0164] 1051. Encode the keywords and entities in the original question into vector representations using a language model, and concatenate the vector representations of the keywords and entities as a final representation of the original question;

[0165] Extract keywords and entities from the original question. This can be achieved through natural language processing (NLP) techniques such as named entity recognition (NER) and keyword extraction algorithms. Use pre-trained language models (such as BERT, GPT, etc.) to encode the extracted keywords and entities into vector representations. These vectors usually contain rich semantic information and can capture the similarities and relationships between words. Finally, the vector representations of keywords and entities are concatenated to form a comprehensive final vector representation that can represent the original question.

[0166] 1052. Using the attention mechanism, a first attention weight of the original question representation is calculated using a learnable first parameter matrix, and a second attention weight of each node representation is calculated using a learnable second parameter matrix;

[0167] To measure the importance of the original question representation, a learnable first parameter matrix is ​​used to calculate the first attention weight. This weight reflects the relative importance of different parts of the original question. Similarly, to measure the importance of each node representation, a learnable second parameter matrix is ​​used to calculate the second attention weight. This weight reflects the relative importance of different nodes in the subgraph or their potential relevance to the original question.

[0168] 1053. Based on the first attention weight and the second attention weight, use a dot product attention function to respectively calculate a similarity score between the original question representation and each node representation;

[0169] Based on the first and second attention weights, a dot product attention function is used to calculate the similarity score between the original question representation and each node representation. This score measures the semantic similarity or relevance between the original question and each node. For each dimension of the original question, it is dot-producted with the corresponding dimension of each node and may be multiplied by the corresponding attention weight. These dot product results are then summed or averaged to get the final similarity score.

[0170] 1054. According to the similarity score corresponding to each node, select a preset number of nodes with the highest scores as target nodes most relevant to the original question.

[0171] According to the similarity scores corresponding to each node, the nodes are sorted so that the nodes most relevant to the original question are ranked first. A preset number of nodes with the highest scores are selected as the target nodes most relevant to the original question. The preset number can be determined according to the specific application scenario and requirements.

[0172] 106. Taking entities corresponding to each target node as candidate answers to the original question, and extracting the shortest paths from each entity in the original question to each candidate answer;

[0173] For each entity in the original problem, it is the starting point of the path search. For each candidate answer (i.e., the entity corresponding to the target node), it is the end point of the path search. For each pair of starting and ending points, use the shortest path algorithm on the graph (such as Dijkstra algorithm, Bellman-Ford algorithm, or breadth-first search algorithm) to find the shortest path.

[0174] In one embodiment, the step 106 further includes:

[0175] 1061. List all nodes and all edges in the subgraph to obtain a node list and an edge list, wherein each edge represents a relationship between two nodes;

[0176] 1062. Construct an adjacency matrix according to the node list and the edge list;

[0177] List all nodes of the subgraph to form a node list. List all edges connecting these nodes to form an edge list. Each edge should be represented as a tuple or list of two nodes. Finally, construct an adjacency matrix based on the node list and edge list.

[0178] The adjacency matrix is ​​a two-dimensional array (or matrix) whose elements represent the connection relationship between nodes in the graph. In an undirected graph, if there is an edge connecting node i and node j, the elements at the corresponding positions (i, j) and (j, i) in the matrix are both 1 (or the value representing the connection weight, if the graph is a weighted graph); if there is no edge connection, the elements at these positions are 0. For a directed graph, if there is an edge from node i to node j, the element at position (i, j) is 1 (or weight), and the element at position (j, i) is 0.

[0179] 1063. Set the entity of the original question as the starting point and the candidate answer as the end point in the adjacency matrix, and initialize a queue for storing the current node and the path from the starting point to the current node;

[0180] 1064. Use a breadth-first search algorithm to traverse the adjacency matrix until an end point is found or all nodes are traversed, and output the shortest path from each entity in the original question to each candidate answer.

[0181] After setting the entity of the original question as the starting point and the candidate answer as the end point in the adjacency matrix, the breadth-first search (BFS) algorithm is used to find the shortest path from the starting point to the end point. In this process, a queue is initialized to store the nodes to be explored and the paths from the starting point to these nodes. In order to store the path directly in the queue, a tuple or list can be pre-defined, which contains the current node and the list of paths from the starting point to the current node (or more concisely, if each node has a unique identifier, then only the sequence of node identifiers passed through needs to be stored).

[0182] 107. Convert each extracted shortest path into a path description in natural language;

[0183] First, identify the starting point and the end point of the path. Then traverse each intermediate node in the path, and finally use a natural language template to construct a description of the path. For example, the beginning sentence usually starts with "starting from..." or a similar statement. For each intermediate node, use statements such as "through..." or "then to...". The sentence ends with "finally arrived at..." and indicates the end point.

[0184] In one embodiment, the above step 107 further includes:

[0185] 1071. Obtain the natural language meaning corresponding to each node and each edge in the subgraph, and generate a label corresponding to each node and each edge based on the natural language meaning;

[0186] 1072. Based on the order of each node and each edge in the corresponding shortest path, the labels corresponding to each node and each edge are used to construct a corresponding natural language description as the path description corresponding to each of the shortest paths.

[0187] First, we need to clarify the specific meaning of each node and each edge in the real world or application scenario. For example, a node may be a person's name, a place name, or a concept, and an edge may be a "friendship", "located in", "contains", etc. Then we extract the passed nodes and edges from the shortest path algorithm. Finally, we construct a coherent natural language description based on the labels of the nodes and edges and their order in the path.

[0188] Assume the following node and edge labels:

[0189] Nodes: A (city A), B (city B), C (city C)

[0190] Sides: AB (airline from A to B), BC (railway from B to C)

[0191] The shortest path identified is A -> B -> C, and the natural language description implementation process is as follows:

[0192] (1) Clarify the meaning of labels: A is city A, B is city B, and C is city C;

[0193] AB represents the route from city A to city B;

[0194] BC represents the railway from city B to city C;

[0195] The shortest path corresponding to the node labels: A ->B ->C;

[0196] (2) Construction description: starting from "City A", then reaching "City B" via "airline", and finally reaching "City C" via "railway";

[0197] (3) Natural language description: “Starting from city A, take the airline to city B, and then take the railway to city C.”

[0198] 108. Input the original question and the path description into a large language model to perform a retrieval enhancement generation task, and output a final answer corresponding to the original question.

[0199] Retrieval-augmented generation is a technology that combines external knowledge retrieval with language model generation. It retrieves relevant information from an external knowledge base before a large language model generates the final text, and then uses this information as input or condition to guide the generation process of the language model, thereby outputting more accurate and easier to understand content.

[0200] In one embodiment, the step 108 further includes:

[0201] 1081. Input the original question and the path description into a large language model;

[0202] Large Language Model (LLM) is a natural language processing technology that can generate natural language text or understand the meaning of language text by using a deep learning model trained with a large amount of text data. Large language models can handle a variety of natural language tasks, such as text classification, question answering, dialogue, text generation, etc.

[0203] 1082. Using a large language model, identify key information in the original question, and integrate the key information with the path description to obtain nodes and edges to be retrieved;

[0204] The large language model needs to understand and analyze the original question and extract key information from it, including:

[0205] Entity: The subject or object involved in the question, such as a person’s name, place name, organization name, etc.

[0206] Attributes: Characteristics or descriptions associated with an entity, such as age, location, relationship, etc.

[0207] Action: The operation or behavior that needs to be performed, such as searching, comparing, sorting, etc.

[0208] Constraints: Restrictions on query results, such as time range, spatial range, specific conditions, etc.

[0209] A path description usually refers to a sequence of nodes and edges that need to be passed from one node to another. In a knowledge graph or graph database, this is usually represented as a query path. The large language model needs to understand the meaning of each node and edge in this path description and the relationship between them.

[0210] The large language model needs to integrate the key information in the original question with the path description, including:

[0211] Match key information with nodes: Match the entities or attributes in the question with the nodes in the path description to determine which nodes are the starting point, end point, or intermediate nodes of the query.

[0212] Construct query logic: Based on the actions and constraints in the problem, as well as the nodes and edges in the path description, construct a logical query path. This may involve combining, sorting, and filtering multiple nodes and edges.

[0213] Generate query statements: Convert the integrated information into specific query statements or query graphs to perform retrieval in graph databases or knowledge graphs.

[0214] Through the above processing of the large language model, a query result containing the nodes and edges to be retrieved will be obtained. These nodes and edges constitute the query path from the original question to the target answer. The large language model can retrieve relevant data and information in the external knowledge graph based on this query path.

[0215] For example, suppose the original question is: "Find information on all science and technology exhibitions held in Beijing in 2023.", and the shortest path description is "Starting from the 'exhibition' node, connect to the 'city' node through the 'venue' edge, and then connect to the 'year' node through the 'time' edge". Key information: The entity is "exhibition", the attributes are "venue" and "time", and the constraints are "2023" and "Beijing". Information integration: Take "exhibition" as the starting node, "city" as the intermediate node, and "year" as the end node, connect "exhibition" and "city" through the "venue" edge, and connect "city" and "year" through the "time" edge. At the same time, add the constraints "Beijing" and "2023". Nodes and edges to be retrieved: The query path obtained is "exhibition-venue-city (Beijing)-time-year (2023)", which means that it is necessary to retrieve information on all exhibitions held in Beijing in 2023.

[0216] 1083. Through the large language model, retrieve the external knowledge graph to obtain detailed information related to the nodes and edges to be retrieved;

[0217] Through the above steps 1081 and 1082, the nodes and edges to be retrieved are obtained. For example, if the original question is "Information of all science and technology exhibitions held in Beijing in 2023", the nodes to be retrieved may include "exhibition", "city" and "year", and the edges may include "venue" and "time". According to the nodes and edges to be retrieved, design a suitable retrieval strategy. This includes determining the search keywords, selecting appropriate retrieval tools or interfaces (such as API, query language, etc.), and considering how to combine these elements to form an effective retrieval query.

[0218] Using a large language model (LLM) for retrieval can help optimize retrieval queries, understand retrieval results, and perform further reasoning or analysis when necessary. Specifically, LLM can be used to:

[0219] Generate search queries: Based on the nodes and edges to be searched, LLM can generate search query statements that conform to syntax and logic.

[0220] Parsing search results: For the returned search results, LLM can help parse the key information and extract detailed data related to the nodes and edges to be retrieved.

[0221] Reasoning and supplementation: When the search results are incomplete or ambiguous, LLM can use its language understanding and reasoning capabilities to make necessary supplements or corrections.

[0222] Submit the designed search query to the external knowledge graph to perform the search operation. The knowledge graph usually contains a large amount of structured data, which is represented in the form of a graph, with nodes representing entities and edges representing the relationship between entities. The search result will be a series of data sets related to the nodes and edges to be searched.

[0223] Extract detailed information related to the nodes and edges to be retrieved from the search results, including the attributes, relationships, and related descriptions or instructions of the entities. This information may need to be further integrated and processed to meet specific needs or format requirements. Output the extracted and integrated detailed information in an appropriate form, which may be text, tables, graphics, or other visual forms. The output results should clearly and accurately reflect the detailed information related to the nodes and edges to be retrieved in the search results.

[0224] For example, suppose we want to retrieve "information about all science and technology exhibitions held in Beijing in 2023". The process of retrieval using a large language model and an external knowledge graph is as follows:

[0225] Nodes and edges to be retrieved: The nodes are "exhibition", "city" (Beijing), and "year" (2023); the edges are "venue" (exhibition→city), and "time" (exhibition→year).

[0226] Retrieval query: generated by a large language model, which may be similar to "query for detailed information about science and technology exhibitions held in Beijing in 2023".

[0227] Search results: The knowledge graph returns a series of qualified exhibition records, including exhibition name, time, venue (city), exhibition type (technology), exhibitors, exhibition introduction and other detailed information.

[0228] Extraction and integration: Extract key information such as exhibition name, time (specific to date), venue (specific to city), exhibition type, etc. from the search results and integrate them according to a certain format.

[0229] Output results: The integrated information is output in the form of a table or list for user viewing and use.

[0230] 1084. Using the retrieved detailed information as context information of the original question and the path description, a final answer corresponding to the original question is generated and output through a large language model.

[0231] Integrate the retrieved details with the original question and path description to form complete contextual information, including:

[0232] Original question: The initial query or question asked by the user.

[0233] Path description: The sequence of nodes and edges used to guide the retrieval process.

[0234] Retrieval results: Detailed information related to the nodes and edges to be retrieved, retrieved from the external knowledge graph.

[0235] The large language model needs to analyze the integrated context information to understand the logical relationship, semantic meaning and key information points. Based on the analyzed context information, the large language model begins to generate answers, including: first extracting key information directly related to the original question from the context, then organizing the extracted information according to a certain logic and structure to form a draft of the answer, and finally converting the organized information into natural and fluent language to generate the final answer text. To ensure the accuracy of the output results, the generated answers can be further verified and corrected to ensure their accuracy and rationality. The large language model may contain some verification mechanisms to check whether the generated answers are consistent with the context information and whether they meet the logical and semantic requirements. In some cases, the generated answers can be displayed to users or other systems to collect external feedback for correction and optimization. Finally, the verified and corrected answers are output to the user. The output answers should be clear, accurate, concise and easy to understand, and can directly answer the original questions.

[0236] For example, suppose the original question is: "Please tell me the names and times of all science and technology exhibitions held in Beijing in 2023." The path description is the steps to guide how to retrieve relevant information from the knowledge graph, and the retrieved detailed information includes the names, times, locations, and other information of a series of exhibitions.

[0237] After integrating this information, the large language model generates the following answer:

[0238] According to your request, we found information about several science and technology exhibitions held in Beijing in 2023. Here are their names and dates:

[0239] Exhibition name: XX Technology Exhibition

[0240] Date: From XX month to XX month, 2023

[0241] Exhibition name: XX Expo

[0242] Date: From XX month to XX month, 2023.

[0243] This embodiment combines the capabilities of logical reasoning and natural language understanding, and through an integrated retrieval enhancement mechanism, further improves the overall accuracy and recall rate of the system. Compared with traditional methods, the efficiency is significantly improved, especially when processing multi-hop and multi-entity queries, the accuracy of the answers is significantly improved. Without the need for additional computing resources, the accuracy and efficiency of question answering are significantly improved, demonstrating its great value and potential in practical applications.

[0244] See also Figure 2 An embodiment of the question-answering device based on retrieval enhancement in the embodiment of the present invention includes:

[0245] An acquisition module 201 is used to acquire an original question raised by a user in natural language;

[0246] A construction module 202 is used to construct a subgraph including all entities and entity association relationships related to the original question;

[0247] An initialization module 203 is used to use the subgraph as a basic unit for graph neural network processing and assign an initial feature vector to each node in the subgraph;

[0248] The reasoning module 204 is used to input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on the attention mechanism for multiple rounds of reasoning to obtain a final representation of each node rich in context;

[0249] A node identification module 205, configured to identify a number of target nodes most relevant to the original question from the final representation of all nodes of the subgraph;

[0250] A path extraction module 206 is used to take the entities corresponding to each target node as candidate answers to the original question, and extract the shortest paths from each entity in the original question to each candidate answer;

[0251] A conversion module 207, used to convert each extracted shortest path into a path description in natural language;

[0252] The generation module 208 is used to input the original question and the path description into the large language model to perform the retrieval enhancement generation task and output the final answer corresponding to the original question.

[0253] Optionally, in one embodiment, the construction module 202 is specifically used for:

[0254] The original question is parsed using natural language processing technology, and several key elements of the original question are extracted from the parsed results;

[0255] Based on each of the key elements, forwardly traverse the preset knowledge graph to obtain a number of first key information directly related to the original question;

[0256] Based on each of the first key information, construct a forward subgraph including all entities and entity association relationships directly related to the original question;

[0257] Based on each entity in the forward subgraph, traverse the knowledge graph in reverse order to obtain a number of second key information directly related to each entity;

[0258] Based on each of the second key information, construct a reverse subgraph including all entities and entity association relationships indirectly related to the original question;

[0259] The forward subgraph and the reverse subgraph are structurally fused to obtain a bidirectional fused subgraph.

[0260] Optionally, in one embodiment, the reasoning module 204 is specifically used for:

[0261] Input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism;

[0262] By using the graph neural network, the initial feature vector corresponding to each node in the subgraph is converted into a continuous vector space representation to obtain an embedding vector of each node and its neighboring nodes in the subgraph;

[0263] Calculate the dot product of the embedding vector between each node and its corresponding neighbor nodes respectively to obtain the original attention score between the corresponding neighbor nodes;

[0264] Normalize the attention scores of all neighbor nodes, and use the normalized attention scores to perform weighted summation on the Value vectors of all neighbor nodes to obtain the final context-rich representation of each node after update;

[0265] Each attention layer of the graph neural network receives the node representation output by the previous attention layer as input, and outputs an updated node representation.

[0266] Optionally, in one embodiment, the node identification module 205 is specifically used for:

[0267] Using a language model to encode the keywords and entities in the original question into vector representations, and concatenating the vector representations of the keywords and entities as the final representation of the original question;

[0268] Using the attention mechanism, the first attention weight of the original question representation is calculated using a learnable first parameter matrix, and the second attention weight of each node representation is calculated using a learnable second parameter matrix;

[0269] Based on the first attention weight and the second attention weight, using a dot product attention function to respectively calculate a similarity score between the original question representation and each node representation;

[0270] According to the similarity scores corresponding to the nodes, a preset number of nodes with the highest scores are selected as target nodes most relevant to the original question.

[0271] Optionally, in one embodiment, the path extraction module 206 is specifically used for:

[0272] List all nodes and all edges in the subgraph to obtain a node list and an edge list, wherein each edge represents a relationship between two nodes;

[0273] Constructing an adjacency matrix according to the node list and the edge list;

[0274] In the adjacency matrix, the entity of the original question is set as the starting point and the candidate answer is set as the end point, and a queue is initialized to store the current node and the path from the starting point to the current node;

[0275] The adjacency matrix is ​​traversed using a breadth-first search algorithm until an end point is found or all nodes are traversed, and the shortest path from each entity in the original question to each candidate answer is output.

[0276] Optionally, in one embodiment, the conversion module 207 is specifically used for:

[0277] Obtaining the natural language meaning corresponding to each node and each edge in the subgraph, and generating labels corresponding to each node and each edge based on the natural language meaning;

[0278] Based on the order of each node and each edge in the corresponding shortest path, the labels corresponding to each node and each edge are used to construct a corresponding natural language description as the path description corresponding to each of the shortest paths.

[0279] Optionally, in one embodiment, the generating module 208 is specifically used for:

[0280] Inputting the original question and the path description into a large language model;

[0281] Using a large language model, key information in the original question is identified, and the key information is integrated with the path description to obtain nodes and edges to be retrieved;

[0282] Through the large language model, the external knowledge graph is retrieved to obtain detailed information related to the nodes and edges to be retrieved;

[0283] The retrieved detailed information is used as the context information of the original question and the path description, and a final answer corresponding to the original question is generated and output through a large language model.

[0284] Since the embodiments of the device part correspond to the embodiments of the above-mentioned method, please refer to the above-mentioned method embodiments for the introduction of the retrieval-enhanced question-and-answer device provided by the present invention. The present invention will not be repeated here, and it has the same beneficial effects as the above-mentioned retrieval-enhanced question-and-answer method.

[0285] above Figure 2 The retrieval-enhanced question-answering apparatus in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The computer device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0286] Figure 3 : is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention. The computer device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 can be short-term storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the computer device 500. Furthermore, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the computer device 500.

[0287] The computer device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 3 The illustrated computer device structure does not constitute a limitation on the computer device, and may include more or fewer components than illustrated, or combine certain components, or arrange the components differently.

[0288] The present invention also provides a computer device, the computer device includes a memory and a processor, the processor includes a high-performance GPU processor, and is used to deploy a basic hardware environment for running a large model, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the retrieval-enhanced question-answering method in the above-mentioned embodiments. The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium, or a volatile computer-readable storage medium, and the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the steps of the retrieval-enhanced question-answering method.

[0289] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0290] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0291] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A question-answering method based on retrieval enhancement, characterized in that: The question answering method based on retrieval enhancement includes: Obtain the original question asked by the user in natural language; Construct a subgraph containing all entities and entity associations related to the original question; Taking the subgraph as a basic unit for graph neural network processing, an initial feature vector is assigned to each node in the subgraph; Input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on the attention mechanism for multiple rounds of reasoning to obtain a final representation of each node rich in context; identifying a number of target nodes that are most relevant to the original question from the final representation of all nodes of the subgraph; Taking the entities corresponding to each target node as candidate answers to the original question, and extracting the shortest paths from each entity in the original question to each candidate answer; Convert each extracted shortest path into a path description using natural language; Inputting the original question and the path description into a large language model to perform a retrieval enhancement generation task, and outputting a final answer corresponding to the original question; The construction of a subgraph containing all entities and entity association relationships related to the original question includes: The original question is parsed using natural language processing technology, and several key elements of the original question are extracted from the parsed results; Based on each of the key elements, forwardly traverse the preset knowledge graph to obtain a number of first key information directly related to the original question; Based on each of the first key information, construct a forward subgraph including all entities and entity association relationships directly related to the original question; Based on each entity in the forward subgraph, traverse the knowledge graph in reverse order to obtain a number of second key information directly related to each entity; Based on each of the second key information, construct a reverse subgraph including all entities and entity association relationships indirectly related to the original question; fusing the forward subgraph with the reverse subgraph to obtain a bidirectional fused subgraph; The step of inputting the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism for multiple rounds of reasoning to obtain a final context-rich representation of each node includes: Input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism; By using the graph neural network, the initial feature vector corresponding to each node in the subgraph is converted into a continuous vector space representation to obtain an embedding vector of each node and its neighboring nodes in the subgraph; Calculate the dot product of the embedding vector between each node and its corresponding neighbor nodes respectively to obtain the original attention score between the corresponding neighbor nodes; Normalize the attention scores of all neighbor nodes, and use the normalized attention scores to perform weighted summation on the Value vectors of all neighbor nodes to obtain the final context-rich representation of each node after update; Each attention layer of the graph neural network receives the node representation output by the previous attention layer as input, and outputs an updated node representation; the identifying of several target nodes most relevant to the original problem from the final representations of all nodes of the subgraph includes: Using a language model to encode the keywords and entities in the original question into vector representations, and concatenating the vector representations of the keywords and entities as the final representation of the original question; Using the attention mechanism, the first attention weight of the original question representation is calculated using a learnable first parameter matrix, and the second attention weight of each node representation is calculated using a learnable second parameter matrix; Based on the first attention weight and the second attention weight, using a dot product attention function to respectively calculate a similarity score between the original question representation and each node representation; According to the similarity scores corresponding to the nodes, a preset number of nodes with the highest scores are selected as target nodes most relevant to the original question.

2. The retrieval-enhanced question-answering method according to claim 1, characterized in that: The step of taking the entities corresponding to each target node as candidate answers to the original question and extracting the shortest paths from each entity in the original question to each candidate answer comprises: List all nodes and all edges in the subgraph to obtain a node list and an edge list, wherein each edge represents a relationship between two nodes; Constructing an adjacency matrix according to the node list and the edge list; In the adjacency matrix, the entity of the original question is set as the starting point and the candidate answer is set as the end point, and a queue is initialized to store the current node and the path from the starting point to the current node; The adjacency matrix is ​​traversed using a breadth-first search algorithm until an end point is found or all nodes are traversed, and the shortest path from each entity in the original question to each candidate answer is output.

3. The retrieval-enhanced question-answering method according to claim 2, characterized in that: The step of converting the extracted shortest paths into path descriptions in natural language comprises: Obtaining the natural language meaning corresponding to each node and each edge in the subgraph, and generating labels corresponding to each node and each edge based on the natural language meaning; Based on the order of each node and each edge in the corresponding shortest path, the labels corresponding to each node and each edge are used to construct a corresponding natural language description as the path description corresponding to each of the shortest paths.

4. The retrieval-enhanced question-answering method according to claim 3, characterized in that: The step of inputting the original question and the path description into a large language model to perform a retrieval enhancement generation task and outputting a final answer corresponding to the original question comprises: Inputting the original question and the path description into a large language model; Using a large language model, key information in the original question is identified, and the key information is integrated with the path description to obtain nodes and edges to be retrieved; Through the large language model, the external knowledge graph is retrieved to obtain detailed information related to the nodes and edges to be retrieved; The retrieved detailed information is used as the context information of the original question and the path description, and a final answer corresponding to the original question is generated and output through a large language model.

5. A question-answering device based on retrieval enhancement, characterized in that: The question-answering device based on retrieval enhancement includes: An acquisition module is used to acquire the original question raised by the user in natural language; A construction module, used to construct a subgraph including all entities and entity association relationships related to the original problem; An initialization module, used to use the subgraph as a basic unit for graph neural network processing and assign an initial feature vector to each node in the subgraph; An inference module, used to input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism for multiple rounds of inference to obtain a final representation of each node rich in context; A node identification module, used for identifying a number of target nodes most relevant to the original question from the final representation of all nodes of the subgraph; A path extraction module, used to take the entities corresponding to each target node as candidate answers to the original question, and extract the shortest paths from each entity in the original question to each candidate answer; A conversion module, used for converting each extracted shortest path into a path description in natural language; A generation module, used to input the original question and the path description into a large language model to perform a retrieval enhancement generation task, and output a final answer corresponding to the original question; Wherein, the building blocks are specifically used for: The original question is parsed using natural language processing technology, and several key elements of the original question are extracted from the parsed results; Based on each of the key elements, forwardly traverse the preset knowledge graph to obtain a number of first key information directly related to the original question; Based on each of the first key information, construct a forward subgraph including all entities and entity association relationships directly related to the original question; Based on each entity in the forward subgraph, traverse the knowledge graph in reverse order to obtain a number of second key information directly related to each entity; Based on each of the second key information, construct a reverse subgraph including all entities and entity association relationships indirectly related to the original question; fusing the forward subgraph with the reverse subgraph to obtain a bidirectional fused subgraph; Wherein, the reasoning module is specifically used for: Input the subgraph and the initial feature vector corresponding to each node into a pre-trained graph neural network based on an attention mechanism; By using the graph neural network, the initial feature vector corresponding to each node in the subgraph is converted into a continuous vector space representation to obtain an embedding vector of each node and its neighboring nodes in the subgraph; Calculate the dot product of the embedding vector between each node and its corresponding neighbor nodes respectively to obtain the original attention score between the corresponding neighbor nodes; Normalize the attention scores of all neighbor nodes, and use the normalized attention scores to perform weighted summation on the Value vectors of all neighbor nodes to obtain the final context-rich representation of each node after update; Each attention layer of the graph neural network receives the node representation output by the previous attention layer as input, and outputs an updated node representation; Wherein, the node identification module is specifically used for: Using a language model to encode the keywords and entities in the original question into vector representations, and concatenating the vector representations of the keywords and entities as the final representation of the original question; Using the attention mechanism, the first attention weight of the original question representation is calculated using a learnable first parameter matrix, and the second attention weight of each node representation is calculated using a learnable second parameter matrix; Based on the first attention weight and the second attention weight, using a dot product attention function to respectively calculate a similarity score between the original question representation and each node representation; According to the similarity scores corresponding to the nodes, a preset number of nodes with the highest scores are selected as target nodes most relevant to the original question.

6. A computer device, characterized in that: The computer device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the computer device to execute the retrieval-enhanced question-answering method according to any one of claims 1 to 4.

7. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the retrieval-enhanced question-answering method according to any one of claims 1 to 4 is implemented.

Images