Intelligent question answering system and method based on question decomposition and community semantic search

Abstract

The invention discloses an intelligent question answering system and method based on question decomposition and community semantic search. According to the method, firstly, a complex question is decomposed into a plurality of single-hop sub-questions through an original loop question decomposition technology, then information needed for answering each sub-question is obtained based on communities in a vector retrieval knowledge graph, and the sub-questions are answered by using a large language model. And finally, generating a final question and answer for the original complex question by combining the LLM with the answers of all the sub-questions. Compared with other methods in the industry, the method has the advantages that the information integration capability is remarkably improved, information fragmentation is avoided through community-level search and abstract generation, and high-quality and logically consistent answers are provided. When a complex query is processed, the effect of the method is superior to that of a general method in the industry and is improved by 29% at most; meanwhile, on the basis of ensuring a better question and answer effect, the efficiency is remarkably improved, and on the premise that the effect exceeds that of an existing optimal method GraphRAG, the token consumption is only 3.2% of that of the existing optimal method GraphRAG.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent question answering systems, and specifically to an intelligent question answering system and method based on question decomposition and community semantic search. Background Art

[0002] Natural Language Processing (NLP) technology has developed rapidly in recent years, especially in the direction of large language models (LLM), such as ChatGPT3, ChatGPT4, and Llama3.1. These models have demonstrated excellent language generation and reasoning capabilities. However, when dealing with complex, multi-step reasoning queries, language models alone have the following key problems: 1. Information hallucination: The answers generated by the model may contain content that is inconsistent with real-world information or irrelevant to the question, especially when complex queries require reasoning or integration of multiple information sources; 2. Context window limitation: LLM has a limited context window. When faced with a large-scale data set that is not in the training corpus, it cannot understand all the information at once.

[0003] In order to solve the above problems when LLM works alone, the Retrieval-Augmented Generation (RAG) technology came into being. Traditional RAG combines semantic vector retrieval to add the text blocks most similar to the question in the private dataset to the model context to reduce hallucinations and improve the accuracy of the answer. However, traditional vector retrieval-based RAG has some inherent defects: 1. Information fragmentation: The returned text blocks can only represent part of the semantics of the original text, and it is difficult to integrate them into a logical overall answer; 2. Limited reasoning ability: The similarity calculated based on vectors essentially only matches texts with similar surface semantics, and cannot successfully match deeply related texts or texts that require inference to derive relationships.

[0004] In order to solve the problems of traditional RAG, the industry has proposed a method called GraphRAG, which is constructed through the knowledge graph to structure the entities and relationships in the data, and realizes information aggregation through community detection. For each question, GraphRAG uses the Map-Reduce algorithm to obtain valuable information in each community, and adds it to the context window of LLM to generate the final answer.

[0005] Although GraphRAG has made breakthroughs in global information integration, its Map-Reduce algorithm relies on frequent calls to LLM to retrieve and process communities, which consumes huge tokens, has extremely high costs for a single question and answer, and its efficiency drops sharply as the size of the data set increases, which seriously limits its promotion in practical applications. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides an intelligent question-answering system and method based on question decomposition and community semantic search, which solves the problems mentioned in the above background technology. Specifically, the present invention is implemented through the following technical solutions: 1) Knowledge graph construction module: used to convert large-scale unstructured private datasets into structured knowledge graphs. This module also clusters entities in the knowledge graph into communities based on semantics and generates descriptions of each community for subsequent retrieval; 2) Problem decomposition and processing module: used to split complex (multi-hop) problems into several simple (single-hop) sub-problems to support vector-based community retrieval; 3) Community retrieval module: used to filter out several community descriptions that are most relevant to the question and add them to the context of LLM to generate answers to sub-questions; 4) Information evaluation and answer generation module: used to evaluate whether the answers to the sub-questions provided are sufficient to answer the original complex question. If not, repeat steps 2 to 3. If sufficient, add all the sub-questions and their answers together with the original question into the LLM context to generate the final answer to the original question. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 Fig. 1 is a flow chart of the RAG method proposed by the present invention; Figure 2 The question-answering effect of the RAG method provided by the present invention on the public dataset UltraDomain; Figure 3 This is a diagram of the subgraph merging algorithm provided by the present invention.

[0008] Beneficial Effects The present invention provides an intelligent question-answering system and method based on question decomposition and community semantic search. Compared with the prior art, it has the following beneficial effects: 1. The information integration capability of the present invention for complex problems is significantly improved: through knowledge graph and community-level summary generation, the system can extract relevant content from multiple source information and effectively combine logical global answers; dynamic scalability is enhanced: by combining vector embedding with community detection, the present invention can maintain high flexibility when processing dynamic data and adapt to the needs of constantly changing data sets and real-time queries.

[0009] 2. The present invention improves the efficiency of multi-hop query: By combining vector retrieval with community search, the present method reduces LLM calls, thereby reducing token consumption and successfully improving efficiency. As shown in the table, the method proposed in the present invention achieves better performance than GraphRAG while consuming only 3.2% of its tokens.

[0010] In summary, the present invention combines knowledge graph, vector retrieval and community detection, and adopts query decomposition to optimize the query process. While solving the defects of the prior art, it realizes efficient information retrieval and integration capabilities. Compared with systems such as GraphRAG, the present invention has obvious advantages in query performance, resource consumption and answer quality. These improvements not only enhance the application value of the system, but also expand its application scenarios in real-time, multi-hop queries. DETAILED DESCRIPTION

[0011] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] Example 1 An intelligent question-answering system based on question decomposition and community semantic search, comprising: Knowledge graph construction module: used to convert large-scale unstructured private data sets into structured knowledge graphs. This module also clusters entities in the knowledge graph into communities based on semantics and generates descriptions of each community for subsequent retrieval. The specific process is as follows: 1) Divide the original large-scale data set into chunks (Chunks) whose size is within the LLM window; 2) LLM is used to extract entities and relationships in each chunk. Each entity has three attributes: name, category, and description. Each relationship has four attributes: source entity, target entity, relationship description, and relationship strength. 3) Model each chunk as a subgraph, where the nodes are the entities extracted in the previous step, the edges are the relationships extracted in the previous step, and the edge weights are the relationship strengths obtained in the previous step; 4) By merging the same entities, all subgraphs are merged to obtain the final knowledge graph for representing the original data set. The merging algorithm implemented by the present invention Figure 3 shown.

[0013] 5) Use the Leiden clustering algorithm on the knowledge graph to discover closely connected entities in the graph; 6) For each community, sort the entities in it according to prominence, then add the entity description to the context of LLM until the window size is reached, and finally use LLM to summarize the entity description and relationship description in the community to obtain the community description.

[0014] 7) Finally, the description of each community is encoded into a high-dimensional vector space using an embedding model and stored in the knowledge base to facilitate the subsequent calculation of the semantic similarity with the question.

[0015] Problem decomposition and processing module: used to split complex (multi-hop) problems into several simple (single-hop) sub-problems to support vector-based community retrieval. The specific process is as follows: 1) The present invention first requires LLM to evaluate the complexity of the problem according to predefined criteria through prompt engineering. Specifically, when a problem requires multiple queries to the knowledge base, the present invention defines it as a complex problem; 2) If LLM identifies the original question as a complex question in the previous step, the present invention requires LLM to decompose it into a simple question, that is, a question that can be answered by querying the knowledge base only once; if LLM considers the original question to be a simple question, the present invention requires LLM to directly return to the original question.

[0016] Community retrieval module: It is used to filter out several community descriptions that are most relevant to the question and add them to the context of LLM to generate answers to sub-questions. The specific process is as follows: 1) The present invention first uses an embedding model to transform the sub-query obtained in the previous step into the vector space where the community description generated previously is located, so as to facilitate the calculation of semantic similarity; 2) Then select the top with the highest semantic similarity 2k community descriptions as candidates, and then use the reranking model to select the top kThe community description is used as the final LLM context, which is required to generate answers to the sub-questions; Information evaluation and answer generation module: used to evaluate whether the answers to the sub-questions provided are sufficient to answer the original complex question. The specific process is as follows: 1) Through the prompt word engineering, if LLM believes that the current question-answer pair is not sufficient to answer the original question, the question decomposition and community search process will be repeated. This process will be repeated until a question-answer pair sufficient to answer the original question is obtained.

[0017] 2) If LLM believes that the existing question-answer pair (i.e., the sub-question and its answer) provides the information needed to answer the original question, it directly generates the answer to the original question.

[0018] Table 1 The effect of this method on question answering in People’s Daily.

[0019] The scores in the table are given by ChatGPT-4o based on publicly recognized evaluation standards in the industry.

[0020] Table 2 Comparison of the efficiency of this method and other mainstream methods The contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0021] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0022] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent question-answering system based on question decomposition and community semantic search, characterized by: 1) Knowledge graph construction module: used to convert large-scale unstructured private datasets into structured knowledge graphs. This module also clusters entities in the knowledge graph into communities based on semantics and generates descriptions of each community for subsequent retrieval; 2) Problem decomposition and processing module: used to split complex (multi-hop) problems into several simple (single-hop) sub-problems to support vector-based community retrieval; 3) Community retrieval module: used to filter out several community descriptions that are most relevant to the question and add them to the context of LLM to generate answers to sub-questions; 4) Information evaluation and answer generation module: used to evaluate whether the answers to the sub-questions provided are sufficient to answer the original complex question.

2. The intelligent question-answering system according to claim 1, wherein: The knowledge graph construction module is used to convert large-scale unstructured private data sets into structured knowledge graphs, cluster entities in the knowledge graph into communities based on semantics, and generate descriptions of each community. The module specifically includes: 1) Divide the original data set into multiple text blocks, and the size of each text block is within the LLM window; 2) LLM is used to extract entities and relations from each text block, where each entity includes name, category and description, and each relation includes source entity, target entity, relation description and relation strength; 3) Model each text block as a subgraph, where nodes are entities, edges are relationships, and edge weights are relationship strengths; 4) Merge multiple subgraphs to obtain the final knowledge graph, and use the Leiden clustering algorithm to semantically cluster the entities in the knowledge graph to discover closely connected entities; 5) Sort the entities in each community by significance and add the entity descriptions to the LLM context until the window size is reached, then use LLM to generate a description for each community; 6) Use the embedding model to encode the description of each community into a high-dimensional vector and store it in the knowledge base.

3. The intelligent question-answering system according to claim 1, wherein: The problem decomposition and processing module is used to decompose a complex problem into several simple sub-problems to support vector-based community retrieval, and the module includes: 1) Through the prompt word engineering, LLM evaluates the complexity of the problem. If the problem is complex, it is decomposed into multiple simple questions. Each simple question only needs to query the knowledge base once to get the answer; 2) If the question is a simple one, the answer will be returned directly.

4. The intelligent question-answering system according to claim 1, wherein: The community search module is used to filter out several community descriptions with the highest degree of relevance to the sub-questions and add them to the LLM context. The module includes: 1) Convert the sub-problem into a vector representation and calculate the semantic similarity between the sub-problem and each community description; 2) Select the top-k community descriptions with the highest semantic similarity, and use the reranking model to select the final top-k community descriptions as the information source in the LLM context to generate answers to sub-questions.

5. The intelligent question-answering system according to claim 1, wherein: The information evaluation and answer generation module is used to evaluate whether the provided sub-question answers are sufficient to answer the original complex question, and the module includes: 1) If LLM believes that the existing sub-question answers are not sufficient to answer the original question, it will redo the process of question decomposition and community retrieval until sufficient answer information is obtained; 2) If LLM believes that the existing answer is sufficient to answer the original question, it will directly generate the final answer to the original question.

6. The intelligent question-answering system according to claim 1, wherein: The embedding model is used to transform community descriptions and sub-problems into high-dimensional vectors and calculate semantic similarity based on the vector space.

7. The intelligent question-answering system according to claim 1, wherein: The Leiden clustering algorithm is used to cluster entities based on the connectivity of entities in a graph and to discover entity groups with similar semantics.

Images