An intelligent question-answering system and method based on problem decomposition and community semantic search

Abstract

The application discloses an intelligent question and answer system and method based on problem decomposition and community semantic search. The method first decomposes a complex problem into multiple single-hop sub-problems through an original cyclic problem decomposition technology, then retrieves the community in the knowledge graph based on a vector to obtain the information required to answer each sub-problem and uses a large language model to answer the sub-problems, and finally uses an LLM to combine the answers to all sub-problems to generate the final answer to the original complex problem. Compared with other methods in the industry, the method significantly improves the information integration capability, avoids information fragmentation through community-level search and summary generation, and provides high-quality and logically consistent answers. In handling complex queries, the method outperforms the industry's general method, with a maximum improvement of 29%. At the same time, the method significantly improves the efficiency while ensuring better question and answer results. Under the premise of better results than the existing optimal method GraphRAG, the token consumption is only 3.2% of that of GraphRAG.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent question answering systems, in particular to an intelligent question answering system and method based on question decomposition and community semantic search. BACKGROUND

[0002] Natural Language Processing (NLP) technology has developed rapidly in recent years, especially in the direction of Large Language Models (LLM), such as ChatGPT3, ChatGPT4, and Llama3.1. These models have shown excellent language generation and reasoning capabilities. However, when dealing with complex, multi-step reasoning queries, relying solely on language models has the following key problems: 1. Information hallucination: the answers generated by the model may contain information that does not conform to the real world or is irrelevant to the question, especially when complex queries require reasoning or integration of multiple information sources; 2. Context window limitation: the context window of LLM is limited, and when faced with large-scale data sets that are not in the training corpus, it cannot understand all the information at once.

[0003] To solve the above problems when LLM works alone, Retrieval-Augmented Generation (RAG) technology has emerged. Traditional RAG combines semantic vector retrieval to add the most similar text blocks in the private data set to the model context to reduce hallucination and improve the accuracy of answers. However, traditional RAG based on vector retrieval has some inherent defects: 1. Information fragmentation: the returned text blocks can only represent part of the semantics of the original text, making it difficult to integrate into a logical whole answer; 2. Limited reasoning ability: the similarity based on vector calculation only matches the surface semantic similarity of the text, and cannot successfully match the text with deep-level associations or relationships that require reasoning.

[0004] To solve the problems of traditional RAG, the industry has proposed a method called GraphRAG, which constructs a knowledge graph to structure entities and relationships in the data, and uses community detection to aggregate information. For each question, GraphRAG uses the Map-Reduce algorithm to obtain valuable information in each community and adds it to the context window of the LLM to generate the final answer.

[0005] Although GraphRAG has made a breakthrough in global information integration, its Map-Reduce algorithm relies on frequent calling of LLM to retrieve and process communities, token consumption is huge, single query cost is extremely high, and efficiency decreases sharply with the increase of the size of the data set, which seriously limits its popularization in practical applications. SUMMARY

[0006] In view of the deficiencies of the prior art, the present application provides an intelligent question answering system and method based on problem decomposition and community semantic search, which solves the problems mentioned in the above background art. Specifically, the present application is realized by the following technical solutions:

[0007] 1) Knowledge graph construction module: used for converting large-scale unstructured private data sets into structured knowledge graphs, and clustering entities in the knowledge graph according to semantics to form communities and generating descriptions of each community for subsequent retrieval;

[0008] 2) Problem decomposition and processing module: used for splitting complex (multi-hop) problems into several simple (single-hop) sub-problems to support vector-based community retrieval;

[0009] 3) Community retrieval module: used for screening several communities with the highest degree of relevance to the problem and adding them to the context of LLM to generate answers to the sub-problems;

[0010] 4) Information evaluation and answer generation module: used for evaluating whether the provided sub-problem answers are sufficient to answer the original complex problem, if not, repeating steps 2 to 3, if yes, adding all sub-problems, their answers and the original problem to the context of LLM to generate the final answer to the original problem. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 Flowchart of the RAG method provided by the present application;

[0012] Figure 2 Subgraph merging algorithm provided by the present application.

[0013] ADVANTAGEOUS EFFECTS

[0014] The present application provides an intelligent question answering system and method based on problem decomposition and community semantic search. Compared with the prior art, the present application has the following advantageous effects:

[0015] 1.The information integration capability of the present application for complex problems is significantly improved: through knowledge graph and community-level summary generation, the system can extract relevant content from multiple source information and effectively combine logical global answers; dynamic expansion is enhanced: by combining vector embedding and community detection, the present application can maintain high flexibility when processing dynamic data, adapting to changing data sets and real-time query requirements.

[0016] 2.The present application improves the efficiency of multi-hop query: by combining vector retrieval and community search, the method reduces LLM calls, thereby reducing token consumption and successfully improving efficiency, as shown in the table, the method proposed by the present application achieves better performance than GraphRAG, with only 3.2% of token consumption.

[0017] In summary, the present application combines knowledge graph, vector retrieval and community detection, optimizes the query process by query decomposition, solves the defects of the prior art, and realizes efficient information retrieval and integration capability. Compared with systems such as GraphRAG, the present application has obvious advantages in query performance, resource consumption and answer quality, etc. These improvements not only improve the application value of the system, but also expand its application scenarios in real-time and multi-hop queries. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described below in a clear and complete manner. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0019] Embodiment 1

[0020] An intelligent question and answer system based on problem decomposition and community semantic search, comprising:

[0021] A knowledge graph construction module: for converting large-scale unstructured private data sets into structured knowledge graphs, the module simultaneously clusters entities in the knowledge graph according to semantics and generates a description of each community for subsequent retrieval, the specific process is as follows:

[0022] 1) The original large-scale data set is divided into text blocks (Chunk) with a size within the LLM window;

[0023] 2) Use LLM to extract entities and relationships in each chunk, each entity has three attributes: name, category and description, and each relationship has four attributes: source entity, target entity, relationship description and relationship strength;

[0024] 3) Model each chunk as a subgraph, where nodes are entities extracted in the previous step, edges are relations extracted in the previous step, and the weight of each edge is the strength of the relation obtained in the previous step;

[0025] 4) Merge all subgraphs by merging the same entities to obtain the final knowledge graph representing the original dataset, and the merging algorithm implemented by the present application is shown in Figure 3;

[0026] 5) Use the Leiden clustering algorithm on the knowledge graph to discover closely connected entities in the graph;

[0027] 6) For each community, sort the entities in it according to their prominence, then add the entity descriptions to the context of the LLM until the window size is reached, and finally use the LLM to summarize the entity descriptions and relation descriptions in the community to obtain a community description;

[0028] ;

[0029] 7) Finally, use an embedding model to encode the description of each community into a high-dimensional vector space and store it in the knowledge base, which facilitates subsequent semantic similarity calculations with questions,

[0030]

[0031] Question decomposition and processing module: used to split complex (multi-hop) questions into several simple (single-hop) sub-questions to support vector-based community retrieval, the specific process is as follows:

[0032] 1) First, through prompt engineering, ask the LLM to evaluate the complexity of the question according to the pre-defined standard, specifically, when a question needs to be queried multiple times in the knowledge base, it is defined as a complex question;

[0033] 2) If the LLM identifies the original question as a complex question in the previous step, ask the LLM to decompose it into a simple question, i.e. a question that only needs to be queried once in the knowledge base; if the LLM considers the original question to be a simple question, ask the LLM to return the original question directly.

[0034] Community retrieval module: used to filter out a number of community descriptions with the highest relevance to the question and add them to the context of the LLM to generate the answer to the sub-question, the specific process is as follows:

[0035] 1) First, use the embedding model to transform the sub-question obtained in the previous step into the vector space of the community descriptions generated earlier, which facilitates semantic similarity calculations;

[0036]

[0037] 2) Then select the top 2k community descriptions with the highest semantic similarity as candidate objects, and then use a re-ranking model to select the top k community descriptions as the final LLM context, and require it to generate answers to the sub-questions;

[0038] ;

[0039] Information evaluation and answer generation module: for evaluating whether the provided sub-question answer is sufficient to answer the original complex question, the specific process is as follows:

[0040] 1) Through prompt word engineering, if the LLM considers that the current question and answer pair is not sufficient to answer the original question, the question decomposition and community retrieval process is performed again, and this process is repeated until a question and answer pair sufficient to answer the original question is obtained;

[0041] 2) If the LLM considers that the existing question and answer pair (i.e., the sub-question and its answer) provides the information required to answer the original question, then directly generate an answer to the original question.

[0042] Table 1 Answer and question effect of the present application

[0043]

[0044] The scores in the table are given by ChatGPT-4o according to publicly recognized evaluation standards in the industry.

[0045] Table 2 Efficiency comparison of the present method and other mainstream methods

[0046]

[0047] The contents not described in detail in the specification all belong to the prior art known to those skilled in the art.

[0048] It should be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment.

[0049] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1. An intelligent question answering system based on problem decomposition and community semantic search, characterized in that: 1) a knowledge graph construction module: for converting a large-scale unstructured private data set into a structured knowledge graph, and simultaneously clustering entities in the knowledge graph according to semantics into communities and generating a description of each community for subsequent retrieval; 2) a problem decomposition and processing module: for splitting a complex problem into several simple sub-problems to support community retrieval based on vectors; 3) a community retrieval module: for screening out several community descriptions with the highest degree of relevance to the problem and adding them to the context of the LLM to generate answers to the sub-problems; an information evaluation and answer generation module: for evaluating whether the provided sub-problem answers are sufficient to answer the original complex problem; wherein the knowledge graph construction module is used to convert a large-scale unstructured private data set into a structured knowledge graph, and to cluster entities in the knowledge graph according to semantics into communities and generate a description of each community, the module specifically comprising: 1) dividing the original data set into multiple text blocks, each text block being within the LLM window in size; 2) extracting entities and relationships from each text block using LLM, wherein each entity includes a name, a category, and a description, and each relationship includes a source entity, a target entity, a relationship description, and a relationship strength; 3) modeling each text block as a sub-graph, where nodes are entities and edges are relationships, and the weight of the edge is the relationship strength; 4) merging multiple sub-graphs to obtain the final knowledge graph, and performing semantic clustering on the entities in the knowledge graph using the Leiden clustering algorithm to discover closely connected entities; 5) sorting the entities in each community by significance and adding entity descriptions to the LLM context until the window size is reached, then using LLM to generate a description of each community; using an embedding model to encode the description of each community into a high-dimensional vector and store it in the knowledge base; wherein the problem decomposition and processing module is used to decompose a complex problem into several simple sub-problems to support community retrieval based on vectors, the module comprising: 1) using prompt word engineering to make LLM evaluate the complexity of the problem, if the problem is a complex problem, it is decomposed into multiple simple problems, each of which only needs one query to the knowledge base to get the answer; if the problem is a simple problem, the problem answer is returned directly; wherein the community retrieval module is used to screen out several community descriptions with the highest degree of relevance to the sub-problem and add them to the LLM context, the module comprising: 1) converting the sub-problem into a vector representation and calculating the semantic similarity between the sub-problem and each community description; 2) selecting the top-k community descriptions with the highest semantic similarity and using a re-ranking model to select the final top-k community descriptions as the information source in the LLM context to generate an answer to the sub-problem.

2. The intelligent question answering system of claim 1, wherein, the information evaluation and answer generation module is used to evaluate whether the provided sub-problem answers are sufficient to answer the original complex problem, the module comprising: 1) If the LLM considers that the existing sub-question answers are not sufficient to answer the original question, the process of question decomposition and community retrieval is restarted until sufficient answer information is obtained; 2) If the LLM considers that the existing answers are sufficient to answer the original question, the final answer to the original question is directly generated.

3. The intelligent question answering system of claim 1, wherein, The embedding model is used to convert community descriptions and sub-questions into high-dimensional vectors, and to calculate semantic similarity based on vector space.

4. The intelligent question answering system of claim 1, wherein, The Leiden clustering algorithm is used to cluster entities based on the connectivity of entities in the graph, and to discover groups of entities with similar semantics.

Images