Knowledge graph enhanced RAG intelligent question and answer method based on complex process

Abstract

The invention provides a knowledge graph enhanced RAG intelligent question-answering method based on a complex process, which comprises the following steps: uploading a heterogeneous data file, creating an embedded vector and a vector index, and obtaining a private database; performing entity relationship extraction on the private database to obtain a knowledge graph, and combining the knowledge graph with the large model to obtain a query model; using the query model to decompose user questions, identify question types, question intentions and entity relationships, then judging whether related entities exist in the knowledge graph, if yes, outputting answers by using the query model, and if not, calling the large language model to perform question and answer reasoning; and constructing a historical question and answer abstract, and optimizing answers. According to the method, the accuracy and the credibility of answering are improved, meanwhile, the visual representation of the knowledge graph also enables the user to more intuitively understand how the system obtains a specific answer, and the interactivity and the credibility between the user and the system are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of search question answering technology, and in particular to a knowledge graph-enhanced RAG intelligent question answering method based on complex processes. Background Art

[0002] With the advent of the big data era, people are faced with the challenge of accurately and quickly acquiring the required knowledge from huge and messy data. Although existing search engines can perform fuzzy matching searches based on keywords, the search results are often unsatisfactory for complex queries or fuzzy query conditions. In addition, existing search methods simply list the search results and lack the ability to deeply understand the search results and perform natural language processing, which limits the efficiency and utilization of knowledge acquisition.

[0003] As an emerging technology, large language models can handle complex queries more efficiently and accurately, provide users with the answers they need, and present the results in a user-friendly way. However, due to the characteristics of their technical architecture, these models have a certain lag in processing real-time information updates, resulting in illusions and errors in the generated answers. Moreover, large models have limited in-depth understanding of professional fields, so they perform poorly when answering questions in professional fields. In order to solve these problems, researchers began to explore how to introduce external knowledge bases to improve the accuracy and authenticity of large model question-answering systems. These external knowledge bases can include various documents, pictures, web page information, etc., but these contents usually exist in an independent form and lack direct associations and connections, resulting in large language models not being able to make good use of these contents and unable to give more accurate and comprehensive answers. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a knowledge graph-enhanced RAG intelligent question-answering method based on complex processes, which improves the accuracy and credibility of the answers. At the same time, the visual representation of the knowledge graph also enables users to more intuitively understand the process of how the system arrives at a specific answer, thereby enhancing the interactivity and trust between users and the system.

[0005] To achieve the above object, the present invention provides the following solution: a knowledge graph-enhanced RAG intelligent question answering method based on complex processes, comprising the following steps:

[0006] Uploading heterogeneous data files, performing data conversion and text block processing on the heterogeneous data files to obtain multiple text blocks, and performing embedding vector generation and vector index creation for each of the text blocks to obtain a private database;

[0007] Extract entity relationships from the private database, construct a knowledge graph, and combine the knowledge graph with the big model to obtain a query model;

[0008] Using the query model, decompose the user's question, identify the question type, question intent and entity relationship, and then determine whether there is a related entity in the knowledge graph. If so, use the query model to output the answer. If not, call the large language model for reasoning question answering;

[0009] A historical question and answer library for storing historical question and answer records is constructed in the query model, and a historical question and answer summary for optimizing answers is generated based on the historical question and answer records.

[0010] Optionally, uploading a heterogeneous data file, performing data conversion and text block processing on the heterogeneous data file to obtain multiple text blocks, and performing embedding vector generation and vector index creation for each text block to obtain a private database, including:

[0011] Load the uploaded heterogeneous data file, use image recognition and natural language processing technology to perform format conversion and data consistency processing, and then divide the heterogeneous data file into blocks to obtain multiple text blocks; when the heterogeneous data file is a PDF file, use a PDF parser to extract the content, and when the heterogeneous data file is other file types, convert the heterogeneous data file of other file types into Markdown format, and then perform data processing;

[0012] Generate a unique identifier for each text block using a hash value, record the position, length and offset of each text block, and encapsulate the recorded information as metadata, and establish an association between the text block and the heterogeneous data file using the Cypher language based on the metadata;

[0013] For each of the text blocks, an embedding vector is generated and a vector index is created, and the embedding vector and the vector index are updated using the Cypher language to obtain a private data set library.

[0014] Optionally, entity relationships are extracted from the private database to construct a knowledge graph, and the knowledge graph is combined with a large model to obtain a query model, including:

[0015] Preset a big model prompt word for guiding the big model, combine the big model configuration information, and call different big models to perform entity extraction and entity relationship extraction on the private database;

[0016] Perform data cleaning and classification on the extraction results, and then classify and save the extraction results.

[0017] According to the extraction results, the preset Cypher language is used to establish connections between entity nodes and text blocks to obtain a knowledge graph, which is visualized and combined with a large model to obtain a query model.

[0018] Optionally, the query model is used to decompose the user question, identify the question type, question intent and entity relationship, and then determine whether there is a relevant entity in the knowledge graph. If so, the query model is used to output the answer. If not, the large language model is called to perform reasoning question answering, including:

[0019] Record the text content of the user's question, and attach a timestamp, relevant context information, and an ID to indicate whether the session has ended.

[0020] Use natural language processing technology to identify the type and intention of user questions, and classify user questions into factual query and fuzzy query categories;

[0021] Extracting question entities from the user's question, and identifying question entity relationships in the private database according to the question entities and the large model prompt words;

[0022] Based on the question entity relationship, the knowledge graph is retrieved to determine whether there is a related question entity. If so, the query model is used to search and output the answer, and a scoring mechanism is set in the query model to indicate the degree of match between the question entity and the knowledge graph. If not, a large language model is called to perform reasoning question and answering.

[0023] Optionally, based on the question entity relationship, the knowledge graph is retrieved to determine whether there is a related question entity. If so, the query model is used to search and output the answer, and a scoring mechanism for indicating the degree of match between the question entity and the knowledge graph is set in the query model, including:

[0024] Perform entity initialization processing on the user query, create an initial block set, and score each initial block according to its importance and context relevance to obtain an initial score;

[0025] Matching the initial block set through the knowledge graph and the private database, aggregating the information of all relevant initial blocks using the collect function to obtain an aggregated block set, and calculating the average score of the aggregated block set according to the initial score;

[0026] Expand the aggregate block using the UNWIND operation, find related entities, and perform entity expansion to obtain an expanded entity set;

[0027] Counting and sorting the extended entity set, calculating the number of entity occurrences, and then limiting the number of returned entities based on the entity calculation and sorting results;

[0028] According to the average score and the ranking result, the optimal expansion path and the number of expansion steps of the entity are selected, and then the expansion path and the expansion entity set are deduplicated to obtain a result entity set;

[0029] The result entity set is returned, and an entity association graph is constructed, the entity association graph is visualized, and the entity association graph is converted into natural language to output the final answer.

[0030] The present invention discloses the following technical effects by providing a knowledge graph-enhanced RAG intelligent question-answering method based on complex processes:

[0031] 1. Personalized question-and-answer service: The present invention can conduct accurate question-and-answer based on the personalized knowledge base provided by the user, avoiding the universal problems in general large-model question-and-answer. Through a customized knowledge base, users can obtain higher-quality answers and responses, thereby significantly improving the user experience. Especially in the field of complex processes, the existing large-model knowledge base often lacks relevant information and cannot effectively answer specific questions. The personalized knowledge base constructed using uploaded external files can provide specific answers to relevant questions, reducing the risk of hallucinations in large models when processing unknown content.

[0032] 2. Visual knowledge display: The present invention displays the user's knowledge base in a visual way, presenting the originally scattered knowledge information in a graphical form, helping users to intuitively understand complex associations. The system can automatically merge relevant information so that users can grasp the connection between knowledge points at a glance when viewing the knowledge graph. This graphical display not only improves the manageability of knowledge, but also promotes the user's in-depth understanding of the knowledge structure, laying a good foundation for subsequent learning and application.

[0033] 3. Multi-data integration advantage: The present invention has the ability to integrate multiple types of data, and builds a more complete and accurate domain-specific knowledge system by continuously expanding and updating the knowledge graph. The system can effectively integrate data from different sources to form a unified knowledge framework. This data integration advantage enables users to obtain the required information more quickly and effectively when facing complex problems, improve work efficiency, and promote scientific decision-making.

[0034] 4. Enhanced intelligent decision support: Through the combination of personalized knowledge base and knowledge graph, the present invention can provide users with more targeted decision support. The system can not only quickly retrieve and analyze data, but also make intelligent recommendations based on the user's historical questions and answers, thereby providing users with more practical suggestions. As data accumulates, the system will continuously optimize its response capabilities and further improve the accuracy and timeliness of intelligent decision-making.

[0035] 5. Continuous learning and adaptability: The knowledge graph construction and update mechanism adopted by the present invention enables the system to have the ability of continuous learning. As users continue to upload new data and information, the system can dynamically update the knowledge base to ensure that it is always up to date. This adaptability enables the system to respond to changing user needs and industry development trends, providing users with lasting support.

[0036] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 A schematic diagram of a method flow chart provided by an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of a knowledge graph construction process based on heterogeneous data files provided in an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of the intelligent question-answering process based on the knowledge graph provided in an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of the application process of the historical question and answer record in the question and answer system provided by an embodiment of the present invention;

[0042] Figure 5 A schematic diagram of the entity relationship-based knowledge graph query process provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] like Figure 1As shown, the present invention provides a knowledge graph-enhanced RAG intelligent question answering method based on complex processes, comprising the following steps:

[0046] 1. If Figure 2 As shown, a heterogeneous data file is uploaded, data conversion and text block processing are performed on the heterogeneous data file to obtain multiple text blocks, and an embedding vector is generated and a vector index is created for each text block to obtain a private database. Including:

[0047] Load the uploaded heterogeneous data files. Heterogeneous data files support multiple file types, such as PDF, Word, PPT, etc. In addition, you can also upload a web address as the data source.

[0048] Image recognition and natural language processing technologies are used to perform format conversion and data consistency processing. Specifically, the images and table data in the file are converted into natural language descriptions and merged with other text content to ensure data consistency. For PDF files, the dedicated PDF parser PyMuPDFLoader in Langchain is used to extract the images, tables, and text content in the page; for other file types, UnstructuredFileLoader is used to load the content. Since UnstructuredFileLoader does not handle tables and images well, it is recommended to convert them to Markdown format before loading other files, which can better handle table data.

[0049] The heterogeneous data file is then divided into blocks to obtain multiple text blocks. Since the file content is usually long and the context length of the large model conversation is limited, the document needs to be sliced ​​to adapt to the processing capacity of the model. The text_splitters method in Langchain is used to divide the document into blocks, and different block sizes are set according to the size of the file.

[0050] After the document is divided into blocks, the blocks are independent and lack relevance, which will affect subsequent queries. Therefore, it is necessary to establish associations for the document blocks. Specifically, a unique identifier is generated for each text block using a hash value, and the position, length and offset of each text block are recorded. The recorded information is encapsulated as metadata. Based on the metadata, the association between the text block and the heterogeneous data file is established using the Cypher language, and the blocks are sorted according to the document order to ensure the sequential connection between the document blocks.

[0051] For each of the text blocks, an embedding vector is generated and a vector index is created. According to the preset embedding model, a vector is generated for each document block, and a vector index is created. Cosine similarity is used as the similarity function, and different vector dimensions are specified according to different preset embedding models. Finally, the embedding vector and vector index are updated using the Cypher language to obtain a private dataset library.

[0052] 2. If Figure 2 As shown, entity relationships are extracted from the private database, a knowledge graph is constructed, and the knowledge graph is combined with the large model to obtain a query model. Including:

[0053] 2.1 Preset big model prompt words for guiding the big model. The main content of the prompt words is to tell the big model what work needs to be done, what are the limitations, and provide some examples for the big model to help it complete the work.

[0054] During the extraction process, attention should be paid to the processing of Markdown format tables, that is, when identifying a specific format, inform the big model that this part is tabular data. Secondly, the extracted content needs to be organized and classified according to the structure of "head entity, head entity type, tail entity, tail entity type, relationship, relationship type". Specifically, the head entity and the tail entity form a relationship pair, the relationship represents the connection between the two entities, and the type is the classification of the relationship. Finally, in order to help the big model better understand the task, a concise and clear example will be provided to show how to annotate the head entity, head entity type, tail entity, tail entity type, relationship, and relationship type for reference by the big model.

[0055] 2.2 In combination with the big model configuration information, different big models are called to perform entity extraction and entity relationship extraction on the private database.

[0056] Use the invoke method in Langchain to extract entities and relationships from the sliced ​​text content. For the same text, you need to call the large model multiple times for extraction, and verify the matching of each extraction result with the original text content by comparison to ensure that all fragments are fully extracted. Based on the effect of each extraction, determine whether it is necessary to continue entity extraction for the next text slice.

[0057] 2.3 Clean the extracted data and process any special characters that may appear to prevent errors during subsequent saving. Then, classify the extracted data according to the extracted content. Although the extraction results of the large model are displayed in the form of {head entity-relationship-tail entity}, when saving, the same type of extraction results will be classified and saved in a unified format.

[0058] 2.3 Based on the extraction results, the preset Cypher language is used to establish connections between entity nodes and text blocks to obtain a knowledge graph, the knowledge graph is visualized, and the knowledge graph is combined with the large model to obtain a query model. The visualization interface can be divided into two types: file relationship and entity relationship, which are used to display the relationship between files and entities and the relationship between entities.

[0059] 3. If Figure 3-4 As shown, the query model is used to decompose user questions, identify question types, question intent and entity relationships, and then determine whether there are relevant entities in the knowledge graph. If so, the query model is used to output answers. If not, the large language model is called for reasoning question answering. Specifically, it includes:

[0060] 3.1 When a user inputs a query or question through the system interface, the system will record the text content of the user's question, and attach a timestamp, relevant context information, and an ID identifier to the user's question to indicate whether the conversation has ended; the ID identifier is used to identify this round of questions and answers. If the ID identifier remains unchanged, it means that this round of conversation has not ended, and the context data provided by the user will be used as a reference for subsequent questions and answers.

[0061] 3.2 Use natural language processing technology to identify the type and intention of user questions, determine the core requirements of the questions, and classify user questions into factual query categories and fuzzy query categories.

[0062] Factual queries refer to users seeking specific, objective, and verifiable factual information, and the answers can be obtained directly from the data in the knowledge graph. Fuzzy queries refer to users' questions that are usually broad, subjective, or have no clear answers, and may contain uncertainty or context-dependent information. Depending on the type of question, the system will select the corresponding query method for processing.

[0063] 3.3 Extract the question entities (such as people, places, time, events, etc.) in the user's question. According to the question entities and the large model prompt words, identify the question entity relationship in the private database; in this process, the results other than the extracted entities are also recorded and saved as context content.

[0064] 3.4 Based on the question entity relationship, the knowledge graph is retrieved to determine whether there is a related question entity. If so, the query model is used to search and output the answer, and a scoring mechanism is set in the query model to indicate the degree of match between the question entity and the knowledge graph. If not, a large language model is called to perform reasoning question and answering.

[0065] like Figure 5 As shown in the figure, the knowledge graph query process based on entity relationships includes:

[0066] 3.41 Perform entity initialization processing on user queries.

[0067] The system receives user queries, identifies relevant entities and creates an initial set of blocks, and scores each initial block based on its importance and contextual relevance to obtain an initial score; among which: the importance of the entity is evaluated by the connectivity of the entity in the knowledge graph; the contextual relevance is calculated by analyzing the match between the query and the block content to calculate the similarity score.

[0068] 3.42 Matching relevant documents of the initialization block set and aggregating information about the matching document content

[0069] The initial block set is matched through the knowledge graph and the private database, and documents or records related to these blocks are retrieved. The information of all relevant initial blocks is aggregated using the collect function to obtain an aggregated block set. According to the initial score, the average score of the aggregated block set is calculated, and the data is passed using the WITH statement to ensure that all relevant information is comprehensively considered.

[0070] 3.43 Expanding the associated entities of an aggregate block collection

[0071] The aggregation block is expanded using the UNWIND operation to find related entities, ensure that all possible related information is extracted, perform entity expansion, and obtain an expanded entity set.

[0072] 3.44 Counting and Sorting Extended Entity Sets

[0073] The extended entity set is counted and sorted according to the importance and relevance of the entities. The number of entity occurrences is calculated using a statistical function, and the number of returned entities is limited based on the entity calculation and sorting results.

[0074] 3.45 Select the best expansion path based on scoring and sorting

[0075] According to the average score and the ranking result, the best expansion path of the entity is selected, and the number of expansion steps is determined according to the similarity of the entity by using a CASE statement.

[0076] The extended path and the extended entity set are then deduplicated, and the toSet and flatten functions in the APOC library are used to remove duplicate items to ensure the uniqueness of the result and obtain the result entity set; and the repeated calculation of multiple paths or entities is eliminated, thereby improving efficiency and accuracy. The deduplicated path and entity information will be retained for subsequent answer generation.

[0077] 3.46 Return the result entity set and construct an entity association graph. This information serves as a key element in generating the final answer, ensuring that the answer content is closely related to the query.

[0078] The entity association graph is visualized and converted into natural language by using the join function of APOC to output the final answer.

[0079] 4. If Figure 3-4 As shown, a historical question and answer library for storing historical question and answer records is constructed in the query model, and based on the historical question and answer records, a historical question and answer summary for optimizing the answer is generated. The historical question and answer summary can be used as context and integrated into the query results. The query results will serve as the main body of the answer, and the summary of the historical question and answer provides background information to help form a preliminary answer. Based on the preliminary answer, the system will use the big model to optimize the answer to generate the answer that best fits the question and makes it easier to understand.

[0080] The above process uses Langchain's HumanMessage class to summarize historical questions and answers into concise messages and use them as conversation content in subsequent answers. In addition, historical question and answer summaries can also help the system ensure that the answer content remains consistent and coherent with previous conversations when processing new questions, thereby improving the continuity of the user experience.

[0081] Therefore, the present invention improves the accuracy and credibility of answers by providing a knowledge graph-enhanced RAG intelligent question-answering method based on complex processes. At the same time, the visual representation of the knowledge graph also enables users to more intuitively understand the process of how the system arrives at a specific answer, thereby enhancing the interactivity and trust between users and the system.

[0082] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0083] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A knowledge graph-enhanced RAG intelligent question answering method based on complex processes, characterized in that: The following steps are involved: Uploading heterogeneous data files, performing data conversion and text block processing on the heterogeneous data files to obtain multiple text blocks, and performing embedding vector generation and vector index creation for each of the text blocks to obtain a private database; Extract entity relationships from the private database, construct a knowledge graph, and combine the knowledge graph with the big model to obtain a query model; Using the query model, decompose the user's question, identify the question type, question intent and entity relationship, and then determine whether there is a related entity in the knowledge graph. If so, use the query model to output the answer. If not, call the large language model for reasoning question answering; A historical question and answer library for storing historical question and answer records is constructed in the query model, and a historical question and answer summary for optimizing answers is generated based on the historical question and answer records.

2. According to claim 1, a complex process-based knowledge graph enhanced RAG intelligent question answering method is characterized in that: Upload heterogeneous data files, perform data conversion and text block processing on the heterogeneous data files to obtain multiple text blocks, generate embedded vectors and create vector indexes for each text block to obtain a private database, including: Load the uploaded heterogeneous data file, use image recognition and natural language processing technology to perform format conversion and data consistency processing, and then divide the heterogeneous data file into blocks to obtain multiple text blocks; when the heterogeneous data file is a PDF file, use a PDF parser to extract the content, and when the heterogeneous data file is other file types, convert the heterogeneous data file of other file types into Markdown format, and then perform data processing; Generate a unique identifier for each text block using a hash value, record the position, length and offset of each text block, and encapsulate the recorded information as metadata, and establish an association between the text block and the heterogeneous data file using the Cypher language based on the metadata; For each of the text blocks, an embedding vector is generated and a vector index is created, and the embedding vector and the vector index are updated using the Cypher language to obtain a private data set library.

3. According to claim 2, a complex process-based knowledge graph enhanced RAG intelligent question answering method is characterized in that: Entity relationships are extracted from the private database to construct a knowledge graph, and the knowledge graph is combined with the big model to obtain a query model, including: Preset a big model prompt word for guiding the big model, combine the big model configuration information, and call different big models to perform entity extraction and entity relationship extraction on the private database; Perform data cleaning and classification on the extraction results, and then classify and save the extraction results. According to the extraction results, the preset Cypher language is used to establish connections between entity nodes and text blocks to obtain a knowledge graph, which is visualized and combined with a large model to obtain a query model.

4. According to claim 3, a complex process-based knowledge graph enhanced RAG intelligent question answering method is characterized in that: The query model is used to decompose user questions, identify question types, question intent, and entity relationships, and then determine whether there are relevant entities in the knowledge graph. If so, the query model is used to output answers. If not, the large language model is called for reasoning question answering, including: Record the text content of the user's question, and attach a timestamp, relevant context information, and an ID to indicate whether the session has ended. Use natural language processing technology to identify the type and intention of user questions, and classify user questions into factual query and fuzzy query categories; Extracting question entities from the user's question, and identifying question entity relationships in the private database according to the question entities and the large model prompt words; Based on the question entity relationship, the knowledge graph is retrieved to determine whether there is a related question entity. If so, the query model is used to search and output the answer, and a scoring mechanism is set in the query model to indicate the degree of match between the question entity and the knowledge graph. If not, a large language model is called to perform reasoning question and answering.

5. According to claim 4, a complex process-based knowledge graph enhanced RAG intelligent question answering method is characterized in that: Based on the question entity relationship, the knowledge graph is retrieved to determine whether there is a related question entity. If so, the query model is used to search and output the answer, and a scoring mechanism is set in the query model to indicate the degree of match between the question entity and the knowledge graph, including: Perform entity initialization processing on the user query, create an initial block set, and score each initial block according to its importance and context relevance to obtain an initial score; Matching the initial block set through the knowledge graph and the private database, aggregating the information of all relevant initial blocks using the collect function to obtain an aggregated block set, and calculating the average score of the aggregated block set according to the initial score; Expand the aggregate block using the UNWIND operation, find related entities, and perform entity expansion to obtain an expanded entity set; Counting and sorting the extended entity set, calculating the number of entity occurrences, and then limiting the number of returned entities based on the entity calculation and sorting results; According to the average score and the ranking result, the optimal expansion path and the number of expansion steps of the entity are selected, and then the expansion path and the expansion entity set are deduplicated to obtain a result entity set; The result entity set is returned, and an entity association graph is constructed, the entity association graph is visualized, and the entity association graph is converted into natural language to output the final answer.

Images