Retrieval enhancement generation method and device based on knowledge graph

Abstract

The invention discloses a retrieval enhancement generation method and device based on a knowledge graph, and relates to the technical field of natural language processing and knowledge management. Comprising the steps of 1, performing data preprocessing, and generating a text block; 2, according to the text blocks, entities and relations are extracted; 3, converting the entity and relation description into vector representation, and storing the vector representation in a vector database; 4, processing a natural language query problem input by the user, and constructing a query context; 5, according to the query context, utilizing the pre-training language model to generate an answer, and utilizing a cache mechanism to optimize response time; according to the invention, efficient and accurate response to natural language query is realized, and the performance and user experience of an intelligent question-answering system and an information management platform are improved.

Description

Technical Field

[0001] The present invention discloses a retrieval enhanced generation method and device based on a knowledge graph, which relates to the technical fields of natural language processing and knowledge management. Background Art

[0002] With the rapid development of information technology, especially the continuous progress of natural language processing (NLP) and knowledge management technologies, intelligent question-answering systems and information retrieval platforms are increasingly widely used in all walks of life. Existing technologies mainly rely on keyword matching methods to respond to users' natural language queries. These methods can, to a certain extent, meet users' information needs by indexing and retrieving a large amount of text data. However, with the rapid increase in data volume and the diversification of user needs, existing methods gradually show many deficiencies in terms of accuracy and efficiency. For example, existing keyword matching methods have obvious limitations in dealing with complex semantic relationships and context understanding. It is difficult to capture the implicit intentions and complex semantic structures in users' queries, resulting in low relevance and accuracy of retrieval results. Secondly, existing retrieval generation methods perform poorly in real-time response and large-scale data processing. A large amount of computing resources are required for vector embedding and similarity calculation, resulting in a long response time and difficulty in meeting the needs of real-time applications. Moreover, existing technologies also have deficiencies in knowledge integration and multi-source information fusion. Summary of the Invention

[0003] In view of the problems of the existing technology, the present invention provides a retrieval enhanced generation method and device based on a knowledge graph to achieve efficient and accurate response to natural language queries and improve the performance and user experience of intelligent question-answering systems and information management platforms.

[0004] The specific solution proposed by the present invention is as follows: The present invention provides a retrieval enhanced generation method based on a knowledge graph, including: Step 1: Perform data preprocessing and generate text blocks: Clean and standardize the text data, encode the processed text data, and split the encoded text into multiple text blocks; Step 2: Extract entities and relationships according to the text blocks: According to the text blocks, identify entities: Use the named entity recognition method NER to identify the entities in the text blocks; Extract the relationships between entities: After identifying the entities, use the semantic relationships between the entities as relationship descriptions; Generate a knowledge graph according to the entities and relationship descriptions; Step 3: Convert the entities and relationship descriptions into vector representations and store them in a vector database; Step 4: Process the natural language query problem input by the user and construct a query context: Extract keywords from natural language query questions using a pre-trained language model, retrieve in a knowledge graph and / or vector database according to the keywords, obtain relevant entities, relationship descriptions, and text fragments, and integrate the relevant entities, relationship descriptions, and text fragments into a query context to provide background support for generating answers; Step 5: Generate an answer using a pre-trained language model based on the query context and optimize the response time using a caching mechanism.

[0005] Furthermore, in step 1 of the retrieval-enhanced generation method based on a knowledge graph, it includes: Encode the text using the following formula: ; where, represents the encoded token sequence, is the original text data, is the encoding model, According to the set maximum number of tokens and the overlapping number of tokens , the encoded text data is segmented into multiple text blocks: ; where, is the total number of text blocks, is the th text block, and there is overlap between each text block to avoid information loss or context breakage caused by segmentation.

[0006] Furthermore, in step 3 of the retrieval-enhanced generation method based on a knowledge graph, it includes: Convert the entity and relationship descriptions into vector representations: Use the pre-trained embedding model , and convert the entity description and the relationship description into vector representations: ; Store the generated vectors and separately in the vector database and the relationship vector database : ; Store the vectors in the vector database using a quantization and de-quantization storage structure: Use the formula: ; ; where , convert the vector into a low-precision integer representation through a quantization function to reduce storage space and computational overhead; restore the precision of the vector when a high-precision vector is needed through a dequantization function to ensure the accuracy of similarity calculation.

[0007] Furthermore, in step 4 of the above-mentioned retrieval enhanced generation method based on a knowledge graph, it includes: Using the formula: ; Retrieve according to the keyword in the knowledge graph and the vector database to obtain relevant entities, relationships, and text fragments. represents the query mode, which determines the scope and depth of the retrieval. The query mode includes local query, global query, and hybrid query. The local query retrieves only in the knowledge graph, the global query conducts a wide retrieval in the vector database, and the hybrid query retrieves in both the knowledge graph and the vector database.

[0008] Furthermore, the cache mechanism involved in step 5 of the above-mentioned retrieval enhanced generation method based on a knowledge graph includes: Calculation cache: Using the formula: ; Calculate the hash value of the query parameter , is the input for hashing the query and the query mode, ensuring that each query has a unique hash value. Match cache: Check whether there is a corresponding answer in the cache through the hash value . If there is a matching hash value in the cache, directly return the answer in the cache to avoid repeated calculation. Update cache: If there is no matching answer in the cache, generate a new answer and store it in the cache. Call cache: In subsequent similar queries, directly call the answer in the cache by matching the hash value to shorten the response time.

[0009] The present invention also provides a retrieval enhanced generation device based on a knowledge graph, including a preprocessing module, a knowledge graph management module, a vector conversion module, a context construction module, an answer generation module, and a cache optimization module. The preprocessing module performs data preprocessing and generates text blocks: Clean and standardize the text data, encode the processed text data, and split the encoded text into multiple text blocks; The knowledge graph management module extracts entities and relationships based on the text blocks: The context construction module identifies entities based on the text blocks: uses the named entity recognition method NER to identify the entities in the text blocks; extracts the relationships between entities: after identifying the entities, uses the semantic relationships between the entities as the relationship descriptions; generates a knowledge graph based on the entities and relationship descriptions; The vector conversion module converts the entity and relationship descriptions into vector representations and stores them in a vector database; The context construction module processes the natural language query problem input by the user and constructs a query context: Uses a pre-trained language model to extract keywords from the natural language query problem, retrieves relevant entities, relationship descriptions, and text fragments in the knowledge graph and / or vector database, and integrates the relevant entities, relationship descriptions, and text fragments into a query context to provide background support for generating an answer; The answer generation module generates an answer based on the query context using a pre-trained language model, and the cache optimization module optimizes the response time using a cache mechanism.

[0010] Furthermore, the knowledge graph management module of the retrieval enhanced generation device based on a knowledge graph encodes the text using the following formula: ; where, represents the encoded token sequence, is the original text data, is the encoding model, According to the set maximum number of tokens and the overlapping number of tokens , the encoded text data is split into multiple text blocks: ; where, is the total number of text blocks, is the th text block, and there is an overlap between each text block to avoid information loss or context breakage caused by splitting.

[0011] Furthermore, the vector conversion module of the retrieval enhanced generation device based on a knowledge graph converts the entity and relationship descriptions into vector representations: uses a pre-trained embedding model , and converts the entity description and the relationship description into vector representations: ; Store the generated vectors and separately in the vector database and the relational vector database as follows: ; Store the vectors in the vector database using a quantization and de - quantization storage structure: Use the formula: ; ; where , convert the vector to a low - precision integer representation through the quantization function to reduce storage space and computational overhead; restore the precision of the vector when high - precision vectors are needed through the de - quantization function to ensure the accuracy of similarity calculations.

[0012] Furthermore, the context construction module of the retrieval - enhanced generation device based on the knowledge graph uses the formula: ; Retrieve relevant entities, relationships, and text fragments in the knowledge graph and the vector database according to the keyword , where represents the query mode, which determines the scope and depth of the retrieval. The query mode includes local query, global query, and hybrid query. The local query retrieves only in the knowledge graph, the global query conducts extensive retrieval in the vector database, and the hybrid query retrieves in both the knowledge graph and the vector database.

[0013] Furthermore, the cache mechanism involved in the cache optimization module of the retrieval - enhanced generation device based on the knowledge graph includes: Computation cache: Use the formula: ; Calculate the hash value of the query parameters , is the input for hashing the query and the query mode, ensuring that each query has a unique hash value, Match cache: Check whether there is a corresponding answer in the cache through the hash value . If a matching hash value exists in the cache, directly return the answer in the cache to avoid repeated calculations. Update cache: If there is no matching answer in the cache, generate a new answer , and store it in the cache Call cache: In subsequent similar queries, directly call the answer in the cache by matching the hash value , shortening the response time

[0014] The advantages of the present invention are as follows: Improve query relevance: By combining the structured information of the knowledge graph with the efficient retrieval ability of the vector database, the present invention can more accurately understand the deep semantics of user queries and provide more relevant retrieval results and generated content

[0015] Enhance knowledge integration ability: Utilizing the multi-level structure of the knowledge graph, it is possible to integrate information from different data sources and formats, constructing a comprehensive and dynamically updated knowledge base. Through structured entity and relationship management, efficient organization and retrieval of information are achieved

[0016] Support large-scale data processing: The efficient storage and retrieval ability of the vector database enables the present invention to process massive datasets and maintain high performance. By optimizing the vector storage structure and indexing mechanism, when processing millions of vector data, the query efficiency can still be maintained at the millisecond level, meeting the requirements of large-scale enterprise applications

[0017] Improve system scalability: By introducing a multi-model support mechanism, it is possible to select the most suitable language model and embedding algorithm according to actual needs, further enhancing the adaptability and flexibility of the system Brief Description of the Drawings

[0018] Figure 1 is a schematic diagram of the method flow of the present invention Detailed Embodiments

[0019] The present invention will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the embodiments cited are not intended to limit the present invention

[0020] Embodiment 1: The present invention provides a retrieval enhanced generation method based on a knowledge graph, including: Step 1: Perform data preprocessing and generate text blocks: Clean and standardize the text data, encode the processed text data, and divide the encoded text into multiple text blocks

[0021] Among them, in Step 1, it may specifically include: Encode the text using the following formula: ; Wherein, Represents the encoded token sequence, is the original text data, is the encoding model, According to the set maximum number of tokens and the overlapping token number , the encoded text data is segmented into multiple text blocks: ; Among them, is the total number of text blocks, is the th text block. There is overlap between each text block to avoid information loss or context breakage caused by segmentation. In this way, the system can capture key information in the text more comprehensively, providing a solid foundation for subsequent entity and relationship extraction.

[0022] Step 2: Extract entities and relationships based on the text blocks: Based on the text blocks, identify entities: Use the named entity recognition method NER to identify entities in the text blocks; such as person names, organization names, place names, technical names, etc. For example, in the sentence "The development of agricultural technology has significantly increased food production and promoted the prosperity of rural economy", "agricultural technology", "food production", and "rural economy" will be identified as entities.

[0023] Extract the relationships between entities: After identifying the entities, use the semantic relationships between the extracted entities as relationship descriptions, such as "influence", "promote", "depend on", etc. For example, in the above sentence, there is a "development and improvement" relationship between "agricultural technology" and "food production", and a "promote prosperity" relationship between "agricultural technology" and "rural economy".

[0024] Generate a knowledge graph based on the entities and relationship descriptions. The knowledge graph can be updated dynamically to reflect the latest data and knowledge structure. A multi-level verification mechanism can also be introduced to cross-verify and check the consistency of the extraction results to ensure the accuracy and consistency of the knowledge graph.

[0025] Step 3: Convert the entity and relationship descriptions into vector representations and store them in a vector database.

[0026] Among them, in step 3, it can specifically include: Convert the entity and relationship descriptions into vector representations: Use a pre-trained embedding model , convert the entity description and the relationship description into vector representations: ; Convert the generated vectors and Stored separately in the vector database and the relational vector database as follows: ; The vector is stored by the vector database using a quantization and dequantization storage structure: Using the formula: ; ; , through the quantization function convert the high-precision vector into a low-precision integer representation to reduce storage space and computational overhead: The quantization function maps the floating-point vector to an integer between 0 and 255, and the dequantization function maps the integer back to an approximate floating-point vector in the same proportion, where means shifting the vector as a whole downwards so that the minimum value aligns with 0, and using compress the vector data into the range of 0-255, and through get an integer between 0 and 255, through the dequantization function restore the precision of the vector when a high-precision vector is needed: through pull back to the original proportion and magnify the integer back to its original size, means adding back the baseline, that is, putting the integer back to its original height to obtain an approximate floating-point vector, ensuring the accuracy of similarity calculation. This not only improves the storage efficiency but also speeds up the calculation speed of vector similarity, enabling the system to maintain high performance when processing large-scale vector data.

[0027] Step 4: Process the natural language query problem input by the user and construct a query context: Use a pre-trained language model to extract keywords from the natural language query problem, retrieve relevant entities, relationship descriptions, and text fragments in the knowledge graph and / or vector database, and integrate the relevant entities, relationship descriptions, and text fragments into a query context to provide background support for generating answers.

[0028] Among them, in Step 4, it may specifically include: Use the language model to extract core keywords from the query : : ; The keyword generation process not only extracts the explicit keywords in the query, but also captures implicit intentions and related terms through semantic analysis to ensure a comprehensive understanding of the query.

[0029] Using the formula: ; According to the keywords in the knowledge graph and the vector database perform a retrieval to obtain relevant entities, relationships, and text fragments. represents the query mode, which determines the scope and depth of the retrieval. The query modes include local query, global query, and hybrid query. The local query retrieves only in the knowledge graph, the global query performs a wide retrieval in the vector database, and the hybrid query retrieves in both the knowledge graph and the vector database.

[0030] During context integration: Integrate the retrieved information into a unified context , and the context integration process includes information screening, weight assignment, and content combination to ensure that the generated answer is both comprehensive and accurate.

[0031] The specific implementation is as follows: Information screening: Based on the relevance and weight of the retrieval results, screen out the most representative and influential relevant entities, relationship descriptions, and text fragments; Weight assignment: Assign weights to different entities, relationship descriptions, and text fragments to ensure that important information has a higher weight and is given priority consideration when generating the answer; Content combination: Combine the screened entities, relationship descriptions, and text fragments to form a coherent and logical context .

[0032] Step 5: Generate an answer according to the query context using a pre-trained language model and optimize the response time using a caching mechanism.

[0033] The caching mechanisms involved include: Computation cache: Using the formula: ; Calculate the hash value of the query parameters , is the input for hashing the query and the query mode to ensure that each query has a unique hash value. Match cache: Check whether there is a corresponding answer in the cache through the hash value . If there is a matching hash value in the cache, directly return the answer in the cache to avoid repeated calculation. Update the cache: If there is no matching answer in the cache, generate a new answer and store it in the cache. Invoke the cache: In subsequent similar queries, directly invoke the answer in the cache by matching the hash value to shorten the response time.

[0034] Embodiment 2: The present invention also provides a retrieval enhanced generation device based on a knowledge graph, including a preprocessing module, a knowledge graph management module, a vector conversion module, a context construction module, an answer generation module, and a cache optimization module. The preprocessing module performs data preprocessing and generates text blocks: Clean and standardize the text data, encode the processed text data, and split the encoded text into multiple text blocks; The knowledge graph management module extracts entities and relationships according to the text blocks: The context construction module identifies entities according to the text blocks: Use the named entity recognition method NER to identify the entities in the text blocks; Extract the relationships between entities: After identifying the entities, use the semantic relationships between the entities as relationship descriptions; Generate a knowledge graph according to the entities and relationship descriptions; The vector conversion module converts the entities and relationship descriptions into vector representations and stores them in a vector database; The context construction module processes the natural language query problem input by the user and constructs a query context: Use a pre-trained language model to extract keywords from the natural language query problem, retrieve relevant entities, relationship descriptions, and text fragments in the knowledge graph and / or vector database according to the keywords, and integrate the relevant entities, relationship descriptions, and text fragments into a query context to provide background support for generating an answer; The answer generation module generates an answer according to the query context using a pre-trained language model, and the cache optimization module optimizes the response time using a cache mechanism.

[0035] Regarding the information interaction, execution process, etc. among the above-mentioned modules in the device, since they are based on the same concept as the method embodiment of the present invention, the specific content can be referred to the description in the method embodiment of the present invention and will not be elaborated here.

[0036] Similarly, the device of the present invention improves query relevance: By combining the structured information of the knowledge graph with the efficient retrieval ability of the vector database, the present invention can more accurately understand the deep semantics of the user's query and provide more relevant retrieval results and generated content.

[0037] Enhance knowledge integration capabilities: By leveraging the multi-level structure of the knowledge graph, it is possible to integrate information from different data sources and formats, constructing a comprehensive and dynamically updated knowledge base. Through structured entity and relationship management, efficient organization and retrieval of information are achieved.

[0038] Support large-scale data processing: The efficient storage and retrieval capabilities of the vector database enable the present invention to handle massive datasets and maintain high-performance performance. By optimizing the vector storage structure and indexing mechanism, the query efficiency can still be maintained at the millisecond level when processing millions of vector data, meeting the requirements of large-scale enterprise-level applications.

[0039] Improve system scalability: By introducing a multi-model support mechanism, the most suitable language model and embedding algorithm can be selected according to actual needs, further enhancing the adaptability and flexibility of the system.

[0040] It should be noted that not all steps and modules in the above processes and device structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted according to needs. The system structure described in the above embodiments can be a physical structure or a logical structure. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities separately, or some components in multiple independent devices may be jointly implemented.

[0041] The above-described embodiments are merely preferred embodiments given to fully illustrate the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present invention are all within the protection scope of the present invention. The protection scope of the present invention is subject to the claims.

Claims

1. A retrieval enhancement generation method based on knowledge graph, characterized by include: Step 1: Preprocess the data and generate text blocks: Clean and standardize the text data, encode the processed text data, and divide the encoded text into multiple text blocks; Step 2: Extract entities and relationships based on the text block: According to the text block, identify entities: use the named entity recognition method NER to identify entities in the text block; extract the relationship between entities: after identifying the entities, use the semantic relationship between the extracted entities as the relationship description; generate a knowledge graph based on the entity and relationship description; Step 3: Convert entity and relationship descriptions into vector representations and store them in a vector database; Step 4: Process the natural language query input by the user and build the query context: Use the pre-trained language model to extract keywords from natural language query questions, search the knowledge graph and / or vector database based on the keywords, retrieve relevant entities, relationship descriptions and text fragments, integrate the relevant entities, relationship descriptions and text fragments into the query context, and provide background support for generating answers; Step 5: Generate answers based on the query context using a pre-trained language model and use a caching mechanism to optimize response time.

2. According to the method for generating enhanced retrieval based on knowledge graph in claim 1, it is characterized by: Step 1 includes: The text is encoded using the following formula: ； in, represents the encoded token sequence, is the original text data, For the encoding model, According to the maximum number of tokens set and the number of overlapping tokens , split the encoded text data into multiple text blocks: ； in, is the total number of text blocks, For the There are overlaps between each text block to avoid information loss or context breakage due to segmentation.

3. According to the method for generating retrieval enhancement based on knowledge graph in claim 1, it is characterized by: Step 3 includes: Converting entity and relation descriptions into vector representations: Leveraging pre-trained embedding models , describe the entity and relationship description Convert to vector representation: ； The generated vector and Stored in vector database and relational vector database middle: ； The vector database uses quantized and dequantized storage structures to store vectors: Using the formula: ； ； in , through the quantization function Vector Convert to low-precision integer representation to reduce storage space and computational overhead; by dequantizing the function Recovering vectors when high precision is required to ensure the accuracy of similarity calculation.

4. The method for generating retrieval enhancement based on knowledge graph according to claim 1, characterized in that Step 4 includes: Using the formula: ； By keyword In the knowledge graph and vector database Search to obtain relevant entities, relations and text fragments. Represents the query mode, which determines the scope and depth of the retrieval. The query modes include local query, global query and hybrid query. Local query only searches in the knowledge graph, global query conducts extensive search in the vector database, and hybrid query searches in both the knowledge graph and the vector database.

5. The method for generating retrieval enhancement based on knowledge graph according to claim 1, characterized in that The caching mechanisms involved in step 5 include: Calculate cache: Use the formula: ； Calculate the hash value of the query parameters , The query and query pattern are hashed inputs, ensuring that each query Each has a unique hash value. Matching cache: by hash value Check if the corresponding answer exists in the cache If there is a matching hash value in the cache, the answer in the cache is returned directly to avoid repeated calculations. Update cache: If there is no matching answer in the cache, generate a new answer , and store it in the cache, Call cache: In subsequent similar queries, by matching the hash value , directly call the answer in the cache to shorten the response time.

6. A retrieval enhancement generation device based on knowledge graph, characterized in that It includes preprocessing module, knowledge graph management module, vector conversion module, context building module, answer generation module and cache optimization module. The preprocessing module performs data preprocessing and generates text blocks: Clean and standardize the text data, encode the processed text data, and divide the encoded text into multiple text blocks; The knowledge graph management module extracts entities and relationships based on text blocks: The context building module identifies entities based on the text block: uses the named entity recognition method NER to identify entities in the text block; extracts the relationship between entities: after identifying the entities, uses the semantic relationship between the extracted entities as the relationship description; generates a knowledge graph based on the entity and relationship description; The vector conversion module converts entity and relationship descriptions into vector representations and stores them in the vector database; The context building module processes the natural language query questions entered by the user and builds the query context: Use the pre-trained language model to extract keywords from natural language query questions, search the knowledge graph and / or vector database based on the keywords, retrieve relevant entities, relationship descriptions and text fragments, integrate the relevant entities, relationship descriptions and text fragments into the query context, and provide background support for generating answers; The answer generation module generates answers based on the query context using a pre-trained language model, and the cache optimization module optimizes the response time using a cache mechanism.

7. According to the knowledge graph-based retrieval enhancement generation device of claim 6, it is characterized in that the knowledge graph management module encodes the text using the following formula: ； in, represents the encoded token sequence, is the original text data, For the encoding model, According to the maximum number of tokens set and the number of overlapping tokens , split the encoded text data into multiple text blocks: ； in, is the total number of text blocks, For the There are overlaps between each text block to avoid information loss or context breakage due to segmentation.

8. The retrieval enhancement generation device based on knowledge graph according to claim 6 is characterized by: The vector conversion module converts entity and relation descriptions into vector representations: using pre-trained embedding models , describe the entity and relationship description Convert to vector representation: ； The generated vector and Stored in vector database and relational vector database middle: ； The vector database uses quantized and dequantized storage structures to store vectors: Using the formula: ； ； in , through the quantization function Vector Convert to low-precision integer representation to reduce storage space and computational overhead; by dequantizing the function Recovering vectors when high precision is required to ensure the accuracy of similarity calculation.

9. The retrieval enhancement generation device based on knowledge graph according to claim 6, characterized in that The context building block utilizes the formula: ； By keyword In the knowledge graph and vector database Search to obtain relevant entities, relations and text fragments. Represents the query mode, which determines the scope and depth of the retrieval. The query modes include local query, global query and hybrid query. Local query only searches in the knowledge graph, global query conducts extensive search in the vector database, and hybrid query searches in both the knowledge graph and the vector database.

10. The retrieval enhancement generation device based on knowledge graph according to claim 6, characterized in that The cache mechanisms involved in the cache optimization module include: Calculate cache: Use the formula: ； Calculate the hash value of the query parameters , The query and query pattern are hashed inputs, ensuring that each query Each has a unique hash value. Matching cache: by hash value Check if the corresponding answer exists in the cache If there is a matching hash value in the cache, the answer in the cache is returned directly to avoid repeated calculations. Update cache: If there is no matching answer in the cache, generate a new answer , and store it in the cache, Call cache: In subsequent similar queries, by matching the hash value , directly call the answer in the cache to shorten the response time.

Images