An enhanced cross-document intelligent retrieval method and system based on knowledge memory

Abstract

The application provides an enhanced cross-document intelligent retrieval method and system based on knowledge memory, relates to the field of information retrieval, and solves the limitation problem of traditional retrieval enhancement generation methods in cross-document information integration, calculation efficiency and processing complex question and answer scenarios; the method comprises the following steps: extracting semantic triples from multiple documents by using a large language model to construct a schema-free knowledge graph; applying a relationship encoder to enhance the semantic connectivity between different semantic triples, so that the information in multiple documents is integrated and used for subsequent comprehensive retrieval; calculating the Hub value and the Authority value of each node in the knowledge graph by using the HITS algorithm; receiving a query request of a user, retrieving the most relevant node based on the Hub value and the Authority value of the node, and providing information matched with the query request; and the application effectively improves the information retrieval and generation capability of the natural language processing system.

Description

Technical Field

[0001] This invention relates to the field of information retrieval, specifically to an enhanced cross-document intelligent retrieval method and system based on knowledge memory, applied to the RAG system of natural language processing. Background Technology

[0002] In recent years, the field of Natural Language Processing (NLP) has made significant progress, particularly in the application of Large Language Models (LLMs). These models, by learning patterns from massive amounts of text data, can generate coherent and informative text, providing powerful support for various application scenarios such as machine translation, content creation, and dialogue systems. With technological advancements, expectations for the quality of generated content have increased, demanding not only high relevance but also accurate reflection of the latest developments in the discussed topic. To meet these needs, Retrieval-Augmented Generation (RAG) methods have been proposed, which improve the quality and accuracy of output by introducing external knowledge sources during the generation process.

[0003] Despite the immense potential of the RAG method, its practical application still faces a number of challenges. First, in the traditional RAG framework, each document is treated as an independent unit of information. While this simplifies the information extraction process, it overlooks a crucial fact: knowledge in the real world is interconnected, not isolated. Therefore, when faced with tasks requiring the integration of information from different sources, single-document-based methods struggle to fully uncover potential knowledge connections, thus limiting the richness and accuracy of the final generated content.

[0004] Furthermore, another significant limitation of existing RAG systems lies in their high computational cost. Typically, these systems employ an iterative retrieval strategy, repeatedly querying relevant information based on initial results until a satisfactory outcome is achieved. This process not only consumes substantial computational resources but also increases response time, impacting user experience. Especially on mobile devices or under poor network conditions, prolonged waiting times can be a major factor in users abandoning the service.

[0005] Finally, traditional RAG solutions fall short in solving problems involving complex logical reasoning. These problems often require extracting and synthesizing information from multiple unrelated documents to arrive at the correct answer. However, due to the lack of an effective cross-document information fusion mechanism, existing technologies struggle to effectively support such tasks, resulting in poor performance when faced with scenarios requiring deep understanding and extensive search capabilities.

[0006] In summary, while current RAG technology has achieved certain successes, it still has significant shortcomings in achieving more efficient, accurate, and adaptable information retrieval and generation. To address this issue, there is an urgent need to develop new solutions to overcome the limitations of single-document processing methods, reduce computational burden, and improve support for complex question-and-answer scenarios. Summary of the Invention

[0007] The purpose of this invention is to address the limitations of traditional retrieval enhancement generation methods in cross-document information integration, computational efficiency, and handling complex question-and-answer scenarios. Therefore, this invention proposes an enhanced cross-document intelligent retrieval method and system based on knowledge memory. Through the application of knowledge graph construction and keyword matching algorithms, this invention effectively improves the information retrieval and generation capabilities of natural language processing systems.

[0008] The present invention employs the following technical solutions to achieve its objective:

[0009] An enhanced cross-document intelligent retrieval method based on knowledge memory includes the following steps:

[0010] S1. Use a large language model to automatically extract corresponding semantic triples from multiple documents. The semantic triples include at least information about individuals, relations and attributes. Construct a patternless knowledge graph based on the extracted semantic triples.

[0011] S2. Apply a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information from multiple documents can be integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval.

[0012] S3. The HITS algorithm is used to calculate the Hub value and Authority value of each node in the knowledge graph; the nodes in the knowledge graph represent entity elements in semantic triples;

[0013] S4. Receive the user's query request, and based on the node's Hub value and Authority value, retrieve and determine the node most relevant to the query request, and then provide information that matches the query request.

[0014] Specifically, in step S1, the large language model is configured to recognize natural language expressions in multiple documents and convert them into structured semantic triples, wherein the information of individuals, relations and attributes is obtained directly from or by reasoning from the content of multiple documents.

[0015] Preferably, the relation encoder applied in step S2 includes a retrieval encoder, which retrieves and associates similar but not identical noun phrases in different semantic triples, and adds additional relation edges between these noun phrases in the knowledge graph.

[0016] Preferably, in step S3, when applying the HITS algorithm, the Hub value and Authority value of all nodes in the knowledge graph are initialized, and then iteratively updated until convergence. This initialization method makes the importance score of each node reflect its role in the knowledge graph.

[0017] Specifically, in step S4, the retrieved relevant nodes are filtered to select the node with the highest comprehensive score; the comprehensive score is the result of weighting the node's Hub value and Authority value.

[0018] Preferably, in step S4, when providing information that matches the query request, the original text segment in the original document is located by tracing back the original semantic triple corresponding to the most relevant node retrieved, and then fed back to the user.

[0019] Preferably, step S4 retrieves multiple relevant nodes corresponding to the query request, sorts the located original text segments according to the relevance score of each node, and provides the user with a sorted list of search results.

[0020] This invention also provides an enhanced cross-document intelligent retrieval system based on knowledge memory, the system comprising:

[0021] The document processing module is configured to automatically extract corresponding semantic triples from multiple documents using a large language model. The semantic triples include at least information about individuals, relations, and attributes, and a patternless knowledge graph is constructed based on the extracted semantic triples.

[0022] The knowledge graph enhancement module is configured to use a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information from multiple documents can be integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval.

[0023] The HITS algorithm calculation module is configured to use the HITS algorithm to calculate the Hub value and Authority value of each node in the knowledge graph; wherein, the nodes in the knowledge graph represent entity elements in semantic triples;

[0024] The query processing module is configured to receive user query requests, retrieve and determine the most relevant node for the query request based on the node's Hub value and Authority value, and provide information that matches the query request.

[0025] The present invention also provides a computer-readable storage medium storing a computer program / instruction thereon, which, when executed by a processor, implements the steps of the aforementioned knowledge-memory-based enhanced cross-document intelligent retrieval method.

[0026] The present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aforementioned knowledge-memory-based enhanced cross-document intelligent retrieval method.

[0027] In summary, due to the adoption of this technical solution, the beneficial effects of this invention are as follows:

[0028] This invention enables the effective integration of cross-document information. Through algorithmic design, it identifies and connects relevant content from different documents, constructing a richer and more comprehensive knowledge graph. This approach is particularly important for handling complex queries, ensuring that even when faced with questions requiring the integration of information from multiple sources, the system can provide accurate and detailed answers, thereby significantly improving the quality and relevance of the output results.

[0029] This invention also represents a significant advancement in computational efficiency. Unlike traditional methods that employ multiple iterative retrieval steps, this invention supports completing all necessary retrieval steps in a single, multi-hop inference process. This approach substantially reduces the computational resources required for each operation and shortens the overall response time. Consequently, users can obtain near-instantaneous feedback, which is crucial for enhancing user experience and promoting the development of real-time application scenarios.

[0030] This invention also performs well in complex question-answering scenarios involving in-depth logical analysis and extensive information retrieval needs. It possesses the ability to extract key points from fragmented information scattered across multiple unrelated documents and formulate coherent and accurate answers accordingly. This enables the system to provide high-quality answers even to challenging queries, fully meeting the growing demand for intelligent information services in modern society. Attached Figure Description

[0031] Figure 1 This is a simplified flowchart illustrating the overall process of the method of the present invention;

[0032] Figure 2 This is a schematic diagram illustrating the process of executing the method of the present invention in conjunction with an example flow. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0034] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0035] Example 1

[0036] An enhanced cross-document intelligent retrieval method based on knowledge memory. Figure 1 A brief overview of the method is shown below, which can be viewed concurrently. The steps of the method are summarized as follows:

[0037] S1. Use a large language model to automatically extract corresponding semantic triples from multiple documents. The semantic triples include at least information about individuals, relations and attributes. Construct a patternless knowledge graph based on the extracted semantic triples.

[0038] S2. Apply a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information from multiple documents can be integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval.

[0039] S3. The HITS algorithm is used to calculate the Hub value and Authority value of each node in the knowledge graph; the nodes in the knowledge graph represent entity elements in semantic triples; the purpose of this step is to achieve a more accurate response to user queries.

[0040] S4. Receive the user's query request, and based on the node's Hub value and Authority value, retrieve and determine the node most relevant to the query request, and then provide information that matches the query request.

[0041] In the above method, the schema-less knowledge graph mentioned in step S1 refers to a knowledge graph that does not require a predefined strict data schema or structure during construction. This means that the knowledge graph can be automatically generated and expanded based on document content, without being restricted by a pre-set schema. The advantage of this is that it can flexibly handle various types of documents, and as more documents are added, the knowledge graph can be continuously enriched and developed, providing more comprehensive information retrieval capabilities. While traditional knowledge graphs maintain data consistency and integrity through strict schema definition, this also limits their flexibility.

[0042] In the above method, the HITS algorithm mentioned in step S3 is a link analysis algorithm. This algorithm evaluates the importance or relevance of each node in the network by calculating two importance indicators, namely the Hub value and the Authority value. In addition to being applied to the link structure between web pages, this algorithm can also be effectively applied to other types of graph structures such as knowledge graphs.

[0043] Therefore, in this embodiment, the HITS algorithm is used to improve the relevance and retrieval efficiency of entity nodes (i.e., noun phrases corresponding to individuals, relations, and attributes in semantic triples) in the knowledge graph. Specifically: the Hub value represents the degree to which an entity acts as a "recommender," that is, its ability to point to other authoritative entities; the Authority value represents the degree to which an entity acts as an "expert" or "authority," that is, the degree to which it is pointed to by other important recommenders.

[0044] In this embodiment, by calculating these values ​​during steps S3 and S4, it is possible to identify which nodes (entities) in the knowledge graph are most important under a specific query request topic, thereby helping the system respond to user query requests more accurately. In this way, when a user enters a query, the system can quickly locate and retrieve the most relevant information based on the node's Hub and Authority values.

[0045] The following is a detailed and preferred description of the method process in this embodiment.

[0046] In step S1, the large language model is configured to recognize natural language expressions in multiple documents and convert them into structured semantic triples, where the information of individuals, relations and attributes is obtained directly from the content of multiple documents or through inference.

[0047] The relation encoder applied in step S2 includes a retrieval encoder, which retrieves and associates similar but not identical noun phrases in different semantic triples, and adds additional relation edges between these noun phrases in the knowledge graph to increase the connectivity and richness of the knowledge graph.

[0048] In step S3, when applying the HITS algorithm, the Hub value and Authority value of all nodes in the knowledge graph are initialized, and then iteratively updated until convergence. This initialization method makes the importance score of each node reflect its role in the knowledge graph.

[0049] In step S4, the retrieved relevant nodes are filtered, and the node with the highest comprehensive score is selected. The comprehensive score is the result of weighting the node's Hub value and Authority value. Furthermore, when providing information matching the query request, the original semantic triples corresponding to the most relevant retrieved nodes are traced back to locate the corresponding original text segments in the original document and fed back to the user, allowing the user to obtain more detailed background information.

[0050] Step S4 retrieves multiple relevant nodes corresponding to the query request, sorts the original text segments located according to the relevance scores of each node, and provides the user with a sorted list of search results, thereby improving the effectiveness of information presentation and user experience.

[0051] This embodiment will then demonstrate the execution process of the above method with a more specific example, which can be viewed simultaneously. Figure 2 This is an illustration. Example explanation: A user's query is "Who is the professor at XX University who researches Alzheimer's disease?" Document 1 content is "Li Ming is a professor at XX University. He specializes in Alzheimer's disease research and has published many related papers." Document 2 content is "XX University has several research projects focusing on neurodegenerative diseases, including Alzheimer's disease. Professor Li Ming is one of the leading researchers in this field." The execution process of the method is described as follows:

[0052] 1. Knowledge Graph Construction:

[0053] 1a. Extracting semantic triples from multiple documents: Using a large language model, semantic triples for constructing a knowledge graph are extracted from two documents. The extraction examples are "Li Ming, position, professor" and "Li Ming, research, Alzheimer's disease". The extracted examples represent information about individuals, relationships, and attributes.

[0054] 1b. Construct a knowledge graph: Add these extracted semantic triples to the knowledge graph to form a structure of nodes and edges;

[0055] 1c. Add extra edges: Use a retrieval encoder to add extra edges for similar but not identical noun phrases to enhance the connectivity of the knowledge graph.

[0056] 2. Application of the HITS algorithm:

[0057] 2a. Input query: Given the query "Who is the professor at XX University who researches Alzheimer's disease?";

[0058] 2b. Select query nodes: Extract the key concepts "XX University" and "Alzheimer's disease" from the query and map them to the corresponding nodes in the knowledge graph;

[0059] 2c. Intelligent search algorithm retrieval: Starting from these query nodes, the HITS algorithm is used to search and calculate to find the scores of nodes related to the query.

[0060] 3. Retrieval and Generation:

[0061] 3a. Associating high-probability nodes with semantic triples: Based on the output of the intelligent search algorithm, high-probability nodes will be associated with corresponding triples;

[0062] 3b. Backtracking to the original text: Find the original text using these semantic triples;

[0063] 3c. Sorting and Output: Sort the original text segments according to the relevance scores of the nodes and output the most relevant text segments.

[0064] Through the above process, the system first extracts semantic triples from the document and constructs the corresponding knowledge graph. Then, it applies an intelligent search algorithm to find the node "Li Ming" that is related to the query, and finally associates it with the original text segment to generate an accurate answer in response to the user's query request.

[0065] Example 2

[0066] Based on Example 1, this example, corresponding to its method, provides an enhanced cross-document intelligent retrieval system based on knowledge memory, which includes:

[0067] The document processing module is configured to automatically extract corresponding semantic triples from multiple documents using a large language model. The semantic triples include at least information about individuals, relations, and attributes, and a patternless knowledge graph is constructed based on the extracted semantic triples.

[0068] The knowledge graph enhancement module is configured to use a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information from multiple documents can be integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval.

[0069] The HITS algorithm calculation module is configured to use the HITS algorithm to calculate the Hub value and Authority value of each node in the knowledge graph; wherein, the nodes in the knowledge graph represent entity elements in semantic triples;

[0070] The query processing module is configured to receive user query requests, retrieve and determine the most relevant node for the query request based on the node's Hub value and Authority value, and provide information that matches the query request.

[0071] Based on this, this embodiment also provides a computer-readable storage medium storing a computer program / instruction thereon, which, when executed by a processor, can implement the steps of the knowledge memory-based enhanced cross-document intelligent retrieval method in Embodiment 1.

[0072] Based on this, this embodiment also provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the steps of the knowledge memory-based enhanced cross-document intelligent retrieval method in Embodiment 1.

[0073] In this embodiment, these computer programs / instructions may be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the function specified in one or more steps of the method.

Claims

1. An enhanced cross-document intelligent retrieval method based on knowledge memory, characterized in that, The method includes the following steps: S1. Use a large language model to automatically extract corresponding semantic triples from multiple documents. The semantic triples include at least information about individuals, relations and attributes. Construct a patternless knowledge graph based on the extracted semantic triples. S2. Apply a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information from multiple documents can be integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval. S3. The HITS algorithm is used to calculate the Hub value and Authority value of each node in the knowledge graph; the nodes in the knowledge graph represent entity elements in semantic triples; S4. Receive the user's query request, and based on the node's Hub value and Authority value, retrieve and determine the node most relevant to the query request, and then provide information that matches the query request.

2. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 1, characterized in that: In step S1, the large language model is configured to recognize natural language expressions in multiple documents and convert them into structured semantic triples, wherein the information of individuals, relations and attributes is obtained directly from or by reasoning from the content of multiple documents.

3. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 1, characterized in that: The relation encoder applied in step S2 includes a retrieval encoder, which retrieves and associates similar but not identical noun phrases in different semantic triples, and adds additional relation edges between these noun phrases in the knowledge graph.

4. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 1, characterized in that: In step S3, when applying the HITS algorithm, the Hub value and Authority value of all nodes in the knowledge graph are initialized, and then iteratively updated until convergence. This initialization method makes the importance score of each node reflect its role in the knowledge graph.

5. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 1, characterized in that: In step S4, the retrieved relevant nodes are filtered to select the node with the highest comprehensive score; the comprehensive score is the result of weighting the node's Hub value and Authority value.

6. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 5, characterized in that: In step S4, when providing information that matches the query request, the original text segment in the original document is located by tracing back the original semantic triple corresponding to the most relevant node retrieved, and then fed back to the user.

7. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 6, characterized in that: Step S4 retrieves multiple relevant nodes corresponding to the query request, sorts the original text segments located according to the relevance score of each node, and provides the user with a sorted list of search results.

8. An enhanced cross-document intelligent retrieval system based on knowledge memory, characterized in that, The system includes: The document processing module is configured to automatically extract corresponding semantic triples from multiple documents using a large language model. The semantic triples include at least information about individuals, relations, and attributes, and a patternless knowledge graph is constructed based on the extracted semantic triples. The knowledge graph enhancement module is configured to use a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information from multiple documents can be integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval. The HITS algorithm calculation module is configured to use the HITS algorithm to calculate the Hub value and Authority value of each node in the knowledge graph; wherein, the nodes in the knowledge graph represent entity elements in semantic triples; The query processing module is configured to receive user query requests, retrieve and determine the most relevant node for the query request based on the node's Hub value and Authority value, and provide information that matches the query request.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that: When the computer program / instruction is executed by the processor, it implements the steps of the knowledge memory-based enhanced cross-document intelligent retrieval method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instruction is executed by the processor, it implements the steps of the knowledge memory-based enhanced cross-document intelligent retrieval method according to any one of claims 1 to 7.

Images