Enhanced cross-document intelligent retrieval method and system based on knowledge memory

Abstract

The invention provides an enhanced cross-document intelligent retrieval method and system based on knowledge memory, relates to the field of information retrieval, and solves the problem of limitation of a traditional retrieval enhanced generation method in cross-document information integration, calculation efficiency and complex question and answer scene processing. The method comprises the following steps of: extracting semantic triples from a plurality of documents by adopting a large language model, and constructing a modelless knowledge graph; a relation encoder is applied to enhance semantic connectivity among different semantic triples, so that information in a plurality of documents is integrated and is used for subsequent comprehensive retrieval; adopting an HITS algorithm to calculate a Hub value and an Authority value of each node in the knowledge graph; a query request of a user is received, the most relevant node is retrieved and determined based on the Hub value and the Authority value of the node, and information matched with the query request is provided; according to the invention, the information retrieval and generation capability of the natural language processing system is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of information retrieval, and is applied to a RAG system for natural language processing, and in particular to an enhanced cross-document intelligent retrieval method and system based on knowledge memory. Background Art

[0002] In recent years, the field of Natural Language Processing (NLP) has made significant progress, especially in the application of Large Language Models (LLMs). These models can generate coherent and informative texts by learning patterns from massive text data, providing strong support for various application scenarios such as machine translation, content creation, and dialogue systems. With the development of technology, people have higher expectations for the quality of generated content, requiring it not only to be highly relevant, but also to accurately reflect the latest developments in the topic being discussed. In order to meet such needs, the Retrieval-Augmented Generation (RAG) method was proposed, which improves the quality and accuracy of the output by introducing external knowledge sources in the generation process.

[0003] Although the RAG method has shown great potential, it still faces a series of challenges in practical application. First, under the traditional RAG framework, each document is treated as an independent information unit for processing. Although this approach simplifies the process of information extraction, it ignores a key fact: knowledge in the real world is interrelated rather than isolated. Therefore, when faced with tasks that require integrating information from different sources, methods based on single document processing are difficult to fully tap the potential knowledge connections, thereby limiting the richness and accuracy of the final generated content.

[0004] In addition, another important limitation of existing RAG systems is their high computational cost. Usually, such systems adopt an iterative retrieval strategy, that is, they repeatedly query relevant information based on the initial generated results until a satisfactory result is achieved. This process not only consumes a lot of computing resources, but also increases the response time, affecting the user experience. Especially on mobile devices or in poor network conditions, long waiting times may become an important factor in users giving up using the service.

[0005] Finally, traditional RAG solutions do not perform well in problem-solving scenarios involving complex logical reasoning. These problems often require extracting and synthesizing information from multiple documents that are not directly related to obtain the correct answer. However, due to the lack of an effective cross-document information fusion mechanism, existing technologies are difficult to effectively support such tasks, resulting in poor performance in situations that require deep understanding and extensive search capabilities.

[0006] In summary, although the current RAG technology has achieved certain success, it still has obvious shortcomings in achieving more efficient, accurate and adaptable information retrieval and generation. To address this problem, it is urgent to develop new solutions to overcome the limitations of a single document processing method, reduce the computational burden, and improve the ability to support complex question-answering scenarios. Summary of the invention

[0007] The purpose of the present invention is to solve the limitations of traditional retrieval enhancement generation methods in cross-document information integration, computational efficiency and processing of complex question-and-answer scenarios. Therefore, an enhanced cross-document intelligent retrieval method and system based on knowledge memory is proposed; the present invention effectively improves the information retrieval and generation capabilities of the natural language processing system through the construction of a knowledge graph and the application of a keyword hitting algorithm.

[0008] The present invention adopts the following technical solutions to achieve the purpose:

[0009] An enhanced cross-document intelligent retrieval method based on knowledge memory includes the following steps:

[0010] S1. Automatically extract corresponding semantic triples from multiple documents using a large language model, wherein the semantic triples include at least information about individuals, relationships, and attributes, and construct a schema-free knowledge graph based on the extracted semantic triples;

[0011] S2. applying a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information in multiple documents can be integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval;

[0012] S3. Calculate the Hub value and Authority value of each node in the knowledge graph using the HITS algorithm; the nodes in the knowledge graph represent entity elements in the semantic triples;

[0013] S4. Receive the user's query request, retrieve and determine the most relevant node corresponding to the query request based on the Hub value and Authority value of the node, and then provide information matching the query request.

[0014] Specifically, in step S1, the large language model is configured to recognize natural language expressions in multiple documents and convert them into structured semantic triples, wherein the information of individuals, relationships and attributes are directly extracted or inferred from the contents of multiple documents.

[0015] Preferably, the relation encoder applied in step S2 includes a retrieval encoder, which is used to retrieve and associate similar but not identical noun phrases in different semantic triples, and add additional relation edges between these name phrases in the knowledge graph.

[0016] Preferably, in step S3, when applying the HITS algorithm, the Hub values ​​and Authority values ​​of all nodes in the knowledge graph are initialized and then iteratively updated until convergence. Through this initialization method, the importance score of each node reflects its role in the knowledge graph.

[0017] Specifically, in step S4, the retrieved related nodes are screened to select the node with the highest comprehensive score; the comprehensive score is a weighted result based on the Hub value and Authority value of the node.

[0018] Preferably, in step S4, when providing information matching the query request, the original semantic triple corresponding to the most relevant node retrieved is back-traced to locate the corresponding original text segment in the original document and feed it back to the user.

[0019] Preferably, step S4 retrieves a plurality of relevant nodes corresponding to the query request, sorts the located original text segments according to the relevance scores of the nodes, and provides the user with a sorted search result list.

[0020] The present invention also provides an enhanced cross-document intelligent retrieval system based on knowledge memory, the system comprising:

[0021] A document processing module, configured to automatically extract corresponding semantic triples from multiple documents using a large language model, wherein the semantic triples include at least information about individuals, relationships, and attributes, and construct a schema-free knowledge graph based on the extracted semantic triples;

[0022] A knowledge graph enhancement module, configured to apply a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information in multiple documents is integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval;

[0023] A HITS algorithm calculation module, configured to use the HITS algorithm to calculate the Hub value and the Authority value of each node in the knowledge graph; wherein the nodes in the knowledge graph represent entity elements in a semantic triple;

[0024] The query processing module is configured to receive a user's query request, retrieve and determine the most relevant node corresponding to the query request based on the Hub value and Authority value of the node, and provide information matching the query request.

[0025] The present invention also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the aforementioned enhanced cross-document intelligent retrieval method based on knowledge memory are implemented.

[0026] The present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the aforementioned enhanced cross-document intelligent retrieval method based on knowledge memory.

[0027] In summary, due to the adoption of this technical solution, the beneficial effects of the present invention are as follows:

[0028] The method of the present invention realizes the effective integration of cross-document information. The present invention identifies and connects related content from different documents through algorithm design to build a richer and more comprehensive knowledge graph; this method is particularly important for processing complex queries, which ensures that even when faced with questions that require comprehensive multi-source information to answer, the system can provide accurate and detailed answers, thereby greatly improving the quality and relevance of the output results.

[0029] The present invention also has some progress in computing efficiency. Unlike the traditional method of using multiple iterative retrieval, the present invention supports completing all necessary retrieval steps in the multi-hop reasoning process at one time. This method greatly reduces the computing resources required for each operation and shortens the overall response time. Therefore, users can get nearly instant feedback, which is of great significance for improving user experience and promoting the development of real-time application scenarios.

[0030] The present invention also performs well in complex question-and-answer scenarios involving deep logical analysis and extensive information search requirements, and has the ability to extract key points from fragmentary information scattered in multiple indirect documents and form coherent and accurate answers based on them. This enables the system to give high-quality answers after retrieval even in the face of challenging inquiries, fully meeting the growing demand for intelligent information services in modern society. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 The overall process diagram of the method of the present invention is briefly described;

[0032] Figure 2 The present invention is a schematic diagram of a process in which the method of the present invention is executed in combination with an example process. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0034] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Example 1

[0036] An enhanced cross-document intelligent retrieval method based on knowledge memory, Figure 1 The process of the method is briefly described in FIG. 1 , which can be viewed simultaneously. The steps of the method are summarized as follows:

[0037] S1. Automatically extract corresponding semantic triples from multiple documents using a large language model, wherein the semantic triples include at least information about individuals, relationships, and attributes, and construct a schema-free knowledge graph based on the extracted semantic triples;

[0038] S2. applying a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information in multiple documents can be integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval;

[0039] S3, using the HITS algorithm to calculate the Hub value and Authority value of each node in the knowledge graph; the nodes in the knowledge graph represent entity elements in the semantic triples; the purpose of this step is to achieve a more accurate response to user queries;

[0040] S4. Receive the user's query request, retrieve and determine the most relevant node corresponding to the query request based on the Hub value and Authority value of the node, and then provide information matching the query request.

[0041] In the above method, the schema-less knowledge graph mentioned in step S1 refers to a knowledge graph that does not require strict pre-defined data schema or structure when it is constructed; this means that the knowledge graph can be automatically generated and expanded based on the document content without being restricted by a pre-set schema. The advantage of this is that it can flexibly handle various types of documents, and as more documents are added, the knowledge graph can be continuously enriched and developed to provide more comprehensive information retrieval capabilities. Although traditional knowledge graphs maintain the consistency and integrity of data through strict schema definition, they also limit their flexibility.

[0042] In the above method, the HITS algorithm mentioned in step S3 is a link analysis algorithm, which evaluates the importance or relevance of nodes by calculating two importance indicators of each node in the network, namely, the Hub value and the Authority value; in addition to being applied to the link structure between web pages, the algorithm can also be effectively applied to other types of graph structures such as knowledge graphs.

[0043] Therefore, in this embodiment, the HITS algorithm is used to improve the relevance and retrieval efficiency of entity nodes in the knowledge graph (i.e., noun phrases corresponding to information such as individuals, relationships, and attributes in semantic triples). Specifically: the Hub value represents the degree to which an entity is a "recommender", that is, its ability to point to other authoritative entities; the Authority value represents the degree to which an entity is an "expert" or "authority", that is, the degree to which it is pointed to by other important recommenders.

[0044] In the process of step S3 and step S4 of this embodiment, by calculating these values, it is possible to identify which nodes (entities) in the knowledge graph are the most important under a specific query request topic, thereby helping the system to respond to the user's query request more accurately. In this way, when the user enters a query, the system can quickly locate and query the most relevant information based on the Hub value and Authority value of the node.

[0045] The following is a detailed and preferred introduction to the method process of this embodiment.

[0046] In step S1, the large language model is configured to recognize natural language expressions in multiple documents and convert them into structured semantic triples, wherein information of individuals, relationships, and attributes are directly extracted or inferred from the contents of multiple documents.

[0047] The relation encoder applied in step S2 includes a retrieval encoder, which is used to retrieve and associate similar but not identical noun phrases in different semantic triples, and add additional relation edges between these name phrases in the knowledge graph to increase the connectivity and richness of the knowledge graph.

[0048] In step S3, when the HITS algorithm is applied, the Hub values ​​and Authority values ​​of all nodes in the knowledge graph are initialized, and then iteratively updated until convergence. Through this initialization method, the importance score of each node reflects its role in the knowledge graph.

[0049] In step S4, the retrieved related nodes are screened and the nodes with the highest comprehensive scores are selected; the comprehensive scores are weighted according to the Hub value and Authority value of the nodes. In addition, when providing information matching the query request, the original semantic triples corresponding to the most relevant nodes retrieved are traced back to locate the corresponding original text in the original document and feedback to the user, so that the user can obtain more detailed background information.

[0050] Step S4 retrieves multiple relevant nodes corresponding to the query request, sorts the located original text segments according to the relevance scores of the nodes, and provides the user with a sorted search result list, thereby improving the effectiveness of information presentation and user experience.

[0051] This embodiment will now illustrate the execution process of the above method with a more specific example, which can be simultaneously referred to in Figure 2 Example description: The user's query request is "Who is the professor who studies Alzheimer's disease at XX University?" Document 1 contains "Li Ming is a professor at XX University. He specializes in Alzheimer's disease research and has published many related papers." Document 2 contains "XX University has several research projects focusing on neurodegenerative diseases, including Alzheimer's disease. Professor Li Ming is one of the leading researchers in this field." The execution process of the method is described as follows:

[0052] 1. Knowledge graph construction:

[0053] 1a. Extracting semantic triples from multiple documents: Using a large language model, semantic triples for constructing a knowledge graph are extracted from two documents. The extracted examples are "Li Ming, position, professor" and "Li Ming, research, Alzheimer's disease". The extracted examples here represent information about individuals, relationships, and attributes.

[0054] 1b. Build a knowledge graph: Add these extracted semantic triples to the knowledge graph to form a structure of nodes and edges;

[0055] 1c. Adding extra edges: Use the retrieval encoder to add extra edges for similar but not identical noun phrases to enhance the connectivity of the knowledge graph.

[0056] 2. Application of HITS algorithm:

[0057] 2a. Input query: Given the query "Who is the professor who studies Alzheimer's disease at XX University?";

[0058] 2b. Select query nodes: extract the key concepts "XX University" and "Alzheimer's disease" in the query and map them to the corresponding nodes in the knowledge graph;

[0059] 2c. Intelligent search algorithm retrieval: Starting from these query nodes, the HITS algorithm is used to search and calculate to find the scores of nodes related to the query.

[0060] 3. Retrieval and generation:

[0061] 3a. Associate high probability nodes with semantic triples: Based on the output of the intelligent search algorithm, high probability nodes are associated with corresponding triples;

[0062] 3b. Go back to the original text: find the original text through these semantic triples;

[0063] 3c. Sorting and output: Sort the original text segments according to the relevance scores of the nodes and output the most relevant ones.

[0064] Through the above process, the system first extracts semantic triples from the document and constructs the corresponding knowledge graph, then applies an intelligent search algorithm to find the node "Li Ming" related to the query, and finally associates it with the original text, generates accurate answers, and responds to the user's query requests.

[0065] Example 2

[0066] Based on Example 1, this embodiment corresponds to the method thereof and provides an enhanced cross-document intelligent retrieval system based on knowledge memory, the system comprising:

[0067] A document processing module, configured to automatically extract corresponding semantic triples from multiple documents using a large language model, wherein the semantic triples include at least information about individuals, relationships, and attributes, and construct a schema-free knowledge graph based on the extracted semantic triples;

[0068] A knowledge graph enhancement module, configured to apply a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information in multiple documents is integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval;

[0069] A HITS algorithm calculation module, configured to use the HITS algorithm to calculate the Hub value and the Authority value of each node in the knowledge graph; wherein the nodes in the knowledge graph represent entity elements in a semantic triple;

[0070] The query processing module is configured to receive a user's query request, retrieve and determine the most relevant node corresponding to the query request based on the Hub value and Authority value of the node, and provide information matching the query request.

[0071] On this basis, this embodiment also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the enhanced cross-document intelligent retrieval method based on knowledge memory in Embodiment 1 can be implemented.

[0072] On this basis, this embodiment also provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the steps of the enhanced cross-document intelligent retrieval method based on knowledge memory in Embodiment 1.

[0073] In this embodiment, these computer programs / instructions may be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more steps in the method.

Claims

1. An enhanced cross-document intelligent retrieval method based on knowledge memory, characterized in that: The method comprises the following steps: S1. Automatically extract corresponding semantic triples from multiple documents using a large language model, wherein the semantic triples include at least information about individuals, relationships, and attributes, and construct a schema-free knowledge graph based on the extracted semantic triples; S2. applying a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information in multiple documents can be integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval; S3. Calculate the Hub value and Authority value of each node in the knowledge graph using the HITS algorithm; the nodes in the knowledge graph represent entity elements in the semantic triples; S4. Receive the user's query request, retrieve and determine the most relevant node corresponding to the query request based on the Hub value and Authority value of the node, and then provide information matching the query request.

2. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 1 is characterized in that: In step S1, the large language model is configured to recognize natural language expressions in multiple documents and convert them into structured semantic triples, wherein the information of individuals, relationships and attributes are directly extracted or inferred from the contents of multiple documents.

3. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 1 is characterized in that: The relation encoder applied in step S2 includes a retrieval encoder, which is used to retrieve and associate similar but not identical noun phrases in different semantic triples, and add additional relation edges between these name phrases in the knowledge graph.

4. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 1 is characterized in that: In step S3, when the HITS algorithm is applied, the Hub values ​​and Authority values ​​of all nodes in the knowledge graph are initialized, and then iteratively updated until convergence. Through this initialization method, the importance score of each node reflects its role in the knowledge graph.

5. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 1 is characterized in that: In step S4, the retrieved related nodes are screened to select the node with the highest comprehensive score; the comprehensive score is a weighted result based on the Hub value and Authority value of the node.

6. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 5 is characterized in that: In step S4, when providing information matching the query request, the original semantic triple corresponding to the most relevant node retrieved is back-traced to locate the corresponding original text segment in the original document and feed it back to the user.

7. The enhanced cross-document intelligent retrieval method based on knowledge memory according to claim 6 is characterized in that: Step S4 retrieves a plurality of relevant nodes corresponding to the query request, sorts the located original text segments according to the relevance scores of the nodes, and provides the user with a sorted search result list.

8. An enhanced cross-document intelligent retrieval system based on knowledge memory, characterized in that: The system comprises: A document processing module, configured to automatically extract corresponding semantic triples from multiple documents using a large language model, wherein the semantic triples include at least information about individuals, relationships, and attributes, and construct a schema-free knowledge graph based on the extracted semantic triples; A knowledge graph enhancement module, configured to apply a relation encoder to enhance the semantic connectivity between different semantic triples in the knowledge graph, so that information in multiple documents is integrated in the knowledge graph based on semantic triples and used for subsequent comprehensive retrieval; A HITS algorithm calculation module, configured to use the HITS algorithm to calculate the Hub value and the Authority value of each node in the knowledge graph; wherein the nodes in the knowledge graph represent entity elements in a semantic triple; The query processing module is configured to receive a user's query request, retrieve and determine the most relevant node corresponding to the query request based on the Hub value and Authority value of the node, and provide information matching the query request.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the enhanced cross-document intelligent retrieval method based on knowledge memory as described in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instruction is executed by a processor, the steps of the enhanced cross-document intelligent retrieval method based on knowledge memory as described in any one of claims 1 to 7 are implemented.

Images