Title and entity retrieval-based multi-document knowledge graph question and answer method and system

Abstract

The invention belongs to the technical field of retrieval enhancement generation. According to the title and entity retrieval-based multi-document knowledge graph question and answer method and system, each document is analyzed to obtain a title set, a table set and a text block set corresponding to each document; generating a plurality of simulation question and answer pairs based on the number of characters of each textbox in the text block set, and performing entity extraction and relation extraction on the simulation question and answer pairs by adopting a large language model to obtain an entity set and a relation set; constructing a title layer according to the title set and the title hierarchical structure, constructing a text block layer according to the text block set and the table set, and constructing an entity network layer according to the entity set and the relation set; determining a sub-knowledge graph of each document according to the title layer, the text block layer and the entity network layer, and determining a multi-document knowledge graph according to the sub-knowledge graph of each document; determining question answers according to the query questions and the multi-document knowledge graph; according to the method and the device, cross-document information reasoning is realized, and question answering precision is improved.

Description

Technical Field

[0001] This invention relates to the field of retrieval enhancement generation technology, specifically to a multi-document knowledge graph question answering method and system based on title and entity retrieval. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development and widespread adoption of large language models, they have been applied in various fields, particularly demonstrating strong potential in question and answering (QA). However, despite the ability of these models to encode vast amounts of document knowledge into their parameters, they still face challenges in highly specialized and vertical applications, including limitations in the timeliness of model parameter knowledge, insufficient coverage of long-tail knowledge, and the potential for illusory production results.

[0004] In recent years, Retrieval Augmented Generation (RAG) has been recognized as an effective method to address issues such as insufficient model knowledge. It retrieves document information through semantic similarity, thereby providing additional knowledge to large language models and enhancing their performance in specific professional domains for knowledge-based question answering. However, when faced with complex and multi-hop Multi-Document Question Answering (MDQA) requirements, it is often necessary to deeply reason about the differences and connections between document content. A single semantic retrieval method is insufficient to effectively capture and recall the cross-document information required to support complex reasoning, resulting in search results that may contain redundant or irrelevant content. This low-quality information, as contextual input, may actually exacerbate the illusion risk of large language models. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a multi-document knowledge graph question answering method and system based on title and entity retrieval. By storing document knowledge as a multi-document knowledge graph with a "title structure-text block-trinity" structure (i.e., a fusion of the title layer, text block layer, and entity network layer), and using the title and triples as the retrieval index for document information, cross-document information reasoning is realized, greatly improving the accuracy of question answering.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a multi-document knowledge graph question answering method based on title and entity retrieval.

[0007] A multi-document knowledge graph question answering method based on title and entity retrieval includes the following process: Each document in a set of related documents is parsed to obtain the title set, table set, and text block set for each document. Based on the number of characters in each text box in the text block set, multiple simulated question-answer pairs are generated. A large language model is used to extract entities and relations from the simulated question-answer pairs to obtain entity sets and relation sets. Construct a heading layer based on the heading set and heading hierarchy, construct a text block layer based on the text block set and table set, and construct an entity network layer based on the entity set and relation set. Based on the title layer, text block layer, and entity network layer, determine the sub-knowledge graph of each document, and based on the sub-knowledge graph of each document, determine the multi-document knowledge graph; The answer to the question is determined based on the query question and the multi-document knowledge graph.

[0008] In one implementation of the first aspect of the present invention, before constructing the title layer knowledge graph based on the title set hierarchical structure, a vectorization process is also included: The document title, tables, simulated question-and-answer pairs, and entity sets are vectorized respectively. The tables are embedded by concatenating the document name, title name, and table header fields.

[0009] In one implementation of the first aspect of the present invention, a title layer knowledge graph is constructed based on the title set and the title hierarchy structure. ,include: ,in, This is the starting title node, indicating the parent or preceding title. This is the final heading, indicating a subordinate or subordinate heading. This indicates the subordinate structural relationship between headings. Representing the The title set of a document; Representing the The title hierarchy of a document.

[0010] In one implementation of the first aspect of the present invention, a text block layer is constructed based on a text block set and a table set. ,include: ,in, Representing the A set of text blocks for a document. Representing the A set of tables for a document; Construct an entity network layer knowledge graph based on entity sets and relation sets. ,include: ,in, This represents the head entity node in the entity network layer knowledge graph. Represents the tail entity node. This represents the relationship between two nodes. Representing the A set of relationships between documents. Representing the An entity set of documents.

[0011] In one implementation of the first aspect of the present invention, determining the answer to the question based on the query question and the overall knowledge graph includes: The query question is broken down into sub-questions to obtain a list of sub-questions; Extract keywords from the list of sub-problems and generate synonyms for the keywords. Each sub-problem corresponds to a set of words consisting of keywords and synonyms. Based on the word set, locate the starting node of the graph inference of the entity network layer and generate the simulated document name for each sub-problem; Based on the sub-problems and the simulated document name, determine Document title and A simulated question-and-answer pair; according to Based on the document title and the large language model, determine the text blocks and tables under the target title; Based on the obtained keywords and synonyms, the semantic similarity of the vectors is matched to obtain the starting node of the entity network layer. The first-order neighbor subgraph of the starting node of the graph reasoning of the entity network layer is obtained. Based on the large language model and the first-order neighbor subgraph, the triples related to each sub-problem are obtained. The text blocks are retrieved in reverse based on the triples. The obtained text blocks, tables, and Each simulated question-and-answer pair serves as a knowledge base, answering queries.

[0012] As a further limitation of the first aspect of the invention, the sub-problem and the simulated document name are determined. Document title and A simulated question-and-answer pair, including: ; ; ; Among them, among them, Indicates the candidate title for the word set Hit score calculation This indicates the total size of the title candidate set. Indicates the first The ranking of each candidate title in the HIT used; For the first A preprocessed set of words for candidate titles Representing the The similarity score of each candidate title. These are the weighting coefficients; By retrieved The document title obtains the hierarchical structure of the document title, and finally combines the semantic understanding capabilities of LLM to accurately locate the text blocks and tables under the target title.

[0013] Secondly, the present invention provides a multi-document knowledge graph question answering system based on title and entity retrieval.

[0014] A multi-document knowledge graph question answering system based on title and entity retrieval includes: The document parsing unit is configured to parse each document in a set of related documents to obtain the title set, table set, and text block set for each document. The entity-relation collection unit is configured to: generate multiple simulated question-and-answer pairs based on the number of characters in each text box in the text block set; and use a large language model to extract entities and relations from the simulated question-and-answer pairs to obtain entity sets and relation sets. The graph hierarchy generation unit is configured to: construct a title layer based on a title set and a title hierarchy structure; construct a text block layer based on a text block set and a table set; and construct an entity network layer based on an entity set and a relation set. The multi-document knowledge graph generation unit is configured to: determine the sub-knowledge graph of each document based on the title layer, text block layer and entity network layer, and determine the multi-document knowledge graph based on the sub-knowledge graph of each document; The query and answer unit is configured to determine the answer to the question based on the query question and the multi-document knowledge graph.

[0015] Thirdly, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the multi-document knowledge graph question-answering method based on title and entity retrieval as described in the first aspect of the present invention.

[0016] Fourthly, the present invention provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed as described in the first aspect of the present invention for multi-document knowledge graph question answering based on title and entity retrieval.

[0017] Fifthly, the present invention provides a computer program product comprising a computer program that, when executed by a processor, implements the multi-document knowledge graph question answering method based on title and entity retrieval as described in the first aspect of the present invention.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This invention innovatively proposes a multi-document knowledge graph question answering method based on title and entity retrieval. Leveraging the powerful performance of large language models, it automates the entire knowledge graph construction process, effectively addressing the complexity and difficulty of traditional knowledge graph construction. By introducing a multi-document knowledge graph structure that integrates title, text block, and entity network layers, it achieves rapid text block retrieval at both coarse-grained document title structures and fine-grained entity triples. This enables the model to quickly locate question-related content in multi-document scenarios, reducing the introduction of irrelevant text and significantly lowering the illusion risk when generating answers using large language models, thus improving the reliability and controllability of the results. Simultaneously, the document title structure provides differences and connections between documents in multi-hop reasoning and complex question scenarios, thereby providing the semantic coherence required for the model's reasoning.

[0019] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0021] Figure 1 A flowchart illustrating a multi-document knowledge graph question answering method based on title and entity retrieval, provided as an exemplary embodiment of the present invention; Figure 2 A schematic diagram of a multi-document knowledge graph question answering system based on title and entity retrieval provided as an exemplary embodiment of the present invention; Figure 3 A schematic diagram of a computer device provided for an exemplary embodiment of the present invention. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] As described in the background section, inefficient retrieval in multi-document question answering scenarios exacerbates the risk of model illusion. Furthermore, existing knowledge graphs (RAGs) treat multiple documents as flattened text collections, ignoring the hierarchical structure within documents and entity relationships across documents, resulting in insufficient knowledge to support the semantic coherence required for complex reasoning. Therefore, this implementation proposes a multi-document knowledge graph question answering method based on title and entity retrieval. It employs a knowledge graph with a "title structure-text block-triple" structure as the external retrieval knowledge base for the RAG, and automatically constructs the knowledge graph using a large language model. Combining semantic similarity with the reasoning capabilities of the large language model, it achieves rapid document information retrieval using a dual-granularity indexing approach based on document title structure and triples.

[0025] More specifically, such as Figure 1 As shown, the process includes the following: S1: Perform document parsing.

[0026] For a set of related documents Docs, the first One document Use document conversion tools Document Convert to Markdown document Markdown format documents can effectively represent structural and diagrammatic information within a document; subsequently, document parsing tools are used. Get Title Set and heading hierarchy ,pass The document is broken down into text blocks to obtain a set of tables. With text block set .

[0027] The specific formula is as follows: (1).

[0028] S2: Knowledge Extraction and Graph Construction.

[0029] S201: Triple extraction.

[0030] For text block sets Leveraging the rich knowledge inherent in LLM (Large Language Model), it first focuses on text blocks. (in, The number of characters is automatically generated. A simulated question-and-answer pair Subsequently, with the help of LLM, Deep semantic analysis is performed, entity extraction and relation extraction operations are executed, and finally an entity set is obtained. and relation set On the one hand, the advantage of this design lies in guiding LLM to extract information through simulated question-and-answer scenarios, enabling the identified entities and relationships to better match the user's actual query intent, thereby significantly improving the accuracy and usability of the subsequent question-and-answer system; on the other hand, the generated... Store it in the database as an additional question-and-answer knowledge base.

[0031] (2); (3); S202: Use the text-embedding-ada-002 embedding model to parse the document titles obtained from the document. ,sheet Simulation problem and entity set Vectorization is performed to determine the starting node of the graph retrieval process. Whether it is a title-level knowledge graph or an entity network-level knowledge graph, the semantic similarity is calculated using vectors to determine the starting node of the graph retrieval.

[0032] In this invention, the vectorization of table tb is specially processed. Considering the redundancy of table data during the embedding process, it was found that only the title and header fields are needed to summarize the core theme of the table. However, for tables that may have the same header fields and titles in different documents, simply embedding based on the title and header cannot distinguish the tables. Therefore, this invention adopts the concatenation method of "document name + title name + header fields" to embed the table, thereby improving the accuracy and distinguishability of table retrieval. This processing method enables the question-answering system to more accurately identify and match the specific table content pointed to by the user's query.

[0033] S203: According to the title set and heading hierarchy Constructing the title layer It can be represented as: (4); in, This is the starting title node, indicating the parent or preceding title. This is the final heading, indicating a subordinate or subordinate heading. This indicates the subordinate structural relationship between headings.

[0034] Then the text block set With tables As a text block layer : (5); Based on the extracted entity set With relation set Constructing the knowledge graph of the entity network layer : (6); in, This represents the head entity node in the entity network layer knowledge graph. Represents the tail entity node. This indicates the relationship between two nodes.

[0035] This is how the document is constructed. A multi-document knowledge graph of "title structure - text block - triple": (7); For document collections A single-document knowledge graph is constructed for each document using formula (7), and then entity fusion is performed on the entity network layer knowledge graph of each document knowledge graph to finally construct the knowledge graph. Its formula is: (8).

[0036] S204: Multi-document Q&A.

[0037] In multi-document question answering scenarios, methods based on dense vector embedding, character-level matching, or hybrid retrieval generally suffer from information overload. Although the retrieved text blocks may contain core segments relevant to the query intent, they may also contain a large amount of redundant information. This not only reduces the accuracy of the retrieval but also creates noise interference for subsequent answer generation. It is worth noting that some of the contextual information considered as interference actually contains important semantic relationships and domain knowledge.

[0038] Therefore, in the multi-document question answering module, by combining the powerful reasoning capabilities of LLM (Large Language Model), a dual-granularity retrieval mechanism is constructed, using the document title as the primary index of the text block content and entity triples as the secondary index of the text block. This enables efficient and accurate retrieval of text blocks from both macro and micro dimensions. The specific process is as follows: S204-1: Problem Breakdown.

[0039] Complex multi-document query problems with multiple hops Break down the problem into subproblems to obtain a list of subproblems. (Right now Figure 1By breaking down complex multi-hop problems into smaller, more manageable tasks, LLM can better grasp their underlying intent and use this as a driving force for iterative retrieval of document information.

[0040] S204-2: Sub-problem information mining.

[0041] List of sub-problems Keyword extraction and synonym generation are performed to obtain a word set. By locating the starting node of entity network layer graph inference through vector matching, the simulated document name of each sub-problem is generated, which serves as an important indicator for title-level retrieval.

[0042] S204-3: Title-level retrieval.

[0043] The keywords, synonyms, and simulated document names generated in S204-2 are retrieved using a hybrid retrieval method that combines dense vector matching and topic term matching. Document title and simulated question and answer pair The calculation formula is as follows: (9); (10); (11); in, Indicates the candidate title for the word set Hit score calculation This indicates the total size of the title candidate set. Indicates the first The ranking of each candidate title in the HIT used; For the first The set of words after preprocessing the title (here, preprocessing the candidate title, specifically by word segmentation); then, through retrieval... The document title obtains the hierarchical structure of the document title, and finally combines the semantic understanding capabilities of LLM to accurately locate the text blocks and tables under the target title.

[0044] The overall calculation process is as follows: First, based on semantic similarity, the top K candidate titles are retrieved from the title vector database, and the similarity score of each candidate title is recorded. Formulas (9) and (10) calculate the hit score of each candidate title. Finally, the final score of each candidate title is obtained according to formula (11). The titles are then reordered to obtain the final titles. In this invention, the simulated document name is the union of the subproblems, and the top K candidate titles are obtained through the vector library.

[0045] S204-4: Entity network layer retrieval.

[0046] Get a set of words The first-order neighbor subgraph of the starting node in the localized entity network layer The triples related to each sub-problem are obtained through a large language model. Ultimately through triples By searching for text blocks in reverse, text block retrieval at the network layer can be achieved.

[0047] S204-5: The above-retrieved results (i.e., text block) (i.e., the table) and (That is, simulated question-and-answer pairs) serve as a knowledge base to answer user questions; if the current information does not fully resolve the issue, the above steps are repeated. In each iteration, the quality of the answer is gradually optimized by dynamically adjusting the retrieval strategy and reorganizing relevant information until the problem is fully resolved.

[0048] In this implementation, the retrieval strategy is dynamically adjusted through iteration on a large model. Specifically, we combine the retrieved information with the original query question Q and a list of sub-questions. As input, the large model first determines whether the retrieved information is sufficient to answer all sub-questions and thus provide a final answer to question Q. If so, it outputs the answer. If the retrieved information is insufficient to provide a final answer, the large model summarizes the information that needs to be retrieved further and generates a new question as input for the next iteration. Figure 2 This paper presents a multi-document knowledge graph question-answering system based on title and entity retrieval, including: Document parsing unit 201 is configured to parse each document in a set of related documents to obtain the title set, table set and text block set corresponding to each document; The entity relation collection unit 202 is configured to: generate multiple simulated question-and-answer pairs based on the number of characters in each text box in the text block set; and use a large language model to extract entities and relations from the simulated question-and-answer pairs to obtain an entity set and a relation set. The graph hierarchy generation unit 203 is configured to: construct a title layer based on a title set and a title hierarchy structure; construct a text block layer based on a text block set and a table set; and construct an entity network layer based on an entity set and a relation set. The multi-document knowledge graph generation unit 204 is configured to: determine the sub-knowledge graph of each document based on the title layer, text block layer and entity network layer, and determine the multi-document knowledge graph based on the sub-knowledge graph of each document; The query and answer unit 205 is configured to determine the answer to the question based on the query question and the multi-document knowledge graph.

[0049] It is understood that the aforementioned units can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of the present invention. The aforementioned units are based on logical functional division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of the present invention, the system may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0050] According to another embodiment of the present invention, the system described in this embodiment can be constructed by running a computer program (including program code) capable of performing the steps involved in the corresponding method of the present invention on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, loaded into the aforementioned computing device through the computer-readable recording medium, and run therein.

[0051] Figure 3 A computer device is shown, which includes a processor 301, a communication interface 302, and a computer-readable storage medium 303. The processor 301, communication interface 302, and computer-readable storage medium 303 can be connected via a bus or other means.

[0052] The communication interface 302 is used to receive and send data. The computer-readable storage medium 303 can be stored in the memory of the electronic device. The computer-readable storage medium 303 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the computer-readable storage medium 303.

[0053] The processor 301 is the computing and control core of the electronic device. It is suitable for implementing one or more instructions, specifically for loading and executing one or more instructions to achieve the corresponding method flow or corresponding function.

[0054] The processor 301 is configured to perform the following process: Each document in a set of related documents is parsed to obtain the title set, table set, and text block set for each document. Based on the number of characters in each text box in the text block set, multiple simulated question-answer pairs are generated. A large language model is used to extract entities and relations from the simulated question-answer pairs to obtain entity sets and relation sets. Construct a heading layer based on the heading set and heading hierarchy, construct a text block layer based on the text block set and table set, and construct an entity network layer based on the entity set and relation set. Based on the title layer, text block layer, and entity network layer, determine the sub-knowledge graph of each document, and based on the sub-knowledge graph of each document, determine the multi-document knowledge graph; The answer to the question is determined based on the query question and the multi-document knowledge graph.

[0055] This invention also provides a computer-readable storage medium, which is a memory device in an electronic device for storing programs and data. It is understood that the computer-readable storage medium here may include both built-in storage media in the electronic device and extended storage media supported by the electronic device. The computer-readable storage medium provides storage space for storing the processing system of the electronic device.

[0056] Furthermore, this storage space also contains one or more instructions suitable for loading and execution by the processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM memory or unstable memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0057] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor loads and executes the one or more instructions stored in the computer-readable storage medium to perform the following process: Each document in a set of related documents is parsed to obtain the title set, table set, and text block set for each document. Based on the number of characters in each text box in the text block set, multiple simulated question-answer pairs are generated. A large language model is used to extract entities and relations from the simulated question-answer pairs to obtain entity sets and relation sets. Construct a heading layer based on the heading set and heading hierarchy, construct a text block layer based on the text block set and table set, and construct an entity network layer based on the entity set and relation set. Based on the title layer, text block layer, and entity network layer, determine the sub-knowledge graph of each document, and based on the sub-knowledge graph of each document, determine the multi-document knowledge graph; The answer to the question is determined based on the query question and the multi-document knowledge graph.

[0058] The present invention also provides a computer program product or computer program comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the following process: Each document in a set of related documents is parsed to obtain the title set, table set, and text block set for each document. Based on the number of characters in each text box in the text block set, multiple simulated question-answer pairs are generated. A large language model is used to extract entities and relations from the simulated question-answer pairs to obtain entity sets and relation sets. Construct a heading layer based on the heading set and heading hierarchy, construct a text block layer based on the text block set and table set, and construct an entity network layer based on the entity set and relation set. Based on the title layer, text block layer, and entity network layer, determine the sub-knowledge graph of each document, and based on the sub-knowledge graph of each document, determine the multi-document knowledge graph; The answer to the question is determined based on the query question and the multi-document knowledge graph.

[0059] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0060] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic cable, digital cable) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data processing device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0061] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-document knowledge graph question answering method based on title and entity retrieval, characterized in that, Includes the following processes: Each document in a set of related documents is parsed to obtain the title set, table set, and text block set for each document. Based on the number of characters in each text box in the text block set, multiple simulated question-answer pairs are generated. A large language model is used to extract entities and relations from the simulated question-answer pairs to obtain entity sets and relation sets. Construct a heading layer based on the heading set and heading hierarchy, construct a text block layer based on the text block set and table set, and construct an entity network layer based on the entity set and relation set. Based on the title layer, text block layer, and entity network layer, determine the sub-knowledge graph of each document, and based on the sub-knowledge graph of each document, determine the multi-document knowledge graph; The answer to the question is determined based on the query question and the multi-document knowledge graph.

2. The multi-document knowledge graph question answering method based on title and entity retrieval as described in claim 1, characterized in that, Before constructing the title-level knowledge graph based on the title hierarchy of the title set, a vectorization process is also included: The document title, tables, simulated question-and-answer pairs, and entity sets are vectorized respectively. The tables are embedded by concatenating the document name, title name, and table header fields.

3. The multi-document knowledge graph question answering method based on title and entity retrieval as described in claim 1, characterized in that, Construct a title-level knowledge graph based on the title set and title hierarchy. ,include: ,in, This is the starting title node, indicating the parent or preceding title. This is the final heading, indicating a subordinate or subordinate heading. This indicates the subordinate structural relationship between headings. Representing the The title set of a document; Representing the The title hierarchy of a document.

4. The multi-document knowledge graph question answering method based on title and entity retrieval as described in claim 1, characterized in that, Construct a text block layer based on the text block set and the table set. ,include: ,in, Representing the A set of text blocks for a document. Representing the A set of tables for a document; Construct an entity network layer knowledge graph based on entity sets and relation sets. ,include: ,in, This represents the head entity node in the entity network layer knowledge graph. Represents the tail entity node. This represents the relationship between two nodes. Representing the A set of relationships between documents. Representing the An entity set of documents.

5. The multi-document knowledge graph question answering method based on title and entity retrieval as described in any one of claims 1-4, characterized in that, Based on the query question and the overall knowledge graph, determine the answer to the question, including: The query question is broken down into sub-questions to obtain a list of sub-questions; Extract keywords from the list of sub-problems and generate synonyms for the keywords. Each sub-problem corresponds to a set of words consisting of keywords and synonyms. Based on the word set, locate the starting node of the graph inference of the entity network layer and generate the simulated document name for each sub-problem; Based on the sub-problems and the simulated document name, determine Document title and A simulated question-and-answer pair; according to Based on the document title and the large language model, determine the text blocks and tables under the target title; Based on the obtained keywords and synonyms, the semantic similarity of the vectors is matched to obtain the starting node of the entity network layer. The first-order neighbor subgraph of the starting node of the graph reasoning of the entity network layer is obtained. Based on the large language model and the first-order neighbor subgraph, the triples related to each sub-problem are obtained. The text blocks are retrieved in reverse based on the triples. The obtained text blocks, tables, and Each simulated question-and-answer pair serves as a knowledge base, answering queries.

6. The multi-document knowledge graph question answering method based on title and entity retrieval as described in claim 5, characterized in that, Based on the sub-problems and the simulated document name, determine Document title and A simulated question-and-answer pair, including: ； ； ； Among them, among them, Indicates the candidate title for the word set Hit score calculation This indicates the total size of the title candidate set. Indicates the first The ranking of each candidate title in the HIT used; For the first A preprocessed set of words for candidate titles Representing the The similarity score of each candidate title. These are the weighting coefficients; By retrieved The document title obtains the hierarchical structure of the document title, and finally combines the semantic understanding capabilities of LLM to accurately locate the text blocks and tables under the target title.

7. A multi-document knowledge graph question-answering system based on title and entity retrieval, characterized in that, include: The document parsing unit is configured to parse each document in a set of related documents to obtain the title set, table set, and text block set for each document. The entity-relation collection unit is configured to: generate multiple simulated question-and-answer pairs based on the number of characters in each text box in the text block set; and use a large language model to extract entities and relations from the simulated question-and-answer pairs to obtain entity sets and relation sets. The graph hierarchy generation unit is configured to: construct a title layer based on a title set and a title hierarchy structure; construct a text block layer based on a text block set and a table set; and construct an entity network layer based on an entity set and a relation set. The multi-document knowledge graph generation unit is configured to: determine the sub-knowledge graph of each document based on the title layer, text block layer and entity network layer, and determine the multi-document knowledge graph based on the sub-knowledge graph of each document; The query and answer unit is configured to determine the answer to the question based on the query question and the multi-document knowledge graph.

8. A computer device, characterized in that, include: Processor and computer-readable storage media; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the multi-document knowledge graph question answering method based on title and entity retrieval as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1 to 6, which is a multi-document knowledge graph question answering method based on title and entity retrieval.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the multi-document knowledge graph question answering method based on title and entity retrieval as described in any one of claims 1 to 6.

Images