Multi-document intelligent question and answer method, device and equipment based on multi-agent cooperation

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a multi-document intelligent question and answer method, device and equipment based on multi-agent collaboration, and the method comprises the steps: partitioning a target document, extracting entities and relationships, constructing a knowledge graph, generating entity description, constructing a community hierarchical structure and generating vector representation; processing the new document to obtain entities and relationships in the new document, updating the knowledge graph, re-dividing communities and updating vector representation; and clustering the entity list in the updated knowledge graph, determining correlation scores of entities and questions to sort the entities, retrieving adjacent entities, relationships, belonging communities and belonging text block sets of the entities meeting preset conditions, and generating corresponding answers based on a large model. According to the method, the key information related to the question can be efficiently, accurately and fully extracted from the document, the language model is guided to generate a high-quality answer, and then the effect of the question answering system in the aspect of detail question answering is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, for example, to a multi-document intelligent question-answering method, device, and equipment based on multi-agent collaboration. Background Art

[0002] In the era of big data, a large amount of knowledge is stored in massive amounts of unstructured text data. How to efficiently retrieve knowledge related to questions from huge document repositories and generate high-quality answers has become a core topic in the field of natural language processing and intelligent question answering.

[0003] Traditional intelligent question-answering systems mainly use the Retrieval-Augmented Generation (RAG) method for processing. The process includes: first, retrieving text fragments that are semantically similar to the question from the text corpus, and then generating the final answer based on these fragments. In this process, semantic-based text fragment retrieval and semantic-understanding-based question-answering generation are two key links.

[0004] In the semantic-based text segment retrieval stage, the traditional RAG method relies on vector matching technology to retrieve text segments from the corpus that are similar to the question semantics. However, this method treats each text segment as an independent information unit, making it difficult to capture the deep-level relationships within the segment and between different segments, resulting in an insufficient understanding of the question semantics, thus limiting the quality of question answering.

[0005] In the question-answer generation process based on semantic understanding, using large language models to generate answers is the current mainstream method. However, when faced with long documents or multiple documents, existing RAG systems have the following prominent problems: 1. It is difficult to provide high-precision detailed answers: Although text fragments reduce redundant information in documents that is not related to the question, they still contain a lot of irrelevant content. Directly inputting semantically similar text fragments into the language model can easily lead to ignoring key information. In addition, since multiple text fragments are independent of each other, it is difficult for the model to integrate information to generate high-quality answers.

[0006] Second, it is difficult to ensure high recall of key information: The length of text processed by the language model is limited, and the number of segments that can be processed is also limited. The key information related to the question may be scattered in multiple segments. The existing RAG system is difficult to provide the model with enough key information to generate a comprehensive answer.

[0007] Therefore, how to efficiently retrieve key information related to the question from massive text data and provide sufficient supporting information for the language model has become an urgent problem that needs to be solved in the intelligent question answering system.

[0008] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this application. Summary of the invention

[0009] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0010] The present disclosure provides a multi-document intelligent question answering method based on multi-agent collaboration, the method comprising: Segment the target document, extract entities and relations, build a knowledge graph, generate entity descriptions, build a community hierarchy, and generate vector representations; Process the new document to obtain the entities and relationships in the new document, update the knowledge graph, re-divide the community, and update the vector representation; Cluster the list of entities in the updated knowledge graph and determine the relevance score of the entity to the question; Sort entities based on relevance scores, and retrieve adjacent entities, relationships, communities, and text blocks for entities that meet preset conditions; Sort and deduplicate adjacent entities, relationships, communities, and text block collections, and generate corresponding answers based on the preset large model.

[0011] In some embodiments, segmenting a target document, extracting entities and relationships, constructing a knowledge graph, generating entity descriptions, constructing a community hierarchy, and generating a vector representation include: Divide each document in the corresponding set of the target document into blocks according to the preset length, and divide the document into text blocks; Based on the preset large model, each text block is processed, entities and relationships are extracted, and the entity set and relationship set of each text block are generated; Merge the entity sets and relationship sets of all text blocks to build a knowledge graph; Generate description information for each entity in the entity set based on entity attributes; Entities are regarded as nodes and relationships as edges. The constructed knowledge graph is represented as an undirected weighted graph. A preset community partitioning algorithm is used to construct a hierarchical community structure, and a community summary is generated for each community. Generate corresponding vector representations for all entities, text blocks to which the entities belong, and community summaries.

[0012] In some embodiments, entities are regarded as nodes and relationships are regarded as edges, the constructed knowledge graph is represented as an undirected weighted graph, a preset community partitioning algorithm is used to construct a hierarchical community structure, and a community summary is generated for each community, including: Taking entities as nodes and relationships as edges, the constructed knowledge graph is represented as an undirected weighted graph, where each node in the undirected weighted graph is regarded as an independent community, and the number of communities in the network is equal to the number of nodes; For each node i, move it to the community C to which the neighboring node belongs in turn, and calculate the change in the modularity gain of the entire graph network before and after moving node i ; Record the maximum modularity gain after moving node i to each neighbor node ,if , then assign node i to The largest community, otherwise it remains unchanged until all nodes cannot move; For each node i that moves to the community C to which the neighboring node belongs, it is regarded as a new node. The weight of the community node is expressed as the weight between nodes in the community, and the weight between communities is expressed as the weight of the edge between new community nodes to build a new upper network; Repeat moving each node i and building a new upper network until the modularity no longer increases; The entity and relationship descriptions in each community are filled into the preset prompt template respectively, and the preset big model is called to summarize the community summary of each community.

[0013] In some embodiments, the modularity gain varies , satisfying the formula: , in, , represents the total weight of all edges connecting node i in the network with nodes in community C; is the edge weight between nodes i and j. If there is no edge between two nodes, then ; represents the sum of the connection weights of node 𝑖 in its original community; represents the sum of the degrees of all nodes in community C; represents the sum of the weights of all the connecting edges of node i; the normalization factor 2m ensures that the modularity is between [−1,1].

[0014] In some embodiments, the new document is processed to obtain entities and relationships in the new document, and the knowledge graph is updated, the community is re-divided, and the vector representation is updated, including: Chunk new documents, extract entities and relations, and generate entity descriptions; Incorporate the entities and relationships extracted from the new document into the original knowledge graph to update the knowledge graph; Re-perform community division on the updated knowledge graph, generate community summaries for the newly added communities, and merge the new documents and their text blocks into the original document set and text block set; Generate new vector representations for newly added entities, text blocks, and community summaries for subsequent retrieval.

[0015] In some embodiments, entities and relationships extracted from new documents are incorporated into the originally constructed knowledge graph, including: For newly added entities, if the corresponding name already exists in the original entity set, the attributes of the corresponding entity will be merged; for numeric attributes with unique characteristics, the original attribute values ​​will be retained unchanged; and for string type attributes, the new and old attribute values ​​will be concatenated and updated; if the newly added entity does not exist in the original entity set, it will be directly added to the entity set; For the relationship extracted from the newly added document, if the two entities connected by it already exist in the original entity set, and the corresponding relationship itself also exists in the original relationship set, the descriptive attributes of the newly added relationship are spliced ​​into the corresponding attributes of the original relationship; if the entities connected by the corresponding relationship do not all exist in the original entity set, or the corresponding relationship does not exist in the original relationship set, it is directly added to the relationship set.

[0016] In some embodiments, clustering the entity list in the updated knowledge graph and determining the relevance score between the entity and the question includes: Cluster the entity list in the updated knowledge graph and obtain the center points of all clusters; Convert the problem q into a vector v q , T clusters whose matching distances meet the preset screening conditions; In T clusters, calculate each entity x and v q The distance between each entity and the question is obtained.

[0017] The present disclosure provides a multi-document intelligent question-answering device based on multi-agent collaboration, the device comprising: The graph construction module is used to segment the target document, extract entities and relationships, build a knowledge graph, generate entity descriptions, build a community hierarchy, and generate vector representations; The graph update module is used to process new documents, obtain entities and relationships in new documents, update the knowledge graph, re-divide communities, and update vector representations; A relevance score determination module, used to cluster the entity list in the updated knowledge graph and determine the relevance score between the entity and the question; A retrieval module, used to sort the entities based on the relevance scores, and retrieve the adjacent entities, relationships, communities and text blocks of the entities that meet the preset conditions; The output module is used to sort and deduplicate adjacent entities, relationships, communities, and text block collections, and generate corresponding answers based on the preset large model.

[0018] An embodiment of the present disclosure provides an electronic device, the device comprising at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute the above-mentioned multi-document intelligent question-answering method based on multi-agent collaboration.

[0019] An embodiment of the present disclosure provides a storage medium storing program instructions, which, when run, execute the above-mentioned multi-document intelligent question-answering method based on multi-agent collaboration.

[0020] The multi-document intelligent question answering method, apparatus, device, and storage medium based on multi-agent collaboration provided by the embodiments of the present disclosure can achieve the following technical effects: The present disclosure divides the target document into blocks, extracts entities and relationships, constructs a knowledge graph, generates entity descriptions, constructs a community hierarchy, and generates vector representations; processes new documents to obtain entities and relationships in the new documents, and updates the knowledge graph, re-divides communities, and updates vector representations; clusters the entity list in the updated knowledge graph, and determines the relevance scores between entities and questions to sort the entities, and retrieves adjacent entities, relationships, communities, and text block collections for entities that meet preset conditions, and generates corresponding answers based on a large model. It can efficiently, accurately, and fully extract key information related to the question from the document, guide the language model to generate high-quality answers, and improve the effectiveness of the question-answering system in detail question-answering.

[0021] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein: Figure 1 It is a flowchart of a multi-document intelligent question-answering method based on multi-agent collaboration provided by an embodiment of the present disclosure; Figure 2is a flowchart of another multi-document intelligent question-answering method based on multi-agent collaboration provided by an embodiment of the present disclosure; Figure 3 is a structural diagram of a multi-document intelligent question-answering device based on multi-agent collaboration provided by an embodiment of the present disclosure; Figure 4 It is a structural diagram of a multi-document intelligent question-answering device based on multi-agent collaboration provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0024] The terms "first", "second", etc. in the embodiments of the present disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0025] Unless otherwise stated, the term "plurality" means two or more.

[0026] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0027] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0028] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0029] In order to solve the above problems, the present disclosure provides a multi-document intelligent question answering method, apparatus, device and storage medium based on multi-agent collaboration.

[0030] The following is a description of the multi-document intelligent question answering method, apparatus, device and storage medium based on multi-agent collaboration provided by the embodiments of the present disclosure in conjunction with the accompanying drawings.

[0031] Figure 1It is a flow chart of a multi-document intelligent question-answering method based on multi-agent collaboration provided by an embodiment of the present disclosure.

[0032] Combination Figure 1 As shown, the multi-document intelligent question answering method based on multi-agent collaboration may include: S101, segmenting the target document, extracting entities and relationships, building a knowledge graph, generating entity descriptions, building a community hierarchy, and generating vector representations; S102, processing the new document to obtain entities and relationships in the new document, updating the knowledge graph, re-dividing the community, and updating the vector representation; S103, clustering the entity list in the updated knowledge graph and determining the relevance score between the entity and the question; S104, sorting the entities based on the relevance scores, and retrieving the adjacent entities, relationships, communities, and text block collections of the entities that meet the preset conditions; S105, sorting and deduplication of adjacent entities, relationships, communities and text block sets, and generating corresponding answers based on a preset large model.

[0033] In some embodiments, segmenting a target document, extracting entities and relationships, constructing a knowledge graph, generating entity descriptions, constructing a community hierarchy, and generating a vector representation include: Divide each document in the corresponding set of the target document into blocks according to the preset length, and divide the document into text blocks; Based on the preset large model, each text block is processed, entities and relationships are extracted, and the entity set and relationship set of each text block are generated; Merge the entity sets and relationship sets of all text blocks to build a knowledge graph; Generate description information for each entity in the entity set based on entity attributes; Entities are regarded as nodes and relationships as edges. The constructed knowledge graph is represented as an undirected weighted graph. A preset community partitioning algorithm is used to construct a hierarchical community structure, and a community summary is generated for each community. Generate corresponding vector representations for all entities, text blocks to which the entities belong, and community summaries.

[0034] In some embodiments, entities are regarded as nodes and relationships are regarded as edges, the constructed knowledge graph is represented as an undirected weighted graph, a preset community partitioning algorithm is used to construct a hierarchical community structure, and a community summary is generated for each community, including: Taking entities as nodes and relationships as edges, the constructed knowledge graph is represented as an undirected weighted graph, where each node in the undirected weighted graph is regarded as an independent community, and the number of communities in the network is equal to the number of nodes; For each node i, move it to the community C to which the neighboring node belongs in turn, and calculate the change in the modularity gain of the entire graph network before and after moving node i ; Record the maximum modularity gain after moving node i to each neighbor node ,if , then assign node i to The largest community, otherwise it remains unchanged until all nodes cannot move; For each node i that moves to the community C to which the neighboring node belongs, it is regarded as a new node. The weight of the community node is expressed as the weight between nodes in the community, and the weight between communities is expressed as the weight of the edge between new community nodes to build a new upper network; Repeat moving each node i and building a new upper network until the modularity no longer increases; The entity and relationship descriptions in each community are filled into the preset prompt template respectively, and the preset big model is called to summarize the community summary of each community.

[0035] In some embodiments, the above modularity gain changes , satisfying the formula: , in, , represents the total weight of all edges connecting node i in the network with nodes in community C; is the edge weight between nodes i and j. If there is no edge between two nodes, then ; represents the sum of the connection weights of node 𝑖 in its original community; represents the sum of the degrees of all nodes in community C; represents the sum of the weights of all the connecting edges of node i; the normalization factor 2m ensures that the modularity is between [−1,1].

[0036] In some embodiments, the new document is processed to obtain entities and relationships in the new document, and the knowledge graph is updated, the community is re-divided, and the vector representation is updated, including: Chunk new documents, extract entities and relations, and generate entity descriptions; Incorporate the entities and relationships extracted from the new document into the original knowledge graph to update the knowledge graph; Re-perform community division on the updated knowledge graph, generate community summaries for the newly added communities, and merge the new documents and their text blocks into the original document set and text block set; Generate new vector representations for newly added entities, text blocks, and community summaries for subsequent retrieval.

[0037] In some embodiments, entities and relationships extracted from new documents are incorporated into the originally constructed knowledge graph, including: For a newly added entity, if the corresponding name already exists in the original entity set, the attributes of the entity are merged; for numeric attributes with unique characteristics, the original attribute values ​​are retained unchanged; and for string type attributes, the new and old attribute values ​​are concatenated and updated; if the newly added entity does not exist in the original entity set, it is directly added to the entity set; For the relationship extracted from the newly added document, if the two entities connected by it already exist in the original entity set, and the relationship itself also exists in the original relationship set, the descriptive attributes of the newly added relationship are spliced ​​into the corresponding attributes of the original relationship; if the entities connected by the relationship do not all exist in the original entity set, or the relationship does not exist in the original relationship set, it is directly added to the relationship set.

[0038] In some embodiments, clustering the entity list in the updated knowledge graph and determining the relevance score between the entity and the question includes: Cluster the entity list in the updated knowledge graph and obtain the center points of all clusters; Convert the problem q into a vector v q , T clusters whose matching distances meet the preset screening conditions; In T clusters, calculate the relationship between each entity x and v q The distance between each entity and the question is obtained.

[0039] Figure 2 is a flowchart of another multi-document intelligent question answering method based on multi-agent collaboration provided by the present disclosure, combined with Figure 2 ,right Figure 1 The specific technical solution may include: 1. Graph Construction For a given document set D={d1,d2,...,d m}, use large models to build knowledge graphs (KG) from documents.

[0040] Specific practices may include: 1.1 Document segmentation. According to the set segment length size, that is, the preset length, the document d i Split, the words of the specified length will overlap between adjacent blocks, and the document d is obtained i Corresponding text block set TB i ={tb1,tb2,...,tb k}; That is to say, each document in the document set D can be divided into blocks according to the set block length, and the document can be divided into text blocks of reasonable length.

[0041] Input: long document d i , the maximum length of the target block size, the overlapping length of adjacent overlapping blocks overlap_length Output: text block set TB i split(d i ): words_list = tokenizer(d i ) words_length = len(word_list) TB i = [], start_index = 0 while start_index <words_length: end_index = min(start_index + size, n) TB i .append(word_list[start_index:end_index]) start_index = start_index + size - overlap_length return TB i TB i = split(d i ) Slicing method idea: First, you can use the tokenizer of the large model to process the document into a word list word_list, and then slice it by block starting from start_index through a loop, taking a sublist of length size each time and adding it to the text block set TB i After each slicing, update the starting index to start_index + size - overlap_length to ensure the correct overlap length between blocks. Repeat until all words are segmented, and finally get the text block set TB i .

[0042] 1.2 Extract entities and relations. i Fill in the prompt template Prompt extract , using the large model to extract each block of text separately from tb iExtract the entity set E from i ={ ei1, ei2, ..., ei n} and its attribute information, and identify E i The relationship between entities in R i ={ri1,ri2,…,ri n | E i}.

[0043] 1.3 Build a knowledge graph. The entities and relationships extracted from multiple text blocks are fused and represented as entity sets E and relationship sets R as the basic units of the knowledge graph KG. During the fusion process, if there are entities with the same name in multiple text blocks, they will be merged into the same entity. Among the attributes of these entities, the numerical attributes remain unique, and the text attributes are merged into the concatenation of the attributes of multiple entities with the same name.

[0044] 1.4 Generate entity descriptions. Based on the attributes of each entity, use the big model to summarize it and generate a summary description of each entity.

[0045] 1.5 Perform community division on the constructed graph. In other words, treat entities as nodes, relationships as edges, and represent the knowledge graph KG as an undirected weighted graph. Use the community division algorithm to build a hierarchical community structure, and use the big model to generate a community summary for each community. The division method is as follows: 1.5.1 Initialization: Taking entities as nodes and relationships as edges, KG is represented as an undirected weighted graph. At this time, each node in the graph is regarded as an independent community, and the number of communities in the network is equal to the number of nodes.

[0046] 1.5.2 Local Movement: For each node i, try to move it to the community C to which the neighboring node belongs, and calculate the change in the modularity gain of the entire graph network before and after moving node i : , in, , represents the total weight of all edges connecting node i in the network with nodes in community C; is the edge weight between nodes i and j. If there is no edge between two nodes, then ; represents the sum of the connection weights of node 𝑖 in its original community; represents the sum of the degrees of all nodes in community C; represents the sum of the weights of all the connecting edges of node i; the normalization factor 2m ensures that the modularity is between [−1,1].

[0047] Record the maximum modularity gain after moving node i to each neighbor node ,if , then assign node i to The largest community, otherwise remains the same.

[0048] When all nodes cannot be moved, proceed to the next step.

[0049] 1.5.3 Treat each community obtained in step 1.5.2 as a new node. The weight of the community node is expressed as the weight between nodes in the community. The inter-community weight is the weight of the edge between the new community nodes, thereby constructing a new upper-layer network.

[0050] 1.5.4 Repeat steps 1.5.2 and 1.5.3 above until the modularity stops increasing.

[0051] 1.5.5 Fill the entity and relationship descriptions in each community into the prompt template respectively, and call the big model to summarize the community summary of each community.

[0052] 1.6 Generate vector representation. All entities, text blocks where entities are located, and community summaries of communities where entities are located are obtained by calling the big model for subsequent retrieval and matching.

[0053] 2. Map update For the newly added documents in D, the following strategy is used to update the document content to the knowledge graph: 2.1 Process new documents. Perform the same operations from 1.1 to 1.4 in graph construction; 2.2 Update the knowledge graph. Compare the entities extracted from the newly added documents with the original entities, merge the same entities and adjacency relationships, and update the graph. The specific update ideas are as follows: 2.2.1 Entity Update: 2.2.1.1 Existing entities: For newly added entities, if their names already exist in the original entity set, the attributes of the entity are merged. For numeric attributes with unique characteristics, the original attribute values ​​are retained unchanged. For string type attributes, the new and old attribute values ​​are concatenated and updated.

[0054] 2.2.1.2 Adding new entities: If the new entity does not exist in the original entity set, it will be added to the entity set directly.

[0055] 2.2.2 Relationship Update: 2.2.2.1 Existing relations: For relations extracted from newly added documents, if both entities connected by them already exist in the original entity set, and the relation itself also exists in the original relation set, the descriptive attributes of the newly added relation are spliced ​​into the corresponding attributes of the original relation.

[0056] 2.2.2.2 Adding a new relationship: If not all entities connected by the relationship exist in the original entity set, or the relationship does not exist in the original relationship set, add it directly to the relationship set.

[0057] 2.3 Re-divide the community. For the updated entities and relationships, perform the operation of step 1.5 in graph construction again based on the original community, divide the new graph into communities, and update the communities and community summaries. Then add new documents and text blocks to the original document set D and text block set TB. That is, re-perform community division on the updated knowledge graph KG and generate community summaries for the newly added communities. At the same time, merge the new documents and their text blocks into the original document set and text block set.

[0058] 2.4 Update vector representation. Call the big model to generate vector representations for updated entities, text blocks, and community summaries. That is, generate new vector representations for newly added entities, text blocks, and community summaries to ensure that the latest information can be used for subsequent retrieval.

[0059] 3. Retrieve the list of entities E related to the question according to question q q , calculate the similarity between q and all question-related entities.

[0060] Specific practices may include: 3.1 Entity list clustering: Use the K-Means mean clustering algorithm to divide all entities into K clusters and obtain the corresponding K cluster centers μ1, μ2, …, μ K . Calculation method: Calculate the Euclidean distance from each entity ei to the cluster center and assign each entity ei to the nearest cluster: ,in, represents the jth cluster center; 3.2 Input question q into the large model to obtain the question vector vq, and match it with the nearest Top-T clusters: , 3.3 In T clusters, calculate the distance between each entity x and vq in each cluster: , represents the semantic relevance score between the entity and the question.

[0061] 4. Keep the Top-K high-scoring entities: According to the relevance scores of entities in multiple clusters calculated in step 3, sort and filter the entity list Eq. Set the threshold k and retain the top-k entities E with the highest scores. q k ={e1,e2,…,e k}; 5. Retrieve entity related communities: For the entity set E related to question q obtained in step 4 q k , search each entity through the knowledge graph, obtain adjacent entities, related relations and communities to which they belong, and obtain the set of text blocks to which each entity belongs to obtain information closely related to the problem.

[0062] 6. Remove redundant elements from the community: According to the entity set E in step 4 q k According to the entity order in , the adjacent entities, relations, communities, corresponding community summaries, and text block sets obtained in step 5 are sorted, and duplicate text blocks are removed. Then, according to the maximum total length limit, text blocks that exceed the length are removed.

[0063] That is, remove redundant elements in the community. Sort and remove duplicates of adjacent entities, relationships, communities, and text block collections. Remove text blocks that exceed the maximum total length limit and retain the most important content.

[0064] 7. The big model generates the final answer: After the sorted entities, relations, community summaries and text blocks are formatted, they are filled into the preset prompt template. The preset Large Language Model (LLM) is used to generate the final answer a according to the prompt template. That is, the entities, relations, communities and text blocks selected in step 6 are formatted and filled into the prompt template, and the final answer output is generated using the large model.

[0065] In the present disclosure, for fine-grained knowledge graph construction, entities and relationships are extracted from multiple documents to construct a knowledge graph that supports word-level detailed information to enhance the system's ability to provide detailed answers.

[0066] In view of the dynamic update of the knowledge graph, the dynamic update of the document corpus is supported to maintain the real-time retrieval and scalability of the system.

[0067] By combining cluster retrieval with vector distance, the retrieval efficiency and information sufficiency of key information can be improved, thereby significantly improving the quality of question and answer.

[0068] The multi-document intelligent question-answering method based on multi-agent collaboration provided by the present disclosure improves the question-answering effect by constructing a knowledge graph and retrieving two agents in fine-grained context. First, a fixed-length segmentation strategy is adopted to divide the document into several text blocks, and entity and relationship extraction operations are performed on each text block. Then, the extraction results of all text blocks are merged to construct a knowledge graph, so as to accurately and fully model the key information in the document as a graph network. Next, vector clustering technology is used to efficiently retrieve entities related to the question, retain the most relevant Top-K entities, and retrieve the adjacent entities, relationships, communities and text blocks of these entities respectively. After sorting and screening, the relevant information is formatted, and finally a large language model is used to generate answers based on this information. In this process, the retrieval efficiency and comprehensiveness of the relevant information are guaranteed by word-level retrieval. The present disclosure can efficiently, accurately and fully extract key information related to the question from long documents and multiple documents, and guide the language model to rely on this information to generate high-quality answers, thereby significantly improving the effect of the intelligent question-answering system in detail question-answering.

[0069] In addition, by constructing a knowledge graph, we can effectively establish the association between different knowledge details within and between documents, and use the graph structure to overcome the scalability bottleneck of the question-answering system, thereby significantly improving the accuracy of multi-document intelligent question-answering. This is to solve the problem that the existing retrieval-augmented generation (RAG)-based intelligent question-answering system has difficulty in providing fine-grained key information when processing long documents and multiple documents, and cannot ensure that the answers are generated based on sufficient retrieval information.

[0070] The present invention improves the performance of the multi-document intelligent question answering system through multi-agent collaboration. Key features include the construction and dynamic update of knowledge graphs, and support for fine-grained contextual retrieval and cluster retrieval combined with vector matching. The system generates answers by retrieving and filtering entities, relationships, and text blocks, combined with a large model, to ensure efficient and accurate answers to questions, overcoming the limitations of traditional RAG methods.

[0071] This paper improves the shortcomings of traditional intelligent question-answering systems, proposes a fine-grained knowledge graph construction and dynamic update mechanism, and enhances the relevance of knowledge inside and outside documents. By combining clustering and vector matching, it improves the efficiency and accuracy of information retrieval, solves the problem of large-scale document and information dispersion, and improves the overall accuracy and scalability of the question-answering system.

[0072] and Figure 1 Corresponding to the multi-document intelligent question answering method based on multi-agent collaboration, the present disclosure also provides a multi-document intelligent question answering device based on multi-agent collaboration, such as Figure 3 As shown, the device may specifically include: A graph construction module 301, which is used to segment the target document, extract entities and relationships, construct a knowledge graph, generate entity descriptions, build a community hierarchy, and generate a vector representation; A graph updating module 302 is used to process the new document, obtain entities and relationships in the new document, update the knowledge graph, re-divide the community, and update the vector representation; A relevance score determination module 303 is used to cluster the entity list in the updated knowledge graph and determine the relevance score between the entity and the question; The retrieval module 304 is used to sort the entities based on the relevance scores, and retrieve the adjacent entities, relationships, communities and text blocks of the entities that meet the preset conditions, and obtain the information that meets the preset close conditions of relevance to the question; The output module 305 is used to sort and deduplicate adjacent entities, relationships, communities and text block sets, and generate corresponding answers based on a preset large model.

[0073] In some embodiments, segmenting a target document, extracting entities and relationships, constructing a knowledge graph, generating entity descriptions, constructing a community hierarchy, and generating a vector representation include: Divide each document in the corresponding set of the target document into blocks according to the preset length, and divide the document into text blocks; Based on the preset large model, each text block is processed, entities and relationships are extracted, and the entity set and relationship set of each text block are generated; Merge the entity sets and relationship sets of all text blocks to build a knowledge graph; Generate description information for each entity in the entity set based on entity attributes; Entities are regarded as nodes and relationships as edges. The constructed knowledge graph is represented as an undirected weighted graph. A preset community partitioning algorithm is used to construct a hierarchical community structure, and a community summary is generated for each community. Generate corresponding vector representations for all entities, text blocks to which the entities belong, and community summaries.

[0074] In some embodiments, entities are regarded as nodes and relationships are regarded as edges, the constructed knowledge graph is represented as an undirected weighted graph, a preset community partitioning algorithm is used to construct a hierarchical community structure, and a community summary is generated for each community, including: Taking entities as nodes and relationships as edges, the constructed knowledge graph is represented as an undirected weighted graph, where each node in the undirected weighted graph is regarded as an independent community, and the number of communities in the network is equal to the number of nodes; For each node i, move it to the community C to which the neighboring node belongs in turn, and calculate the change in the modularity gain of the entire graph network before and after moving node i ; Record the maximum modularity gain after moving node i to each neighbor node ,if , then assign node i to The largest community, otherwise it remains unchanged until all nodes cannot move; For each node i that moves to the community C to which the neighboring node belongs, it is regarded as a new node. The weight of the community node is expressed as the weight between nodes in the community, and the weight between communities is expressed as the weight of the edge between new community nodes to build a new upper network; Repeat moving each node i and building a new upper network until the modularity no longer increases; The entity and relationship descriptions in each community are filled into the preset prompt template respectively, and the preset big model is called to summarize the community summary of each community.

[0075] In some embodiments, the modularity gain varies , satisfying the formula: , in, , represents the total weight of all edges connecting node i in the network with nodes in community C; is the edge weight between nodes i and j. If there is no edge between two nodes, then ; represents the sum of the connection weights of node 𝑖 in its original community; represents the sum of the degrees of all nodes in community C; represents the sum of the weights of all the connecting edges of node i; the normalization factor 2m ensures that the modularity is between [−1,1].

[0076] In some embodiments, the new document is processed to obtain entities and relationships in the new document, and the knowledge graph is updated, the community is re-divided, and the vector representation is updated, including: Chunk new documents, extract entities and relations, and generate entity descriptions; Incorporate the entities and relationships extracted from the new document into the original knowledge graph to update the knowledge graph; Re-perform community division on the updated knowledge graph, generate community summaries for the newly added communities, and merge the new documents and their text blocks into the original document set and text block set; Generate new vector representations for newly added entities, text blocks, and community summaries for subsequent retrieval.

[0077] In some embodiments, entities and relationships extracted from new documents are incorporated into the originally constructed knowledge graph, including: For a newly added entity, if the corresponding name already exists in the original entity set, the attributes of the entity are merged; for numeric attributes with unique characteristics, the original attribute values ​​are retained unchanged; and for string type attributes, the new and old attribute values ​​are concatenated and updated; if the newly added entity does not exist in the original entity set, it is directly added to the entity set; For the relationship extracted from the newly added document, if the two entities connected by it already exist in the original entity set, and the relationship itself also exists in the original relationship set, the descriptive attributes of the newly added relationship are spliced ​​into the corresponding attributes of the original relationship; if the entities connected by the relationship do not all exist in the original entity set, or the relationship does not exist in the original relationship set, it is directly added to the relationship set.

[0078] In some embodiments, clustering the entity list in the updated knowledge graph and determining the relevance score between the entity and the question includes: Cluster the entity list in the updated knowledge graph and obtain the center points of all clusters; Convert the problem q into a vector v q , T clusters whose matching distances meet the preset screening conditions; In T clusters, calculate each entity x and v q The distance between each entity and the question is obtained.

[0079] Combination Figure 4 As shown, the embodiment of the present disclosure also provides a multi-document intelligent question-answering device 400 based on multi-agent collaboration, including a processor 404 and a memory 401. Optionally, the system may also include a communication interface 402 and a bus 403. Among them, the processor 404, the communication interface 402, and the memory 401 can communicate with each other through the bus 403. The communication interface 402 can be used for information transmission. The processor 404 can call the logical instructions in the memory 401 to execute the multi-document intelligent question-answering method based on multi-agent collaboration of the above embodiment.

[0080] In addition, the logic instructions in the memory 401 described above may be implemented in the form of software functional units and when sold or used as independent products, may be stored in a computer-readable storage medium.

[0081] The memory 401 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 404 executes the function application and data processing by running the program instructions / modules stored in the memory 401, that is, the multi-document intelligent question answering method based on multi-agent collaboration in the above embodiment is implemented.

[0082] The memory 401 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 401 may include a high-speed random access memory and may also include a non-volatile memory.

[0083] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as a multi-document intelligent question-answering method based on multi-agent collaboration.

[0084] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0085] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0086] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent possible changes only. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. As used in the description of the embodiments, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or including (comprising) refer to the existence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or these groups. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same or similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0087] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and simplicity of description, the specific working process of the systems, devices and units described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0088] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0089] The flowchart and block diagram in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

[0090] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0091] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0092] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0093] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0094] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0095] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0096] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of this disclosure can be achieved, and this document is not limited here.

[0097] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A multi-document intelligent question answering method based on multi-agent collaboration, characterized in that: The method comprises: Segment the target document, extract entities and relations, build a knowledge graph, generate entity descriptions, build a community hierarchy, and generate vector representations; Process the new document to obtain the entities and relationships in the new document, update the knowledge graph, re-divide the community, and update the vector representation; Cluster the list of entities in the updated knowledge graph and determine the relevance score of the entity to the question; Sorting the entities based on the relevance scores, and retrieving the adjacent entities, relationships, communities, and text block collections of the entities that meet the preset conditions; Sort and deduplicate adjacent entities, relationships, communities, and text block collections, and generate corresponding answers based on the preset large model.

2. The method according to claim 1, characterized in that The target document is segmented, entities and relationships are extracted, knowledge graphs are constructed, entity descriptions are generated, community hierarchies are constructed, and vector representations are generated, including: Divide each document in the corresponding set of the target document into blocks according to the preset length, and divide the document into text blocks; Based on the preset large model, each text block is processed, entities and relationships are extracted, and the entity set and relationship set of each text block are generated; Merge the entity sets and relationship sets of all text blocks to build a knowledge graph; Generate description information for each entity in the entity set based on entity attributes; Entities are regarded as nodes and relationships as edges. The constructed knowledge graph is represented as an undirected weighted graph. A preset community partitioning algorithm is used to construct a hierarchical community structure, and a community summary is generated for each community. Generate corresponding vector representations for all entities, text blocks to which the entities belong, and community summaries.

3. The method according to claim 2, characterized in that The entities are regarded as nodes and the relationships are regarded as edges, and the constructed knowledge graph is represented as an undirected weighted graph. A preset community partitioning algorithm is used to construct a hierarchical community structure, and a community summary is generated for each community, including: Taking entities as nodes and relationships as edges, the constructed knowledge graph is represented as an undirected weighted graph, where each node in the undirected weighted graph is regarded as an independent community, and the number of communities in the network is equal to the number of nodes; For each node i, move it to the community C to which the neighboring node belongs in turn, and calculate the change in the modularity gain of the entire graph network before and after moving node i ; Record the maximum modularity gain after moving node i to each neighbor node ,if , then assign node i to The largest community, otherwise it remains unchanged until all nodes cannot move; For each node i that moves to the community C to which the neighboring node belongs, it is regarded as a new node. The weight of the community node is expressed as the weight between nodes in the community, and the weight between communities is expressed as the weight of the edge between new community nodes to build a new upper network; Repeat moving each node i and building a new upper network until the modularity no longer increases; The entity and relationship descriptions in each community are filled into the preset prompt template respectively, and the preset big model is called to summarize the community summary of each community.

4. The method according to claim 3, characterized in that The modularity gain changes , satisfying the formula: ， in, , represents the total weight of all edges connecting node i in the network with nodes in community C; is the edge weight between nodes i and j. If there is no edge between two nodes, then ; represents the sum of the connection weights of node 𝑖 in its original community; represents the sum of the degrees of all nodes in community C; represents the sum of the weights of all the connecting edges of node i; the normalization factor 2m ensures that the modularity is between [−1,1].

5. The method according to claim 1, characterized in that The new document is processed to obtain entities and relationships in the new document, and the knowledge graph is updated, the community is re-divided, and the vector representation is updated, including: Chunk new documents, extract entities and relations, and generate entity descriptions; Incorporate the entities and relationships extracted from the new document into the original knowledge graph to update the knowledge graph; Re-perform community division on the updated knowledge graph, generate community summaries for the newly added communities, and merge the new documents and their text blocks into the original document set and text block set; Generate new vector representations for newly added entities, text blocks, and community summaries for subsequent retrieval.

6. The method according to claim 5, characterized in that The process of incorporating the entities and relationships extracted from the new document into the originally constructed knowledge graph includes: For newly added entities, if the corresponding name already exists in the original entity set, the attributes of the corresponding entity will be merged; for numeric attributes with unique characteristics, the original attribute values ​​will be retained unchanged; and for string type attributes, the new and old attribute values ​​will be concatenated and updated; if the newly added entity does not exist in the original entity set, it will be directly added to the entity set; For the relationship extracted from the newly added document, if the two entities connected by it already exist in the original entity set, and the corresponding relationship itself also exists in the original relationship set, the descriptive attributes of the newly added relationship are spliced ​​into the corresponding attributes of the original relationship; if the entities connected by the corresponding relationship do not all exist in the original entity set, or the corresponding relationship does not exist in the original relationship set, it is directly added to the relationship set.

7. The method according to claim 1, characterized in that The step of clustering the entity list in the updated knowledge graph and determining the relevance score between the entity and the question includes: Cluster the entity list in the updated knowledge graph and obtain the center points of all clusters; Convert the problem q into a vector v q , T clusters whose matching distances meet the preset screening conditions; In T clusters, calculate the relationship between each entity x and v q The distance between each entity and the question is obtained.

8. A multi-document intelligent question-answering device based on multi-agent collaboration, characterized in that: The device comprises: The graph construction module is used to segment the target document, extract entities and relationships, build a knowledge graph, generate entity descriptions, build a community hierarchy, and generate vector representations; The graph update module is used to process new documents, obtain entities and relationships in new documents, update the knowledge graph, re-divide communities, and update vector representations; A relevance score determination module, used to cluster the entity list in the updated knowledge graph and determine the relevance score between the entity and the question; A retrieval module, used to sort the entities based on the relevance scores, and retrieve the adjacent entities, relationships, communities and text blocks of the entities that meet the preset conditions; The output module is used to sort and deduplicate adjacent entities, relationships, communities, and text block collections, and generate corresponding answers based on the preset large model.

9. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; It is characterized in that the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1 to 7.

Images