Tree diagram and knowledge graph retrieval enhanced large model inference method and system

Abstract

The application discloses a tree diagram and knowledge graph retrieval enhanced large model reasoning method and system, which comprises the following steps: cutting a document into text blocks, identifying entities, relationships and factual descriptions of the text blocks to form a knowledge graph of the text blocks; constructing a factual summary of the document based on the knowledge graph of the text blocks, recursively clustering all documents using a Gaussian mixture model to generate a similarity tree structure diagram; obtaining a user question, selecting a reasoning path from the similarity tree structure diagram through a large language model debate iteration method, obtaining a corresponding document cluster, then pruning the cluster based on the user question and the confidence score of the document, and selecting a plurality of text blocks from the pruned document, and finally generating a reasoning result based on the knowledge graph of the selected text blocks. The application relates to the technical field of natural language processing, and can realize retrieval enhancement based on a tree diagram and a knowledge graph, accurately query relevant information when a large model answers a complex multi-hop question, and effectively guarantee reasoning accuracy.

Description

Technical Field

[0001] The present invention relates to a large model reasoning method and system based on tree diagram and knowledge graph retrieval enhancement, and relates to the field of communications. Background Art

[0002] The development of large language models has brought many opportunities and new directions to industry development. However, because large language models often rely on internal knowledge, they cannot guarantee the most accurate and relevant answers to specific questions when handling tasks that require a large amount of facts and external knowledge. Furthermore, they are prone to hallucinations and reasoning errors when information is missing. Therefore, retrieval enhancement has emerged. Retrieval enhancement combines the advantages of information retrieval and generation, overcoming the limitations of large language models in handling complex questions, acquiring external knowledge, and ensuring accurate answers. It is suitable for tasks that require a large amount of external knowledge and reasoning capabilities, such as question-answering systems, dialogue generation, and multi-step reasoning tasks.

[0003] Traditional retrieval enhancement retrieves relevant content from large text blocks. There is no clear structure between text blocks, and the information is scattered. It is impossible to obtain the relationship between entities or text blocks during retrieval. Therefore, retrieval enhancement based on knowledge graphs is proposed. Knowledge graphs can provide structured data, allowing large language models to more clearly capture the relationship and context between entities, reducing the ambiguity of information. At the same time, when the question involves multi-step reasoning, complex background knowledge, or multi-level relationships between entities, the ability of text block-based retrieval enhancement may be limited. Retrieval enhancement based on knowledge graphs can enable the model to perform more complex reasoning. The structured data in the knowledge graph can help the model perform multi-hop reasoning when answering questions.

[0004] Knowledge graphs focus on relationships between entities but ignore the hierarchical structure of information, making it difficult to answer questions requiring a high level of detail. Tree-structured graphs, on the other hand, organize information hierarchically, making relationships clearer. For example, in a tree-structured graph, a parent node may represent a preliminary answer to a question, while child nodes contain more detailed information. This hierarchical structure helps models better understand and process complex queries.

[0005] Therefore, how to achieve retrieval enhancement based on tree diagrams and knowledge graphs, accurately query relevant information when large language models answer complex multi-hop questions, and effectively ensure the accuracy of large language model reasoning has become a technical issue that technical personnel have focused on. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a large model reasoning method and system based on tree diagram and knowledge graph retrieval enhancement, which can realize retrieval enhancement based on tree diagram and knowledge graph, and accurately query relevant information when the large language model answers complex multi-hop questions, thereby effectively ensuring the accuracy of large language model reasoning.

[0007] To achieve the above objectives, the present invention provides a large model reasoning method based on tree diagram and knowledge graph retrieval enhancement, including:

[0008] Step 1: Obtain several documents and cut each document into multiple text blocks. Then identify the entities, relationships, and factual descriptions of each text block to form a knowledge graph for each text block.

[0009] Step 2: Build a factual overview of the document based on the knowledge graph of the text block, and then use the Gaussian mixture model to recursively cluster all documents to generate a similarity tree structure diagram consisting of all documents;

[0010] Step 3: Obtain the user question and select an inference path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document cluster. Then, based on the confidence scores of the user question and the document, prune the document cluster and select several text blocks from each pruned document. Finally, based on the knowledge graph of the selected text blocks, generate the inference result of the user question.

[0011] In step 1, the entities, relationships, and factual descriptions of each text block are identified to form a knowledge graph for each text block, which further includes:

[0012] Construct entity extraction templates, relationship construction templates, and factual description templates, then insert the content of the text block into the template to obtain text prompts, and then input the text prompts into the large language model to obtain the entity, relationship, and factual descriptions of each text block.

[0013] In step 3, the user question is obtained, and through the large language model debate iterative method, an inference path is selected from the similarity tree structure diagram to obtain the corresponding document cluster, which further includes:

[0014] Step A1: Assign three identities to the large language model, each designed to perform different tasks:

[0015] (1) Debater: Based on the given user question, the debater selects a path on the similarity tree and explains the reasoning. Based on the feedback from critics and other debaters, the debater adjusts or continues to defend the reasoning.

[0016] (2) Critic: Analyze the flaws in each debater's reasons for choosing the path;

[0017] (3) Summarizer: Determine whether the debaters reach a consensus on the final result, summarize the final selected document clusters, and give the final reasoning results;

[0018] Step A2: Use different large language models as debaters. Given a user question and a similarity tree structure, the debaters propose different path choices and reasons for their choices.

[0019] Step A3: Each debater refers to the other debaters' views, defends his or her own argument, and points out the flaws of other debaters;

[0020] Step A4: The critics look for loopholes and errors in each debater's defense and convey them to the debaters;

[0021] Step A5: After receiving the errors pointed out by other debaters and critics, the debater will refer to the other debaters' opinions and revise his or her own views.

[0022] Step A6: After a round of debate, the summarizer determines whether a consensus is reached on the debate results and whether further debate is needed. If further debate is needed, the process returns to step A3 to continue the next round of debate. If consensus is reached, the summarizer summarizes the final reasoning results and selects a reasoning path from the similarity tree structure diagram based on the user's question.

[0023] In order to achieve the above object, the present invention also provides a large model reasoning system based on tree diagram and knowledge graph retrieval enhancement, the system is used to execute the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement, the system includes:

[0024] A knowledge graph construction device is used to obtain a number of documents, cut each document into multiple text blocks, and then identify the entities, relationships and factual descriptions of each text block to construct a knowledge graph for each text block;

[0025] A tree graph construction device is used to construct a factual overview of the document based on the knowledge graph of the text block, and then recursively cluster all documents using a Gaussian mixture model to generate a similarity tree structure graph consisting of all documents;

[0026] The inference device is used to obtain user questions and select an inference path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document cluster. Then, based on the confidence scores of the user questions and documents, the document clusters are pruned, and several text blocks are selected from each pruned document. Finally, based on the knowledge graph of the selected text blocks, the inference result of the user question is generated.

[0027] In order to achieve the above object, the present invention further provides a computing device, comprising:

[0028] memory and processor;

[0029] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement are implemented.

[0030] In order to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement.

[0031] Compared with the existing technology, the beneficial effects of the present invention are: the present invention realizes retrieval enhancement based on tree structure and knowledge graph. The knowledge graph focuses on the relationship between entities. The retrieval enhancement based on the knowledge graph can enable the model to perform more complex reasoning. The structured data in the knowledge graph can help the model perform multi-hop reasoning when answering questions, while the tree structure focuses on the hierarchical structure of information. The retrieval enhancement based on the tree structure can handle more complex reasoning paths, helping the large language model to refine the problem layer by layer and gradually go deeper into each branch. In this way, the knowledge graph is applied to the detailed information inside the text block, and the tree structure is applied to the external clustering information of the document. It can combine the advantages of both and enhance the processing of information; in the retrieval of the tree structure, the current technical solutions often use large language models to iterate and select a reasoning path, but when the problem spans multiple research objects, it is difficult to ensure the accuracy of the reasoning path. Therefore, the present invention also proposes a method of multiple debates-pruning optimization for path retrieval, using multiple large language models in an iterative process of debaters proposing arguments-debators raising questions-debators defending-evaluators raising questions-debators modifying arguments, and finally using summarizers to draw conclusions. The final reasoning conclusion can effectively improve the accuracy of path selection in this way; in cluster screening, traditional technical solutions often use small models for evaluation, but because the training of small models depends on the data set, it is difficult to achieve generalization. Therefore, the present invention also proposes to use a large language model to generate confidence scores, and perform pruning optimization through thresholds. The confidence score is calculated through the credibility of each token, the similarity with the question, the similarity with the keyword of the factual summary of the document, and the stop word score. When the confidence score of any token in the document reasoning sequence is greater than the threshold, the document is considered relevant and retained, otherwise it is regarded as irrelevant and pruned and deleted; current retrieval enhancement often uses similarity scores as a way to evaluate relevance, but ignores the influence of the structure of the text block in the document, and it is difficult to answer questions with high summary or conceptual requirements. The present invention also comprehensively considers the structural characteristics of the text block (the relationship with other text blocks), the relationship between the text block and the question, and the relationship between the text block and the question keywords, and uses a large language model to design parameters according to the characteristics of the question to iteratively generate the relevant scores of the text blocks, effectively ensuring the quality of the scores and improving the retrieval recall rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of a large model reasoning method based on tree diagram and knowledge graph retrieval enhancement shown in an exemplary embodiment of the present invention.

[0033] Figure 2 FIG. 4 is a schematic diagram of a similarity tree structure diagram according to an exemplary embodiment of the present invention.

[0034] Figure 3It is a structure schematic diagram of a large model reasoning system based on tree graph and knowledge graph retrieval enhancement shown by an exemplary embodiment of the present application.

[0035] Figure 4 It is a structure schematic diagram of a computer device shown by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings.

[0037] As shown in Figure 1 The present application is a large model reasoning method based on tree graph and knowledge graph retrieval enhancement, which comprises:

[0038] Step one, obtain a plurality of documents, and cut each document into a plurality of text blocks, then identify the entity, relationship and factual description of each text block to constitute the knowledge graph of each text block;

[0039] Step two, based on the knowledge graph of the text block, build a factual summary of the document, then use a Gaussian mixture model to recursively cluster all documents, thereby generating a similarity tree structure graph composed of all documents;

[0040] Step three, obtain a user question, select a reasoning path from the similarity tree structure graph by a large language model debate iteration method to obtain a corresponding document cluster, then prune the document cluster based on the user question and the confidence score of the document, and select a plurality of text blocks from each document after pruning, and finally generate a reasoning result of the user question based on the knowledge graph of the selected text blocks.

[0041] In step one, identifying the entity, relationship and factual description of each text block to constitute the knowledge graph of each text block can further comprise:

[0042] Building an entity extraction template, a relationship building template and a factual description template, then inserting the content of the text block into the template to obtain a text prompt, and then inputting the text prompt into a large language model to obtain the entity, relationship and factual description of each text block.

[0043] The entity extraction template can be as follows:

[0044] "Goal-

[0045] Identify all entities of these types from the text according to the provided text and entity type list.

[0046] Steps-

[0047] 1. Output |START|, indicating the start of entity extraction.

[0048] 2. Identify all entities. For each identified entity, extract the following information:

[0049] -entity_name:Entity name

[0050] -entity_type: One of the following types: [{entity_types}]

[0051] -entity_description: A comprehensive description of the entity's attributes and activities

[0052] Format each entity as

[0053] ("entity"{tuple_delimiter}<entity_name> {tuple_delimiter}<entity_type> {tuple_delimiter}<entity_description> )

[0054] 3. Output all entities identified in step 2 as a single list. Use **{record_delimiter}** as the list delimiter.

[0055] 4. When completed, output {completion_delimiter}

[0056] ######################

[0057] -Real Data-

[0058] ######################

[0059] Entity_types:{entity_types}

[0060] Text:{input_text}

[0061] ######################

[0062] Output:”

[0063] Specific examples are as follows:

[0064] input_text: The organization plans to hold meetings on Monday and Thursday, and the organization plans to release its latest policy decisions on Thursday at 1:30 pm, followed by a press conference, and the organization's chairman will answer questions from reporters. Investors expect the Market Strategy Committee to maintain the benchmark interest rate at 3.5% to 3.75%.

[0065] entity_types: ORGANIZATION, PERSON

[0066] Extraction results:

[0067] ("entity"{tuple_delimiter} The organization

[0068] {tuple_delimiter}ORGANIZATION{tuple_delimiter} The organization is the Federal Reserve Board, which will set interest rates on Monday and Thursday)

[0069] ("entity"{tuple_delimiter}

[0070] {tuple_delimiter}PERSON{tuple_delimiter} The chairman of the organization)

[0071] The relationship construction template can be as follows:

[0072] “-Goal-

[0073] According to the provided text and entity list, identify the relationships between all these entities in the text.

[0074] -Steps-

[0075] 1. Output |START|, indicating the start of relationship construction.

[0076] 2. Identify the relationship between entities, extracting the following information:

[0077] -source_entity: Source entity name

[0078] -target_entity: Target entity name

[0079] -relationship_description: Explain the reason why the source entity and the target entity are related to each other

[0080] -relationship_strength: Numerical score of the strength of the relationship between the source entity and the target entity

[0081] Format each relationship as

[0082] ("relationship"{tuple_delimiter}<source_entity> {tuple_delimiter}<target_entity> {tuple_delimiter}<relationship_description> {tuple_delimiter}<relationship_strength> )

[0083] 3. Output all the relationships identified in step 2 as a single list.

[0084] **{record_delimiter}** serves as the list delimiter.

[0085] 4. When completed, output {completion_delimiter}

[0086] ######################

[0087] -Real Data-

[0088] ######################

[0089] Entity:{entity}

[0090] Text:{input_text}

[0091] ######################

[0092] Output:”

[0093] Specific examples are as follows:

[0094] The agency plans to hold meetings on Monday and Thursday. It plans to release its latest policy decision at 1:30 p.m. Thursday, followed by a press conference where the chairman will answer questions from reporters. Investors expect the market strategy committee to maintain the benchmark interest rate within a range of 3.5% to 3.75%.

[0095] entity:**Organization,**

[0096] Build results:

[0097] ("relationship"{tuple_delimiter}**{tuple_delimiter}**institution{tuple_delimiter}**is**the**institution's president, who will answer questions{tuple_delimiter}9 at the press conference)

[0098] A factual description template might look like this:

[0099] “Your task is to construct a factual description in text format based on the provided entities and the relationships between them.

[0100] Entity:{entity}

[0101] Relationship:{relationship}

[0102] Output:”

[0103] Specific examples are as follows:

[0104] Entity: **, ** is the chairman of the ** institution; ** institution, ** institution is the ** Federal Reserve, which will set interest rates on Mondays and Thursdays.

[0105] relationship: ** is the chairman of the ** agency and will answer questions at the press conference.

[0106] Build results:

[0107] He is the chairman of the Federal Reserve Board.

[0108] The knowledge graph can be saved in graphml format and the factual description can be saved in parquet format.

[0109] Figure 2 FIG. 1 shows a schematic diagram of an embodiment of a similarity tree structure diagram. Figure 2 As shown in the figure, the similarity tree structure diagram takes all documents as leaf nodes, divides all documents into different document clusters through the factual overview of the documents, and then recursively calculates until the root node. In this way, through clustering recursion, a hierarchical similarity tree structure diagram is generated. Figure 2 It can be seen that document 1 and document 3 belong to one document cluster, ..., document 2 and document n belong to one document cluster.

[0110] Step 2 may further include:

[0111] Step 21: Generate a factual summary of each document based on the factual descriptions of all text blocks contained in the document;

[0112] Step 22, initialize the factual summary of each document into a document vector;

[0113] The vectorized representation of each document can be obtained using embedding model initialization;

[0114] Step 23, based on all document vectors, use Gaussian Mixture Model (GMM) for clustering, thereby dividing all documents into multiple document clusters;

[0115] Step 24, summarize the factual summary of all documents under each document cluster using a large language model to generate summary information, thereby obtaining a summary vector corresponding to each document cluster, and then continue to use Gaussian Mixture Model (GMM) for clustering based on the summary vectors corresponding to all document clusters until the root node is reached.

[0116] In step 24, the summary template for summarizing the summary of all documents under each document cluster using a large language model can be as follows:

[0117] "Your task is to generate a comprehensive summary of the following data.

[0118] Given a series of factual descriptions, please combine all these descriptions into a complete description.

[0119] Make sure to include all the information collected in the descriptions. If the provided descriptions contradict each other, please resolve these contradictions and provide a consistent and coherent summary.

[0120] #######

[0121] -Data-

[0122] Description List: {description_list}

[0123] #######

[0124] Output:”

[0125] In step three, obtain the user question, and select a reasoning path from the similarity tree structure diagram through the method of large language model debate iteration to obtain the corresponding document cluster, which can further include:

[0126] Step A1, assign three identities to the large language model, respectively designed for different specific tasks, which are:

[0127] (1) Debater: According to the given user question, select a path on the similarity tree structure diagram and explain the reasoning; According to the feedback of the critic and other debaters, the debater will also adjust or continue to defend the reasoning;

[0128] (2) Critics: analyze the flaws in each debater's path selection reason;

[0129] (3) Summarizer: determine whether the debaters agree on the final result and summarize the finally selected document cluster, giving the final reasoning result;

[0130] Step A2, set different temperatures or use different large language models as debaters, given the user question and the similarity tree structure diagram, the debaters propose different path selections and selection reasons;

[0131] Step A3, each debater refers to the opinions of other debaters, defends his own debate, and points out the flaws of other debaters;

[0132] Step A4, the critics find the flaws and potential errors in each debater's defense and convey them to the debaters;

[0133] Step A5, the debaters get the errors pointed out by other debaters and critics, refer to the opinions of other debaters, and correct their own opinions;

[0134] Step A6, after a round of debate, the summarizer determines whether the debaters agree on the final result and whether further debate is needed. If further debate is needed, return to step A3 to continue the next round of debate. If agreement is reached, the summarizer summarizes the final reasoning result and selects a reasoning path from the similarity tree structure diagram according to the user question.

[0135] The multi-role large language model multiple debate method improves the accuracy of path retrieval, selects a reasoning path on the similarity tree structure diagram, and narrows down the search range to a document cluster.

[0136] Since the document cluster contains a series of documents, each document has its own factual summary, in step three, based on the user question and the confidence score of the document, the document cluster is pruned. Taking document j in the document cluster as an example, it can further include:

[0137] Step B1, according to the user question and the factual summary of document j, generate a reasoning answer sequence T through a large language model: T = {t1, t2,..., t m}, where t1, t2,..., t m are the tokens segmented in the reasoning answer;

[0138] Step B2, calculate the credibility of each token in T: where H i is the credibility of the i-th token, t i is the i-th token, and pi (t i ) is the occurrence of t at position i in the vocabulary V i The probability, p i (v) is the probability of v appearing at position i in vocabulary V, where v is a word in vocabulary V;

[0139] Step B3: Calculate the similarity score between each token in T and the keywords of the user question and the factual summary of the document;

[0140] The similarity score can be obtained by calculating the cosine similarity;

[0141] Step B4: Calculate the stop word score of each token in T: Determine t i Whether it belongs to the stop word set, if yes, then its stop word score is 0; if not, then its stop word score is 1;

[0142] Step B5: Calculate the confidence score of each token in T: score(t i )=H i *A i *B i *S i , where A i 、B i 、S i They are t i Similarity score with user questions, similarity score with keywords in the factual summary of the document, and stop word score;

[0143] Step B6: Select the maximum value from the confidence scores of all tokens in T. The maximum value is the confidence score of document j. Then determine whether the confidence score of document j is greater than the threshold. If so, retain document j in the document cluster. If not, prune and delete document j from the document cluster.

[0144] In step 3, a structural score for each text block can be generated iteratively, and then k text blocks with the largest structural scores are selected. In step 3, several text blocks are selected from each pruned document. Taking document j as an example, the following can be further included:

[0145] Step C1: During initialization, the structural scores of all text blocks in document j are calculated, and the structural scores of all text blocks form a text block structural score matrix: Among them, π(0) represents the initialization text block structural score matrix, n is the number of text blocks in document j, that is, the structural scores of all text blocks in document j are initialized to Initialize the number of iterations t to 0;

[0146] Step C2: Calculate the similarity score between the factual descriptions of every two text blocks in document j, and construct a similarity matrix A of document j from all similarity scores;

[0147] The vector model can be used to calculate the similarity score between the factual descriptions of each two text blocks. Any element a in the similarity matrix A gh Represents the similarity score between the factual descriptions of the g-th and h-th text blocks in the document;

[0148] Step C3, iteratively calculate the text block structural score matrix for the next round: π(t+1)=(1-α-β)Aπ(t)+αp+βq, where π(t+1) and π(t) represent the text block structural score matrices obtained in rounds t+1 and t, respectively. p is a matrix consisting of similarity score vectors between all text blocks and user questions. q is a matrix consisting of similarity score vectors between keywords in all text blocks and user questions. α and β are the ratios of p and q, respectively, which can be set by the large language model according to the type of user question. For example, for comprehensive questions, the values ​​of α and β need to be set to be smaller. For local retrieval questions, the values ​​of α and β need to be set to be larger. For conceptual questions, the value of α needs to be set to be larger. For clearly directed questions, the value of β needs to be set to be larger.

[0149] Step C4: Determine whether the difference between π(t+1) and π(t) is within a certain threshold range. If so, it indicates that it has converged to a stable score and proceeds to the next step; if not, add 1 to t and then go to step C3;

[0150] Step C5: Sort all text blocks of document j in order of structural scores from high to low, and then select k text blocks with the highest ranking.

[0151] After selecting several text blocks, we can use the sub-questions to perform iterative reasoning based on the knowledge graph of the selected text blocks to obtain the first sub-graph set, and then use the large language model to perform pruning to obtain the second sub-graph set, thereby obtaining the final reasoning result. In step three, based on the knowledge graph of the selected text blocks, the reasoning result of the user question is generated, which can further include:

[0152] Step D1, initialize the iterative subproblem as the user problem;

[0153] Step D2: Based on the knowledge graph of all selected text blocks, find k entities related to the iterative sub-problem, and form a sub-graph with these k entities. Then, use the large language model to infer the answer S corresponding to the iterative sub-problem.

[0154] Step D3: Determine whether the large language model considers answer S to be the correct answer or whether the number of iterations has reached the upper limit. If so, proceed to step D4; if not, based on answer S and the iterative sub-problem, obtain the next iterative sub-problem through the large language model, and then proceed to step D2;

[0155] Step D4: All subgraphs obtained in the iterative process are combined into a first subgraph set, and then the first subgraph set is pruned using the large language model to remove irrelevant or duplicate information, thereby generating a second subgraph set;

[0156] Step D5: The large language model performs reasoning based on the second subgraph set, thereby generating a reasoning result for the user question.

[0157] Through this iterative approach, we can solve the difficulties brought by multi-hop problems, refine the retrieved effective information, and improve the accuracy of the answers.

[0158] like Figure 3 As shown, the present invention provides a large model reasoning system based on tree diagram and knowledge graph retrieval enhancement, including:

[0159] A knowledge graph construction device is used to obtain a number of documents, cut each document into multiple text blocks, and then identify the entities, relationships and factual descriptions of each text block to construct a knowledge graph for each text block;

[0160] A tree graph construction device is used to construct a factual overview of the document based on the knowledge graph of the text block, and then recursively cluster all documents using a Gaussian mixture model to generate a similarity tree structure graph consisting of all documents;

[0161] The inference device is used to obtain user questions and select an inference path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document cluster. Then, based on the confidence scores of the user questions and documents, the document clusters are pruned, and several text blocks are selected from each pruned document. Finally, based on the knowledge graph of the selected text blocks, the inference result of the user question is generated.

[0162] See also Figure 4 , Figure 4 4 is a block diagram of a computing device 400 according to an exemplary embodiment of this specification. Components of the computing device 400 include, but are not limited to, a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and a database 450 is used to store data.

[0163] The computing device 400 also includes an access device 440 that enables the computing device 400 to communicate via one or more networks 460. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and the like.

[0164] In one embodiment of the present specification, the above components of the computing device 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 4 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0165] Computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC. Computing device 400 may also be a mobile or stationary server or cloud server, etc.

[0166] The processor 420 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned large model reasoning method based on tree diagram and knowledge graph retrieval enhancement.

[0167] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device is based on the same concept as the technical solution of the large-scale model reasoning method based on tree diagrams and knowledge graph retrieval enhancement. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the large-scale model reasoning method based on tree diagrams and knowledge graph retrieval enhancement.

[0168] An embodiment of this specification also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the above-mentioned large model reasoning method based on tree diagram and knowledge graph retrieval enhancement are implemented.

[0169] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of this storage medium is based on the same concept as the large-model inference method based on tree diagrams and knowledge graph retrieval enhancement described above. For details not described in detail in the technical scheme of the storage medium, please refer to the description of the technical scheme of the large-model inference method or system based on tree diagrams and knowledge graph retrieval enhancement described above.

[0170] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned large model reasoning method based on tree diagram and knowledge graph retrieval enhancement.

[0171] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of this computer program and the technical scheme of the large-model reasoning method based on tree diagrams and knowledge graph retrieval enhancement described above are based on the same concept. For details not described in detail in the technical scheme of the computer program, please refer to the description of the technical scheme of the large-model reasoning method or system based on tree diagrams and knowledge graph retrieval enhancement described above.

[0172] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0173] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium.

[0174] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0175] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A large model reasoning method based on tree diagram and knowledge graph retrieval enhancement, characterized in that: Includes: Step 1: Obtain several documents and cut each document into multiple text blocks. Then identify the entities, relationships, and factual descriptions of each text block to form a knowledge graph for each text block. Step 2: Build a factual overview of the document based on the knowledge graph of the text block, and then use the Gaussian mixture model to recursively cluster all documents to generate a similarity tree structure diagram consisting of all documents; Step 3: Obtain the user question and select an inference path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document cluster. Then, based on the confidence scores of the user question and the document, prune the document cluster and select several text blocks from each pruned document. Finally, based on the knowledge graph of the selected text blocks, generate the inference result of the user question. In step 1, the entities, relationships, and factual descriptions of each text block are identified to form a knowledge graph for each text block, which further includes: Construct entity extraction templates, relationship construction templates, and factual description templates, then insert the content of the text block into the template to obtain text prompts, and then input the text prompts into the large language model to obtain the entity, relationship, and factual descriptions of each text block. In step 3, the user question is obtained, and through the large language model debate iterative method, an inference path is selected from the similarity tree structure diagram to obtain the corresponding document cluster, which further includes: Step A1: Assign three identities to the large language model, each designed to perform different tasks: (1) Debater: Based on the given user question, the debater selects a path on the similarity tree and explains the reasoning. Based on the feedback from critics and other debaters, the debater adjusts or continues to defend the reasoning. (2) Critic: Analyze the flaws in each debater's reasons for choosing the path; (3) Summarizer: Determine whether the debaters reach a consensus on the final result, summarize the final selected document clusters, and give the final reasoning results; Step A2: Use different large language models as debaters. Given a user question and a similarity tree structure, the debaters propose different path choices and reasons for their choices. Step A3: Each debater refers to the other debaters' views, defends his or her own argument, and points out the flaws of other debaters; Step A4: The critics look for loopholes and errors in each debater's defense and convey them to the debaters; Step A5: After receiving the errors pointed out by other debaters and critics, the debater will refer to the other debaters' opinions and revise his or her own views. Step A6: After a round of debate, the summarizer determines whether a consensus is reached on the debate results and whether further debate is needed. If further debate is needed, the process returns to step A3 to continue the next round of debate. If consensus is reached, the summarizer summarizes the final reasoning results and selects a reasoning path from the similarity tree structure diagram based on the user's question.

2. The method according to claim 1, characterized in that Step 2 further includes: Step 21: Generate a factual summary of each document based on the factual descriptions of all text blocks contained in the document; Step 22: Initialize the factual summary of each document into a document vector; Step 23: Based on all document vectors, clustering is performed using a Gaussian mixture model (GMM) to divide all documents into multiple document clusters. Step 24: Summarize the factual overview of all documents under each document cluster through the large language model to generate summary information, thereby obtaining the summary vector corresponding to each document cluster. Then, based on the summary vectors corresponding to all document clusters, continue to cluster using the Gaussian mixture model (GMM) until the root node is reached.

3. The method according to claim 1, characterized in that In step 3, based on the confidence scores of user questions and documents, document clusters are pruned, which further includes: Step B1: Based on the user question and the factual overview of document j, generate the inference answer sequence T through the large language model: T = {t1, t2, ..., t m }, where t1, t3, ..., t m They are the tokens after segmentation in the reasoning answer; Step B2: Calculate the credibility of each token in T: Among them, H i is the credibility of the i-th token, t i is the i-th token, p i (t i ) is the occurrence of t at position i in the vocabulary V i The probability, p i (v) is the probability of v appearing at position i in vocabulary V, where v is a word in vocabulary V; Step B3: Calculate the similarity score between each token in T and the keywords of the user question and the factual summary of the document; Step B4: Calculate the stop word score of each token in T: Determine t i Whether it belongs to the stop word set, if yes, then its stop word score is 0; if not, then its stop word score is 1; Step B5: Calculate the confidence score of each token in T: score(t i )=H i *A i *B i *S i , where A i 、B i 、S i They are t i Similarity score with user questions, similarity score with keywords in the factual summary of the document, and stop word score; Step B6: Select the maximum value from the confidence scores of all tokens in T. The maximum value is the confidence score of document j. Then determine whether the confidence score of document j is greater than the threshold. If so, retain document j in the document cluster. If not, prune and delete document j from the document cluster.

4. The method according to claim 1, wherein In step 3, several text blocks are selected from each pruned document, further including: Step C1: During initialization, the structural scores of all text blocks in document j are calculated, and the structural scores of all text blocks form a text block structural score matrix: Among them, π(0) represents the initialization text block structural score matrix, n is the number of text blocks in document j, that is, the structural scores of all text blocks in document j are initialized to Initialize the number of iterations t to 0; Step C2: Calculate the similarity score between the factual descriptions of every two text blocks in document j, and construct a similarity matrix A of document j from all similarity scores; Step C3, iteratively calculate the text block structural score matrix for the next round: π(t+1)=(1-α-β)Aπ(t)+αp+βq, where π(t+1) and π(t) represent the text block structural score matrices obtained in rounds t+1 and t, respectively; p is a matrix consisting of similarity score vectors between all text blocks and user questions; q is a matrix consisting of similarity score vectors between keywords in all text blocks and user questions; α and β are the ratios of p and q, respectively; Step C4: Determine whether the difference between π(t+1) and π(t) is within a certain threshold range. If so, it indicates that it has converged to a stable score and proceeds to the next step; if not, add 1 to t and then go to step C3; Step C5: Sort all text blocks of document j in order of structural scores from high to low, and then select k text blocks with the highest ranking.

5. The method according to claim 1, wherein In step 3, based on the knowledge graph of the selected text block, the reasoning results of the user question are generated, which further includes: Step D1, initialize the iterative subproblem as the user problem; Step D2: Based on the knowledge graph of all selected text blocks, find k entities related to the iterative sub-problem, and form a sub-graph with these k entities. Then, use the large language model to infer the answer S corresponding to the iterative sub-problem. Step D3: Determine whether the large language model considers answer S to be the correct answer or whether the number of iterations has reached the upper limit. If so, proceed to step D4. If not, based on answer S and the iterative sub-problem, obtain the next iterative sub-problem through the large language model, and then proceed to step D2. Step D4: All subgraphs obtained in the iterative process are used to form a first subgraph set, and then the first subgraph set is pruned using the large language model to generate a second subgraph set; Step D5: The large language model performs reasoning based on the second subgraph set, thereby generating a reasoning result for the user question.

6. A large model reasoning system based on tree diagram and knowledge graph retrieval enhancement, characterized by: The system is used to execute the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement as claimed in claim 1, and the system includes: A knowledge graph construction device is used to obtain a number of documents, cut each document into multiple text blocks, and then identify the entities, relationships and factual descriptions of each text block to construct a knowledge graph for each text block; A tree graph construction device is used to construct a factual overview of the document based on the knowledge graph of the text block, and then recursively cluster all documents using a Gaussian mixture model to generate a similarity tree structure graph consisting of all documents; The inference device is used to obtain user questions and select an inference path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document cluster. Then, based on the confidence scores of the user questions and documents, the document clusters are pruned, and several text blocks are selected from each pruned document. Finally, based on the knowledge graph of the selected text blocks, the inference result of the user question is generated.

7. A computing device, characterized in that include: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that It stores computer-executable instructions, which, when executed by a processor, implement the steps of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement as described in any one of claims 1-5.

Images