Large model reasoning method and system based on tree diagram and knowledge graph retrieval enhancement

Abstract

The invention discloses a large model reasoning method and system based on tree diagram and knowledge graph retrieval enhancement, and the method comprises the steps: segmenting a document into text blocks, recognizing the entities, relationships and factual descriptions of the text blocks, and forming a knowledge graph of the text blocks; constructing a factual summary of the documents based on the knowledge graph of the text blocks, and performing clustering recursion on all the documents by using a Gaussian mixture model to generate a similarity tree structure graph; the method comprises the following steps: obtaining a user problem, selecting a reasoning path from a similarity tree structure diagram through a large language model debate iteration method, obtaining a corresponding document cluster, pruning the cluster based on the user problem and a confidence score of a document, selecting a plurality of text blocks from the pruned document, and finally obtaining a knowledge graph based on the selected text blocks. And generating a reasoning result. The invention relates to the technical field of natural language processing, can realize retrieval enhancement based on a tree diagram and a knowledge graph, and accurately queries related information when a large model answers a complex multi-hop question, so that the reasoning accuracy is effectively ensured.

Description

Technical Field

[0001] The present invention relates to a large model reasoning method and system based on tree diagram and knowledge graph retrieval enhancement, and relates to the field of communications. Background Art

[0002] The development of large language models has brought many opportunities and directions for the development of the industry. However, because large language models often rely on the knowledge within the model, when dealing with tasks that rely on a large amount of facts and external knowledge, the model cannot guarantee the most accurate and relevant answers to specific questions. At the same time, it is easy to produce hallucinations and reasoning errors when information is missing. Therefore, retrieval enhancement came into being. Retrieval enhancement combines the advantages of information retrieval and generation, overcomes the limitations of large language models in dealing with complex problems, acquiring external knowledge, and ensuring the accuracy of answers, and is suitable for tasks that require a large amount of external knowledge and reasoning ability, such as question-answering systems, dialogue generation, and multi-step reasoning tasks.

[0003] Traditional retrieval enhancement retrieves relevant content from a large number of text blocks. There is no clear structure between text blocks, and the information is scattered. It is impossible to obtain the relationship between entities or text blocks during retrieval. Therefore, retrieval enhancement based on knowledge graph is proposed. Knowledge graph can provide structured data, so that large language models can more clearly capture the relationship and context between entities, reducing the ambiguity of information. At the same time, when the problem involves multi-step reasoning, complex background knowledge, or multi-level relationships between entities, the ability of retrieval enhancement based on text blocks may be limited, while retrieval enhancement based on knowledge graph can enable the model to perform more complex reasoning. The structured data in the knowledge graph can help the model perform multi-hop reasoning when answering questions.

[0004] Knowledge graphs focus on the relationship between entities, but ignore the hierarchical structure of information, making it difficult to answer questions with high hierarchical requirements. Tree structure diagrams can organize information hierarchically, making the relationship between information clearer. For example, in a tree structure diagram, the parent node can represent the preliminary answer to a question, while the child node contains more detailed information. This hierarchical structure helps the model better understand and process complex queries.

[0005] Therefore, how to achieve retrieval enhancement based on tree diagrams and knowledge graphs, accurately query relevant information when the large language model answers complex multi-hop questions, and thus effectively ensure the accuracy of large language model reasoning has become a technical issue that technical personnel have focused on. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a large model reasoning method and system based on tree diagram and knowledge graph retrieval enhancement, which can realize retrieval enhancement based on tree diagram and knowledge graph, and accurately query relevant information when the large language model answers complex multi-hop questions, thereby effectively ensuring the accuracy of large language model reasoning.

[0007] In order to achieve the above object, the present invention provides a large model reasoning method based on tree diagram and knowledge graph retrieval enhancement, including:

[0008] Step 1: Get several documents and cut each document into multiple text blocks, then identify the entities, relationships and factual descriptions of each text block to form a knowledge graph for each text block;

[0009] Step 2: Build a factual overview of the document based on the knowledge graph of the text block, and then use the Gaussian mixture model to recursively cluster all documents to generate a similarity tree structure diagram consisting of all documents;

[0010] Step 3: Obtain user questions, and select a reasoning path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document clustering. Then, based on the confidence scores of the user questions and documents, prune the document clusters, and select several text blocks from each pruned document. Finally, based on the knowledge graph of the selected text blocks, generate the reasoning results of the user questions.

[0011] In order to achieve the above object, the present invention also provides a large model reasoning system based on tree diagram and knowledge graph retrieval enhancement, including:

[0012] A knowledge graph construction device is used to obtain a number of documents, cut each document into multiple text blocks, and then identify the entities, relations and factual descriptions of each text block to construct a knowledge graph for each text block;

[0013] A tree graph construction device is used to construct a factual overview of the document based on the knowledge graph of the text block, and then recursively cluster all the documents using a Gaussian mixture model to generate a similarity tree structure graph consisting of all the documents;

[0014] The inference device is used to obtain user questions, select an inference path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document cluster, and then prune the document cluster based on the confidence scores of the user questions and documents, and select several text blocks from each pruned document, and finally generate the inference result of the user question based on the knowledge graph of the selected text blocks.

[0015] In order to achieve the above object, the present invention further provides a computing device, comprising:

[0016] Memory and processor;

[0017] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement are implemented.

[0018] In order to achieve the above-mentioned objectives, the present invention also provides a computer-readable storage medium, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention realizes retrieval enhancement based on tree structure and knowledge graph. The knowledge graph focuses on the association between entities. The retrieval enhancement based on the knowledge graph can enable the model to perform more complex reasoning. The structured data in the knowledge graph can help the model to perform multi-hop reasoning when answering questions, while the tree structure focuses on the hierarchical structure of information. The retrieval enhancement based on the tree structure can handle more complex reasoning paths, help the large language model to refine the problem layer by layer, and gradually go deep into each branch. In this way, the knowledge graph is applied to the detailed information inside the text block, and the tree structure is applied to the external clustering information of the document, which can combine the advantages of both and enhance the processing of information; in the retrieval of the tree structure, the current technical solutions often use large language models to iterate to select a reasoning path, but when the problem spans multiple research objects, it is difficult to ensure the accuracy of the reasoning path. Therefore, the present invention also proposes a method of multiple debates-pruning optimization for path retrieval, using multiple large language models in an iterative process of debaters proposing arguments-debators raising questions-debators defending-evaluators raising questions-debators modifying arguments, and finally concludes with the summarizer The final reasoning conclusion can effectively improve the accuracy of path selection in this way; in cluster screening, traditional technical solutions often use small models for evaluation, but because the training of small models depends on data sets, it is difficult to achieve generalization. Therefore, the present invention also proposes to use a large language model to generate confidence scores, and perform pruning optimization through thresholds. The confidence score is calculated through the credibility of each token, the similarity with the question, the keyword similarity with the factual summary of the document, and the stop word score. When the confidence score of any token in the document reasoning sequence is greater than the threshold, the document is considered to be relevant and retained, otherwise it is considered irrelevant and pruned and deleted; current retrieval enhancement often uses similarity scores as evaluation relevance, but ignores the influence of the structure of the text block in the document, and it is difficult to answer questions with high summary or conceptual requirements. The present invention also comprehensively considers the structural characteristics of the text block (the relationship with other text blocks), the relationship between the text block and the question, and the relationship between the text block and the question keywords, and uses a large language model to design parameters according to the characteristics of the question to iteratively generate relevant scores for the text block, effectively ensuring the quality of the score and improving the retrieval recall rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of a large model reasoning method based on tree diagram and knowledge graph retrieval enhancement shown in an exemplary embodiment of the present invention.

[0021] Figure 2 It is a schematic diagram of a similarity tree structure diagram shown in an exemplary embodiment of the present invention.

[0022] Figure 3It is a structural diagram of a large model reasoning system based on tree diagram and knowledge graph retrieval enhancement shown in an exemplary embodiment of the present invention.

[0023] Figure 4 It is a schematic diagram of the structure of a computer device according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings.

[0025] like Figure 1 As shown, the present invention provides a large model reasoning method based on tree diagram and knowledge graph retrieval enhancement, including:

[0026] Step 1: Get several documents and cut each document into multiple text blocks, then identify the entities, relationships and factual descriptions of each text block to form a knowledge graph for each text block;

[0027] Step 2: Build a factual overview of the document based on the knowledge graph of the text block, and then use the Gaussian mixture model to recursively cluster all documents to generate a similarity tree structure diagram consisting of all documents;

[0028] Step 3: Obtain user questions, and select a reasoning path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document clustering. Then, based on the confidence scores of the user questions and documents, prune the document clusters, and select several text blocks from each pruned document. Finally, based on the knowledge graph of the selected text blocks, generate the reasoning results of the user questions.

[0029] In step 1, the entities, relations, and factual descriptions of each text block are identified to form a knowledge graph for each text block, which may further include:

[0030] Construct entity extraction templates, relationship construction templates and factual description templates, then insert the content of the text block into the template to obtain text prompts, and then input the text prompts into the large language model to obtain the entity, relationship and factual descriptions of each text block.

[0031] An entity extraction template could look like this:

[0032] "-Goal-

[0033] Given a provided text and a list of entity types, identify all of these types of entities from the text.

[0034] -Steps-

[0035] 1. Output |START|, indicating the start of entity extraction.

[0036] 2. Identify all entities. For each identified entity, extract the following information:

[0037] -entity_name:Entity name

[0038] -entity_type: One of the following types: [{entity_types}]

[0039] -entity_description: A comprehensive description of the entity's attributes and activities

[0040] Format each entity as

[0041] ("entity"{tuple_delimiter}<entity_name> {tuple_delimiter}<entity_type> {tuple_delimiter}<entity_description> )

[0042] 3. Output all entities identified in step 2 as a single list. Use **{record_delimiter}** as the list delimiter.

[0043] 4. When completed, output {completion_delimiter}

[0044] ######################

[0045] -Real Data-

[0046] ######################

[0047] Entity_types:{entity_types}

[0048] Text:{input_text}

[0049] ######################

[0050] Output:”

[0051] Specific examples are as follows:

[0052] input_text:Verdantis' central bank plans to hold meetings on Monday and Thursday. The bank plans to release its latest policy decision at 1:30 p.m. on Thursday, followed by a press conference where central bank chairman Martin Smith will answer questions from reporters. Investors expect the market strategy committee to keep the benchmark interest rate in the range of 3.5% to 3.75%.

[0053] entity_types:ORGANIZATION,PERSON

[0054] Extract results:

[0055] ("entity"{tuple_delimiter}Central institution

[0056] {tuple_delimiter}ORGANIZATION{tuple_delimiter}The central authority is the Federal Reserve Board of Verdantis, which sets interest rates on Mondays and Thursdays)

[0057] ("entity"{tuple_delimiter}Martin Smith

[0058] {tuple_delimiter}PERSON{tuple_delimiter}Martin Smith is the Chairman of the Central Authority)

[0059] A relationship building template could look like this:

[0060] "-Goal-

[0061] Given a provided text and a list of entities, identify the relationships between all these entities from the text.

[0062] -Steps-

[0063] 1. Output |START|, indicating the start of relationship building.

[0064] 2. Identify the relationships between entities and extract the following information:

[0065] -source_entity: source entity name

[0066] -target_entity: target entity name

[0067] -relationship_description: Explains why the source and target entities are related to each other

[0068] -relationship_strength: A numerical score for the strength of the relationship between the source and target entities

[0069] Format each relationship as

[0070] ("relationship"{tuple_delimiter}<source_entity> {tuple_delimiter}<target_entity> {tuple_delimiter}<relationship_description> {tuple_delimiter}<relationship_strength> )

[0071] 3. Output all the relationships identified in step 2 as a single list.

[0072] **{record_delimiter}** acts as the list delimiter.

[0073] 4. When completed, output {completion_delimiter}

[0074] ######################

[0075] -Real Data-

[0076] ######################

[0077] Entity:{entity}

[0078] Text:{input_text}

[0079] ######################

[0080] Output:”

[0081] Specific examples are as follows:

[0082] input_text:Verdantis' central bank plans to hold meetings on Monday and Thursday. The bank plans to release its latest policy decision at 1:30 p.m. on Thursday, followed by a press conference where central bank chairman Martin Smith will answer questions from reporters. Investors expect the market strategy committee to keep the benchmark interest rate in the range of 3.5% to 3.75%.

[0083] entity: Central Agency, Martin Smith

[0084] Build results:

[0085] ("relationship"{tuple_delimiter}Martin Smith{tuple_delimiter}Central Agency{tuple_delimiter}Martin Smith is the chairman of the Central Agency and will answer questions at the press conference{tuple_delimiter}9)

[0086] A factual description template might look like this:

[0087] “Your task is to construct a factual description in text format based on the provided entities and the relationships between them.

[0088] Entity:{entity}

[0089] Relationship:{relationship}

[0090] Output:”

[0091] Specific examples are as follows:

[0092] Entity: Martin Smith, Martin Smith is the chairman of the central agency; the central agency, the central agency is the Verdantis Federal Reserve, which will set interest rates on Mondays and Thursdays.

[0093] relationship: Martin Smith is the central body chairman and will answer questions at the press conference.

[0094] Build results:

[0095] Martin Smith is the Chairman of the Central Authority, which is the Federal Reserve Board of Verdantis

[0096] The knowledge graph can be saved in graphml format and the factual description can be saved in parquet format.

[0097] Figure 2 FIG. 4 is a schematic diagram showing an embodiment of a similarity tree structure diagram. Figure 2 As shown in the figure, the similarity tree structure diagram takes all documents as leaf nodes, divides all documents into different document clusters through the factual overview of the documents, and then recursively calculates until the root node. In this way, through clustering recursion, a hierarchical similarity tree structure diagram is generated. Figure 2 It can be seen that document 1 and document 3 belong to one document cluster, ..., document 2 and document n belong to one document cluster.

[0098] Step 2 may further include:

[0099] Step 21, generating a factual summary of each document based on the factual description of all text blocks contained in the document;

[0100] Step 22, initializing the factual summary of each document into a document vector;

[0101] You can use the embedding model to initialize the vectorized representation of each document.

[0102] Step 23: Based on all document vectors, clustering is performed using a Gaussian mixture model (GMM), thereby dividing all documents into multiple document clusters;

[0103] Step 24: Summarize the factual overview of all documents under each document cluster through a large language model to generate summary information, thereby obtaining a summary vector corresponding to each document cluster, and then continue to use a Gaussian mixture model (GMM) to perform clustering based on the summary vectors corresponding to all document clusters until the root node is reached.

[0104] In step 24, the summary template for summarizing the overview of all documents under each document cluster through the large language model can be as follows:

[0105] “Your task is to generate a comprehensive summary of the following data.

[0106] Given a series of factual descriptions, stitch all of them together into a complete description.

[0107] Make sure to include all the information gathered in the description. If the descriptions provided are contradictory, resolve the contradictions and provide a consistent, coherent summary.

[0108] #######

[0109] -Data-

[0110] Description List:{description_list}

[0111] #######

[0112] Output:”

[0113] In step 3, the user question is obtained, and a reasoning path is selected from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document clustering, which can further include:

[0114] Step A1: Assign three identities to the large language model, each designed to perform different specific tasks:

[0115] (1) Debater: Based on the given user question, the debater selects a path on the similarity tree diagram and explains the reasoning. Based on the feedback from critics and other debaters, the debater will also adjust or continue to defend the reasoning.

[0116] (2) Critic: Analyze the flaws in each debater’s reasons for choosing the path;

[0117] (3) Summarizer: Determine whether the debaters reach a consensus on their final results, summarize the final selected document clusters, and give the final reasoning results;

[0118] Step A2: Set different temperatures or use different large language models as debaters. Given the user question and similarity tree structure diagram, the debater proposes different path choices and reasons for the choices.

[0119] Step A3: Each debater refers to the views of other debaters, defends his or her own debate, and points out the loopholes of other debaters;

[0120] Step A4: The critic looks for loopholes and potential errors in each debater's defense and then conveys them to the debater;

[0121] Step A5: After the debaters have received the mistakes pointed out by other debaters and critics, they refer to the opinions of other debaters and revise their own opinions.

[0122] Step A6: After a round of debate, the summarizer determines whether a consensus is reached on the debate results and whether further debate is needed. If further debate is needed, the process returns to step A3 to continue the next round of debate. If a consensus is reached, the summarizer summarizes the final reasoning results and selects a reasoning path from the similarity tree structure diagram according to the user's question.

[0123] The method of multiple debates of multi-role large language model improves the accuracy of path retrieval, selects a reasoning path on the similarity tree structure diagram, and narrows the retrieval scope to a document cluster.

[0124] Since the document cluster contains a series of documents, each document has its own factual overview. In step 3, based on the user question and the confidence score of the document, the document cluster is pruned. Taking document j in the document cluster as an example, it can further include:

[0125] Step B1: Generate an inference answer sequence T based on the user question and the factual summary of document j through a large language model: T = {t1, t2, ..., t m}, where t1, t2, ..., t m They are the tokens after segmentation in the reasoning answer;

[0126] Step B2: Calculate the credibility of each token in T: Among them, H i is the credibility of the ith token, t i is the i-th token, p i (t i ) is the occurrence of position i in the vocabulary V i The probability, p i (v) is the probability of v appearing at position i in vocabulary V, where v is a word in vocabulary V;

[0127] Step B3, calculate the similarity score between each token in T and the keywords of the user question and the factual summary of the document;

[0128] The similarity score can be obtained by calculating the cosine similarity;

[0129] Step B4: Calculate the stop word score of each token in T: Determine t i Whether it belongs to the stop word set, if yes, its stop word score is 0; if no, its stop word score is 1;

[0130] Step B5: Calculate the confidence score of each token in T: score(t i )=H i *A i *B i *S i , where A i , B i , S i They are t i Similarity score with user questions, similarity score with keywords of the factual summary of the document, and stop word score;

[0131] Step B6: Select the maximum value from the confidence scores of all tokens in T, which is the confidence score of document j. Then determine whether the confidence score of document j is greater than a threshold. If so, retain document j in the document cluster. If not, prune and delete document j from the document cluster.

[0132] In step 3, the structural score of each text block can be generated by iteration, and then k text blocks with the largest structural scores are selected. In step 3, several text blocks are selected from each pruned document. Taking document j as an example, it can further include:

[0133] Step C1: During initialization, the structural scores of all text blocks in document j are calculated, and the structural scores of all text blocks form a text block structural score matrix: Among them, π(0) represents the initialization text block structural score matrix, n is the number of text blocks in document j, that is, the structural scores of all text blocks in document j are initialized to Initialize the number of iterations t to 0;

[0134] Step C2, calculating the similarity score between the factual descriptions of every two text blocks in document j, and constructing a similarity matrix A of document j from all similarity scores;

[0135] The vector model can be used to calculate the similarity score between the factual descriptions of each two text blocks. Any element a in the similarity matrix A gh Represents the similarity score between the factual descriptions of the g-th and h-th text blocks in the document;

[0136] Step C3, iteratively calculate the text block structural score matrix of the next round: π(t+1)=(1-α-β)Aπ(t)+αp+βq, where π(t+1) and π(t) represent the text block structural score matrices obtained in the t+1th and tth rounds, respectively, p is a matrix composed of similarity score vectors between all text blocks and user questions, q is a matrix composed of similarity score vectors between keywords of all text blocks and user questions, α and β are the ratio values ​​of p and q, respectively, which can be set by the large language model according to the type of user questions. For example, for comprehensive questions, the values ​​of α and β need to be set to be relatively small, for local retrieval questions, the values ​​of α and β need to be set to be relatively large, for conceptual questions, the value of α needs to be set to be relatively large, and for questions with clear directions, the value of β needs to be set to be relatively large;

[0137] Step C4: determine whether the difference between π(t+1) and π(t) is within a certain threshold range. If yes, it means that it has converged to a stable score and proceed to the next step. If not, add 1 to t and go to step C3.

[0138] Step C5: sort all text blocks of document j in order of structural scores from high to low, and then select k text blocks with the highest ranking.

[0139] After selecting several text blocks, we can use sub-questions to perform iterative reasoning based on the knowledge graph of the selected text blocks to obtain the first sub-graph set, and then use the large language model to perform pruning to obtain the second sub-graph set, thereby obtaining the final reasoning result.

[0140] In step 3, based on the knowledge graph of the selected text block, the reasoning result of the user question is generated, which may further include:

[0141] Step D1, initializing the iterative subproblem as a user problem;

[0142] Step D2: According to the knowledge graph of all selected text blocks, k entities related to the iterative sub-problem are found, and a sub-graph is formed by the k entities. Then, the answer S corresponding to the iterative sub-problem is obtained through reasoning with the large language model;

[0143] Step D3: determine whether the large language model considers answer S to be the correct answer or the number of iterations reaches the upper limit. If yes, go to step D4; if not, based on answer S and the iterative sub-problem, obtain the next iterative sub-problem through the large language model, and then go to step D2;

[0144] Step D4: All subgraphs obtained in the iterative process constitute a first subgraph set, and then use the large language model to prune the first subgraph set to remove irrelevant or repeated information, thereby generating a second subgraph set;

[0145] Step D5: The large language model performs reasoning based on the second sub-graph set, thereby generating a reasoning result of the user question.

[0146] Through this iterative approach, we can solve the difficulties brought by multi-hop problems, refine the retrieved effective information, and improve the accuracy of the answers.

[0147] like Figure 3 As shown, the present invention is a large model reasoning system based on tree diagram and knowledge graph retrieval enhancement, including:

[0148] A knowledge graph construction device is used to obtain a number of documents, cut each document into multiple text blocks, and then identify the entities, relations and factual descriptions of each text block to construct a knowledge graph for each text block;

[0149] A tree graph construction device is used to construct a factual overview of the document based on the knowledge graph of the text block, and then recursively cluster all the documents using a Gaussian mixture model to generate a similarity tree structure graph consisting of all the documents;

[0150] The inference device is used to obtain user questions, select an inference path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document cluster, and then prune the document cluster based on the confidence scores of the user questions and documents, and select several text blocks from each pruned document, and finally generate the inference result of the user question based on the knowledge graph of the selected text blocks.

[0151] See also Figure 4 , Figure 4 4 is a block diagram of a computing device 400 shown in an exemplary embodiment of this specification. The components of the computing device 400 include but are not limited to a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and the database 450 is used to store data.

[0152] The computing device 400 also includes an access device 440 that enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and the like.

[0153] In one embodiment of the present specification, the above components of the computing device 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 4 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0154] The computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a PC. The computing device 400 may also be a mobile or stationary server or a cloud server, etc.

[0155] Processor 420 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned large model reasoning method based on tree diagram and knowledge graph retrieval enhancement.

[0156] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement belong to the same concept. For details not described in detail in the technical scheme of the computing device, please refer to the description of the technical scheme of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement.

[0157] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned large model reasoning method based on tree diagram and knowledge graph retrieval enhancement.

[0158] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium belongs to the same concept as the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement. For details not described in detail in the technical scheme of the storage medium, please refer to the description of the technical scheme of the large model reasoning method or system based on tree diagram and knowledge graph retrieval enhancement.

[0159] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned large model reasoning method based on tree diagram and knowledge graph retrieval enhancement.

[0160] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement belong to the same concept. For details not described in detail in the technical scheme of the computer program, please refer to the description of the technical scheme of the large model reasoning method or system based on tree diagram and knowledge graph retrieval enhancement.

[0161] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0162] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0163] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0164] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A large model reasoning method based on tree diagram and knowledge graph retrieval enhancement, characterized in that: Included are: Step 1: Get several documents and cut each document into multiple text blocks, then identify the entities, relationships and factual descriptions of each text block to form a knowledge graph for each text block; Step 2: Build a factual overview of the document based on the knowledge graph of the text block, and then use the Gaussian mixture model to recursively cluster all documents to generate a similarity tree structure diagram consisting of all documents; Step 3: Obtain user questions, and select a reasoning path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document clustering. Then, based on the confidence scores of the user questions and documents, prune the document clusters, and select several text blocks from each pruned document. Finally, based on the knowledge graph of the selected text blocks, generate the reasoning results of the user questions.

2. The method according to claim 1, characterized in that In step 1, the entities, relations, and factual descriptions of each text block are identified to form a knowledge graph for each text block, which further includes: Construct entity extraction templates, relationship construction templates and factual description templates, then insert the content of the text block into the template to obtain text prompts, and then input the text prompts into the large language model to obtain the entity, relationship and factual descriptions of each text block.

3. The method according to claim 1, characterized in that Step 2 further includes: Step 21, generating a factual summary of each document based on the factual description of all text blocks contained in the document; Step 22, initializing the factual summary of each document into a document vector; Step 23: Based on all document vectors, clustering is performed using Gaussian mixture model GMM, thereby dividing all documents into multiple document clusters; Step 24: Summarize the factual overview of all documents under each document cluster through a large language model to generate summary information, thereby obtaining a summary vector corresponding to each document cluster, and then continue to use the Gaussian mixture model GMM for clustering based on the summary vectors corresponding to all document clusters until the root node is reached.

4. The method according to claim 1, characterized in that: In step 3, the user question is obtained, and a reasoning path is selected from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document clustering, which further includes: Step A1: Assign three identities to the large language model, each designed to perform different tasks: (1) Debater: Based on the given user question, the debater selects a path on the similarity tree diagram and explains the reasoning. Based on the feedback from critics and other debaters, the debater adjusts or continues to defend the reasoning. (2) Critic: Analyze the flaws in each debater’s reasons for choosing the path; (3) Summarizer: Determine whether the debaters reach a consensus on their final results, summarize the final selected document clusters, and give the final reasoning results; Step A2: Set different temperatures or use different large language models as debaters. Given the user question and similarity tree structure diagram, the debater proposes different path choices and reasons for the choices. Step A3: Each debater refers to the views of other debaters, defends his or her own debate, and points out the loopholes of other debaters; Step A4: The critic looks for loopholes and errors in each debater's defense and then conveys them to the debater; Step A5: After the debaters have received the mistakes pointed out by other debaters and critics, they refer to the opinions of other debaters and revise their own opinions. Step A6: After a round of debate, the summarizer determines whether a consensus is reached on the debate results and whether further debate is needed. If further debate is needed, the process returns to step A3 to continue the next round of debate. If a consensus is reached, the summarizer summarizes the final reasoning results and selects a reasoning path from the similarity tree structure diagram according to the user's question.

5. The method according to claim 1, characterized in that In step 3, based on the confidence scores of user questions and documents, document clusters are pruned, further including: Step B1: Generate an inference answer sequence T based on the user question and the factual summary of document j through a large language model: T = {t1, t2, ..., t m }, where t1, t2, ..., t m They are the tokens after segmentation in the reasoning answer; Step B2: Calculate the credibility of each token in T: Among them, H i is the credibility of the ith token, t i is the i-th token, p i (t i ) is the occurrence of position i in the vocabulary V i The probability, p i (v) is the probability of v appearing at position i in vocabulary V, where v is a word in vocabulary V; Step B3, calculate the similarity score between each token in T and the keywords of the user question and the factual summary of the document; Step B4: Calculate the stop word score of each token in T: Determine t i Whether it belongs to the stop word set, if yes, its stop word score is 0; if no, its stop word score is 1; Step B5: Calculate the confidence score of each token in T: score(t i )=H i *A i *B i *S i , where A i , B i , S i They are t i Similarity score with user questions, similarity score with keywords of the factual summary of the document, and stop word score; Step B6: Select the maximum value from the confidence scores of all tokens in T, which is the confidence score of document j. Then determine whether the confidence score of document j is greater than a threshold. If so, retain document j in the document cluster. If not, prune and delete document j from the document cluster.

6. The method according to claim 1, characterized in that In step 3, a number of text blocks are selected from each pruned document, further including: Step C1: During initialization, the structural scores of all text blocks in document j are calculated, and the structural scores of all text blocks form a text block structural score matrix: Among them, π(0) represents the initialization text block structural score matrix, n is the number of text blocks in document j, that is, the structural scores of all text blocks in document j are initialized to Initialize the number of iterations t to 0; Step C2, calculating the similarity score between the factual descriptions of every two text blocks in document j, and constructing a similarity matrix A of document j from all similarity scores; Step C3, iteratively calculate the next round of text block structural score matrix: π(t+1)=(1-α-β)Aπ(t)+αp+βq, where π(t+1) and π(t) represent the text block structural score matrices obtained in the t+1th and tth rounds, respectively, p is a matrix composed of similarity score vectors between all text blocks and user questions, q is a matrix composed of similarity score vectors between keywords of all text blocks and user questions, and α and β are the ratios of p and q, respectively; Step C4: determine whether the difference between π(t+1) and π(t) is within a certain threshold range. If yes, it means that it has converged to a stable score and proceed to the next step. If not, add 1 to t and go to step C3. Step C5: sort all text blocks of document j in order of structural scores from high to low, and then select k text blocks with the highest ranking.

7. The method according to claim 1, characterized in that In step 3, based on the knowledge graph of the selected text block, the reasoning results of the user question are generated, which further includes: Step D1, initializing the iterative subproblem as a user problem; Step D2: According to the knowledge graph of all selected text blocks, k entities related to the iterative sub-problem are found, and a sub-graph is formed by the k entities. Then, the answer S corresponding to the iterative sub-problem is obtained through reasoning with the large language model; Step D3: determine whether the large language model considers answer S to be the correct answer or the number of iterations reaches the upper limit. If yes, go to step D4; if not, based on answer S and the iterative sub-problem, obtain the next iterative sub-problem through the large language model, and then go to step D2; Step D4: All subgraphs obtained in the iterative process constitute a first subgraph set, and then prune the first subgraph set using the large language model to generate a second subgraph set; Step D5: The large language model performs reasoning based on the second sub-graph set, thereby generating a reasoning result of the user question.

8. A large model reasoning system based on tree diagram and knowledge graph retrieval enhancement, characterized in that: Included are: A knowledge graph construction device is used to obtain a number of documents, cut each document into multiple text blocks, and then identify the entities, relations and factual descriptions of each text block to construct a knowledge graph for each text block; A tree graph construction device is used to construct a factual overview of the document based on the knowledge graph of the text block, and then recursively cluster all the documents using a Gaussian mixture model to generate a similarity tree structure graph consisting of all the documents; The inference device is used to obtain user questions, select an inference path from the similarity tree structure diagram through the large language model debate iteration method to obtain the corresponding document cluster, and then prune the document cluster based on the confidence scores of the user questions and documents, and select several text blocks from each pruned document, and finally generate the inference result of the user question based on the knowledge graph of the selected text blocks.

9. A computing device, characterized in that include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, implement the steps of the large model reasoning method based on tree diagram and knowledge graph retrieval enhancement as described in any one of claims 1 to 7.

Images