Clustering algorithm and large model-based inference tree generation method and system
By using a clustering algorithm and a large-scale model-based inference tree generation method, the problem of low retrieval efficiency and accuracy of the RAG model is solved, achieving efficient and accurate knowledge graph retrieval and inference tree generation, which is suitable for large-scale model applications in vertical fields.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-08
- Publication Date
- 2026-03-24
AI Technical Summary
Existing retrieval augmentation (RAG) models have low retrieval efficiency and accuracy, and are prone to retrieving redundant data, making it difficult to meet the needs of building large models in vertical fields.
A reasoning tree generation method based on clustering algorithms and large models is adopted. Similar hyperedges are divided by hypergraph modularity clustering algorithm, keywords and entities are extracted by large models, the scope of knowledge graph retrieval is limited, and the expansion of reasoning trees is controlled by accumulating feature values of topological structure and semantics, and the expansion of reasoning trees is stopped in a timely manner.
It improves the efficiency and accuracy of knowledge graph retrieval, reduces the querying of redundant data, and achieves efficient knowledge graph retrieval and reasoning tree generation.
Smart Images
Figure CN121722808A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of large models, and particularly relates to a reasoning tree generation method and system based on a clustering algorithm and a large model. BACKGROUND
[0002] With the rapid development of large models, they have shown amazing capabilities in many natural language processing tasks, especially in knowledge question answering, content generation and code writing. However, although large models perform well in many fields, in the vertical fields of many industry applications, large models are still in the early stage and far from the ideal state.
[0003] Retrieval-augmented Generation (RAG), as a new technology combining large model generation and existing human knowledge, has received widespread attention since it was proposed in 2020. The retrieval-augmented generation model combines language models and information retrieval technology, has the ability to retrieve information from an existing knowledge base, and guides content generation based on this, thereby reducing "hallucination" output, and thus attracting attention in the construction of large models in vertical fields. However, current RAG is mostly based on databases, and the efficiency and accuracy of the retrieval process are low, and it is easy to retrieve redundant data.
[0004] To solve the above problems, it is necessary to provide a reasoning tree generation method based on a clustering algorithm and a large model. SUMMARY
[0005] In view of the problems of low retrieval efficiency and accuracy of RAG, and easy retrieval of redundant data, the application provides a reasoning tree generation method and system based on a clustering algorithm and a large model, which limits the knowledge graph retrieval range and stops the reasoning tree expansion in time through the clustering algorithm and the large model, effectively improving the retrieval efficiency and progress, and reducing the query of redundant data.
[0006] To achieve the above purpose, the application adopts the following technical solutions: In the first aspect, the application provides a reasoning tree generation method based on a clustering algorithm and a large model, including the following steps: (1) extracting hyperedges and entities from unstructured text data used to construct a knowledge graph using a large model, each hyperedge corresponding to a text block in the unstructured text data, and each text block corresponding to one or more hyperedges; (2) using a hypergraph modularity clustering algorithm to divide similar hyperedges into the same cluster and recording the clustering results; (3) receiving a user input natural language question, extracting keywords of the question using a large model, and searching for a batch of candidate seed hyperedge sets according to the vector similarity of the keywords and the hyperedges; (4) Take the superedge with the highest similarity from the candidate seed superedge set as the starting point of the reasoning tree, find adjacent superedges through entity association, preferentially select superedges belonging to the same cluster as the last superedge added to the reasoning tree for expansion, and update the reasoning tree; if there is no adjacent superedge to expand the reasoning tree and the upper limit has not been reached, add the superedges in the candidate seed superedge set to the reasoning tree in order of similarity from high to low; (5) When the reasoning tree is updated, build a context containing superedges, entities, and text blocks corresponding to the superedges, submit the large model to determine whether to continue expanding or directly give an answer; if it continues to expand, return to step (4).
[0007] Further, the unstructured text data is stored in a block form, each text block containing content text and a text block index, used to build a hypergraph; the superedges and entities are stored in a vector database in the form of a bipartite hypergraph.
[0008] Further, the superedges and entities are extracted by the large model through a preset prompt word template, the superedge represents an n-ary relationship between entities, which includes an original text segment, an entity subset, and a superedge score, the original text segment is a part of the text block, and the entity subset is a subset of the entire entity set.
[0009] Further, in step (2), the similar superedges are divided into the same cluster by using a hypergraph modularity clustering algorithm, specifically including: For the current cluster module division, the maximum number of entity vertices of each superedge falling into the same module is calculated, and when the value exceeds half of the total entity vertices of the superedge, the superedge is determined to support the current division; the module refers to the modular grouping formed after the hypergraph clustering, and each module contains a group of superedges with similar semantics; The expected value of vertex distribution under the given superedge size and module volume is estimated by a binomial distribution, as a degree penalty; The real edge contribution is compared with the degree penalty, and the hypergraph modularity is calculated by combining the standardization parameter, the module division is adjusted by iteration to maximize the hypergraph modularity value, and the clustering result is output.
[0010] Further, the process of expanding the reasoning tree in step (4) includes: Find all related entities of the current reasoning tree, use the entities as a bridge to search for adjacent superedges associated with the entities in the hypergraph; Under the premise of preferentially selecting superedges belonging to the same cluster as the last superedge added to the reasoning tree to join the reasoning tree, the adjacent superedges retrieved are sorted, and the sorting priority is: preferentially according to the topological structure characteristic value, and then according to the semantic accumulation characteristic value; the topological structure characteristic value refers to the degree of the superedge in the knowledge graph, and the semantic accumulation value refers to the number of times the superedge appears in the knowledge graph construction process.
[0011] Further, when the inference tree size reaches a preset upper limit, the expansion is terminated and a large model internal knowledge answer or a refusal answer is enabled.
[0012] Further, when the seed hyperedge and the adjacent hyperedge are exhausted, it is determined that the inference fails and a large model internal knowledge answer or a refusal answer is enabled.
[0013] Further, the large model judgment process in step (5) is specifically: Starting from the hyperedge in the inference tree, the associated entities are obtained and are sorted in priority according to a topological structure characteristic value and secondly according to a semantic accumulation characteristic value; the topological structure characteristic value refers to the degree of the entity in the knowledge graph, and the semantic accumulation value refers to the number of times of appearance of the entity in the knowledge graph construction process; meanwhile, the original text block corresponding to the hyperedge is retrieved to form three parts of contents of the hyperedge, the entity and the original text block; The three parts of contents are integrated into structured data in a csv format as inference basis, the inference basis is embedded in a preset query template, and a large model is called to answer; The answer result of the large model is parsed, if the answer result shows that the inference basis is sufficient, the inference tree is stopped from being expanded and the answer is returned, otherwise the inference tree is continuously expanded.
[0014] Further, the keywords corresponding to the natural language question input by the user and the final answer output by the large model are added to the cache for subsequent same or similar query quick response.
[0015] The second part discloses an inference tree generation system based on a clustering algorithm and a large model, which is used for realizing the inference tree generation method based on the clustering algorithm and the large model.
[0016] The present application has the following beneficial effects: The present application discloses an inference tree generation method based on a clustering algorithm and a large model, which realizes an efficient knowledge graph retrieval mode by limiting the knowledge graph retrieval range and timely stopping the inference tree expansion, has good practical value and good reference significance.
[0017] The present application improves the knowledge graph retrieval efficiency and optimizes the retrieval process. The present application enhances the timeliness of the inference tree termination through the large model technology and guarantees the accuracy and credibility of the inference result. BRIEF DESCRIPTION OF DRAWINGS
[0018] Figure 1 FIG. 1 is a flowchart of the inference process based on the clustering algorithm and the large model of the present application; Figure 2 FIG. 2 is a flowchart of the generation process of the inference tree of the present application. DETAILED DESCRIPTION
[0019] The application will be further described and illustrated with reference to the specific embodiments. The embodiments are only exemplary and do not limit the scope of the disclosure. The technical features of various embodiments of the application can be combined accordingly without conflict.
[0020] As shown in Figure 1 The application proposes a reasoning tree generation method based on a clustering algorithm and a large model, and the specific process is as follows: S1, using a large model to extract hyperedges and entities from the original text; wherein the hyperedges and entities are stored in the corresponding text vector representation using bipartite hypergraph; S2, using a hypergraph module degree clustering algorithm to divide similar hyperedges into the same cluster, and recording in the corresponding file; S3, using a large model to extract the keywords of the question, and searching for related hyperedges as seed hyperedges according to the vector similarity; S4, taking the seed hyperedge as the starting point, finding the adjacent hyperedge to join the reasoning tree, and preferentially selecting the hyperedge belonging to the same cluster as the last time the reasoning tree is joined.
[0021] S5, when the reasoning tree is updated, constructing the corresponding context, submitting the large model, and determining whether to stop expanding or directly giving the answer by the large model.
[0022] Each part is as follows: Step S1 aims to extract structured knowledge from unstructured text data and construct a bipartite hypergraph knowledge graph, and the specific implementation includes: S11, text preprocessing: Unstructured original text data (such as vertical field technical documents) is stored in block form, as a text block number file and a text block file. In this embodiment, the text block number format is as follows: {“__id__”: “…”}, and the text block format is as follows: {“text block id”:{“tokens”: “…”, “content”: “…”, “chunk_order_index”: “…”,“full_doc_id”: “…”,} Each text block contains a unique identifier, content text, and its corresponding file number index.
[0023] S12, hyperedge and entity extraction Using a large model combined with a preset prompt word template to extract hyperedges and entities from text blocks. Specifically, the hyperedge represents an n-ary relationship, and in some embodiments, the hyperedge format is as follows: {“__id__”: “…”, “hyperedge_name”:“…”}, and the entity format is as follows: {“__id__”: “…”, “entity_name”: “…”}, the entity is stored separately and is associated with the corresponding hyperedge.
[0024] The hyperedge contains an original text segment, a connotative entity subset, and a hyperedge score, wherein the original text segment is a partial segment in the text block, and the connotative entity subset is a subset of the entire entity set.
[0025] S13, knowledge storage The hyperedge and the entity are stored in a vector database in the form of a bipartite hypergraph, each entity and hyperedge has a corresponding vector representation, and the entities are connected by the hyperedge to form a structured knowledge network.
[0026] Step S2 aims to group hyperedges with similar semantics through a hypergraph modularity clustering algorithm to reduce reasoning redundancy. The specific process includes: S21, edge contribution calculation The edge contribution is calculated by the maximum number of vertices of each hyperedge falling into the same module and the total number of vertices of the edge, which is used to measure the degree to which the hyperedge supports the current division. Specifically, for the current module division, the maximum number of entity vertices of each hyperedge falling into the same module is counted, and if the number is greater than half of the total number of vertices, it is determined that the hyperedge supports the current division, indicating that most of the vertices of the edge belong to the same module.
[0027] S22, degree penalty estimation Based on the assumption of random distribution, the expected probability of vertex distribution under the given hyperedge size and module volume is estimated by binomial distribution as the degree penalty; the degree penalty is the expected part of the module degree, which is used to compare with the real edge number to measure the deviation of the real distribution from the random distribution; S23, module degree optimization Finally, the hypergraph modularity is calculated according to the real edge contribution, the degree penalty, and the standardization parameter; the module division is adjusted iteratively (such as module merging or splitting) to maximize the modularity until the final division is determined, and the division result is recorded.
[0028] In some embodiments of the present application, the clustering storage format is as follows: {“__id__”: “…”, “hyperedge_name”: “…”, “cluster_id”: “…”}.
[0029] Step S3 is used to process user queries and locate the reasoning starting point.
[0030] S31, keyword extraction After receiving the user's natural language question, the present application extracts the keywords of the question, such as entities or action phrases, by using a prompt word template and a large model.
[0031] S32, similarity retrieval The cosine similarity between the keywords and the hyperedge vector representation is calculated, and the top N hyperedges with the highest similarity are retrieved from the hypergraph to form a candidate seed hyperedge set. This process ensures efficiency through vectorized retrieval and provides an accurate starting point for the inference tree.
[0032] Step S4 expands the inference tree based on the seed hyperedge using a clustering priority strategy. The specific implementation is as follows: S41, adjacent hyperedge discovery The hyperedge with the highest similarity is taken from the candidate seed hyperedge set as the starting point of the inference tree, and adjacent hyperedges are searched. Specifically, all related entities of the current inference tree are searched, and adjacent hyperedges are searched using entities as a stepping stone. The selected adjacent hyperedge is selected from all adjacent hyperedges of the inference tree.
[0033] S42, adjacent hyperedge expansion Under the premise of preferentially selecting hyperedges belonging to the same cluster as the last added to the inference tree, the retrieved adjacent hyperedges are sorted in priority order: first by topological structure feature value, and second by semantic accumulation feature value. Here, the topological structure feature value refers to the degree of the hyperedge in the knowledge graph, and the semantic accumulation value refers to the number of times the hyperedge appears in the knowledge graph construction process. If the inference tree has no expandable adjacent hyperedges and has not reached the upper limit, the hyperedges in the candidate seed hyperedge set are added to the inference tree in order of similarity from high to low; The size of the inference tree has a predetermined upper limit. When the upper limit is reached, the internal knowledge is used to answer or reject the answer.
[0034] When the seed hyperedge and adjacent hyperedge are exhausted, the reasoning fails, and the internal knowledge is used to answer or reject the answer.
[0035] The dynamic expansion mechanism of the inference tree is shown in Figure 2 The initial inference tree is taken as the starting point, and the large model (LLM) is used for judgment to intelligently control the expansion of the inference tree, achieving efficient and accurate question answering. Starting from the initial inference tree, the tree contains the seed hyperedge and its associated entities selected through the previous steps. The structured information of the inference tree is converted into natural language text, and the knowledge of hyperedges, entities, etc. is integrated into the prompt content that the LLM can understand. The converted text is submitted as text input to the LLM for processing. The LLM evaluates the current inference tree information and produces a judgment result. This judgment branch is as follows: Clear answer: If the LLM believes that the information contained in the current inference tree is sufficient, it can directly answer the question and return the result, and the entire reasoning process is successfully terminated.
[0036] Unknown (unknown): if the LLM considers that the current inference tree is insufficient information to draw a clear answer, the flow enters the next control link. At this time, a second judgment is made to check if the size of the inference tree reaches the maximum number of nodes, if it reaches the maximum number of nodes, it is no longer allowed to expand, and the system will answer based on the internal knowledge of the LLM. This safety mechanism prevents infinite loops and ensures that a response will eventually be given (which may be based on the model's own knowledge or may be "refuse to answer"). If the maximum number of nodes has not been reached, there is still room for expansion, and the operation of extending the inference tree will be performed. Return to the previous method step to find and add new adjacent hyperedges to enrich the inference tree. Then, the updated inference tree will again go through the above cycle until the output result is obtained.
[0037] Step S5 controls the expansion of the inference tree through dynamic judgment to ensure efficiency and accuracy. The specific implementation is as follows: S51, context construction When the inference tree is updated, the associated entities are obtained from the hyperedge, the associated entities are sorted according to the topological structure feature value first and the semantic accumulation feature value second, and the corresponding original text block is retrieved to form three parts of data: hyperedge, entity, and original text block. Here, the topological structure feature value refers to the degree of the entity in the knowledge graph, and the semantic accumulation value refers to the number of times the entity appears in the knowledge graph construction process.
[0038] Using the above three parts of content, a csv format content is constructed as the basis for reasoning.
[0039] S52, parse large model answer Add the reasoning basis to the query template, call the large model to answer and parse, if the reasoning basis is insufficient to give an answer, continue to expand the inference tree, if the answer can be deduced, stop expanding the inference tree, and return the answer.
[0040] In this embodiment, the answer is further added to the cache for quick response next time.
[0041] In this embodiment, a reasoning tree generation system based on clustering algorithm and large model is also provided, comprising: A knowledge extraction module for extracting hyperedges and entities from unstructured text data using a large model, each hyperedge corresponding to a text block in the unstructured text data, and each text block corresponding to one or more hyperedges; A clustering processing module for dividing similar hyperedges into the same cluster using the hypergraph module degree clustering algorithm and recording the clustering results; A query processing module for receiving a user input natural language question, extracting the keywords of the question using a large model, and searching for a set of candidate seed hyperedge sets according to the vector similarity between the keywords and the hyperedges; The reasoning tree management module is configured to take the hyperedge with the highest similarity from the candidate seed hyperedge set as the starting point of the reasoning tree, find adjacent hyperedges through entity association, preferentially select a hyperedge belonging to the same cluster as the hyperedge added to the reasoning tree last time for expansion, and update the reasoning tree; if the reasoning tree has no expandable adjacent hyperedge and has not reached the upper limit, hyperedges in the candidate seed hyperedge set are sequentially added to the reasoning tree in descending order of similarity; The judgment control module is configured to, when the reasoning tree is updated, construct a context containing the hyperedge, the entity, and the text block corresponding to the hyperedge, and submit the large model to judge whether to continue expansion or directly give an answer; if the expansion is continued, the reasoning tree management module is returned.
[0042] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts are described in the method embodiment. The implementation method of the remaining modules is not described here. The system embodiments described above are only illustrative. The units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all modules can be selected to achieve the purpose of the present application according to actual needs. Those skilled in the art can understand and implement it without creative labor.
[0043] The system embodiment of the present application can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The system embodiment can be realized by software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running through the processor of the device with data processing capability.
[0044] The above-described embodiments only express several embodiments of the present application, which are described in detail and specifically, but should not be understood as limiting the scope of the present application. For those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are within the scope of protection of the present application.
Claims
1. A method for generating inference trees based on clustering algorithms and large models, characterized in that, Includes the following steps: (1) Use large models to extract hyperedges and entities from unstructured text data used to construct knowledge graphs. Each hyperedge corresponds to a text block in the unstructured text data, and each text block corresponds to one or more hyperedges. (2) Use the hypergraph modularity clustering algorithm to divide similar hyperedges into the same cluster and record the clustering results; (3) Receive natural language questions input by users, extract keywords of the questions using a large model, and search a set of candidate seed hyperedges based on the vector similarity between the keywords and the hyperedges; (4) Take the superedge with the highest similarity from the candidate seed superedge set as the starting point of the reasoning tree, find adjacent superedges through entity association, and prioritize the selection of superedges that belong to the same cluster as the superedges added to the reasoning tree last time for expansion and update the reasoning tree; If the inference tree has no adjacent superedges that can be expanded and has not reached the upper limit, add the superedges from the candidate seed superedge set to the inference tree in descending order of similarity. (5) When the inference tree is updated, construct a context containing hyperedges, entities and the text blocks corresponding to the hyperedges, and submit it to the large model to determine whether to continue expansion or give the answer directly; if to continue expansion, return to step (4).
2. The inference tree generation method based on clustering algorithm and large model according to claim 1, characterized in that, The unstructured text data is stored in chunks, each containing the content text and a text chunk index, used to construct a hypergraph; hyperedges and entities are stored in a vector database in the form of a bipartite hypergraph.
3. The inference tree generation method based on clustering algorithm and large model according to claim 1, characterized in that, The hyperedges and entities are extracted from the large model using preset prompt word templates. The hyperedge represents an n-ary relation between entities, which includes the original text fragment, the subset of intensional entities, and the hyperedge score. The original text fragment is a part of the text block, and the subset of intensional entities is a subset of the entire entity set.
4. The inference tree generation method based on clustering algorithm and large model according to claim 1, characterized in that, In step (2), similar hyperedges are divided into the same cluster using the hypergraph modularity clustering algorithm, specifically including: For the current clustering module partitioning, calculate the maximum number of entity vertices that the entity vertices of each hyperedge fall into the same module. When this value exceeds half of the total number of entity vertices of the hyperedge, the hyperedge is considered to support the current partitioning. A module refers to a modular grouping formed after hypergraph clustering, and each module contains a set of semantically similar hyperedges. The expected value of the vertex distribution under a given hyperedge size and module volume is estimated using a binomial distribution and used as a degree penalty. The true edge contribution is compared with the degree penalty, and the hypergraph module degree is calculated by combining standardized parameters. The module partitioning is iteratively adjusted to maximize the hypergraph module degree value, and the clustering results are output.
5. The inference tree generation method based on clustering algorithm and large model according to claim 1, characterized in that, The process of expanding the reasoning tree in step (4) includes: Find all relevant entities in the current inference tree, and use these entities as stepping stones to retrieve adjacent hyperedges associated with the entities in the hypergraph; Under the premise of prioritizing the selection of hyperedges belonging to the same cluster as those previously added to the inference tree, the retrieved adjacent hyperedges are sorted with the following priority: first by topological structure feature value, and second by semantic accumulation feature value. The topological structure feature value refers to the degree of the hyperedge in the knowledge graph, and the semantic accumulation value refers to the number of times the hyperedge appears during the knowledge graph construction process.
6. The inference tree generation method based on clustering algorithm and large model according to claim 5, characterized in that, When the size of the reasoning tree reaches the preset limit, the expansion is terminated and the knowledge within the large model is used to answer or refuse to answer.
7. The inference tree generation method based on clustering algorithm and large model according to claim 5, characterized in that, When both the seed hyperedge and adjacent hyperedges are exhausted, the reasoning is deemed to have failed, and the internal knowledge of the large model is used to answer or the answer is rejected.
8. The inference tree generation method based on clustering algorithm and large model according to claim 1, characterized in that, The large model judgment process in step (5) is as follows: Starting from the hyperedges in the reasoning tree, related entities are obtained and sorted first by topological structure feature value, and then by semantic accumulation feature value. The topological structure feature value refers to the degree of the entity in the knowledge graph, and the semantic accumulation value refers to the number of times the entity appears in the knowledge graph construction process. At the same time, the original text block corresponding to the hyperedge is retrieved to form three parts: hyperedge, entity, and original text block. The three parts are integrated into structured data in CSV format, which serves as the basis for reasoning. The reasoning basis is then embedded into a preset query template, and the large model is invoked to provide the answer. Analyze the response results of the large model. If the response results indicate sufficient reasoning, stop expanding the reasoning tree and return the answer; otherwise, continue expanding the reasoning tree.
9. The inference tree generation method based on clustering algorithm and large model according to claim 8, characterized in that, The keywords corresponding to the natural language question input by the user and the final answer output by the large model are added to the cache for quick response to subsequent identical or similar queries.
10. A reasoning tree generation system based on clustering algorithms and large models, used to implement the reasoning tree generation method based on clustering algorithms and large models as described in claim 1, characterized in that, The system includes: The knowledge extraction module is used to extract hyperedges and entities from unstructured text data using a large model. Each hyperedge corresponds to a text block in the unstructured text data, and each text block corresponds to one or more hyperedges. The clustering module is used to divide similar hyperedges into the same cluster using the hypergraph modularity clustering algorithm and record the clustering results. The query processing module is used to receive natural language questions input by users, extract keywords from the questions using a large model, and search for a set of candidate seed hyperedges based on the vector similarity between the keywords and the hyperedges. The reasoning tree management module is used to extract the hyperedge with the highest similarity from the candidate seed hyperedge set as the starting point of the reasoning tree, find adjacent hyperedges through entity association, prioritize the selection of hyperedges that belong to the same cluster as the hyperedges that were added to the reasoning tree last time for expansion, and update the reasoning tree. If the inference tree has no adjacent superedges that can be expanded and has not reached the upper limit, add the superedges from the candidate seed superedge set to the inference tree in descending order of similarity. The decision control module is used to construct a context containing hyperedges, entities, and the text blocks corresponding to the hyperedges when updating the inference tree, and submit it to the large model to determine whether to continue expansion or directly provide an answer; if expansion continues, it returns to the inference tree management module.