End-to-end construction method for multi-level mesh knowledge graph for multiple long documents

Abstract

The invention provides a multi-level mesh knowledge graph end-to-end construction method for a plurality of long documents, which is suitable for the plurality of long documents and comprises the steps of document slicing and preprocessing; a label layer construction and combination step: integrating the initial keywords of all the effective fragments to form an original keyword set; the original keyword set is input into a dynamic spectral clustering double-judgment merging module to be processed, and a tag set is generated by merging keywords; wherein the processing of the dynamic spectral clustering double-judgment merging module comprises dynamic capacity spectral clustering and double-judgment merging; a node layer merging and map generation step: inputting the initial node set after self-supervision optimization into the dynamic spectral clustering double-judgment merging module for processing, and generating a final node set through merging nodes; and integrating the final node set, the final relationship set and the label layer graph, and constructing a multi-stage mesh knowledge graph.

Description

Technical Field

[0001] This invention relates to artificial intelligence large language model technology, and particularly to knowledge graph construction technology for multiple long documents. Background Technology

[0002] With the continuous evolution of artificial intelligence technology, knowledge graphs have become one of the core infrastructures supporting semantic understanding, intelligent reasoning, and large-scale model knowledge enhancement, and are widely used in many high-tech fields such as education, healthcare, and finance. However, traditional knowledge graph construction methods rely heavily on expert manual design and annotation, resulting in problems such as low efficiency, poor scalability, and high costs.

[0003] Although automatic extraction methods based on Large Language Models (LLMs) have alleviated the human burden to some extent in recent years, they still face significant challenges: most systems cannot achieve true end-to-end automation and still require manual intervention in node cleaning, relationship alignment, and hierarchical division; the generated knowledge structures are often limited to tree-like or flat network representations, making it difficult to reflect the multi-level associations and complex semantic interactions that are widespread in real knowledge systems; on the other hand, invalid content in documents (such as headers, footers, appendices, etc.) and text slicing caused by the input length limit of large language models further introduce noise and redundancy, resulting in knowledge fragmentation, entity duplication, and semantic inconsistency, which seriously restricts the quality and application of knowledge graphs.

[0004] The aforementioned problems are particularly prominent in the construction of knowledge graphs in the field of education.

[0005] Against the backdrop of current digitalization in education and teaching reform, curriculum systems are increasingly evolving towards interdisciplinary integration, diversified resources, and personalized learning. Many courses no longer rely solely on a single textbook but integrate multiple reference books, academic literature, experimental cases, and other materials to form a more open and dynamic knowledge resource network. However, traditional knowledge graph construction methods mostly still rely on a single textbook, failing to fully adapt to the demands of the new curriculum system for multi-source, heterogeneous, and cross-media knowledge integration, and facing several key limitations in practical application:

[0006] New courses typically encompass multiple textbooks, academic papers, and online resources, resulting in diverse knowledge sources, varying structures, and differing presentation styles. Traditional knowledge graph generation methods often extract entities and relationships based on the chapter structure of a single textbook, lacking mechanisms for semantic alignment and redundancy resolution across multi-source knowledge systems. This leads to conceptual conflicts, repetitions, or missing connections in different materials, making it difficult to form a logically consistent knowledge network.

[0007] Although multiple reference materials support a course, their knowledge organization may be based on different logics (such as by topic, historical context, problem orientation, etc.). Knowledge graphs generated by traditional methods are often limited to tree-like hierarchies or simple connections, making it difficult to effectively present complex semantic relationships across resources and chapters (such as concept evolution paths and interdisciplinary connections), which restricts students' holistic and in-depth understanding of the knowledge system.

[0008] In conclusion, although knowledge graphs are considered a key tool for the digital transformation of education, their current construction methods are still insufficient to fully adapt to new curriculum systems supported by multiple textbooks and documents. Future efforts need to overcome key technological bottlenecks such as multi-source knowledge fusion, dynamic semantic alignment, and cross-modal association to support more flexible, open, and adaptable smart teaching scenarios. Summary of the Invention

[0009] To address the core challenges of automated knowledge graph construction from multiple document resources, such as difficulties in multi-source knowledge fusion, flat and monotonous structural representation, and knowledge fragmentation caused by redundant noise, this paper proposes a multi-level network knowledge graph construction method based on dynamic spectral clustering and dual-decision. This method aims to achieve fully automated, end-to-end knowledge extraction, fusion, and structured organization of long documents such as textbooks, generating a hierarchical, semantically rich, and highly consistent multi-level network knowledge graph to meet the urgent need for intelligent construction of cross-resource knowledge systems.

[0010] The technical problem to be solved by this invention is an end-to-end construction method for a multi-level network knowledge graph of multiple long documents, comprising the following steps:

[0011] (1) Document slicing and preprocessing steps: Slice the input long document to obtain field fragments; extract the effective fragments and their corresponding initial keywords for each field fragment in parallel; (2) Tag layer construction and merging steps: (2.1) Integrate the initial keywords of all effective segments to form the original keyword set; (2.2) The original keyword set is input into the dynamic spectral clustering double-decision merging module for processing, and a tag set is generated by merging the keywords; wherein, the processing of the dynamic spectral clustering double-decision merging module includes: a. Dynamic capacity spectrum clustering: Cluster the original keyword set and dynamically divide the processing batches according to capacity constraints; b. Consolidation of dual judgments: For each batch, the following judgments shall be executed sequentially:

[0012] The first level of judgment: a large-scale generalization judgment based on keyword names, based on semantic similarity matching, to screen high-confidence keyword merging candidate pairs, and then hand them over to a large language model for preliminary identification and merging of keywords with the same or highly similar semantics; The second level of judgment: Based on the description, a small-scale fine-grained judgment is made by a large language model in combination with detailed information to make a final decision on the keyword merging candidate pairs and generate a tag set; (2.3) Based on the tag set, semantic relationships between tags are created through a large language model to construct a tag layer graph; (3) Node and relation extraction steps: Extract node names, node descriptions, relation source nodes, relation target nodes and relation descriptions from effective fragments using a large language model to form an initial node set and an initial relation set; and perform self-supervised optimization on the initial node set and the initial relation set to remove unreasonable or duplicate items; (4) Steps for merging node layers and generating the graph: (4.1) The initial node set after self-supervised optimization is input into the dynamic spectral clustering double-decision merging module for processing, and the final node set is generated by merging the nodes; wherein, the processing of the dynamic spectral clustering double-decision merging module includes: a. Dynamic capacity spectrum clustering: Cluster the initial node set and dynamically divide the processing batches according to capacity constraints; b. Consolidation of dual judgments: For each batch, the following judgments shall be executed sequentially: The first level of judgment: a large-scale generalized judgment based on node names, based on semantic similarity matching, to select high-confidence keyword merging candidate pairs, and then hand them over to a large language model for preliminary identification and merging of nodes with the same or highly similar semantics. The second level of judgment: a small-scale fine-grained judgment based on node descriptions, where the large language model combines node descriptions to make a final decision on the candidate pairs of node merging and generate the final set of nodes; (4.2) Update the node anchors in the initial relation set synchronously according to the final node set to obtain the final relation set; (4.3) Integrate the final node set, the final relationship set, and the tag layer graph to construct a multi-level network knowledge graph.

[0013] This invention forms a preliminary knowledge fragment and tagging system by slicing long documents into fixed-length segments, performing semantic denoising, extracting and merging keywords, and creating connections. Then, based on the powerful natural language understanding capabilities of a large language model, it automatically judges and extracts node names, descriptions, and their semantic relationships, followed by multiple rounds of rationality judgment and intelligent merging. Only a long document needs to be input to construct a complete, semantically consistent, multi-level network knowledge graph end-to-end.

[0014] The rationality judgment and intelligent merging, specifically the dynamic spectral clustering dual-judgment merging steps proposed in this invention, are as follows:

[0015] In the dynamic spectral clustering dual-judgment merging step, the extracted keywords and nodes are first subjected to dynamic capacity spectral clustering, and the processing batches are adaptively divided according to the label distribution and quantity to maximize the processing efficiency of the large language model. Then, the semantic relevance between nodes is calculated using a similarity matching algorithm to screen high-confidence candidate pairs. Then, a large-scale generalization judgment is performed based on the node name, and the model initially identifies potential similar nodes. Finally, a small-scale fine-grained judgment based on description is introduced, and the detailed description information of the nodes is integrated and handed over to the large model for deep semantic discrimination to ensure the accuracy of the merging decision. After merging, similarity verification is performed again, thereby significantly improving the conciseness and consistency of knowledge points.

[0016] The beneficial effects of this invention are that it effectively solves the problems of knowledge fragmentation and redundancy caused by text slicing in the traditional knowledge graph construction process of long documents, and breaks through the limitations of tree structures or flat networks in the expression of complex knowledge. Attached Figure Description

[0017] Figure 1 Flowchart of an end-to-end construction method for a multi-level network knowledge graph based on dynamic spectral clustering and dual-decision;

[0018] Figure 2 Flowchart of the dynamic spectral clustering double-judgment merging method;

[0019] Figure 3 A schematic diagram of the three-level architecture of the knowledge graph generated by this method;

[0020] Figure 4 This is a schematic diagram of the local interconnections of the knowledge graph generated by this method. Detailed Implementation

[0021] The example uses a textbook as a case study of a long document, illustrating an end-to-end automatic knowledge graph construction method based on a large language model. The dynamic spectral clustering double-judgment merging step is implemented through a dynamic spectral clustering double-judgment merging module. Essentially, this module is an intelligent deduplication and merging unit. Its input is the set of entities to be cleaned, and its output is the cleaned and merged set of entities. Through two calls, the first call takes the original keyword set extracted from all valid slices as input and outputs a deduplicated and merged tag set as output. The second call takes the set of all nodes extracted from all slices and optimized under self-supervised supervision, with each node including a name and description attribute. The output of the second call is a concise set of nodes after deep merging and deduplication, with synchronized relationship updates.

[0022] The dynamic spectral clustering dual-judgment merging module ensures the conciseness and consistency of the knowledge graph at both the label and node levels.

[0023] The dynamism is mainly reflected in the preparation stage of dynamic capacity spectral clustering, which aims to intelligently adapt to the imbalance of input data and optimize the calling efficiency of large language model LLM. Specifically, it is reflected in: (1) the dynamism of the number of clusters: it does not pre-determine how many clusters to form, but rather forms several clusters of different sizes naturally according to the distribution of the data itself. (2) the dynamism of batch partitioning: it dynamically packages these clusters of different sizes into several batches through a capacity constraint optimization problem and a greedy allocation strategy. Under the premise of not exceeding the single processing limit of LLM, it makes the maximum use of each batch.

[0024] The dual-judgment process employs a coarse-to-fine, two-stage verification process to ensure that the merging decision is both comprehensive and accurate.

[0025] The first judgment is a large-scale generalization judgment based on names, aiming for high recall through coarse screening. It quickly identifies all potential candidate pairs with similar names from a large batch. Specifically, a dynamically packaged list of batch keywords / node names is sent to the LLM (Local Management Module), and pre-designed prompts, such as requesting the merging of synonyms or the deletion of non-knowledge points, are used to provide initial merging suggestions to the LLM.

[0026] The second decision is a small-scale, fine-grained judgment based on the description, aiming for high accuracy through fine filtering. It makes the final decision on the candidate pairs to be merged generated in the first decision. Specifically, the keywords / nodes suggested for merging in the first decision, along with their respective detailed descriptions, are combined into a small judgment task and submitted to the LLM.

[0027] like Figure 1 As shown, the specific methods for constructing knowledge graphs are as follows:

[0028] (1) Divide the textbook content into several field segments to avoid exceeding the input length limit of the large language model; at the same time, improve the accuracy of model judgment by refining the slices.

[0029] (2) Input the slices into the intelligent filtering router in parallel. This part is responsible for guiding the large language model to determine whether the slice belongs to invalid content that does not contain knowledge structure, such as cover, introduction, table of contents, appendix, etc. If it belongs to invalid content, mark it as "skip"; if it is valid teaching content, output the clean text after removing noise such as headers, footers, and code to improve the accuracy of subsequent self-supervised extraction, and extract the keywords corresponding to the slice.

[0030] (3) Integrate the keywords of all slices and input them into the dynamic spectral clustering double judgment merging module: First, perform dynamic capacity spectral clustering on the keywords for similarity matching, and input them into the large language model in batches for large-scale generalization judgment based on name; combine the corresponding descriptions into the model for small-scale fine judgment based on description, and finally merge similar nodes and update the slice keywords.

[0031] (4) The processed keywords are subjected to dynamic capacity spectral clustering again and then fed into the large model in batches. The large model creates the association between the labels and stores them in the database.

[0032] (5) Based on the judgment result of step (2), the slices that have been marked as "skipped" are directly marked as null and stored in the database; for the remaining denoised (redundant) segments, the large language model LLM is guided by prompt words to extract the node name, node description, relation source node, relation target node, relation description and relation keywords.

[0033] (6) Input the nodes and relationships extracted in step (5) back into the large language model LLM. Guide the model to judge their rationality through prompt words, and remove nodes and relationships that are logically unreasonable or repetitive, so as to realize the self-supervised processing of LLM.

[0034] (7) Use the keywords obtained in the textbook cleaning stage as tags and assign them to the nodes extracted from the corresponding slices.

[0035] (8) Remove duplicate nodes and relationships extracted from all slices and integrate them into the database.

[0036] (9) Input all nodes into the dynamic spectral clustering double-judgment merging module again for node merging and simplification, and update the corresponding node anchor points in the relationship synchronously to ensure the uniformity and consistency of the graph structure.

[0037] (10) The course is used as the central point of the first level, the tags are used as the divergence points of the second level, and the knowledge points as nodes are used as the divergence points of the third level to construct the final multi-level network knowledge graph.

[0038] By following the above process, you only need to input the electronic textbook to generate a complete, detailed, hierarchical, network-like knowledge graph from end to end.

[0039] The specific dynamic spectral clustering double-decision merging method is given below:

[0040] Since the input length of large language models is limited, it is necessary to slice the textbook. Different segments of one or more textbooks may involve the same content, but due to translation / generalization ability of large language models, nodes with different names are extracted. To address this problem, a node cleaning method is proposed to achieve intelligent merging of similar nodes.

[0041] 1) Dynamic capacity spectral clustering

[0042] The dynamic capacity spectral clustering step models the batch partitioning problem as a capacity-constrained optimization problem. Its objective function is to maximize batch capacity utilization, and the constraint is that the number of elements in each batch does not exceed the preset number of elements in the target batch. The clustering results are dynamically packaged into batches using a greedy allocation strategy.

[0043] Transform the batch partitioning problem into a capacity-constrained optimization problem:

[0044]

[0045] Constraints:

[0046]

[0047]

[0048]

[0049] in, This indicates the total number of batches generated in the end; This indicates the total number of elements in the input cluster; Indicates the first The number of elements in each batch, where i and k are both batch sequence variables; Indicates the number of elements in the target batch; Indicates the first The clustering is calculated from input data, where j is the category index variable.

[0050] This yields the initial batches. A greedy allocation strategy is then used to maximize the capacity of each batch and reduce the number of calls to the large model.

[0051]

[0052] in, This indicates the clusters that will be merged; Indicates the current batch; This indicates that clusters will be merged into the current batch; This indicates that clustering will be postponed to a new batch.

[0053] 2) First-level judgment: Large-scale generalization judgment based on name

[0054] In the name-based large-scale generalization decision step, designed prompts and predefined merging and deletion rules guide the large language model to identify similar keywords / nodes in each cluster, perform preliminary screening of keyword / node names, and output the merging results. Specifically, this decision process follows the following core principles:

[0055] (2.1) Merging rules:

[0056] The model is required to identify nodes with the same or highly similar semantics and group them together, i.e., merge them.

[0057] Generate a concise, standard name summary for each group and list the original list of nodes to be merged (keywords). Keywords must be selected directly from the input and cannot be created manually. Each group must contain at least two nodes.

[0058] If there are no nodes to merge, output an empty array.

[0059] For example, different names representing the same concept, such as "greedy algorithm", "greedy algorithm" and its full English name "Greedy Algorithm", should be merged into a unified and standardized name, such as "greedy algorithm".

[0060] (2.2) Deletion rules:

[0061] Remove nodes that are not related to knowledge points (such as irrelevant text or erroneous entries) or nodes that are too specific to knowledge points (such as detailed descriptions rather than the concept itself).

[0062] If there are no nodes to delete, output an empty array.

[0063] (2.3) Output Specifications:

[0064] The decision is output in a structured data format, including a merge list and a delete list. Each item in the merge list contains a condensed standard name summary and a list of the original keywords / nodes to be merged, and the original name set must all come from the input data. The delete list contains all keywords / nodes to be deleted.

[0065] By applying the above rules, this step can efficiently perform preliminary cleaning of a large number of keywords / nodes, laying the foundation for subsequent detailed judgment.

[0066] 3) Semantic matching

[0067] For the keywords / nodes extracted from the large model that need to be merged, perform semantic similarity matching and find the best match from this batch:

[0068]

[0069] in, A string representing the query keyword / node to be matched; This represents the set of elements in the current batch, which contains n elements. This represents the k-th element; Indicates the element that best matches; This represents the highest similarity score. This represents the scoring function. The elements here are the corresponding keywords or nodes.

[0070] For the matched results, set a confidence threshold for filtering to avoid mistakenly merging irrelevant elements:

[0071]

[0072] in, This represents the confidence threshold. Only the best matching element that is greater than or equal to the confidence threshold can be output as the matching result. If the best matching element is less than the confidence threshold, the matching result output is empty. This represents the matching function.

[0073] 4) Second-level judgment: Small-scale fine-grained judgment based on description

[0074] The keywords / nodes are then fed back to the main model, which is required to carefully evaluate the rationality of this merging process based on the keywords / nodes and descriptions. If the merging is reasonable, these keywords / nodes are merged; otherwise, they are skipped.

[0075] Taking the node combination processing of the dynamic spectral clustering dual-decision merging module as an example, the entire process of the dynamic spectral clustering dual-decision merging module is as follows: Figure 2 As shown:

[0076] 1. Node clustering stage

[0077] 1.1 Node Data Input: First, obtain the node data.

[0078] 1.2 Grouping by Label: Group node data according to labels.

[0079] 1.3 Batch Size Judgment:

[0080] If it is a small batch, perform the merge small batch operation.

[0081] For large batches, use KMeans splitting and clustering combined with a greedy algorithm to optimize batch operations.

[0082] If it is a moderate batch, proceed directly to the next step.

[0083] 1.4 Add to queue: After the above processing, add the node to the queue to complete the node clustering stage.

[0084] 2. Node merging phase

[0085] 2.1 First LLM Analysis (based on node name only): For the nodes in the queue, the first Large Language Model (LLM) analysis is performed based on the node name.

[0086] 2.2 Merge suggestion judgment: if yes, proceed to the semantic matching step; otherwise, do not merge.

[0087] 2.3 Semantic Matching and In-Depth Validation:

[0088] After performing semantic matching, a description is introduced.

[0089] Perform description-based LLM verification (more accurate similarity judgment).

[0090] Perform semantic matching again.

[0091] 2.4 Merging Nodes: After the above verification, the final merging nodes are determined.

[0092] For the tag layer, semantic relationships between tags are created based on the tag set using a large language model to construct a tag layer graph. For the node layer, the node anchors in the initial relationship set are updated synchronously according to the final node set to obtain the final relationship set. Finally, the final node set, the final relationship set, and the tag layer graph are integrated to construct a multi-level network knowledge graph.

[0093] Experimental verification

[0094] 1) Comparison with open-source knowledge graph generation methods

[0095] Table 1 compares this method with widely used open-source knowledge graph generation projects, using average degree and average clustering coefficient as statistical indicators.

[0096] The following explains the formula for calculating the indicator:

[0097] Average degree captures global connectivity by averaging the number of edges per node. A higher value indicates a more integrated knowledge representation, enabling more efficient cross-node traversal. Its calculation formula is as follows:

[0098]

[0099] in, Represents a set of nodes. Represents a node The degree is the number of edges connected to the node.

[0100] The average clustering coefficient is commonly used to evaluate the connectivity of local neighborhoods. A higher value indicates denser local neighborhood connections, leading to more accurate retrieval. Its calculation formula is as follows:

[0101]

[0102] in, Represents a node Clustering coefficient, Represented by node The total number of triangular structures centered on the center.

[0103] As can be seen from Table 1, our method outperforms the other three open-source knowledge graph generation projects in both of these metrics.

[0104] Table 1 Comparison of map construction quality

[0105]

[0106] 2) Comparison with commonly available knowledge graphs

[0107] This study compares common knowledge graph examples on the market. The graphs from Higher Education Press and Chaoxing use a tree-like hierarchical structure to display the relationships between knowledge points, but neglect the connections between individual knowledge points. Zhihuishu offers two solutions: ring graphs and network graphs. Ring graphs suffer from the same problem as the previous two. While network graphs can show the connections between individual knowledge points, for the same book corresponding to the same course, this method extracts 138.46% more nodes and 185.71% more relationships.

[0108] Figure 3 A complete overview of the knowledge graph generated by the method of this invention is presented, which clearly shows that the entire graph is divided into a three-level architecture: the first level is centered on courses, the second level is radiating outwards as tags, and the third level is radiating outwards as nodes. From Figure 4 The partial graph of the knowledge graph generated by the method of this invention shows that each level is connected to the levels above and below, and there are also reasonable connections between the nodes within each level that have been determined by the large model.

[0109] 3) Summary of the effectiveness comparison of its own modules

[0110] Table 2 compares the number of words in the textbook text before and after textbook cleaning, the number of nodes before and after self-supervised optimization, and the number of nodes before and after the dynamic spectral clustering double-judgment merging module. Without the above modules, the generated knowledge graph has an average degree of 2.06 and an average clustering coefficient of 0.065. Compared with the data in Table 1, the average degree increased by 39.32% and the average clustering coefficient increased by 512.31% after processing with this method.

[0111] Table 2 Comparison before and after treatment

[0112]

[0113] Based on the data in Tables 1 and 2, we can conclude that this method, through the layered processing of three modules—textbook cleaning, self-supervised optimization, and dynamic spectral clustering and merging—significantly reduces the size of text and nodes while greatly improving the core topological indicators of the knowledge graph, effectively realizing the construction of a high-quality knowledge graph from massive amounts of text.

Claims

1. An end-to-end method for constructing a multi-level network knowledge graph for multiple long documents, characterized in that, Includes the following steps: (1) Document slicing and preprocessing steps: Slice multiple long input documents to obtain field fragments; extract valid fragments and their corresponding initial keywords for each field fragment in parallel; (2) Tag layer construction and merging steps: (2.1) Integrate the initial keywords of all effective segments to form the original keyword set; (2.2) The original keyword set is input into the dynamic spectral clustering double-decision merging module for processing, and a tag set is generated by merging the keywords; wherein, the processing of the dynamic spectral clustering double-decision merging module includes: a. Dynamic capacity spectrum clustering: Cluster the original keyword set and dynamically divide the processing batches according to capacity constraints; b. Consolidation of dual judgments: For each batch, the following judgments shall be executed sequentially: The first level of judgment: a large-scale generalization judgment based on keyword names, based on semantic similarity matching, to screen high-confidence keyword merging candidate pairs, and then hand them over to a large language model for preliminary identification and merging of keywords with the same or highly similar semantics; The second level of judgment: Based on the description, a small-scale fine-grained judgment is made by a large language model in combination with detailed information to make a final decision on the keyword merging candidate pairs and generate a tag set; (2.3) Based on the tag set, semantic relationships between tags are created through a large language model to construct a tag layer graph; (3) Node and relation extraction steps: Extract node names, node descriptions, relation source nodes, relation target nodes and relation descriptions from effective fragments using a large language model to form an initial node set and an initial relation set; and perform self-supervised optimization on the initial node set and the initial relation set to remove unreasonable or duplicate items; (4) Steps for merging node layers and generating the graph: (4.1) The initial node set after self-supervised optimization is input into the dynamic spectral clustering double-decision merging module for processing, and the final node set is generated by merging the nodes; wherein, the processing of the dynamic spectral clustering double-decision merging module includes: a. Dynamic capacity spectrum clustering: Cluster the initial node set and dynamically divide the processing batches according to capacity constraints; b. Consolidation of dual judgments: For each batch, the following judgments shall be executed sequentially: The first level of judgment: a large-scale generalized judgment based on node names, based on semantic similarity matching, to select high-confidence keyword merging candidate pairs, and then hand them over to a large language model for preliminary identification and merging of nodes with the same or highly similar semantics. Second-level decision: Small-scale fine-grained decision based on node description, the large language model combined with node description makes the final decision on the node merging candidate pairs to generate the final node set; (4.2) The node anchors in the initial relation set are updated synchronously according to the final node set to obtain the final relation set; (4.3) Integrate the final node set, the final relationship set, and the tag layer graph to construct a multi-level network knowledge graph.

2. The method according to claim 1, characterized in that, The dynamic capacity spectrum clustering step models the batch partitioning problem as a capacity-constrained optimization problem. Its objective function is to maximize the batch capacity utilization rate, and the constraint is that the number of elements in each batch does not exceed the preset number of elements in the target batch. The clustering results are dynamically packaged into batches using a greedy allocation strategy.

3. The method according to claim 1, characterized in that, In the name-based large-scale generalization decision, the large language model is guided to perform the following operations using predefined prompt words: Merging rules: Identify nodes or keyword names that are semantically identical or highly similar, generate a standard name for each group, and list the original names to be merged; Deletion rules: Delete entries that are not knowledge points or are too specific; the judgment results are output in the format of structured merge lists and deletion lists.

4. The method according to claim 1, characterized in that, The semantic similarity matching is achieved by calculating the similarity score between the query string and the set of elements within the batch, and its function is expressed as: ; in, This represents the query string to be matched. This represents the set of elements in the current batch, which contains n elements. This represents the k-th element; Indicates the element that best matches; This represents the highest similarity score. Represents the scoring function; A threshold is set for filtering, and only the best matching elements with a confidence level greater than or equal to the confidence threshold can be used as high-confidence node merging candidate pairs.

Images