Method, system and device for managing and scheduling based on multi-level knowledge aggregation data

Abstract

The application relates to the technical field of electric digital data processing, and particularly discloses a management and scheduling method, system and device based on multi-level knowledge aggregation data. A server computing port is used to pre-process an input document, cut the document into text blocks based on the original structure of the document, and mark three-dimensional metadata to form a text block set with a link relationship. Then, two-dimensional characteristic data of each text block is obtained, a double-dimensional fusion adaptation index is determined, a matching composite vector weight fusion proportion is matched, vectorization processing is performed on each text block, and a composite vector is obtained. Based on the composite vector, hierarchical clustering processing is performed to obtain hierarchical clustering units, and a clustering quality index is evaluated. According to the index, secondary splitting is performed on the clustering units. Meanwhile, multi-level summarization is performed on each hierarchical clustering unit to obtain hierarchical clustering summaries. A server user port extracts the granularity of a user question, schedules each hierarchical clustering summary, and performs hierarchical boundary automatic checking in the scheduling process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital data processing, in particular to a management and scheduling method, system and device based on multi-level knowledge aggregation data. BACKGROUND

[0002] In the prior art, to solve the problem that traditional RAG (Retrieval Augmentation Generation) cannot obtain a global perspective, mainly rely on methods such as graph RAG and long text RAG. Graph RAG constructs an entity knowledge graph, divides communities and generates summaries, integrates information using community summaries to improve the comprehensiveness and diversity of answers; long text RAG uses hybrid retriever, LLM enhanced information extractor and other components to integrate global information and factual details, enhance the understanding and answering ability of long text context questions, and thus make up for the limitations of traditional RAG.

[0003] For example, the Chinese invention patent with publication number CN118673126A discloses a RAG question and answer method, system and medium based on a knowledge graph, belonging to the technical field of natural language processing, which is used to solve the following problems: there is an urgent need for new technical means to fully utilize the structured information and global perspective of the knowledge graph, and to improve the accuracy, comprehensiveness and consistency of the RAG system when processing complex problems. The method includes: preprocessing the pre-stored knowledge base file; then performing entity information extraction to obtain entity nodes and entity node description vectors; performing community detection and clustering on all entity nodes to obtain community reports; constructing a knowledge graph index structure according to the entity nodes, entity node description vectors and community reports; determining the associated entity nodes according to the similarity between the user query vector and the entity node description vector; then performing related information mining in the knowledge graph index structure, and inputting the mined related information into a large language model to generate an answer.

[0004] For example, the Chinese invention patent with publication number CN118503350A discloses a process optimization design method and system for improving the accuracy of a large model RAG, belonging to the technical field of large models. The following technical solutions are implemented: extracting text information from a document, dividing the text information into text blocks according to the number of characters; extracting summaries from the text blocks through a summary extraction model, and performing text vectorization on the extracted summaries through a text vectorization model and storing them in a vector library; a user inputs a question, processes the user question and vectorizes it; according to the vectorized user question, search in the vector library and filter the search results to remove irrelevant data; input the searched knowledge and the user question into a large model for answering.

[0005] In combination with the above technical solutions, it is found that the existing RAG optimization schemes mostly rely on fixed word number segmentation of text blocks and fixed level clustering in the clustering process. Since the RAG optimization method needs to consider the inherent hierarchical relationship of multi-level knowledge itself, these existing methods not only destroy the hierarchical independence and structure of multi-level knowledge, making it difficult to adapt to the inherent logic of multi-level knowledge, but also blur the hierarchical boundaries, making it difficult to accurately locate the knowledge of a specific level during management and scheduling, affecting the efficiency and accuracy of multi-level knowledge scheduling in RAG optimization. SUMMARY

[0006] In view of the deficiencies of the prior art, the present application provides a management and scheduling method, system and device based on multi-level knowledge aggregated data, which can effectively solve the problems involved in the background art.

[0007] To achieve the above object, the present application is implemented by the following technical solutions: The first aspect of the present application provides a management and scheduling method based on multi-level knowledge aggregated data, comprising: a server computing port pre-processes an input document, performs cutting processing on the input document based on the native structure of the input document, obtains each text block of the input document, labels three-dimensional metadata for each text block, and forms a text block set with link relationships; obtaining the two-dimensional characteristic data of each text block in the text block set, determining the two-dimensional fusion adaptation index of each text block, matching to obtain the composite vector weight fusion proportion, performing vectorization processing on each text block to obtain the composite vector of each text block; performing hierarchical clustering processing based on the composite vector of each text block to obtain each hierarchical clustering unit, evaluating the clustering quality index of each hierarchical clustering unit, and performing secondary splitting of each hierarchical clustering unit based on the clustering quality index; performing multi-level summarization on each hierarchical clustering unit to obtain each hierarchical clustering summary, the server user port extracts the granularity of the user question, schedules each hierarchical clustering summary, and simultaneously performs hierarchical boundary automatic verification during the scheduling process.

[0008] The second aspect of this invention provides a management and scheduling system based on multi-level knowledge aggregation data, comprising: a text block segmentation module, used by the server computing port to preprocess the input document, segment the input document based on its original structure to obtain text blocks, annotate each text block with three-dimensional metadata, and form a set of text blocks with linked relationships; a composite vector processing module, used to obtain the two-dimensional characteristic data of each text block in the text block set, determine the two-dimensional fusion adaptation index of each text block, match the composite vector weight fusion ratio, perform vectorization processing on each text block to obtain a composite vector of each text block; a clustering quality evaluation module, used to perform hierarchical clustering processing based on the composite vector of each text block to obtain each level of clustering units, evaluate the clustering quality index of each level of clustering units, and perform secondary splitting on each level of clustering units based on the clustering quality index; and a multi-level scheduling module, used to perform multi-level summarization on each level of clustering units to obtain each level of clustering summary, the server user port extracts the granularity of user questions, schedules each level of clustering summary, and performs automatic hierarchical boundary verification during the scheduling process.

[0009] A third aspect of the present invention provides an electronic device, characterized in that the electronic device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the above-described method.

[0010] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0011] (1) This invention provides a management and scheduling method, system, and device based on multi-level knowledge aggregation data. The server computing port preprocesses the input document, cuts it into text blocks based on its original structure, and annotates three-dimensional metadata to form a set of text blocks with linked relationships. Subsequently, it acquires the two-dimensional characteristic data of each text block, determines the two-dimensional fusion adaptation index, matches the composite vector weight fusion ratio, and performs vectorization processing on each text block to obtain a composite vector. Based on the composite vector, it performs hierarchical clustering processing to obtain clustering units at each level, evaluates the clustering quality index, and performs secondary splitting of the clustering units according to the index. At the same time, it performs multi-level summarization on each level clustering unit to obtain each level cluster summary. The server user port extracts the granularity of user questions, schedules each level cluster summary, and performs automatic hierarchical boundary verification during the scheduling process.

[0012] (2) By dynamically adjusting the semantic similarity threshold and the center similarity threshold, this invention can flexibly perform secondary splitting based on the specific situation of the current clustering unit. For example, when the average semantic similarity within a sub-cluster is lower than the initial threshold, the semantic similarity adaptation threshold is dynamically adjusted by calculating the deviation and mapping it to obtain the correction parameter, thereby ensuring the cohesion of the sub-cluster. This dynamic adjustment mechanism not only avoids the problems of insufficient or excessive clustering that may be caused by fixed thresholds, but also adaptively optimizes the granularity and quality of the clustering results according to the actual distribution characteristics of the text data. In addition, when the center vector similarity of different sub-clusters is too high, the clustering discrimination is further optimized by merging and then splitting again, ensuring the rationality and consistency of the clustering results.

[0013] (3) The present invention utilizes the two-dimensional fusion adaptation index multiple times throughout various stages, including text block vectorization, clustering quality assessment, and hierarchical scheduling. In the vectorization stage, the two-dimensional fusion adaptation index is used to match the fusion ratio of composite vector weights, ensuring a more reasonable fusion of semantic vectors and hierarchical feature vectors, thus generating more representative composite vectors. In the clustering quality assessment stage, the two-dimensional fusion adaptation index is used to calculate clustering quality indicators, providing a basis for secondary splitting. In the hierarchical scheduling stage, the two-dimensional fusion adaptation index is used again to determine the granularity range of the user's question, thereby selecting the most suitable scheduling strategy. This multiple-use mechanism allows the two-dimensional fusion adaptation index to fully play its role in multi-level knowledge aggregation, not only improving the semantic and hierarchical feature fusion effect of text blocks but also optimizing clustering quality and scheduling accuracy, resulting in a comprehensive improvement in the overall system performance.

[0014] (4) Most existing technologies rely on fixed-number text block segmentation and fixed-level clustering, which destroys the hierarchical independence and structure of multi-level knowledge and is difficult to adapt to the inherent logic of multi-level knowledge. However, this invention preserves the hierarchical relationship and semantic structure of the text by segmenting text blocks based on the original structure of the input document and annotating with three-dimensional metadata, thus ensuring the integrity and independence of multi-level knowledge. Secondly, existing technologies lack dynamic evaluation and optimization mechanisms for clustering quality during the clustering process, which can easily lead to inaccurate clustering results. This invention can effectively improve the quality and adaptability of clustering results by introducing clustering quality indicators and a secondary splitting mechanism. In addition, existing technologies have difficulty accurately locating knowledge at specific levels during the scheduling process, while this invention can ensure the accuracy and consistency of scheduling results by generating target level paths and automatically verifying execution level boundaries. These improvements enable this invention to not only improve efficiency and accuracy when processing complex multi-level knowledge aggregation data, but also enhance the flexibility and scalability of the system, making it suitable for various application scenarios, such as text classification, topic mining, and knowledge graph construction. Attached Figure Description

[0015] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the method steps of the present invention.

[0017] Figure 2 This is a schematic diagram of the system module connections of the present invention.

[0018] Figure 3 This is a flowchart of the document preprocessing process.

[0019] Figure 4 This is a flowchart of the hierarchical clustering process. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0021] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0022] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0023] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0024] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0025] Reference Figure 1As shown, the first aspect of the present invention provides a management and scheduling method based on multi-level knowledge aggregation data, including: preprocessing the input document on the server computing port, performing segmentation processing on the input document based on the original structure of the input document to obtain each text block of the input document, and labeling each text block with three-dimensional metadata to form a set of text blocks with linked relationships.

[0026] The specific process of splitting the above text block set is as follows: Figure 3 As shown, Figure 3 The document preprocessing flowchart illustrates the process: After receiving the input document, the server's computing port first segments the document based on its original structure (such as paragraphs and chapters) to obtain individual text blocks. Subsequently, each text block is annotated with three-dimensional metadata, including a unique hierarchical path code, the link relationship between parent and child blocks, and expiration tags (such as segmentation timestamps and text block validity periods), ultimately forming a set of linked text blocks.

[0027] Specifically, the formation process of a collection of linked text blocks is as follows:

[0028] The original structure of the input document is paragraphs and chapters, and the input document is divided into several text blocks according to the paragraphs and chapters.

[0029] The annotation of three-dimensional metadata for each text block specifically includes:

[0030] The first dimension: assigning a unique hierarchical path encoding to each text block.

[0031] The second dimension: link each text block upwards to its parent block and downwards to its child blocks.

[0032] The third dimension: bind time-limited tags to each text block, including the cutting timestamp and the validity period of the text block.

[0033] Finally, the text blocks are integrated into a set of text blocks with linked relationships.

[0034] Obtain the two-dimensional characteristic data of each text block in the text block set, determine the two-dimensional fusion adaptation index of each text block, match the composite vector weight fusion ratio, perform vectorization processing on each text block, and obtain the composite vector of each text block.

[0035] Furthermore, vectorization processing is performed on each text block. The specific processing procedure is as follows:

[0036] Each text block is converted into a vector using a text embedding model, denoted as the semantic vector of each text block;

[0037] Extract the hierarchical feature data of each text block, including the hierarchical depth of each text block, the semantic similarity of the parent level of each text block, and the distribution density of the same level of each text block, to form the hierarchical feature vector of each text block. The hierarchical feature data can be directly extracted from the hierarchical tags in the document preprocessing stage of the server computing port.

[0038] The text embedding model described above converts each text block into a vector. Specifically, the text embedding model encodes the semantic information of the text block by converting the text block into a "token sequence" (the smallest processing unit of the model) that the model can recognize, thereby transforming unstructured text into structured numerical vectors.

[0039] The two-dimensional fusion adaptation index of each text block is averaged to obtain the document's two-dimensional fusion adaptation index. This index is then matched with the predefined composite vector weight fusion ratio corresponding to each two-dimensional fusion adaptation index interval to determine the interval to which the document's two-dimensional fusion adaptation index belongs, and the composite vector weight fusion ratio corresponding to that interval is obtained.

[0040] The semantic vectors of each text block and the hierarchical feature vectors of each text block are fused together according to the composite vector weight fusion ratio to form a hierarchical-semantic composite vector, thus obtaining the composite vector of each text block. The composite vector weight fusion ratio represents the proportion of the semantic vector in the fusion process.

[0041] The fusion process described above involves expanding the hierarchical feature vector to the dimension of the semantic vector through repeated padding or linear mapping (e.g., expanding a 3-dimensional hierarchical feature vector to 1536 dimensions). The expanded values ​​maintain the proportional relationship of the original hierarchical features (e.g., if the original dimension is 2, the values ​​at all corresponding positions after expansion are scaled by 2 / 3, ensuring that the hierarchical features can be recognized in the high-dimensional space). The expanded hierarchical feature vector and the semantic vector are then fused according to the composite vector weight ratio, with a weighted sum of α and 1-α. Example: If α = 0.7, the semantic vector has a dimension value of 0.3, and the corresponding dimension of the expanded hierarchical feature vector has a value of 0.2, then the composite vector for that dimension has the value of: 0.7 × 0.3 + 0.3 × 0.2 = 0.27.

[0042] The timeliness weight adaptation factor of each text block is obtained and coupled with the composite vector of each text block. Specifically, the timeliness weight adaptation factor of each text block is multiplied with the composite vector of each text block and the average value is taken to obtain the document-level vector. At the same time, the composite vectors of each text block at the same level are averaged to obtain the global vector of each level.

[0043] The aforementioned timeliness weight adaptation factor is specifically obtained by extracting the cutting timestamp of each text block, performing difference processing with the current time point to obtain the cutting time span of each text block, and mapping it to obtain the first timeliness weight adjustment coefficient. At the same time, the expiration time point of the validity period of each text block is extracted, and the difference processing with the current time point is performed to obtain the valid time span of each text block, and mapping it to obtain the second timeliness weight adjustment coefficient. The first timeliness weight adjustment coefficient and the second timeliness weight adjustment coefficient are multiplied by the timeliness weight preset factor to obtain the timeliness weight adaptation factor.

[0044] The above mapping yields the first coefficient for adjusting the timeliness weight. Specifically, it involves obtaining a mapping set between the segmentation time span and the first coefficient for adjusting the timeliness weight from the knowledge aggregation information database, and then inputting the segmentation time span of each real-time text block into the mapping set to obtain the first coefficient for adjusting the timeliness weight of each text block. The above mapping also yields the second coefficient for adjusting the timeliness weight. Specifically, it involves obtaining a mapping set between the effective time span and the second coefficient for adjusting the timeliness weight from the knowledge aggregation information database, and then inputting the effective time span of each real-time text block into the mapping set to obtain the second coefficient for adjusting the timeliness weight of each text block.

[0045] In this embodiment, the information value of different text blocks decays at different rates over time (such as data in technical documents and clauses in policy documents). The timeliness weighting adaptation factor can dynamically adjust the weights to make the document-level vector highlight the latest information and avoid outdated information or long-unchanging content from excessively affecting the timeliness representation of the document-level vector. The core function of the timeliness weighting adaptation factor is to inject the value judgment of the time dimension into the weighted average of the composite vector, so that the generated document-level vector can not only aggregate the semantic and hierarchical features of the text blocks, but also dynamically reflect the timeliness value of the information. Ultimately, this improves the accuracy of the document-level vector in tasks such as retrieval, clustering, and comparison, and is especially suitable for scenarios that need to distinguish the value of new and old information (such as technical reports, policy documents, and industry news).

[0046] Specifically, the two-dimensional fusion adaptation index of each text block is determined, and the specific determination process is as follows:

[0047] The two-dimensional characteristic data of each text block includes the information entropy of each text block, the granularity level depth of each text block, the hierarchical binding strength of each text block, and the inter-sentence similarity fluctuation value of each text block, wherein the information entropy can be calculated by the information entropy formula.

[0048] The information entropy, granularity level depth, hierarchical binding strength, and inter-sentence similarity fluctuation of each text block are normalized. An influencing parameter is introduced to correct the normalization results, resulting in a two-dimensional fusion adaptation index for each text block. The specific analysis method is as follows:

[0049]

[0050] In the formula, FA i Let i be the two-dimensional fusion adaptation index of the i-th text block, where i is the number of each text block. [1, M], where M is the total amount of text blocks, HX i Let GL be the information entropy of the i-th text block. i BS is the granularity level depth of the i-th text block. i SS represents the hierarchical affiliation binding strength of the i-th text block. i Let f1 be the inter-sentence similarity fluctuation value of the i-th text block, f2 be the influence parameter corresponding to the predefined information entropy in the knowledge aggregation information base, f3 be the influence parameter corresponding to the predefined granularity level depth in the knowledge aggregation information base, f4 be the influence parameter corresponding to the predefined hierarchical affiliation binding strength in the knowledge aggregation information base, and f5 be the influence parameter corresponding to the predefined inter-sentence similarity fluctuation value in the knowledge aggregation information base.

[0051] It should be explained that the above-mentioned granularity level depth refers to the hierarchical position depth of the text block in a multi-level knowledge structure, which is directly extracted from the hierarchical tags in the document preprocessing stage. For example, L1 (chapter level, depth 1), L2 (sub-chapter level, depth 2), L3 (paragraph level, depth 3), where L represents the level. The larger the depth value, the finer the granularity. The hierarchical affiliation binding strength refers to the degree of semantic association between the text block and its parent level (the next higher level of structure). The cosine similarity between the semantic vector of the text block and the global vector of its parent level is the binding strength. The inter-sentence similarity fluctuation value refers to the standard deviation of the semantic similarity of all adjacent sentences within the text block, reflecting the coherence of the content within the text block.

[0052] The aforementioned information entropy, granularity level depth, hierarchical attribution binding strength, and inter-sentence similarity fluctuation value are respectively mapped to the preset influence parameters corresponding to information entropy, granularity level depth, hierarchical attribution binding strength, and inter-sentence similarity fluctuation value in the knowledge aggregation information base to form a mapping set. The real-time information entropy, granularity level depth, hierarchical attribution binding strength, and inter-sentence similarity fluctuation value are then brought into the mapping set to obtain the influence parameters corresponding to information entropy, granularity level depth, hierarchical attribution binding strength, and inter-sentence similarity fluctuation value.

[0053] In this embodiment, multivariate analysis is performed using information entropy, granularity level depth, hierarchical binding strength, and inter-sentence similarity fluctuation values. Specifically, the correlation between these parameters is considered. Text blocks with higher information entropy often contain more diverse vocabulary and topics, and their granularity level depth may be deeper because more detailed hierarchical division is needed to accurately describe their complex content. Therefore, the two-dimensional fusion adaptation index of the text block will be larger. At the same time, the hierarchical binding strength of such text blocks may be weaker because the diversity and complexity of their content makes their association with upper and lower level text blocks less close. Text blocks with larger inter-sentence similarity fluctuation values ​​may mean that there are significant thematic or semantic changes between sentences within them, which may lead to an increase in granularity level depth to better distinguish sentences with different themes. Furthermore, text blocks with larger inter-sentence similarity fluctuation values ​​may also be related to hierarchical binding strength. If the semantic changes between sentences exceed the semantic range of the current level, then the binding strength with the upper-level text block may weaken.

[0054] Hierarchical clustering is performed based on the composite vectors of each text block to obtain clustering units at each level. The clustering quality index of each clustering unit is evaluated, and a secondary split is performed on each clustering unit based on the clustering quality index.

[0055] The hierarchical clustering process involves a secondary split, the specific process of which is as follows: Figure 4 As shown, Figure 4 The flowchart illustrates the hierarchical clustering process. Based on the composite vectors of each text block, the system employs the HDBSCAN algorithm to progressively perform hierarchical clustering according to the hierarchical order of the text blocks, obtaining clustering units at each level and evaluating the clustering quality index. If the clustering quality index of a certain level clustering unit is lower than the predefined fit index, a secondary split is performed on that clustering unit. The secondary split calculates the semantic similarity matrix, re-clusters to obtain sub-clusters, and verifies them to ensure that the sub-clusters meet the partitioning conditions. After the verification passes, the secondary split is completed. Subsequently, multi-level summarization is performed on each level clustering unit.

[0056] Furthermore, hierarchical clustering is performed, and the specific execution process is as follows:

[0057] The HDBSCAN algorithm is used to perform hierarchical clustering in a progressive manner according to the hierarchical order of text blocks. The level at which clustering is currently performed is recorded as the current level. The specific clustering process is as follows:

[0058] The total number of text blocks at the current level is obtained, which can be extracted from the segmented records. This total number is mapped to obtain the cluster correction factor for the current level, and coupled with a predefined preset number of first clusters. Specifically, the minimum cluster correction factor for the current level is multiplied by the first minimum cluster threshold to obtain the number of first clusters. The number of first clusters represents the minimum number of data samples contained in a valid cluster when clustering the composite vector of text blocks using a clustering algorithm.

[0059] The above mapping yields the cluster correction factor for the current level. Specifically, it obtains the mapping set between the total number of text blocks and the cluster correction factor from the knowledge aggregation information base, and then inputs the real-time total number of text blocks at the current level into the mapping set to obtain the cluster correction factor for the current level.

[0060] Extract the composite vector of all text blocks at the current level and perform clustering. If the number of the first cluster is greater than or equal to the number of the first cluster boundaries, the clustering units at the current level are obtained, and the clustering quality index of the clustering units at the current level is evaluated.

[0061] The cosine similarity of the composite vector of the current-level cluster unit with the global vector of the current-level cluster unit is calculated to obtain the cluster similarity of the current level. This similarity is then multiplied by a predefined cluster similarity influence factor to obtain the primary clustering quality index of the current-level cluster unit. The number of text blocks in the current-level cluster unit is obtained and subtracted from the predefined number of matching text blocks for each cluster unit to obtain the current-level cluster text bias. A clustering quality correction factor is then found and multiplied by the primary clustering quality index of the current-level cluster unit to obtain the sub-index of clustering quality for the current-level cluster unit.

[0062] The above-mentioned method for finding cluster quality correction factors involves obtaining a mapping set between cluster text bias and cluster quality correction factors from a knowledge aggregation information database, inputting the real-time current-level cluster text bias into the mapping set, and finding the corresponding cluster quality correction factor in the mapping set.

[0063] The two-dimensional fusion adaptation index of each text block at the current level is extracted and averaged to obtain the mean of the two-dimensional fusion adaptation index at the current level. This mean is mapped to the clustering quality influence parameter, which is then multiplied by the clustering quality sub-index of the current level clustering unit to obtain the clustering quality index of the current level clustering unit. Similarly, the clustering quality index of each level clustering unit is obtained.

[0064] The above mapping yields the clustering quality impact parameter. Specifically, it obtains the mapping set between the mean of the two-dimensional fusion adaptation index and the clustering quality impact parameter from the knowledge aggregation information base, and uses the mean of the two-dimensional fusion adaptation index at the current level in real time to obtain the clustering quality impact parameter.

[0065] Obtain the total number of all text blocks in the next level, map it to the cluster correction factor of the next level, and couple it with the predefined number of second clusters. Specifically, multiply the cluster correction factor of the next level by the predefined number of second clusters to obtain the number of second cluster boundaries.

[0066] The above mapping yields the cluster correction factor for the next level. Specifically, it obtains the mapping set between the total number of text blocks and the cluster correction factor from the knowledge aggregation information base, and then inputs the real-time total number of text blocks for the next level into the mapping set to obtain the cluster correction factor for the next level.

[0067] For the clustering of all text blocks at the current level, extract all text blocks at the next level contained in each text block at the current level, and only cluster these text blocks at the next level. If the number of the second clusters is greater than or equal to the number of the second cluster boundaries, obtain the clustering of all text blocks at the next level under the current level's text block clustering. At the same time, evaluate the clustering quality index of the next level clustering units, and so on, traversing to the finest level.

[0068] By employing a layer-by-layer clustering approach, text blocks are organized into a structure with a clear hierarchical relationship. The clustering results at each level not only reflect the thematic or semantic similarity of the text blocks at that level but also demonstrate the semantic connections and progressive relationships between different levels. This hierarchical structure makes the organization of text data more aligned with the cognitive habits of computing devices, facilitating understanding and analysis. Starting from the top level and delving deeper into the finest granular level, it provides both a macroscopic global view and allows for detailed analysis of fine-grained text blocks. This combined approach enables the system to simultaneously meet users' needs for grasping the overall text structure and exploring in-depth details, making it suitable for various application scenarios such as text classification, topic mining, and knowledge graph construction.

[0069] Specifically, the clustering units at each level are split twice. The specific analysis process is as follows:

[0070] The clustering quality index of each level of clustering unit is compared with the predefined clustering quality fit index. If the clustering quality index of a certain level of clustering unit is less than the clustering quality fit index, then the clustering unit at that level is determined to be split twice; otherwise, the split is not performed.

[0071] The second splitting process is specifically as follows:

[0072] Obtain the semantic vectors of all text blocks within the current clustering unit. Based on the cosine similarity algorithm, calculate the semantic similarity between the text block vectors within the current clustering unit, generating a similarity matrix. The value in the a-th row and b-th column of the matrix represents the semantic similarity between the a-th and b-th text blocks, with a value range of [0,1]. Then, using the HDBSCAN clustering algorithm as input, automatically output several sub-clusters of the current clustering unit, where each sub-cluster satisfies the following partitioning condition:

[0073] 1) The number of each sub-cluster is greater than or equal to the number of cluster boundaries.

[0074] 2) No two text blocks in the same sub-cluster have a semantic similarity less than the initial semantic similarity threshold.

[0075] The sub-clusters after partitioning are validated:

[0076] The average semantic similarity between all text block pairs within each sub-cluster is denoted as the average semantic similarity within each sub-cluster. If the average semantic similarity within each sub-cluster is greater than or equal to the initial semantic similarity threshold, and the center similarity between the center vectors of different sub-clusters is less than or equal to the center similarity threshold, then the secondary splitting process is completed. The center vector of the sub-cluster is represented as the average of all semantic vectors within the sub-cluster.

[0077] If the average semantic similarity within a sub-cluster is less than the initial semantic similarity threshold, the difference between the average semantic similarity within the sub-cluster and the initial semantic similarity threshold is calculated to obtain the average semantic similarity deviation within the sub-cluster. This deviation is then mapped to a semantic similarity correction parameter, which is coupled with the initial semantic similarity threshold. Specifically, the semantic similarity correction parameter is multiplied by the initial semantic similarity threshold to obtain the semantic similarity adaptation threshold. A second split is then performed based on this threshold. It should be noted that the aforementioned semantic similarity correction parameter is used to adjust the initial semantic similarity threshold upwards to the semantic similarity adaptation threshold.

[0078] The above mapping yields a semantic similarity correction parameter. Specifically, it obtains a mapping set between the average semantic similarity deviation and the semantic similarity correction parameter from the knowledge aggregation information database, and then incorporates the real-time average semantic similarity deviation within the sub-clusters into the mapping set to obtain the semantic similarity correction parameter.

[0079] If the center similarity between the center vectors of different sub-clusters is greater than the center similarity threshold, then the two different sub-clusters need to be merged and then a second split is performed based on the initial semantic similarity threshold.

[0080] It should be noted that after the split is completed, a clear hierarchical label is added to each sub-cluster, in the format of original cluster ID - sub-cluster number (e.g., the original L2 cluster is L2-3, and after splitting, the sub-clusters are labeled L2-3-1 and L2-3-2, and are forced to inherit the parent hierarchical label of the original cluster, such as still belonging to the L1-2 cluster, to ensure that the hierarchical relationship remains unchanged).

[0081] By comparing the clustering quality metrics of each level of clustering units with predefined fit metrics, a secondary split is only performed when the clustering quality fails to meet the standards. This mechanism ensures that the clustering results at each level meet the quality requirements, avoiding the impact of low-quality clustering units on the overall analysis, thereby significantly improving the quality of the entire clustering system. The secondary split process re-clusters based on the semantic similarity matrix, enabling finer segmentation of text blocks. By setting the partitioning conditions for sub-clusters (such as the number of clusters, internal semantic similarity, etc.), each sub-cluster is ensured to have high semantic cohesion and discriminability, thus optimizing the granularity of clustering and making it more consistent with the actual semantic structure of the text.

[0082] Multi-level summaries are performed on each level of clustering units to obtain the cluster summaries at each level. The granularity of user questions is extracted from the server user port, and the cluster summaries at each level are scheduled. At the same time, automatic verification of level boundaries is performed during the scheduling process.

[0083] The generation of multi-level summaries follows a logic of starting from the bottom up and progressing hierarchically. Based on the preprocessed text blocks, vectorization results, and hierarchical clustering results, document-level summaries, cluster-level summaries, and global-level summaries are generated sequentially. The generation process of each level depends on the output of the level below.

[0084] The aforementioned document-level summary involves concatenating all text blocks of a single document in their original hierarchical order, retaining the hierarchical tags as input, and ensuring that the server computing port can recognize the document's structural logic. After automatically generating a knowledge summary for the document using the LLM (Large Language Model) on the server computing port, a structured summary of the single document is output, including an overview of the hierarchical structure, core viewpoints, and key information, with a length approximately proportional to the original text.

[0085] The aforementioned cluster-level summaries are generated by sorting the document-level vectors and cluster center vectors of the same-level cluster unit in descending order of similarity, resulting in a sorted list. Summaries with document similarity greater than a threshold in the sorted list are prioritized as input. The LLM library on the server's computing port automatically generates cluster-level summaries of the same level, thus outputting summaries reflecting the commonalities and differences of similar documents under a specific topic, clarifying the position of each cluster unit within the knowledge system.

[0086] The aforementioned global-level summary, based on a sorted list, forms topic groups (such as technical principles, application scenarios, and development trends), and is arranged in logical order (such as chronological order or causal relationships) as input. The LLM is then invoked to generate the global-level summary, which outputs a global knowledge framework covering the entire knowledge base, clearly presenting the topic distribution, hierarchical relationships, and evolutionary logic.

[0087] The above-mentioned server user port extracts the granularity of user questions, specifically by obtaining the word count and the number of domain keywords in the user question. The word count, expressed as the total number of characters in the user question (including punctuation, excluding spaces), reflects the level of detail in the question description and can be extracted from the server user port records. The number of domain keywords represents the number of domain-specific terms and core concepts appearing in the question, obtained through searching a domain keyword database.

[0088] The first granularity of the user question is obtained by multiplying the word count of the user question by a predefined word count weighting factor. The second granularity of the user question is obtained by multiplying the number of domain keywords in the user question by a predefined domain keyword quantity weighting factor. These two granularities are then added together to obtain the final granularity of the user question. It should be noted that both the word count and the number of domain keywords in the user question have been normalized.

[0089] Furthermore, the clustering summaries at each level are scheduled, and the specific analysis process is as follows:

[0090] The granularity of the user's question is compared with predefined granularity intervals to determine the interval to which the user's question belongs:

[0091] Each particle size interval includes a first particle size interval, a second particle size interval, and a third particle size interval.

[0092] If the granularity of the user's question belongs to the first granularity interval, then the global-level summary, high-level cluster summary, and high-level vector are scheduled first, followed by the mid-level cluster summary associated with the high-level cluster, and then the fine-grained block is assisted in scheduling. The weight ratio of each scheduling satisfies the weight ratio set of the first granularity interval.

[0093] It should be explained that the set of weight ratios for the first granularity interval mentioned above includes three types of weight ratios: the first type, the second type, and the third type. The first type of weight ratio is used to configure the knowledge data to be scheduled first, the second type is used to configure the knowledge data to be scheduled second, and the third type is used to configure the knowledge data to be scheduled second.

[0094] If the granularity of the user's question belongs to the second granularity interval, the mid-level cluster summary and mid-level hierarchical vector under the corresponding high-level clustering will be scheduled first to assist in scheduling the fine-grained blocks associated with the mid-level clustering. The weight ratio of each scheduling session satisfies the weight ratio set of the second granularity interval.

[0095] It should be explained that the set of weight proportions for the second granularity interval mentioned above includes both the first type of weight ratio and the second type of weight ratio. The first type of weight ratio is used to configure priority scheduling knowledge data, and the second type of weight ratio is used to configure auxiliary scheduling knowledge data.

[0096] If the granularity of the user's problem belongs to the third granularity range, the target cluster is located through the high-middle layer hierarchy, and then the fine-grained blocks under that cluster are scheduled.

[0097] It should be explained that in this embodiment, the high-level layer can be represented as level L1, the middle layer as level L2, and the fine-grained block as level L3. During document segmentation, text blocks with hierarchical paths are generated strictly according to the original structure (chapter, sub-chapter, paragraph). The high-level layer (L1) corresponds to the first-level heading (chapter) in the original document structure, and is the top-level framework of the document. The L1 text block itself is the chapter title and overview, and its subordinate middle-level (L2) and fine-grained (L3) text blocks are associated with it (i.e., "L1 is the parent level of L2, and L2 is the parent level of L3"). Example: A document with ID Doc-2025 has its L1 text block path encoded as Doc-2025-L1-1 (Chapter 1), Doc-2025-L1-2 (Chapter 2), etc. The first level (L1) corresponds to the second-level headings (sub-sections) under the first-level headings in the original document structure. It is a further subdivision of the L1 topic. Each L2 text block must be bound to a unique L1 parent level (e.g., "1.1 Major Model Technology Breakthrough" belongs to "1. Core Technologies of Artificial Intelligence"), and its subordinate fine-grained (L3) text blocks are related. Fine-grained (L3) text blocks correspond to the paragraphs or sections under the second-level headings in the original document structure. They are a specific expansion of the L2 sub-topic. Each L3 text block must be bound to a unique L2 parent level (e.g., a paragraph belongs to "1.1 Major Model Technology Breakthrough"), and is the most specific information unit in the document.

[0098] Based on the granularity range of the user's question (first, second, third), the scheduling will strictly follow the hierarchical path relationship, prioritizing the call of the corresponding level's summary or text block, and assigning weights according to rules.

[0099] By comparing the granularity of user questions with predefined granularity intervals, the most appropriate scheduling strategy can be selected based on the complexity and specific needs of the user question. For example: The first granularity interval is suitable for user questions that are relatively macroscopic or broad. By prioritizing the scheduling of global-level summaries and high-level cluster summaries, a holistic answer can be quickly provided, while fine-grained blocks are scheduled to supplement details. The third granularity interval is suitable for user questions that are very specific and require in-depth analysis of fine-grained details. By directly locating the target cluster and scheduling fine-grained blocks, the most accurate answer can be provided. This hierarchical scheduling approach avoids a full search of the entire dataset, instead gradually narrowing the search scope according to the granularity of the user question. For example, for questions in the first granularity interval, high-level content is scheduled first to quickly provide a macroscopic answer, avoiding unnecessary fine-grained searches. For questions in the third granularity interval, the target cluster is directly located and fine-grained blocks are scheduled, avoiding invalid searches of high- and mid-level content.

[0100] Specifically, automatic hierarchical boundary verification is performed during the scheduling process. The specific verification process is as follows:

[0101] Before scheduling begins, a target hierarchical path is generated based on the granularity of the user's question, which defines the allowed range of levels and their hierarchical relationships.

[0102] For example: If the granularity of the user's question belongs to the third granularity interval (such as "the specific steps of block L2-3 in document 2"), the target hierarchical path is document 2 to L1-a certain cluster to block L2-3. If the granularity of the user's question belongs to the second granularity interval (such as "the common features of L2 cluster 3"), the target hierarchical path is "global topic A to L1-2 cluster to L2-3 cluster. The path contains a unique identifier for each level (such as L1 cluster ID: C1-2, L2 block ID: B2-3)" as a verification benchmark.

[0103] For all candidate knowledge units returned by the scheduling process, the hierarchical identifiers of all candidate knowledge units are automatically extracted. Candidate knowledge units include text blocks, multi-level summaries, and multi-level clustering results. The hierarchical identifiers are pre-bound during the preprocessing stage and are formatted as DocID-hierarchical depth-unit ID-parent hierarchical ID. For example, a text block might be identified as "Doc3-L2-5-L1," indicating the 5th block at level L2 of document 3, with a parent level of L1-1. A clustering summary might be identified as C2-3(L2)-L1-2, indicating the 3rd cluster at level L2, with a parent level of L1-2. The hierarchical identifiers of each candidate knowledge unit are compared with the target hierarchical path. This includes checking if the hierarchical depth of the candidate knowledge unit matches the hierarchical depth in the target path and automatically checking if the parent hierarchical ID in the candidate knowledge unit's hierarchical identifier matches the previous level cluster ID in the target hierarchical path. If any inconsistency or mismatch exists, it is determined to be a cross-hierarchical error and automatically filtered out of the scheduling list.

[0104] For example, if the target path requires access to L2 level content, but the candidate knowledge unit is L3 level (depth 3), then it is initially determined to be a level inconsistency.

[0105] For example, if a candidate knowledge unit is an L2 block, its parent level ID must be equal to the ID of the L1 level in the target path. For instance, if the target L1 is L1-2, then the parent ID of the candidate L2 block must be L1-2. If a candidate unit is an L2 cluster, its parent cluster ID must be equal to the ID of the L1 cluster in the target path. For instance, if the target L1 cluster is C1-3, then the parent ID of the candidate L2 cluster must be C1-3.

[0106] By generating a target hierarchical path, the allowed hierarchical range and subordinate relationships are explicitly specified. During scheduling, the hierarchical identifiers of candidate knowledge units are compared one by one to check whether their hierarchical depth matches the target path and whether their parent hierarchical ID matches the cluster ID of the previous level in the target path. This mechanism ensures that all candidate knowledge units returned by scheduling strictly conform to the predefined hierarchical structure, avoiding cross-hierarchical errors and guaranteeing the accuracy and consistency of hierarchical relationships. By automatically filtering out candidate knowledge units that do not conform to the target hierarchical path, invalid or irrelevant content that may be introduced due to scheduling errors or data inconsistencies can be effectively removed. This makes the final returned scheduling results more accurate, reliable, and better meet the actual needs of the user's problem.

[0107] Reference Figure 2 As shown, a second aspect of the present invention provides a management and scheduling system based on multi-level knowledge aggregation data, comprising: a text block segmentation module, a composite vector processing module, a clustering quality assessment module, a multi-level scheduling module, and a knowledge aggregation information database. The knowledge aggregation information database is used to store preset values ​​for various parameters.

[0108] The text block segmentation module is connected to the composite vector processing module, which in turn is connected to the clustering quality assessment module. The clustering quality assessment module is also connected to the multi-level scheduling module. All of these modules are connected to the knowledge aggregation information base.

[0109] The text block segmentation module is used by the server computing port to preprocess the input document. Based on the original structure of the input document, it performs segmentation processing to obtain the text blocks of the input document, and annotates the three-dimensional metadata of each text block to form a set of text blocks with links.

[0110] The composite vector processing module is used to obtain the two-dimensional characteristic data of each text block in the text block set, determine the two-dimensional fusion adaptation index of each text block, match the composite vector weight fusion ratio, perform vectorization processing on each text block, and obtain the composite vector of each text block.

[0111] The clustering quality assessment module is used to perform hierarchical clustering based on the composite vector of each text block to obtain clustering units at each level, evaluate the clustering quality index of each clustering unit, and perform secondary splitting of each clustering unit based on the clustering quality index.

[0112] The multi-level scheduling module is used to perform multi-level summaries on clustering units at each level to obtain cluster summaries at each level. The server user port extracts the granularity of user questions and schedules the cluster summaries at each level. At the same time, automatic verification of level boundaries is performed during the scheduling process.

[0113] A third aspect of the present invention provides an electronic device, characterized in that the electronic device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the above-described method.

[0114] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined by the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A management and scheduling method based on multi-level knowledge aggregation data, characterized in that, include: The server computing port preprocesses the input document, performs segmentation based on the original structure of the input document to obtain text blocks, and annotates each text block with three-dimensional metadata to form a set of text blocks with linked relationships. Obtain the two-dimensional characteristic data of each text block in the text block set, determine the two-dimensional fusion adaptation index of each text block, match the composite vector weight fusion ratio, perform vectorization processing on each text block, and obtain the composite vector of each text block. Hierarchical clustering is performed based on the composite vectors of each text block to obtain clustering units at each level. The clustering quality index of each clustering unit is evaluated, and the clustering units at each level are split a second time based on the clustering quality index. Multi-level summaries are performed on each level of clustering units to obtain the cluster summaries at each level. The granularity of user questions is extracted from the server user port, and the cluster summaries at each level are scheduled. At the same time, automatic verification of level boundaries is performed during the scheduling process.

2. The management and scheduling method based on multi-level knowledge aggregation data according to claim 1, characterized in that: The specific process for forming the set of linked text blocks is as follows: The original structure of the input document is paragraphs and chapters, and the input document is divided into several text blocks according to the paragraphs and chapters; The annotation of three-dimensional metadata for each text block specifically includes: First dimension: Mark each text block with a unique hierarchical path encoding; The second dimension: link each text block upwards to its parent block and pre-link its child blocks downwards; The third dimension: bind time-limited tags to each text block, where the time-limited tags include the cutting timestamp and the validity period of the text block; Finally, the text blocks are integrated into a set of text blocks with linked relationships.

3. The management and scheduling method based on multi-level knowledge aggregation data according to claim 1, characterized in that: The vectorization process for each text block is as follows: Each text block is converted into a vector using a text embedding model, denoted as the semantic vector of each text block; Extract the hierarchical feature data of each text block, including the hierarchical depth of each text block, the semantic similarity of the parent level of each text block, and the distribution density of the same level of each text block, and form the hierarchical feature vector of each text block. The two-dimensional fusion adaptation index of each text block is averaged to obtain the document's two-dimensional fusion adaptation index. This index is then matched with the predefined composite vector weight fusion ratio corresponding to each two-dimensional fusion adaptation index interval to determine the interval to which the document's two-dimensional fusion adaptation index belongs, and the composite vector weight fusion ratio corresponding to that interval is obtained. The semantic vector of each text block and the hierarchical feature vector of each text block are fused together according to the composite vector weight fusion ratio to form a hierarchical-semantic composite vector, thus obtaining the composite vector of each text block. The timeliness weight adaptation factor of each text block is obtained and coupled with the composite vector of each text block to obtain the document-level vector. At the same time, the composite vectors of each text block at the same level are averaged to obtain the global vector of each level.

4. The management and scheduling method based on multi-level knowledge aggregation data according to claim 3, characterized in that: The specific process for determining the two-dimensional fusion adaptation index of each text block is as follows: The two-dimensional characteristic data of each text block includes the information entropy of each text block, the granularity level depth of each text block, the hierarchical binding strength of each text block, and the inter-sentence similarity fluctuation value of each text block. The information entropy, granularity level depth, hierarchical binding strength, and inter-sentence similarity fluctuation of each text block are normalized. An influencing parameter is introduced to correct the normalization result, resulting in a two-dimensional fusion adaptation index for each text block.

5. The management and scheduling method based on multi-level knowledge aggregation data according to claim 1, characterized in that: The execution process of the hierarchical clustering is as follows: The HDBSCAN algorithm is used to perform hierarchical clustering in a progressive manner according to the hierarchical order of text blocks. The level at which clustering is currently performed is recorded as the current level. The specific clustering process is as follows: Get the total number of text blocks at the current level, map it to the cluster correction factor at the current level, and couple it with the predefined number of the first cluster to get the number of the first cluster boundaries; Extract the composite vector of all text blocks in the current level and perform clustering. If the number of the first cluster is greater than or equal to the number of the first cluster boundaries, the clustering units of the current level are obtained, and the clustering quality index of the current level clustering units is evaluated. Get the total number of all text blocks in the next level, map it to the cluster correction factor of the next level, and couple it with the predefined number of second clusters to get the number of second cluster boundaries; For the clustering of all text blocks at the current level, extract all text blocks at the next level contained in each text block at the current level, and only cluster these text blocks at the next level. If the number of the second clusters is greater than or equal to the number of the second cluster boundaries, obtain the clustering of all text blocks at the next level under the current level's text block clustering. At the same time, evaluate the clustering quality index of the next level clustering units, and so on, traversing to the finest level.

6. The management and scheduling method based on multi-level knowledge aggregation data according to claim 5, characterized in that: The specific analysis process for performing secondary splitting on each level of clustering units is as follows: The clustering quality index of each level of clustering unit is compared with the predefined clustering quality fit index. If the clustering quality index of a certain level of clustering unit is less than the clustering quality fit index, it is determined that the level of clustering unit will be split twice; otherwise, the split will not be performed. The second splitting process is specifically as follows: Obtain the semantic vectors of all text blocks within the current clustering unit. Calculate the semantic similarity between the text block vectors within the current clustering unit based on the cosine similarity algorithm, generating a similarity matrix. Then, apply the HDBSCAN clustering algorithm again, using the similarity matrix as input, to automatically output several sub-clusters of the current clustering unit. The partitioning condition for each sub-cluster is as follows: 1) The number of sub-clusters is greater than or equal to the number of cluster boundaries; 2) No two text blocks in the same sub-cluster have a semantic similarity less than the initial semantic similarity threshold; The sub-clusters after partitioning are validated: The average semantic similarity between all text block pairs within each sub-cluster is denoted as the average semantic similarity within each sub-cluster. If the average semantic similarity within each sub-cluster is greater than or equal to the initial semantic similarity threshold, and the center similarity between the center vectors of different sub-clusters is less than or equal to the center similarity threshold, then the second splitting process is completed. The center vector of the sub-cluster is represented as the average of all semantic vectors within the sub-cluster. If the average semantic similarity within a sub-cluster is less than the initial semantic similarity threshold, then the difference between the average semantic similarity within the sub-cluster and the initial semantic similarity threshold is processed to obtain the average semantic similarity deviation within the sub-cluster. This deviation is then mapped to obtain the semantic similarity correction parameter, which is coupled with the initial semantic similarity threshold to obtain the semantic similarity adaptation threshold. Based on the semantic similarity adaptation threshold, a second split is then performed. If the center similarity between the center vectors of different sub-clusters is greater than the center similarity threshold, then the two different sub-clusters need to be merged and then a second split is performed based on the initial semantic similarity threshold.

7. The management and scheduling method based on multi-level knowledge aggregation data according to claim 1, characterized in that: The specific analysis process for scheduling the clustering summaries at each level is as follows: The granularity of the user's question is compared with predefined granularity intervals to determine the interval to which the user's question belongs: Each particle size interval includes a first particle size interval, a second particle size interval, and a third particle size interval; If the granularity of the user's question belongs to the first granularity interval, then the global level summary, high-level cluster summary and high-level vector are scheduled first, then the mid-level cluster summary of high-level association is scheduled, and fine-grained blocks are assisted in scheduling. The weight ratio of each scheduling satisfies the weight ratio set of the first granularity interval. If the granularity of the user's question belongs to the second granularity interval, the mid-level cluster summary and mid-level hierarchical vector under the corresponding high-level clustering will be scheduled first to assist in scheduling the fine-grained blocks associated with the mid-level clustering. The weight ratio of each scheduling satisfies the weight ratio set of the second granularity interval. If the granularity of the user's problem belongs to the third granularity range, the target cluster is located through the high-middle layer hierarchy, and then the fine-grained blocks under that cluster are scheduled.

8. The management and scheduling method based on multi-level knowledge aggregation data according to claim 7, characterized in that: The automatic hierarchical boundary verification is performed during the scheduling process, and the specific verification process is as follows: Before scheduling begins, a target hierarchical path is generated based on the granularity of the user's question, which is the allowed range of levels and their subordinate relationships. For all candidate knowledge units returned by the scheduling process, the hierarchical identifiers of all candidate knowledge units are automatically extracted. Candidate knowledge units include text blocks, multi-level summaries, and multi-level clustering results. The hierarchical identifiers of each candidate knowledge unit are compared with the target hierarchical path. This includes checking whether the hierarchical depth of the candidate knowledge unit is consistent with the hierarchical depth in the target path and automatically checking whether the parent level ID in the hierarchical identifier of the candidate knowledge unit matches the previous level cluster ID in the target hierarchical path. If there is an inconsistency or mismatch, it is determined to be a cross-level error and is automatically filtered out from the scheduling list.

9. A system applying the management and scheduling method based on multi-level knowledge aggregation data as described in any one of claims 1-8, characterized in that: include: The text block segmentation module is used by the server computing port to preprocess the input document. Based on the original structure of the input document, it performs segmentation processing on the input document to obtain each text block of the input document. It annotates each text block with three-dimensional metadata to form a set of text blocks with links. The composite vector processing module is used to obtain the two-dimensional characteristic data of each text block in the text block set, determine the two-dimensional fusion adaptation index of each text block, match the composite vector weight fusion ratio, perform vectorization processing on each text block, and obtain the composite vector of each text block. The clustering quality assessment module is used to perform hierarchical clustering based on the composite vector of each text block to obtain clustering units at each level, evaluate the clustering quality index of each clustering unit, and perform secondary splitting of each clustering unit based on the clustering quality index. The multi-level scheduling module is used to perform multi-level summaries on clustering units at each level to obtain cluster summaries at each level. The server user port extracts the granularity of user questions and schedules the cluster summaries at each level. At the same time, automatic verification of level boundaries is performed during the scheduling process.

10. An electronic device, characterized in that, The electronic device includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to perform the method of any one of claims 1 to 8.

Images