A RAG database construction method based on information compression and pruning

Abstract

This invention proposes a method for constructing a RAG database based on information compression and pruning. The steps are as follows: constructing node objects and a hierarchical clustering tree; compressing the text information of the node objects in the clustering tree; and pruning the clustering tree after compressing some of the node object text information based on a QR-like decomposition method. By compressing the sentence unit text information in the child nodes of the clustering tree and updating the embedding vectors of the compressed child nodes, this invention can filter and store the most valuable external information according to the actual needs of the large language model. This not only reduces the storage of invalid data in the RAG database but also ensures that the text information stored in the RAG database always has gain. Simultaneously, pruning the clustering tree after compressing some of the node object text information based on a QR-like decomposition method effectively identifies and deletes redundant child node objects with high semantic similarity, reducing the storage overhead of node objects in the RAG database.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence large language models and relates to a method for constructing a retrieval-enhanced RAG database. Specifically, it relates to a method for constructing a retrieval-enhanced RAG database based on information compression and pruning, which can be used in natural language processing, information retrieval, and external knowledge base management of large language models. Background Technology

[0002] With the rapid development of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) technology has gradually become an important means to improve the inference capabilities of models. In RAG systems, external knowledge bases are typically stored as vector databases. By retrieving relevant information through similarity, the language model is provided with additional context, allowing it to refer to more information when generating answers, especially when large models cannot directly obtain answers from their parameters. RAG technology improves model performance by combining external knowledge sources. One of the core issues in RAG technology for large language models is how to efficiently construct the RAG database.

[0003] The basic process of building a RAG database is as follows: (1) Text segmentation and vectorization: First, the text to be retrieved is segmented into several small sentence units, usually paragraphs or sentence fragments of fixed length. Then, these sentence units are vectorized into fixed-length embedding vectors through an embedding model for subsequent retrieval and matching. These vectors are used to build the RAG database, and each vector represents the semantic information of a sentence unit. (2) Database storage of sentence units and embedding vectors: The above-segmented sentence units and embedding vectors are stored in the database according to certain relational rules or data structures to form the RAG database. Common RAG databases combine a single sentence unit and its corresponding embedding vector into the smallest query unit and store them directly in the database in order.

[0004] To improve retrieval efficiency in large-scale text databases, for example, P Sarthi et al. proposed a method for constructing a RAG database based on a hierarchical iterative clustering tree RAPTOR in their 2024 paper "Recursive Abstractive Processing for Tree-organized Retrieval." This method first segments the text into several sentence units, then vectorizes each sentence unit into an embedding vector using an embedding model and combines it with other sentence units to form node objects. A clustering algorithm is then used to aggregate similar sentence units together, and a large language model is used to semantically summarize these clusters, generating higher-level parent node sentence units. This process is iteratively repeated to gradually construct a multi-level sentence unit tree structure. This method, through iterative clustering of node objects, gives the RAG database a better data management structure and retrieval efficiency. However, this method introduces redundant information between node sentence units during the construction of the sentence unit tree structure, resulting in a large amount of similar or duplicate text content in the database. This not only increases the storage overhead of the RAG database but also affects the efficiency of the large language model in retrieving the RAG database and the quality of the responses generated based on the search results. Moreover, this method directly stores the sentence units segmented from the original text as node objects, resulting in poor adaptability of the text information in the RAG database to specific large language models. For knowledge in certain specific domains, the text retrieved from the RAG database contains less effective information required by large language models, resulting in lower quality of generated answers. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and propose a RAG database construction method based on information compression and pruning, which solves the technical problems of large storage overhead of RAG database and poor adaptability of text information to specific large language models in the existing technology.

[0006] To achieve the above objectives, the technical solution adopted by the present invention includes the following steps:

[0007] (1) Construct node objects:

[0008] Obtain N text data, each containing multiple sentences, and segment each text data according to the delimiter to obtain M sentence units from the N text data. Then, vectorize each sentence unit through an embedding model, and combine the embedding vector of each sentence unit with the sentence unit to form a node object, resulting in M ​​node objects, where N≥1000.

[0009] (2) Construct a hierarchical clustering tree:

[0010] Perform S iterations of semantic clustering on M node objects, and construct an S+1 layer clustering tree with M node objects as leaf nodes and G node objects as parent nodes, which are the summary sentence units of multiple grouped child nodes obtained in each iteration of semantic clustering and their corresponding embedding vector combinations.

[0011] (3) Compress the text information of node objects in the clustering tree:

[0012] The text information of sentence units in the child nodes covered by their parent nodes in the clustering tree is compressed, and the embedding vectors of the child nodes corresponding to the compressed sentence unit text information are updated based on the embedding model to obtain a clustering tree with compressed text information of some node objects.

[0013] (4) Pruning the clustering tree after compressing the text information of some node objects based on a QR-like decomposition method:

[0014] The child node objects contained in each parent node in the clustering tree after the text information of some node objects is compressed are sorted in descending order of importance metric scores. Then, based on the class orthogonal triangular QR decomposition method, the linear independence coefficient of the embedding vector of each sorted child node object contained in each parent node is calculated. Then, the child node objects with independence coefficients lower than the independence threshold ∈ are pruned to obtain the RAG database containing the structure and information of the pruned clustering tree.

[0015] Compared with existing technologies, the present invention has the following advantages:

[0016] 1. This invention compresses the text information of sentence units in child nodes covered by their parent nodes in a clustering tree, and updates the embedding vectors of the child nodes corresponding to the compressed sentence unit text information based on an embedding model. This allows for the filtering and storage of the most valuable external information, avoiding the excessive storage of invalid data in the database in existing technologies. It ensures that the text information stored in the RAG database always has gain, thereby improving the inference performance of the overall RAG system.

[0017] 2. This invention prunes the clustering tree after compressing the text information of some node objects based on a QR-like decomposition method, effectively identifying and deleting redundant child node objects with high semantic similarity, reducing the storage overhead of node objects in the RAG database, and avoiding the impact of excessive redundant information in the database on storage space occupation and data management complexity in existing technologies. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the implementation of the present invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0020] Reference Figure 1 The present invention includes the following steps:

[0021] Step 1) Construct node objects:

[0022] (1a) Obtain 1000 text data entries, each containing multiple sentences, and perform initial segmentation based on common delimiters including periods ".", exclamation marks "!", question marks "?", and newlines to ensure that each sentence is treated as an independent unit. This operation is performed using regular expressions, which can effectively handle different types of text delimiters. Each segmented sentence is then encoded using the tokenizer of the BERT embedding model to calculate the number of tokens in each encoded block. The purpose is to determine the number of tokens occupied by each sentence, which facilitates subsequent control over the maximum number of tokens in a text block.

[0023] (1b) Based on the maximum allowed number of tokens (100), multiple sentences are gradually merged into a single text block. If adding a sentence does not exceed the maximum token limit, the sentence is added to the current text block; if adding the sentence would exceed the limit, the current text block is saved, and a new text block is constructed. When a sentence's token count exceeds the maximum limit, it is further segmented using punctuation marks such as commas, semicolons, and colons to ensure that the length of each part is within the allowed range. This process constructs suitable text blocks by gradually adding smaller clauses.

[0024] (1c) When constructing adjacent text blocks, the `overlap` parameter is set to retain a portion of the content at the end of the previous text block and copy this portion to the beginning of the next text block. This creates a certain contextual association between text blocks, preventing the loss of important contextual information during segmentation. The final result is a list of 16,000 sentence units. The length of each sentence unit is controlled within the specified maximum number of tokens, and the overlap between text blocks is set as needed.

[0025] (1d) Obtain the nomic-embed-text embedding model, which includes a cascaded word segmenter, an embedding layer, an encoder composed of multiple cascaded Transformer blocks, and an output layer. Each Transformer block contains a cascaded self-attention network and two perceptron MLP feedforward networks. The embedding layer consists of a 30000×768 word embedding matrix, where 30000 represents the total number of subwords or word identifiers. The encoder is composed of 12 cascaded Transformer blocks, and the output layer consists of a single Transformer block.

[0026] The (1e)nomic-embed-text input layer's word segmenter breaks down each sentence unit into 100 sub-words or word identifiers, and then the embedding layer maps the sub-words or word identifiers into 100 fixed-dimensional 768-dimensional vectors through a word embedding matrix;

[0027] (1f) The multi-layer Transformer of the nomic-embed-text encoder layer uses a self-attention mechanism to model the context of 100 vectors for each sentence unit, resulting in 100 context-related vectors. Then, the input and output layers use feature dimensionality reduction techniques to reduce the dimensionality of the context-related vectors, resulting in a fixed-dimensional 768-dimensional embedding vector.

[0028] (1g) Combine the vectorized embedding vector of each sentence unit with the sentence unit to form a node object, resulting in 16,000 node objects;

[0029] Step 2) Construct a hierarchical clustering tree:

[0030] (2a) Initialize the number of iterations for semantic clustering to s, and let s = 1. Initialize node object O1 to the 16,000 node objects obtained in step 1), and use them as the leaf nodes of the clustering tree.

[0031] Write a prompt template `prompt_sum` for summarizing sentence units in a large language model:

[0032] "Write a summary of the following, including as many key details aspossible:{sentences}:"

[0033] Where {sentences} represents the set of text content of sentence units of all child nodes under the cluster group.

[0034] (2b) The embedding vector of node object O1 is reduced to a dimension of 10 using the uniform manifold approximation and projective UMAP to alleviate the problem that distance metrics may perform poorly when measuring similarity in the high-dimensional space of Gaussian Mixture Models (GMMs). Then, the Bayesian Information Criterion (BIC) is used to select the Gaussian model to determine the optimal number of clusters. BIC not only penalizes model complexity but also rewards goodness of fit. The BIC for a given GMM is... Where T is the number of text segments (or data points), and p is the number of model parameters. This is the maximum value of the model likelihood function. In GMM, the number of parameters p is a function of the dimension of the input vector and the number of clusters. Based on the optimal number of clusters and GMM clustering, node objects O1 are clustered into G... s Semantic groups;

[0035] (2c) The prompt method of the large language model Qwen2, based on the summary sentence unit template, summarizes all sentence units in each group to obtain the summary sentence unit: The obtained summary sentence unit Sum:

[0036] Sum=RAG_LLM(prompt_sum,sentences)

[0037] Then, perform the same vectorization operation as in step 1) to vectorize each summary sentence unit, obtaining a summary embedding vector. Finally, combine each summary sentence unit and its corresponding embedding vector into a parent node object, resulting in G. s Each parent node object, and the resulting G s O node objects s This is the s-th layer of the clustering tree.

[0038] (2d) Iterative semantic clustering finally yields a 5-layer clustering tree consisting of 16,000 leaf nodes, 695 parent nodes in the first layer, 156 parent nodes in the second layer, 23 parent nodes in the third layer, and 5 parent nodes in the fourth layer.

[0039] Step 3) Compress the text information of the node objects in the clustering tree:

[0040] (3a) Use a large model to quantify the additional information provided in the child node text that exceeds the summary text of the parent node, using a scoring system of 1-6. Develop a prompt template based on the large language model to calculate the incremental score of one sentence unit text information relative to another sentence unit text information, prompt_add:

[0041] "Here is a summary of a long text and an excerpt from that text. Please evaluate whether the excerpt provides important information that is not included in the summary. Use an integer score from 1to 6to measure the amount of new information provided by the excerpt.

[0042] Summary:

[0043] {sentence2}

[0044] Excerpt:

[0045] {sentence1}

[0046] Please answer using Arabic numerals only 1,2,3,4,5,6:”

[0047] Where {sentence2} represents the summary text content of the parent node, and {sentence1} represents the text content of the child node; then, write the text compression prompt template prompt_com:

[0048] "Here is a summary of a long text and an excerpt from that text. Carefully read both and identify the important additional information in the excerpt that is not mentioned in the summary. Concisely summarize this additional information.

[0049] Summary:

[0050] {sentence2}

[0051] Excerpt:

[0052] {sentence1}:”

[0053] (3b) Based on incremental score calculation, the Qwen2 large language model's prompt method calculates the incremental score of the sentence unit text information of each child node in the first 4 layers of a 5-layer clustering tree relative to the sum of sentence unit text information of its parent node. The incremental score A of the k-th child node contained in the j-th parent node of the s-th layer is used. sjk The calculation formula is:

[0054] A sjk =RAG_LLM(prompt_add,sentence sjk ,sentence sj )

[0055] Where sentence1 represents the child node sentence unit, and sentence2 represents the parent node sentence unit;

[0056] (3c) The text of the parent node is a semantic summary of all child nodes. Therefore, by comparing the text of the child nodes with that of the parent node, it is possible to assess whether the child node provides additional information beyond the summary of the parent node. If most of the content of the child node has been completely covered by the parent node, it indicates that the text of the child node has high redundancy and is suitable for compression. Determine whether the incremental score of each child node is less than the incremental score threshold of 4. If so, compress the sentence unit text of that byte point using the prompt method of the large language model Qwen2 based on the text information compression prompt template.

[0057] sentence sjk =RAG_LLM(prompt_com,sentence sjk ,sentence sj )

[0058] Otherwise, the sentence unit text of that node remains unchanged.

[0059] (3d) Perform the same vectorization operation as in step 1), vectorize the sentence units of the child nodes corresponding to the compressed sentence unit text information based on the embedding model nomic-embed-text, and replace the original embedding vector of the child node with the vectorized embedding vector of dimension 768.

[0060] (3e) Replace the original embedding vector of the child node after text information compression with a 768-dimensional compressed embedding vector to obtain a clustering tree after text information compression of some node objects.

[0061] Step 4) Prune the clustering tree after compressing the text information of some node objects based on the QR-like decomposition method:

[0062] (4a) The information content of each child node's text is measured using Qwen2, calculating the amount of new information each child node can provide to the Qwen2 knowledge base. This information content measurement model can identify unique content in child nodes, especially content not mentioned in the parent node summary, and knowledge that Qwen2 has not yet learned from other data. The higher the information content of a child node, the greater its importance to the retrieval generation task. Information quantification (1-6 score) is achieved using a large model prompt method. A prompt template, prompt_imp, is written to calculate the incremental score of sentence unit text information relative to the Qwen2 internal parameter knowledge base based on the large language model prompt method.

[0063] "I am providing you with a summary of a long text and a specific excerpt from that text. What is the article title of the original text? If you know the title of the article from which this excerpt is taken, meaning you are familiar with the content of the original article, please assess how much important information this excerpt provides that you did not already know from the original.

[0064] Use an integer score from 1 to 6 to measure the amount of new information provided by the excerpt.

[0065] If you do not know the title of the article,answer'6'.If this excerptprovides no important information to you,answer'1'.

[0066] Summary:

[0067] {sentence2}

[0068] Excerpt:

[0069] {sentence1}

[0070] Please answer using Arabic numerals only 1,2,3,4,5,6:”

[0071] (4b) The Qwen2 large language model, based on incremental score calculation of the prompt template, calculates the incremental scores of the sentence unit text information of the child nodes in the first 4 layers of the 5-layer clustering tree after the compression of the text information of some node objects, relative to the knowledge base of the internal parameters of the Qwen2 large language model. Among them, the importance metric score Z of the k-th child node contained in the j-th parent node of the s-th layer is calculated. sjk The calculation formula is:

[0072] Z sjk =RAG_LLM(prompt_imp,sentence sjk ,sentence sj )

[0073] (4c) In order to make it easier to delete nodes with less information, the child node objects under the same cluster group are arranged in descending order to obtain the cluster tree after sorting the importance of the grouped child nodes.

[0074] (4d) Combine the embedding vectors of all child nodes in the clustering tree that have been sorted by importance measure into an embedding vector matrix E for that parent node. sj =[r sj1 ,r sj2 ,...,r sjk ,...,r sjK ], among which, E sj Let r represent the embedding vector matrix of the j-th parent node at level s in the sorted clustering tree, where s∈[2,3,...,5], j∈[1,2,...,J], J represents the number of parent nodes at level s, and k∈[1,2,...,K], where K represents the number of child nodes contained in the j-th parent node at level s; sjk Let E represent the embedding vector of the k-th child node contained in the j-th parent node of the s-th layer in the sorted clustering tree. Therefore, the dimension of the embedding matrix is ​​768×K, where 768 is the dimension of each embedding vector. Then, the UMAP dimensionality reduction algorithm is used to reduce the dimension of the embedding vectors in each embedding matrix to 10 dimensions, resulting in an embedding vector matrix E of dimension 10×K. sj :

[0075] E sj =UMAP(E sj (n_components=10)

[0076] Where n_components is the target dimension for dimensionality reduction in the UMAP algorithm. If the dimension of the embedding vector is small, dimensionality reduction is not required, and the original embedding matrix can be used directly.

[0077] (4e) Initialize the orthogonal vector matrix Q of the j-th parent node in the s-th layer of the clustering tree. sj =[q sj1 ,q sj2 ,...,q sjk ,...,q sjK Let the first column of the Q matrix be equal to the first column r1, q of its embedding vector matrix. sj1 =r sj1 The linear independence coefficient of this vector is set to 1, L sj1 =1, where L sjk The linear independence coefficient of the embedding vector of the i-th child node of the j-th parent node in the s-th level of the clustering tree is also the linear independence score of the node object;

[0078] (4f) Calculate the embedding vector r of the child node sjk The orthogonalization matrix Q of its parent node sj The projection vector onto the first K' column vectors is given, where K' represents the number of child nodes in the parent node that are ranked before this child node. Subtracting this projection vector yields the residual vector q of the child node. sjk :

[0079]

[0080] · represents the inner product operation, ||·|| represents the L1 norm operation, and ∑ represents the summation operation;

[0081] (4g) Calculate the ratio of the norm of the residual vector of the child node object to the norm of the embedding vector to obtain the linear independence coefficient of the embedding vector of that child node:

[0082]

[0083] The closer this ratio is to 1, the more linearly independent the current node object is from the previous node objects in the vector space; the closer the ratio is to 0, the more redundant the content of the current node object is from the previous node objects. In the evaluation of vector linear independence, i.e. node redundancy, in QR-like decomposition, vectors ranked later in the order map most of their own vector components into the projection of the preceding vectors, usually resulting in a smaller L. Therefore, nodes with less information measurement in step 4) are ranked later to make them easier to delete in subsequent pruning steps.

[0084] (4h) Determine the linear independence of each node object based on a preset threshold of 0.01. If the linear independence score of a node object is lower than 0.01, remove the node object; otherwise, retain the node object. Then, obtain a RAG database containing the structure and information of the pruned clustering tree.

[0085] The effects of the present invention will be further explained below with reference to simulation experiments.

[0086] 1. Simulation conditions:

[0087] The hardware platform for the simulation experiment was: Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz

[0088] It features a 2.81GHz CPU, 32GB of RAM, an NVIDIA GeForce RTX 2080 graphics processor, and runs on Ubuntu 22.04, Python 3.12, and PyTorch 2.4.1.

[0089] 2. Simulation content and result analysis:

[0090] The size and question-answering accuracy of the RAG databases constructed by this invention and existing technologies on two typical RAG datasets, QuALITY and QASPER, were compared through simulation. The results are shown in Table 1. Question-answering accuracy is the ratio of the number of correctly answered questions to the total number of questions. A correct answer is defined as one where the answer output by the large language model is exactly the same as the actual answer. Question-answering accuracy reflects the effectiveness of the text information stored in the RAG database. Database compression rate represents the reduction in text length, token number, and node number stored in the RAG database compared to existing technologies.

[0091] Table 1

[0092]

[0093] As can be seen from Table 1, the RAG database obtained by this invention can effectively reduce the memory requirements of the database while ensuring the validity of the stored text information.

Claims

1. A method for constructing a RAG database with information compression and pruning, characterized in that, Includes the following steps: (1) Construct node objects: Obtain N text data, each containing multiple sentences, and segment each text data according to the delimiter to obtain M sentence units from the N text data. Then, vectorize each sentence unit through an embedding model, and combine the embedding vector of each sentence unit with the sentence unit to form a node object, resulting in M ​​node objects, where N≥1000. (2) Construct a hierarchical clustering tree: Perform S iterations of semantic clustering on M node objects, and construct an S+1 layer clustering tree with M node objects as leaf nodes and G node objects as parent nodes, which are the summary sentence units of multiple grouped child nodes obtained in each iteration of semantic clustering and their corresponding embedding vector combinations. (3) Compress the text information of node objects in the clustering tree: The text information of sentence units in the child nodes covered by their parent nodes in the clustering tree is compressed, and the embedding vectors of the child nodes corresponding to the compressed sentence unit text information are updated based on the embedding model to obtain a clustering tree with compressed text information of some node objects. (4) Pruning the clustering tree after compressing the text information of some node objects based on a QR-like decomposition method: The child node objects contained in each parent node in the clustering tree after the text information of some node objects is compressed are sorted in descending order of importance metric scores. Based on the class orthogonal triangular QR decomposition method, the linear independence coefficient of the embedding vector of each sorted child node object contained in each parent node is calculated. Then, the child node objects with independence coefficients lower than the independence threshold ∈ are pruned to obtain the retrieval enhancement database containing the structure and information of the pruned clustering tree.

2. The method according to claim 1, characterized in that, The embedding model described in step (1) includes a cascaded word segmenter, an embedding layer, an encoder composed of multiple cascaded Transformer blocks, and an output layer. The Transformer blocks contain cascaded self-attention networks and two perceptron MLP feedforward networks.

3. The method according to claim 2, characterized in that, The steps described in step (1) for vectorizing each sentence unit using an embedding model are as follows: (1a) The input layer's word segmenter breaks down each sentence unit into token_len sub-words or word identifiers; the embedding layer maps each sub-word or word identifier to a D1-dimensional vector through a word embedding matrix; (1b) The multi-layer Transformer of the encoder layer uses a self-attention network to model the context of each vector, and obtains token_len context-related vectors. The output layer reduces the dimensionality of the token_len context-related vectors obtained by modeling, and obtains the embedding vector with dimension D2 corresponding to each sentence unit.

4. The method according to claim 1, characterized in that, The steps for performing S iterations of semantic clustering on the M node objects described in step (2) are as follows: (2a) Initialize the number of iterations for semantic clustering to s, and let s = 1. Initialize node object O1 to have M leaf node objects. (2b) Perform Gaussian mixture semantic clustering on node object O1 to obtain G s The semantic groups are divided into semantic groups, and the prompt method of the large language model RAG_LLM is used to summarize all sentence units in each semantic group, resulting in the summarized sentence unit Sum: Sum=RAG_LLM(prompt_sum,sentences) Here, `prompt_sum` represents the summary prompt template, and `sentences` represents the text content of all sentence units in each semantic group. Then, each summary sentence unit is vectorized, and each summary sentence unit and its vectorized summary embedding vector are combined into a node object. The resulting G... s O node objects s As the s-th layer of the clustering tree; (2c) Determine whether S = s. If yes, use the G node objects obtained from the S iterations of semantic clustering as the S+1 level of the clustering tree. Otherwise, let s = s+1 and O1 = O s and perform step (2b), where ∑ represents the summation operation.

5. The method according to claim 1, wherein The compression of the sentence unit text information in the child nodes covered by their parent nodes in the clustering tree described in step (3) is achieved as follows: (3a) The prompt method based on the large language model RAG_LLM is used to calculate the incremental score of the sentence unit text information of each child node covered by its parent node in the clustering tree relative to the sum of sentence unit text information of its parent node, where the incremental score A of the k-th child node contained in the j-th parent node of the s-th layer is calculated. sjk The calculation formula is: A sjk =RAG_LLM(prompt_add,sentence sjk ,sentence sj ) Where, prompt_add represents the prompt template for calculating the incremental score of text information of one sentence unit relative to the text information of another sentence unit, sentence sjk The sentence unit represents the sentence unit of the k-th child node contained in the j-th parent node of the s-th layer. sj This represents the sentence unit of the j-th parent node in the s-th layer; (3b) Determine whether A satisfies A < C with a pre-set incremental score threshold C. If so, compress the sentence unit text information of the child node based on the prompt method of the large language model RAG_LLM: sentence sjk =RAG_LLM(prompt_com,sentence sjk ,sentence sj ) where prompt_com represents the prompt template for compressing the sentence unit text content; otherwise, keep the sentence unit text information of the child node unchanged.

6. The method according to claim 5, characterized in that, The update method for updating the embedding vector of the child node corresponding to the compressed sentence unit text information based on the embedding model described in step (3) is as follows: Vectorize the sentence unit of the child node corresponding to the compressed sentence unit text information based on the embedding model, and replace the original embedding vector of the child node with the embedding vector of dimension D2 after vectorization.

7. The method according to claim 5, characterized in that, In step (4), the multiple child node objects contained in each parent node in the clustering tree after the partial node object text information is compressed are sorted in descending order of importance metric scores, where the importance metric score Z of the kth child node contained in the jth parent node in the sth layer is... sjk The calculation formula is: Z sjk =RAG_LLM(prompt_imp,sentence sjk ,sentence sj ) where prompt_imp represents the prompt template for calculating the incremental score of the child node sentence unit text information relative to the internal parameter knowledge base of the large language model RAG_LLM.

8. The method according to claim 7, characterized in that, The linear independence coefficient of the embedding vector of each sorted child node object contained in each parent node, as described in step (4), is calculated using the orthogonal triangular QR decomposition method described in step (4). The linear independence coefficient L of the k-th child node contained in the j-th parent node at level s is... sjk The calculation formula is: Where s∈[2,3,...,S+1], j∈[1,2,...,J], J represents the number of parent nodes in the s-th level, k∈[1,2,...,K], K represents the number of child nodes contained in the j-th parent node in the s-th level; r sjk q represents the embedding vector of the k-th child node contained in the j-th parent node of the s-th level in the sorted clustering tree. sjk q represents the residual vector after orthogonal projection of the embedding vector of the child node. sjk' represents the residual vector after orthogonal projection of the embedding vector of the k'-th child node preceding this child node, where K' represents the number of child nodes preceding this child node; · represents the inner product operation, ||·|| represents the L1 norm operation, and ∑ represents the summation operation.

Images