Method, device and medium for search enhancement generation based on hierarchical tree and self-reflection

Abstract

The present application relates to the technical field of artificial intelligence, in particular to a retrieval enhancement generation method, device and medium based on hierarchical tree and self-reflection, the method comprising: constructing a multi-level tree structure knowledge base, the knowledge base comprising a plurality of tree index structures constructed based on text similarity, enterprise organizational structure and job level structure; receiving user query and user identity information; generating a plurality of sub-queries corresponding to different retrieval intentions based on the user query and the user identity information; performing multi-path retrieval of each sub-query in the corresponding tree index structure to obtain a relevant knowledge fragment set; generating a final answer based on a self-reflection mechanism, while realizing knowledge tracing. The present application solves the problems of low knowledge base quality, poor retrieval result relevance and single retrieval path existing in traditional RAG technology, and significantly improves the accuracy, relevance and interpretability of enterprise system question answering.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a retrieval enhancement generation method, device, and medium based on hierarchical trees and self-reflection. Background Technology

[0002] With the rapid development of generative large language models (such as Qwen and DeepSeek), natural language question answering services are widely used in various fields. However, pure large language models rely on fixed knowledge from the training phase and cannot automatically adapt to dynamic changes in external information (such as iterative updates of corporate rules and regulations), leading to errors or outdated issues in the generated data. To overcome this deficiency, retrieval-enhanced generation (RAG) technology has emerged.

[0003] The core architecture of RAG technology consists of an "indexing module + retrieval generation module". It improves the quality of answers by introducing an external knowledge base: the indexing module parses and segments the original text, transforms it into searchable knowledge fragments, and builds a knowledge base; the retrieval generation module retrieves knowledge related to the user's query from the knowledge base, combines the knowledge with the question as input with prompt word engineering, and generates accurate and evidence-based answers through a large language model.

[0004] However, the current baseline RAG technology, in the specific scenario of corporate rules and regulations Q&A, is limited by technical design flaws and struggles to adapt to the core characteristics of corporate rules and regulations (rigor, hierarchy (applicable to different departments and job levels), and dynamism), exhibiting significant limitations.

[0005] The knowledge base is of poor quality: the baseline RAG divides text into blocks based on a fixed term length, resulting in knowledge fragments that lack hierarchical relationships and are scattered in a two-dimensional plane. Searching requires a global traversal and comparison of all fragments, which is inefficient; furthermore, because the fragments do not reflect the hierarchical logic of the content, they cannot meet the enterprise's search needs for a "general overview + detailed provisions" of regulations.

[0006] Insufficient relevance and consistency of search results: The search query directly uses the user's original question without considering the contextual characteristics of the company's system (such as departmental responsibilities and job level permissions) to distinguish the relevance of paragraphs to the query purpose and semantics. This results in knowledge that matches the user's actual needs (such as "the reimbursement system of a certain department" or "the attendance rules of grassroots employees") not being retrieved.

[0007] The retrieval path is singular and the results are incomplete: knowledge fragments are obtained only through a single query as the input context for the large language model, without covering multi-dimensional retrieval paths such as corporate organizational structure and job level system; moreover, the fragments are directly input into the model to generate answers after retrieval, lacking the validity verification and supplementary retrieval mechanism for the retrieved content, and failing to supplement potential necessary knowledge (such as the institutional connection clauses for cross-departmental collaboration), resulting in limited coverage of results.

[0008] In summary, the aforementioned shortcomings of the baseline RAG technology make it difficult to meet the needs of enterprise users for "accurate matching of departments / job levels, balancing general and detailed information, and covering multiple scenarios" in their Q&A of rules and regulations. There is an urgent need for an optimized RAG method that is adapted to the enterprise rules and regulations scenario to solve the problems of existing technology in terms of retrieval efficiency, result relevance, and comprehensiveness. Summary of the Invention

[0009] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0010] According to a first aspect of the present invention, a retrieval enhancement generation method based on hierarchical trees and self-reflection is provided, the method comprising the following steps:

[0011] S100, Construct a multi-level tree structure knowledge base. The knowledge base includes multiple tree index structures formed by organizing knowledge fragments based on semantic dimension, organizational structure dimension and job level dimension. The multiple tree index structures establish the relationship between dimensions through a unified cross-reference mapping table.

[0012] S200, receive user queries and user identity information, the user identity information including department identifier and job level permission value.

[0013] S300: Based on the user query and user identity information, the query intent is decomposed to generate multiple sub-queries corresponding to the semantic dimension, organizational structure dimension, and job level dimension, respectively.

[0014] S400 employs a parallel retrieval strategy, sending each subquery to the corresponding tree index structure for retrieval, and during the retrieval process, collaborative verification of retrieval results across dimensions is achieved based on the cross-reference mapping table to obtain a set of relevant knowledge fragments.

[0015] S500 evaluates the set of knowledge fragments based on a self-reflection mechanism and generates a final answer.

[0016] According to a second aspect of the present invention, an electronic device is provided, including a processor and a memory; the processor executes the steps of the method described in the first aspect of the present invention by invoking a program or instructions stored in the memory.

[0017] According to a third aspect of the present invention, a computer-readable storage medium is provided that stores a program or instructions that cause a computer to perform the steps of the method described in the first aspect of the present invention.

[0018] Compared with the prior art, the technical solution provided by the present invention can bring the following significant beneficial effects:

[0019] (1) Significantly improved the accuracy and comprehensiveness of retrieval: By constructing multiple tree-shaped index structures based on different dimensions (semantics, organization, job level) and rewriting and expanding user queries with multiple intents, concurrent retrieval of the knowledge base from multiple angles and levels was achieved. This overcomes the limitations of the traditional single retrieval path and can more comprehensively and accurately recall the knowledge fragments most relevant to the user query and its own context (department, job level), effectively solving the problems of "incomplete retrieval results" and "low relevance".

[0020] (2) Significantly enhanced factual accuracy and contextual relevance of generated answers: By introducing a self-reflective (Self-RAG) mechanism, reflective markers (such as relevance, support, and usability) are used to critically evaluate and filter retrieved knowledge during the generation process. This acts like a built-in "quality inspector," ensuring that the large language model generates answers based solely on highly relevant and supportive knowledge, effectively suppressing the "illusion" phenomenon and making the generated answers not only accurate but also highly relevant to the user's specific identity and query intent.

[0021] (3) Improved system performance and user experience: The tree-structured knowledge base organization method, compared with the traditional flat knowledge block storage, allows for more efficient tree traversal or tree folding algorithms during retrieval, reducing the number of vectors that need to be compared for similarity, thereby improving retrieval efficiency while ensuring recall effect. In addition, the final knowledge tracing function enhances the credibility and interpretability of the answers, allowing users to know the specific institutional source of the answers, thus improving the user experience.

[0022] (4) Deep integration of technology and business scenarios: This invention is not a general RAG improvement, but is specifically designed for the complex and domain-specific scenario of "corporate system Q&A". By introducing "organizational structure" and "job level structure" as the core dimensions of knowledge organization and query rewriting, the technical solution is closely integrated with the actual management structure and permission system of the enterprise, solving the pain points in enterprise knowledge management and having high practical value and application feasibility.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 The flowchart illustrates the retrieval enhancement generation method based on hierarchical trees and self-reflection provided in this embodiment of the invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0028] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0029] Explanation of related terms in this invention

[0030] 1. RAG (Retrieval Augmented Generation): A natural language processing framework that enhances the accuracy and richness of generated responses by retrieving relevant information fragments from external knowledge bases and integrating them as context into the input of a generative model. This technique effectively mitigates the factual errors or outdated information problems that may arise from purely generative models.

[0031] 2. RAPTOR (Recursive Abstractive Processing for Tree-Organized Retrieval): A hierarchical retrieval technology. It generates a multi-level tree-like index structure by recursively abstracting and summarizing text blocks. The top level of this structure is a highly summarized summary, and the bottom level contains the specific text blocks, enabling efficient and accurate information location from coarse-grained to fine-grained during retrieval.

[0032] 3. Self-RAG (Self-Reflective Retrieval-Augmented Generation): An advanced RAG paradigm. It introduces a reflective mechanism into the generation process, evaluating retrieved knowledge through specific critical markers (such as relevance and support) to dynamically determine whether retrieval is necessary and how to utilize the retrieval results, ultimately generating answers that are more factually accurate, context-relevant, and logically coherent.

[0033] 4. Embedding Vector: A distributed representation technique that maps discrete symbols (such as words, sentences, and paragraphs) to a continuous high-dimensional vector space. In this space, entities with similar semantics are closer in distance to each other. This invention utilizes embedded vectors to represent text semantics and calculates semantic similarity based on these vectors to achieve efficient retrieval.

[0034] 5. Token: The smallest semantic unit obtained after processing the original text by a word segmentation algorithm. It can be a word, subword, or character, etc. The token sequence constitutes the basic input and output form of a large language model, and the model's understanding and generation are both calculated using tokens as the basic unit.

[0035] This invention aims to provide a Q&A service for corporate rules and regulations. Based on the traditional baseline RAG technology, it combines RAPTOR technology to construct a new RAG technology framework based on a self-reflection framework. This improves the quality of RAG Q&A, enhances the semantic relevance and consistency between search results and user queries, broadens the search path, and makes the search results more comprehensive, thereby solving the three main problems and limitations faced by the baseline RAG technology.

[0036] This invention proposes a retrieval enhancement generation method based on hierarchical trees and self-reflection. This method combines additional information such as the relationship between the user's department and the company's organizational structure, and the user's own job level, to more efficiently and comprehensively retrieve the most relevant knowledge fragments to the user's query, generating accurate, reliable, and traceable high-quality answers. Through this method, enterprise users can quickly obtain information with internal company characteristics based on a knowledge base, thereby enabling internal company analysis.

[0037] Furthermore, such as Figure 1 As shown, the retrieval enhancement generation method based on hierarchical trees and self-reflection provided in this embodiment of the invention may include the following steps:

[0038] S100, Construct a multi-level tree structure knowledge base. The knowledge base includes multiple tree index structures formed by organizing knowledge fragments based on semantic dimension, organizational structure dimension and job level dimension. The multiple tree index structures establish the relationship between dimensions through a unified cross-reference mapping table.

[0039] Traditional baseline RAGs simply divide document content into chunks based on expected chunk lengths. This knowledge chunking data structure resembles a two-dimensional plane, with different chunks randomly scattered across it. Retrieving chunks containing valid information requires a global traversal search and comparison, which is inefficient. In RAPTOR's tree structure, knowledge chunks are not simply randomly scattered across a plane, but rather arranged semantically at the bottom layer of the tree. Using specific clustering methods, the content of the bottom-layer chunks is summarized and aggregated into upper-level nodes. Each upper-level node includes relevant information from its upstream and downstream nodes. The document content presents a clear hierarchy, generating sufficient contextual connections and providing ample reference for the final result generation.

[0040] In this invention, based on the segmentation technology of baseline RAG, RAPTOR technology is introduced to construct a tree-structured hierarchical knowledge segmentation structure. Combining the actual characteristics of enterprise rules and regulations documents, three tree-structured knowledge segments are constructed: a text similarity clustering tree based on Gaussian Mixture Model (GMM), a job level clustering tree based on enterprise job level architecture, and an organizational clustering tree based on enterprise organizational structure. This embeds, clusters, and summarizes knowledge text from bottom to top, layering and summarizing information in the knowledge base from three aspects. This approach effectively captures both high-level and low-level details in the text, overcoming the limitation of baseline RAG's poor understanding of context.

[0041] Furthermore, S100 specifically includes:

[0042] S110, preprocess the original text to obtain the preprocessed text.

[0043] In this embodiment of the invention, the original text is a corporate rules and regulations document, covering basic management and general principles, human resources and labor relations management, operations and administration, finance and audit management, compliance and risk management, safety production and environmental management, intellectual property management, and other related systems. The texts of these various rules and regulations documents are systematically collected through communication with relevant functional departments within the company.

[0044] In this embodiment of the invention, the preprocessing operation includes format standardization and content filtering:

[0045] Format standardization: The original text includes multiple formats such as OFD, DOC, DOCX, and PDF. Tools for parsing OFD format files are relatively scarce, so the file format can be manually converted to Word or PDF format using file format conversion tools such as Foxit PhantomPDF, or directly through a virtual printer driver. For PDF format documents, a distinction is made between native PDFs that can be directly extracted and image-based PDFs obtained through scanning, etc. Native PDF format is preferred to improve parsing quality, while image-based PDF documents are processed using a combination of optical character recognition and layout analysis.

[0046] Content filtering: Non-institutional content documents, including appointment documents and notices, contained in the text are identified and filtered out to retain only the pure corporate rules and regulations.

[0047] S120 performs parsing, knowledge segmentation, and vectorization on the preprocessed text to obtain multiple knowledge fragments and their corresponding embedding vectors.

[0048] To transform the collected text set of company rules and regulations into searchable question-and-answer data, the preprocessed DOC, DOCX, and PDF documents should be parsed to extract their text content, resulting in the parsed text content, specifically including:

[0049] For DOCX format documents, text content can be extracted based on the original paragraph structure using document parsing tools; for example, the python-docx package can be used to load and read file content, thereby obtaining the file content of docx format text based on the original paragraphs of the file.

[0050] For DOC format documents, first convert them to DOCX format, and then extract the text content using a document parsing tool. For example, using the LibreOffice software on Ubuntu, the format conversion can be completed based on the command line command "libreoffice –convert-todocx [filename.doc]" (for Windows systems, file conversion can be performed using other office software such as Microsoft Word), and then the file can be parsed using the python-docx package.

[0051] For PDF documents, differentiate between processing methods: For native PDF documents, use text extraction tools to obtain text and metadata; for image-based PDF documents, use optical character recognition (OCR) tools combined with layout analysis tools to extract and reconstruct the text content. For example, for native PDF files, Python tools such as PyPDF, pdfplumber, and pdfminer can be used to load and read the file, extracting its complex formatting and content; for image-based PDFs, OCR methods should be used for content extraction and understanding. Specifically, this includes various OCR tools such as EasyOCR and PaddleOCR to extract the content, combined with layout detection solutions such as DeepDoc to identify the complex file structure and infer the content of the image-based PDF file.

[0052] The parsed text content is processed into knowledge blocks to generate knowledge fragments suitable for retrieval, including:

[0053] First, initial block division is performed based on paragraph boundaries;

[0054] For paragraphs longer than 512 words, they are further divided into blocks according to sentence boundaries to ensure that the length of each knowledge segment does not exceed 512 words.

[0055] For sentences with a length exceeding 512 words in extreme cases that cannot be segmented based on sentence length, forced segmentation based on text length is implemented.

[0056] Furthermore, the knowledge fragments obtained after segmentation are vectorized, converting them from natural language form into dense vector representations that reflect the semantic information of the knowledge fragments. This invention employs a pre-trained text embedding model to map each knowledge fragment into a 1024-dimensional embedding vector. For example, the latest Qwen3-Embedding-8B model is used for knowledge fragment vectorization. This model is specifically designed for text embedding and ranking tasks, has 8 billion parameters, allows for customizable output dimensions, can generate embedding vectors up to 4096 dimensions, and possesses long text understanding and reasoning capabilities.

[0057] S130, Based on the embedding vector, a first tree index structure is constructed, wherein the nodes of the first tree index structure are formed based on text semantic clustering.

[0058] Furthermore, S130 specifically includes:

[0059] S1301 uses a Gaussian mixture model to perform soft clustering on the embedding vectors of knowledge fragments, generating bottom-level cluster nodes.

[0060] GM (Gazette-Meaning) is a flexible and powerful statistical model that can not only estimate probability density but also be applied to cluster analysis. In the clustering process, GMM assumes that the data consists of multiple Gaussian distributions, each corresponding to a cluster. The probability of each data point belonging to a cluster is determined by the weights of the Gaussian distributions. Therefore, unlike traditional hard clustering methods (where each data point belongs to only one category), GMM-based clustering is a soft clustering, where the same data point may be included in different categories simultaneously. This is more consistent with the actual situation in document processing. A knowledge fragment may contain content from different topics, and these should naturally be included in clusters of different topics simultaneously, rather than being mutually exclusive. Based on this model, the posterior probability of each knowledge fragment belonging to each predefined cluster is calculated, achieving soft partitioning and allowing the same knowledge fragment to be assigned to multiple semantic categories with different probabilities. The embedding vectors of knowledge fragments are aggregated into different categories through GMM clustering. Based on the GMM soft clustering results, each identified semantic cluster is mapped to an initial tree node, called the bottom-level cluster node. Each bottom-level node records the identification information of the knowledge fragments it contains and their corresponding membership probabilities. For example, given the 10 embedding vectors mentioned above, which are classified into 5 classes by the GMM, there are 5 bottom-level clustering nodes above the underlying text blocks. These nodes will first contain the node's own number, and then the numbers of its child nodes or the text blocks they contain.

[0061] In one embodiment of the present invention, clustering is performed using the GaussianMixture class from the scikit-learn library. The number of cluster components is set to 10, the covariance type is set to full, and the expectation-maximization (EM) algorithm is used for model fitting. The model input is a matrix composed of 1024-dimensional embedding vectors of all knowledge fragments. After fitting, the predict_proba method is called to obtain the posterior probability matrix of each knowledge fragment belonging to each cluster. Here, the probability threshold θ is set to 0.2, meaning that associations with a posterior probability greater than 0.2 are retained, thereby achieving soft clustering.

[0062] Next, the obtained posterior probability matrix is ​​traversed, and a node object is created for each cluster (i.e., a Gaussian component of the GMM). This node object contains the following attributes: node ID (automatically generated unique identifier), a list of child node IDs (initially empty), and a list of associated knowledge fragment IDs and their membership probabilities. After this step, the first level of the tree nodes is formed.

[0063] In another embodiment of the present invention, the number of GMM clustering components is dynamically adjusted by the silhouette coefficient, the clustering granularity is determined according to the topic density of the institutional text, and the embedding vectors of knowledge fragments are softly clustered to form the bottom clustering nodes.

[0064] The technical solution described in this embodiment is adapted to the knowledge organization scenario of corporate rules and regulations texts. Addressing the core characteristics of corporate rules and regulations texts—"strong correlation between clauses, diverse thematic dimensions (such as financial reimbursement, human resource management, compliance and risk control, etc.), and clear hierarchical logic"—it solves the problems of "overly coarse clustering (multi-theme mixing)" or "overly fine clustering (theme fragmentation)" caused by the failure to consider the differences in theme density when processing such texts using traditional fixed-component GMM (Gaussian Mixture Model) clustering. By introducing a contour coefficient, it achieves dynamic optimization of the number of clustering components and, combined with the characteristics of soft clustering, preserves the multi-theme membership relationships of knowledge fragments, providing a high-quality underlying clustering foundation for the subsequent construction of a hierarchical semantic clustering tree (first tree index structure).

[0065] The silhouette coefficient is used to quantify the quality of clustering. Its value ranges from -1 to 1. The closer the coefficient value is to 1, the higher the similarity of samples within the same cluster (good cohesion) and the lower the similarity of samples between different clusters (good separation). When the coefficient value is ≤0, it indicates that there are sample classification errors or topic mixing problems in the clustering results.

[0066] The number of cluster components refers to the number of Gaussian distribution components preset in the GMM model, which corresponds to the number of underlying clusters formed after clustering. Its value directly determines the clustering granularity (the more components, the finer the clustering granularity, and the more singular the knowledge fragments contained in a single cluster; the fewer components, the coarser the clustering granularity, and the more diverse the knowledge fragments contained in a single cluster).

[0067] The subject density of a policy text refers to the proportion of fragments belonging to the same policy subject (such as "travel allowance standards" or "labor contract renewal procedures") within a unit quantity of knowledge fragments. When the subject density is ≥ g1, it is defined as a "high subject density region"; when the subject density is ≤ g2, it is defined as a "low subject density region". g1 is a first preset value, and in an illustrative embodiment, g1 = 60%. g2 is a second preset value, and in an illustrative embodiment, g2 = 30%.

[0068] The number of GMM clustering components is dynamically adjusted by the silhouette coefficient, and the clustering granularity is determined according to the topic density of the institutional text. Soft clustering is performed on the embedding vectors of knowledge fragments to form the bottom-level clustering nodes, specifically including:

[0069] Step 1: Determine the initial search range for the number of cluster components

[0070] Using the preprocessed set of embedded vectors for enterprise policy knowledge fragments (denoted as V, containing Q vectors, Q≥100, corresponding to Q knowledge fragments) as input, and based on the statistical results of the topic dimension of the enterprise policy text, the initial search interval for the clustering components is set to [K]. min ,K max ],in:

[0071] The topic dimension statistics are obtained by labeling knowledge fragments with topics (using a large language model combined with a corporate policy terminology database, which includes primary topics such as "finance, human resources, compliance, administration" and their subordinate secondary topics), and the number of independent topics T in the text is obtained.

[0072] Preferably, K min =max(roundup(T / 2),5),K max =min(2T, 30), ensuring that the initial search range covers both the potential optimal number of cluster components and avoids wasting computational resources due to an excessive number of components (e.g., when T=10, K...). min =5,K max =20). Roundup() means rounding up.

[0073] Step 2: Dynamic optimization of clustering components based on silhouette coefficient

[0074] For the initial search interval [K] min K max The number of candidate clustering components K (K is an integer, and K is greater than or equal to 1) within each cluster. min ≤K≤K max ), perform the following operations:

[0075] (1) Constructing a K-component GMM model: The expectation-maximization (EM) algorithm is used to fit the vector set V. The mean, covariance matrix and mixing weights of each Gaussian distribution are iteratively optimized. The iteration termination condition is "the difference between the log-likelihood function values ​​of two adjacent iterations is ≤10". -4 "or "the number of iterations is ≥100";

[0076] (2) Calculate the silhouette coefficient of the clustering results:

[0077] The overall silhouette coefficient of the K-component clustering result is S(K) = (1 / Q) × ∑ Q j=1 s j , where s j For the j-th embedding vector v in the vector set V j Profile coefficients (j=1 to Q), s j =(b j -a j ) / max(a j b j );a j For v j a is the average distance to all other vectors within its cluster. j Characterizing cohesion, the smaller the distance, the better; b j For the embedding vector v jThe minimum average distance b between the vector and all other clusters j b j To characterize the separation degree, the greater the distance, the better.

[0078] (3) Determine the optimal number of cluster components K: Iterate through all candidate K values ​​and select the K that makes the overall profile coefficient S(K) reach the maximum value as the optimal number of cluster components K; if there are multiple K values ​​corresponding to the same maximum S(K), select the smallest K value that makes the "theme purity in the high theme density area cluster greater than or equal to the preset theme purity, for example, 90%" (theme purity = number of single theme fragments in the cluster / total number of fragments in the cluster).

[0079] Step 3: Adapt and adjust the clustering granularity based on topic density.

[0080] Based on the optimal clustering component number K obtained in step 2, the clustering granularity is further fine-tuned by combining the topic density distribution of the institutional text to ensure that the clustering results are adapted to the topic distribution characteristics of the enterprise's institutional text.

[0081] (1) Topic density calculation: For the knowledge fragments corresponding to the vector set V, divide them into regions according to the preset first-level topics of enterprise system (such as "finance" and "human resources"), and calculate the topic density D of each region. i (i is the first-level topic number, where i ranges from 1 to H, and H is the number of first-level topics).

[0082] (2) Clustering granularity adjustment rules:

[0083] If a certain region D i ≥g1 (High topic density region): Increase the number of corresponding clusters in this region by 1-2, that is, increase the number of local cluster components based on K, so that single topic fragments in this region form independent clusters and avoid topic mixing;

[0084] If a certain region D i ≤g2 (low topic density region): merge adjacent clusters within this region that have a semantic similarity ≥0.7 (calculated by vector cosine similarity) (i.e., reduce the number of local cluster components based on K to avoid topic fragmentation).

[0085] After adjustment, the final number of cluster components K is obtained. final And K final ∈[K–2, K+2], ensuring the overall number of clusters remains stable.

[0086] Step 4: Execution of soft clustering based on GMM and generation of underlying clustering nodes

[0087] Based on the final number of cluster components K final Constructing a GMM model involves performing soft clustering on the knowledge fragment embedding vector set V to generate underlying cluster nodes, specifically including:

[0088] (1) GMM model fitting: The EM algorithm is used to fit the vector set V to obtain K. final The parameters of a Gaussian distribution, where each Gaussian distribution corresponds to a potential cluster;

[0089] (2) Posterior probability calculation: For each knowledge fragment embedding vector, calculate its membership in each cluster c (c takes values ​​from 1 to K). final The posterior probability P of ).

[0090] (3) Soft clustering classification: Set a posterior probability threshold θ (θ=0.2, optimized based on enterprise system text clustering experiments), if v j The posterior probability P(c|v) of belonging to cluster c j If )≥θ, then v j The corresponding knowledge fragment is associated with cluster c, achieving a soft clustering effect where "a knowledge fragment can belong to multiple clusters" (adapting to the "cross-topic clauses" characteristic in corporate policy fragments, such as "cross-departmental collaboration expense reimbursement clauses" belonging to both the "finance" and "administration" clusters).

[0091] (4) Construction of underlying cluster nodes: A bottom-level cluster node is constructed for each cluster c, and the node contains the following attributes:

[0092] The node's unique identifier ID (e.g., "CLUSTER-BASE-001");

[0093] The set of knowledge fragments associated within a cluster (containing the unique ID of each knowledge fragment and its corresponding posterior probability P(c|v)) j ));

[0094] The topic tags for knowledge fragments within a cluster (the large language model summarizes the content of all fragments within the cluster, generating 1-2 core topics, such as "financial reimbursement - domestic business travel");

[0095] The average embedding vector of the in-cluster vectors (used for subsequent clustering and retrieval of higher-level nodes).

[0096] This embodiment has the following advantages:

[0097] Clustering quality optimization: By dynamically adjusting the number of cluster components through the silhouette coefficient, the overall silhouette coefficient S(K) of the clustering results is ≥0.6 (compared to S(K)≤0.4 for GMM clustering with a fixed number of components), which significantly improves the cohesion and separation of clusters and avoids "scattered clauses on the same topic" or "mixed clauses on different topics" in corporate policy documents;

[0098] Enhanced adaptability: The clustering granularity is adjusted according to the theme density of the policy text, achieving fine-grained clustering in high theme density areas (such as the "concentrated area of ​​financial reimbursement clauses") and coarse-grained clustering in low theme density areas (such as the "dispersed area of ​​cross-departmental collaboration clauses"), thus adapting to the heterogeneity of the theme distribution of corporate policy texts.

[0099] Enhanced flexibility: The soft clustering mechanism allows knowledge fragments to be associated across clusters, solving the problem that traditional hard clustering (where a fragment belongs to only one cluster) cannot handle "cross-topic clauses" of enterprises, and providing a more comprehensive underlying knowledge association relationship for the construction of subsequent hierarchical semantic clustering trees.

[0100] S1302, generate summary content for each bottom cluster, and vectorize the summary content to form upper-level nodes.

[0101] For each bottom-level cluster node, a pre-trained large language model is used to summarize the content of all knowledge fragments under that node, generating a natural language summary text. This summary represents the core institutional themes or content outlines covered by the node. Subsequently, the summary text is converted into a summary vector using the same embedding model as the knowledge fragments, completing the distributed semantic representation of the node. Specifically, for each generated bottom-level cluster node, the original text content of all its associated knowledge fragments is collected. These text contents are concatenated and input into the large language model (specifically, the Qwen2-7B-Instruct model is used in this invention) to generate a summary. The prompt word template is designed as follows: "You are a corporate policy expert. Please strictly summarize the core themes and key points based on the following text content and generate a concise summary. Text content: [TEXT]". After the summary is generated, the summary text is converted into a 1024-dimensional summary vector using the same Qwen3-Embedding-8B model as step S120 and stored in the node object.

[0102] For example, if node 1 contains the content of text chunk 1 and text chunk 2, the large language model will summarize the content of these two chunks and express it in natural language, storing it in node 1 as well. Then, this natural language summary is also converted into an embedding vector, thus forming a complete tree node at the next level based on the underlying text chunks.

[0103] S1303, recursively execute the clustering and summary generation steps until the preset termination condition is met, thereby forming a multi-level semantic clustering tree and obtaining the first tree index structure.

[0104] Using the summary vectors of each bottom-level cluster node obtained in S1302 as the new input dataset, the clustering, node generation, and summary vectorization processes in S131 to S132 are repeated. Higher-level tree nodes and their summaries are recursively constructed from bottom to top until a preset recursion termination condition is reached. Each recursion generates a higher-level tree node, representing a more abstract topic summary. This ultimately forms a complete tree index structure with the root node at the top level and leaf nodes linking to original knowledge fragments—the first tree index structure. Higher-level nodes correspond to more abstract and generalized institutional topic summaries. The recursion termination condition is set as follows: the number of nodes in the current layer is less than or equal to 3, or the recursion depth reaches 5 levels.

[0105] In this embodiment of the invention, the final generated first tree-shaped index structure is serialized in JSON format and persistently stored. The stored information includes: the IDs, levels, summary vectors, summary texts, and a list of child node IDs for all nodes, as well as the mapping relationship between each node and its associated knowledge fragment IDs and probabilities. This metadata file is loaded by the indexing module for subsequent retrieval operations.

[0106] S140, Based on a predefined organizational structure, construct a second tree-shaped index structure, wherein the nodes of the second tree-shaped index structure correspond to different organizational departments, and associate knowledge fragments with the corresponding department nodes.

[0107] Furthermore, S140 specifically includes:

[0108] S1401, Based on the actual organizational hierarchy and departmental division of the target company, construct a tree-like framework with organizational departments as nodes.

[0109] Corporate policies refer to a set of rules, procedures, and policies that an enterprise follows in its operations, encompassing all aspects of its operations. An enterprise's organizational framework is its actual organizational structure and management system, including its organizational hierarchy and departmental divisions. This framework forms the foundation of the enterprise's operations, determining how resources are allocated, work is coordinated, and goals are achieved. There is a close, interdependent relationship between corporate policies and the organizational framework. Different departments or organizations within an enterprise may have different applicable policy provisions. Establishing an organizational framework—a structural framework based on the organizational structure's clustering tree—aims to better systematize knowledge from the perspective of specific departments / organizations within the enterprise.

[0110] For a specific enterprise, there are multiple different departmental levels, each level contains multiple different departments, and each department may have multiple different sub-departments. Therefore, based on the specific circumstances of the enterprise, an organizational framework can be constructed. In this framework, the enterprise as a whole is used as the root node, and it expands downwards according to different departmental levels (including but not limited to subsidiaries, business groups, first-level departments, second-level departments, and sub-departments) to form a tree structure with parent-child relationships. Finally, the leaf nodes correspond to the most basic departmental units in the enterprise. This framework serves as the skeleton of the second tree-like index structure, used for the mounting and organization of subsequent knowledge fragments.

[0111] S1402, The large language model is used to automatically determine and establish the association between each knowledge fragment and the department nodes in the tree framework.

[0112] This step specifically includes:

[0113] Develop a prompt template: Design structured prompts for LLM, clearly defining the task as determining the relevant department or organization based on knowledge fragments. The template should include a task description, output format constraints, and placeholders. An example is shown below:

[0114] ### Role

[0115] You are a corporate policy analysis expert, skilled at accurately determining the applicable internal departments of a company based on the content of policy provisions.

[0116] ### Task

[0117] Based on the given "[Organizational Framework List]" and "[Knowledge Fragment Content]", please determine the specific department that this policy provision regulates, binds, or applies to. Department names must strictly derive from the provided framework list.

[0118] ### Output Requirements

[0119] 1. The output must strictly follow the following format: each department name on a separate line and wrapped in a specific tag.

[0120] 2. Output only the department name, without any additional explanation.

[0121] ### Output Format Example

[0122] <beg>Human Resources Department<END|>

[0123] <beg>Finance Department<END|>

[0124] ### List of Enterprise Organizational Frameworks

[0125] {enterprise_organizational_structure}

[0126] ### Knowledge Fragment Content

[0127] {chunk_content}

[0128] Use LLM for association: Fill the prompt word template with the content of each knowledge fragment and the list of enterprise organizational frameworks, call LLM (such as Qwen2-7B-Instruct) for processing, and obtain its output list of department names.

[0129] Parse the output and establish links: Parse the LLM output and extract the data. <beg>and<END|> Label the department name of the package. Link the unique identifier of this knowledge fragment to the corresponding department node in the enterprise organizational framework. A knowledge fragment can be associated with multiple department nodes simultaneously.

[0130] S1403, generate department node summaries from bottom to top and form a second tree index structure.

[0131] Based on the link relationships established in S1402, a summary of each department node in the organizational tree is generated, forming the final tree index structure. The specific steps are as follows:

[0132] S14301, Generate Leaf Node Summary: Starting from the leaf department nodes (i.e., nodes without sub-departments) of the enterprise organizational framework, traverse each leaf node. Input the text content of all knowledge fragments directly associated with that node into the LLM, and generate a natural language summary that summarizes the key points of the department's system.

[0133] S14302, Generate internal node summaries: Traverse non-leaf nodes from bottom to top. For each such node, input the summary text of all its sub-departments and the content of its directly related knowledge fragments into the LLM to generate a more general summary that represents the overall system and regulations of the node and its subordinate departments.

[0134] S14303, Vectorized Storage: Use a specified text embedding model (such as Qwen3-Embedding-8B) to convert the generated summary text of each node into a vector representation (such as a 1024-dimensional vector), and store it together with other metadata of the node (including node ID, parent node ID, list of child node IDs, list of associated knowledge fragment IDs, etc.).

[0135] S14304, Construct a complete index: Recursively execute the above process until the root node, and finally form a complete organized clustering tree with related knowledge fragments and vectorized summaries, namely the second tree index structure. The summaries of all nodes and the embedded vector representations of the underlying blocks will be used as the final vector knowledge base content for question answering.

[0136] S150, based on a predefined job level architecture, construct a third tree index structure, wherein the node content of the third tree index structure is organized differently according to job level.

[0137] Furthermore, S150 specifically includes the following steps:

[0138] S1501, traverse the first tree index structure and generate a job grade differential summary.

[0139] Based on the first tree-shaped index structure (i.e., semantic clustering tree) generated in the preceding steps, each node in the tree is traversed. For each node, its summary text and the content of the associated original knowledge fragments are used as input, and a large language model is invoked to generate multiple differentiated summary versions suitable for different job levels.

[0140] The specific implementation process includes:

[0141] S15011 is a predefined corporate job level system that includes all job levels within the company (such as "employee", "manager", "director", "senior executive").

[0142] S15012, Generate Job Level Summary for Each Node: For the content of the current node, use a large language model (such as Qwen2-7B-Instruct) to generate summaries applicable to each of the above job levels. An example of the prompt word template used is as follows:

[0143] "### Task

[0144] You are a corporate human resources and policy management expert. Based on the given policy content, please generate a policy summary applicable to personnel at [position level]. Please highlight the clauses that personnel at this position must know, comply with, or implement, and omit irrelevant content.

[0145] ### Original Text of the Policy

[0146] {node_content}

[0147] ### Require

[0148] The summary should be concise, accurate, and directly address the role responsibilities and authority of [a specific job level].

[0149] S15013, Handling irrelevant job levels: If the node content is completely unrelated to a specific job level, the large language model generates a null value summary (null or empty string), indicating that the job level is not applicable to the node's system content.

[0150] S15014, Store Summary: Store each job level and its corresponding summary text as key-value pairs in the current node object.

[0151] S1502 involves vectorizing all the job-level differentiated summaries generated in S1501 and constructing a job-level clustering tree as the third tree-structured index. Specifically:

[0152] Vectorization: Each job title summary text is converted into an embedding vector (e.g., 1024-dimensional) using a specified text embedding model (e.g., Qwen3-Embedding-8B). If the summary is null, a zero vector or a specific identifier vector is stored.

[0153] Construct a job-level clustering tree: Preserve the complete tree topology of the first tree index structure (i.e., the parent-child hierarchical structure between nodes). Replace the original general summary in each node with a summary dictionary with job level as the key. The key of this dictionary is the job level name, and the value is the corresponding vectorized summary. This ultimately forms a tree structure isomorphic to the first tree index structure, but with node content transformed into job-level indexed summaries, which is the third tree index structure (job-level clustering tree).

[0154] In this embodiment of the invention, the final third tree-shaped index structure is persistently stored. The stored information includes: the tree structure relationships, the summary text of all job titles for each node, and their corresponding embedding vectors. This structure is independent of other tree-shaped index structures and is specifically designed to support efficient retrieval from a job title perspective.

[0155] In one specific embodiment of the present invention, a multimodal hybrid storage architecture is adopted to realize the persistence and management of the multi-level tree structure knowledge base. This architecture is composed of a relational database, a vector database, and an object storage system, as detailed below:

[0156] MySQL relational database: Used to store all structured metadata and relationships. This mainly includes: unique identifiers for knowledge fragments, original document metadata (such as document name, version, and department), the hierarchical relationship between knowledge fragments and source documents, node information for each tree-like index structure (semantic tree, organizational tree, hierarchical tree) (including node ID, level, parent node ID, list of child node IDs, etc.), and the association mapping between nodes and knowledge fragments. This component is responsible for maintaining data integrity and consistency and supports complex relational queries.

[0157] ElasticSearch Vector Database: Used to store and index all vectorized data, providing high-performance Approximate Nearest Neighbor (ANN) retrieval capabilities. Specifically, it stores: the original embedding vectors of all knowledge fragments, the summary vectors of each node in the semantic clustering tree, the department summary vectors of each node in the organizational clustering tree, and the job-level differentiated summary vectors of each node in the job-level clustering tree. Specifically, to ensure retrieval efficiency and isolation, the vector data of these three tree-structured indexes are stored in three independent indexes within ElasticSearch.

[0158] MINIO Distributed Object Storage System: Used to store unstructured raw data files, serving as a persistent backup and source data management component for underlying data. Specific stored content includes: original documents in various formats of company rules and regulations (such as PDF, DOCX, etc.), and multimedia resources such as images and charts extracted from these documents. This system provides highly available and scalable raw data storage for the entire knowledge base.

[0159] Through the S100 series steps described above, combined with this hybrid storage architecture, the entire enterprise rules and regulations knowledge base was constructed. This achieved end-to-end, integrated storage and management from original documents and structured metadata to multi-dimensional vectorized indexes, providing a solid data foundation for subsequent efficient retrieval and question answering.

[0160] S160, Establish the cross-reference mapping table to record the positional relationship of each knowledge fragment in the three tree-shaped index structures.

[0161] In this embodiment of the invention, the cross-reference mapping table uses the globally unique identifier of a knowledge fragment as the key and records the position information of the corresponding node identifier in the first, second, and third tree index structures as the value, to achieve rapid association and location of knowledge fragments across different dimensional indexes. During the construction of the first, second, and third tree index structures, a cross-reference mapping table is simultaneously established and maintained. For each knowledge fragment, when it is associated with a node in any tree index structure, the mapping relationship between the fragment identifier and the corresponding node identifier is recorded in the mapping table, thereby forming a global cross-reference network.

[0162] S200 receives user queries and user identity information.

[0163] Specifically, the user query refers to a natural language question entered by the user, typically involving inquiries about company rules and regulations. The user identity information is contextual information related to the current user's identity, including at least the user's department identifier and job level permission value. The department identifier is a code or name used to uniquely identify an organizational department within the company; for example, the department code "HR001" or the full name "Human Resources Department". The job level permission value is a numerical or grade code used to characterize the user's position and data access permissions within the organizational hierarchy; for example, the value "5" or code "M3" representing "Middle Manager". In one embodiment, user identity information can be automatically obtained from the user's system login session, user profile database, or declaration information provided by a Single Sign-On (SSO) system. This step receives the user's input query string through the front-end interface of the enterprise Q&A system and synchronously obtains the contextual information associated with the user session through the back-end service interface, providing a basis for subsequent personalized query rewriting and retrieval.

[0164] S300, based on the user query and user identity information, generate multiple sub-queries corresponding to different search intentions.

[0165] Furthermore, the S300 specifically includes:

[0166] S310 combines user queries, department identifiers and job level permissions in user identity information, and historical dialogue records into structured prompts.

[0167] The prompt words explicitly instruct the large language model to generate specific types of query statements, as shown in the following example:

[0168] "You are a professional query analysis and rewriting expert. Please generate three queries with different focuses based on the historical conversations, user identity information, and user questions below."

[0169] ### Task Description

[0170] Please generate three query statements: a basic query statement, an organizational structure query statement, and a job level query statement.

[0171] - Basic query statement: should be generated solely based on the user's question, without considering other contexts.

[0172] - Organizational structure query statement: The query should be generated from the perspective of organizational structure, taking into account the user's question and their department ({department}).

[0173] - Job level query statement: The query should be generated from the perspective of job level permissions, taking into account the user's question and their job level ({level}).

[0174] - The output must be a valid JSON object and contain only this object; no other text is allowed.

[0175] - The keys of the JSON object must be: base_query, department_query, level_query.

[0176] ### Historical Dialogue Records

[0177] {history}

[0178] ### User Identity Information

[0179] Department: {department}

[0180] Job level: {level}

[0181] ### User Issues

[0182] {query}

[0183] Please generate the rewritten query statement and return it in JSON format:

[0184] S320, the prompt words are processed using a large language model, and the search intent is decomposed based on the retrieval intent in terms of semantic dimension, organizational structure dimension, and job level dimension, generating multiple query statements corresponding to different retrieval intents as subqueries.

[0185] Specifically, the prompt words in S310 are input into the large language model (such as Qwen2-7B-Instruct). The model generates and returns a JSON object according to the instructions. This object contains multiple query statements corresponding to different search intentions.

[0186] The query statement includes at least one basic query statement, one organizational structure-related query statement, and one job level-related query statement. The basic query statement, the organizational structure-related query statement, and the job level-related query statement are used to perform retrieval in the first tree index structure, the second tree index structure, and the third tree index structure, respectively.

[0187] S330, parse and output the multiple subqueries, including:

[0188] The three query statements are parsed from the output of the large language model and returned in a predetermined JSON format. The JSON key names correspond to the semantic subquery, the organization subquery, and the job title subquery, respectively.

[0189] Capture the return results of the large language model, parse its JSON output, and extract the string values ​​corresponding to the three keys base_query, department_query, and level_query as subqueries for subsequent retrieval steps.

[0190] S400, each of the subqueries is retrieved in the corresponding tree index structure to obtain a set of relevant knowledge fragments.

[0191] Furthermore, the S400 specifically includes:

[0192] S410, convert each subquery into a vector representation.

[0193] The model and its configuration used in the vectorization process are consistent with those used when building the knowledge base. Specifically, the Qwen3-Embedding-8B model is used to convert each query statement into a 1024-dimensional embedding vector to ensure the consistency of the vector space.

[0194] S420: For each subquery, in its corresponding tree index structure, two strategies are used to perform the retrieval: tree traversal retrieval and tree folding retrieval.

[0195] Tree traversal is a retrieval method based on a tree structure, which includes: starting from the root node of the tree structure, calculating the similarity between the query vector and the summary vectors of each node in the current level. The top K nodes with the highest similarity (K=3 in one embodiment of this invention) are selected to proceed to the next level, and this process is repeated at each level until the leaf nodes are reached. Finally, multiple low-level knowledge fragments that are semantically closest to the query statement are retrieved.

[0196] Specifically, after obtaining the vectorized representation of the query statement via S410, the top level of the tree structure is first accessed. The process traverses the top level, comparing the representation with all tree nodes and performing similarity comparisons with the embedded vectors summarized within each node. The tree node with the highest relevance to the query statement is found and retained. The process then moves to the next level of the tree structure, following the same logic, until the bottom level is reached. At this point, the retrieval yields the knowledge fragment most semantically similar to the query statement.

[0197] In this invention, cosine similarity is used as the metric for vector similarity. Cosine similarity effectively captures the directional consistency of vectors, ignoring deviations caused by the absolute length of the vectors. Specifically, cosine similarity normalizes the similarity to the interval [-1, 1] by calculating the cosine of the angle between two vectors. The closer the value is to 1, the more consistent the vector directions and the closer their semantics. Cosine similarity is decoupled from the Euclidean norm of vectors, making it computationally efficient and suitable for handling high-dimensional sparse data tasks such as text similarity, recommendation systems, and embedding comparisons.

[0198] Tree folding is a retrieval method based on tree folding structures. This method does not distinguish the hierarchical relationships between nodes in a tree-structured knowledge base. Instead, it performs a one-time cosine similarity calculation between the query vector and the summary vectors of all nodes in the target tree index structure. The knowledge fragments associated with the top M nodes (M=10 in one embodiment of this invention) with the highest global similarity are directly selected as the retrieval results.

[0199] S430, integrate all knowledge fragments obtained through different retrieval paths and different tree index structures, and perform collaborative verification of retrieval results across dimensions based on the cross-reference mapping table to form the final set of knowledge fragments.

[0200] In steps S410 to S420, retrieval of three different tree-structured indexes was completed, resulting in multiple preliminary search result sets. The core task in this step is to utilize the cross-reference mapping table to achieve collaborative verification across dimensions, rather than simply merging results. Collaborative verification aims to identify and prioritize knowledge fragments that are proven to be highly relevant across multiple dimensions, thereby improving the quality and consistency of the final set. In an illustrative embodiment, S430 may include the following steps, which deeply integrate the collaborative verification mechanism:

[0201] S4301, Result association and deduplication based on cross-reference mapping table:

[0202] All knowledge fragments from different tree-structured indexes and retrieval paths are merged. Based on the cross-reference mapping table, multiple retrieval results from different dimensions that point to the same knowledge fragment can be quickly identified and associated. Subsequently, deduplication is performed based on the globally unique identifier of the knowledge fragment, ensuring that each unique knowledge fragment is represented only once in subsequent processing.

[0203] S4302, Collaborative Verification and Scoring Supported by Multi-Dimensional Evidence:

[0204] This step is the core of collaborative verification. For each deduplicated knowledge fragment, the cross-reference mapping table is queried to obtain the path and context information retrieved in different dimension indices, and collaborative verification scoring is performed based on this multi-dimensional evidence.

[0205] The collaborative verification score is calculated by weighting the consistency verification score and the cross-verification strength score. The consistency verification score assesses whether the inherent multidimensional attributes of a knowledge fragment are consistent and whether it matches the user's context. For example, a fragment about "travel reimbursement" should semantically belong to the finance topic, organizationally be related to the finance and sales departments, and have a version applicable to "employees" in terms of job level. Fragments with high consistency receive bonus points. Its calculation can be further decomposed into a weighted average of three sub-item scores:

[0206] Topic Consistency: Calculate the cosine similarity between the semantic vector of the user query and the summary vector of the topic cluster node to which the knowledge fragment belongs in the first tree index (semantic tree).

[0207] Departmental Consistency: Check if the user's department appears in the list of associated department nodes in the second tree index (organization tree) for this knowledge fragment. Return 1 if yes, 0 otherwise.

[0208] Job title consistency: Check whether the user's job title appears in the list of applicable job title nodes in the third tree index (job title tree) for this knowledge fragment. Return 1 if yes, 0 otherwise.

[0209] The cross-verification strength score is based on the number of paths a knowledge fragment is retrieved from different dimension indexes, and is used to assess the degree to which the fragment is supported by retrieval paths from different dimensions. For example, if a fragment is found simultaneously through three paths: the semantic tree's "travel" topic search, the organizational tree's "sales department" path search, and the job level tree's "employee" search, its cross-verification strength is high, indicating that it highly matches the user's multidimensional intent in the query, and its collaborative verification score will be significantly improved. Define M as the number of different dimension tree indexes hit by the knowledge fragment (i.e., hitting 1, 2, or 3 dimensions), then the cross-verification strength score = M / 3. For example, if a fragment is retrieved simultaneously by the semantic tree and the organizational tree (M=2), its score is 2 / 3 ≈ 0.67.

[0210] S4303, Database-based minimum filtering and global re-sorting:

[0211] Integrate and sort the knowledge fragments that have undergone collaborative verification and scoring:

[0212] Database partitioning and screening: To ensure representativeness of each dimension, a segment with the highest similarity is retained from the original search results (before collaborative verification) for each tree-like index structure.

[0213] Weighted ranking: The minimum acceptable fragment is merged with the remaining fragments, and a weighted global re-ranking is performed based on the collaborative verification score as the primary ranking criterion, combined with the semantic similarity to the original user query. Fragments with higher collaborative verification scores rank higher.

[0214] S4304, Final set generation:

[0215] From the reordered list, the top N knowledge fragments are selected and combined with the fragments selected from the minimum threshold of the database to form the final set of knowledge fragments. Through this mechanism, the final set not only includes the most representative content across all dimensions but also prioritizes high-quality "consensus" fragments that have undergone multi-dimensional cross-validation, providing optimal contextual information for generating accurate and consistent final answers. In a preferred embodiment of the invention, N=7 is set to achieve an optimal balance between the limitation on the length of generated context and the richness of answers.

[0216] S500 evaluates the set of knowledge fragments based on a self-reflection mechanism and generates a final answer.

[0217] In step S400, knowledge retrieval based on the user question is completed, yielding N knowledge fragments most relevant to the user question (N=10 in one embodiment). This step introduces the Self-Reflective Retrieval Enhancement Generation (Self-RAG) framework to critically evaluate the retrieval results and generate high-quality answers, specifically including:

[0218] S510 uses predefined reflection tags to evaluate the retrieved knowledge fragments. The evaluation process comprehensively utilizes the correlation information of the knowledge fragments in multiple dimension indexes to obtain the corresponding evaluation results.

[0219] In one embodiment of the present invention, the reflection tag may be a three-dimensional tag, including relevance, support and usability tags.

[0220] The relevance tag determines whether the retrieved knowledge fragments are relevant to the user's query, with a "yes" or "no" result. The support tag determines whether the final answer is adequately supported by the knowledge fragments, also with a "yes" or "no" result. The usability tag assesses the overall usefulness of the knowledge fragments in generating the final answer, scoring it on an integer scale of 1-5. These tags enable the model to filter the validity of search results and assess their quality, providing control signals for subsequent generation.

[0221] In another embodiment of the present invention, the reflection marker may be a five-dimensional marker, including: relevance, support and availability markers, timeliness markers and compliance markers. The timeliness marker is used to verify whether the corresponding system of the fragment is the latest version, and the result is "yes / no". The compliance marker is used to determine whether the fragment complies with the current compliance requirements of the enterprise, such as GDPR, industry regulatory provisions, etc., and the result is "yes / no".

[0222] The evaluation process comprehensively utilizes the correlation information of knowledge fragments across multiple dimensions of the index, including:

[0223] Based on the node affiliation information of knowledge fragments in the multiple tree index structures, the consistency with user queries and user identity context is verified to enhance the accuracy of the evaluation.

[0224] And / or,

[0225] The evaluation signals obtained from the same knowledge fragment in different dimensional index contexts are fused to generate a comprehensive evaluation result.

[0226] Furthermore, the evaluation process comprehensively utilizes the correlation information of knowledge fragments across multiple dimensions, specifically manifested in the following two core mechanisms:

[0227] a) Consistency verification mechanism based on multi-dimensional node affiliation

[0228] This mechanism aims to address the problem in traditional evaluation methods that focus solely on the semantic relevance of knowledge fragments to queries, while neglecting their matching degree with the user's actual identity context. The specific implementation process is as follows:

[0229] Information Acquisition: For each retrieved knowledge fragment, the system immediately obtains its node affiliation information in different tree index structures by querying the cross-reference mapping table, including:

[0230] The relevant topic cluster node in the first tree index structure (semantic clustering tree) (e.g., "financial reimbursement - domestic business travel");

[0231] The associated department nodes (e.g., "Finance Department", "Sales Department") in the second tree-structured index (organizational tree);

[0232] Applicable job level nodes (e.g., "Manager" and "Employee") in the third tree-structured index (job level clustering tree).

[0233] Context consistency check: The system checks the consistency between the node attribution information mentioned above and the current user query and user identity information (department, job level). For example:

[0234] Departmental consistency: If a user is from the "Sales Department" but the knowledge fragment is only associated with the "R&D Department" in the organizational structure tree, the system will mark that the fragment is inconsistent in the departmental context, even if it is semantically related, and will lower its usability assessment score.

[0235] Job level consistency: If a user's job level is "employee" and a knowledge fragment is marked in the job level tree as only applicable to "senior executives", the system will determine that the fragment does not match the permission context, thereby significantly reducing its availability and avoiding providing users with information that they do not have permission to access or that is not applicable.

[0236] Semantic focus: If a knowledge fragment belongs to the topic of "safe production" in the semantic tree, but the user query is specifically about "financial system", even if it is retrieved through fuzzy keyword matching, the system will question its core relevance based on the inconsistency of the topic node affiliation.

[0237] Enhanced accuracy: Through this multidimensional consistency verification, the evaluation mechanism can identify knowledge fragments that are "semantically relevant but context-inconsistent," thereby effectively avoiding generating irrelevant or unauthorized answers and greatly enhancing the business accuracy and security of the evaluation results.

[0238] b) Weighted fusion mechanism based on multi-dimensional evaluation signals

[0239] This mechanism aims to address the limitations of a single-dimensional assessment perspective by integrating complementary evidence from different dimensions to form a more comprehensive and robust integrated assessment. The specific implementation process is as follows:

[0240] Dimensional evaluation: The system first performs a preliminary evaluation of the same knowledge fragment in the context of each dimension index.

[0241] In the semantic dimension, the main evaluation is the semantic similarity (cosine similarity) between the fragment and the user query.

[0242] At the organizational level, the main assessment focuses on the matching degree between the relevant departments of the fragment and the user's department (perfect match, peer departments, unrelated departments).

[0243] In terms of job level, the main evaluation focuses on the fit between the job level applicable to the segment and the user's job level (fully fit, needs adjustment, not applicable).

[0244] Signal mapping and normalization: The evaluation results from the different dimensions mentioned above (such as similarity scores and match levels) are uniformly mapped to the core dimensions of the reflective label (relevance, support, and usability), and normalized into comparable numerical scores. For example, a high degree of fit in job level will translate into a bonus in the "usability" dimension.

[0245] Weighted fusion: The system assigns appropriate weights to evaluation signals of different dimensions according to specific business scenarios, performs weighted calculations, and generates the final comprehensive evaluation result.

[0246] Weighting example: For questions like "querying specific department's policies", the matching signal of the organization dimension will receive higher weight; for questions like "querying general business processes", the relevance signal of the semantic dimension will have the highest weight.

[0247] Integration Benefits: When a knowledge fragment receives high ratings across three dimensions—semantics, organization, and job level—its final score will be significantly higher than a fragment that excels only in a single dimension. This integration mechanism ensures that the knowledge fragments ultimately adopted are high-quality fragments that withstand scrutiny in multiple aspects, thus laying a solid foundation for generating accurate, relevant, secure, and identity-appropriate final answers.

[0248] Through the synergistic operation of the two mechanisms a) and b) mentioned above, the self-reflective evaluation process of this invention achieves a leap from "single-point evaluation" to "three-dimensional evaluation", fully exploring and utilizing the multi-dimensional related information contained in the multi-level tree structure knowledge base.

[0249] S520, Based on the evaluation results and knowledge fragments, generate the final answer.

[0250] In scenarios where reflection tags are three-dimensional, the evaluated knowledge fragments, reflection tag results, and user queries are combined into prompts, and a large language model (such as the Qwen3-32B model in one embodiment) is invoked to generate the final answer. During the generation process, the model weights and selects the search content based on the reflection tags to ensure that the output answer is not only accurate but also highly relevant to the user query, while avoiding the generation of content that is not supported by evidence or is irrelevant to the question.

[0251] The prompt word template is designed as follows (example shown below):

[0252] "You are a friendly and professional Q&A assistant, specializing in corporate policy Q&A. Please provide safe, accurate, and helpful answers to user questions based on the following knowledge base content."

[0253] ### Answer Requirements

[0254] 1. Please answer strictly according to the knowledge base content, and cite specific clauses or content from relevant knowledge fragments as much as possible.

[0255] 2. Please determine whether the knowledge base content is relevant to the user's question. If the knowledge base content is completely irrelevant to the user's question, please reply directly: "Sorry, the answer you are looking for was not found in the knowledge base."

[0256] 3. Answers should be concise, clear, and conform to the company's regulations.

[0257] ### Knowledge Base Content

[0258] {retrieved_knowledge}

[0259] ### User Issues

[0260] {user_query}.

[0261] In a scenario where the reflection label is a five-dimensional label, S520 may include:

[0262] (1) If the timeliness of a knowledge fragment is marked as "no" or the compliance is marked as "no", a supplementary search will be automatically triggered to obtain the latest version or compliance-adapted version of the relevant institutional clauses.

[0263] In this embodiment of the invention, the supplementary retrieval is not a full re-retrieval, but rather based on "precise positioning using the original fragment identifier," and the process is as follows:

[0264] Step 1: Extract the core identifiers of the original fragment

[0265] Extract two key pieces of information from the knowledge fragments marked "Timeliness - No" or "Compliance - No":

[0266] A unique ID for each clause (e.g., "Civil Code Number: CW-2023-001", assigned to each fragment during knowledge base construction).

[0267] Thematic keywords (such as "travel allowance - domestic business trip - department manager level") are extracted from the fragment content by the large language model.

[0268] Step 2: Targeted Search of the Database

[0269] Based on the tag type, retrieve the corresponding data source database:

[0270] If it is "Timeliness - No": Retrieve the "Institution Version Repository" maintained by S600 (Dynamic Synchronization Module). This repository is synchronized with the latest policies in the enterprise OA system in real time and indexed by "Clause ID + Version Number" (e.g., "CW-2023-001-V2" represents the second version of the clause with the number CW-2023-001).

[0271] If the answer is "Compliance - No": Search "Enterprise Compliance Clause Library", which contains external regulations (such as national laws and industry standards) and internal compliance manuals, and is indexed by "Compliance Area + Clause Topic" (such as "Data Compliance - Customer Information Retention Period").

[0272] Step 3: Matching and Replacing Results

[0273] For "time-sensitive supplements": find a fragment with the same "clause ID and latest version number" as the original fragment and directly replace the original outdated fragment; if no fragment is found (e.g., the original clause has been completely repealed and there is no alternative clause), mark it as "no valid alternative version" and filter it in S522 later;

[0274] For "compliance supplements": Find fragments that "match the subject keywords and meet current compliance requirements" (which may come from external regulations or internal revised systems), and add them as "compliance-adapted versions" to the knowledge fragment set, placing them alongside the original fragments (for reference and comparison when generating answers later).

[0275] Supplementary retrieval will be performed in a maximum of 2 iterations:

[0276] First search: Searching for segments initially marked "No";

[0277] Second search: If the new fragments obtained in the first search are still marked as "Timeliness - No" or "Compliance - No" (e.g., the retrieved version is not the latest), then search again;

[0278] If a valid fragment cannot be obtained after two iterations, stop the search and mark the fragment as "unable to supplement a valid version".

[0279] (2) Filter the segments with "relevance - no" and "support - no" in the five-dimensional tags, and retain the segments with usability scores greater than or equal to the preset score, such as 3 points, to form an optimized set of knowledge segments.

[0280] The core dimensions for usability scoring include:

[0281] Job level suitability: Does the snippet match the user's job level (e.g., if the user is a "junior employee", are the snippets targeted at "junior employees" rather than "senior executives"?)

[0282] Scenario practicality: Can the clip directly solve the user's problem (e.g., when a user asks about "travel allowance standards", does the clip clearly provide specific figures such as "domestic business trip allowance of 150 yuan per day")?

[0283] Content completeness: Does the fragment contain the core elements of the terms (e.g., "reimbursement process" should include key steps such as "submitting materials, departmental review, financial review, and payment"? If the fragment only mentions "submitting materials", the completeness is insufficient and the usability score is low).

[0284] The quantitative criteria for usability scoring can be:

[0285] 1 point: Completely unsuitable for the user's job level, contains no useful information, and is incomplete;

[0286] 2 points: Partially matches the user's job level and contains a small amount of relevant information, but cannot directly solve the problem;

[0287] 3 points: Basically suitable for user job level, contains core information that can solve user problems, and the content is complete;

[0288] 4 points: Highly adaptable to user job levels, includes detailed problem solutions, and comes with operation examples (such as the "Reimbursement Process" segment with "Employee A's Reimbursement Example in March 2025: Reimbursement Arrives 3 Business Days After Submission of Materials").

[0289] 5 stars: Fully adapted to user job levels, including problem solutions, tips on common pitfalls (such as "When submitting expense reports, please note: the invoice header must be the company's full name, otherwise it will not be approved") and suggestions for cross-departmental collaboration (such as "If the finance department does not review in a timely manner, you can contact the department's administrative specialist for coordination").

[0290] If multiple knowledge fragments have the same score (e.g., all 3 points), they will be sorted according to the following priority (prioritizing the retention of preceding fragments to avoid set redundancy):

[0291] The "latest version / compliant version" obtained from the supplementary search takes precedence over the original fragment;

[0292] Fragments that simultaneously hit three tree-structured indexes ("semantic + organizational + job level" - with high cross-index correlation) are preferred over fragments that only hit a single index;

[0293] Segments with a length of 200-500 words (balancing content completeness and conciseness) are preferred over those that are too short (<200 words, insufficient information) or too long (>500 words, too much redundant information).

[0294] The optimized set will be accompanied by "filter tags" (such as "supplementary timeliness fragments" and "high availability 3 points"). When generating the final answer, the large language model will prioritize generating content based on the "supplementary fragments" and "4-5 point fragments", and will explicitly cite specific clauses in the fragments (such as "According to Article 3.2 of the Financial Reimbursement System (2025 Edition): the domestic business trip allowance standard for grassroots employees is RMB 150 per day"), further improving the credibility and explainability of the answer.

[0295] The purpose of steps (1) and (2) above is to:

[0296] 1) Resolve the problem of "outdated system leading to incorrect answers".

[0297] If the traditional RAG retrieves the "Travel Allowance Standard" that was repealed in 2023, it will directly generate an incorrect answer based on that fragment; however, this solution supplements the retrieval with the latest version from 2025 through S521, and then retains it through S522, ensuring that the answer is based on the currently effective system.

[0298] 2) Avoid the problem of "lack of compliance risk warnings"

[0299] If a user asks about the "customer information retention period" and the original fragment is "5 years" (non-compliant), S521 will add the compliant fragment "The Personal Information Protection Law requires that the information be retained for no more than 3 years". S522 will retain this compliant fragment, and the final answer will also indicate "The original policy terms have been updated, and the current compliance requirement is: the retention period shall not exceed 3 years", thus avoiding corporate compliance risks.

[0300] 3) Improve the "match between answers and user job level"

[0301] If the user is a "grassroots employee", the "Executive-Exclusive Reimbursement Green Channel" segment (usability score 2, job level not suitable) will be filtered by S522; only the "Grassroots Employee Reimbursement Process" (usability score 4) will be retained to ensure that the answer does not contain content that the user is not entitled to know or that is not applicable.

[0302] (3) Combine the optimized knowledge fragments with the user query to generate the final answer.

[0303] Furthermore, the method also includes the following steps:

[0304] S600, the final answer is segmented and vectorized, and a visual knowledge tracing is constructed by calculating the similarity with knowledge fragments.

[0305] To verify the interpretability and credibility of the answers, the source of the generated final answers is traced, specifically including:

[0306] S601, Sentence Segmentation and Vectorization: The final answer is segmented into sentences, and each sentence is converted into a sentence vector using a vectorization model (Qwen3-Embedding-8B);

[0307] S602, Similarity Calculation: Calculate the cosine similarity between each sentence vector and each knowledge segment vector in the knowledge base;

[0308] S603, Source Association: Associate sentences with similarity exceeding a preset threshold (e.g., the threshold is set to 0.6 in one embodiment) with their corresponding knowledge fragments to generate a tree-shaped index path visualization diagram, showing the retrieval path corresponding to each part of the answer, such as "semantic tree → financial clustering → reimbursement sub-node → clause ID123", thereby clearly identifying the source basis of each part of the answer and hiding the clause traceability details that the user does not have permission to view based on the user's job level.

[0309] Through the aforementioned self-reflection mechanism, this invention significantly improves the factual accuracy, contextual relevance, and verifiability of the generated answers.

[0310] In summary, the retrieval enhancement generation method based on hierarchical trees and self-reflection provided in this invention focuses on retrieval enhancement generation in the context of enterprise policy question-and-answer scenarios. It constructs a multi-level tree-structured knowledge base using computer technologies (such as vectorization of large language models, Gaussian mixture model clustering, and vector database retrieval), generates sub-queries by combining user identity information (department, job level), and then evaluates knowledge fragments and generates answers through a self-reflection mechanism. It has at least the following technical effects:

[0311] (1) Improve search efficiency and accuracy

[0312] By introducing a RAPTOR hierarchical tree structure, this invention hierarchically summarizes and generalizes information in the knowledge base, enabling the capture of both high-level and low-level details in the text. This tree structure makes the retrieval process more efficient, quickly finding the documents or paragraphs most relevant to the user's query.

[0313] Compared to traditional RAG technology, the present invention significantly improves retrieval efficiency and accuracy. Traditional RAG technology may suffer from low relevance and consistency of search results, while the present invention, through hierarchical retrieval using a tree structure, can better understand the context of the user's query, thereby improving the relevance and accuracy of the search results.

[0314] (2) Enhance the factual accuracy and relevance of generated content.

[0315] By introducing the self-reflection mechanism of the Self-RAG framework, this invention uses special reflective tags during the generation process to evaluate the relevance, support, and usability of the retrieved text. These reflective tags help the model determine the validity of the retrieved content and provide feedback during the generation process, thereby optimizing the generated results.

[0316] Compared to traditional RAG technology, this invention generates more accurate and relevant content. Traditional RAG technology may suffer from the problem of mismatch between generated and retrieved content (i.e., the illusion problem), while this invention, through a self-reflection mechanism, can significantly improve the factual accuracy and relevance of the generated content.

[0317] (3) Improve the coherence and logic of generated content.

[0318] This invention combines the RAPTOR hierarchical tree structure and the Self-RAG framework to generate content that is not only accurate but also possesses higher contextual relevance and coherence. The hierarchical retrieval of the tree structure and the introduction of the self-reflection mechanism enable the generated content to better reflect the user's query intent and context.

[0319] Compared to traditional RAG technology, the content generated by this invention is more coherent and logically sound. Traditional RAG technology may suffer from poor contextual understanding when generating content, while this invention, through hierarchical retrieval of a tree structure and a self-reflection mechanism, can generate more coherent and logically sound content.

[0320] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0321] This invention also provides a computer-readable storage medium storing computer-executable instructions for performing the methods described in this invention.

[0322] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0323] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.< / beg> < / beg> < / beg>

Claims

1. A method for generating search enhancement based on hierarchical tree and self-reflection, characterized in that, The method comprises the following steps: S100, constructing a multi-level tree structure knowledge base, the knowledge base comprising a plurality of tree index structures formed by organizing knowledge fragments based on semantic dimensions, organizational structure dimensions and job level dimensions, and the plurality of tree index structures establishing the association relationship between the dimensions through a unified cross-reference mapping table; S200, receiving a user query and user identity information, the user identity information comprising a department identifier and a job level permission value; S300, based on the user query and the user identity information, performing query intent decomposition to generate a plurality of sub-queries corresponding to the semantic dimensions, the organizational structure dimensions and the job level dimensions respectively; S400, using a parallel retrieval strategy, sending each sub-query to the corresponding tree index structure for retrieval, and based on the cross-reference mapping table, implementing collaborative verification of the retrieval results between the dimensions during the retrieval process to obtain a relevant knowledge fragment set; S500, evaluating the knowledge fragment set based on a self-reflection mechanism and generating a final answer; S100 specifically comprises: S110, preprocessing the original text to obtain preprocessed text; S120, parsing, knowledge partitioning and vectorization processing the preprocessed text to obtain a plurality of knowledge fragments and their corresponding embedding vectors; S130, based on the embedding vectors, constructing a first tree index structure, wherein the nodes of the first tree index structure are formed based on text semantic clustering; S140, based on a predefined organizational structure, constructing a second tree index structure, wherein the nodes of the second tree index structure correspond to different organizational departments, and the knowledge fragments are associated with the corresponding department nodes; S150, based on a predefined job level structure, constructing a third tree index structure, wherein the node content of the third tree index structure is organized differently according to job levels; S160, establishing the cross-reference mapping table to record the positional association relationship of each knowledge fragment in the three tree index structures; S150 specifically comprises: S151, traversing the first tree index structure and generating job level differentiated summaries; S152, vectorizing all generated job level differentiated summaries and constructing a job level clustering tree as the third tree index structure.

2. The method of claim 1, wherein, S130 specifically comprises: S131, using a Gaussian mixture model to perform soft clustering on the embedding vectors of the knowledge fragments to form bottom-level clusters; S132, generating summary content for each bottom-level cluster and vectorizing the summary content to form upper-level nodes; S133, recursively performing the clustering and summary generation steps until a preset termination condition is met, thereby forming a multi-level semantic clustering tree as the first tree index structure.

3. The method of claim 1, wherein, S140 specifically comprises: S141, constructing a tree framework with organizational department nodes according to the actual organizational level and department division relationship of the target enterprise; S142, using a large language model to automatically determine and establish the association relationship between each knowledge fragment and the department nodes in the tree framework; S143, generating department node summaries from bottom to top and forming a second tree index structure.

4. The method of claim 1, wherein, S300 specifically comprises: S310, combine the user query, the department identifier and the job level permission value in the user identity information, and the historical dialogue record into a structured prompt word; S320, process the prompt word by using a large language model, perform query intent decomposition based on the search intent of the semantic dimension, the organizational structure dimension, and the job level dimension, and generate a plurality of query sentences respectively corresponding to different search intents as the sub-queries; S330, parse and output the plurality of sub-queries.

5. The method of claim 1, wherein, S400 specifically includes: S410, convert each sub-query into a vector representation; S420, for each sub-query, in its corresponding tree index structure, respectively adopt two strategies of tree traversal retrieval and tree folding retrieval to perform retrieval; S430, integrate all knowledge fragments obtained through different retrieval paths and different tree index structures, and realize collaborative verification of retrieval results between dimensions based on the cross-reference mapping table, to form a final knowledge fragment set.

6. The method of claim 1, wherein, The self-reflection mechanism in S500 includes: S510, evaluate the retrieved knowledge fragments using predefined reflection markers, and the evaluation process comprehensively utilizes the associated information of the knowledge fragments in multiple dimension indexes to obtain corresponding evaluation results; S520, generate a final answer based on the evaluation results and the knowledge fragments; The method further includes: S600, sentence vectorization is performed on the final answer, and a visual knowledge provenance is constructed by calculating the similarity with the knowledge fragments.

7. An electronic device, comprising: It includes a processor and a memory; The processor is used to execute the steps of the method according to any one of claims 1 to 6 by calling the programs or instructions stored in the memory.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium is used to store programs or instructions, which make the computer execute the steps of the method according to any one of claims 1 to 6.

Images