Enterprise RAG Intelligent Question Answering System Based on Vector Database Reordering

Abstract

This invention relates to the field of intelligent retrieval technology, specifically to an enterprise RAG intelligent question-answering system based on vector database re-ranking. Specifically, it includes: a document retrieval module, which retrieves candidate document blocks containing business metadata from a vector database; a comprehensive scoring module, which calculates semantic relevance scores using a post-interaction architecture based on bidirectional token importance weighting, performs path semantic matching and context-aware rule evaluation based on a preset metadata ontology graph to obtain business attribute scores, and analyzes evidence subgraphs to obtain factual path scores; it integrates semantic relevance scores, business attribute scores, and factual path scores to generate a comprehensive relevance score; and a ranking output module, which optimizes and re-ranks based on a preset penalty mechanism and the comprehensive relevance score to generate an optimized context set, and calls a generation model based on the optimized context set to output the answer. This invention can balance semantic accuracy, business compliance, and factual reliability, generating more trustworthy, high-quality enterprise-level answers.

Description

Technical Field

[0001] This invention relates to the field of intelligent retrieval technology, specifically to an enterprise RAG intelligent question-answering system based on vector database reordering. Background Technology

[0002] In the process of enterprise digital transformation, how to accurately and efficiently acquire information from massive and heterogeneous internal knowledge bases is a key technological challenge for improving organizational decision-making capabilities and operational efficiency. Traditional knowledge retrieval methods have limitations in understanding complex natural language queries and cannot meet the needs of in-depth and accurate question answering.

[0003] In existing technologies, Retrieval Augmentation (RAG) architecture has become the mainstream solution to this problem. This architecture performs preliminary semantic retrieval using a vector database to recall a set of candidate documents, and then optimizes this set through a re-ranking process to improve the quality of the final generated answer. However, in enterprise implementation, the core retrieval and ranking process of existing solutions has room for optimization. First, the preliminary vector retrieval results may contain semantic biases and not perfectly match the user's true query intent. Second, the computationally complex high-precision re-ranking models used to improve ranking accuracy are often accompanied by significant response delays, making it challenging to meet real-time interaction requirements while ensuring effectiveness. In addition, existing ranking mechanisms do not adequately consider the business attributes of documents. How to effectively integrate metadata such as document timeliness and source authority with content relevance, and how to effectively integrate heterogeneous information from multiple source databases, are also ongoing technical directions in this field.

[0004] In summary, existing technologies suffer from the inability to integrate multi-dimensional information, resulting in limited quality of the final answer. To address this, we propose an enterprise RAG intelligent question-answering system based on vector database re-sorting. Summary of the Invention

[0005] The purpose of this invention is to provide an enterprise RAG intelligent question-answering system based on vector database reordering for intelligent information retrieval. This invention aims to generate more trustworthy, high-quality enterprise-level answers by comprehensively considering semantic accuracy, business compliance, and factual reliability through a multi-dimensional intelligent reordering engine.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An enterprise RAG intelligent question answering system based on vector database reordering includes:

[0008] The document retrieval module receives user queries and retrieves candidate document blocks containing business metadata from a preset vector database based on the user queries.

[0009] The comprehensive scoring module employs a post-interaction architecture based on bidirectional token importance weighting to perform similarity aggregation operations on pre-encoded document term vectors and query term vectors to calculate semantic relevance scores. Based on a pre-defined metadata ontology graph, it performs path semantic matching and context-aware rule evaluation on business metadata and business constraints generated based on user queries to obtain business attribute scores. It extracts key entities from user queries and candidate document blocks and performs relation tree expansion to generate evidence subgraphs. By analyzing the internal topological structure and semantic coherence of the evidence subgraphs, it obtains factual path scores. Finally, it dynamically integrates semantic relevance scores, business attribute scores, and factual path scores to generate a comprehensive relevance score.

[0010] The sorting output module optimizes and reorders candidate document blocks based on a preset penalty mechanism and a comprehensive relevance score, generating an optimized context set; and then calls the generation model based on the optimized context set to output the answer.

[0011] Preferably, the document retrieval module includes:

[0012] The user query is preprocessed, including identifying key terms and correcting errors to generate optimized query text. The optimized query text is then decomposed into retrieval intent subtasks, which are converted into corresponding specialized query vectors using an embedding model. These specialized query vectors are combined to form heterogeneous query vectors. The heterogeneous query vectors are then used to perform a similarity search in a preset vector database to obtain a candidate result set. Finally, the candidate result set is fused to generate a candidate document block containing business metadata.

[0013] Preferably, the calculation of the semantic relevance score includes:

[0014] Candidate document blocks and user queries are input into a pre-defined bidirectional encoder representation model to generate corresponding document term vectors and query term vectors. Based on the word frequency of the document term vector in the candidate document block and its inverse document frequency in a pre-constructed inverse document frequency dictionary, the document-side importance weight corresponding to the document term vector is calculated and stored. Based on the inverse document frequency dictionary, the query term vector is searched and assigned a corresponding query-side importance weight. A context transformation matrix is ​​generated for the local context of the query term vector, and the pre-encoded document term vector is transformed using the context transformation matrix to obtain the transformed document term vector. The cosine similarity between the query term vector and the transformed document term vector is calculated, and the cosine similarity is weighted and aggregated using the query-side importance weight and the document-side importance weight to generate a semantic relevance score.

[0015] Preferably, the process of deriving the business attribute score includes:

[0016] The user query is parsed using a natural language understanding model to extract business constraints, including categorical and numerical constraints. For the categorical constraints, path semantic matching is performed, which includes searching for relationship paths between the categorical constraints and business metadata based on a preset metadata ontology graph. The relationship paths are converted into path semantic vectors, and the similarity between the path semantic vectors and the query term vectors is calculated to generate a hierarchical relationship score. For the numerical constraints, context-aware rule evaluation is performed, which includes mapping the business metadata and the numerical values ​​in the constraints to a preset fuzzy logic set, dynamically adjusting the membership boundaries of the fuzzy logic set based on the context of the user query, and calculating a rule satisfaction score based on the adjusted membership. The hierarchical relationship score and the rule satisfaction score are multiplied to generate a business attribute score.

[0017] Preferably, the process of deriving the factual path score includes:

[0018] Based on user queries and candidate document blocks, query entities and document entities are extracted. The query entities are used as the root nodes of a pre-built enterprise knowledge graph, and a relationship tree expansion operation with a preset depth is performed to generate an evidence subgraph. By analyzing the inherent topological structure and semantic coherence of the evidence subgraph, a structure density score, a core node contribution score, and a path consistency score are calculated. The structure density score is obtained by calculating the interconnectivity of nodes in the evidence subgraph; the core node contribution score is obtained by calculating the network centrality of the document entity as a node in the evidence subgraph; and the path consistency score is obtained by encoding the relationship paths connecting the query entity and the document entity in the evidence subgraph into path semantic vectors and calculating the average pairwise similarity of the path semantic vectors. The structure density score, the core node contribution score, and the path consistency score are then fused to generate a factual path score.

[0019] Preferably, the generation of the comprehensive relevance score includes:

[0020] By analyzing user queries, the user's main intent is identified. Based on the main intent, a gated weighted network is used to dynamically generate weight coefficients for the semantic relevance score, business attribute score, and factual path score. The semantic relevance score, business attribute score, and factual path score are then fused based on the weight coefficients to generate a preliminary comprehensive score. The distribution consistency among the semantic relevance score, business attribute score, and factual path score is calculated to obtain the score confidence level. The preliminary comprehensive score is then calibrated and adjusted using the score confidence level to generate the final comprehensive relevance score.

[0021] Preferably, the sorting output module includes:

[0022] The candidate document blocks are initially sorted based on the comprehensive relevance score, and an empty optimization context set is set. The selection steps are executed iteratively until the number of document blocks in the optimization context set reaches a preset threshold. The selection steps include: for candidate document blocks that have not yet been selected into the optimization context set, calculating the semantic overlap between the candidate document block and the document blocks already included in the optimization context set to generate a redundancy penalty score; adjusting the comprehensive relevance score of the candidate document block using the redundancy penalty score to obtain a candidate score; and selecting candidate document blocks based on the candidate score and adding them to the optimization context set.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] 1. This invention introduces a "two-way token importance weighted post-interaction architecture". On the one hand, it uses the post-interaction mode to pre-process most of the heavy computation, ensuring millisecond-level online response speed. On the other hand, by intelligently weighting the core keywords in the query and document, the system can accurately focus on the core information that determines relevance, improving the accuracy of semantic matching and solving the fundamental contradiction in the prior art that it is difficult to achieve both high precision and high efficiency.

[0025] 2. Through a business attribute evaluation engine based on a "metadata ontology graph" and mechanisms such as "path semantic matching" and "context-aware rule evaluation", this invention enables the system to deeply understand the complex organizational structure, document specifications and dynamic business rules within an enterprise, ensuring that the final output is not only correct in content, but also fully compliant and usable in business logic, thereby enhancing the reliability and practicality of the system in complex enterprise environments.

[0026] 3. This invention constructs a fact-scoring model based on "relationship tree expansion," which determines the strength of factual evidence by generating and evaluating the inherent topological structure, node importance, and logical consistency of the "evidence subgraph." This paradigm of overall quality assessment of the evidence network can effectively identify candidate documents based on coincidence or weak associations, ensuring that the context provided to downstream models is factually reliable and logically rigorous. This reduces the risk of factual errors in the generated answers and enhances user trust. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the structure of the enterprise RAG intelligent question answering system based on vector database reordering of the present invention;

[0028] Figure 2 This is a flowchart of the document recall module of the present invention;

[0029] Figure 3 This is a schematic diagram of the structure of the comprehensive scoring module of the present invention;

[0030] Figure 4 This is a flowchart of the sorting output module of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Please see Figures 1 to 4 This invention provides an enterprise RAG intelligent question answering system based on vector database reordering, and the technical solution is as follows:

[0033] An enterprise RAG intelligent question answering system based on vector database reordering includes the following modules:

[0034] The document retrieval module receives user queries and retrieves candidate document blocks containing business metadata from a preset vector database based on the user queries.

[0035] The comprehensive scoring module employs a post-interaction architecture based on bidirectional token importance weighting to perform similarity aggregation operations on pre-encoded document term vectors and query term vectors to calculate semantic relevance scores. Based on a pre-defined metadata ontology graph, it performs path semantic matching and context-aware rule evaluation on business metadata and business constraints generated based on user queries to obtain business attribute scores. It extracts key entities from user queries and candidate document blocks and performs relation tree expansion to generate evidence subgraphs. By analyzing the internal topological structure and semantic coherence of the evidence subgraphs, it obtains factual path scores. Finally, it dynamically integrates semantic relevance scores, business attribute scores, and factual path scores to generate a comprehensive relevance score.

[0036] The sorting output module optimizes and reorders candidate document blocks based on a preset penalty mechanism and a comprehensive relevance score, generating an optimized context set; and then calls the generation model based on the optimized context set to output the answer.

[0037] Example 1

[0038] This embodiment provides a specific application of an enterprise RAG intelligent question answering system based on vector database reordering. Its typical application scenario is that a user makes a complex query to the internal knowledge base of organization A, and organization A introduces an enterprise RAG intelligent question answering system based on vector database reordering.

[0039] See Figure 1 The system proposed in this invention specifically includes:

[0040] An enterprise RAG intelligent question answering system based on vector database reordering includes:

[0041] The document retrieval module receives user queries and retrieves candidate document blocks containing business metadata from a preset vector database based on the user queries.

[0042] The comprehensive scoring module employs a post-interaction architecture based on bidirectional token importance weighting to perform similarity aggregation operations on pre-encoded document term vectors and query term vectors to calculate semantic relevance scores. Based on a pre-defined metadata ontology graph, it performs path semantic matching and context-aware rule evaluation on business metadata and business constraints generated based on user queries to obtain business attribute scores. It extracts key entities from user queries and candidate document blocks and performs relation tree expansion to generate evidence subgraphs. By analyzing the internal topological structure and semantic coherence of the evidence subgraphs, it obtains factual path scores. Finally, it dynamically integrates semantic relevance scores, business attribute scores, and factual path scores to generate a comprehensive relevance score.

[0043] The sorting output module optimizes and reorders candidate document blocks based on a preset penalty mechanism and a comprehensive relevance score, generating an optimized context set; and then calls the generation model based on the optimized context set to output the answer.

[0044] Furthermore, the step of receiving user queries and retrieving candidate document blocks containing business metadata from a preset vector database based on the user query, corresponding to the system's document retrieval module, specifically includes:

[0045] See Figure 2 A user submits a query: "Find the latest semiconductor industry technology research reports about Company B in the first half of the year, with the source being the chief analyst team." The system first preprocesses the query text, and the key term recognition model identifies domain-specific terms such as "semiconductor industry," "technology research report," and "chief analyst team," and generates an optimized query text based on these terms.

[0046] The optimized query text is decomposed into multiple intent subtasks. This decomposition is specifically implemented using a pre-trained sequence labeling model based on the BERT architecture. This model is trained to identify entities, required document types, and different types of phrases serving as key constraints within the query text. The identified phrases are then used as subtask content for semantic intent, entity intent, and constraint intent, respectively. Based on the intent subtasks, corresponding specialized query vectors are generated: Semantic intent: "In-depth research report on semiconductor industry technology" generates a semantic vector; Entity intent: "Company B" generates an entity vector; Constraint intent: "First half of the year" and "Chief Analyst Team" generate constraint vectors. Institution A's internal knowledge base includes: Market research reports: written by analysts at different levels, containing a large amount of text, tables, and charts, with high timeliness requirements; Internal compliance manual: stipulating guidelines for information disclosure and investment behavior, possessing high authority; Listed company financial reports and announcements: containing both structured and unstructured data.

[0047] All documents are segmented, vectorized, and stored in a vector database. The system uses semantic vectors, entity vectors, and constraint vectors to perform similarity searches in the vector database in parallel, obtaining three candidate result sets. These result sets are then merged using a ranking fusion algorithm to generate an initial candidate set containing 50 candidate document blocks, along with their complete business metadata. For example, the ranking fusion algorithm can be a reciprocal ranking fusion, whereby the final score for each candidate document block is determined by the sum of its reciprocals of its ranking in each candidate result set. To avoid low rankings having too much influence, a smoothing constant k is introduced during the calculation; for example, k can be 60. Through query intent decomposition and multi-vector parallel retrieval, the fuzzy natural language problem is transformed into a precise combinatorial search, improving the comprehensiveness and diversity of the retrieved results and providing a richer candidate base for subsequent precise ranking.

[0048] Furthermore, the post-interaction architecture based on bidirectional token importance weighting performs similarity aggregation operations on the pre-encoded document lexical vectors and query lexical vectors to calculate semantic relevance scores; based on a preset metadata ontology graph, business attribute scores are obtained by performing path semantic matching and context-aware rule evaluation on business metadata and business constraints generated based on user queries; key entities of user queries and candidate document blocks are extracted and relation tree expansion is performed to generate evidence subgraphs, and factual path scores are obtained by analyzing the internal topological structure and semantic coherence of the evidence subgraphs; a comprehensive relevance score is generated by dynamically fusing semantic relevance scores, business attribute scores, and factual path scores, corresponding to the system's comprehensive scoring module, specifically including:

[0049] See Figure 3 The system will score each document block in the initial candidate set, such as document block A and document block B.

[0050] Document Block A: From "Q1 Semiconductor Technology Frontier Insight Report", metadata is [Author: "Zhang San", Team: "Chief Analyst Team", Date: "03-15", Type: "Industry In-Depth Report", Confidentiality Level: "Internal Public"].

[0051] Document Block B: From "Company B Analysis Report", metadata is [Author: "Li Si", Team: "Ordinary Analyst Team", Date: "04-20", Type: "Company News", Confidentiality Level: "Internal Public"].

[0052] The system generates weighted term vectors for the query and document blocks A and B. For example, because "technical research report" in the query and "technological frontier insights" in document block A have similar contexts, the system generates similar context transformation matrices for them.

[0053] The context transformation matrix is ​​generated as follows: First, the vectors of the query word and the two words before and after it are extracted and concatenated to form a combined vector as the input of the network. This input is fed into a pre-trained transformation matrix generation network, which is a feedforward neural network. Its input layer receives a combined vector formed by concatenating the embedding vectors of the query word and the two words before and after it. It contains two hidden layers using the ReLU activation function. Its output layer is a square matrix with the same dimension as the word vector. This square matrix is ​​the transformation matrix that can represent the semantics of the local context. The pre-training objective of the transformation matrix generation network is to reduce the deviation of the transformation matrix generated by the network from the direction of the query term vector when applied to relevant document terms, and increase the deviation when applied to irrelevant document terms. Specifically, this is optimized using a triplet loss function. For a query term, positive examples are taken from corresponding terms in manually verified relevant documents, while negative examples are randomly sampled from irrelevant documents. This ensures that there is a preset minimum difference between the similarity scores of relevant pairs and the similarity scores of irrelevant pairs, which can be set to 0.2.

[0054] The system pre-calculates the inverse document frequency (IVF) for all terms in the knowledge base and constructs an IVF dictionary. This IVF dictionary is built through statistical analysis of all document content within Organization A's internal knowledge base. During similarity calculation, the system assigns a normalized IVF value as an importance weight to each term. Specifically, the IVF value is linearly mapped to a preset weight range, for example, [0.1, 1.5]. The importance weight for the term "semiconductor" can be calculated as 1.4, while the importance weights for the terms "of" and "about" can be calculated as 0.1 and 0.3, respectively. The transformed document term vector is obtained by matrix multiplication of the previously generated transformation matrix with the original document term vector. The system first calculates the weighted similarity between the query term and the transformed document term, forming a similarity score matrix. Then, it sums all the scores in this matrix and normalizes them using a minimum-maximum method, using this as the final aggregation result. For example, the semantic relevance score S_sem(A) is 0.92, and S_sem(B) is 0.85. The post-interaction architecture ensures millisecond-level response and utilizes two-way token weighting and context transformation to focus on core words, eliminate semantic ambiguity, and improve the accuracy and interpretability of semantic matching.

[0055] The system parses the following constraints for the user query: Type: "Technical Research Report", Team: "Chief Analyst Team", Date: (01-01, 06-30). The system pre-constructs a metadata ontology graph, which is a knowledge base defining the enterprise's internal metadata standards and hierarchical relationships. Its core node types include "Document Type", "Publishing Department", and "Author Rank", while relationship types define the subordinate or categorical relationships between them. For example, in the document type, "In-depth Industry Report" is a subclass of "Research Report"; in the analyst category, "Chief Analyst Zhang San" belongs to the "Chief Analyst Team".

[0056] For document block A, the system calculates the business attribute score through the following steps:

[0057] First, hierarchical relationship matching is performed: In the metadata ontology graph, document block A's [Type: "Industry In-Depth Report"] is a subclass of constraint [Type: "Technical Research Report"], with a path distance of 1; [Team: "Chief Analyst Team"] is a perfect match for the constraint; the text description of each node and edge in the relationship path is converted into its corresponding embedding vector and combined into a vector sequence. The vector sequence is input into a two-layer bidirectional long short-term memory network, and the final hidden state is taken as the "path semantic vector" representing the semantics of the entire path. The cosine similarity between this vector and the query term vector is calculated; finally, the hierarchical relationship score is 0.95.

[0058] Secondly, context-aware rule evaluation is performed: the date "03-15" in document block A falls entirely within the constrained time interval, and because the query includes "latest," the preset rule mapping table is triggered, tightening the "fractional half-life" parameter from the default 30 days to 15 days. The rule mapping table maps keywords in the query to parameter adjustments in the timeliness membership function. The rule mapping table specifically defines the logic of keyword and parameter adjustments; for example, keywords like "latest" and "recent" will have their half-life shortened by multiplying by 0.5, keywords like "urgent" and "expedited" will have their half-life multiplied by 0.2, and keywords like "review" and "archive" will have their half-life relaxed by multiplying by 2.0. The membership function is a smooth decay curve, with an output range of 0 to 100 points. The "score half-life" is the time required for the score to decay from 100 to 50. The decay pattern of this curve can be precisely defined: the score remains full within the full-score period after document publication, and the duration of this period is set to maintain a fixed proportional relationship with the currently effective "score half-life" parameter, for example, one-third of the half-life duration. After the full-score period, the score begins to decline smoothly over time, and the trajectory of the smooth decline curve is completely constrained and determined by the scores at two known key time points: firstly, at the end of the "full-score period," the score must be full (1.0); secondly, at the number of days specified by the "score half-life," the score must be half (0.5). When the interval between the publication time and the query time is within the "full-score period," the rule satisfaction score will be directly assigned the full score of 1.0. Finally, the hierarchical relationship score and the rule satisfaction score are multiplied, and the final business attribute score is 0.95.

[0059] For document block B, hierarchical relationship matching and context-aware rule evaluation are performed. Hierarchical relationship matching: The [Team: "Ordinary Analyst Team"] in document block B does not conform to the constraint, and the hierarchical relationship score is calculated as 0.3. Context-aware rule evaluation: The date "04-20" in document B falls entirely within the constraint time range, and because the query includes "latest," the dynamically adjusted fuzzy logical boundary gives a rule satisfaction score of 1.0. Using a product operation, the final business attribute score for B is 0.3. By constructing a metadata ontology graph and performing path semantic matching and context-aware rule evaluation, abstract business rules are transformed into computable scores, enabling the system to accurately execute complex compliance screening and ensuring the compliance of the final answer.

[0060] The system pre-constructs an enterprise knowledge graph, which is automatically built from public financial reports, industry news, and internal reports using entity and relationship extraction techniques. The knowledge graph includes entity types and relationship types; for example, entity types may include "company," "financial product," and "executive," while relationship types may include "investment relationship," "competitive relationship," and "employment relationship." Starting with the query entity "Company B," a relationship tree expansion operation with a depth of 3 is performed in the enterprise knowledge graph. This expansion uses a breadth-first search strategy, prioritizing relationship types relevant to the application scenario, such as "investment," "shareholding," and "executive employment." The execution depth is set to an optimized value that balances capturing multiple layers of indirect relationships with controlling computational complexity, ultimately generating an evidence subgraph containing "industry," "core products," "key executives," and "competitors."

[0061] Multi-factor scoring was performed on document block A: Document A not only contains "Company B", but also mentions the core product "Chip X" and the competitor "Company C". These entities are all in the evidence subgraph and are in a core position.

[0062] The structure density score is calculated by normalizing the ratio of the actual number of edges in the evidence subgraph to the theoretically maximum number of edges. For example, the structure density score of document block A is calculated to be 0.8.

[0063] The core node contribution score is obtained by calculating and normalizing the betweenness centrality of the document entity within the evidence subgraph. The betweenness centrality measures the frequency with which a node acts as a "bridge" on the shortest path between other node pairs. The system calculates the betweenness centrality of the entity nodes in document block A and takes the maximum value as the core node contribution score. For example, the core node contribution score of document block A is calculated to be 0.9.

[0064] The path consistency score is obtained by encoding all paths connecting the query entity and the document entity in the subgraph into multiple path semantic vectors using the aforementioned bidirectional long short-term memory network, and then calculating the average pairwise cosine similarity between these vectors. The path consistency score of document block A is calculated to be 0.95.

[0065] The three scores are fused to generate the fact path score for document block A, which is calculated to be 0.86. This fusion is achieved through a pre-defined weighted summation logic, where the weights of each component are optimized using a grid search. The goal of this grid search optimization is to maximize the normalized cumulative gain of the final ranking result on a reserved validation set with manually labeled relevance levels. For document block B, which only mentions the entity "Company B," its fact path score is lower than A. By constructing an evidence subgraph and analyzing its internal structure, core nodes, and path consistency, fact assessment is transformed from linear path search to evidence network analysis. This improves the ability to uncover non-obvious relationships between entities, enhancing the robustness of fact assessment and the reliability of the final answer.

[0066] In deriving the fact path score, metadata attributes including timestamps and source levels are pre-added to each relation edge of the enterprise knowledge graph. The timestamps are time information extracted based on business metadata, and the source levels are pre-defined levels based on the authority of the original documents. The timestamps and source levels of each relation edge in the path are used as weighting factors to modulate the path consistency score, and the final fact path score is calculated. By incorporating the timeliness of facts and the authority of sources into the scoring model, a deep and dynamic evaluation of evidence quality can be achieved, improving the accuracy and credibility of the final generated answer.

[0067] Analyzing user queries reveals that the query intent includes both business constraints (chief analyst) and factual inquiries (company B). A gated weighted network dynamically generates weights, for example, "W_sem: 0.2, W_biz: 0.5, W_fact: 0.3". This gated weighted network is a lightweight feedforward neural network. Its input is the overall semantic vector generated by an encoder model from the user's original query text. The network contains one hidden layer, and the output layer uses a Softmax activation function to output three weights whose sum is 1. This network is jointly trained end-to-end with the final generative model. Its core training mechanism uses a clearly defined weighted combined loss function to convert answer quality into a computable supervisory signal to inversely adjust the weights. The weighted combined loss function consists of two parts:

[0068] The system pre-constructs a training dataset with multi-level relevance annotations. The training dataset consists of query instances, each containing: the user's query text; a list of candidate document blocks generated by the document retrieval module for that query; and relevance level annotations given by internal business experts to the document blocks in the list, indicating their relevance to the query. The relevance levels are divided into five categories: "Perfect" (directly and completely answers the query), "Excellent" (contains the core answer but requires some reasoning), "Good" (relevant but not core), "Average" (topic relevant but no answer), and "Irrelevant."

[0069] Generation Loss: This section employs standard cross-entropy loss, evaluating the accuracy and fluency of the generated content by comparing the model's generated answers with the standard reference answers. This loss function is primarily used to optimize the downstream generation model, ensuring its output of high-quality text.

[0070] Ranking Loss: This part uses a list-based Softmax cross-entropy ranking loss for optimization. For a list containing multiple candidate document blocks, the system generates a "predicted probability distribution" based on the comprehensive relevance score of each document block using the Softmax function. An ideal probability distribution is constructed based on the training data. In this ideal probability distribution, the target probability of a candidate document block is set according to its manually labeled relevance level to the query. Five relevance levels, from "perfect" to "irrelevant," are mapped to non-negative scores, such as 4, 3, 2, 1, and 0. Softmax normalization is then performed based on these non-negative scores to generate a smoother ideal probability distribution. The goal of the training process is to minimize the difference between the "predicted probability distribution" and the "ideal probability distribution" by adjusting the model's internal parameters.

[0071] The two losses mentioned above are weighted and summed with preset weights to generate the final supervision signal. The weight of the generation loss can be preset to 0.6, and the weight of the ranking loss can be preset to 0.4. By backpropagating the gradient signal to the gating network, it can learn how to dynamically allocate weights according to user intent. When the query is biased towards seeking specific facts, the weight of the fact path score is increased; when the query contains strict business filtering conditions, the weight of the business attribute score is increased.

[0072] The semantic relevance score, business attribute score, and factual path score are weighted and summed to generate a preliminary comprehensive score. For example, the preliminary comprehensive score for document block A is calculated to be 0.917. The score distribution is analyzed for confidence calibration. For example, the distribution of "0.92, 0.95, 0.86" in document block A is concentrated and highly consistent, with a score confidence score of C(A) = 0.98. This score confidence score quantifies the distribution consistency by calculating the statistical standard deviation of the three input scores. Specifically, the standard deviation is normalized, and the normalized value is subtracted from 1 to obtain the confidence score representing consistency. The comprehensive relevance score is the product of the preliminary comprehensive score and the score confidence score. For example, the comprehensive relevance score for document block A could be 0.899. This intent-gated weighting achieves dynamic focusing of the scoring, and the score confidence score is then used to calibrate the results, improving the responsiveness of the comprehensive relevance score to user query intent and enhancing the decision-making accuracy in handling multi-dimensional conflicting information.

[0073] In the process of generating the comprehensive relevance score, an adaptive weight recalibration step is performed, specifically including: comparing the business attribute score with a preset veto threshold; when the business attribute score is lower than the veto threshold, assigning a preset invalid value to the comprehensive relevance score of the candidate document block; when the business attribute score is not lower than the veto threshold, the system calculates the distribution consistency of the semantic relevance score, business attribute score, and fact path score to obtain a score confidence level; if the score confidence level is lower than a preset confidence level threshold, outlier score items are identified and a preset punitive decay is applied to the weight coefficients corresponding to the outlier score items; the weight values ​​reduced due to decay are redistributed to the remaining score items according to the original weight ratio to generate recalibrated weight coefficients. By dynamically suppressing outliers and enhancing the weights of consensus items, the comprehensive score is effectively prevented from being misled by a single abnormal score in most conventional query scenarios, improving the robustness of the scoring system and the reliability of the final decision.

[0074] Furthermore, the process of optimizing and reordering candidate document blocks based on a preset penalty mechanism and a comprehensive relevance score to generate an optimized context set; and then calling the generation model based on the optimized context set to output the answer, corresponding to the system's sorting output module, specifically includes:

[0075] See Figure 4 Initially sort all 50 candidate document blocks according to their final composite scores; for example, document block A is ranked first. Set an empty optimization context set with a target size of 3.

[0076] Perform iterative selection operations:

[0077] First iteration: Select document block A with the highest score and add it to the optimization context set.

[0078] Second iteration: Assume that the initially ranked second document block D has a comprehensive relevance score of 0.850, and its content has a semantic vector cosine similarity of 0.95 with A. The system calculates a redundancy penalty score for it. The third-ranked document block E has a comprehensive relevance score of 0.83, its content is complementary to A, and its semantic vector cosine similarity is 0.20. After adjustment, E's dynamic candidate score is greater than D's. The adjustment is implemented through multiplicative decay logic: the dynamic candidate score is equal to its original comprehensive score multiplied by the coefficient after redundancy penalty. The redundancy penalty score itself is obtained by calculating the maximum value of the semantic vector cosine similarity between the current candidate document block and the selected document block. The decay coefficient is equal to 1 minus the product of the redundancy penalty score and the preset penalty factor. The preset penalty factor is between 0 and 1, and can be set to 0.7 based on expert experience. After calculation, D's dynamic score is 0.285, while E's dynamic score is 0.714. Based on E's dynamic candidate score, document block E is selected and added to the optimization context set.

[0079] Third iteration: Repeat the above process to select the next document block that is least redundant with both A and E and has the highest score.

[0080] In the process of generating redundancy penalty scores, candidate document blocks are combined with selected document blocks in the optimization context set to form document pairs. These document pairs are used as input to a natural language inference model, which outputs the logical relationship between the candidate and selected document blocks. This logical relationship includes contradictory and implication relationships. When the logical relationship is contradictory, the candidate score for the candidate document block is set to 0. When the logical relationship is implication, the semantic overlap between the candidate and selected document blocks is considered as the maximum value when calculating the redundancy penalty score. By introducing logical-level judgments, documents with contradictory content or contained information are actively excluded, ensuring the logical consistency and information density of the final optimization context set, and improving the rigor and reliability of the generated answer.

[0081] After iteration, the optimized context set, containing three document blocks, with the highest information density and no redundancy, is submitted along with the original query to the downstream generative model to generate the final, well-organized answer. By using iterative optimization and redundancy penalties, the ranking objective is shifted from selecting the best individual to constructing the best combination, improving the logical coherence and information density of the final generated answer and enhancing its reliability.

[0082] This invention solves the problems of semantic bias, lack of business rules, and insufficient factual basis caused by single-dimensional retrieval by constructing a complete system architecture that includes document retrieval, multi-dimensional scoring, and optimized ranking. It ensures the semantic accuracy, business compliance, and factual reliability of the final generated answer.

[0083] Example 2

[0084] This example uses the financial institution in Example 1 as the application background. The user submits a query: "Find all internal operating guidelines issued by the compliance department after the first quarter that are related to 'internal risk control' and whose confidentiality level must be 'confidential'."

[0085] Upon receiving the query, the system first retrieves an initial candidate set containing multiple candidate document blocks using the document retrieval module. Then, the comprehensive scoring module scores each candidate document block.

[0086] Candidate document block X: originates from "Internal Risk Control Operation Rules", and its associated business metadata is [Issuing Department: "Compliance Department", Release Date: "05-20", Document Type: "Internal Operation Guide", Security Classification: "Confidential"].

[0087] Candidate document block Y: originates from "Latest Market Risk Analysis", and its business metadata is [Publishing Department: "Market Research Department", Publication Date: "06-10", Document Type: "Industry Analysis", Confidentiality Level: "Internal Public"].

[0088] Candidate document block Z: It comes from "Customer Identification Guidelines", and its business metadata is [Issuing Department: "Compliance Department", Release Date: "04-15", Document Type: "Internal Operation Guide", Confidentiality Level: "Internal Public"].

[0089] The system uses a language model fine-tuned based on sequence labeling tasks to accurately parse the user's natural language query into a set of structured business constraints: [Content keywords: "internal risk control", document type: "internal operation guide", issuing department: "compliance department", issuance date: "greater than 03-31", security level: "confidential"].

[0090] For document block X: its "Publishing Department" and "Document Type" perfectly match the constraints, and the path distance on the preset metadata ontology graph is zero, therefore the hierarchical relationship score is calculated as 1.0; its "Publishing Date" and "Confidentiality Classification" also perfectly satisfy the constraints, therefore the rule satisfaction score is calculated as 1.0. Finally, through product operation, its business attribute score is 1.0.

[0091] For document block Y: its "Publishing Department" (Market Research Department) and "Document Type" (Industry Analysis) do not meet the constraints, its hierarchical relationship score is calculated as 0.1, and its business attribute score is calculated as 0.1 accordingly.

[0092] For document block Z: its "Publishing Department" and "Document Type" both satisfy the constraints, and the hierarchical relationship score is calculated to be 0.9. However, during the context-aware rule evaluation, the system finds that its "Security Classification" does not satisfy the rigid constraint that "the security classification must be 'Confidential'". Therefore, its rule satisfaction score is directly determined to be 0. Through multiplication, its final business attribute score is 0.

[0093] Although document block Z achieves a semantic relevance score of 0.95, its business attribute score is set to 0 due to the "veto rule," resulting in a final overall relevance score approaching 0 after dynamic fusion. In contrast, document block X scores above 0.9 in all sub-items across semantic, business, and factual levels, generating a higher overall relevance score than document block Z. By achieving a deep understanding of category constraints through path semantic matching and then dynamically adapting numerical constraints using context-aware rule evaluation, the intelligence and accuracy of business attribute scoring are improved, enhancing the compliance and applicability of the final answer.

[0094] Example 3

[0095] This embodiment uses the financial institution in Embodiment 1 as the application background. The user puts forward a query: "Analyze the possible indirect relationship between 'Company B' and 'Capital Y', such as through jointly held companies or the employment history of key executives."

[0096] The system extracts the query entities "Company B, Capital Y" from the query. Using these two entities as root nodes, the system performs a parallel "relationship tree expansion" operation of depth 3 within a pre-built enterprise knowledge graph, generating an "evidence subgraph." For example, the "evidence subgraph" contains 25 entity nodes and 40 relationship edges. The knowledge graph contains the following facts: "Capital Y" invests in "Technology Z," "Executive A" is a co-founder of "Company B," and "Executive A" is a director of "Technology Z." At this point, the evidence subgraph constructs an indirect path connecting "Capital Y" and "Company B."

[0097] During this process, the document retrieval module retrieves a candidate document K, which mentions that "Mr. Jia, a board member and executive of 'Z Technology,' was a co-founder of Company B." The system will perform multi-factor scoring on candidate document K, as follows: Since the evidence subgraph connects the two originally isolated query entities through the nodes "Z Technology" and "Executive Jia" to form a path and a closed loop, its average node degree reaches 3.2. After normalization, the structure density score is 0.85.

[0098] Core node contribution score: The document entities "Executive A" and "Z Technology" extracted from candidate document K are calculated by the system. The network centrality of these two entities in the evidence subgraph is found to be 0.48 for the betweenness centrality of the "Executive A" node and 0.35 for the betweenness centrality of the "Z Technology" node. Both are key bridge nodes in the subgraph. After aggregation and normalization, the core node contribution score is 0.95.

[0099] Path consistency score: In the evidence subgraph, the three key paths connecting the query entity and the document entity "Executive A" are encoded into path semantic vectors. The average pairwise cosine similarity between these three paths is calculated to be 0.88, indicating that the evidence chain focuses on the theme of "investment and executive appointment". Therefore, the path consistency score is 0.90.

[0100] Although the text content of candidate document K may not simultaneously contain "Company B" and "Y Capital," its semantic relevance score may only be 0.45; however, due to its structure density score of 0.85, core node contribution score of 0.95, and path consistency score of 0.90, the factual path score of candidate document K can be calculated as 0.91. This factual path score is obtained by weighted summation of the three sub-scores; for example, the core node contribution score has a weight of 0.5, the path consistency score has a weight of 0.3, and the structure density score has a weight of 0.2. In the final dynamic fusion, this factual path score (0.91) will significantly improve the overall ranking of candidate document K. By constructing a comprehensive evidence subgraph through relation tree expansion and then using a multi-factor model to quantitatively evaluate the intrinsic quality of the evidence subgraph, the depth and robustness of factual evaluation are improved, thus enhancing the factual reliability of the final answer.

[0101] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An enterprise RAG intelligent question-answering system based on vector database reordering, characterized in that, include: The document retrieval module receives user queries and retrieves candidate document blocks containing business metadata from a preset vector database based on the user queries. The comprehensive scoring module uses a post-interaction architecture based on bidirectional token importance weighting to perform similarity aggregation operations on pre-encoded document term vectors and query term vectors to calculate semantic relevance scores. The calculation of semantic relevance score includes: inputting candidate document blocks and user queries into a preset bidirectional encoder representation model to generate corresponding document term vectors and query term vectors; and calculating and storing the document-side importance weights corresponding to the document term vectors based on the term frequencies of the document term vectors in the candidate document blocks and the inverse document frequencies in a pre-built inverse document frequency dictionary. Based on the inverse document frequency dictionary, the query term vector is searched and assigned a corresponding query importance weight; A context transformation matrix is ​​generated for the local context of the query term vector. The pre-encoded document term vector is then transformed using the context transformation matrix to obtain the transformed document term vector. The cosine similarity between the query term vector and the transformed document term vector is calculated. The cosine similarity is then weighted and aggregated using the importance weights of the query end and the document end to generate a semantic relevance score. Based on a pre-defined metadata ontology graph, business attribute scores are obtained by performing path semantic matching and context-aware rule evaluation on business metadata and business constraints generated based on user queries. Extract key entities from user queries and candidate document blocks and perform relation tree expansion to generate evidence subgraphs. Analyze the internal topological structure and semantic coherence of the evidence subgraphs to obtain fact path scores. Generate a comprehensive relevance score by dynamically fusing semantic relevance scores, business attribute scores, and fact path scores. The sorting output module optimizes and reorders candidate document blocks based on a preset penalty mechanism and a comprehensive relevance score, generating an optimized context set. The model outputs the answer based on the optimized context set.

2. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1, characterized in that, The document retrieval module includes: The user query is preprocessed, including identifying key terms in the user query and performing query error correction to generate optimized query text; the optimized query text is decomposed into retrieval intent subtasks, which are converted into corresponding specialized query vectors through an embedding model, and the specialized query vectors are combined to form a heterogeneous query vector. The heterogeneous query vectors are used to perform a similarity search in a preset vector database to obtain a candidate result set, and the candidate result set is then merged to generate a candidate document block containing business metadata.

3. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1, characterized in that, The derived business attribute score includes: The user query is parsed using a natural language understanding model to extract business constraints that include categorical and numerical constraints. For the categorical constraints, path semantic matching is performed, which includes searching the relationship path between the categorical constraints and the business metadata based on a preset metadata ontology graph. The relationship path is converted into a path semantic vector, and the similarity between the path semantic vector and the query term vector is calculated to generate a hierarchical relationship score. For the numerical constraint, a context-aware rule evaluation is performed. The context-aware rule evaluation includes mapping the business metadata and the numerical values ​​in the constraint to a preset fuzzy logic set, dynamically adjusting the membership boundary of the fuzzy logic set based on the context of the user query, and calculating the rule satisfaction score based on the adjusted membership. The business attribute score is generated by multiplying the hierarchical relationship score and the rule satisfaction score.

4. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1, characterized in that, The derived fact path score includes: Based on user queries and candidate document blocks, query entities and document entities are extracted; the query entities are used as the root nodes of a pre-built enterprise knowledge graph, and a relation tree expansion operation with a preset depth is performed to generate an evidence subgraph; by analyzing the internal topological structure and semantic coherence of the evidence subgraph, the structure density score, core node contribution score, and path consistency score are calculated. The structure density score is obtained by calculating the interconnectivity of nodes in the evidence subgraph. The core node contribution score is obtained by calculating the network centrality of the document entity as a node in the evidence subgraph. The path consistency score is obtained by encoding the relationship path connecting the query entity and the document entity in the evidence subgraph into a path semantic vector and calculating the average pairwise similarity of the path semantic vector. The structure density score, the core node contribution score and the path consistency score are fused to generate the factual path score.

5. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1, characterized in that, The generated comprehensive relevance score includes: By analyzing user queries, the user's main intent is identified. Based on the main intent, a gated weighted network is used to dynamically generate weight coefficients for the semantic relevance score, business attribute score, and fact path score. Based on the weight coefficients, the semantic relevance score, business attribute score, and fact path score are fused to generate a preliminary comprehensive score. The confidence level of the score is obtained by calculating the distribution consistency among the semantic relevance score, business attribute score and fact path score; the confidence level of the score is then used to calibrate and adjust the preliminary comprehensive score to generate the final comprehensive relevance score.

6. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1, characterized in that, The sorting output module includes: The candidate document blocks are initially sorted based on the comprehensive relevance score, and an empty optimization context set is set. The selection steps are executed iteratively until the number of document blocks in the optimization context set reaches a preset threshold. The selection steps include: for candidate document blocks that have not yet been selected into the optimization context set, calculating the semantic overlap between the candidate document block and the document blocks already included in the optimization context set to generate a redundancy penalty score; adjusting the comprehensive relevance score of the candidate document block using the redundancy penalty score to obtain a candidate score; and selecting candidate document blocks based on the candidate score and adding them to the optimization context set.

Images