Vector database reordering-based enterprise RAG intelligent question-answering system

Abstract

The invention relates to the technical field of intelligent retrieval, in particular to an enterprise RAG intelligent question answering system based on vector database reordering. The system specifically comprises: a document recall module, which retrieves a vector database to obtain candidate document blocks containing business metadata; the comprehensive scoring module is used for calculating a semantic correlation score by adopting a later-stage interaction architecture based on bidirectional token importance weighting, performing path semantic matching and context sensing rule evaluation according to a preset metadata ontology graph to obtain a service attribute score, and analyzing the evidence sub-graph to obtain a fact path score; fusing the semantic correlation score, the service attribute score and the fact path score to generate a comprehensive correlation score; and the sorting output module performs optimization resorting based on a preset punishment mechanism and the comprehensive correlation score to generate an optimized context set, and calls a generation model to output answers based on the optimized context set. According to the method, semantic accuracy, business compliance and fact reliability can be considered, and more trustworthy high-quality enterprise-level answers can be generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent retrieval, in particular to an enterprise RAG intelligent question and answer system based on vector database reordering. BACKGROUND

[0002] In the process of enterprise digital transformation, how to accurately and efficiently obtain information from a large number of heterogeneous internal knowledge bases is a key technical challenge to improve organizational decision-making ability and operational efficiency. Traditional knowledge retrieval methods have limitations in understanding complex natural language queries and cannot meet the needs of deep and accurate question and answer.

[0003] In the prior art, the architecture based on retrieval augmented generation (RAG) has become the mainstream solution to this problem. This architecture performs preliminary semantic retrieval through a vector database to recall a candidate document set, and then optimizes the set through a reordering link to improve the quality of the final generated answer. However, in enterprise practice, the core retrieval and sorting process of the existing solution has room for optimization. First, the preliminary vector retrieval results may have semantic deviations and do not completely match the user's true intent. Second, to improve the sorting accuracy, the use of complex high-precision reordering models often accompanies significant response delays, and it is a challenge to ensure effectiveness while meeting real-time interaction requirements. In addition, the existing sorting mechanism does not adequately consider the business attributes of the documents, and how to effectively integrate document timeliness, source authority and other metadata with content relevance, as well as how to effectively integrate heterogeneous information from multiple source databases, are also technical directions that the field continues to focus on.

[0004] In summary, the existing technology has the problem of being unable to integrate multi-dimensional information, resulting in limited quality of the final answer. Therefore, an enterprise RAG intelligent question and answer system based on vector database reordering is proposed. SUMMARY

[0005] The purpose of the present application is to provide an enterprise RAG intelligent question and answer system based on vector database reordering for intelligent information retrieval. The present application aims to generate more reliable high-quality enterprise-level answers by considering semantic accuracy, business compliance and factual reliability through a multi-dimensional intelligent reordering engine.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solutions: The enterprise RAG intelligent question and answer system based on vector database reordering comprises: a document recall module that receives a user query, retrieves a preset vector database based on the user query to obtain a candidate document block containing business metadata; The comprehensive score module adopts a late interaction architecture based on bidirectional token importance weighting to perform similarity aggregation operation on the pre-encoded document token vector and the query token vector, calculates a semantic relevance score; based on a preset metadata ontology graph, through path semantic matching and context-aware rule evaluation on the business metadata and the business constraint condition generated based on the user query, a business attribute score is obtained; key entities of the user query and the candidate document block are extracted and a relationship tree expansion is performed to generate an evidence subgraph, and through analysis of the internal topological structure and semantic coherence of the evidence subgraph, a fact path score is obtained; and a comprehensive relevance score is generated by dynamically fusing the semantic relevance score, the business attribute score and the fact path score. The ranking output module optimizes and reorders the candidate document block based on a preset penalty mechanism and the comprehensive relevance score, and generates an optimized context set; and calls a generation model based on the optimized context set to output an answer.

[0007] Preferably, the document recall module comprises: The user query is subjected to text preprocessing, which includes identifying key terms of the user query and performing query error correction to generate an optimized query text; the optimized query text is decomposed into retrieval intent subtasks, the retrieval intent subtasks are converted into corresponding specialized query vectors by an embedding model, and the specialized query vectors are combined to form a heterogeneous query vector; the heterogeneous query vector is used to perform similarity search in a preset vector database to obtain a candidate result set, and the candidate result set is fused to generate a candidate document block containing business metadata.

[0008] Preferably, the calculation of the semantic relevance score comprises: The candidate document block and the user query are input into a preset bidirectional encoder representation model to generate corresponding document token vectors and query token vectors; based on the term frequency of the document token vector in the candidate document block and the inverse document frequency in a pre-constructed inverse document frequency dictionary, the document token vector corresponding document end importance weight is calculated and stored; based on the inverse document frequency dictionary, the query token vector is looked up and assigned a corresponding query end importance weight; a context transformation matrix is generated for the local context of the query token vector, and the pre-encoded document token vector is transformed into a transformed document token vector using the context transformation matrix; the cosine similarity between the query token vector and the transformed document token vector is calculated, and the cosine similarity is weighted and aggregated using the query end importance weight and the document end importance weight to generate a semantic relevance score.

[0009] Preferably, the calculation of the business attribute score comprises: The user query is analyzed by a natural language understanding model to extract business constraint conditions including category type constraints and numerical type constraints; for the category type constraints, path semantic matching is performed, which includes searching for a relationship path of the category type constraint and business metadata based on a preset metadata ontology graph; the relationship path is converted into a path semantic vector, the similarity of the path semantic vector and the query word vector is calculated, and a hierarchical relationship score is generated; for the numerical type constraints, context-aware rule evaluation is performed, which includes mapping the business metadata and the numerical value in the constraint condition to a preset fuzzy logic set, dynamically adjusting the membership boundary of the fuzzy logic set based on the context of the user query, and calculating a rule satisfaction score according to the adjusted membership; the hierarchical relationship score and the rule satisfaction score are multiplied to generate a business attribute score.

[0010] Preferably, the fact path score comprises: Based on the user query and the candidate document block, query entities and document entities are extracted; the query entities are taken as root nodes of a pre-constructed enterprise knowledge graph, and a relationship tree expansion operation with a preset depth is performed to generate an evidence subgraph; by analyzing the internal topological structure and semantic coherence of the evidence subgraph, a structure density score, a core node contribution score, and a path consistency score are calculated; the structure density score is obtained by calculating the node interconnection degree of the evidence subgraph, the core node contribution score is obtained by calculating the network centrality of the document entity as a node in the evidence subgraph, and the path consistency score is obtained by encoding the relationship path connecting the query entities and the document entities in the evidence subgraph into a path semantic vector and calculating the average pairwise similarity of the path semantic vector; the structure density score, the core node contribution score, and the path consistency score are fused to generate a fact path score.

[0011] Preferably, the generating of the comprehensive relevance score comprises: By analyzing the user query, the main intention of the user is identified, and based on the main intention, the weight coefficients of the semantic relevance score, the business attribute score, and the fact path score are dynamically generated by a gating weighted network, and the preliminary comprehensive score is generated by fusing the semantic relevance score, the business attribute score, and the fact path score based on the weight coefficients; the distribution consistency between the semantic relevance score, the business attribute score, and the fact path score is calculated to obtain a score confidence; the preliminary comprehensive score is calibrated and adjusted to generate the final comprehensive relevance score using the score confidence.

[0012] Preferably, the sorting output module comprises: The candidate document blocks are initially ranked based on the comprehensive correlation scores, and an empty optimization context set is set; a selection step is cyclically performed until the number of document blocks in the optimization context set reaches a preset threshold, and the selection step comprises: for the candidate document blocks that have not been selected into the optimization context set, a redundancy penalty score is generated by calculating the semantic overlap degree between the candidate document blocks and the document blocks already contained in the optimization context set; the comprehensive correlation score of the candidate document block is adjusted by using the redundancy penalty score to obtain a candidate score; and the candidate document block is selected based on the candidate score and added to the optimization context set.

[0013] Compared with the prior art, the application has the following beneficial effects: 1、The application introduces a "two-way token importance weighted late interaction architecture", which on the one hand prepositions most heavy computing in the late interaction mode to ensure millisecond-level online response speed, and on the other hand intelligently weights the core keywords in the query and the document to enable the system to accurately focus on the core information that determines the correlation, thereby improving the accuracy of semantic matching and solving the fundamental contradiction between high precision and high efficiency in the prior art.

[0014] 2、The application uses a business attribute evaluation engine based on a "metadata ontology graph", and mechanisms such as "path semantic matching" and "context-aware rule evaluation", so that the system can deeply understand the complex organizational structure, document specification and dynamic business rules within an enterprise, thereby ensuring that the final output answer is not only correct in content but also completely compliant and usable in business logic, and enhancing the reliability and practicality of the system in a complex enterprise environment.

[0015] 3、The application constructs a fact scoring model based on a "relation tree extension", which judges the strength of the fact evidence by generating and evaluating the internal topological structure, node importance and logical consistency of the "evidence subgraph". This paradigm for overall quality evaluation of the evidence network can effectively identify candidate documents based on coincidence or weak association, ensure that the context provided to the downstream model is factually reliable and logically rigorous, thereby reducing the risk of fact errors in generated answers and improving user trust. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 It is a structural schematic diagram of the enterprise RAG intelligent question and answer system based on vector database reordering of the application; Figure 2 It is a workflow diagram of the document recall module of the application; Figure 3 It is a structural schematic diagram of the comprehensive scoring module of the application; Figure 4 It is a workflow diagram of the sorting output module of the application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work belong to the scope of protection of the present application.

[0018] Please refer to Figures 1 to 4 The present application provides an enterprise RAG intelligent question answering system based on vector database reordering, and the technical solutions are as follows: The enterprise RAG intelligent question answering system based on vector database reordering comprises the following modules: The document recall module receives a user query, retrieves a preset vector database based on the user query to obtain a candidate document block containing business metadata; The comprehensive scoring module performs similarity aggregation operation on the pre-encoded document token vector and the query token vector by using a late interaction architecture based on bidirectional token importance weighting, calculates a semantic relevance score, performs path semantic matching and context-aware rule evaluation on the business metadata and the business constraint condition generated based on the user query based on a preset metadata ontology graph, obtains a business attribute score, extracts key entities of the user query and the candidate document block, and performs relationship tree expansion to generate an evidence subgraph, obtains a fact path score by analyzing the internal topological structure and semantic coherence of the evidence subgraph, and generates a comprehensive relevance score by dynamically fusing the semantic relevance score, the business attribute score and the fact path score; The sorting output module optimally reorders the candidate document block based on a preset penalty mechanism and the comprehensive relevance score, generates an optimized context set, and calls a generation model to output an answer based on the optimized context set.

[0019] Embodiment one The present embodiment provides a specific application of the enterprise RAG intelligent question answering system based on vector database reordering, and a typical application scenario is that a user makes a complex query to an internal knowledge base of an organization A, and the organization A introduces the enterprise RAG intelligent question answering system based on vector database reordering.

[0020] Please refer to Figure 1 The system provided in the present application specifically comprises: The enterprise RAG intelligent question answering system based on vector database reordering comprises: The document recall module receives a user query, retrieves a preset vector database based on the user query to obtain a candidate document block containing business metadata; The comprehensive score module adopts a late interaction architecture based on bidirectional token importance weighting to perform similarity aggregation operation on the pre-encoded document token vector and the query token vector, calculates a semantic relevance score; based on a preset metadata ontology graph, the business metadata and the business constraint conditions generated based on the user query are subjected to path semantic matching and context-aware rule evaluation, to obtain a business attribute score; key entities of the user query and the candidate document block are extracted and a relationship tree expansion is performed to generate an evidence subgraph, and a fact path score is obtained by analyzing the internal topological structure and semantic coherence of the evidence subgraph; a comprehensive relevance score is generated by dynamically fusing the semantic relevance score, the business attribute score and the fact path score; The ranking output module optimizes and reorders the candidate document blocks based on a preset penalty mechanism and the comprehensive relevance score, to generate an optimized context set; and the answer is output based on the optimized context set.

[0021] Further, the receiving user query, based on the user query, retrieves a preset vector database to obtain candidate document blocks containing business metadata, which corresponds to the document recall module of the system, specifically comprising: Referring to Figure 2 , the user proposes a query: "Find the latest semiconductor industry technology research report on company B for the first half of the year, and require the source to be the chief analyst team." The system first preprocesses the query text, and a key term recognition model identifies the domain-specific vocabulary of "semiconductor industry", "technology research report" and "chief analyst team", and generates an optimized query text based on the domain-specific vocabulary.

[0022] The optimized query text is decomposed into multiple intent subtasks, and the decomposition is specifically implemented by a pre-trained sequence labeling model based on the BERT architecture. The model is trained to identify entities, required document types and different types of phrases as key constraint conditions contained in the query text, and the identified phrases of each type are respectively used as the subtask content of semantic intent, entity intent and constraint intent. Based on the intent subtask, corresponding specialized query vectors are generated: semantic intent: "semiconductor industry technology in-depth research report" generates a semantic vector; entity intent: "company B" generates an entity vector; constraint intent: "first half of the year" and "chief analyst team" generate constraint vectors. The internal knowledge base of institution A includes market research reports: written by analysts of different levels, containing a large amount of text, tables and charts, with high timeliness requirement; internal compliance manual: specifies the guidelines for information disclosure and investment behavior, with high authority; listed company financial reports and announcements: structured and unstructured data coexist.

[0023] All documents are chunked, vectorized and stored in a vector database. The system utilizes semantic vectors, entity vectors and constraint vectors to perform similarity search in the vector database in parallel, obtaining three candidate result sets. Through a ranking fusion algorithm, the result sets are merged, and finally an initial candidate set containing 50 candidate document blocks is generated, and its complete business metadata is loaded, for example, the ranking fusion algorithm can be reverse ranking fusion, specifically: for each candidate document block, its final score is determined by the sum of the reciprocal of its ranking in each candidate result set, to avoid the excessive influence of low ranking, a smoothing constant k is introduced when calculating, for example, k can be taken as 60. Through query intent decomposition and multi-vector parallel retrieval, ambiguous natural language questions are converted into precise combined searches, improving the comprehensiveness and diversity of the recall results, and providing a richer candidate basis for subsequent precise ranking.

[0024] Further, the late interaction architecture based on bidirectional token importance weighting is used to perform similarity aggregation operation on the pre-encoded document token vector and the query token vector, calculate the semantic relevance score; based on the pre-set metadata ontology graph, the business metadata and the business constraint condition generated based on the user query are matched in path semantics and evaluated in context awareness rules, to obtain the business attribute score; the key entities of the user query and the candidate document block are extracted and the relationship tree expansion is performed to generate an evidence subgraph, and the fact path score is obtained by analyzing the internal topological structure and semantic coherence of the evidence subgraph; the comprehensive relevance score is generated by dynamically fusing the semantic relevance score, the business attribute score and the fact path score, corresponding to the comprehensive scoring module of the system, specifically including: Referring to Figure 3 , the system will score each document block in the initial candidate set, such as document block A and document block B.

[0025] Document block A: from "Q1 Semiconductor Technology Frontier Insight Report", metadata is [Author: "Zhang San", Team: "Chief Analyst Team", Date: "03-15", Type: "Industry Depth Report", Secret Level: "Internal Public"].

[0026] Document block B: from "Company B Analysis Report", metadata is [Author: "Li Si", Team: "Ordinary Analyst Team", Date: "04-20", Type: "Company Dynamics", Secret Level: "Internal Public"].

[0027] The system generates weighted token vectors for the query and document blocks A and B. For example, "technology research report" in the query and "technology frontier insight" in document block A are similar in context, and the system generates a similar context transformation matrix for them, The context transformation matrix is generated in the following manner: firstly, the query word and the vectors of the two word before and after it are extracted and spliced to form a combined vector as the input of the network; the input is sent to a pre-trained transformation matrix generation network, which is a feedforward neural network, the input layer of which receives the combined vector formed by splicing the embedding vectors of the query word and the two words before and after it, contains two hidden layers using ReLU activation function, and the output layer is a square matrix with the same dimension as the word vector, which is the transformation matrix representing the local context semantics. The pre-training goal of the transformation matrix generation network is to make the transformation matrix generated by the network reduce the deviation degree of the query word vector direction when applied to the relevant document word, and increase the deviation degree of the query word vector direction when applied to the irrelevant document word. Specifically, a triplet loss function is used for optimization, for a query word, the positive example comes from the corresponding word in the artificially confirmed relevant document, and the negative example is randomly sampled from the irrelevant document; ensure that there is a preset minimum gap between the similarity score of the relevant pair and the similarity score of the irrelevant pair, which can be set to 0.2.

[0028] The system precomputes the inverse document frequency for all words in the knowledge base and constructs an inverse document frequency dictionary, which is constructed by statistical analysis of the content of all documents in the internal knowledge base of the organization A. When calculating the similarity, the system assigns the normalized inverse document frequency value of each word as the importance weight, specifically, the inverse document frequency value is linearly mapped to a preset weight interval, for example, [0.1, 1.5], the importance weight of the word "semiconductor" can be calculated as 1.4, and the importance weights of the words "of" and "about" can be calculated as 0.1 and 0.3 respectively; the transformed document word vector is obtained by matrix multiplication of the generated transformation matrix and the original document word vector. The system first calculates the weighted similarity of the query word and the transformed document word to form a similarity score matrix, then sums all the scores in the matrix and normalizes them by the maximum and minimum value method to obtain the final aggregation result. For example, the semantic relevance score S_sem(A) is 0.92 and S_sem(B) is 0.85. The late interaction architecture ensures millisecond-level response, and the bidirectional token weighting and context transformation focus on core words, eliminate semantic ambiguity, and improve the accuracy and interpretability of semantic matching.

[0029] For user query analysis of constraint conditions: type: "technical research report", team: "chief analyst team", date: (01-01, 06-30). The system pre-constructs a metadata ontology graph, which is a knowledge base defining the enterprise internal metadata standard and hierarchical relationship, and the core node types include "document type", "release department", "author job level", and the relationship type defines the subordinate or classification relationship between them. For example, in the document type, "industry depth report" is a subclass of "research report"; in the analyst, "chief analyst Zhang San" belongs to "chief analyst team".

[0030] For document block A, the system calculates the business attribute score by the following steps: First, hierarchical relationship matching: in the metadata ontology graph, the [type: "industry depth report"] of the document block A is a subclass of the constraint [type: "technical research report"], and the path distance is 1; [team: "chief analyst team"] is completely matched with the constraint; the text description of each node and edge in the relationship path is converted into its corresponding embedded vector combination into a vector sequence, the vector sequence is input into a two-layer structure of bidirectional long short-term memory network, and the final hidden state is taken as the "path semantic vector" representing the semantic of the whole path, and the cosine similarity of the vector and the query word vector is calculated; finally, the hierarchical relationship score is 0.95.

[0031] Second, context-aware rule evaluation is performed: the date "03-15" of document chunk A falls completely within the constraint time interval, and since the query contains "latest", a preset rule mapping table is triggered to tighten the "score half-life" parameter from the default 30 days to 15 days; the rule mapping table maps keywords in the query to parameter adjustments of the timeliness membership function. The rule mapping table specifically defines the logic of keywords and parameter adjustments, for example, the "latest" "recent" keyword will shorten the half-life by multiplying it by 0.5, and the "urgent" "emergency" keyword will multiply the half-life by 0.2; the "review" "archive" keyword will relax the half-life by multiplying it by 2.0. The membership function is a smooth decay curve, and the data output range of the function is 0 to 100 points, and the "score half-life" is the time required for the score to decay from 100 points to 50 points, and the decay mode of the curve can be precisely defined: the full score period after the document is published is maintained at full score, and the length of the period is set to maintain a fixed proportional relationship with the currently effective "score half-life" parameter, for example, set to one-third of the half-life length; after the full score period, the score starts to decline smoothly with time, and the smooth decline curve is completely constrained and determined by the scores at two known key time points: one is at the end of the "full score period", the score must be full (1.0); the second is at the "score half-life" specified day, the score must be half (0.5); when the interval between the publication time and the query time is within the "full score period", the rule satisfaction score will be directly assigned to full score 1.0. Finally, the hierarchical relationship score and the rule satisfaction score are multiplied, and the business attribute score is finally 0.95.

[0032] For document chunk B, hierarchical relationship matching and context-aware rule evaluation are performed. Hierarchical relationship matching: the [team: "ordinary analyst team"] of document chunk B does not match the constraint, and the hierarchical relationship score is calculated as 0.3. Context-aware rule evaluation: the date "04-20" of document B falls completely within the constraint time interval, and since the query contains "latest", the dynamically adjusted fuzzy logic boundary gives a rule satisfaction score of 1.0. Using the product operation, the final B business attribute score is 0.3. By constructing a metadata ontology graph and performing path semantic matching and context-aware rule evaluation, abstract business rules are converted into computable scores, enabling the system to accurately perform complex compliance screening and ensuring the compliance of the final answer.

[0033] The system pre-constructs an enterprise knowledge graph, which is automatically constructed by using entity and relationship extraction technology from public financial reports, industry news and internal reports. The knowledge graph contains entity types and relationship types, for example, entity types can include "company", "financial product", "top management", and relationship types can include "investment relationship", "competitive relationship", "position relationship". Starting from the query entity "Company B", a relationship tree expansion operation with a depth of 3 is performed in the enterprise knowledge graph, the expansion adopts a breadth-first search strategy, and preferentially expands relationship types related to application scenarios, such as "investment", "shareholding", and "top management position", and the execution depth is set to an optimized value that balances the capture of multiple layers of indirect relationships and control of computational complexity, and finally generates an evidence subgraph containing "industry", "core product", "main executives", and "competitors".

[0034] Multi-factor scoring is performed on document block A: Document A not only contains "Company B", but also mentions the core product "Chip X" and the competitor "Company C", all of which are in the evidence subgraph and in the core position.

[0035] The structural density score is obtained by calculating the ratio of the actual number of edges in the evidence subgraph to the maximum number of edges that can theoretically exist and normalizing it, for example, the structural density score of document block A is calculated as 0.8. The core node contribution score is obtained by calculating the betweenness centrality of the document entity in the evidence subgraph and normalizing it, the betweenness centrality measures the frequency of a node appearing as a "bridge" in the shortest path between other node pairs, the system calculates the betweenness centrality of the entity nodes in document block A, and takes the maximum value as the core node contribution score, for example, the core node contribution score of document block A is calculated as 0.9. The path consistency score is obtained by encoding all paths connecting the query entity and the document entity in the subgraph into multiple path semantic vectors using the aforementioned bidirectional long short-term memory network, and then calculating the average pairwise cosine similarity between these vectors, the path consistency score of document block A is calculated as 0.95. The three scores are fused to generate a fact path score of document block A, which is calculated as 0.86, and the fusion is achieved by a preset weighted sum logic, wherein the weight of each component is obtained by grid search optimization, and the goal of the grid search optimization is to maximize the normalized discounted cumulative gain index of the final ranking result on the reserved verification set with manually annotated relevance levels. For document block B, only the "company B" entity is mentioned, and its fact path score is lower than that of A. By constructing an evidence subgraph and analyzing its internal structure, core nodes and path consistency, the fact evaluation is changed from linear path search to evidence network analysis, which improves the mining ability of non-obvious associations between entities and improves the robustness of fact evaluation and the reliability of the final answer.

[0036] In the process of deriving the fact path score, metadata attributes including a timestamp and a source level are added in advance to each relationship edge of the enterprise knowledge graph; the timestamp is time information extracted based on business metadata, and the source level is a level division preset according to the authority of the original document; the timestamp and the source level of each relationship edge of the path are used as weighting factors to modulate the path consistency score, and the final fact path score is calculated. By integrating the timeliness of the fact and the authority of the source into the scoring model, deep dynamic evaluation of the evidence quality can be realized, and the accuracy and reliability of the finally generated answer are improved.

[0037] The user query is analyzed to identify that the query intent includes both business constraints (chief analyst) and fact queries (company B), and a gating weighted network is dynamically generated to generate weights, such as "W_sem: 0.2, W_biz: 0.5, W_fact: 0.3". The gating weighted network is a lightweight feedforward neural network, the input of which is an overall semantic vector generated by an encoder model from the original user query text, the network includes one hidden layer, and the output layer uses a Softmax activation function to output weight values whose sum is 1. The network and the final generation model are trained end-to-end, and the core training mechanism is to use a clearly defined weighted combination loss function to convert the answer quality into a calculable supervised signal to adjust the weights in reverse; the weighted combination loss function consists of two parts: The system pre-constructs a training dataset with multiple levels of relevance annotations. The training dataset is composed of query instances, which include: user query text; a list of candidate document blocks generated by the document recall module for the query; and relevance level annotations given by business experts within the enterprise to the document blocks in the list and related to the query. The relevance level is divided into five grades: "perfect" (can directly and completely answer the query), "excellent" (contains core answers but requires a small amount of reasoning), "good" (related content but not core), "general" (theme related but no answer), and "irrelevant".

[0038] Generation loss: This part adopts the standard cross-entropy loss to evaluate the accuracy and fluency of the generated content by comparing the differences between the model-generated answers and the standard reference answers. This loss function is mainly used to optimize the downstream generation model to ensure that it outputs high-quality text.

[0039] Ranking loss: This part adopts the list-wise Softmax cross-entropy ranking loss to optimize. For a list containing multiple candidate document blocks, the system generates a "predicted probability distribution" based on the comprehensive relevance scores of each document block through the Softmax function. Based on the training data, an ideal probability distribution is constructed, in which the target probability of the candidate document block is set according to its relevance level with the query's artificial annotation. The five levels of relevance from "perfect" to "irrelevant" are mapped to non-negative scores, such as 4, 3, 2, 1, and 0. The non-negative scores are normalized by Softmax to generate a smoother ideal probability distribution. The goal of the training process is to minimize the differences between the "predicted probability distribution" and the "ideal probability distribution" by adjusting the internal parameters of the model.

[0040] The two parts of the loss are weighted and summed with preset weights to generate the final supervision signal. The generation loss weight can be preset as 0.6, and the ranking loss weight can be preset as 0.4. By backpropagating the gradient signal to the gating network, it can learn how to dynamically allocate weights according to user intent. When the query is biased towards seeking specific facts, the weight of the fact path score is increased; when the query contains strict business filtering conditions, the weight of the business attribute score is increased.

[0041] The semantic relevance score, business attribute score, and fact path score are weighted and summed according to the weights to generate a preliminary comprehensive score, for example, the preliminary comprehensive score of document block A is calculated as 0.917. The score distribution is analyzed for confidence calibration, for example, the "0.92, 0.95, 0.86" distribution of document block A is highly consistent, and the score confidence is C(A)=0.98. The score confidence quantifies the consistency of the distribution by calculating the statistical standard deviation of the three input scores. Specifically, the standard deviation is normalized, and the normalized value is subtracted from 1 to obtain the confidence score representing consistency. The comprehensive relevance score is the product of the preliminary comprehensive score and the score confidence, for example, the comprehensive relevance score of document block A can be 0.899. Through intent gating weighting, the score is dynamically focused, and the score confidence is used to calibrate the result, improving the response effect of the comprehensive relevance score to user query intent and improving the decision-making accuracy of handling multi-dimensional conflicting information.

[0042] In the process of generating the comprehensive relevance score, an adaptive weight recalibration step is performed, specifically comprising: comparing the business attribute score with a preset veto threshold; when the business attribute score is lower than the veto threshold, assigning the comprehensive relevance score of the candidate document block as a preset invalid value; when the business attribute score is not lower than the veto threshold, the system calculates the distribution consistency of the semantic relevance score, the business attribute score and the factual path score to obtain a score confidence; if the score confidence is lower than a preset confidence threshold, identifying an outlier score item and applying a preset punitive attenuation to the weight coefficient corresponding to the outlier score item; re-distributing the weight values reduced due to attenuation to the remaining score items according to the original weight proportion to generate the recalibrated weight coefficient. By dynamically suppressing outlier items and enhancing the weight of consensus items, the comprehensive score is effectively avoided from being misled by a single abnormal score in most conventional query scenarios, and the robustness of the scoring system and the reliability of the final decision are improved.

[0043] Further, the candidate document blocks are optimized and reordered based on the preset penalty mechanism and the comprehensive relevance score to generate an optimized context set; the answer is output based on the optimized context set calling the generation model, corresponding to the ranking output module of the system, specifically comprising: Referring to Figure 4 All 50 candidate document blocks are initially ranked according to their final comprehensive scores, for example, document block A ranks first. An empty optimized context set is set, and the target size can be 3.

[0044] Iterative selection operation is performed: First iteration: select the document block A with the highest score and add it to the optimized context set.

[0045] Second iteration: assuming that the document block D initially ranked second has a comprehensive relevance score of 0.850, and its content has a semantic vector cosine similarity of 0.95 with A, the system calculates a redundancy penalty score for it; the comprehensive relevance score of the document block E ranked third is 0.83, and its content is complementary to A with a semantic vector cosine similarity of 0.20; after adjustment, the dynamic candidate score of E is greater than that of D, and the adjustment is realized through multiplication attenuation logic: the dynamic candidate score is equal to the original comprehensive score multiplied by the coefficient after redundancy penalty score attenuation, and the redundancy penalty score itself is obtained by calculating the maximum value of the semantic vector cosine similarity between the current candidate document block and the selected document block, and the attenuation coefficient is equal to 1 minus the product of the redundancy penalty score and a preset penalty factor, and the preset penalty factor is between 0 and 1, and can be set to 0.7 based on expert experience; after calculation, the dynamic score of D is 0.285, and the dynamic score of E is 0.714; document block E is selected according to the dynamic candidate score of E and added to the optimized context set.

[0046] Third iteration: Repeat the above process to select the next document block that is least redundant to both A and E and has the highest score.

[0047] In the process of generating the redundancy penalty score, the candidate document block to be selected is combined with the selected document blocks in the optimization context set as a document pair, the document pair is input into the natural language inference model, and the logical relationship between the candidate document block and the selected document block is output, the logical relationship including a contradiction relationship and an implication relationship; when the logical relationship is the contradiction relationship, the candidate score of the candidate document block is set to 0; when the logical relationship is the implication relationship, the semantic overlap between the candidate document block and the selected document block is considered as the maximum value in the calculation of the redundancy penalty score. By introducing the judgment at the logical level, the documents with contradictory content or contained information are actively excluded, ensuring the logical self-consistency and information density of the final optimization context set, and improving the rigor and reliability of the generated answer.

[0048] After the iteration, the optimization context set containing 3 document blocks with the highest information density and no redundancy is submitted to the downstream generation model together with the original query to generate the final answer with clear organization. Through the iterative optimization and redundancy penalty, the sorting target is changed from selecting the optimal individual to constructing the optimal combination, improving the logical coherence and information density of the final generated answer, and improving the reliability of the answer.

[0049] The present application solves the problems of semantic deviation, missing business rules and insufficient factual basis caused by single-dimensional retrieval by constructing a complete system architecture including document recall, multi-dimensional scoring and optimization sorting, ensuring the accuracy of the semantic of the finally generated answer, the compliance of the business and the reliability of the facts.

[0050] Embodiment two This embodiment takes the financial institution in embodiment one as the application background, and the user proposes a query: "Find all internal operation guidelines published by the compliance department after the first quarter, related to 'internal risk control', and with a confidentiality level of 'confidential'." After receiving the query, the system first recalls an initial candidate set containing multiple candidate document blocks through the document recall module. Then, the comprehensive scoring module scores each candidate document block: Candidate document block X: derived from "internal risk control operation details", and the associated business metadata is [publishing department: "compliance department", publishing date: "05-20", document type: "internal operation guide", confidentiality level: "confidential"].

[0051] Candidate document block Y: derived from "latest market risk analysis", and the business metadata is [publishing department: "market research department", publishing date: "06-10", document type: "industry analysis", confidentiality level: "internal disclosure"].

[0052] Candidate document block Z: derived from "Customer Identity Recognition Guidelines", with business metadata [publishing department: "Compliance Department", publishing date: "04-15", document type: "Internal Operating Guidelines", classified level: "Internal Disclosure"].

[0053] The system accurately parses the user's natural language query into a set of structured business constraints through a language model fine-tuned based on a sequence labeling task: [content keyword: "internal risk control", document type: "internal operating guidelines", publishing department: "compliance department", publishing date: "greater than 03-31", classified level: "confidential"].

[0054] For document block X: its "publishing department" and "document type" fully match the constraints, and the path distance on the preset metadata ontology graph is zero, so the hierarchical relationship score is calculated as 1.0; its "publishing date" and "classified level" also fully meet the constraint conditions, so the rule satisfaction score is calculated as 1.0. Finally, through the product operation, its business attribute score is 1.0.

[0055] For document block Y: its "publishing department" (Market Research Department) and "document type" (Industry Analysis) do not meet the constraint conditions, its hierarchical relationship score is calculated as 0.1, and its business attribute score is correspondingly calculated as 0.1.

[0056] For document block Z: its "publishing department" and "document type" both meet the constraints, and the hierarchical relationship score is calculated as 0.9. However, when performing context-aware rule evaluation, the system finds that its "classified level" does not meet the rigid constraint "classified level must be 'confidential'" in the constraint condition, so its rule satisfaction score is directly determined as 0. Through the product operation, its final business attribute score is 0.

[0057] Although the semantic relevance score of document block Z can reach 0.95, due to the action of the "veto rule", its business attribute score is set to 0, and its final comprehensive relevance score tends to 0 after dynamic fusion. While document block X has each sub-score in the semantic, business, and factual levels above 0.9, the generated comprehensive relevance score is greater than that of document block Z. Through path semantic matching to achieve deep understanding of category constraints, and then using context-aware rule evaluation for dynamic adaptation of numerical constraints, the intelligence and accuracy of business attribute scoring are improved, and the compliance and applicability of the final answer are improved.

[0058] Embodiment Three This embodiment takes the financial institution in Embodiment One as the application background, and the user proposes the query: "Analyze the possible non-direct association between 'Company B' and 'Capital Y', such as through common holding companies or key executive job histories." The system extracts the query entities "Company B, Capital Y" from the query. The system performs a "relation tree expansion" operation with a depth of 3 in the pre-built enterprise knowledge graph with these two entities as root nodes, generating an "evidence subgraph", for example, the "evidence subgraph" contains 25 entity nodes and 40 relation edges. There is a fact in the knowledge graph that "Y Capital" invests in "Z Technology", "Manager A" is the co-founder of "Company B", and "Manager A" is the director of "Z Technology". At this time, the evidence subgraph constructs a non-direct path connecting "Y Capital" and "Company B".

[0059] During this process, the document recall module recalls a candidate document K, which mentions "Manager A, the board member of Z Technology, was the co-founder of Company B". The system will score the candidate document K with multiple factors as follows: Since the evidence subgraph connects the two originally isolated query entities through the "Z Technology" and "Manager A" nodes to form a path and a closed loop, the average node degree reaches 3.2, and after normalization calculation, the structural density score is 0.85.

[0060] Core node contribution score: The system calculates the network centrality of the two entities "Manager A, Z Technology" in the evidence subgraph extracted from the candidate document K. It is found that the betweenness centrality of the "Manager A" node is 0.48 and the betweenness centrality of the "Z Technology" node is 0.35, both of which are key bridge nodes in the subgraph. After aggregation and normalization calculation, the core node contribution score is 0.95.

[0061] Path consistency score: In the evidence subgraph, there are three key paths connecting the query entities and the document entity "Manager A". After encoding these three paths into path semantic vectors, the average pairwise cosine similarity between them is calculated to be 0.88, indicating that the evidence chain focuses on the theme of "investment and managerial position". Therefore, the path consistency score is 0.90.

[0062] Although the text content of candidate document K can not have both "Company B" and "Y Capital" appearing at the same time, its semantic relevance score can only be 0.45; but since the structure density score is 0.85, the core node contribution score is 0.95, and the path consistency score is 0.90, the fact path score of candidate document K can be calculated as 0.91, which is obtained by weighted summation of the three sub-scores, for example, the core node contribution score weight is 0.5, the path consistency score weight is 0.3, and the structure density score weight is 0.2. In the final dynamic fusion, this fact path score (0.91) will significantly improve the comprehensive ranking of candidate document K. By constructing a comprehensive evidence subgraph through relationship tree expansion and then using a multi-factor model to quantitatively evaluate the internal quality of the evidence subgraph, the depth and robustness of fact evaluation are improved, and the fact reliability of the final answer is improved.

[0063] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. The enterprise RAG intelligent question answering system based on vector database reordering is characterized by: include: The document recall module receives user queries and searches the preset vector database based on the user queries to obtain candidate document blocks containing business metadata; The comprehensive scoring module uses a late interaction architecture based on bidirectional token importance weighting to perform similarity aggregation operations on the pre-encoded document word meta-vectors and query word meta-vectors to calculate the semantic relevance score; Based on the preset metadata ontology graph, the business attribute score is obtained by performing path semantic matching and context-aware rule evaluation on the business metadata and the business constraints generated based on the user query; Extract key entities from user queries and candidate document blocks and perform relationship tree expansion to generate an evidence subgraph. Analyze the intrinsic topological structure and semantic coherence of the evidence subgraph to derive a fact path score. Dynamically fuse the semantic relevance score, business attribute score, and fact path score to generate a comprehensive relevance score. The sorting output module optimizes and re-ranks the candidate document blocks based on the preset penalty mechanism and comprehensive relevance scores to generate an optimized context set; The generative model is called based on the optimized context set to output the answer.

2. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1 is characterized in that: The document recall module includes: Performing text preprocessing on user queries, including identifying key terms in the user queries and performing query error correction to generate optimized query text; decomposing the optimized query text into retrieval intent subtasks, converting the retrieval intent subtasks into corresponding specialized query vectors through an embedding model, and combining the specialized query vectors to form a heterogeneous query vector; A similarity search is performed in a preset vector database using the heterogeneous query vector to obtain a candidate result set, and the candidate result set is fused to generate a candidate document block containing business metadata.

3. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1 is characterized in that: Calculating the semantic relevance score includes: Input the candidate document chunk and the user query into a preset bidirectional encoder representation model to generate corresponding document word meta-vectors and query word meta-vectors; calculate and store the document-side importance weight corresponding to the document word meta-vector based on the word frequency of the document word meta-vector in the candidate document chunk and the inverse document frequency in a pre-built inverse document frequency dictionary; Based on the inverse document frequency dictionary, searching for and assigning corresponding query-end importance weights to the query term meta-vectors; A context transformation matrix is ​​generated for the local context of the query word element vector, and the pre-encoded document word element vector is transformed using the context transformation matrix to obtain a transformed document word element vector; the cosine similarity between the query word element vector and the transformed document word element vector is calculated, and the cosine similarity is weightedly aggregated using the query-side importance weight and the document-side importance weight to generate a semantic relevance score.

4. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1 is characterized in that: Determining the business attribute score includes: Parsing the user query using a natural language understanding model to extract business constraints including categorical constraints and numerical constraints; performing path semantic matching on the categorical constraints, wherein the path semantic matching includes searching for a relationship path between the categorical constraints and the business metadata based on a preset metadata ontology graph; Converting the relationship path into a path semantic vector, calculating the similarity between the path semantic vector and the query term vector, and generating a hierarchical relationship score; performing context-aware rule evaluation on the numerical constraint, wherein the context-aware rule evaluation includes mapping the business metadata and the numerical values ​​in the constraint condition to a preset fuzzy logic set, dynamically adjusting the membership boundary of the fuzzy logic set based on the context of the user query, and calculating a rule satisfaction score based on the adjusted membership; The hierarchical relationship score is multiplied by the rule satisfaction score to generate a business attribute score.

5. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1 is characterized in that: Determining the fact path score includes: Extract query entities and document entities based on user queries and candidate document blocks; use the query entity as the root node of a pre-built enterprise knowledge graph, perform a relationship tree expansion operation with a preset depth to generate an evidence subgraph; calculate the structural density score, core node contribution score, and path consistency score by analyzing the intrinsic topological structure and semantic coherence of the evidence subgraph; The structural density score is obtained by calculating the degree of node interconnection in the evidence subgraph, the core node contribution score is obtained by calculating the network centrality of the document entity as a node in the evidence subgraph, and the path consistency score is obtained by encoding the relationship path connecting the query entity and the document entity in the evidence subgraph into a path semantic vector and calculating the average pairwise similarity of the path semantic vector; the structural density score, the core node contribution score and the path consistency score are fused to generate a fact path score.

6. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1 is characterized in that: Generating a comprehensive relevance score comprises: By analyzing the user query, the user's main intention is identified. Based on the main intention, a weight coefficient of the semantic relevance score, the business attribute score and the fact path score is dynamically generated through a gated weighted network. The semantic relevance score, the business attribute score and the fact path score are fused based on the weight coefficient to generate a preliminary comprehensive score; The distribution consistency among the semantic relevance score, the business attribute score and the fact path score is calculated to obtain a score confidence; and the preliminary comprehensive score is calibrated and adjusted using the score confidence to generate a final comprehensive relevance score.

7. The enterprise RAG intelligent question-answering system based on vector database reordering according to claim 1 is characterized in that: The sorting output module includes: The candidate document blocks are initially sorted based on the comprehensive relevance scores, and an empty optimization context set is set; the selection step is executed cyclically until the number of document blocks in the optimization context set reaches a preset threshold, and the selection step includes: for candidate document blocks that have not yet been selected into the optimization context set, calculating the semantic overlap between the candidate document blocks and the document blocks already included in the optimization context set to generate a redundancy penalty score; using the redundancy penalty score to adjust the comprehensive relevance score of the candidate document block to obtain a candidate score; and selecting a candidate document block based on the candidate score and adding it to the optimization context set.

Images