A modular RAG system and method supporting implicit space intent routing and multi-granularity dynamic context injection

Abstract

This invention relates to the field of modular RAG processing technology, and discloses a modular RAG system and method supporting latent space intent routing and multi-granularity dynamic context injection, including the following steps: S1, in the offline database entry stage, the document is parsed into a graph structure with multiple granular levels and parent pointers between levels, and the content of fine-grained nodes and the parent pointers are stored; S2, in the retrieval stage, the continuous latent states output by the early layers of the forward propagation of the generative model are intercepted, and they are decomposed into multiple parallel sub-intent query representations, etc. This invention innovatively uses a graph-based "parent-child pointer mapping" to replace physical text writing. Only the plain text of fine-grained slices and their hierarchical pointers are stored in the base database. When a retrieval hit occurs, the corresponding macro-level paragraph background and global summary are dynamically traced and assembled through pointer queries with extremely low time complexity.

Description

Technical Field

[0001] This invention belongs to the field of modular RAG processing technology, and more specifically relates to a modular RAG system and method that supports latent space intent routing and multi-granularity dynamic context injection. Background Technology

[0002] With the widespread application of large language models in question answering for private knowledge bases in vertical domains, the Retrieval Augmentation (RAG) paradigm is evolving from "naive single retrieval" to complex "modular RAG". However, existing cutting-edge RAG systems and agent architectures still have inherent technical bottlenecks when facing enterprise-level production environments and deep cross-document logical connections.

[0003] Overall, existing large models have limitations when processing complex private knowledge bases, mainly due to the following issues:

[0004] (1) The problem of context fragmentation and storage redundancy caused by static segmentation and forced splicing

[0005] During the knowledge base construction phase, splitting long documents into flat data chunks disrupts the global context logic, leading to serious issues such as unclear pronoun references or missing subjects, which greatly reduces the accuracy of vector matching. This not only causes the model to exhibit local cognitive limitations during generation, but also results in uncontrollable increases in storage costs for existing solutions that attempt to supplement the context.

[0006] (2) High latency and discrete space error accumulation problem in multi-hop retrieval in complex cross-document reasoning

[0007] When users pose complex, multi-hop questions that require traversing multiple documents or logical nodes to answer, traditional intelligent agents rely heavily on discrete natural language spaces for reasoning (i.e., large models first autoregressively generate a "thinking text" and then use it for retrieval). This pattern of repeatedly generating text in the natural language dimension has two fatal flaws: first, autoregressive generation causes the system response latency to increase exponentially, making it impossible to meet real-time interaction needs; second, it is extremely prone to error accumulation, where small misunderstandings in early text generation can be amplified in subsequent rounds of queries, leading to the collapse of the retrieval chain.

[0008] (3) The problem of black-box credulity and factual illusion caused by the lack of independent verification loop.

[0009] Although existing retrieval enhancement generation systems have introduced some reflective mechanisms, most of them tightly couple the retrieval quality evaluation process with the final generation process of the large model. If the retrieval recalls noisy fragments containing erroneous facts and outdated indicators, the system, lacking a hard dynamic verification and blocking mechanism independent of the black box model, will often blindly believe these abnormal data, thus inevitably conforming to the spread of erroneous information and outputting serious illusions that violate objective facts. Summary of the Invention

[0010] This invention proposes a modular RAG system and method that supports latent space intent routing and multi-granularity dynamic context injection, aiming to solve the problems of context fragmentation and storage redundancy caused by static segmentation and forced splicing in the prior art.

[0011] To achieve the above objectives, the basic solution of this invention provides a modular RAG method that supports latent space intent routing and multi-granularity dynamic context injection, comprising the following steps:

[0012] S1. During the offline data entry stage, the document is parsed into a graph structure with multiple granular levels and parent pointers between levels, and the content of fine-grained nodes and the parent pointers are stored.

[0013] S2. During the retrieval phase, the continuous hidden states output by the early layers of the forward propagation of the generative model are intercepted and decomposed into multiple parallel sub-intent query representations.

[0014] S3. The graph structure is retrieved concurrently using the multiple sub-intent query representations, and the fine-grained nodes that are found in the retrieval are cascaded and assembled along the parent pointer to obtain a multi-granularity context.

[0015] S4. A verification unit, independent of the generative model, calculates the matching confidence between the multi-granularity context and the query before the generative model is input into the generative model.

[0016] S5. Perform three-state control based on the matching confidence: when the high threshold is reached, allow the generation model to generate an answer; when the low threshold is reached, block the input; when the two thresholds are between, extract the difference representation between the query and the context, and send it back to the mapping decomposition step as a feedback signal to trigger a targeted secondary retrieval for missing features.

[0017] In one possible design, during step S1, when the document is loaded into the database offline, the gateway parses the long document into a directed tree graph according to its logical structure. vertex set Includes document-level nodes Paragraph-level nodes and sentence-level slicing edge set A function for pointing to parent nodes from bottom to top. ,Right now , The system calls the lightweight model for and Generate global summary When storing data, do not write the summary. The text is stored only in the vector database. Plain text and its pointers .

[0018] In one possible design, in step S2, when the user queries the large language model Q, the orchestrator intercepts the query in the early forward propagation layer of the Transformer. Let the number of truncation layers be k, and extract the continuous hidden state feature matrix from its output. :

[0019] ;

[0020] Will via pre-trained lightweight projection network Perform multi-head dimensionality reduction mapping to divide it into n parallel sub-intent latent vectors in a continuous high-dimensional mathematical space. : .

[0021] In one possible design, in step S3, the sub-intent latent vector set By bypassing the time-consuming natural language decoder, the query vector is sent concurrently to the vector database as an independent query vector for multi-path retrieval. Assume the user query hits the leaf node set. For any hit slice Execute using pointers Complex dynamic context assembly, ultimately feeding the complete context to the model. Defined as:

[0022]

[0023] in, This represents the n sub-intent hidden vectors extracted from the early hidden states via the projection network. Represents the parent pointer function: slice The paragraph-level parent node, This refers to tracing back to the document-level parent node. This represents the pre-generated summary content for the node. Plain text representing slice x itself. This indicates a sequence concatenation operation.

[0024] In one possible design, step S4 introduces a cross-attention scorer independent of the generative model, calculating the matching confidence score before the recall context C is fed into the generative model. :

[0025]

[0026] in, These represent the vector representation of the user query and the vector representation of the retrieved and assembled context, respectively.

[0027] In one possible design, step S5 involves setting a high confidence threshold. With low confidence threshold System control flow Defined as a piecewise function:

[0028]

[0029] Among them, if the confidence score matches Triggering hard blocking Termination will recall context Input the generated model, do not perform fallback generation without retrieval, and return a rejection result to the user, indicating that the current retrieval results are insufficient to support a reliable answer;

[0030] If the score falls into the critical range, it indicates that the context is relevant but information is missing, and the critical engine will calculate accordingly. and Difference vector ,Will As a negative feedback compensation signal, it is sent back to the editor to trigger a targeted secondary search based on the missing features;

[0031] in, This indicates that the query side can learn a linear projection matrix. Indicates query-side embedding. This indicates that the context side can learn linear projection matrices. This indicates context-side embedding.

[0032] In one possible design, during step S3, when performing multi-way retrieval, the time overhead of multi-hop retrieval is collapsed into a single forward propagation and concurrent query:

[0033] ;

[0034] The present invention also provides a system including a storage component and a processing unit, wherein the storage component stores machine-executable instructions that can be executed by the processing unit, and the processing unit executes the machine-executable instructions to implement the method as described above.

[0035] The present invention also provides a machine-readable storage medium comprising a computer-readable storage carrier having program code storage functionality, wherein the program code carried by the carrier contains executable instructions for the method as described above.

[0036] The beneficial effects of this invention include:

[0037] This invention innovatively employs a graph-based "parent-child pointer mapping (Parent_ID)" to replace physical text writing. Only fine-grained slices of plain text and their hierarchical pointers are stored in the base database. When a retrieval hit occurs, the corresponding macro-level paragraph background and global summary (i.e., Cx=Summ(P(P(x)))⊕Summ(P(x))⊕Text(x)) are dynamically traced and assembled through pointer lookup with extremely low time complexity (i.e., Cx=Summ(P(P(x)))⊕Summ(P(x))⊕Text(x))) are assembled, achieving a "storage-computation separation" context processing paradigm. This reduces storage costs while restoring the context. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a system architecture diagram of a modular RAG method that supports latent space intent routing and multi-granular dynamic context injection. Detailed Implementation

[0040] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0041] (1) Regarding the problem of "fragmented context":

[0042] How existing technologies work (refer to ContextualRetrieval Industry Frontiers): To address the loss of context caused by data truncation, the latest existing technologies propose implementing "dynamic context injection" during data access. This means instead of simply truncating the text, a large model is used to generate a global context summary for the entire document, which is then forcibly appended to the header of each data block, attempting to ensure that the data blocks carry the global context during retrieval.

[0043] Technical drawbacks: This flattened, forced splicing scheme is extremely inefficient. First, it leads to a dramatic increase in storage redundancy in the vector database (each small slice requires a large number of repetitive document summaries); second, it fails to establish a true "parent-child document" graph cascading relationship. Even when a fine-grained fragment is found during retrieval, the system still cannot dynamically expand outwards or trace back its complete logical chain based on structured routing.

[0044] (2) Regarding the problem of "multi-hop retrieval delay and error":

[0045] How existing technologies work (refer to the ModularRAG review and traditional agent architectures): For complex problems, existing technologies introduce a "rewrite-retrieve-read" framework. The system allows a large model to act as an agent, "guessing" the direction of the answer before retrieval, generating rewritten query statements or intermediate thought text in a discrete natural language space, and then initiating the retrieval step by step.

[0046] Technical drawbacks: As revealed by the latest LatentRAG research, this scheme, which relies on intermediate states of natural language, significantly slows down inference. Each retrieval must wait for the time-consuming autoregressive generation to complete, and the transformation of this discrete text is extremely fragile. Once a slight semantic deviation occurs during the intermediate thinking process, it will become a definite input contamination of subsequent iterations, completely lacking the ability to perform continuous vector correction in a single forward propagation.

[0047] (3) Regarding the issue of "model credulity and illusion":

[0048] How existing technologies work (referencing self-reflective mechanisms like eRAG): To achieve dynamic verification, existing technologies attempt to introduce "reflection" and "critique" mechanisms. By fine-tuning a large model, the model autonomously predicts and outputs confidence identifiers (such as "supportive," "irrelevant," etc.) while reading search results and generating answers, in order to judge the quality of the search results.

[0049] Technical drawbacks: This "internal dynamic verification" scheme forcibly binds the critique mechanism and the generation module within the same large model. Because the large model itself has a tendency to produce illusions, making it "both the generator and the judge" easily leads to erroneously high scores due to blind confidence. Existing technologies lack an independent pre-evaluation engine, making it impossible to achieve strict hard blocking before generation, nor can they utilize confidence levels to dynamically trigger and guide precise secondary compensation retrieval.

[0050] Example:

[0051] like Figure 1 As shown, this embodiment of the invention provides a modular RAG method that supports latent space intent routing and multi-granularity dynamic context injection, including the following steps:

[0052] S1. During the offline data entry stage, the document is parsed into a graph structure with multiple granular levels and parent pointers between levels, and the content of fine-grained nodes and the parent pointers are stored.

[0053] S2. During the retrieval phase, the continuous hidden states output by the early layers of the forward propagation of the generative model are intercepted and decomposed into multiple parallel sub-intent query representations.

[0054] S3. The graph structure is retrieved concurrently using the multiple sub-intent query representations, and the fine-grained nodes that are found in the retrieval are cascaded and assembled along the parent pointer to obtain a multi-granularity context.

[0055] S4. A verification unit, independent of the generative model, calculates the matching confidence between the multi-granularity context and the query before the generative model is input into the generative model.

[0056] S5. Perform three-state control based on the matching confidence: when the high threshold is reached, allow the generation model to generate an answer; when the low threshold is reached, block the input; when the two thresholds are between, extract the difference representation between the query and the context, and send it back to the mapping decomposition step as a feedback signal to trigger a targeted secondary retrieval for missing features.

[0057] In step S1, during offline data import, the gateway parses the long document into a directed tree graph according to its logical structure. vertex set Includes document-level nodes Paragraph-level nodes and sentence-level slicing edge set A function for pointing to parent nodes from bottom to top. ,Right now , The system calls the lightweight model for and Generate global summary When storing data, do not write the summary. The text is stored only in the vector database. Plain text and its pointers .

[0058] In step S2, when the user queries When the input is a large language model (decoder-only, autoregressive generative architecture) used as the main generative model, the orchestrator intercepts the input in the early forward propagation layers of the model's own Transformer stack. Let the number of interception layers be k ( , (Total number of layers in the model), only the first few layers are taken. Layer-encoded continuous hidden state feature matrix :

[0059] ;

[0060] Regarding the choice of the number of truncation layers k: Let the total number of layers in the main generator model Transformer be L, and the number of truncation layers satisfy the following: The smaller k is, the lower the forward computation overhead and the faster the retrieval starts, but... The semantic abstraction it carries is insufficient, and the intended decomposition tends to be biased towards surface-level lexical features; the larger k is, the better. The stronger the semantic sufficiency, the higher the interception latency, gradually approaching the overhead of the complete forward propagation. Therefore, the selection of k follows the latency-semantic sufficiency trade-off principle: prioritizing early to mid-early layers to ensure that the hidden state has formed a stable query semantic representation, while avoiding entering deep layers that are mainly for generating the final answer. In this invention, the total number of layers in the main generation model Transformer is L, numbered from layer 1 to layer L in sequence from the input side. Layers 1 to L... The first layer is called the early layer; the second layer is called the early layer. Layer to the first The first layer is called the early to middle layer; the second layer is called the middle to early middle layer. Layers L to L are referred to as the middle to late stages to the deep stages. The preference for early to early-middle stages means that the truncated layer number k preferentially selects layers that satisfy… The integer means that the interception range covers the early layer and extends to the mid-early layer, but does not enter the deep layer which is mainly for generating the final answer.

[0061] Based on experience, it is advisable For example, for model take In practical deployment, the optimal k can be determined by grid search within the aforementioned range using a joint metric of retrieval recall (or subsequent confidence score S) and end-to-end latency on the validation set; alternatively, it can be adaptively adjusted according to query complexity, using a smaller k for simple queries and a larger k for multi-hop complex queries, while always maintaining a consistent k. This ensures that interception occurs before the final autoregressive response is decoded.

[0062] Will via pre-trained lightweight projection network Perform multi-head dimensionality reduction mapping to divide it into n parallel sub-intent latent vectors in a continuous high-dimensional mathematical space. :

[0063] .

[0064] Regarding sub-intention projection networks The structure and training method are further explained below:

[0065] The projection network is a lightweight multi-head mapping module independent of the main generator model. The parameters of the main generator model are kept frozen. Specifically, the early hidden state feature matrix is ​​first pooled to obtain the global representation of the query; then, intermediate features are extracted by a shared multilayer perceptron backbone; finally, n parallel projection heads output n sub-intent latent vectors, whose dimensions are consistent with the slice embedding dimensions in the vector database, so that they can be directly used as query vectors for retrieval.

[0066] To ensure that the n sub-intent latent vectors have distinguishable semantic meanings and can be used for retrieval independently, rather than being meaningless dimensionality reduction segments, the projection network is trained offline according to the retrieval-oriented objective. Supervision signals are derived from pre-labeled sets of gold-standard support slices corresponding to each sub-question in multi-hop question-answering samples; if necessary, a teacher model can also decompose complex queries into several natural language sub-queries and encode them into teacher representations as auxiliary supervision. Negative samples use slices from other queries in the same batch, or non-support slices from the same document.

[0067] During training, a contrastive learning loss is applied to each projection head to bring its output sub-intent latent vector closer to its corresponding gold-standard support slice in the embedding space and further away from the negative sample slice, thus ensuring that each sub-intent can independently complete one effective retrieval. Simultaneously, an inter-head diversity constraint is introduced to suppress excessive similarity or degradation of output vectors from different projection heads into the same direction, preventing the n-way segmentation from losing its complementarity. If a teacher subquery representation is used, a distillation constraint can be further added to align each sub-intent with the semantics of interpretable sub-questions.

[0068] During the inference phase, the projection network parameters are fixed, and a forward mapping is performed only once on the captured early hidden states to obtain n sub-intent latent vectors, which are then concurrently sent to the leaf vector library for retrieval. Thus, the sub-intent decomposition is explicitly constrained by the training objective of "being able to hit complementary evidence separately," rather than a simple dimensionality split.

[0069] The number of sub-intents, n, is a fixed positive integer by default (or n=3), consistent with the number of projection heads, and is shared by offline training and online inference. It can also be adaptively set according to query complexity: the orchestrator selects from a preset set (e.g., {1,2,3}) based on the discreteness of early hidden states or a lightweight complexity estimator; for simple / single-hop queries, n=1 is used, enabling only single-head projection and without forced multi-path splitting; for multi-hop complex queries, n>1 is used, enabling multi-head concurrent retrieval. When n=1, it degenerates into single-path latent space retrieval, still projected and aligned to the library vector space.

[0070] The early hidden states of the main model have a dimension of d, and the vector library slices are encoded by independent embedding models with a dimension of d. The two are usually different. Each head of the projection network linearly (or via an MLP) maps the pooled query representation to... dimensional intention vector Furthermore, it uses the same metric (such as cosine similarity) and the same normalization method as the library-side embedding, thereby enabling... Since it resides in the same retrieval space as the library vectors, it can be directly searched using Top-K nearest neighbors. During the training phase, the library-side embeddings are used as a fixed target space, and comparative learning is employed to... Pull the library vectors to the gold standard slice to complete cross-encoder space alignment.

[0071] In step S3, the sub-intent latent vector set By directly bypassing the time-consuming natural language decoder, the query vector is concurrently sent to the vector database as an independent query vector for multi-way retrieval, thus collapsing the time overhead of multi-hop retrieval into a single forward propagation and concurrent query: Suppose a user query hits the leaf node set. For any hit slice Execute using pointers Complex dynamic context assembly, ultimately feeding the complete context to the model. Defined as:

[0072]

[0073] in This indicates a sequence concatenation operation.

[0074] In step S4, a cross-attention scorer independent of the generative model is introduced to calculate the matching confidence score before the recall context C is fed into the generative model. :

[0075] .

[0076] Among them, query representation With context representation It is obtained by an encoder independent of the main generator model. In one implementation, the same embedding model as the leaf vector library is used: the user query is Q-encoded to obtain... The assembled multi-granularity context C encoding is obtained In another implementation, the scorer incorporates a lightweight dual-tower or shared encoder to encode the Q and C responses respectively. Regardless of the implementation, this encoder is decoupled from the main generative model parameters and does not participate in the autoregressive generation of the final answer.

[0077] The cross-attention scorer is trained offline independently, while the master generative model is frozen. Training samples are constructed as (query, context, label) triplets: positive samples are the query and its supporting context (or manually confirmed retrieval results), labeled 1; negative samples are irrelevant context, incorrectly concatenated context, or noisy context containing conflicting / outdated information, labeled 0; partially relevant intermediate samples can also be introduced, labeled with soft tags of 0-1. The loss function uses binary cross-entropy (or FocalLoss) to make the output S approximate the sample labels. Thus, S reflects the matching confidence of "whether the current context is sufficient to correctly support the query," naturally falling within [0,1]. Before deployment, temperature scaling or Platt calibration can be performed on the validation set to roughly align the value of S with the empirical accuracy.

[0078] High threshold With low threshold The validation set is labeled according to business risk, satisfying 0 ≤ < ≤1. Specifically, the release accuracy, rejection rate, and false negative rate were statistically analyzed at different thresholds: Take the lowest score that makes the "acceptability rate of responses after approval" reach a preset lower limit (e.g., ≥α); The highest score that makes the "proportion of truly unanswerable / dirty data in the blocked samples" reach a preset lower limit (e.g., ≥β) is selected. The interval between the two is used to trigger a secondary retrieval of the difference vector. The threshold can be configured by domain or role, and can also be periodically recalibrated based on online feedback after going live. However, during inference, the currently configured pair of thresholds is always used for a single request.

[0079] In one possible design, step S5 involves setting a high confidence threshold. With low confidence threshold System control flow Defined as a piecewise function:

[0080]

[0081] Among them, if the confidence score matches Triggering hard blocking Termination will recall context Input the generated model, do not perform fallback generation without retrieval, and return a rejection result to the user, indicating that the current retrieval results are insufficient to support a reliable answer;

[0082] If the score falls into the critical range, it indicates that the context is relevant but information is missing, and the critical engine will calculate accordingly. and Difference vector ,Will As a negative feedback compensation signal, it is sent back to the editor to trigger a targeted secondary search based on the missing features;

[0083] in, This indicates that the query side can learn a linear projection matrix. Indicates query-side embedding. This indicates that the context side can learn linear projection matrices. This indicates context-side embedding.

[0084] If the score falls into the critical range, it indicates that the context is relevant but information is missing, and the critical engine will calculate accordingly. and Difference vector ,Will As a negative feedback compensation signal, it is sent back to the orchestrator, triggering a targeted secondary search based on missing features.

[0085] in, , To represent the query With context representation The linear projection matrix maps to the same difference space, with dimensions consistent with the query space used in the secondary retrieval. Both are learnable parameters.

[0086] During training, the master generator model is frozen; on critical intervals or constructed relevant samples, gold-labeled missing support slices are used as supervision: This allows the model to be trained... The resulting compensation query can retrieve the missing slice, thereby enabling , Learn to map what features are still missing relative to the query context. , It can be trained jointly with a cross-attention scorer, or the feedback head can be trained separately after the scorer is fixed. The parameters are fixed during the inference phase, and only forward computation is performed. It then sends a response to the arranger to initiate a targeted secondary search.

[0087] Difference vector After being fed back to the latent space orchestrator, the initial query representation is applied using an incremental overlay method. Specifically, the orchestrator aggregates the latent vectors of the initial sub-intents to obtain an aggregated query representation; the difference vectors are then overlaid onto this aggregated query representation using compensation coefficients and normalized to obtain a secondary retrieval query; this secondary retrieval query is then used to perform a targeted retrieval on the leaf vector library, and the new hit results are merged with the initial hits, deduplicated, and then reassembled into a multi-granularity context. Thus, The missing feature direction of the query relative to the current context is represented and superimposed on the original query to achieve a targeted secondary retrieval based on the missing feature. The specific steps for calculating the secondary retrieval vector are as follows:

[0088] (1) Calculation This represents the direction of missing features relative to the current context;

[0089] (2) Implicit vector of the first round Aggregation :

[0090] (3) Generate secondary search query: ,in The compensation coefficient;

[0091] (4) with Perform a Top-K search on the leaf vector library again, and reassemble the context after merging the results with the first round of hits.

[0092] Therefore, based on missing features means: retaining the relevant semantics from the first round. Under the premise, along The missing direction of the query is corrected to achieve targeted supplementary inspection.

[0093] Compared with existing modular RAG and agent technologies, this invention has significant technical advantages in terms of storage complexity, inference time complexity, and anti-hallucination mechanism:

[0094] (1) Eliminate storage redundancy and achieve Cascaded Retrieval (Solving the problems of storage overload and context fragmentation caused by forced summary splicing): Existing ContextualRetrieval technology repeatedly stores massive summaries in each underlying slice, resulting in extremely high space complexity. This invention uses "graph pointer mapping" instead of physical text splicing, enabling a single retrieval to hit fine-grained slices. At that time, only need to go through Global context assembly can be completed with pointer lookup of minimal complexity.

[0095]

[0096] This mechanism optimizes the storage complexity of the entire knowledge base. While perfectly realizing the dynamic cascading restoration of macro-context, it reduces the response latency in the retrieval and routing stage.

[0097] Specifically, multi-granularity graph construction based on virtual pointer mapping and Dynamic cascaded reconstruction technology overcomes the significant storage redundancy caused by existing technologies (such as ContextualRetrieval) that forcibly physically splice summaries to the underlying slices. This invention innovatively employs a graph-based "parent-child pointer mapping (Parent_ID)" to replace physical text writing. Only the plain text of fine-grained slices and their hierarchical pointers are stored in the base database. Upon retrieval, a pointer lookup with extremely low time complexity dynamically traces upwards and assembles the corresponding macro-level paragraph background and global summary (i.e., ...). This is a context processing paradigm of "separation of storage and computation", which restores the context while reducing storage costs.

[0098] (2) Avoid autoregressive latency and discrete error accumulation (solving the drawbacks of extremely high latency and error caused by agent architecture relying on multi-step inference of natural language). The time consumption of traditional agents in handling multi-hop problems increases linearly (or even exponentially) with the number of inference steps k: This invention intercepts early hidden states in a continuous high-dimensional space. And reduce the dimensionality to map into concurrent sub-intent latent vectors. This reduces the time overhead of multi-hop retrieval to a single forward propagation and concurrent query: This not only significantly reduces system response latency, but also operates entirely within a continuous floating-point feature space. Internal operations completely avoid irreversible semantic errors and cascade amplification during discrete natural language conversion.

[0099] The continuous latent space feature interception and dimensionality reduction routing technology for early forward propagation of large models completely overturns the high-latency mode of traditional agents that rely on large models to autoregressively generate "discrete natural language thinking text" to guide retrieval. The ingenuity of this invention lies in: directly intercepting the continuous hidden state matrix of the early layers of the forward propagation of large Transformer models. Furthermore, a pre-trained projection network is introduced to directly "segment" complex multi-hop intentions into multiple parallel sub-intention latent vectors in a continuous high-dimensional mathematical space. This technique skips the natural language decoder, transforming serial text reasoning into parallel vector addressing, and physically avoids the cascading amplification of errors caused by autoregressive latency and discrete word meaning.

[0100] (3) Construct an objective closed loop and a physical blocking mechanism to prevent hallucinations (Solve the blind faith drawback caused by the internal reflection mechanism acting as both judge and player). Existing internal verification mechanisms such as Self-RAG, due to their inability to get rid of the hallucination tendency of the generative model itself, are very prone to giving high scores to incorrect contexts (i.e., conditional probability). This invention introduces an independent cross-scoring mechanism that is completely decoupled from the generative model. Through absolute double threshold logic, when At this time, the system physically cuts off the input stream (hard blocking), which can significantly reduce the hallucination caused by recalling dirty data; and utilizes difference vectors in the critical state. By performing targeted secondary compensation, truly controllable and highly robust factual generation is achieved. Pre-generation cross-validation and difference vectors are completely decoupled from the generation black box. The negative feedback loop breaks through the internal verification limitations of existing technologies such as Self-RAG, which require the user to act as both player and referee. This invention constructs a lightweight cross-attention scorer independent of the main generator model and designs an innovative "dual-threshold tri-state routing" logic. Its core innovation lies not only in the "physical hard blocking" of low-quality data, but also in its ability to calculate and extract the difference vector between the query and the context when facing critical confidence levels. This difference vector, acting as a precise negative feedback compensation signal, can automatically trigger targeted secondary searches for missing features, giving the system true self-correction and dynamic evolution capabilities.

[0101] Figure 1 Introduction to the middle section:

[0102] Dynamic graph context: such as Figure 1 As shown in the second part, this part is responsible for receiving the original document, building the parent-child cascade index, and solving storage redundancy and contextual gaps.

[0103] Latent space intent orchestrator: such as Figure 1 As shown in the first part, it is responsible for capturing shallow continuous vectors of the model and distributing them in a dimensionality-reduced manner, replacing natural language rewriting and solving latency and error.

[0104] Independently generated front cross-validation engine: such as Figure 1 The third part, as an independent evaluator, executes hard blocking and secondary compensation loops to resolve internal blind faith and hallucinations.

[0105] The present invention also provides a system including a storage component and a processing unit, wherein the storage component stores machine-executable instructions that can be executed by the processing unit, and the processing unit executes the machine-executable instructions to implement the method as described above.

[0106] The present invention also provides a machine-readable storage medium comprising a computer-readable storage carrier having program code storage functionality, wherein the program code carried by the carrier contains executable instructions for the method as described above.

[0107] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A modular RAG method supporting latent space intent routing and multi-granularity dynamic context injection, characterized in that, Includes the following steps: S1. During the offline data entry stage, the document is parsed into a graph structure with multiple granular levels and parent pointers between levels, and the content of fine-grained nodes and the parent pointers are stored. S2. During the retrieval phase, the continuous hidden states output by the early layers of the forward propagation of the generative model are intercepted and decomposed into multiple parallel sub-intent query representations. S3. The graph structure is retrieved concurrently using the multiple sub-intent query representations, and the fine-grained nodes that are found in the retrieval are cascaded and assembled along the parent pointer to obtain a multi-granularity context. S4. Before the multi-granularity context is input into the generative model, a verification unit independent of the generative model calculates the matching confidence between the context and the query. S5. Perform three-state control based on the matching confidence: when the high threshold is reached, allow the generation model to generate an answer; when the low threshold is reached, block the input; when the two thresholds are between, extract the difference representation between the query and the context, and send it back to the mapping decomposition step as a feedback signal to trigger a targeted secondary retrieval for missing features.

2. The method according to claim 1, characterized in that, In step S1, during offline data import, the gateway parses the long document into a directed tree graph according to its logical structure. , where vertex set Includes document-level nodes Paragraph-level nodes and sentence-level slicing edge set A function for pointing to parent nodes from bottom to top. ,Right now , The system calls the lightweight model for and Generate global summary When storing data, do not write the summary. The text is stored only in the vector database. Plain text and its pointers .

3. The method according to claim 2, characterized in that, In step S2, when the user queries the large language model Q, the orchestrator intercepts the query in the early forward propagation layer of the Transformer. Let the number of truncation layers be k, and extract the continuous hidden state feature matrix from its output. : ； Will via pre-trained lightweight projection network Perform multi-head dimensionality reduction mapping to divide it into n parallel sub-intent latent vectors in a continuous high-dimensional mathematical space. : .

4. The method according to claim 3, characterized in that, In step S3, the sub-intent latent vector set By bypassing the time-consuming natural language decoder, the query vector is sent concurrently to the vector database as an independent query vector for multi-path retrieval. Assume the user query hits the leaf node set. For any hit slice Execute using pointers Complex dynamic context assembly, ultimately feeding the complete context to the model. Defined as: in, This represents the n sub-intent hidden vectors extracted from the early hidden states via the projection network. Represents the parent pointer function: slice The paragraph-level parent node, This refers to tracing back to the document-level parent node. This represents the pre-generated summary content for the node. Plain text representing slice x itself. This indicates a sequence concatenation operation.

5. The method according to claim 4, characterized in that, In step S4, a cross-attention scorer independent of the generative model is introduced to calculate the matching confidence score before the recall context C is fed into the generative model. : in, These represent the vector representation of the user query and the vector representation of the retrieved and assembled context, respectively.

6. The method according to claim 5, characterized in that, In step S5, a high confidence threshold is set. With low confidence threshold System control flow Defined as a piecewise function: Among them, if the confidence score matches Triggering hard blocking Termination will recall context Input the generated model, do not perform fallback generation without retrieval, and return a rejection result to the user, indicating that the current retrieval results are insufficient to support a reliable answer; If the score falls into the critical range, it indicates that the context is relevant but information is missing, and the critical engine will calculate accordingly. and Difference vector ,Will As a negative feedback compensation signal, it is sent back to the editor to trigger a targeted secondary search based on the missing features; in, This indicates that the query side can learn a linear projection matrix. Indicates query-side embedding. This indicates that the context side can learn linear projection matrices. This indicates context-side embedding.

7. The method according to claim 5 or 6, characterized in that, In step S3, during multi-way retrieval, the time overhead of multi-hop retrieval is reduced to a single forward propagation and concurrent query: .

8. A system, characterized in that, The method includes a storage component and a processing unit, wherein the storage component stores machine-executable instructions that can be executed by the processing unit, and the processing unit executes the machine-executable instructions to implement the method according to any one of claims 1-7.

9. A machine-readable storage medium, characterized in that, A computer-readable storage medium having program code storage capabilities, wherein the program code carried by the medium contains executable instructions for the method of any one of claims 1-7.

Images