Dynamic Feature Retrieval Generation and Management Method Based on Cross-Attention Mechanism

Abstract

This invention discloses a dynamic feature retrieval generation and management method based on a cross-attention mechanism, belonging to the field of natural language processing technology. The method includes: Step 1, converting knowledge base document fragments into atomic knowledge units, each atomic knowledge unit including question-answer pairs, encoded as key-value pairs, and appended with dynamic priority weights; Step 2, replacing self-attention queries with a dual-channel query structure, one channel for cross-attention, generating a cross-attention query vector through linear transformation, and using the key-value pairs of the atomic knowledge units; Step 3, training a cross-attention adapter, freezing the language model weights, optimizing the adapter parameters, and dynamically adjusting the loss function using prediction parameters based on the contextual complexity of the input sequence. This method can achieve a deep association between user input and knowledge fragments, resolve semantic ambiguity, balance knowledge injection and contextual information, avoid generation distortion, and meet the stringent semantic accuracy requirements of professional fields.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and more specifically, to a method for dynamic feature retrieval, generation, and management based on a cross-attention mechanism. Background Technology

[0002] As the application of Large Language Models (LLM) in text generation, dialogue interaction, and knowledge-based question answering has deepened, its generalization ability has been verified. However, in scenarios such as enterprise private domain corpora that require specific knowledge support, LLM lacks targeted knowledge modeling and is unable to meet the needs of high-precision semantic understanding and generation. To address this, Retrieval-Enhanced Generation (RAG) technology has become a solution. Its approach is to recall relevant corpus fragments from external knowledge bases and use them together with user input as the basis for LLM generation.

[0003] However, traditional RAG methods rely heavily on static semantic similarity for retrieval. They first convert corpus fragments into vectors and then rank and recall data based on similarity to the user's input vector. This approach has significant limitations in practical applications. When faced with complex contexts or polysemous expressions, the retrieval accuracy is greatly limited. For example, in a medical scenario, when a user asks about "postoperative complication management," the types of complications and corresponding solutions vary significantly depending on the surgery. Static vectors cannot adjust the retrieval direction based on the real-time context of "specific surgical type." They cannot adapt to the dynamic context features of user input, cannot resolve semantic ambiguities, and cannot capture the deep semantic connections hidden between texts, resulting in retrieval results that often deviate from the user's true intent.

[0004] From a technical perspective, these problems stem from the inherent flaws of the static vector similarity calculation paradigm, which cannot effectively model the dynamic semantic interaction between user input and knowledge fragments. The traditional RAG retrieval process is fixed; no matter how the user input context changes, the extraction rules and matching logic of retrieval features remain unchanged, making it impossible to accurately capture fine-grained semantic relevance, ultimately limiting the accuracy of retrieval results from the bottom up.

[0005] In the polysemous contexts of private domain corpora such as finance and healthcare, these corpora contain a large number of domain-specific terms and are often polysemous. For example, "risk hedging" in the financial field has different meanings in the context of stocks and futures. However, in pursuit of retrieval efficiency, traditional RAGs often simplify the semantic modeling process by using fast vector similarity calculations to achieve efficient recall of large-scale corpora. They do not deeply analyze the contextual dependencies between sentences or analyze the deep intent of user queries. This design that prioritizes efficiency over accuracy makes it difficult to balance retrieval efficiency and semantic accuracy, which restricts its application in high-precision fields. Summary of the Invention

[0006] To address the problems existing in the prior art, the present invention aims to provide a dynamic feature retrieval generation and management method based on a cross-attention mechanism, which can realize a deep association between user input and knowledge fragments, resolve semantic ambiguity, balance knowledge injection and contextual information, avoid generation distortion, and meet the stringent requirements of professional fields for semantic accuracy.

[0007] To solve the above problems, the present invention adopts the following technical solution:

[0008] Firstly, a dynamic feature retrieval generation and management method based on a cross-attention mechanism includes:

[0009] Step 1: Convert knowledge base document fragments into atomic knowledge units. Each atomic knowledge unit includes question-answer pairs, encoded as key-value pairs, and assigned dynamic priority weights.

[0010] Step 2: Replace the self-attention query with a dual-channel query structure, one channel of which is used for cross-attention. Generate a cross-attention query vector through linear transformation, using key-value pairs of atomic knowledge units, and calculate the cross-attention matrix based on the cross-attention query vector and the key-value pairs of atomic knowledge units.

[0011] Step 3: Train the cross-attention adapter, freeze the language model weights, optimize the adapter parameters, and dynamically adjust the loss function using prediction parameters based on the contextual complexity of the input sequence.

[0012] Step 4: Based on the cross-attention matrix, calculate the relevance score between the input sequence and each atomic knowledge unit, weight and fuse the attention scores using a preset sequence position weight function, and apply the contribution coefficient;

[0013] Step 5: Based on the relevance score, construct a knowledge topology network based on the relevance score and semantic overlap, and retrieve a subset of atomic knowledge units using a graph diffusion algorithm;

[0014] Step 6: During the generation process, a gating function based on the matching degree between the current generation context and the atomic knowledge unit is used to dynamically control the knowledge injection;

[0015] Step 7: Based on the generated results and user feedback, adjust the dynamic priority weights and contribution coefficients, and update the parameters using incremental learning.

[0016] Further, step 1 includes:

[0017] Step 11: Based on the document's logical structure and semantic density, identify semantic unit boundaries by analyzing punctuation density, term frequency, and sentence complexity.

[0018] Step 12: Within the semantic unit, analyze the logical relationships between statements, identify the semantic patterns of question-answer, argument-evidence, and problem-solution, extract key propositions as questions and associate supporting content as answers, form question-answer pairs and attach structural tags;

[0019] Step 13: Perform dual-channel encoding on the question-answer pairs, encode the question text into a discriminative vector, encode the answer text into a capacity vector, and map it to a unified semantic space through projection transformation to form key-value pairs;

[0020] Step 14: Calculate the semantic similarity between atomic knowledge units using key-value pair semantic vectors, construct a knowledge unit association network, and calculate the initial priority weights based on node centrality index and connection density.

[0021] Further, step 2 includes:

[0022] Step 21: Perform semantic role analysis on each token in the input sequence, assign semantic role labels, and allocate two query dimension spaces for each token based on the semantic role labels, including internal semantic association channels and external knowledge interaction channels;

[0023] Step 22: Using semantic role labels and query dimension space, perform channel-specific transformation on the initial representation of each token. The internal semantic association channel adopts linear transformation, while the external knowledge interaction channel adopts non-linear transformation. The transformation matrix is ​​grouped with shared parameters according to the semantic role labels.

[0024] Furthermore, step 2 also includes:

[0025] Step 23: Perform semantic focusing processing on the query vector of the external knowledge interaction channel, including calculating the initial relevance distribution with the key vector of the knowledge unit, identifying the semantic focus region, and performing local semantic enhancement transformation;

[0026] Step 24: Dynamically fuse the output of the internal semantic association channel and the output of the enhanced cross-attention channel, and calculate the fusion weight based on the current decoding state and the semantic complexity of the input sequence.

[0027] Further, step 3 includes:

[0028] Step 31: Calculate the semantic distribution dispersion and context dependency strength of tokens in the input sequence to generate a context complexity score;

[0029] Step 32: Using context complexity scores, generate global prediction parameters at the sequence level and local prediction parameters at the token subset level, and fuse multi-scale prediction parameters through a nonlinear combination function;

[0030] Step 33: Based on multi-scale prediction parameters, adjust the weight coefficients and weighting factors of the standard loss function, adjust the overall weight coefficients of the loss function using global prediction parameters, and introduce token-level weighting factors using local prediction parameters;

[0031] Step 34: Under the guidance of the reconstructed loss function, optimize and update the adapter parameters. Identify the update direction of key parameters based on the gradient distribution, prioritize the adjustment of parameter paths corresponding to high-complexity contexts, and control the update magnitude through gradient pruning mechanism while keeping the language model weights frozen.

[0032] Further, step 4 includes:

[0033] Step 41: Identify and filter semantic redundant connections in the cross-attention matrix based on attention kurtosis, including analyzing the distribution characteristics of attention weights to identify abnormal connection points with high variance, and performing smoothing suppression on redundant connections.

[0034] Step 42: Based on the purified cross-attention matrix, calculate the initial relevance score of each atomic knowledge unit, including the purified attention weights of all tokens in the aggregated input sequence, using the geometric mean aggregation function, and record the distribution characteristics of the attention tokens.

[0035] Furthermore, step 4 also includes:

[0036] Step 43: Utilize the distribution characteristics of the tokens that have attracted attention, apply a preset sequence position weight function to adjust the initial relevance score, including assigning importance coefficients based on the relative position of the tokens in the sequence, and assigning higher weight coefficients to the tokens of key semantic nodes, and obtain the position-aware relevance score through weighted fusion.

[0037] Step 44: Based on the location-aware relevance score, the contribution coefficient is applied for final balancing, including dynamically adjusting the contribution coefficient according to the semantic complexity of the input sequence and the size of the knowledge base, and generating the final relevance score through a combination of preset nonlinear functions.

[0038] Further, step 5 includes:

[0039] Step 51: Construct a weighted directed graph based on relevance scores and semantic overlap, where nodes represent atomic knowledge units, node weights are assigned after normalization of relevance scores, and edge weights are calculated by measuring the cosine similarity of the knowledge unit key vectors and combining them with contextual dependencies.

[0040] Step 52: Perform multi-round graph diffusion processing on the weighted directed graph. During the diffusion process, each node calculates the attention score based on the edge weight and its own node weight, and dynamically adjusts the information transmission intensity.

[0041] Step 53: Based on the diffused node scores, an adaptive threshold mechanism based on the statistical characteristics of score distribution is used to extract a subset of atomic knowledge units, and the subset is then subjected to redundancy filtering.

[0042] Further, step 6 includes:

[0043] Step 61: Monitor changes in the generated context state, including calculating the semantic completeness of the generated sequence and the dispersion of the predicted probability distribution, and quantifying the degree of uncertainty;

[0044] Step 62: Based on the generated context state, calculate the semantic matching degree of the atomic knowledge unit, including direct semantic similarity and semantic coherence with subsequent generation direction, and use bidirectional semantic projection to evaluate the degree of fit.

[0045] Step 63: Based on the semantic matching degree evaluation results, execute the dynamic gating function to adjust the intensity ratio and timing of knowledge injection according to the matching degree score and the degree of contextual uncertainty.

[0046] Further, step 7 includes:

[0047] Step 71: Extract the semantic consistency, factual accuracy and logical coherence quality indicators of the generated results, analyze the explicit ratings and implicit behavioral data of user feedback, calculate the feedback confidence score, establish the correlation mapping between quality indicators and feedback confidence to generate a comprehensive evaluation signal.

[0048] Step 72: Based on the comprehensive evaluation signal and confidence weight, the dynamic priority weight and contribution coefficient are incrementally adjusted. Reinforcement learning optimization or smooth adjustment strategies are adopted according to the confidence score, and a tracking mechanism for parameter update trajectory is established.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] (1) This solution achieves dynamic semantic adaptation through cross-attention mechanism, which can adjust the retrieval weight in real time according to the user input context, get rid of the limitation of traditional RAG relying on static vector similarity. For polysemous query scenarios in finance, medical care and other fields, it can accurately capture context changes and avoid the deviation caused by single static representation. For example, in complex polysemous term queries, it can dynamically match deep semantic needs, improve the retrieval accuracy in complex contexts, and make the retrieval results more in line with the user's true intention.

[0051] (2) This scheme explicitly models the deep semantic relationship between user input and knowledge fragments, calculates the relevance score between the input sequence and the atomic knowledge unit through the cross attention matrix, and constructs a knowledge topology network by combining semantic overlap. Compared with the problem that traditional RAG is difficult to characterize deep relationships, this scheme can capture fine-grained semantic relationships, such as parsing the logical relationship between the core needs in the user query and the knowledge unit, effectively eliminating semantic ambiguity, greatly improving the relevance of the search results, and reducing the matching error caused by the lack of semantic relationship.

[0052] (3) This scheme balances the relationship between knowledge injection and input context by contribution coefficient, avoiding over-reliance on external knowledge that leads to generation distortion. At the same time, the training process covers scenarios such as single knowledge point, multiple knowledge point and rejection question answering. In high-precision scenarios such as financial compliance query and medical diagnosis assistance, it can ensure that knowledge injection can efficiently support generation, maintain generation stability, adapt to different task types, improve model robustness, and solve the problem that traditional RAG is difficult to balance between accuracy and stability.

[0053] (4) This solution uses the cross-attention mechanism to closely integrate knowledge retrieval and generation, eliminating the error accumulation caused by the decoupling of traditional RAG retrieval and generation. From atomic knowledge representation and model training to retrieval recall and generation control, the semantic logic of each link is coherent. For example, the relevance score in the retrieval stage directly guides the knowledge injection in the generation stage, so that the generated content not only meets the input requirements but also accurately integrates knowledge, improving the quality of question answering and user experience. It is especially suitable for professional fields with extremely high requirements for semantic accuracy. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0055] Figure 1 This is a flowchart of the dynamic feature retrieval generation and management method based on the cross-attention mechanism of the present invention;

[0056] Figure 2 This is a flowchart illustrating the workflow of the dynamic feature retrieval, generation, and management method based on the cross-attention mechanism of this invention. Detailed Implementation

[0057] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0058] Please see Figures 1 to 2 The specific steps of the dynamic feature retrieval generation and management method based on the cross-attention mechanism are as follows:

[0059] Step 1: Convert knowledge base document fragments into atomic knowledge units. Each atomic knowledge unit includes question-answer pairs, encoded as key-value pairs, and assigned dynamic priority weights.

[0060] Step 2: Replace the self-attention query with a dual-channel query structure, with one channel used for cross-attention. Generate a cross-attention query vector through linear transformation, using key-value pairs of atomic knowledge units, and calculate the cross-attention matrix based on the cross-attention query vector and the key-value pairs of atomic knowledge units.

[0061] Step 3: Train the cross-attention adapter, freeze the language model weights, optimize the adapter parameters, and dynamically adjust the loss function using prediction parameters based on the contextual complexity of the input sequence.

[0062] Step 4: Based on the cross-attention matrix, calculate the relevance score between the input sequence and each atomic knowledge unit, weight and fuse the attention scores using a preset sequence position weight function, and apply the contribution coefficient;

[0063] Step 5: Based on the relevance score, construct a knowledge topology network based on the relevance score and semantic overlap, and retrieve a subset of atomic knowledge units using a graph diffusion algorithm;

[0064] Step 6: During the generation process, a gating function based on the matching degree between the current generation context and the atomic knowledge unit is used to dynamically control the knowledge injection;

[0065] Step 7: Based on the generated results and user feedback, adjust the dynamic priority weights and contribution coefficients, and update the parameters using incremental learning.

[0066] Step 1 further includes the following steps: Step 11, based on the document's logical structure and semantic density, semantic unit boundaries are identified by analyzing punctuation density, term frequency, and sentence complexity. The specific steps are as follows:

[0067] Firstly, based on the document's logical structure and semantic density, multi-dimensional text feature analysis is used to collaboratively identify semantic unit boundaries. Punctuation density analysis focuses on punctuation marks with semantic separation functions, such as periods, semicolons, and colons. The distribution frequency and interval patterns of these punctuation marks within different paragraphs are statistically analyzed. Abrupt changes in punctuation density, such as a sudden shift from a low-frequency to a high-frequency range, often correspond to natural breaks in semantic content, potentially marking the end of a complete semantic expression and the beginning of a new one. Terminology frequency analysis targets domain-specific terms or core vocabulary. By statistically analyzing the frequency and concentration of these terms in text fragments, areas with high-frequency and concentrated terminology often carry relatively complete domain semantic information and can serve as core semantic units. Anchor point analysis and sentence complexity analysis quantify the simplicity and complexity of sentences by analyzing their grammatical structure, such as subject-verb-object hierarchy, number of modifiers, and types of conjunctions in complex sentences. When there is a significant leap in sentence complexity, such as transitioning from a simple short sentence to a long and complex sentence containing multiple layers of modifiers and adverbs, or vice versa, it usually means a shift in semantic expression logic, which can serve as an auxiliary verification basis for the boundaries of semantic units. By cross-validating and associating the results of the above three feature analyses, the semantic coherence of the document text is judged paragraph by paragraph and sentence by sentence, and the specific boundaries of semantic units are finally determined. This ensures that the internal logic of each divided semantic unit is coherent and semantically complete, and that there are clear semantic distinctions between different units, avoiding semantic fragmentation or redundancy during subsequent atomic knowledge extraction.

[0068] Step 12: Within the semantic unit, analyze the logical relationships between statements, identify question-answer, argument-evidence, and problem-solution semantic patterns, extract key propositions as questions and associate supporting content as answers, forming question-answer pairs and attaching structural tags. The specific operations are as follows:

[0069] Building upon the independent semantic units defined in Step 11, we delve deeper into the semantic units themselves. By analyzing the logical relationships between statements, we identify specific semantic patterns and extract the core components of atomic knowledge units, such as question-and-answer pairs, while simultaneously completing structured labeling. First, we analyze the logical relationships of all statements within a semantic unit, identifying causal, progressive, parallel, and explanatory relationships. From this, we select semantic patterns with a clear "core content - supporting content" correspondence, specifically including question-and-answer patterns, argument-evidence patterns, and problem-solution patterns. The question-and-answer pattern identifies explicit interrogative sentences in the text, containing interrogative words such as "what," "how," and "why," or implicit interrogative sentences, and locates the explanatory statements that immediately follow to directly answer the question, forming a semantic correspondence of "question statement - answer content." The argument-evidence pattern distinguishes argument statements that express viewpoints, conclusions, or judgments, typically containing words such as "is," "should," "has," and "indicates." For the "core viewpoint - supporting evidence" semantic association, the system first extracts statements describing the problem, deficiency, or difficulty, including problem-characteristic words such as "problem," "shortcoming," "difficulty," and "challenge," and then matches them with statements proposing solutions, implementation steps, or optimization strategies to form a "problem description - solution" semantic correspondence. For the above three modes, the core propositions are extracted as questions in the question-and-answer pair, such as the core question in the question mode, the main point of the argument mode, and the essence of the problem mode. At the same time, the corresponding supporting content, such as the details of the answer, the full text of the argument, and the content of the solution, are extracted as the answers in the question-and-answer pair. Finally, a corresponding structural label is attached to each generated question-and-answer pair, such as "question-answer type," "argument-evidence type," and "problem-solution type." This label is used to clearly identify the semantic mode category of the question-and-answer pair.

[0070] Step 13: Perform dual-channel encoding on the question-answer pairs, encoding the question text into a discriminative vector and the answer text into a capacity vector, and then map them to a unified semantic space through projection transformation to form key-value pairs. The specific operations are as follows:

[0071] To ensure that question-answer pairs can adapt to the semantic computation requirements of subsequent cross-attention retrieval and knowledge generation stages, a dual-channel encoding strategy is adopted to differentiate the question-answer pairs. Projection transformation is used to unify the vector space, ultimately forming a standardized key-value pair representation. This dual-channel encoding strategy designs differentiated encoding objectives based on the different semantic functions of questions and answers: For question text, the encoding objective is to improve its semantic distinguishability, meaning that the distinguishability vector obtained through encoding should clearly reflect the semantic differences between different questions, avoiding semantic confusion caused by similar question expressions, such as synonym substitution or sentence structure transformation. To achieve this objective, the encoding model focuses on capturing core interrogative words, limiting conditions, and domain-specific expressions in the question. The encoding model can use methods such as Sentence Correction. The Transformers series of models, by adjusting the weight distribution of the attention mechanism, ensures that key information contributes more to the vector representation, guaranteeing significant distinguishability of vectors for different questions in the semantic space. For the answer text, the encoding goal is to increase its information capacity; that is, the capacity vector must fully carry the factual information, logical reasoning process, and detailed descriptions contained in the answer. Encoding preserves the contextual relevance and information integrity of the answer text. By optimizing the feature extraction strategy of the encoding model, it reduces the loss of key details due to information compression, ensuring that the answer vector comprehensively reflects the knowledge content. After obtaining the initial discriminative and capacity vectors through dual-channel encoding, since the two types of vectors may originate from different encoding subspaces, such as dimensionality... If the feature distributions differ and cannot be directly used for subsequent cross-attention calculations, they need to be mapped to a unified semantic space through projection transformation. Specifically, a linear projection matrix adapted to the semantic features of the text is used to perform spatial transformations on the discriminant vector and the capacity vector respectively. The parameters of the projection matrix are optimized through pre-training on the domain corpus to ensure that the two types of vectors after transformation have the same dimension and consistent semantic metric, which meets the format requirements of key-value vectors in the cross-attention mechanism. Finally, the question discriminant vector after projection transformation is defined as the key, and the answer capacity vector is defined as the value, forming a key-value pair structure. This structure not only fully preserves the semantic correspondence between question and answer pairs, but also provides standardized input for the cross-attention calculation of the subsequent knowledge retrieval model.

[0072] Step 14: Utilize key-value pair semantic vectors to calculate the semantic similarity between atomic knowledge units, construct a knowledge unit association network, and calculate initial priority weights based on node centrality and connection density. The specific operations are as follows:

[0073] Based on the key-value pair semantic vectors generated in the previous steps, a knowledge unit association network is constructed. The initial priority weights of atomic knowledge units are calculated by analyzing the network topology features. First, the semantic similarity between different atomic knowledge units is calculated: using the key vector or the fusion vector of the key vector and value vector of the atomic knowledge unit as the semantic representation carrier, a similarity calculation method suitable for text vectors, such as cosine similarity, is adopted to quantify the degree of association between any two atomic knowledge units in terms of semantic content. When the semantic similarity score of two units is higher than a preset threshold, which is set according to the characteristics of the knowledge base domain and the semantic association requirements, such as 0.7, it is determined that there is a valid semantic association between the two, and an undirected connection edge is established in the network, thereby forming a knowledge unit association network with atomic knowledge units as nodes and semantic associations as edges. Subsequently, the topological characteristics of the network were analyzed, focusing on calculating two types of indicators: first, node centrality, including degree centrality (determined by the number of effective connecting edges a node possesses), betweenness centrality (determined by the frequency with which a node lies on the shortest path between any two other nodes), and compact centrality (determined by the reciprocal of the average shortest path length from a node to all other nodes in the network). A higher node centrality score indicates a closer direct or indirect connection between the atomic knowledge unit and other units in the network, typically representing stronger coreity and knowledge radiation capabilities within the knowledge base; second, connection density, i.e., the actual number of nodes in the network... The ratio of the number of effective connections to the theoretically possible number of connections, which can be obtained by multiplying the total number of nodes by (total number of nodes - 1) / 2, allows us to calculate the connection density of each node's local subnetwork. For example, the subnetwork formed by the node's neighbors with a radius of 2 can determine the semantic aggregation degree of the knowledge region where the node is located. Regions with higher connection density have stronger semantic synergy and correlation among knowledge units. Finally, the initial priority weights are calculated by combining node centrality and connection density. A weighted combination formula is used to fuse the two types of indicators, which can be expressed as:

[0074] ;

[0075] in, Let be the initial priority weight of the i-th atomic knowledge unit; The normalized centrality score of the i-th node, with a value range of [0,1]; The normalized connection density score of the local subnetwork where the i-th node is located, with a value range of [0,1]. and Let be the weighting coefficient, satisfying Pre-set according to domain requirements, such as in knowledge-first scenarios. , This formula originates from the logic that core knowledge units must simultaneously possess high relevance and strong semantic aggregation. By linearly weighting and balancing the contributions of the two types of indicators, it ensures that the weight calculation results can objectively reflect the importance of atomic knowledge units. Through the above calculation, each atomic knowledge unit obtains a corresponding initial priority weight. The higher the weight value of the unit, the higher the probability of it being recalled first in the subsequent retrieval process, thereby achieving efficient scheduling of core knowledge.

[0076] In a preferred embodiment of the present invention, the generation of the cross-attention matrix in step 2 is specifically performed as follows:

[0077] The generation of the cross-attention matrix is ​​based on the representation of atomic knowledge units and the dual-channel query optimization of the input sequence. It is constructed step by step following a rigorous semantic interaction and feature transformation logic. First, the key-value pairs of atomic knowledge units provide interaction objects for cross-attention calculation. These key-value pairs are generated by semantic unit segmentation of knowledge base document fragments, question-answer pair extraction, and dual-channel encoding. The question text is encoded as a discriminative vector, and the answer text is encoded as a capacity vector. After projection transformation, they are mapped to a unified semantic space to ensure semantic interactivity with the input sequence. The input sequence needs to be preprocessed to generate an appropriate query vector. Through semantic role analysis, channel-specific transformation, semantic focusing processing, and dynamic fusion, the generated query vector maintains the internal logical consistency of the input sequence and fully integrates the external knowledge interaction requirements. Finally, through similarity calculation, such as cosine similarity and dot product similarity, the semantic association strength between each token and each atomic knowledge unit is quantified to form the cross-attention matrix.

[0078] Step 2 also includes the following operations:

[0079] Step 21: Perform semantic role analysis on each token in the input sequence, assign semantic role labels, and allocate two query dimension spaces for each token based on the semantic role labels, including internal semantic association channels and external knowledge interaction channels. The specific operations are as follows:

[0080] First, semantic role analysis is performed on each token in the input sequence. This process typically employs a model based on a pre-trained semantic role annotation framework, such as PropBank or FrameNet. Combining the grammatical structure and contextual information of the input sequence, the semantic function and contribution of each token in the sentence are identified, and then specific semantic role labels are assigned. Common labels include predicates (core tokens representing actions or states); agents (the main tokens initiating actions); patients (the tokens to which actions are performed); time adverbs (tokens defining the time of actions); and place adverbs (tokens defining the location of actions), etc. These labels clearly define the semantic location and functional requirements of the tokens in the input sequence. Subsequently, based on the assigned semantic role labels, two independent query dimensions are allocated to each token. The allocation logic, which consists of internal semantic association channels and external knowledge interaction channels, stems from the differentiated semantic processing needs of tokens with different semantic roles. For example, predicate tokens need to maintain strong logical associations with other tokens in the input sequence to ensure sentence grammatical and semantic coherence, as well as to acquire external knowledge to verify their semantic accuracy, such as the standardized expression of actions within the domain. Therefore, two channels are needed to carry the corresponding needs respectively. On the other hand, time adverbial tokens focus more on the temporal logical connection with other tokens in the input sequence, with lower external knowledge interaction requirements, but still need external channels to assist in verifying the rationality of time expressions. Therefore, a dual-channel design is also used to cover potential needs. Through this channel allocation based on semantic roles, it can be ensured that the semantic processing of each token accurately matches its functional positioning, avoiding the limitations of a single query space that cannot take into account both internal associations and external interactions.

[0081] Step 22: Using semantic role labels and the query dimension space, perform channel-specific transformations on the initial representation of each token. Internal semantic association channels use linear transformations, while external knowledge interaction channels use non-linear transformations. The transformation matrix is ​​grouped with shared parameters based on the semantic role labels. The specific operations are as follows:

[0082] To address the differentiated semantic requirements of the two query dimensions, a channel-specific transformation is performed on the initial representation of each token. The initial representation of each token is typically a word embedding vector output by a pre-trained language model. Semantic role grouping is used to achieve parameter sharing of the transformation matrix, balancing model performance and parameter quantity. For internal semantically related channels, a linear transformation is used to process the initial representation of the token, specifically through a learnable linear transformation matrix. Bias term with inner channel The initial representation x is transformed into an internal channel feature vector. The reason for choosing linear transformation is that the goal of the internal channel is to maintain the semantic coherence and logical consistency of the input sequence. Linear transformation can achieve adaptive transformation of the feature space while preserving the initial semantic information of the tokens, avoiding the additional semantic distortion introduced by nonlinear transformation, and ensuring that the association between tokens within the sequence is not destroyed. For the external knowledge interaction channel, since it needs to handle the complex semantic mapping that may exist between tokens and external knowledge units, such as the ambiguity of domain terms and cross-domain semantic associations, nonlinear transformation is adopted, using linear matrices. external channel bias term The initial representation is transformed into an external channel feature vector using a nonlinear activation function σ, which can be either a GELU function or a ReLU function. Nonlinear activation functions can capture complex nonlinear relationships in the semantic space, making the external channel feature vectors more adaptable to subsequent interactions with the knowledge unit key vectors. Simultaneously, based on the semantic role labels assigned in step 21, the transformation matrix is ​​grouped using parameter sharing: semantic roles are divided into core semantic roles, such as predicates, agents, and patients, which directly affect the core semantics of the sentence, and auxiliary semantic roles, such as time adverbs and place adverbs, which play a modifying and supplementary role. Each of these two types of roles is assigned an independent set of parameters. , The parameters are shared by all tokens of the same type of role. This grouping strategy can significantly reduce the number of model parameters and reduce the risk of overfitting. It can also ensure that tokens with the same semantic function maintain consistent processing in channel transformation and improve the model's generalization ability to similar semantic scenarios.

[0083] Step 23: Perform semantic focusing processing on the query vector of the external knowledge interaction channel, including calculating the initial relevance distribution with the knowledge unit key vector, identifying the semantic focus region, and performing local semantic enhancement transformation. The specific operations are as follows:

[0084] First, calculate the initial correlation distribution between the external channel feature vector and the knowledge unit key vector: the feature vector output by the external channel. as query vector The bond vector of all atomic knowledge units in the knowledge base (j=1,2,...,Z, where Z is the total number of atomic knowledge units) Perform similarity calculations, such as cosine similarity or dot product similarity, to obtain the similarity between each key vector and... correlation score All scores constitute the initial correlation distribution. This distribution intuitively reflects the semantic association strength between the current token and each knowledge unit. Subsequently, semantic focus regions are identified based on the initial relevance distribution: by setting a relevance threshold, which can be obtained by adding one standard deviation to the mean of distribution S, or by Top-K filtering, which can obtain the top 5 key vectors with the highest scores, a subset with significantly higher relevance scores than other key vectors is selected. The semantic range of the knowledge units corresponding to this subset is the semantic focus region. These knowledge units have the highest semantic relevance to the current token and are the objects of subsequent knowledge interaction. Identifying the focus region can effectively narrow the retrieval range of external knowledge and reduce the interference of irrelevant knowledge. Finally, a local semantic enhancement transformation is performed: for the key vector corresponding to the semantic focus region, its average feature vector is calculated. And calculated through dot product and Similarity score ,Will As an enhancement coefficient, for The enhanced external channel vector is obtained by weighting adjustment. This local enhancement strategy can be further strengthened. The semantic features related to the focal region enable it to more accurately point to the relevant knowledge units in subsequent cross-attention calculations, thereby improving the ability of the external channel vector to capture useful knowledge.

[0085] Step 24: Dynamically fuse the output of the internal semantic association channel and the output of the enhanced cross-attention channel. Calculate the fusion weight based on the current decoding state and the semantic complexity of the input sequence. The specific operation is as follows:

[0086] This step generates a final feature vector that balances intra-sequence logical coherence and external knowledge relevance by dynamically fusing the output vectors of the internal semantic association channel and the enhanced external knowledge interaction channel. The fusion process calculates dynamic fusion weights based on the current decoding state and the semantic complexity of the input sequence to ensure that the fusion result adapts to the current generation requirements. First, two key reference indicators are obtained: one is the current decoding state, i.e., the current hidden layer state of the generator model decoder. This state integrates the semantic information of the generated content and can be calculated. With internal channel vector cosine similarity The fusion weights are calculated based on two metrics: first, the matching degree between the internal channel output and the current generation context; and second, the semantic complexity C of the input sequence, which is calculated using the semantic distribution dispersion of the tokens and the strength of contextual dependencies in previous steps. A higher C value indicates a more complex semantic structure in the current input sequence and a stronger need for supplementary external knowledge. A fusion weight calculation function is constructed based on these two metrics, specifically the internal channel weights. External channel weights ;when A higher C value indicates a high degree of matching between the internal channel vector and the current decoding state, requiring an increase in the internal channel weights to maintain logical coherence in the sequence. When C is high, 1-C decreases, and the external channel weights increase accordingly to introduce more external knowledge to support complex semantic understanding. In the formula, Represents the contribution weight of the internal channel vectors in the fusion result. Represents the contribution weight of the enhanced external channel vector; Used for balance The influence of C on the weights, The default value is 0.5; C represents the similarity between the decoded state and the internal channel vectors, with a value ranging from [0,1]; C represents the semantic complexity, with a value ranging from [0,1]. Finally, a weighted summation is used to dynamically fuse the two channel vectors, generating the final feature vector. This vector preserves the internal logical connections of the input sequence and integrates the necessary external knowledge, which can efficiently support subsequent cross-attention retrieval and knowledge enhancement generation.

[0087] In a preferred embodiment of the present invention, step 3 further includes the following operation steps:

[0088] Step 31: Calculate the semantic distribution dispersion and contextual dependency strength of tokens in the input sequence to generate a contextual complexity score. The specific operations are as follows:

[0089] First, the semantic distribution dispersion of each token in the input sequence is calculated: a high-dimensional semantic vector is extracted for each token using a pre-trained language model, such as BERT or GPT. The high-dimensional semantic vector represents the token's position and category attributes in the semantic space. Then, the distribution dispersion is calculated based on the semantic vectors of all tokens, typically using Shannon entropy or variance as a metric. If Shannon entropy is used, the probability distribution of the semantic vectors across preset semantic categories, such as domain-specific semantic categories, must first be statistically analyzed. A higher entropy value indicates a more diverse range of categories covered by the token's semantic vectors, a more dispersed semantic expression, and higher contextual complexity. If variance is used, the overall variance of each dimension of the semantic vector is calculated. A larger variance indicates a more dispersed distribution of token semantics in space and stronger semantic diversity. Second, the context dependency strength of the tokens is calculated: this is achieved by analyzing the semantic association between each token and its context tokens, typically using a large window size for the context tokens. The context complexity score is 5-10. Specifically, the attention weights output by the self-attention mechanism can be used to statistically analyze the distribution of attention weights of the current token to the context tokens. The variance or coefficient of variation of the weights can be calculated. The larger the variance or coefficient of variation, the stronger the dependence of the current token on some context tokens and the weaker the dependence on others, indicating an imbalance in the semantic association of the context and a higher contextual logical complexity. Alternatively, the semantic synergy between the token and the context tokens can be calculated through mutual information. The greater the fluctuation of the mutual information value, the more complex the contextual dependency relationship. Finally, the semantic distribution dispersion and the context dependency strength are normalized and mapped to the [0,1] interval. By pre-setting weight coefficients, such as 0.6 for semantic distribution dispersion and 0.4 for context dependency strength (which can be adjusted according to domain requirements), a weighted sum is performed to obtain the context complexity score of each input sequence. The score ranges from [0,1]. The closer the score is to 1, the more complex the sequence context and the higher the learning difficulty of the model.

[0090] Step 32: Using context complexity scores, global prediction parameters are generated at the sequence level, and local prediction parameters are generated at the token subset level. Multi-scale prediction parameters are then fused using a nonlinear combination function. The specific operations are as follows:

[0091] First, global prediction parameters are generated: Based on the sequence-level context complexity score obtained in step 31, a non-linear transformation, such as the Softmax function or the Sigmoid function, is used to map it to the [0,1] interval to obtain global prediction parameters. These parameters reflect the overall learning difficulty of the input sequence. For example, when the sequence context complexity score is 0.8, it indicates high complexity, and the global prediction parameter after the Sigmoid transformation may be close to 0.9, indicating that the entire sequence requires more learning resources from the model; while when the complexity score is 0.2, it indicates low complexity, and the global prediction parameter may be close to 0.9. A value close to 0.3 indicates low sequence learning difficulty, requiring minimal resource allocation. The next step involves generating local prediction parameters: first, the input sequence is divided into multiple token subsets based on the semantic role labels of the tokens. Examples include a core semantic token subset containing predicates, agents, and patients, and an auxiliary semantic token subset containing adverbs of time and place. Then, the average contextual complexity score of all tokens within each subset is calculated, representing the subset-level average complexity. Finally, this average value is mapped to the corresponding subset's local prediction parameters using a linear transformation matrix. The parameter values ​​are also determined in the same way. The local prediction parameters, defined as [0,1], reflect the differences in learning difficulty among different token subsets. For example, the average complexity of the core semantic token subset is 0.7, and its local prediction parameter might be 0.8, while the average complexity of the auxiliary semantic token subset is 0.3, and its local prediction parameter might be 0.4. This clearly distinguishes the priority of different subsets. Finally, a nonlinear fusion of multi-scale prediction parameters is performed: a nonlinear combination function, such as the GELU activation function or attention fusion mechanism, is used to fuse the global prediction parameters with each local prediction parameter. First, the global parameters are used as the fusion benchmark. The contribution of each local parameter to the global parameter is calculated using attention weights. The contribution of the core semantic subset is usually higher than that of the auxiliary subset. The global parameters and the weighted local parameters are then nonlinearly integrated using the GELU function to obtain the final multi-scale prediction parameters. The reason for using nonlinear fusion is that the influence of global and local parameters on loss adjustment is not a simple linear superposition. When the global complexity is high, the contribution of the local parameters of the core semantic subset to loss adjustment needs to be nonlinearly amplified. Nonlinear functions can accurately capture this complex relationship and ensure that the fused parameters can fully reflect the contextual difficulty at different granularities.

[0092] Step 33: Based on multi-scale prediction parameters, adjust the weight coefficients and weighting factors of the standard loss function. Use global prediction parameters to adjust the overall weight coefficients of the loss function, and use local prediction parameters to introduce token-level weighting factors. The specific operations are as follows:

[0093] First, we need to clarify the basics of the standard loss function, which is usually the cross-entropy loss function:

[0094] ;

[0095] Where N is the number of tokens. For the first The real label of a token To predict probabilities for the model, The cross-entropy loss function measures the deviation between the model's predictions and the actual values, but it cannot adjust the learning emphasis according to the difficulty of the context. Therefore, the overall weight coefficients of the loss function are adjusted based on the global prediction parameter: let the global prediction parameter be G, taking values ​​[0,1]; the preset coefficient is... Typically, it is set to 0.5-1.0, and adjusted according to training needs. The adjusted overall weight coefficient is then... This adjustment logic stems from the need for higher learning weights for complex sequences. When G approaches 1, it indicates high contextual complexity. Increasing G amplifies the loss value of the entire sequence, causing the model to allocate more resources to optimizing the prediction results of that sequence during backpropagation; when G is close to 0, it indicates low context complexity. The loss value remains relatively stable when it approaches 1, avoiding overfitting caused by the model focusing too much on simple samples. Simultaneously, a token-level weighting factor is introduced based on locally predicted parameters: Let the first... The local prediction parameters corresponding to each token are: The value ranges from [0,1]; the preset coefficient is... Typically, it is set to 0.3-0.8, adjusted according to the importance of the token. The loss weighting factor for each token is:

[0096] ;

[0097] To address the need for higher learning weights for key tokens, Tokens close to 1, such as tokens representing a subset of the core semantics, Increasing the value of this token in the total loss increases the model's focus on optimizing the prediction accuracy of such tokens and reducing prediction errors for key semantics. Based on these adjustments, the final reconstructed loss function is:

[0098] ;

[0099] The derivation of this formula is based on the concept of dual-dimensional weight enhancement. First, it amplifies the overall loss of complex sequences through global weights, and then highlights the loss contribution of key tokens through local weights. The two work together to ensure that the loss function accurately matches the learning needs of different contexts and tokens. In the formula, This represents the final loss value after reconstruction. These are the global weighting coefficients. For the first Local weighting factors for each token.

[0100] Step 34: Guided by the reconstructed loss function, optimize and update the adapter parameters. Identify the update direction of key parameters based on the gradient distribution, prioritize the adjustment of parameter paths corresponding to high-complexity contexts, and control the update magnitude through gradient pruning mechanism while keeping the language model weights frozen. The specific operations are as follows:

[0101] First, strictly maintain the language model weights frozen. Throughout the training process, fix all parameters of the pre-trained language model, such as Sentence Transformers for semantic vector extraction and the Decoder-Only model for basic semantic understanding. Only the parameters of the cross-attention adapter, including the linear transformation matrix of the cross-attention query vector, the projection matrix of the key-value vector, and the bias terms, are considered as optimizable parameters. This operation aims to protect the general semantic understanding ability and domain knowledge already learned by the pre-trained model, avoid damaging the basic semantic modeling ability during adapter parameter optimization, and significantly reduce the number of trainable parameters, thereby reducing the risk of model overfitting and training computation costs. Second, identify the key parameter update direction based on gradient distribution: During each backpropagation gradient calculation, calculate the gradient norm of each adapter parameter, such as the L2 norm. Sort and select key parameters by the size of the gradient norm. The larger the gradient norm, the more significant the impact on the loss function after reconstruction. These parameters usually correspond to the model's learning path in highly complex contexts, such as the parameters for assigning cross-attention weights to handle polysemous terms. For key parameters, a priority update strategy is adopted: In the parameter optimization step, such as the Adam optimizer, a higher learning rate is assigned to key parameters, typically 1.2-1.5 times that of ordinary parameters, or key parameters are updated first in the parameter update order. This ensures that the model prioritizes solving adaptation problems in high-complexity contexts and quickly improves retrieval accuracy in complex scenarios. Simultaneously, a gradient pruning mechanism is introduced to control the parameter update magnitude: A threshold for the gradient norm is set, such as 1.0-2.0, and adjusted according to model convergence. After each gradient calculation, if the gradient norm of a parameter exceeds the threshold, the gradient is scaled proportionally to the gradient norm or the threshold, keeping the gradient norm within the threshold range. This mechanism effectively prevents gradient explosion caused by complex context samples during training. In high-complexity contexts, model prediction errors may be large, easily generating excessively large gradients. Gradient pruning can limit the gradient to a reasonable range, avoiding excessive parameter update magnitudes that could lead to model training oscillations or convergence difficulties. Finally, the loss function is reconstructed... Under the guidance of [the relevant authority], through multiple rounds of iterative execution, the process of forward calculation of loss, backpropagation to calculate gradient, priority update of key parameters, and gradient pruning to stabilize training is carried out until the loss value of the model on the validation set tends to stabilize and the retrieval accuracy reaches the preset index. The adapter parameters are then optimized and updated. At this point, the adapter can accurately realize the dynamic cross-attention calculation of the input sequence and atomic knowledge units, meeting the retrieval needs in complex contexts.

[0102] In a preferred embodiment of the present invention, step 4 further includes the following operation steps:

[0103] Step 41 involves identifying and filtering semantically redundant connections in the cross-attention matrix based on attention kurtosis. This includes analyzing the distribution characteristics of attention weights to identify anomalous connection points with high variance, and performing smoothing suppression on redundant connections. The specific operations are as follows:

[0104] First, attention weight distribution concentration is determined through attention kurtosis analysis. Attention kurtosis is a statistical indicator that measures the steepness of the weight distribution. A high kurtosis value indicates that attention weights are highly concentrated on a few key connections, i.e., knowledge unit connections that are semantically strongly related to the input sequence, with fewer redundant connections. A low kurtosis value indicates a flat and dispersed weight distribution, with many connections having weight values ​​in the middle range. Some of these connections may be redundant terms without actual semantic relevance, such as spurious connections caused by random noise. Based on the kurtosis analysis results, the distribution characteristics of attention weights are further analyzed to identify abnormal connection points with high variance: the variance of the weights in the row and column of each connection point in the cross-attention matrix is ​​calculated. If the weight value of a connection point deviates significantly from the mean of its row or column, such as exceeding the mean ± 2 standard deviations, then... Furthermore, if the corresponding variance value is higher than a preset threshold, which is determined statistically based on the weight distribution of the model training data, it is identified as an abnormal connection point. These connections usually originate from accidental semantic overlap between tokens and knowledge units, rather than genuine associations, and are considered redundant items that need to be filtered out. Finally, a smoothing suppression process is performed on the identified redundant connections: Gaussian smoothing filtering or weight threshold pruning strategies are adopted. For Gaussian smoothing, a local smoothing window is constructed, and the weight values ​​of redundant connection points are replaced with the average of the weights of the surrounding valid connections within the window, thus weakening the influence of redundant weights. For threshold pruning, a lower limit threshold for weights is set, and the weights of redundant connections below the threshold are directly set to 0, completely eliminating invalid connections. Through the above processing, redundant and abnormal connections in the cross-attention matrix are effectively filtered out, and the weight distribution is more focused on genuine semantic associations.

[0105] Step 42: Based on the purified cross-attention matrix, calculate the initial relevance score for each atomic knowledge unit, including the purified attention weights of all tokens in the aggregated input sequence, using the geometric mean aggregation function, and record the distribution characteristics of the attention tokens. The specific operations are as follows:

[0106] First, clarify the information of the purified cross-attention matrix, and the information of each element in the matrix. Represents the input sequence number 1 The token and the first The purified attention weights of each atomic knowledge unit, having eliminated redundant interference, directly reflect the strength of the basic semantic association between the two. To obtain the comprehensive association strength between a single atomic knowledge unit and the entire input sequence, the weights of all tokens in the input sequence for that knowledge unit need to be aggregated. Here, the geometric mean aggregation function is chosen instead of the arithmetic mean because the geometric mean is more sensitive to high weight values. If a knowledge unit has extremely high weights with a few core semantic tokens, such as predicates and agent tokens, but low weights with other auxiliary tokens, the geometric mean can effectively amplify the contribution of high-weight tokens, avoiding the arithmetic mean's flattening effect due to low-weight tokens masking the strong association between the knowledge unit and the core tokens. At the same time, the geometric mean can avoid the distortion of results caused by individual abnormally high weights, such as residual noise that has not been completely filtered, ensuring that the aggregation result is closer to the actual semantic association. Specifically, the attention weights of the first atomic knowledge unit are calculated as follows: The formula for calculating the initial relevance score of each atomic knowledge unit is:

[0107] ;

[0108] in For the first The initial relevance score of each knowledge unit, The formula represents the total number of tokens in the input sequence. It stems from the understanding that core tokens dominate semantic association. The strength of the association between the input sequence and knowledge units is primarily determined by the weights of the core tokens. The geometric mean, through product and square root operations, ensures that the contribution of high-weight tokens dominates the final score, while balancing the combined influence of multiple tokens. Compared to the arithmetic mean, its result more accurately reflects the true semantic association between knowledge units and the input sequence. While calculating the initial score, it simultaneously records the distribution characteristics of the tokens of interest: statistically analyzing the distribution of the tokens of interest... The position of a token in the input sequence whose weight value is higher than a preset association threshold, such as the position of the token at the beginning of the sequence. The middle After Semantic role labels, such as core semantic roles and auxiliary semantic roles, as well as weight contribution ratios, such as the proportion of the sum of the weights of high-weight tokens to the total weights of all tokens. These distribution features directly reflect which positions and types of tokens in the input sequence play a dominant role in the association of the knowledge unit; among them, the preset association threshold can be the average of all weights in the matrix.

[0109] Step 43: Utilize the distribution characteristics of the tokens of interest, and apply a preset sequence position weighting function to adjust the initial relevance score. This includes assigning importance coefficients based on the relative position of the tokens in the sequence, and assigning higher weight coefficients to tokens of key semantic nodes. The position-aware relevance score is obtained through weighted fusion. The specific operations are as follows:

[0110] First, the design logic of the pre-defined sequence position weight function is clarified. Tokens at different positions in the input sequence carry significantly different semantic functions. Typically, tokens at the beginning of the sequence summarize the theme or core requirements, such as the core question in a user's query; tokens in the middle of the sequence provide supplementary details or logical elaboration; and tokens at the end of the sequence often summarize or define limiting conditions, such as constraints in a query. Furthermore, the semantic importance of the core semantic role token is higher than that of auxiliary semantic role tokens regardless of its position. Therefore, the position weight function needs to consider both the relative position and semantic role of the tokens, constructing a two-dimensional weight coefficient system for relative positions. Non-linear functions, such as S-curves or piecewise linear functions, are used for assignment. For example, the sequence position is divided into the first 20%, the middle 60%, and the last 20%, and assigned basic weight coefficients of 0.8, 0.5, and 0.7 respectively. The specific weights can be adjusted according to the characteristics of the domain corpus. For semantic roles, a role weight coefficient of 1.2 is added to the core semantic role token, and 1.0 is added to the auxiliary semantic role token. The final position weight coefficient of a single token is: relative position basic coefficient × semantic role added coefficient. Subsequently, combined with the distribution characteristics of the attention tokens recorded in step 42, the initial relevance score is adjusted by weighted fusion: for the first... For each knowledge unit, firstly, extract the position weight coefficients of all followed tokens from the distribution characteristics, where all followed tokens have weights higher than the association threshold. Then, calculate the average of these coefficients as the mean position weight of the knowledge unit. Then, the initial correlation score was... and Perform a product operation to obtain the position-aware correlation score. For example, if the tokens of interest for a certain knowledge unit are mostly concentrated at the beginning of the sequence and mainly play core semantic roles, then... The possible score is 0.8 × 1.2 = 0.96. With an initial score of 0.6, the positional awareness score would be 0.6 × 0.96 = 0.576. If the tokens of another knowledge unit are concentrated in the middle of the sequence and mostly play auxiliary roles, The initial score is 0.6, so the position awareness score is 0.6 × 0.5 = 0.3. This adjustment logic can effectively distinguish between knowledge units where the token is located in a core position or role and those located in a non-core position or role. The former carries stronger semantic association evidence, so its position awareness score is strengthened, while the latter has weaker semantic association evidence, so its score is reasonably reduced, ensuring that the relevance score can accurately reflect the impact of the token's position and role on semantic association.

[0111] Step 44: Based on the location-aware relevance score, apply the contribution coefficient for final balancing. This includes dynamically adjusting the contribution coefficient according to the semantic complexity of the input sequence and the size of the knowledge base, and generating the final relevance score through a combination of preset nonlinear functions. The specific operations are as follows:

[0112] First, the dynamic adjustment mechanism of the contribution coefficient is clarified. The contribution coefficient C∈(0,1] is used to control the contribution ratio of atomic knowledge units to the relevance of the input sequence. Its value needs to be adjusted in conjunction with the semantic complexity of the input sequence and the size of the knowledge base. When the semantic complexity of the input sequence is high, such as containing ambiguous terms or complex logical relationships, the auxiliary role of external knowledge needs to be enhanced. In this case, C is adjusted towards 1.0. For example, when the complexity score is 0.8, C=0.9. When the size of the knowledge base is large, such as the total number of atomic knowledge units... To avoid overestimating the relevance scores of a large number of knowledge units, which could lead to search confusion, C needs to be appropriately reduced, such as in terms of scale. At this point, C=0.7. The adjustment of the contribution coefficient is achieved through a preset mapping function, which takes the semantic complexity score and the knowledge base size as input. The semantic complexity score is generated in step 31, and the real-time C value is calculated through linear interpolation to ensure that C can dynamically adapt to the knowledge injection needs in different scenarios. Subsequently, the final relevance score is constructed based on the contribution coefficient and the location-aware relevance score, and the two are fused using a preset nonlinear function. The specific formula is as follows:

[0113] ;

[0114] in For the first The final relevance score of each atomic knowledge unit, For non-linear activation functions, variants of the Sigmoid or Softmax functions are typically chosen, where C is the dynamically adjusted contribution coefficient. The formula for the location-aware relevance score is derived from the need to "balance knowledge contribution and semantic rationality": directly comparing C with... Linear multiplication may cause fractions to exceed a reasonable range, such as C=0.9 and... When the product is 0.81, if the scores of multiple knowledge units are close, it can easily lead to insufficient discrimination. However, nonlinear functions can map the fusion result to the [0,1] interval, and at the same time, the nonlinear characteristics of the function amplify the difference between high-scoring and low-scoring knowledge units, thereby improving the discrimination of retrieval recall; for example, the Sigmoid function can be used. The coefficient 5 is used to enhance the nonlinear discrimination. hour, ≈0.98, when hour, The relevance score is approximately 0.62, which significantly widens the gap. Through the above processing, the final relevance score balances the relationship between knowledge injection and input context, and has clear distinguishability. It can be directly used for subsequent TopK retrieval and retrieval, ensuring that the recalled knowledge units accurately match the semantic requirements of the input sequence.

[0115] In a preferred embodiment of the present invention, step 5 further includes the following operation steps:

[0116] Step 51: Construct a weighted directed graph based on relevance scores and semantic overlap, where nodes represent atomic knowledge units. Node weights are assigned after normalization using relevance scores, and edge weights are calculated by measuring the cosine similarity of the knowledge unit key vectors and combining it with contextual dependencies. The specific operations are as follows:

[0117] First, the node definition of the graph is determined, and each atomic knowledge unit is directly used as an independent node of the graph. The core attribute of the node is the weight, which is generated by normalizing the knowledge unit relevance score obtained in step 4. Considering that the absolute values ​​of the relevance scores of different knowledge units vary greatly, such as some scores being in the range of 0.1-0.3 and some in the range of 0.6-0.8, directly using the original scores will lead to inconsistent weight scales, affecting the accuracy of subsequent graph calculations. Therefore, the Min-Max normalization method is used to map the relevance scores of all knowledge units to the range of [0,1]. The normalized value is the weight of the corresponding node. The closer the weight is to 1, the closer the semantic association between the knowledge unit and the input sequence, and the higher its importance in the graph.

[0118] Next, the directed edges and edge weights of the graph are constructed. The calculation of edge weights needs to combine two core dimensions of semantic overlap: the cosine similarity of the key vectors of knowledge units and the contextual dependency. First, the cosine similarity of the key vectors of any two knowledge units is calculated. This similarity can quantify the semantic distance between the two in a unified semantic space. The higher the similarity, the closer their core semantics are, and this serves as the basic value for the edge weight. Then, the contextual dependency between the two knowledge units in the original knowledge base document is analyzed in depth. If the two are in adjacent paragraphs or sentences in the document and have a clear logical connection, such as causal relationship or explanatory relationship, it can be judged by analyzing the conjunctions in the document, such as "therefore," "that is," and "for example," indicating that the two have a strong dependency in the actual knowledge system. To enhance the semantic relevance of the knowledge units, the basic edge weights need to be multiplied by a factor of 1.2 to reflect the semantic connection brought about by the context. If two knowledge units have no direct adjacent relationship in the document and their logical connection is weak, the basic edge weights remain unchanged. Simultaneously, the direction of the edges is determined based on the semantic support capability of the knowledge units: comparing the semantic interpretation capabilities of two knowledge units for the input sequence, indirectly reflected by node weights, indicates that higher node weights indicate stronger interpretation capabilities. A directed edge is formed from the knowledge unit with stronger semantic support capability to the knowledge unit with weaker support capability. For example, if the node weight of knowledge unit A is 0.9 and the node weight of knowledge unit B is 0.6, and A and B have a semantic connection, then a directed edge is constructed from A to B, with the edge weight being the value adjusted for context. In this way, the final weighted directed graph reflects both the importance of the knowledge units themselves (represented by node weights) and the strength and direction of the semantic connection between units (represented by edge weights), providing comprehensive structural and semantic information for graph diffusion retrieval.

[0119] Step 52: Perform multi-round graph diffusion processing on the weighted directed graph. During the diffusion process, each node calculates its attention score based on the edge weights and its own node weights, and dynamically adjusts the information transmission intensity. The specific operations are as follows:

[0120] By performing multi-round graph diffusion processing on the weighted directed graph constructed in step 51, the dynamic transmission and aggregation of knowledge unit information is realized, which strengthens the weight of core knowledge units and reasonably adjusts the weight of peripheral knowledge units, thereby more clearly highlighting knowledge units that are strongly related to the input sequence. The logic of graph diffusion is to simulate the process of knowledge propagation between related units. Each round of diffusion calculates the attention score based on the node weight and edge weight, dynamically controlling the intensity of information transmission and ensuring that information is concentrated on core nodes with close semantic association. Each round of diffusion begins with the information reception and aggregation of each node: for the current node, firstly, all directed edges pointing to itself are collected, that is, edges from other nodes to this node. The edge weight corresponding to each edge and the current node weight of the starting node of the edge are extracted. The edge weight and the starting node weight are multiplied to obtain the preliminary intensity of the information transmission from the starting node to the current node. This calculation logic is based on the understanding that the intensity of information transmission is jointly determined by the degree of association and the value of information. The degree of association is represented by the edge weight, and the value of information is represented by the starting node weight. A large edge weight indicates a close association, and a large starting node weight indicates that the information is valuable. The product of the two can accurately reflect the actual utility of information transmission. Subsequently, the initial transmission intensities of all points to the current node are summed, and the sum is divided by the sum of the individual initial transmission intensities to obtain the attention score of the starting node to the current node. This score represents the contribution ratio of the starting node's information to the current node's aggregation process. Based on the attention scores of all starting nodes, the current node performs weighted aggregation of the semantic information of each starting node, such as core semantic words and logical relationships, and then updates its own node weight. Starting nodes with higher attention scores have a higher proportion of information in the aggregation and have a greater impact on the update of the current node's weight.

[0121] During the multi-round diffusion process, the intensity of information transmission is dynamically adjusted based on the attention score of each round: if the attention score of a starting node to the current node is higher than the preset strong contribution threshold for two consecutive rounds, such as 0.3, it indicates that there is a stable and strong semantic relationship between the starting node and the current node. In subsequent rounds, the transmission coefficient of the edge weight between the two will be appropriately increased, such as multiplied by 1.1, to further strengthen the information transmission of the path; if the attention score is lower than the weak contribution threshold for two consecutive rounds, such as 0.1, it indicates that the information value of the starting node to the current node is low, and the transmission coefficient will be reduced, such as multiplied by 0.8, to weaken the information transmission. Through 3-5 rounds of dynamic diffusion, the number of rounds is set according to the size of the knowledge base. The larger the scale, the more rounds are required to ensure that the information is fully disseminated. The node weight of the core knowledge unit will gradually accumulate and increase, while the weight of the peripheral knowledge unit will gradually decrease, eventually forming a hierarchical node weight distribution.

[0122] Step 53: Based on the diffused node scores, an adaptive threshold mechanism based on the statistical characteristics of score distribution is used to extract a subset of atomic knowledge units, and the subset is subjected to redundancy filtering. The specific operations are as follows:

[0123] First, an adaptive threshold screening based on the statistical characteristics of the score distribution is performed. A fixed threshold, such as 0.5, has the drawback of not being able to adapt to the node weight distribution corresponding to different input sequences. When most node weights are low, such as concentrated in the 0.2-0.4 range, a fixed threshold will result in no nodes being selected; when most node weights are high, such as concentrated in the 0.6-0.8 range, a fixed threshold will filter out a large number of marginal nodes. Therefore, the complete distribution characteristics of the scores of all atomic knowledge unit nodes after diffusion are first statistically analyzed, focusing on calculating the mean μ and standard deviation of the scores. The mean reflects the overall level of the node scores, while the standard deviation reflects the dispersion of the scores. The adaptive threshold is set to μ+1.5. The logic behind selecting this threshold is to ensure that the scores of the selected nodes are significantly higher than the overall average, exceeding the mean by 1.5 standard deviations, thus belonging to core knowledge units; while avoiding an excessively high threshold due to an excessively large standard deviation, which would cause some important nodes to be missed. Based on this threshold, knowledge units with node scores higher than the threshold are selected to form an initial subset. For example, the mean of a batch of node scores might be μ=0.4, and the standard deviation might be... =0.2, and the threshold is 0.4 + 1.5 × 0.2 = 0.7. Only nodes with scores higher than 0.7 are retained to ensure the coreness of the initial subset.

[0124] Subsequently, redundancy filtering is performed on the initial subset to avoid redundant information generated later due to highly overlapping semantic knowledge units within the subset. The core criterion for redundancy judgment is the semantic overlap between knowledge units, which is achieved by calculating the Jaccard similarity of the core semantic vocabulary sets corresponding to the key vectors of any two knowledge units within the subset: first, the core semantic vocabulary corresponding to the key vector of each knowledge unit is extracted, and the top 10 words with the highest weights are selected using a keyword extraction algorithm such as TF-IDF to obtain two vocabulary sets; then, the Jaccard similarity of the sets is calculated, which is the intersection size of the two sets divided by the union size. The higher the similarity, the stronger the overlap between the two knowledge units. The more overlapping the core semantics of a knowledge unit, the higher the redundancy. A redundancy threshold of 0.7 is set. If the Jaccard similarity between two knowledge units is higher than 0.7, it is considered redundant. At this point, the node scores of the two knowledge units are compared, and the knowledge unit with the higher score is retained. A higher score means that its semantic association with the input sequence is closer and its information value is higher. Knowledge units with lower scores are eliminated to avoid semantic repetition. Through the collaborative processing of adaptive threshold screening and redundancy filtering, the final subset of atomic knowledge units contains core knowledge that is strongly related to the input sequence and has no semantic redundancy, which can provide high-quality and efficient knowledge support for subsequent generation stages.

[0125] In a preferred embodiment of the present invention, step 6 further includes the following operation steps:

[0126] Step 61: Monitor changes in the generated context state, including calculating the semantic completeness of the generated sequence and the dispersion of the predicted probability distribution, and quantifying the degree of uncertainty. The specific operations are as follows:

[0127] First, calculate the semantic completeness of the generated sequence: using the input sequence, such as a user query, as a benchmark, the core semantic elements extracted from it, such as key issues, requirement constraints, and domain-specific terms, are used. Semantic matching algorithms, such as element coverage calculation based on the bag-of-words model or element alignment analysis based on semantic vectors, are employed to statistically analyze the percentage of core semantic elements covered in the generated sequence. Simultaneously, the logical coherence of the generated content is considered by analyzing the completeness of causal and progressive logical relationships between statements, such as the presence of unexplained concepts or incomplete logical derivations. A semantic completeness scoring model is constructed, with the output score ranging from [0,1]. A score closer to 1 indicates a more comprehensive and logically coherent coverage of the core input requirements; conversely, a lower score indicates semantic gaps or logical discontinuities in the generated content, potentially requiring external knowledge supplementation. Second, calculate the dispersion of the predicted probability distribution: focusing on the model's prediction of the next token to be generated, the model's output is extracted. The probability distribution of all candidate tokens is analyzed using information entropy as a quantitative indicator of dispersion. A higher entropy value indicates a more dispersed probability distribution of candidate tokens, suggesting greater uncertainty about the model's generation direction for the next token. Conversely, a lower entropy value indicates that the probability is concentrated in a few candidate tokens, indicating a clear generation direction. Finally, uncertainty is quantified based on semantic completeness and predicted probability dispersion: a fusion function is constructed to weight the semantic completeness score and the predicted probability dispersion (i.e., entropy). For example, when semantic completeness is below 0.5 and dispersion is high, the entropy value is greater than a preset threshold, indicating high uncertainty in the current context. This means the model neither meets core semantic requirements nor has a clear generation direction, requiring knowledge injection. Conversely, when semantic completeness is high (greater than 0.8) and dispersion is low, the entropy value is less than a preset threshold, indicating low uncertainty and stable model generation, requiring no additional knowledge injection. This multi-dimensional quantification method allows for real-time and objective capture of the dynamic changes in the generated context.

[0128] Step 62: Based on the generated context state, calculate the semantic matching degree of the atomic knowledge units, including direct semantic similarity and semantic coherence with subsequent generation directions. Use bidirectional semantic projection to evaluate the degree of fit. The specific operations are as follows:

[0129] First, direct semantic similarity is calculated by comparing the semantic vectors of the generated sequence with the key vectors of the atomic knowledge units. The key vectors, encoded by a pre-trained language model, integrate the overall semantic information of the generated content and are generated in step 13 to represent the core semantics of the knowledge unit. Additionally, cosine similarity is commonly used as a metric; a higher similarity score indicates a greater overlap between the core semantics of the knowledge unit and the semantics of the currently generated sequence, resulting in stronger direct fit. For example, if the generated sequence revolves around "contraindications to diabetes medication use," and the key vector of a certain knowledge unit corresponds to "contraindications for oral hypoglycemic agents in type 2 diabetes," then the direct semantic similarity between the two will be significantly higher than that of the knowledge unit corresponding to "hypertensive drug dosage adjustment." Second, semantic coherence with subsequent generation directions is evaluated: based on the currently generated sequence, the generation model predicts future... The semantic vectors of 1-3 tokens to be generated are used, which are the semantic representations of the predicted generation direction. Then, the similarity between the key vector of the atomic knowledge unit and the predicted semantic vector is calculated. This similarity reflects the support ability of the knowledge unit for the subsequent generation direction. A high similarity indicates that the semantics of the knowledge unit can be naturally connected to the subsequent generated content, avoiding semantic jumps. A low similarity may lead to logical breaks in the generated content. Finally, bidirectional semantic projection is used to evaluate the degree of fit: considering that there may be local semantic deviations between the generated sequence semantic vector and the knowledge unit key vector, such as different expressions of the same concept leading to vector differences, the two types of vectors are projected onto a preset shared semantic subspace, which is obtained through pre-training on domain corpus. This can eliminate the vector offset caused by expression differences. The cosine similarity of the projected vectors is calculated as a correction and supplement to the direct semantic similarity and subsequent coherence. The calculation results of the above three dimensions are combined and weighted, such as direct semantic similarity weight of 0.5, subsequent coherence weight of 0.3, and bidirectional projection fit weight of 0.2, to obtain the final semantic matching score. The score takes the value of [0,1], which provides a basis for subsequent gating function decisions.

[0130] Step 63: Based on the semantic matching evaluation results, execute the dynamic gating function to adjust the intensity ratio and timing of knowledge injection according to the matching score and the degree of contextual uncertainty. The specific operations are as follows:

[0131] First, the logic of the dynamic gating function is defined. The output of the gating function is the knowledge injection weight, represented by the control strength ratio, and the injection trigger signal, represented by the control timing. Its decision-making follows the business rule that the higher the semantic matching degree, the higher the contextual uncertainty, and the stronger the knowledge injection demand, to avoid semantic redundancy caused by indiscriminate injection or generation defects caused by the lack of injection at key nodes. The calculation of the knowledge injection weight adopts the Sigmoid activation function to construct the formula: ;

[0132] Where g is the knowledge injection weight, with a value of [0,1]; The Sigmoid function maps the input to the [0,1] interval, ensuring that the weights are controllable; The semantic matching score. To account for the degree of uncertainty in the context, and The balancing coefficient is preset to a range of [0.8, 1.2] and is optimized using domain training data to balance the impact of matching degree and uncertainty on the weights. The derivation of this formula stems from the fact that knowledge injection needs are a synergistic result of matching degree and uncertainty. Matching degree determines whether knowledge is suitable, while uncertainty determines whether knowledge is needed. The two integrate the core influencing factors through a linear combination. The Sigmoid function ensures that the weight output is smooth and within a reasonable range through non-linear mapping, avoiding injection imbalance caused by extreme values. g directly controls the proportion of semantic information of knowledge units integrated into the generation process. When g is close to 1, the semantic information of knowledge units is deeply integrated into the generation. When g is close to 0, only a small amount or no knowledge is integrated. Secondly, the timing of knowledge injection is determined: the gating function decides the timing by monitoring changes in the injection trigger signal. When g exceeds the preset injection threshold for two consecutive generation steps, such as 0.6, and the currently generated token is a logical node, such as the first token after conjunctions like "for example" or "therefore," or the starting token of a domain concept explanation, knowledge injection is triggered. At this time, the injected knowledge can naturally connect with the current generation logic, avoiding interruption of the semantic flow. If g is below the threshold or the current token is a non-logical node, such as a modifying adjective or punctuation mark, the injection is delayed until the trigger condition is met. Through this dynamic gating mechanism that controls the strength of the weight and the timing of the signal, deep collaboration between knowledge injection and the generation process can be achieved. This ensures that knowledge is accurately supplemented when needed, while avoiding excessive injection that interferes with the continuity of generation, ultimately improving the quality and naturalness of knowledge enhancement generation.

[0133] In a preferred embodiment of the present invention, step 7 further includes the following operation steps:

[0134] Step 71: Extract the semantic consistency, factual accuracy, and logical coherence quality indicators of the generated results; parse the explicit ratings and implicit behavioral data of user feedback; calculate the feedback confidence score; and establish a correlation mapping between the quality indicators and the feedback confidence score to generate a comprehensive evaluation signal. The specific operations are as follows:

[0135] First, three quality indicators are extracted from the generated results. The semantic consistency indicator is obtained by encoding the semantic vector of the generated results using a pre-trained language model, and then calculating the cosine similarity between this cosine similarity and the semantic vector of the input sequence, as well as the semantic vector of the injected atomic knowledge units. The average of these two values ​​is taken as the quantification result. The closer this value is to 1, the smaller the semantic deviation between the generated content and the input requirements and injected knowledge, and the stronger the consistency. The factual accuracy indicator is obtained by comparing factual statements in the generated results, such as domain data, concept definitions, and rule descriptions, with authoritative atomic knowledge units in the knowledge base sentence by sentence. The percentage of statements without factual bias is considered, with a higher percentage indicating higher accuracy. If factual conflicts exist, such as data errors or conceptual confusion, the percentage is reduced according to the severity of the conflict, such as a 10% reduction for minor conflicts and a 50% reduction for severe conflicts. The logical coherence index analyzes the logical relationships between statements in the generated results, such as causality, progression, parallelism, and contrast. A logical relationship identification model is used to quantify the completeness and rationality of the logical chain. For example, if there are unexplained causal breakpoints or contradictory progressive relationships, the coherence score decreases, and vice versa. All three indicators are eventually normalized to the [0,1] interval.

[0136] Secondly, analyze user feedback data to distinguish between explicit ratings and implicit behavioral data: Explicit ratings directly extract users' star ratings of the generated results, such as 1-5 stars, or satisfaction scores, such as 0-10 points, which are normalized and used as explicit feedback values; Implicit behavioral data is obtained by analyzing users' interaction with the generated results, such as reading time. Reading time exceeding 1.5 times the average reading time is considered positive feedback, while reading time less than 0.5 times is considered negative feedback; clicking to view details, supplementary explanations, etc., are considered positive feedback; modifying the original query and resubmitting is considered negative feedback on the current generated result. All implicit behaviors are assigned preset weights, such as positive behavior weight +0.2 and negative behavior weight -0. 1. Quantify implicit feedback values. Then, calculate the feedback confidence score, which measures the reliability of user feedback: For explicit ratings, if a user submits multiple ratings under the same query and the difference is less than a preset threshold (e.g., 0.1), or if the user is a highly trustworthy user marked by the system (e.g., a domain expert), the confidence score increases, up to a maximum of 1.0; if the rating fluctuates greatly or the user has no historical trustworthy interaction records, the confidence score decreases, down to a minimum of 0.3; For implicit behavioral data, if multiple implicit behaviors point to a consistent feedback direction (e.g., long reading time plus clicking details are both positive), the confidence increases; if the behaviors are contradictory (e.g., long reading time but resubmitting the query), the confidence decreases.

[0137] Finally, a correlation mapping between quality indicators and feedback confidence levels is established to generate a comprehensive evaluation signal, using a weighted fusion formula: ;

[0138] in To comprehensively evaluate the signal, the value is [0,1]. To calculate the weighted average of the three quality metrics for the generated results, the weights for semantic consistency, factual accuracy, and logical coherence were set to 0.3, 0.4, and 0.3, respectively. For normalized explicit feedback values, For explicit feedback confidence scores. For normalized implicit feedback values, This represents the implicit feedback confidence score. and For the balance coefficient, satisfying , preset , This formula can be optimized according to specific domain scenarios. Its derivation stems from the need to consider both objective system quality and subjective user feedback. Quality indicators reflect the objective performance of the generated results, while feedback data reflects the user's subjective experience. By merging the two through confidence-weighted averaging, biases arising from single-dimensional evaluations can be avoided, such as high objective quality but poor user experience, or vice versa. The synergistic effect of the parameters in the symbols ensures… It can not only reflect the technical performance of the generated results, but also meet the actual needs of users, providing a comprehensive evaluation basis for subsequent parameter adjustments.

[0139] Step 72: Based on the comprehensive evaluation signal and confidence weight, the dynamic priority weight and contribution coefficient are incrementally adjusted. A reinforcement learning optimization or smoothing adjustment strategy is adopted according to the confidence score, and a tracking mechanism for the parameter update trajectory is established. The specific operations are as follows:

[0140] First, clarify the core logic of incremental adjustments: comprehensive evaluation of signals. If the value is higher than the preset excellent threshold, such as 0.8, it indicates that the current dynamic priority weights and contribution coefficients are well-suited to the current scenario, and only minor adjustments are needed based on newly generated results and feedback to maintain system stability; when... If the value is below the preset acceptable threshold, such as 0.6, it indicates a significant parameter mismatch issue, and further adjustments are needed. Targeted adjustments should be made based on the identified defects, such as... Low accuracy stems from poor factual accuracy, potentially due to biases in high-priority knowledge units; it also stems from semantic redundancy, possibly due to excessively high contribution coefficients; when... When the value is between the two thresholds, make a moderate adjustment according to the degree of defect.

[0141] Secondly, a differentiated adjustment strategy is selected based on the feedback confidence score: when the feedback confidence score is higher than the high confidence threshold, such as 0.8, a reinforcement learning optimization strategy is adopted to comprehensively evaluate the signal. As a reward signal, construct a reward function:

[0142] ;

[0143] in Adjust the penalty coefficient for the parameter range. To adjust the parameters and avoid over-adjustment, if R is positive, it indicates that the current parameter adjustment direction is correct, and optimization will continue in that direction in the next round of adjustment, such as further increasing the dynamic priority weight of a certain knowledge unit. If R is negative, the adjustment will be reversed to correct the deviation. The effective adjustment direction will be reinforced through a reward mechanism to accelerate parameter convergence. When the feedback confidence score is lower than the low confidence threshold, such as 0.5, a smooth adjustment strategy will be adopted—limiting the parameter adjustment range to a preset small range, such as the dynamic priority weight not exceeding ±0.1 and the contribution coefficient not exceeding ±0.05 each time. The moving average algorithm, such as the average of the last 3 evaluation signals, will be used to filter out noise interference from low confidence feedback, avoid large fluctuations in parameters due to unreliable feedback, and ensure a smooth adjustment process.

[0144] In terms of specific parameter adjustments, the dynamic priority weighting focuses on atomic knowledge units that are recognized by both user feedback and quality indicators. Knowledge units that contribute significantly to the generated results, such as those with high factual accuracy or those linked to user clicks to view details, are prioritized based on comprehensive evaluation signals. The product of feedback confidence score and the weight of the knowledge unit is increased. For knowledge units that cause generation defects, such as factual errors or semantic redundancy, their weight is reduced according to the degree of the defect. The contribution coefficient is adjusted according to the knowledge dependence of the generated result: if there is excessive knowledge injection in the generated result, such as semantic redundancy or deviation from the input requirements, the contribution coefficient is reduced to reduce the impact of knowledge on generation; if there is insufficient knowledge injection, such as missing facts or semantics and knowledge disconnect, the contribution coefficient is increased to strengthen knowledge support. Finally, a parameter update trajectory tracking mechanism is established: the timestamp of each parameter adjustment, the comprehensive evaluation signal value that triggers the adjustment, the feedback confidence score, the parameter value before adjustment, the adjustment range, the adjustment strategy type, and the change of quality indicators of the generated result after adjustment are recorded to form a complete parameter adjustment log. This log can be used to trace back the parameter optimization process, analyze the effectiveness of parameter adjustment in different scenarios, provide data support for subsequent optimization strategy iterations, and ensure the reproducibility and interpretability of parameter adjustment.

[0145] The above description is merely a preferred embodiment of the present invention; however, the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and its improved concepts, should be covered within the scope of protection of the present invention.

Claims

1. A dynamic feature retrieval generation and management method based on a cross-attention mechanism, characterized in that, The method comprises the following steps: Step 1: converting the knowledge base document fragments into atomic knowledge units, each atomic knowledge unit comprising a question-answer pair, encoded as a key-value pair, and attached with a dynamic priority weight; Step 2: replacing the self-attention query with a dual-channel query structure, one channel for cross-attention, generating a cross-attention query vector through linear transformation, using the key-value pair of the atomic knowledge unit, and calculating the cross-attention matrix according to the cross-attention query vector and the key-value pair of the atomic knowledge unit; Step 3: training the cross-attention adapter, freezing the language model weight, optimizing the adapter parameter, and dynamically adjusting the loss function based on the input sequence context complexity; Step 4: based on the cross-attention matrix, calculating the relevance score of the input sequence and each atomic knowledge unit, weighting and fusing the attention score through a preset sequence position weight function, and applying a contribution coefficient; Step 5: based on the relevance score, constructing a knowledge topology network based on the relevance score and semantic overlap, and retrieving the atomic knowledge unit subset through a graph diffusion algorithm; Step 6: in the generation process, using a gating function based on the matching degree of the current generation context and the atomic knowledge unit to dynamically control knowledge injection; Step 7: based on the generation result and user feedback, adjusting the dynamic priority weight and contribution coefficient, and updating the parameters using incremental learning. 2.The cross-attention mechanism based dynamic feature retrieval generation and management method according to claim 1, characterized in that, The step 1 comprises: Step 11: based on the document logical structure and semantic density, identifying the semantic unit boundary by analyzing the punctuation symbol density, term frequency, and sentence complexity; Step 12: within the semantic unit, analyzing the logical relationship between sentences, identifying question-answer, argument-evidence, and problem-solution semantic patterns, extracting key propositions as questions and associating supporting content as answers, forming question-answer pairs with structure labels; Step 13: dual-channel encoding of question-answer pairs, encoding question text into distinguishability vectors and answer text into capacity vectors, and mapping to a unified semantic space through projection transformation to form key-value pairs; Step 14: using the key-value semantic vector, calculating the semantic similarity between atomic knowledge units, constructing a knowledge unit association network, and calculating the initial priority weight based on node centrality indicators and connection density. 3.The cross-attention mechanism based dynamic feature retrieval generation and management method according to claim 2, characterized in that, The step 2 comprises: Step 21: performing semantic role analysis on each token in the input sequence, assigning semantic role labels, and assigning two query dimension spaces to each token based on the semantic role labels, including an internal semantic association channel and an external knowledge interaction channel; Step 22: using semantic role labels and query dimension spaces, performing channel-specific transformation on the initial representation of each token, where the internal semantic association channel uses linear transformation and the external knowledge interaction channel uses nonlinear transformation, and the transformation matrix is parameter-shared grouped according to the semantic role labels. 4.The method of claim 3, wherein, The step 2 further comprises: Step 23: performing semantic focusing processing on the external knowledge interaction channel query vector, including calculating the initial relevance distribution with the knowledge unit key vector, identifying the semantic focus area, and performing local semantic enhancement transformation; Step 24, dynamically fuse the internal semantic association channel output and the enhanced cross-attention channel output, calculate the fusion weight based on the current decoding state and the input sequence semantic complexity. 5.The cross-attention mechanism based dynamic feature retrieval generation and management method according to claim 4, characterized in that, The step 3 includes: Step 31, calculate the semantic distribution dispersion and context dependence intensity of the tokens in the input sequence, and generate a context complexity score; Step 32, use the context complexity score to generate a global pre-judgment parameter at the sequence level and a local pre-judgment parameter at the token subset level, and fuse the multi-scale pre-judgment parameters through a nonlinear combination function; Step 33, based on the multi-scale pre-judgment parameters, adjust the weight coefficient and weighting factor of the standard loss function, use the global pre-judgment parameter to adjust the overall weight coefficient of the loss function, and use the local pre-judgment parameter to introduce the token-level weighting factor; Step 34, under the guidance of the reconstructed loss function, optimize and update the adapter parameters, identify the key parameter update direction according to the gradient distribution, preferentially adjust the parameter path corresponding to the high complexity context, and control the update amplitude through the gradient clipping mechanism, while keeping the language model weight frozen. 6.The cross-attention mechanism based dynamic feature retrieval generation and management method according to claim 5, characterized in that, The step 4 includes: Step 41, based on the attention kurtosis, identify and filter the semantic redundant connections in the cross-attention matrix, including analyzing the distribution characteristics of the attention weight to identify abnormal connection points with high variance values, and performing smoothing suppression processing on the redundant connections; Step 42, based on the purified cross-attention matrix, calculate the initial relevance score of each atomic knowledge unit, including aggregating the purified attention weight of all tokens in the input sequence, using a geometric mean aggregation function, and recording the distribution characteristics of the tokens being concerned. 7.The cross-attention mechanism based dynamic feature retrieval generation and management method according to claim 6, characterized in that, The step 4 also includes: Step 43, use the distribution characteristics of the tokens being concerned to apply a preset sequence position weight function to adjust the initial relevance score, including assigning importance coefficients according to the relative position of the tokens in the sequence, and assigning higher weight coefficients to the tokens of key semantic nodes, and obtaining the position-aware relevance score through weighted fusion; Step 44, based on the position-aware relevance score, apply a contribution coefficient for final balancing processing, including dynamically adjusting the contribution coefficient according to the input sequence semantic complexity and the knowledge base size, and generating the final relevance score through a preset nonlinear function combination. 8.The cross-attention mechanism based dynamic feature retrieval generation and management method according to claim 7, characterized in that, The step 5 includes: Step 51, based on the relevance score and semantic overlap, construct a weighted directed graph, where the nodes represent atomic knowledge units, the node weights are assigned after normalization of the relevance score, and the edge weights are calculated by measuring the cosine similarity of the knowledge unit key vectors and combining the context dependence relationship; Step 52, perform multi-round graph diffusion processing on the weighted directed graph, and during the diffusion process, each node calculates an attention score based on the edge weight and its own node weight, and dynamically adjusts the information transmission intensity; Step 53, based on the node scores after diffusion, use an adaptive threshold mechanism based on score distribution statistical characteristics to extract a subset of atomic knowledge units, and perform redundancy filtering on the subset. 9.The cross-attention mechanism based dynamic feature retrieval generation and management method according to claim 8, characterized in that, The step 6 includes: Step 61, monitor the generation context state change, including calculating the semantic completeness of the generated sequence and the dispersion of the predicted probability distribution, quantifying the degree of uncertainty; Step 62, based on the generation context state, calculate the semantic matching degree of the atomic knowledge unit, including direct semantic similarity and semantic coherence with the subsequent generation direction, use bidirectional semantic projection to evaluate the adaptation degree; Step 63, based on the evaluation results of semantic matching degree, execute the dynamic gating function, adjust the strength ratio and injection timing of knowledge injection according to the matching degree score and the degree of context uncertainty. 10.The cross-attention mechanism based dynamic feature retrieval generation and management method according to claim 9, characterized in that, The step 7 includes: Step 71, extract the semantic consistency, factual accuracy and logical coherence quality indicators of the generation result, analyze the explicit score and implicit behavior data of the user feedback, calculate the feedback confidence score, and establish the association mapping of quality indicators and feedback confidence to generate a comprehensive evaluation signal; Step 72, based on the comprehensive evaluation signal and the confidence weight, incrementally adjust the dynamic priority weight and contribution coefficient, use reinforcement learning type optimization or smooth adjustment strategy according to the confidence score, and establish a tracking mechanism for parameter update trajectory.

Images