Dynamic feature retrieval generation and management method based on cross attention mechanism

Abstract

The invention discloses a dynamic feature retrieval generation and management method based on a cross attention mechanism, which belongs to the technical field of natural language processing, and comprises the following steps: step 1, converting knowledge base document fragments into atomic knowledge units, each atomic knowledge unit comprising question and answer pairs, codes as key value pairs, and adding dynamic priority weights; self-attention query is replaced with a double-channel query structure, one channel is used for cross attention, a cross attention query vector is generated through linear transformation, and key value pairs of atomic knowledge units are used; and step 3, training a cross attention adapter, freezing the weight of the language model, optimizing parameters of the adapter, and dynamically adjusting a loss function by using pre-judgment parameters based on input sequence context complexity. By means of the method, deep correlation description of user input and knowledge fragments can be achieved, semantic ambiguity is eliminated, knowledge injection and context information are balanced, distortion is avoided, and the strict requirement for semantic precision in the professional field is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, more particularly, to a dynamic feature retrieval generation and management method based on a cross-attention mechanism. BACKGROUND

[0002] With the gradual deepening of the application of large language models (LLM) in text generation, dialogue interaction, knowledge question and answer, etc. scenarios, its generalization ability has been verified, but in the context of enterprise private domain corpus which requires specific knowledge support, LLM is difficult to meet the high-precision semantic understanding and generation requirements due to the lack of targeted knowledge modeling. Therefore, the retrieval augmented generation (RAG) technology has become a solution. The idea is to recall relevant corpus fragments from an external knowledge base and use them together with user input as the basis for LLM generation.

[0003] However, traditional RAG methods rely on static semantic similarity to achieve retrieval, first converting corpus fragments into vectors, and then sorting and recalling by calculating the similarity between the user input vector. This approach has obvious limitations in practical applications. When faced with complex contexts or ambiguous expressions, the retrieval accuracy is greatly limited. For example, in the medical field, users may ask about "postoperative complication treatment". The types of complications and coping strategies for different surgeries are significantly different. Static vectors cannot adjust the retrieval direction according to the real-time context of "specific surgery type". They cannot adapt to the dynamic context features of user input, cannot resolve semantic ambiguity, and cannot capture the deep semantic relationships between texts, resulting in retrieval results often deviating from the user's true intention.

[0004] From a technical perspective, these problems are due to the inherent limitations of the static vector similarity calculation paradigm, which cannot effectively model the dynamic semantic interaction between user input and knowledge fragments. The retrieval process of traditional RAG is fixed, regardless of how the user input context changes, the extraction rules and matching logic of the retrieval features remain unchanged, which cannot accurately capture fine-grained semantic relevance, ultimately limiting the accuracy of the retrieval results from the bottom up.

[0005] In the context of ambiguous expressions in private domain corpus such as finance and medicine, the corpus in such scenarios contains a large number of domain-specific terms, and ambiguous expressions are common. For example, "risk hedging" in the financial field has different meanings in the stock and futures scenarios. However, traditional RAG, in pursuit of retrieval efficiency, uses fast vector similarity calculation to achieve efficient recall of large-scale corpus, often simplifying the semantic modeling process without analyzing the context dependence between sentences or analyzing the deep intent of user queries in detail. This design, which prioritizes efficiency over accuracy, makes it difficult to balance retrieval efficiency and semantic accuracy, limiting its application in high-precision fields. SUMMARY

[0006] In view of the problems in the prior art, the purpose of the present application is to provide a dynamic feature retrieval generation and management method based on a cross-attention mechanism, which can realize deep association description of user input and knowledge fragments, eliminate semantic ambiguity, balance knowledge injection and context information, avoid generation distortion, and meet the stringent requirements of professional fields for semantic accuracy.

[0007] To solve the above problems, the present application adopts the following technical solutions:

[0008] In a first aspect, a dynamic feature retrieval generation and management method based on a cross-attention mechanism includes:

[0009] Step 1: Convert the knowledge base document fragments into atomic knowledge units, each atomic knowledge unit including a question-answer pair, encoded as a key-value pair, and with a dynamic priority weight attached;

[0010] Step 2: Replace the self-attention query with a dual-channel query structure, one channel of which is used for cross-attention, generating a cross-attention query vector through linear transformation, using the key-value pair of the atomic knowledge unit, and calculating according to the cross-attention query vector and the key-value pair of the atomic knowledge unit to obtain a cross-attention matrix;

[0011] Step 3: Train the cross-attention adapter, freeze the language model weight, optimize the adapter parameter, and dynamically adjust the loss function using the pre-judgment parameter based on the context complexity of the input sequence;

[0012] Step 4: Based on the cross-attention matrix, calculate the relevance score of the input sequence and each atomic knowledge unit, weight and fuse the attention score through a preset sequence position weight function, and apply a contribution coefficient;

[0013] Step 5: Based on the relevance score, construct a knowledge topology network based on the relevance score and semantic overlap degree, and retrieve the atomic knowledge unit subset through a graph diffusion algorithm;

[0014] Step 6: In the generation process, use a gating function based on the matching degree of the current generation context and the atomic knowledge unit to dynamically control knowledge injection;

[0015] Step 7: Based on the generation result and user feedback, adjust the dynamic priority weight and the contribution coefficient, and update the parameters using incremental learning.

[0016] Further, the step 1 includes:

[0017] Step 11: Based on the document logical structure and semantic density, identify the semantic unit boundary by analyzing the punctuation symbol density, term occurrence frequency, and sentence complexity;

[0018] Step 12, analyze the logical relationship between sentences in the semantic unit, identify the question-answer, argument-evidence and problem-solution semantic patterns, extract key propositions as questions and associate supporting content as answers, form question-answer pairs and attach structure labels;

[0019] Step 13, encode the question-answer pair in two channels, encode the question text as a distinguishability vector and the answer text as a capacity vector, and map them to a unified semantic space through projection transformation to form key-value pairs;

[0020] Step 14, use the key-value pair semantic vector to calculate the semantic similarity between atomic knowledge units, construct a knowledge unit association network, and calculate the initial priority weight based on node centrality index and connection density.

[0021] Further, the step 2 comprises:

[0022] Step 21, perform semantic role analysis on each token in the input sequence, assign semantic role labels, and assign two query dimension spaces to each token based on the semantic role labels, including an internal semantic association channel and an external knowledge interaction channel;

[0023] Step 22, use semantic role labels and query dimension spaces to perform channel-specific transformation on the initial representation of each token, where the internal semantic association channel uses linear transformation and the external knowledge interaction channel uses nonlinear transformation, and the transformation matrix is parameter-shared and grouped according to the semantic role labels.

[0024] Further, the step 2 further comprises:

[0025] Step 23, perform semantic focusing processing on the external knowledge interaction channel query vector, including calculating the initial relevance distribution with the knowledge unit key vector, identifying the semantic focus area, and performing local semantic enhancement transformation;

[0026] Step 24, dynamically fuse the internal semantic association channel output and the enhanced cross-attention channel output, and calculate the fusion weight based on the current decoding state and the input sequence semantic complexity.

[0027] Further, the step 3 comprises:

[0028] Step 31, calculate the semantic distribution dispersion and context dependence strength of tokens in the input sequence to generate context complexity scores;

[0029] Step 32, use the context complexity score to generate global prediction parameters at the sequence level and local prediction parameters at the token subset level, and fuse the multi-scale prediction parameters through a nonlinear combination function;

[0030] Step 33, based on the multi-scale prediction parameters, the standard loss function is adjusted by weight coefficient and weighting factor, the global prediction parameter is used to adjust the overall weight coefficient of the loss function, and the local prediction parameter is used to introduce the token level weighting factor;

[0031] Step 34, under the guidance of the reconstructed loss function, the adapter parameter is optimized and updated, the key parameter update direction is identified according to the gradient distribution, the parameter path corresponding to the high complexity context is preferentially adjusted, and the update amplitude is controlled through the gradient clipping mechanism, while the language model weight is kept frozen.

[0032] Further, the step 4 comprises:

[0033] Step 41, based on the attention peak degree, the semantic redundant connections in the cross attention matrix are identified and filtered, including analyzing the distribution characteristics of the attention weight to identify abnormal connection points with high variance value, and performing smoothing suppression processing on the redundant connections;

[0034] Step 42, based on the purified cross attention matrix, the initial relevance score of each atomic knowledge unit is calculated, including aggregating the purified attention weight of all tokens in the input sequence, using the geometric mean aggregation function, and recording the distribution characteristics of the token being concerned.

[0035] Further, the step 4 further comprises:

[0036] Step 43, using the distribution characteristics of the token being concerned, a preset sequence position weight function is applied to adjust the initial relevance score, including assigning importance coefficients according to the relative position of the token in the sequence, and assigning higher weight coefficients to the tokens of the key semantic nodes, and obtaining the position-aware relevance score through weighted fusion;

[0037] Step 44, based on the position-aware relevance score, a contribution coefficient is applied for final balancing processing, including dynamically adjusting the contribution coefficient according to the semantic complexity of the input sequence and the size of the knowledge base, and generating the final relevance score through the combination of the preset nonlinear function group.

[0038] Further, the step 5 comprises:

[0039] Step 51, based on the relevance score and the semantic overlap degree, a weighted directed graph is constructed, wherein the nodes represent atomic knowledge units, the node weights are assigned after normalization of the relevance score, and the edge weights are calculated by measuring the cosine similarity of the knowledge unit key vector and combining the context dependence relationship;

[0040] Step 52, multi-round graph diffusion processing is performed on the weighted directed graph, and in the diffusion process, each node calculates the attention score according to the edge weight and its own node weight, and dynamically adjusts the information transmission strength.

[0041] Step 53, based on the diffusion of the node score, an adaptive threshold mechanism based on the score distribution statistical characteristics is used to extract the atomic knowledge unit subset, and the subset is filtered for redundancy.

[0042] Further, the step 6 comprises:

[0043] Step 61, monitor the generated context state change, including calculating the semantic integrity of the generated sequence and the dispersion of the prediction probability distribution, quantifying the degree of uncertainty;

[0044] Step 62, based on the generated context state, calculate the semantic matching degree of the atomic knowledge unit, including direct semantic similarity and semantic coherence with the subsequent generation direction, use bidirectional semantic projection to evaluate the adaptation degree;

[0045] Step 63, based on the evaluation result of the semantic matching degree, execute the dynamic gating function, adjust the strength ratio and injection timing of knowledge injection according to the matching degree score and context uncertainty degree.

[0046] Further, the step 7 comprises:

[0047] Step 71, extract the semantic consistency, factual accuracy and logical coherence quality indicators of the generated results, analyze the explicit ratings and implicit behavior data of user feedback, calculate the feedback confidence score, and establish the association mapping of quality indicators and feedback confidence to generate a comprehensive evaluation signal;

[0048] Step 72, based on the comprehensive evaluation signal and confidence weight, incrementally adjust the dynamic priority weight and contribution coefficient, use reinforcement learning type optimization or smoothing adjustment strategy according to the confidence score, and establish a tracking mechanism for parameter update trajectory.

[0049] Compared with the prior art, the beneficial effects of the present application are:

[0050] (1) The scheme realizes dynamic semantic adaptation through cross attention mechanism, can adjust the retrieval weight in real time according to the user input context, gets rid of the limitation of traditional RAG relying on static vector similarity, can accurately capture the context change for polysemy query scene in financial, medical and other fields, avoid the deviation caused by single static representation, such as in complex polysemy term query, can dynamically match the deep semantic demand, improve the retrieval accuracy in complex context, make the retrieval result more close to the real intention of the user.

[0051] (2) The scheme explicitly models the deep semantic association between user input and knowledge fragments, calculates the relevance score of the input sequence and atomic knowledge units through the cross-attention matrix, and also constructs a knowledge topology network combining semantic overlap. Compared with the traditional RAG which is difficult to depict deep relationships, this scheme can capture fine-grained semantic associations, such as analyzing the logical association between the core demand in the user query and the knowledge unit, effectively resolving semantic ambiguity, greatly improving the relevance of the search results, and reducing the matching errors caused by the lack of semantic association.

[0052] (3) The scheme balances the relationship between knowledge injection and input context through the contribution coefficient, avoiding excessive reliance on external knowledge leading to generation distortion. The training process covers single knowledge point, multiple knowledge points and rejection question answering scenarios. In high-precision scenarios such as financial compliance queries and medical diagnosis assistance, it can not only ensure efficient knowledge injection to support generation, but also maintain generation stability, adapt to different task types, improve model robustness, and solve the problem of traditional RAG in balancing precision and stability.

[0053] (4) The scheme closely integrates knowledge retrieval and generation through the cross-attention mechanism, eliminating the error accumulation caused by the decoupling of traditional RAG retrieval and generation. From atomic knowledge representation, model training to retrieval recall and generation control, each link is semantically logical and coherent. For example, the relevance score in the retrieval stage directly guides the knowledge injection in the generation stage, making the generated content not only meet the input demand, but also accurately integrate knowledge, improving the quality of question answering and user experience, especially suitable for professional field applications with high semantic precision requirements. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, a brief introduction will be given below to the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0055] Fig. 1 Flowchart of the dynamic feature retrieval generation and management method based on the cross-attention mechanism of the present application;

[0056] Fig. 2 Workflow diagram of the dynamic feature retrieval generation and management method based on the cross-attention mechanism of the present application. DETAILED DESCRIPTION

[0057] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described; obviously, the described embodiments are only a part of the embodiments of the present application, and not all the embodiments; based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0058] Please refer to Figs. 1-2 The specific operation steps of the dynamic feature retrieval generation and management method based on the cross-attention mechanism are as follows:

[0059] Step 1, convert the knowledge base document segment into an atomic knowledge unit, each atomic knowledge unit includes a question and answer pair, is encoded as a key-value pair, and is additionally attached with a dynamic priority weight;

[0060] Step 2, replace the self-attention query with a double-channel query structure, one of which is used for cross-attention, generate a cross-attention query vector through linear transformation, use the key-value pair of the atomic knowledge unit, and calculate according to the cross-attention query vector and the key-value pair of the atomic knowledge unit to obtain a cross-attention matrix;

[0061] Step 3, train the cross-attention adapter, freeze the language model weight, optimize the adapter parameter, and dynamically adjust the loss function based on the pre-judgment parameter of the input sequence context complexity;

[0062] Step 4, based on the cross-attention matrix, calculate the relevance score of the input sequence and each atomic knowledge unit, weight and fuse the attention score through a preset sequence position weight function, and apply a contribution coefficient;

[0063] Step 5, based on the relevance score, construct a knowledge topology network based on the relevance score and semantic overlap degree, and retrieve the atomic knowledge unit subset through a graph diffusion algorithm;

[0064] Step 6, in the generation process, use a gating function based on the matching degree of the current generation context and the atomic knowledge unit to dynamically control knowledge injection;

[0065] Step 7, based on the generation result and user feedback, adjust the dynamic priority weight and the contribution coefficient, and update the parameters using incremental learning.

[0066] Step 1 further includes the following operation steps: Step 11, based on the document logical structure and semantic density, identify the semantic unit boundary by analyzing the punctuation symbol density, term occurrence frequency and sentence complexity, and the specific operation is as follows:

[0067] First, the logical structure and semantic density of the document are used as the basis for judgment. The semantic unit boundary is identified through multi-dimensional text feature analysis. The punctuation mark density analysis focuses on punctuation marks with semantic separation functions, such as periods, semicolons, and colons. The distribution frequency and interval law of such punctuation marks in different paragraphs of the text are analyzed. The sudden change in punctuation mark density, such as from low frequency to high frequency, corresponds to the natural break point of semantic content, which may be the end of a complete semantic expression and the beginning of a new semantic expression. Term frequency analysis targets domain-specific terms or core vocabulary. By analyzing the frequency and concentration of terms in text segments, areas with high frequency and concentrated distribution of terms often carry relatively complete domain semantic information and can be used as core range anchors for semantic units. Sentence complexity analysis analyzes the grammatical structure of sentences, such as subject-predicate-object hierarchy, modifier component quantity, and complex sentence conjunction word type, to quantify the simplicity and complexity of sentence structure. When there is a significant transition in sentence complexity, such as from a simple short sentence to a long and difficult sentence containing multiple attributive and adverbial phrases, or vice versa, it usually means a change in semantic expression logic and can be used as an auxiliary verification basis for semantic unit boundaries. By cross-checking and correlating the results of the above three feature analyses, the semantic coherence of the document text is judged segment by segment and sentence by sentence. The specific boundaries of the semantic units are finally determined to ensure that each divided semantic unit is logically coherent and semantically complete, and there is a clear semantic distinction between different units, avoiding semantic fragmentation or redundancy during subsequent atomic knowledge extraction.

[0068] Step 12: Within the semantic unit, analyze the logical relationship between sentences, identify question-answer, argument-evidence, and problem-solution semantic patterns, extract key propositions as questions and associate supporting content as answers, form question-answer pairs with structure labels, and the specific operations are as follows:

[0069] On the basis of the independent semantic units divided in step 11, further in-depth semantic unit is carried out, the logical association relationship between sentences is analyzed, the specific semantic mode is recognized and the core composition of the atomic knowledge unit is extracted, such as the question and answer pair, and the structured label is completed. First, the logical relationship of all sentences in the semantic unit is combed, the correlation types such as cause and effect, progression, parallelism, explanation and description between sentences are recognized, and the semantic mode with clear corresponding relationship of "core content-supporting content" is screened out from them, including the question-answer mode, the argument-datum mode and the problem-solution mode. The question-answer mode is formed by identifying the explicit interrogative sentence in the text, containing the interrogative words such as "what", "how" and "why", or the implicit interrogative tone sentence, locating the explanatory sentence directly following it to answer the question, forming the semantic correspondence of "question expression-answer content". The argument-datum mode is formed by distinguishing the argument sentence expressing the viewpoint, conclusion or judgment, usually containing the judgment words such as "is", "should", "has" and "indicates", and the datum sentence used to support the argument, such as data, cases and theoretical basis, establishing the semantic association of "core viewpoint-supporting evidence". The problem-solution mode is formed by extracting the sentence describing the problem to be solved, defect or difficulty, containing the problem representation words such as "problem", "deficiency", "difficulty" and "challenge", and matching the sentence of the solution method, implementation step or optimization strategy for the problem, constituting the semantic correspondence of "problem description-solution".

[0070] Step 13, the question and answer pair is encoded in double channel, the problem text is encoded into distinguishable vector, the answer text is encoded into capacity vector, and is mapped to uniform semantic space through projection transformation, forms key value pair, specific operation is as follows:

[0071] To adapt the question and answer pair to the semantic calculation requirements of the subsequent cross-attention retrieval and knowledge generation link, a double-channel encoding strategy is adopted to differentially process the question and answer pair, and a projection transformation is used to realize the unification of the vector space, and finally a standardized key-value pair representation is formed. The double-channel encoding strategy designs differential encoding targets according to the different semantic functions of the question and answer: for the question text, the encoding goal is to improve its semantic distinguishability, that is, the distinguishability vector obtained through encoding needs to clearly reflect the semantic differences between different questions, avoiding semantic confusion caused by similar question expressions such as synonym replacement and sentence transformation. To achieve this goal, the encoding model will focus on capturing the core question words, limiting conditions and domain-specific expressions in the question. The encoding model can use models such as the Sentence Transformers series, adjust the weight distribution of the attention mechanism, and make these key information have a higher contribution in the vector representation, ensuring that the vectors of different questions have significant distinguishability in the semantic space. For the answer text, the encoding goal is to improve its information capacity, that is, the capacity vector needs to completely carry the factual information, logical reasoning process and detailed description contained in the answer. The encoding will preserve the context association and information integrity of the answer text, and through optimizing the feature extraction strategy of the encoding model, it reduces the loss of key details caused by information compression, ensuring that the answer vector can fully reflect the knowledge content. After completing the double-channel encoding to obtain the initial distinguishability vector and capacity vector, since the two types of vectors may come from different encoding subspaces, such as differences in dimensions or feature distribution, they cannot be directly used for subsequent cross-attention calculation. Projection transformation is needed to map them to a unified semantic space: a linear projection matrix suitable for text semantic features is used to convert the distinguishability vector and the capacity vector in space respectively. The parameters of the projection matrix are optimized through domain corpus pre-training to ensure that the two types of vectors after transformation have the same dimension and consistent semantic measurement standard, meeting the format requirements of the key-value vectors in the cross-attention mechanism. Finally, the question distinguishability vector after projection transformation is defined as the key (Key), and the answer capacity vector is defined as the value (Value), forming a key-value pair structure. This structure not only retains the semantic correspondence of the question and answer pair, but also provides a standardized input for the subsequent cross-attention calculation of the knowledge retrieval model.

[0072] Step 14, using the key-value pair semantic vectors, calculate the semantic similarity between atomic knowledge units, construct a knowledge unit association network, and calculate the initial priority weight based on node centrality index and connection density. The specific operations are as follows:

[0073] Based on the key-value pair semantic vectors generated in the previous steps, a knowledge unit association network is constructed, and the initial priority weight of the atomic knowledge unit is calculated by analyzing the network topology characteristics. First, the semantic similarity between different atomic knowledge units is calculated: taking the key vector or the fusion vector of the key vector and the value vector of the atomic knowledge unit as the semantic representation carrier, a similarity calculation method suitable for text vectors is used, such as cosine similarity, to quantify the association degree of any two atomic knowledge units in semantic content. When the semantic similarity score of two units is higher than a preset threshold, the threshold is set according to the field characteristics and semantic association requirements of the knowledge base, such as 0.7, it is determined that there is an effective semantic association between the two, and a non-directional connection edge is established in the network, thereby forming a knowledge unit association network with atomic knowledge units as nodes and semantic associations as edges. Subsequently, the topological characteristics of the association network are analyzed, and two types of indicators are calculated: one is the node centrality, including the degree centrality, which is determined by the number of effective connection edges possessed by the node, the betweenness centrality, which is determined by the frequency of the node on the shortest path between any two nodes, and the closeness centrality, which is determined by the reciprocal of the average shortest path length from the node to all other nodes in the network. The higher the node centrality score, the closer the direct or indirect association between the atomic knowledge unit and other units in the network, which usually represents its stronger core and knowledge radiation capability in the knowledge base; the second is the connection density, which is the ratio of the number of actual existing effective connection edges in the network to the number of all possible connection edges in theory. The number of all possible connection edges in theory can be obtained by node total number x (node total number-1) / 2. By calculating the connection density of the local sub-network where each node is located, such as the sub-network composed of neighborhood nodes with the node as the center and a radius of 2, the semantic aggregation degree of the knowledge area where the node is located can be determined. The higher the connection density of the area, the stronger the semantic synergy and association between knowledge units. Finally, the initial priority weight is calculated by combining the node centrality and the connection density: a weighted combination formula is used to realize the fusion of the two types of indicators. The formula can be expressed as: ;

[0074] wherein, is the initial priority weight of the i-th atomic knowledge unit; is the normalized centrality score of the i-th node, with a value range of [0, 1]; is the normalized connection density score of the local sub-network where the i-th node is located, with a value range of [0, 1]; and are weight coefficients, satisfying , which is preset according to the field requirements, such as , The formula is derived from the logic that the core knowledge unit needs to have high correlation and strong semantic aggregation, balances the contributions of the two types of indexes through linear weighting, and ensures that the weight calculation result can objectively reflect the importance of the atomic knowledge unit; through the above calculation, each atomic knowledge unit obtains a corresponding initial priority weight, and the higher the weight value of the unit, the higher the probability of being recalled preferentially in the subsequent retrieval process, thereby realizing efficient scheduling of core knowledge.

[0075] In a preferred embodiment of the application, the generation of the cross-attention matrix in step 2 is specifically operated as follows:

[0076] The generation of the cross-attention matrix is based on the optimization of the double-channel query of the atomic knowledge unit representation and the input sequence, and is gradually constructed following the strict semantic interaction and feature transformation logic. First, the key-value pairs of the atomic knowledge unit provide interactive objects for cross-attention calculation. These key-value pairs are generated through semantic unit division of knowledge base document fragments, question and answer pair extraction, and double-channel encoding. The question text is encoded as a distinguishability vector, and the answer text is encoded as a capacity vector. After projection transformation and mapping to a unified semantic space, the semantic interactivity with the input sequence is ensured. The input sequence needs to be preprocessed to generate an adaptive query vector. Through semantic role analysis, channel-specific transformation, semantic focusing processing, and dynamic fusion, the generated query vector not only maintains the internal logical consistency of the input sequence, but also fully integrates external knowledge interaction requirements. Finally, the semantic association strength between each token and each atomic knowledge unit is quantified through similarity calculation, such as cosine similarity and dot product similarity, to form the cross-attention matrix.

[0077] In step 2, the following operations are also performed:

[0078] In step 21, semantic role analysis is performed on each token in the input sequence, semantic role labels are assigned, and two query dimension spaces are assigned to each token based on the semantic role labels, including an internal semantic association channel and an external knowledge interaction channel. The specific operations are as follows:

[0079] First, semantic role labeling is performed on each token in the input sequence, which usually adopts a model based on a pre-trained semantic role labeling framework, such as the PropBank or FrameNet system, which combines the syntactic structure and context information of the input sequence to identify the semantic function and contribution of each token in the sentence, and then assigns specific semantic role labels, common labels include predicate, representing the core token of the action or state; agent, the subject token initiating the action; patient, the object token of the action; time adverbial, token limiting the time of the action; place adverbial, token limiting the location of the action, etc. These labels clearly define the semantic positioning and functional requirements of tokens in the input sequence. Subsequently, based on the assigned semantic role labels, two independent query dimension spaces are assigned to each token, namely the internal semantic association channel and the external knowledge interaction channel. This allocation logic is based on the differentiated needs of tokens with different semantic roles for semantic processing: for example, predicate tokens need to maintain strong logical association with other tokens in the input sequence to ensure syntactic and semantic coherence, and obtain external knowledge to verify the semantic accuracy of the action specification in the domain, so two channels are needed to carry the corresponding needs; while time adverbial tokens focus more on the temporal logical connection with other tokens in the input sequence, and have lower external knowledge interaction needs, but still need external channels to assist in verifying the rationality of the time expression, so the dual-channel design is also used to cover potential needs. Through this semantic role-based channel allocation, the semantic processing process of each token can accurately match its functional positioning, avoiding the limitations of a single query space that cannot balance internal association and external interaction.

[0080] Step 22, using semantic role labels and query dimension spaces, performing channel-specific transformation on the initial representation of each token, where the internal semantic association channel uses linear transformation, and the external knowledge interaction channel uses nonlinear transformation, and the transformation matrix is parameter-shared grouped according to the semantic role label. The specific operation is as follows:

[0081] According to the differentiated semantic needs of the two query dimension spaces, channel-specific transformation is performed on the initial representation of each token, which is usually the word embedding vector output by the pre-trained language model, and parameter sharing of the transformation matrix is achieved through semantic role grouping to balance model performance and parameter quantity. For the internal semantic association channel, linear transformation is used to process the initial representation of the token, specifically through a learnable linear transformation matrix and the bias term of the internal channel convert the initial representation x into an internal channel feature vector The reason for choosing linear transformation is that the goal of the internal channel is to maintain the semantic coherence and logical consistency of the input sequence. Linear transformation can achieve adaptive transformation of the feature space while preserving the initial semantic information of the tokens, avoiding the additional semantic distortion introduced by nonlinear transformation, and ensuring that the association between tokens within the sequence is not destroyed. For the external knowledge interaction channel, since it needs to handle the complex semantic mapping that may exist between tokens and external knowledge units, such as the ambiguity of domain terms and cross-domain semantic associations, nonlinear transformation is adopted, using linear matrices. external channel bias term The initial representation is transformed into an external channel feature vector using a nonlinear activation function σ, which can be either a GELU function or a ReLU function. Nonlinear activation functions can capture complex nonlinear relationships in the semantic space, making the external channel feature vectors more adaptable to subsequent interactions with the knowledge unit key vectors. Simultaneously, based on the semantic role labels assigned in step 21, the transformation matrix is ​​grouped using parameter sharing: semantic roles are divided into core semantic roles, such as predicates, agents, and patients, which directly affect the core semantics of the sentence, and auxiliary semantic roles, such as time adverbs and place adverbs, which play a modifying and supplementary role. Each of these two types of roles is assigned an independent set of parameters. , The parameters are shared by all tokens of the same type of role. This grouping strategy can significantly reduce the number of model parameters and reduce the risk of overfitting. It can also ensure that tokens with the same semantic function maintain consistent processing in channel transformation and improve the model's generalization ability to similar semantic scenarios.

[0082] Step 23: Perform semantic focusing processing on the query vector of the external knowledge interaction channel, including calculating the initial relevance distribution with the knowledge unit key vector, identifying the semantic focus region, and performing local semantic enhancement transformation. The specific operations are as follows:

[0083] First, calculate the initial correlation distribution between the external channel feature vector and the knowledge unit key vector: the feature vector output by the external channel. as query vector The bond vector of all atomic knowledge units in the knowledge base (j=1,2,...,Z, where Z is the total number of atomic knowledge units) Perform similarity calculations, such as cosine similarity or dot product similarity, to obtain the similarity between each key vector and... correlation score All scores constitute the initial correlation distribution. This distribution intuitively reflects the semantic association strength between the current token and each knowledge unit. Subsequently, semantic focus regions are identified based on the initial relevance distribution: by setting a relevance threshold, which can be obtained by adding one standard deviation to the mean of distribution S, or by Top-K filtering, which can obtain the top 5 key vectors with the highest scores, a subset with significantly higher relevance scores than other key vectors is selected. The semantic range of the knowledge units corresponding to this subset is the semantic focus region. These knowledge units have the highest semantic relevance to the current token and are the objects of subsequent knowledge interaction. Identifying the focus region can effectively narrow the retrieval range of external knowledge and reduce the interference of irrelevant knowledge. Finally, a local semantic enhancement transformation is performed: for the key vector corresponding to the semantic focus region, its average feature vector is calculated. And calculated through dot product and Similarity score ,Will As an enhancement coefficient, for The enhanced external channel vector is obtained by weighting adjustment. This local enhancement strategy can be further strengthened. The semantic features related to the focal region enable it to more accurately point to the relevant knowledge units in subsequent cross-attention calculations, thereby improving the ability of the external channel vector to capture useful knowledge.

[0084] Step 24: Dynamically fuse the output of the internal semantic association channel and the output of the enhanced cross-attention channel. Calculate the fusion weight based on the current decoding state and the semantic complexity of the input sequence. The specific operation is as follows:

[0085] This step generates a final feature vector that balances intra-sequence logical coherence and external knowledge relevance by dynamically fusing the output vectors of the internal semantic association channel and the enhanced external knowledge interaction channel. The fusion process calculates dynamic fusion weights based on the current decoding state and the semantic complexity of the input sequence to ensure that the fusion result adapts to the current generation requirements. First, two key reference indicators are obtained: one is the current decoding state, i.e., the current hidden layer state of the generator model decoder. This state integrates the semantic information of the generated content and can be calculated. With internal channel vector cosine similarity The fusion weights are calculated based on two metrics: first, the matching degree between the internal channel output and the current generation context; and second, the semantic complexity C of the input sequence, which is calculated using the semantic distribution dispersion of the tokens and the strength of contextual dependencies in previous steps. A higher C value indicates a more complex semantic structure in the current input sequence and a stronger need for supplementary external knowledge. A fusion weight calculation function is constructed based on these two metrics, specifically the internal channel weights. External channel weights ; when higher, it means that the internal channel vector matches the current decoding state, and the internal channel weight needs to be improved to maintain the sequence logical coherence; when the value of C is higher, 1-C is reduced, and the external channel weight is improved accordingly to introduce more external knowledge to support complex semantic understanding, in the formula, represents the contribution weight of the internal channel vector in the fusion result, represents the contribution weight of the enhanced external channel vector; for balancing the influence of C on the weight, , preset as 0.5; is the similarity between the decoding state and the internal channel vector, and the value range is [0, 1]; C is the semantic complexity, and the value range is [0, 1]; finally, the dynamic fusion of the two channel vectors is realized by weighted summation to generate the final feature vector , which not only retains the internal logical association of the input sequence, but also integrates the necessary external knowledge, which can efficiently support the subsequent cross-attention retrieval and knowledge-enhanced generation.

[0086] In a preferred embodiment of the present application, step 3 further includes the following operation steps:

[0087] Step 31, calculate the semantic distribution dispersion and context dependence strength of the token in the input sequence to generate a context complexity score, and the specific operation is as follows:

[0088] First, the semantic distribution dispersion of each token in the input sequence is calculated: a high-dimensional semantic vector of each token is extracted by a pre-trained language model, the pre-trained language model adopts BERT, GPT series, and the high-dimensional semantic vector can represent the position and category attribute of the token in the semantic space; then, the distribution dispersion is calculated based on the semantic vectors of all tokens, and the Shannon entropy or variance is usually used as a measurement index, if the Shannon entropy is used, the probability distribution of the semantic vector on the preset semantic category, such as the domain-specific semantic category, needs to be counted first, the higher the entropy value, the more diverse the categories covered by the token semantic vector, the more dispersed the semantic expression, and the higher the context complexity; if the variance is used, the overall variance of the value of each dimension of the semantic vector is calculated, the larger the variance, the more dispersed the distribution of the token semantic in the space, and the stronger the semantic diversity; second, the context dependence strength of the token is calculated: it is realized by analyzing the semantic correlation degree of each token and its context token, and the context token is usually taken as a context of about 5-10 window size, the attention weight distribution of the current token to the context token can be calculated by using the attention weight output by the self-attention mechanism, the variance or coefficient of variation of the weight is calculated, the larger the variance or coefficient of variation, the stronger the dependence of the current token on part of the context token, and the weaker the dependence on another part, the context semantic correlation is uneven, and the context logic complexity is higher; the semantic synergy of the token and the context token can also be calculated by mutual information, the larger the fluctuation range of the mutual information value, the more complex the context dependence relationship, and finally, the semantic distribution dispersion and the context dependence strength are normalized respectively and mapped to the [0, 1] interval, the weighted sum is calculated by a preset weight coefficient, such as a semantic distribution dispersion weight of 0.6 and a context dependence strength weight of 0.4, which can be adjusted according to the domain requirements, to obtain the context complexity score of each input sequence, the score value range is [0, 1], the closer the score to 1, the more complex the sequence context, and the higher the model learning difficulty.

[0089] In step 32, the context complexity score is used to generate a global pre-judgment parameter at the sequence level, a local pre-judgment parameter at the token subset level, and a multi-scale pre-judgment parameter fused by a nonlinear combination function, and the specific operations are as follows:

[0090] First, generate the global prediction parameter: for the sequence-level context complexity score obtained in step 31, through a nonlinear transformation such as the Softmax function or the Sigmoid function, map it to the interval [0, 1] to obtain the global prediction parameter. This parameter reflects the learning difficulty of the input sequence from the overall level. For example, when the sequence context complexity score is 0.8, it indicates high complexity, and the global prediction parameter after Sigmoid transformation may be close to 0.9, indicating that the entire sequence requires the model to invest more learning resources. When the complexity score is 0.2, it indicates low complexity, and the global prediction parameter may be close to 0.3, representing low sequence learning difficulty and no need for excessive resource allocation. Second, generate the local prediction parameter: first, divide the input sequence into multiple token subsets according to the semantic role labels of the tokens, such as the core semantic token subset containing the predicate, agent, and patient, and the auxiliary semantic token subset containing the time adverbial and place adverbial. Then calculate the average of the context complexity scores of all tokens in each token subset, representing the average complexity at the subset level. Finally, map this average value to the local prediction parameter of the corresponding subset through a linear transformation matrix. The parameter value is also [0, 1]. The local prediction parameter can reflect the learning difficulty difference of different token subsets. For example, the average complexity of the core semantic token subset is 0.7, and its local prediction parameter may be 0.8. The average complexity of the auxiliary semantic token subset is 0.3, and its local prediction parameter may be 0.4, clearly distinguishing the priority of different subsets. Finally, nonlinearly fuse the multi-scale prediction parameters: use a nonlinear combination function such as the GELU activation function or the attention fusion mechanism to fuse the global prediction parameter and each local prediction parameter. First, use the global parameter as the fusion reference, calculate the contribution of each local parameter to the global parameter through attention weights, and the contribution of the core semantic subset is usually higher than that of the auxiliary subset. Then, nonlinearly integrate the global parameter and the weighted local parameters through the GELU function to obtain the final multi-scale prediction parameter. The reason for using nonlinear fusion is that the influence of global and local parameters on loss adjustment is not simply linearly superimposed. When the global complexity is high, the contribution of the local parameter of the core semantic subset to the loss adjustment needs to be nonlinearly amplified. The nonlinear function can accurately capture this complex relationship to ensure that the fused parameter can fully reflect the context difficulty at different granularities.

[0091] Step 33, based on the multi-scale prediction parameter, adjust the weight coefficient and weighting factor of the standard loss function. Use the global prediction parameter to adjust the overall weight coefficient of the loss function, and use the local prediction parameter to introduce the token-level weighting factor. The specific operation is as follows:

[0092] First, clarify the basis of the standard loss function, usually use the cross-entropy loss function: ;

[0093] where N is the number of tokens, is the true label of the th token, is the model prediction probability, is the cross-entropy loss function, which can measure the deviation between model prediction and true value, but cannot adjust the learning focus according to the context difficulty. Subsequently, the overall weight coefficient of the loss function is adjusted based on the global prediction parameter: let the global prediction parameter be G, taking values [0, 1]; the preset coefficient is , usually taking 0.5-1.0, which is adjusted according to training requirements, then the adjusted overall weight coefficient is This adjustment logic is derived from the need for higher learning weight for complex sequences. When G is close to 1, it indicates high context complexity, , which is increased, so that the loss value of the entire sequence is amplified, and the model will invest more resources to optimize the prediction result of this sequence when backpropagation; when G is close to 0, it indicates low context complexity, , close to 1, the loss value remains relatively stable, avoiding overfitting caused by the model's excessive focus on simple samples. At the same time, a token-level weighting factor is introduced based on the local prediction parameter: let the local prediction parameter corresponding to the th token be , taking values [0, 1]; the preset coefficient is , usually taking 0.3-0.8, which is adjusted according to the importance of the token, then the loss weighting factor of the th token is: ;

[0094] In view of the need for higher learning weight for key tokens, , close to 1, such as tokens in the core semantic subset, their , the proportion of the loss of this token in the total loss is increased, and the model will focus on optimizing the prediction accuracy of such tokens, reducing the prediction error of key semantics. By comprehensively adjusting the above, the final reconstructed loss function is: ;

[0095] The derivation logic of this formula is based on the concept of double-dimensional weight enhancement. First, the overall loss of complex sequences is amplified through global weight, and then the loss contribution of key tokens is highlighted through local weight. The two work together to make the loss function accurately match the learning needs of different contexts and tokens; in the formula, is the final loss value after reconstruction, is the global weight coefficient, is the local weighting factor of the th token.

[0096] Step 34, under the guidance of the reconstructed loss function, the optimization update of the adapter parameters is carried out, the key parameter update direction is identified according to the gradient distribution, the parameter path corresponding to the high complexity context is adjusted preferentially, and the update amplitude is controlled through the gradient clipping mechanism, while the language model weight is kept frozen, and the specific operation is as follows:

[0097] First, strictly keep the language model weight frozen, fix all parameters of the pre-trained language model in the whole training process, such as Sentence Transformers used for semantic vector extraction and Decoder-Only model used for basic semantic understanding, only the parameters of the cross-attention adapter, including the linear transformation matrix of the cross-attention query vector, the projection matrix of the key value vector and the bias term, are used as the trainable parameters, the purpose of this operation is to protect the general semantic understanding ability and domain knowledge learned by the pre-trained model, avoid the destruction of the basic semantic modeling ability in the adapter parameter optimization process, at the same time, greatly reduce the number of trainable parameters, reduce the risk of model overfitting and training calculation cost, secondly, based on the gradient distribution to identify the key parameter update direction: in each back propagation calculation of gradient, the gradient norm of each adapter parameter is calculated, such as L2 norm, the key parameters are selected by sorting the gradient norm, the larger the gradient norm of the parameter, the more significant the influence on the reconstructed loss function, and such parameters usually correspond to the learning path of the model in the high complexity context, such as the parameters of cross-attention weight distribution processing polysemous terms; for key parameters, the priority update strategy is adopted: in the parameter optimization step, such as Adam optimizer, the key parameters are allocated higher learning rate, usually 1.2-1.5 times of the ordinary parameters, or the key parameters are updated preferentially in the parameter update order, to ensure that the model solves the adaptation problem in the high complexity context preferentially, and quickly improves the retrieval accuracy of complex scenes; at the same time, the gradient clipping mechanism is introduced to control the parameter update amplitude: the threshold of gradient norm is set, such as 1.0-2.0, which is adjusted according to the model convergence, after calculating the gradient, if the gradient norm of a parameter exceeds the threshold, the gradient is scaled by the ratio of the gradient norm or the threshold, so that the gradient norm is controlled within the threshold range, this mechanism can effectively prevent the gradient explosion problem caused by complex context samples in the training process, the model prediction error may be larger in the high complexity context, which is easy to produce super large gradient, gradient clipping can limit the gradient in a reasonable range, avoid the model training shock or convergence difficulty caused by too large parameter update amplitude, finally, in the reconstructed loss function Under the guidance of the teacher, through multiple iterations, forward calculation of loss, back propagation of gradient, key parameter priority update, gradient clipping stable training process, until the model loss value on the validation set tends to be stable and the retrieval accuracy reaches the preset index, complete the optimization update of the adapter parameter, at this time the adapter can accurately realize the dynamic cross attention calculation of input sequence and atomic knowledge unit, meet the retrieval demand under complex context.

[0098] In a preferred embodiment of the present application, step 4 further comprises the following operation steps:

[0099] Step 41, based on the attention peak degree, identify and filter the semantic redundant connections in the cross attention matrix, including analyzing the distribution characteristics of attention weight to identify abnormal connection points with high variance value, and performing smoothing suppression processing on redundant connections, the specific operation is as follows:

[0100] Firstly, the distribution concentration of attention weight is judged by attention peak degree analysis. Attention peak degree is a statistical indicator to measure the steepness of weight distribution. When the peak value is high, the attention weight is highly concentrated in a few key connections, that is, the knowledge unit connection with strong semantic correlation to the input sequence, and there are few redundant connections. When the peak value is low, the weight distribution is flat and dispersed, and a large number of connection weights are in the middle interval, some of which may be redundant items without actual semantic association, such as false connections caused by random noise. Based on the peak analysis result, the distribution characteristics of attention weight are further analyzed to identify abnormal connection points with high variance value: the weight variance of each connection point in the cross attention matrix is calculated, if the weight value of a connection point deviates significantly from the mean value of the row or column it belongs to, such as more than ± 2 standard deviations of the mean value, and the corresponding variance value is higher than the preset threshold, which is determined according to the weight distribution statistics of the model training data, it is determined as an abnormal connection point. Such connections are usually caused by accidental semantic overlap between token and knowledge unit, rather than real association, and belong to redundant items that need to be filtered. Finally, the identified redundant connections are subjected to smoothing suppression processing: Gaussian smoothing filtering or weight threshold clipping strategy is adopted. For Gaussian smoothing, the weight value of the redundant connection point is replaced by the mean value of the effective connection weight in the window, and the influence of the redundant weight is weakened. For threshold clipping, set the lower weight threshold, and set the redundant connection weight below the threshold to 0, completely eliminate the invalid connection. Through the above processing, the redundant and abnormal connections in the cross attention matrix are effectively filtered, and the weight distribution is more focused on the real semantic association.

[0101] Step 42, based on the purified cross attention matrix, calculate the initial relevance score of each atomic knowledge unit, including aggregating the purified attention weight of all tokens in the input sequence, using the geometric mean aggregation function, and recording the distribution characteristics of the token of interest, the specific operation is as follows:

[0102] First, clarify the information of the purified cross-attention matrix, and the information of each element in the matrix. Represents the input sequence number 1 The token and the first The purified attention weights of each atomic knowledge unit, having eliminated redundant interference, directly reflect the strength of the basic semantic association between the two. To obtain the comprehensive association strength between a single atomic knowledge unit and the entire input sequence, the weights of all tokens in the input sequence for that knowledge unit need to be aggregated. Here, the geometric mean aggregation function is chosen instead of the arithmetic mean because the geometric mean is more sensitive to high weight values. If a knowledge unit has extremely high weights with a few core semantic tokens, such as predicates and agent tokens, but low weights with other auxiliary tokens, the geometric mean can effectively amplify the contribution of high-weight tokens, avoiding the arithmetic mean's flattening effect due to low-weight tokens masking the strong association between the knowledge unit and the core tokens. At the same time, the geometric mean can avoid the distortion of results caused by individual abnormally high weights, such as residual noise that has not been completely filtered, ensuring that the aggregation result is closer to the actual semantic association. Specifically, the attention weights of the first atomic knowledge unit are calculated as follows: The formula for calculating the initial relevance score of each atomic knowledge unit is: ;

[0103] in For the first The initial relevance score of each knowledge unit, The formula represents the total number of tokens in the input sequence. It stems from the understanding that core tokens dominate semantic association. The strength of the association between the input sequence and knowledge units is primarily determined by the weights of the core tokens. The geometric mean, through product and square root operations, ensures that the contribution of high-weight tokens dominates the final score, while balancing the combined influence of multiple tokens. Compared to the arithmetic mean, its result more accurately reflects the true semantic association between knowledge units and the input sequence. While calculating the initial score, it simultaneously records the distribution characteristics of the tokens of interest: statistically analyzing the distribution of the tokens of interest... The position of a token in the input sequence whose weight value is higher than a preset association threshold, such as the position of the token at the beginning of the sequence. The middle After Semantic role labels, such as core semantic roles and auxiliary semantic roles, as well as weight contribution ratios, such as the proportion of the sum of the weights of high-weight tokens to the total weights of all tokens. These distribution features directly reflect which positions and types of tokens in the input sequence play a dominant role in the association of the knowledge unit; among them, the preset association threshold can be the average of all weights in the matrix.

[0104] Step 43: Utilize the distribution characteristics of the tokens of interest, and apply a preset sequence position weighting function to adjust the initial relevance score. This includes assigning importance coefficients based on the relative position of the tokens in the sequence, and assigning higher weight coefficients to tokens of key semantic nodes. The position-aware relevance score is obtained through weighted fusion. The specific operations are as follows:

[0105] First, the design logic of the pre-defined sequence position weight function is clarified. Tokens at different positions in the input sequence carry significantly different semantic functions. Typically, tokens at the beginning of the sequence summarize the theme or core requirements, such as the core question in a user's query; tokens in the middle of the sequence provide supplementary details or logical elaboration; and tokens at the end of the sequence often summarize or define limiting conditions, such as constraints in a query. Furthermore, the semantic importance of the core semantic role token is higher than that of auxiliary semantic role tokens regardless of its position. Therefore, the position weight function needs to consider both the relative position and semantic role of the tokens, constructing a two-dimensional weight coefficient system for relative positions. Non-linear functions, such as S-curves or piecewise linear functions, are used for assignment. For example, the sequence position is divided into the first 20%, the middle 60%, and the last 20%, and assigned basic weight coefficients of 0.8, 0.5, and 0.7 respectively. The specific weights can be adjusted according to the characteristics of the domain corpus. For semantic roles, a role weight coefficient of 1.2 is added to the core semantic role token, and 1.0 is added to the auxiliary semantic role token. The final position weight coefficient of a single token is: relative position basic coefficient × semantic role added coefficient. Subsequently, combined with the distribution characteristics of the attention tokens recorded in step 42, the initial relevance score is adjusted by weighted fusion: for the first... For each knowledge unit, firstly, extract the position weight coefficients of all followed tokens from the distribution characteristics, where all followed tokens have weights higher than the association threshold. Then, calculate the average of these coefficients as the mean position weight of the knowledge unit. Then, the initial correlation score was... and Perform a product operation to obtain the position-aware correlation score. For example, if the tokens of interest for a certain knowledge unit are mostly concentrated at the beginning of the sequence and mainly play core semantic roles, then... The possible score is 0.8 × 1.2 = 0.96. With an initial score of 0.6, the positional awareness score would be 0.6 × 0.96 = 0.576. If the tokens of another knowledge unit are concentrated in the middle of the sequence and mostly play auxiliary roles, The initial score is 0.6, so the position awareness score is 0.6 × 0.5 = 0.3. This adjustment logic can effectively distinguish between knowledge units where the token is located in a core position or role and those located in a non-core position or role. The former carries stronger semantic association evidence, so its position awareness score is strengthened, while the latter has weaker semantic association evidence, so its score is reasonably reduced, ensuring that the relevance score can accurately reflect the impact of the token's position and role on semantic association.

[0106] Step 44: Based on the location-aware relevance score, apply the contribution coefficient for final balancing. This includes dynamically adjusting the contribution coefficient according to the semantic complexity of the input sequence and the size of the knowledge base, and generating the final relevance score through a combination of preset nonlinear functions. The specific operations are as follows:

[0107] First, the dynamic adjustment mechanism of the contribution coefficient is clarified. The contribution coefficient C∈(0,1] is used to control the contribution ratio of atomic knowledge units to the relevance of the input sequence. Its value needs to be adjusted in conjunction with the semantic complexity of the input sequence and the size of the knowledge base. When the semantic complexity of the input sequence is high, such as containing ambiguous terms or complex logical relationships, the auxiliary role of external knowledge needs to be enhanced. In this case, C is adjusted towards 1.0. For example, when the complexity score is 0.8, C=0.9. When the size of the knowledge base is large, such as the total number of atomic knowledge units... To avoid overestimating the relevance scores of a large number of knowledge units, which could lead to search confusion, C needs to be appropriately reduced, such as in terms of scale. At this point, C=0.7. The adjustment of the contribution coefficient is achieved through a preset mapping function, which takes the semantic complexity score and the knowledge base size as input. The semantic complexity score is generated in step 31, and the real-time C value is calculated through linear interpolation to ensure that C can dynamically adapt to the knowledge injection needs in different scenarios. Subsequently, the final relevance score is constructed based on the contribution coefficient and the location-aware relevance score, and the two are fused using a preset nonlinear function. The specific formula is as follows: ;

[0108] in For the first The final relevance score of each atomic knowledge unit, For non-linear activation functions, variants of the Sigmoid or Softmax functions are typically chosen, where C is the dynamically adjusted contribution coefficient. The formula for the location-aware relevance score is derived from the need to "balance knowledge contribution and semantic rationality": directly comparing C with... Linear multiplication may cause fractions to exceed a reasonable range, such as C=0.9 and... At this time, the product is 0.81, if the scores of multiple knowledge units are close, it is easy to cause insufficient discrimination, and the nonlinear function can map the fusion result to the interval [0, 1], and at the same time, through the nonlinear characteristics of the function, the gap between the high-score knowledge unit and the low-score knowledge unit is enlarged, and the discrimination of the retrieval recall is improved; for example, the Sigmoid function is used The coefficient 5 is used to enhance the nonlinear discrimination, when , ≈0.98, when , ≈0.62, the gap is significantly enlarged, through the above processing, the final correlation score balances the relationship between knowledge injection and input context, and has clear discrimination, and can be directly used for subsequent TopK retrieval recall, so as to ensure that the knowledge unit recalled accurately matches the semantic requirements of the input sequence.

[0109] In a preferred embodiment of the application, step 5 further comprises the following operation steps:

[0110] Step 51, based on the correlation score and the semantic overlap, a weighted directed graph is constructed, wherein the node represents an atomic knowledge unit, the node weight is given after normalization of the correlation score, and the edge weight is calculated by measuring the cosine similarity of the knowledge unit key vector and combining the context dependency relationship. The specific operation is as follows:

[0111] Firstly, the node definition of the graph is determined, each atomic knowledge unit is directly taken as an independent node of the graph, and the core attribute of the node is the weight, which is generated by the normalization processing of the knowledge unit correlation score obtained in step 4. Considering that the correlation scores of different knowledge units have large absolute value differences, such as some scores may be in the interval of 0.1-0.3, and some may be in the interval of 0.6-0.8, directly using the original score will cause the weight scale to be inconsistent, affecting the accuracy of subsequent graph calculation. Therefore, the Min-Max normalization method is used to map all knowledge unit correlation scores to the interval [0, 1], and the normalized value is the weight of the corresponding node. The closer the weight is to 1, the closer the semantic association between the knowledge unit and the input sequence is, and the higher the importance of the knowledge unit in the graph is.

[0112] The directed edges and edge weights of the constructed graph are then built, and the calculation of the edge weights needs to combine the two core dimensions of the semantic overlap degree, the cosine similarity of the knowledge unit key vector and the context dependency relationship. First, the cosine similarity of any two knowledge unit key vectors is calculated. This similarity can quantify the semantic distance of the two in the unified semantic space. The higher the similarity, the more similar the core semantics of the two. This is used as the basis value of the edge weight. Then, the context dependency relationship of the two knowledge unit keys in the original knowledge base document is analyzed in depth. If the two are in adjacent paragraphs or sentences in the document and there is an explicit logical association, such as a cause-and-effect relationship, an explanation and description relationship, and the association words in the document such as "therefore", "that is", and "for example" are analyzed, it is indicated that the two have strong dependency in the actual knowledge system. The basis edge weight needs to be enhanced by a coefficient, such as multiplied by an enhancement coefficient of 1.2, to reflect the semantic association brought by the context. If the two have no direct adjacent relationship in the document and the logical association is weak, the basis edge weight is kept unchanged. At the same time, the direction of the edge is determined according to the semantic support ability of the knowledge unit. The semantic interpretation ability of the two knowledge units to the input sequence is compared, which is indirectly reflected by the node weight. The interpretation ability of the node weight is high. From the knowledge unit with strong semantic support ability to the knowledge unit with weak support ability, a directed edge is formed. For example, the node weight of knowledge unit A is 0.9, the node weight of knowledge unit B is 0.6, and A and B have a semantic association. A directed edge from A to B is constructed, and the edge weight is the value adjusted by the context. In this way, the final weighted directed graph can not only reflect the importance of the knowledge unit itself, represented by the node weight, but also reflect the semantic association strength and direction between units, represented by the edge weight, providing comprehensive structure and semantic information for graph diffusion retrieval.

[0113] Step 52, multiple rounds of graph diffusion processing are performed on the weighted directed graph. During the diffusion process, each node calculates the attention score according to the edge weight and its own node weight, and dynamically adjusts the information transmission strength. The specific operation is as follows:

[0114] The dynamic transmission and aggregation of knowledge unit information are realized by performing multiple rounds of graph diffusion processing on the weighted directed graph constructed in step 51, the weight of the core knowledge unit is strengthened, and the weight of the edge knowledge unit is reasonably adjusted, so that the knowledge unit strongly related to the input sequence is more clearly highlighted. The logic of graph diffusion is to simulate the propagation process of knowledge among related units. Each round of diffusion is based on node weight and edge weight to calculate attention score, dynamically control the strength of information transmission, and ensure that information is concentrated in the core node with close semantic association. Each round of diffusion process starts from information reception and aggregation of each node: for the current node, first collect all directed edges pointing to itself, that is, edges from other nodes to the current node, extract the edge weight corresponding to each edge and the current node weight of the starting node of the edge; multiply the edge weight and the starting node weight to get the preliminary strength of the information transmission from the starting node to the current node. This calculation logic is based on the cognition that the strength of information transmission is determined by the degree of association and the value of information, where the degree of association is represented by the edge weight, and the value of information is represented by the starting node weight. The greater the edge weight, the closer the association, and the greater the starting node weight, the more valuable the information, and the product of the two can accurately reflect the actual utility of information transmission. Then, sum up the preliminary transmission strength of all edges pointing to the current node, and divide the single preliminary transmission strength by the total to get the attention score of the starting node to the current node. This score represents the proportion of the starting node information in the aggregation process of the current node. The current node aggregates the semantic information of each starting node according to the attention score, such as core semantic words and logical relationships, and then updates its node weight. The starting node with a high attention score has a higher proportion in the aggregation and a greater impact on the update of the current node weight.

[0115] During the multiple diffusion processes, the information transmission strength is dynamically adjusted according to the attention score of each round: if the attention score of a starting node to the current node is higher than the preset strong contribution threshold, such as 0.3, for two consecutive rounds, it means that there is a stable and strong semantic association between the starting node and the current node, and the transmission coefficient of the edge weight between them will be appropriately increased, such as multiplied by 1.1, to further strengthen the information transmission of this path. If the attention score is lower than the weak contribution threshold, such as 0.1, for two consecutive rounds, it means that the information value of the starting node to the current node is low, and the transmission coefficient will be reduced, such as multiplied by 0.8, to weaken the information transmission. Through 3-5 rounds of dynamic diffusion, the number of rounds is determined by the size of the knowledge base, the larger the size, the more the rounds, to ensure sufficient information propagation. The node weight of the core knowledge unit gradually accumulates and increases, and the weight of the edge knowledge unit gradually decreases, finally forming a clear hierarchical node weight distribution.

[0116] Step 53, based on the node score after diffusion, an adaptive threshold mechanism based on score distribution statistical characteristics is used to extract the subset of atomic knowledge units, and the subset is filtered for redundancy. The specific operation is as follows:

[0117] First, the adaptive threshold screening based on the score distribution statistical characteristics is performed. The defect of the fixed threshold such as 0.5 is that it cannot adapt to the node weight distribution corresponding to different input sequences. When most node weights are low, such as concentrated in 0.2-0.4, the fixed threshold will cause no node to be selected. When most node weights are high, such as concentrated in 0.6-0.8, the fixed threshold will screen out a large number of edge nodes. Therefore, the complete distribution characteristics of all atomic knowledge unit node scores after diffusion are first calculated, and the mean μ and standard deviation of the scores are calculated. The mean reflects the overall level of the node score, and the standard deviation reflects the dispersion degree of the score. The adaptive threshold is set to μ+1.5 . The selection logic of this threshold is to ensure that the selected node score is significantly higher than the overall average level, exceeding the mean by 1.5 standard deviations, and belongs to the core knowledge unit. It also avoids the threshold being too high due to a large standard deviation, causing some important nodes to be missed. According to this threshold, knowledge units with node scores higher than the threshold are selected to form an initial subset. For example, if the mean μ of a batch of node scores is 0.4 and the standard deviation is 0.2, the threshold is 0.4+1.5×0.2=0.7. Only nodes with scores higher than 0.7 are retained to ensure the core nature of the initial subset.

[0118] Subsequently, the initial subset is filtered for redundancy to avoid the existence of highly overlapping knowledge units in the subset, which may lead to redundant information in the subsequent generation. The core basis for redundancy judgment is the semantic overlap degree between knowledge units. The Jaccard similarity of the core semantic vocabulary sets corresponding to the key vectors of any two knowledge units in the subset is calculated to achieve this: first, extract the core semantic vocabulary corresponding to the key vector of each knowledge unit by using a keyword extraction algorithm such as TF-IDF to screen the top 10 weighted vocabularies, obtaining two vocabulary sets. Then, calculate the Jaccard similarity of the sets, which is the size of the intersection of the two sets divided by the size of the union. The higher the similarity, the more overlapping the core semantics of the two knowledge units, and the higher the redundancy. Set the redundancy judgment threshold to 0.7. If the Jaccard similarity of two knowledge units is higher than 0.7, it is determined that there is redundancy. At this time, compare the node scores of the two, and retain the knowledge unit with a higher score, as it is more closely related to the input sequence and has higher information value. Remove the knowledge unit with a lower score to avoid semantic repetition. Through the cooperative processing of adaptive threshold screening and redundancy filtering, the final subset of atomic knowledge units contains both core knowledge strongly related to the input sequence and no semantic redundancy, providing high-quality and efficient knowledge support for subsequent generation.

[0119] In a preferred embodiment of the present application, step 6 further comprises the following operation steps:

[0120] Step 61, monitor the generated context state changes, including calculating the semantic completeness of the generated sequence and the dispersion of the prediction probability distribution, quantifying the degree of uncertainty, the specific operation is as follows:

[0121] First, calculate the semantic completeness of the generated sequence: take the core semantic elements extracted from the input sequence such as user query as the benchmark, such as key problem points, requirement limiting conditions, and domain-specific terminology, calculate the number of core semantic elements covered in the generated sequence through semantic matching algorithm such as element coverage calculation based on bag-of-words model or element alignment degree analysis based on semantic vector, and at the same time, combine the logical coherence of the generated content, analyze whether the causal relationship, progression and other logical relationships between sentences are complete, such as whether there are unexplained concepts or incomplete logical deductions, to build a semantic completeness scoring model. The semantic completeness score output by this model ranges from 0 to 1. The closer the score is to 1, the more comprehensive the coverage of the generated sequence to the input core demand and the more coherent the logic. On the contrary, it means that the current generated content has semantic gaps or logical gaps, which may need external knowledge to supplement. Second, calculate the dispersion of the prediction probability distribution: focus on the prediction results of the next token to be generated by the model, extract the probability distribution of all candidate tokens output by the model, and use information entropy as the quantitative indicator of dispersion. The higher the entropy value, the more dispersed the probability distribution of the candidate token, and the more uncertain the model's generation direction for the next token. The lower the entropy value, the more concentrated the probability on a few candidate tokens, and the more explicit the model's generation direction. Finally, quantify the degree of uncertainty based on semantic completeness and prediction probability dispersion: build a fusion function to combine the semantic completeness score and the prediction probability dispersion, i.e. the entropy value, for example, when the semantic completeness is less than 0.5 and the dispersion is high, the entropy value is greater than the preset threshold, it is determined that the current context has high uncertainty, which means that the model neither meets the core semantic demand nor has a clear generation direction, and it needs to inject knowledge. When the semantic completeness is high, greater than 0.8, and the dispersion is low, the entropy value is less than the preset threshold, it is determined that the degree of uncertainty is low, and the model generation state is stable, without the need for additional knowledge injection. Through this multi-dimensional quantification method, the dynamic changes of the generated context can be captured in real time and objectively.

[0122] Step 62, based on the generated context state, calculate the semantic matching degree of atomic knowledge units, including direct semantic similarity and semantic coherence with subsequent generation direction, use bidirectional semantic projection to evaluate the adaptation degree, the specific operation is as follows:

[0123] First, the direct semantic similarity is calculated, and the semantic vector of the generated sequence is calculated with the key vector of the atomic knowledge unit. The semantic vector is obtained by encoding the pre-trained language model, which integrates the overall semantic information of the generated content. The key vector is generated by step 13, which represents the core semantics of the knowledge unit. In addition, the cosine similarity is used as the measurement index. The higher the similarity score, the higher the overlap between the core semantics of the knowledge unit and the semantics of the current generated sequence, the stronger the direct adaptability. For example, if the generated sequence is about "diabetes drug contraindications", the key vector of a certain knowledge unit corresponds to "contraindications for oral hypoglycemic drugs for type 2 diabetes", and the direct semantic similarity between the two is significantly higher than that of the knowledge unit corresponding to "dose adjustment of antihypertensive drugs". Second, the semantic coherence of the subsequent generation direction is evaluated: first, based on the current generated sequence, the semantic vector of the future 1-3 tokens to be generated is predicted by the generation model, i.e. the semantic representation of the generation direction is predicted, and then the similarity between the key vector of the atomic knowledge unit and the predicted semantic vector is calculated. The similarity reflects the support ability of the knowledge unit to the subsequent generation direction. The higher the similarity, the more natural the semantic connection between the knowledge unit and the subsequent generated content, avoiding semantic jumps. The lower the similarity, the more likely it is to cause logical breaks in the generated content. Finally, the adaptation degree is evaluated using bidirectional semantic projection: considering that there may be local semantic deviations between the semantic vector of the generated sequence and the key vector of the knowledge unit, such as different expressions of the same concept leading to vector differences, the two types of vectors are projected into a pre-set shared semantic subspace, which is pre-trained based on domain corpus, which can eliminate the vector offset caused by expression differences. The cosine similarity of the projected vectors is calculated as a correction and supplement to the direct semantic similarity and subsequent coherence. The results of the above three dimensions are combined by weighted summation, such as direct semantic similarity weight 0.5, subsequent coherence weight 0.3, and bidirectional projection adaptation weight 0.2, and then the final semantic matching score is obtained. The score value is [0, 1], which provides a basis for the subsequent gating function decision.

[0124] Step 63, based on the semantic matching degree evaluation result, the dynamic gating function is executed, and the intensity ratio and injection timing of knowledge injection are adjusted according to the matching degree score and context uncertainty degree. The specific operation is as follows:

[0125] First, define the logic of the dynamic gating function. The output of the gating function is the knowledge injection weight, which is represented by the control intensity ratio, and the injection trigger signal, which is represented by the control timing. The decision-making basis follows the business rule that the higher the semantic matching degree and the higher the context uncertainty, the stronger the need for knowledge injection, avoiding semantic redundancy caused by indiscriminate injection or generation defects caused by missing key nodes. The calculation of the knowledge injection weight uses the Sigmoid activation function to construct the formula: ;

[0126] where g is the knowledge injection weight, taking value [0, 1]. Sigmoid function, the input is mapped to the interval [0, 1], ensuring that the weight is controllable; The semantic matching degree score is, The context uncertainty degree is, And The balance coefficient, the preset value range is [0.8, 1.2], is optimized through domain training data, and is used to balance the influence of matching degree and uncertainty on the weight; the derivation of the formula is that the knowledge injection demand is the collaborative result of matching degree and uncertainty, the matching degree determines whether the knowledge is suitable, and the uncertainty determines whether the knowledge is needed, both of which integrate the core influencing factors through linear combination, and the Sigmoid function ensures that the weight output is smooth and in a reasonable interval, avoiding the imbalance caused by extreme values; g directly controls the proportion of semantic information of the knowledge unit in the generation process, and when g is close to 1, the semantic information of the knowledge unit is deeply integrated into the generation, and when g is close to 0, only a small amount or no knowledge is integrated. Secondly, determine the knowledge injection time: the gating function decides the time by monitoring the change of the injection trigger signal, when g is higher than the preset injection threshold, such as 0.6, for two consecutive generation steps, and the current generated token is the first token after the logical node, such as “for example” “therefore” and the like, or the starting token of the domain concept explanation, trigger knowledge injection, at this time, the injected knowledge can naturally connect the current generated logic, avoiding interrupting the semantic flow; if g is lower than the threshold or the current token is a non-logical node, such as an adjectival modifier or a punctuation mark, the injection is delayed until the trigger condition is met, through this dynamic gating mechanism of weight control intensity and signal control time, the deep collaboration of knowledge injection and generation process can be realized, which not only ensures that the knowledge is accurately supplemented when needed, but also avoids excessive injection interference with the generation continuity, finally improves the quality and naturalness of knowledge-enhanced generation.

[0127] In a preferred embodiment of the present application, step 7 further comprises the following operation steps:

[0128] Step 71, extracting the semantic consistency, factual accuracy and logical coherence quality indicators of the generated result, analyzing the explicit score and implicit behavior data of the user feedback, calculating the feedback confidence score, establishing the association mapping of the quality indicators and the feedback confidence to generate a comprehensive evaluation signal, and the specific operations are as follows:

[0129] First, three quality indicators of the generated results are extracted. The semantic consistency indicator is obtained by encoding the semantic vector of the generated results through the pre-trained language model, and then performing cosine similarity calculation with the semantic vector of the input sequence and the semantic vector of the injected atomic knowledge unit respectively, and taking the average of the two as the quantitative result. The closer the value is to 1, the smaller the semantic deviation of the generated content from the input demand and the injected knowledge, and the stronger the consistency. The factual accuracy indicator is obtained by comparing the factual expressions in the generated results, such as domain data, concept definition, and rule description, with the authoritative atomic knowledge units in the knowledge base sentence by sentence, and calculating the proportion of expressions without factual deviation. The higher the proportion, the higher the accuracy. If there is a factual conflict, such as data error or concept confusion, the proportion is reduced according to the severity of the conflict, such as a 10% reduction for a minor conflict and a 50% reduction for a serious conflict. The logical coherence indicator is obtained by analyzing the logical relationships between sentences in the generated results, such as cause and effect, progression, parallelism, and transition, and using a logical relationship recognition model to quantify the integrity and rationality of the logical chain. For example, if there are unexplained cause and effect breakpoints or contradictory progression relationships, the coherence score decreases, and vice versa. The three indicators are finally normalized to the [0, 1] interval.

[0130] Second, analyze user feedback data and distinguish between explicit ratings and implicit behavior data. Explicit ratings directly extract user ratings of the generated results, such as 1-5 stars or satisfaction ratings, such as 0-10 points, and normalize them as explicit feedback values. Implicit behavior data is obtained by analyzing user interactions with the generated results, such as reading time, which is considered positive feedback if it is 1.5 times longer than the average reading time, and negative feedback if it is less than 0.5 times. Clicking on details, supplementary explanations, and other operations are considered positive feedback. Re-submitting after modifying the original query is considered negative feedback on the current generated result. Each type of implicit behavior is quantified as an implicit feedback value based on pre-set weights, such as a positive behavior weight of +0.2 and a negative behavior weight of -0.1. Then, calculate the feedback confidence score, which measures the reliability of user feedback. For explicit ratings, if a user submits ratings multiple times for the same query and the difference is less than a pre-set threshold, such as 0.1, or the user is a high-confidence user marked by the system, such as a domain expert user, the confidence score increases, up to 1.0. If the ratings fluctuate greatly or the user has no historical reliable interaction record, the confidence score decreases, down to 0.3. For implicit behavior data, if multiple implicit behaviors point in the same feedback direction, such as long reading time and clicking on details, which are positive, the confidence increases. If the behaviors are contradictory, such as long reading time but re-submitting the query, the confidence decreases.

[0131] Finally, establish a correlation mapping between quality indicators and feedback confidence to generate a comprehensive evaluation signal, using a weighted fusion formula: ;

[0132] wherein For comprehensive evaluation of the signal, the value is [0, 1]; For generating the weighted mean of the three quality indicators of the result, the weights of semantic consistency, factual accuracy and logical coherence are 0.3, 0.4 and 0.3 respectively; For the normalized explicit feedback value, For the explicit feedback confidence score, For the normalized implicit feedback value, For the implicit feedback confidence score, And For the balance coefficient, it satisfies , the preset , , which can be optimized according to the field scene. The derivation of this formula is based on the demand that the objective quality of the system and the subjective feedback of the user need to be considered together. The quality indicator reflects the objective performance of the generated result, and the feedback data reflects the subjective experience of the user. After the weighted fusion of the two, the deviation of single-dimensional evaluation can be avoided, such as high objective quality but poor user experience, or vice versa. The synergistic effect of the parameters in the formula ensures Both the technical performance of the generated result and the actual needs of the user can be reflected, and a comprehensive evaluation basis is provided for subsequent parameter adjustment.

[0133] Step 72, based on the comprehensive evaluation signal and the confidence weight, the dynamic priority weight and the contribution coefficient are incrementally adjusted. According to the high and low of the confidence score, the reinforcement learning type optimization or the smoothing adjustment strategy is adopted, and the tracking mechanism of the parameter update trajectory is established. The specific operation is as follows:

[0134] First of all, the core logic of incremental adjustment is clear: the comprehensive evaluation signal is higher than the preset excellent threshold, such as 0.8, which means that the current dynamic priority weight and the contribution coefficient are adapted to the current scene, and only need to be fine-tuned based on the newly generated result and the feedback to maintain the stability of the system; when is lower than the preset qualified threshold, such as 0.6, which means that the parameters have obvious adaptation problems and need to be adjusted according to the defect direction reflected by , such as is caused by poor factual accuracy, which may be caused by deviation of high priority knowledge unit; caused by semantic redundancy, which may be caused by too high contribution coefficient; when is between the two thresholds, moderate adjustment is made according to the proportion of the defect degree.

[0135] Secondly, according to the feedback confidence score, the differential adjustment strategy is selected: when the feedback confidence score is higher than the high confidence threshold, such as 0.8, the reinforcement learning type optimization strategy is adopted, and the comprehensive evaluation signal is used as the reward signal to build the reward function: ;

[0136] wherein is a parameter adjustment amplitude penalty coefficient, is a parameter adjustment amplitude, to avoid over-adjustment, if R is positive, it indicates that the current parameter adjustment direction is correct, continue to optimize in this direction in the next round of adjustment, such as further increasing the dynamic priority weight of a certain knowledge unit; if R is negative, adjust in the opposite direction to correct the deviation, strengthen the effective adjustment direction through the reward mechanism, accelerate the parameter convergence, when the feedback confidence score is lower than the low confidence threshold, such as 0.5, a smoothing adjustment strategy is adopted-the parameter adjustment amplitude is limited in a pre-set small range, such as the dynamic priority weight is not more than ±0.1 each time, the contribution coefficient is not more than ±0.05 each time, the moving average algorithm, such as the mean of the last 3 evaluation signals, is used to filter the noise interference of low-confidence feedback, avoid large fluctuations of parameters caused by unreliable feedback, and ensure the stability of the adjustment process.

[0137] In the specific parameter adjustment, the adjustment of the dynamic priority weight focuses on the atomic knowledge unit that is recognized by both the user feedback and the quality indicator, for the knowledge unit that contributes significantly to the generated result, such as high factual accuracy, the user clicks to view the details of the associated knowledge unit, and the priority weight is increased by the product of the comprehensive evaluation signal and the feedback confidence; for the knowledge unit that leads to generation defects, such as factual errors and semantic redundancy, the weight is reduced according to the corresponding defect degree, the adjustment of the contribution coefficient is based on the knowledge dependency of the generated result: if there is excessive injection of knowledge in the generated result, such as semantic redundancy and deviation from the input requirements, the contribution coefficient is reduced to reduce the impact of knowledge on generation; if there is insufficient injection of knowledge, such as factual omission and semantic disconnection with knowledge, the contribution coefficient is increased to strengthen the knowledge support, finally, a parameter update trajectory tracking mechanism is established: the timestamp of each parameter adjustment, the comprehensive evaluation signal value that triggers the adjustment, the feedback confidence score, the parameter value before adjustment, the adjustment amplitude, the adjustment strategy type, and the quality indicator change of the generated result after adjustment are recorded, forming a complete parameter adjustment log, which can be used to trace back the parameter optimization process, analyze the effectiveness of parameter adjustment in different scenarios, provide data support for subsequent optimization strategy iteration, and at the same time ensure the reproducibility and explainability of parameter adjustment.

[0138] The above merely describes the preferred embodiments of the present application; however, the protection scope of the present application is not limited thereto. Any person skilled in the art can make equivalent replacements or changes to the technical solutions of the present application and the improved concepts thereof within the technical scope disclosed by the present application, which should be covered by the protection scope of the present application.

Claims

1. A dynamic feature retrieval, generation, and management method based on cross-attention mechanism, characterized in that, include: Step 1: Convert knowledge base document fragments into atomic knowledge units. Each atomic knowledge unit includes question-answer pairs, encoded as key-value pairs, and assigned dynamic priority weights. Step 2: Replace the self-attention query with a dual-channel query structure, with one channel used for cross-attention. Generate a cross-attention query vector through linear transformation, using key-value pairs of atomic knowledge units, and calculate the cross-attention matrix based on the cross-attention query vector and the key-value pairs of atomic knowledge units. Step 3: Train the cross-attention adapter, freeze the language model weights, optimize the adapter parameters, and dynamically adjust the loss function using prediction parameters based on the contextual complexity of the input sequence. Step 4: Based on the cross-attention matrix, calculate the relevance score between the input sequence and each atomic knowledge unit, weight and fuse the attention scores using a preset sequence position weight function, and apply the contribution coefficient; Step 5: Based on the relevance score, construct a knowledge topology network based on the relevance score and semantic overlap, and retrieve a subset of atomic knowledge units using a graph diffusion algorithm; Step 6: During the generation process, a gating function based on the matching degree between the current generation context and the atomic knowledge unit is used to dynamically control the knowledge injection; Step 7: Based on the generated results and user feedback, adjust the dynamic priority weights and contribution coefficients, and update the parameters using incremental learning.

2. The dynamic feature retrieval generation and management method based on cross-attention mechanism according to claim 1, characterized in that, Step 1 includes: Step 11: Based on the document's logical structure and semantic density, identify semantic unit boundaries by analyzing punctuation density, term frequency, and sentence complexity. Step 12: Within the semantic unit, analyze the logical relationships between statements, identify the semantic patterns of question-answer, argument-evidence, and problem-solution, extract key propositions as questions and associate supporting content as answers, form question-answer pairs and attach structural tags; Step 13: Perform dual-channel encoding on the question-answer pairs, encode the question text into a discriminative vector, encode the answer text into a capacity vector, and map it to a unified semantic space through projection transformation to form key-value pairs; Step 14: Calculate the semantic similarity between atomic knowledge units using key-value pair semantic vectors, construct a knowledge unit association network, and calculate the initial priority weights based on node centrality index and connection density.

3. The dynamic feature retrieval generation and management method based on cross-attention mechanism according to claim 2, characterized in that, Step 2 includes: Step 21: Perform semantic role analysis on each token in the input sequence, assign semantic role labels, and allocate two query dimension spaces for each token based on the semantic role labels, including internal semantic association channels and external knowledge interaction channels; Step 22: Using semantic role labels and query dimension space, perform channel-specific transformation on the initial representation of each token. The internal semantic association channel adopts linear transformation, while the external knowledge interaction channel adopts non-linear transformation. The transformation matrix is ​​grouped with shared parameters according to the semantic role labels.

4. The dynamic feature retrieval generation and management method based on cross-attention mechanism according to claim 3, characterized in that, Step 2 also includes: Step 23: Perform semantic focusing processing on the query vector of the external knowledge interaction channel, including calculating the initial relevance distribution with the key vector of the knowledge unit, identifying the semantic focus region, and performing local semantic enhancement transformation; Step 24: Dynamically fuse the output of the internal semantic association channel and the output of the enhanced cross-attention channel, and calculate the fusion weight based on the current decoding state and the semantic complexity of the input sequence.

5. The dynamic feature retrieval generation and management method based on cross-attention mechanism according to claim 4, characterized in that, Step 3 includes: Step 31: Calculate the semantic distribution dispersion and context dependency strength of tokens in the input sequence to generate a context complexity score; Step 32: Using context complexity scores, generate global prediction parameters at the sequence level and local prediction parameters at the token subset level, and fuse multi-scale prediction parameters through a nonlinear combination function; Step 33: Based on multi-scale prediction parameters, adjust the weight coefficients and weighting factors of the standard loss function, adjust the overall weight coefficients of the loss function using global prediction parameters, and introduce token-level weighting factors using local prediction parameters; Step 34: Under the guidance of the reconstructed loss function, optimize and update the adapter parameters. Identify the update direction of key parameters based on the gradient distribution, prioritize the adjustment of parameter paths corresponding to high-complexity contexts, and control the update magnitude through gradient pruning mechanism while keeping the language model weights frozen.

6. The dynamic feature retrieval generation and management method based on cross-attention mechanism according to claim 5, characterized in that, Step 4 includes: Step 41: Identify and filter semantic redundant connections in the cross-attention matrix based on attention kurtosis, including analyzing the distribution characteristics of attention weights to identify abnormal connection points with high variance, and performing smoothing suppression on redundant connections. Step 42: Based on the purified cross-attention matrix, calculate the initial relevance score of each atomic knowledge unit, including the purified attention weights of all tokens in the aggregated input sequence, using the geometric mean aggregation function, and record the distribution characteristics of the attention tokens.

7. The dynamic feature retrieval generation and management method based on cross-attention mechanism according to claim 6, characterized in that, Step 4 also includes: Step 43: Utilize the distribution characteristics of the tokens that have attracted attention, apply a preset sequence position weight function to adjust the initial relevance score, including assigning importance coefficients based on the relative position of the tokens in the sequence, and assigning higher weight coefficients to the tokens of key semantic nodes, and obtain the position-aware relevance score through weighted fusion. Step 44: Based on the location-aware relevance score, the contribution coefficient is applied for final balancing, including dynamically adjusting the contribution coefficient according to the semantic complexity of the input sequence and the size of the knowledge base, and generating the final relevance score through a combination of preset nonlinear functions.

8. The dynamic feature retrieval generation and management method based on cross-attention mechanism according to claim 7, characterized in that, Step 5 includes: Step 51: Construct a weighted directed graph based on relevance scores and semantic overlap, where nodes represent atomic knowledge units, node weights are assigned after normalization of relevance scores, and edge weights are calculated by measuring the cosine similarity of the knowledge unit key vectors and combining them with contextual dependencies. Step 52: Perform multi-round graph diffusion processing on the weighted directed graph. During the diffusion process, each node calculates the attention score based on the edge weight and its own node weight, and dynamically adjusts the information transmission intensity. Step 53: Based on the diffused node scores, an adaptive threshold mechanism based on the statistical characteristics of score distribution is used to extract a subset of atomic knowledge units, and the subset is then subjected to redundancy filtering.

9. The dynamic feature retrieval generation and management method based on cross-attention mechanism according to claim 8, characterized in that, Step 6 includes: Step 61: Monitor changes in the generated context state, including calculating the semantic completeness of the generated sequence and the dispersion of the predicted probability distribution, and quantifying the degree of uncertainty; Step 62: Based on the generated context state, calculate the semantic matching degree of the atomic knowledge unit, including direct semantic similarity and semantic coherence with subsequent generation direction, and use bidirectional semantic projection to evaluate the degree of fit. Step 63: Based on the semantic matching degree evaluation results, execute the dynamic gating function to adjust the intensity ratio and timing of knowledge injection according to the matching degree score and the degree of contextual uncertainty.

10. The dynamic feature retrieval generation and management method based on cross-attention mechanism according to claim 9, characterized in that, Step 7 includes: Step 71: Extract the semantic consistency, factual accuracy and logical coherence quality indicators of the generated results, analyze the explicit ratings and implicit behavioral data of user feedback, calculate the feedback confidence score, establish the correlation mapping between quality indicators and feedback confidence to generate a comprehensive evaluation signal. Step 72: Based on the comprehensive evaluation signal and confidence weight, the dynamic priority weight and contribution coefficient are incrementally adjusted. Reinforcement learning optimization or smooth adjustment strategies are adopted according to the confidence score, and a tracking mechanism for parameter update trajectory is established.

Images