Knowledge base intelligent retrieval method based on large language model

Abstract

The invention provides a knowledge base intelligent retrieval method based on a large language model, which comprises the following steps: integrating user query and at least three rounds of time sequence contexts, and adopting standardized text preprocessing to generate structured input; using a pre-training language model and a position awareness attention mechanism to jointly encode dialogue content to form a context sensitive semantic vector; a two-channel vector updating structure is constructed, and a knowledge vector state is jointly controlled through semantic consistency comparative learning and attention distribution evolution detection; a differentiable memory gating unit is adopted to update and retain dynamic vectors; a vector backtracking mechanism is designed to improve multi-round semantic coherence, multi-modal user feedback signals are collected, and continuous online optimization of gating parameters is supported.

Description

Technical Field

[0001] This invention relates to the field of multi-turn dialogue semantic modeling and dynamic knowledge vector update and optimization technology, and in particular to a knowledge base intelligent retrieval method based on a large language model. Background Technology

[0002] Currently, intelligent knowledge base retrieval and multi-turn dialogue state modeling based on large language models have become important research directions in the field of natural language processing. Mainstream technologies generally employ pre-trained language models (such as BERT and GPT) to encode user queries and historical dialogue context, transforming multi-turn dialogues into vector representations to achieve semantic retrieval and intelligent responses. Building upon this foundation, related research is increasingly focusing on the dynamic changes in the state of knowledge representation within multi-turn dialogues, enhancing semantic awareness and response coherence by introducing conversation history, context windows, and attention mechanisms.

[0003] Currently, while existing technologies have achieved a certain degree of dialogue context modeling and knowledge state tracking, the following prominent technical bottlenecks and unmet needs still exist: (1) Existing multi-turn dialogue knowledge vector modeling methods mostly adopt static or weakly dynamic update mechanisms, which are insufficient in handling the contradiction between semantic consistency and context evolution. Knowledge vectors usually maintain the core historical semantics but are difficult to effectively adapt to drastic changes in the dialogue context. Once the user raises a new intention or the topic jumps, the model is prone to semantic drift or forgetting key historical states, resulting in the response content being out of touch with the historical context, affecting the accuracy of retrieval and response; (2) Mainstream methods lack detailed criteria and mechanisms for determining whether knowledge vectors should be updated and how to smoothly transition. The semantics of "anchor points" in dialogues are difficult to capture dynamically, and the determination of evolutionary signals is often ambiguous, lacking fine-grained gating that follows the actual changes in dialogue. This leads to some systems being either overly conservative and unable to adapt to new intentions in dialogue in a timely manner, or updating too frequently, losing the accumulation of core semantics, and failing to balance coherence and flexibility in multi-turn interactions; (3) The state evolution of knowledge vectors lacks a traceable and controllable management mechanism. Most publicly available solutions do not provide efficient semantic back lookup and consistency compensation methods. Once a semantic shift occurs in the system, it is difficult to promptly correlate with historical states to achieve self-correction. Most caching and retrieval technologies are only used to accelerate vector lookup rather than support context consistency guarantees, which inevitably affects the robustness and interpretability of the system. (4) Currently, only a very few technologies have established closed-loop feedback optimization, that is, a dynamic update strategy that automatically adjusts the knowledge vector based on actual user interaction data (such as dwell time, depth of follow-up questions, etc.). They lack online fine-tuning and adaptive optimization capabilities, and cannot continuously optimize the evolution process of the knowledge vector state according to the real needs of the scenario and user feedback. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an intelligent knowledge base retrieval method based on a large language model.

[0005] The technical solution of this invention is implemented as follows: a knowledge base intelligent retrieval method based on a large language model, comprising: S1: Receive the current user's query text and historical dialogue context sequence data, wherein the dialogue context sequence contains the text content and timestamp information of at least three consecutive dialogue rounds; S2: Construct a dialogue state-aware encoder based on a pre-trained language model and a position-aware attention mechanism. Input the current user query text and the historical dialogue context sequence into the encoder to generate an initial semantic vector containing context sensitivity and output a vector space representation with a dimension of 768. S3: Construct a dual-channel vector update structure, where the first channel is the semantic consistency channel: construct a set of semantic anchor points based on the key dialogue rounds within the sliding window, calculate the cosine similarity between the current semantic vector and the anchor points through a contrastive learning mechanism, and generate stability weight coefficients; the second channel is the context evolution channel: use the attention distribution change rate detection algorithm to identify semantic shift trends and output evolution signal strength values; S4: Establish a differentiable memory gated unit, which receives the stability weight coefficient and the evolution signal strength value as input parameters, and generates dynamically updated weight parameters through the sigmoid activation function. When the stability weight coefficient is greater than 0.8 and the evolution signal strength value is less than 0.2, the current semantic vector remains unchanged; otherwise, the current semantic vector and the newly generated semantic vector are fused using a weighted residual connection method to obtain the updated target knowledge vector. S5: Construct a vector backtracking mechanism to store the target knowledge vector and its timestamp information after each update into a lightweight vector cache pool, and construct a semantic similarity retrieval structure based on the inverted index to support the most recent state backtracking operation based on the maximum inner product search in subsequent dialogue rounds; S6: Collect user response feedback signals and implicit behavior data, including page dwell time, follow-up question depth, and click sequence. Calculate the gating parameters and adjust the gradient based on the feedback signals. Update the weight parameter matrix of the memory gating unit through online incremental learning to complete the system closed-loop optimization.

[0006] The intelligent knowledge base retrieval method based on a large language model provided by this invention has the following beneficial effects: (1) This invention significantly improves the adaptability to dynamic semantic evolution by introducing a dialogue state-aware encoder and a dual-channel vector update mechanism. It adopts a dual-channel architecture with semantic consistency channel and context evolution channel working together. On the one hand, by constructing a set of semantic anchor points within a sliding window and calculating semantic distance, it effectively identifies and maintains the stability of the core dialogue topic and prevents knowledge drift caused by noise interference. On the other hand, it detects semantic shift trends based on the rate of change of attention distribution, timely captures new intentions or topic jumps, and triggers reasonable knowledge update signals. This dual-criteria mechanism overcomes the lag and oversensitivity defects caused by relying solely on recent input or fixed memory length, and realizes accurate perception and response guidance of the user's intention evolution process, thereby significantly enhancing the system's coherent understanding ability and context fusion depth in complex multi-turn dialogue scenarios. (2) This invention designs a state-based management framework that combines a differentiable memory gating unit with a vector backtracking mechanism, realizing a smooth transition and reversible verification of knowledge evolution. The gating mechanism dynamically adjusts the fusion strength of new and old knowledge. When the semantics are consistent and there are no obvious evolution signals, the original vector is maintained to ensure stability. When reasonable semantic migration is detected, residual connections are used to achieve progressive updates to avoid semantic breaks caused by sudden changes. At the same time, the lightweight vector cache pool, combined with timestamps and semantic similarity indexes, supports fast retrieval and consistency comparison of historical states. It can actively backtrack key nodes in subsequent rounds to verify the logical coherence of the current reasoning path, further enhancing the reliability and interpretability of multi-round reasoning. (3) This invention introduces an online fine-tuning mechanism based on user behavior feedback, constructing a closed-loop optimized learning system. Unlike traditional offline training modes that cannot respond to personalized expressions and scenario differences in actual use, this invention utilizes implicit user feedback signals (such as response dwell time, follow-up question frequency, and jump behavior) as reward indicators to dynamically adjust the control parameters in the gating unit, enabling the system to autonomously optimize the knowledge update strategy based on actual interaction effects. This mechanism not only enhances the model's adaptability to different user language habits and interaction rhythms but also realizes the transformation from "passive response" to "active learning," improving the relevance and satisfaction of recommendation and retrieval results. Attached Figure Description

[0007] Figure 1 This is a flowchart of the intelligent knowledge base retrieval method based on a large language model according to the present invention; Figure 2 This is a sub-flowchart of the intelligent knowledge base retrieval method based on a large language model according to the present invention. Detailed Implementation

[0008] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0009] The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the invention. Furthermore, reference numerals and / or letters may be repeated in different examples; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0010] like Figure 1 As shown, this invention provides an intelligent knowledge base retrieval method based on a large language model, specifically including: S1: Receive the current user's query text and historical dialogue context sequence data, wherein the dialogue context sequence contains the text content and timestamp information of at least three consecutive dialogue rounds; S2: Construct a dialogue state-aware encoder based on a pre-trained language model and a position-aware attention mechanism. Input the current user query text and the historical dialogue context sequence into the encoder to generate an initial semantic vector containing context sensitivity and output a vector space representation with a dimension of 768. S3: Construct a dual-channel vector update structure, where the first channel is the semantic consistency channel: construct a set of semantic anchor points based on the key dialogue rounds within the sliding window, calculate the cosine similarity between the current semantic vector and the anchor points through a contrastive learning mechanism, and generate stability weight coefficients; the second channel is the context evolution channel: use the attention distribution change rate detection algorithm to identify semantic shift trends and output evolution signal strength values; S4: Establish a differentiable memory gated unit, which receives the stability weight coefficient and the evolution signal strength value as input parameters, and generates dynamically updated weight parameters through the sigmoid activation function. When the stability weight coefficient is greater than 0.8 and the evolution signal strength value is less than 0.2, the current semantic vector remains unchanged; otherwise, the current semantic vector and the newly generated semantic vector are fused using a weighted residual connection method to obtain the updated target knowledge vector. S5: Construct a vector backtracking mechanism to store the target knowledge vector and its timestamp information after each update into a lightweight vector cache pool, and construct a semantic similarity retrieval structure based on the inverted index to support the most recent state backtracking operation based on the maximum inner product search in subsequent dialogue rounds; S6: Collect user response feedback signals and implicit behavior data, including page dwell time, follow-up question depth, and click sequence. Calculate the gating parameters and adjust the gradient based on the feedback signals. Update the weight parameter matrix of the memory gating unit through online incremental learning to complete the system closed-loop optimization.

[0011] Step S1: Receive the current user's query text and historical dialogue context sequence data, wherein the dialogue context sequence contains the text content and timestamp information of at least three consecutive dialogue rounds. Specifically, this includes: S1.1: Perform raw text acquisition processing on the current user query text from the user interaction module to extract user input content in natural language form, which serves as the basic input unit for semantic modeling; The current user query text data output by the user interaction module is monitored by a text stream monitoring mechanism (parameters: character encoding UTF-8, buffer size 1-4096 bytes, preferably 1024-2048 bytes) to capture the user's natural language input signal in real time and convert it into a string object that can be read by the downstream processing module. For the captured raw string object, a regular expression parsing algorithm is used to filter illegal characters and verify the legality of input, and generate a text content result containing only parsable natural language symbols; Furthermore, a language recognition algorithm (parameters: supports Chinese and English, confidence threshold 0.85) is used to automatically determine multiple languages ​​and identify the language type labels required for subsequent word segmentation and encoding strategies. Furthermore, by using a unified character set encoding algorithm (parameter: target encoding UTF-8), cross-platform consistent encoding processing is achieved, and a standardized character sequence that can be directly passed to the semantic modeling unit is obtained; Furthermore, by using the data frame construction method (parameters: fields = original text, language tag, capture timestamp), the standardized character sequence is encapsulated into a text data frame object, providing a structured input basis for subsequent contextual retrieval steps; The above algorithm transforms the captured character stream from the previous step into natural language text data with a unified encoding format and recognition labels, achieving the expected technical effect of structuring and standardizing the input data in the semantic modeling stage. For example, in an intelligent customer service system, the current user's query text is transmitted to the server via the WebSocket protocol, using a UTF-8 encoding listening mechanism with a buffer size configured to 2048 bytes, capturing continuous input character streams in real time. The captured raw string is: "Please help me check last month's bill". After filtering out control characters using a regular expression parsing algorithm, the plain text content is retained. The language recognition model calculates the probability of this input Chinese based on the training corpus to be 0.987, which is greater than the confidence threshold of 0.85, and is therefore classified as Chinese. Subsequently, a unified encoding algorithm is used to standardize the Chinese character sequence into UTF-8 format, generating a byte sequence that can be used by a deep learning network. The encapsulated data frame structure includes the fields: {raw text: "Please help me check last month's bill", language tag: "zh-CN", capture timestamp: 1678945632}. This structured data frame is directly passed to the dialogue state management module, where it is combined with the historical dialogue context in subsequent step S1.2 to perform retrieval and construction, realizing context-aware processing of multi-turn dialogues, reducing system response time, and significantly improving semantic modeling accuracy. S1.2: Based on the dialogue state manager, perform a retrieval operation on the historical dialogue context sequence to obtain the text content of at least three consecutive dialogue rounds and their corresponding timestamp information in order to construct a complete context-aware window; The input data includes a historical dialogue context log file stored by the dialogue state manager, which contains a text content field and a corresponding timestamp field. The timestamps are recorded at millisecond resolution to ensure accurate sorting of dialogue rounds. An index retrieval method (parameters: round index key = dialogue round number, timestamp key = millisecond-level timestamp) is used to quickly locate historical dialogue context log files in order to extract continuous round data within a specific time range; Furthermore, by using a sliding window retrieval algorithm (parameters: window length of 3-20 rounds, preferably 5-8 rounds, window step size = 1 round), the text content of consecutive rounds within the target time range is extracted, and a set of original context text containing no less than three rounds is obtained; Furthermore, a structured data aggregation method is adopted (parameters: aggregation key = round number, additional field = timestamp) to achieve the pairing and fusion of text content and timestamp, and generate serialized context record units for subsequent context-aware window construction; Furthermore, by using a window boundary verification algorithm (parameter: time difference threshold = preset maximum allowed interval, specifically 3000-60000 milliseconds, preferably 5000-30000 milliseconds), the time interval consistency verification of the extracted continuous round text set is realized, and a context sample set that conforms to the continuous dialogue logic is generated; Through the above algorithm processing method, the original user query data in the previous step is supplemented into a context-aware window containing at least three consecutive rounds of text and corresponding timestamps, so as to realize the complete context input from the dialogue state manager to the semantic modeling module; For example, in an enterprise intelligent customer service scenario, the system performs a retrieval operation on historical interaction logs stored in the dialogue state manager. The round index key is set to the dialogue round number, the timestamp key to a UTC millisecond timestamp, and the sliding window length is set to 5 rounds with a step size of 1 round to ensure sufficient contextual information coverage. When a user sends a query request for the 15th round, the system quickly locates the interaction records from rounds 11 to 15 through index retrieval and extracts the text and timestamps of these rounds from the logs. Window boundary validation is performed on the extracted results, with the maximum allowed time difference threshold set to 30,000 milliseconds. If the time interval between adjacent rounds does not exceed the threshold, the set is deemed to meet the continuity requirement. Finally, a context-aware window containing the text and timestamps of the five rounds is constructed for input to the subsequent dialogue state-aware encoder. Under the same scenario, the encoder can significantly improve its ability to recognize cross-round referentials and ellipsis expressions, enhancing the semantic coherence of multi-round dialogues. S1.3: Use timestamp information to perform time sequence verification on the text content in the historical dialogue context sequence to ensure the temporal consistency of the dialogue rounds and form an ordered dialogue context sequence input; Based on the processing objective of sub-step S1.3, at least three consecutive dialogue rounds of text and timestamps from S1.2 are used as input objects. Time sequence verification is performed to ensure the temporal consistency of context data and an ordered queue is output. A timestamp parsing method (parameter: timestamp precision in milliseconds) is used to parse the time information of each round of dialogue records and map the parsing results into a unified time value format for subsequent sorting processing; Furthermore, a time-based sorting algorithm (parameters: ascending order mode, stable sorting strategy) is used to sort the parsed timestamp vectors and generate a sorted index structure that corresponds one-to-one between dialogue content and timestamps. Furthermore, a continuity check algorithm (parameter: time interval threshold set to 5000 milliseconds) is used to detect the time interval within adjacent sorted indices and obtain the time interval vector result, which is used to determine whether the dialogue rounds are continuous. Furthermore, by using an abnormal round elimination method (parameter: elimination rule is that the time interval is greater than the threshold or the timestamp is repeated), the abnormal index items in the time interval vector are eliminated, and a verified timestamp and text matching sequence is generated. By using a doubly linked list reconstruction method, the verified timestamp and text matching sequence are transformed into an ordered dialogue context sequence input, thus achieving strict dependence on the temporal structure of the data in the subsequent semantic modeling stage. For example, in an enterprise's internal intelligent customer service scenario, the timestamps of three consecutive rounds of dialogue are "2024-03-21 14:22:05.123", "2024-03-21 14:22:08.452", and "2024-03-21 14:22:12.981", which, when parsed into a unified millisecond-level numerical format, are respectively , , An ascending stable sorting algorithm is used to obtain a sorted index structure [0,1,2] that maintains the original sequence order. The time interval is calculated using a time interval detection method. milliseconds and The milliseconds were all less than the 5000 millisecond threshold, therefore no abnormal turns were determined. The final ordered dialogue context sequence maintains the original text order, ensuring that the temporal relationship of the dialogue can be accurately reflected in the subsequent S1.4 standardization preprocessing and S2 semantic encoding, thereby significantly improving the consistency and contextual integrity of multi-turn semantic modeling; S1.4: Perform standardized preprocessing operations on the current user query text and the text content in the historical dialogue context sequence, including word segmentation, stop word filtering and stemming, to generate a text representation in a unified format; S1.5: Based on the standardized current user query text and historical dialogue context sequence text, perform the dialogue state identifier generation operation to form a structured input data format for subsequent semantic modeling.

[0012] Step S2: Construct a dialogue state-aware encoder based on a pre-trained language model and a position-aware attention mechanism. Input the current user query text and the historical dialogue context sequence into the encoder to generate an initial semantic vector containing context sensitivity, and output a 768-dimensional vector space representation. Specifically, this includes: S2.1: Construct a basic semantic coding network based on the BERT-base pre-trained language model to perform word-level vectorization encoding on the input text in order to obtain a context-independent initial embedding representation; For the standardized current user query text and historical dialogue context sequence text obtained after step S1, a basic semantic encoding network is constructed using the BERT-base pre-trained language model (parameters: 12 model layers, 12 self-attention heads, and 768 hidden unit dimensions) to achieve word-level vectorized encoding for natural language input. Furthermore, the WordPiece word segmentation algorithm (parameters: dictionary size 30000, maximum word segment length 128) is used to achieve sub-word level segmentation of the input text and obtain the corresponding word segment sequence index data. Furthermore, by using the embedding matrix lookup method (parameter: embedding matrix dimension 768×30000), the word slice index is mapped to a continuous vector space, and an initial vector representation matrix containing each word slice is obtained; Furthermore, by using a positional encoding generation method (parameter: the sine and cosine encoding period length is equal to the maximum length of the word sequence), the positional information of each word segment vector is enhanced, and an input representation matrix superimposed with positional relationships is obtained, which is used to improve the sensitivity of the BERT encoder to the sequence order; Furthermore, by using a multi-layer Transformer encoding module (parameters: 12 encoding layers, 3072 dimensions of hidden units in the feed-forward network), the contextual features of the input representation matrix are modeled, and the semantic information intermediate representation matrix is ​​extracted. The output is used as a context-independent initial embedding representation. Through the BERT-base semantic encoding network described above, the word segmentation and embedding representations of the previous step are transformed into semantic vector matrices, realizing word-level vectorized encoding of the input text, and providing an initial feature space for the subsequent introduction of position-aware attention mechanisms and context-sensitive modeling. For example, in a multi-turn dialogue scenario for enterprise intelligent customer service, WordPiece segmentation is performed on the standardized current user query text "Please help me check the order status" and the historical context "Yesterday I bought a mobile phone" "When will it arrive?", resulting in a word fragment sequence ["please", "help", "me", "query", "order", "status"] and historical sequence word fragment indices, totaling 42 word fragments. Using a BERT embedding matrix with a dictionary size of 30,000 and an embedding dimension of 768, each word fragment index is mapped to a 768-dimensional vector. Positional information is added to each word fragment vector using a sine and cosine positional encoding function. After 12 layers of Transformer encoding, a context-independent embedding matrix of the word sequence is obtained, with a shape of 42×768. This embedding matrix serves as the basic semantic representation, achieving complete capture of the semantic features of the input text in actual verification, significantly improving the ability to model the dialogue sequence and state when subsequently introducing position-aware attention. S2.2: Construct a position-aware attention mechanism module. This module generates position encoding vectors based on dialogue turn timestamp information and performs weighted fusion of the position encoding vectors through a learnable attention weight matrix to enhance the model's sensitivity to dialogue temporal relationships. Based on the input current user query text and historical dialogue context timestamp information, a time encoding generation method is used (parameters: timestamp sequence Δt, maximum hour span). This process converts discrete-time markers into continuous numerical representations, forming an initial time position encoding vector. Furthermore, through the sine-cosine periodic function mapping method (parameter: frequency coefficient) Using the dimension index k), the encoding of multi-scale periodic position patterns is achieved, and the original position vector set composed of different frequency components is obtained. The formula for calculating the k-th dimension encoding is as follows:

[0013] in, For the position encoding matrix, The standardized real value corresponding to the timestamp. For the k-th dimension frequency coefficient; Furthermore, a learnable location embedding mapping network (parameter: embedding matrix) is used. This enables non-linear projection of the original set of position vectors, generating precise position encoding vectors that the model can train. Furthermore, a location-aware attention weight fusion algorithm (parameters: query vector Q, key vector K, value vector V, location encoding vector P) is employed to jointly model the location encoding and attention calculation input, thereby obtaining a location fusion attention score matrix. The calculation formula is:

[0014] in, The dimension of the key vector. Let T be the position encoding matrix, and T be the matrix transpose. Furthermore, by using the attention score weighted vector fusion method (parameters: attention score A, value vector V), position-time sensitive information is incorporated into the context representation of the encoder, and a fused position-aware hidden layer representation output is generated. By using positional encoding generation and weighted fusion processing, the word-level embedding representation from the previous step is enhanced into contextual representation data that is sensitive to the temporal relationship of dialogue turns, thereby achieving the effect of temporal identification and sequential dependency capture technology when the model processes multi-turn dialogue input. For example, in an intelligent customer service dialogue scenario, the historical dialogue contains three consecutive rounds with timestamps of 10:00, 10:02, and 10:05, with a maximum hourly span. Set to 1 hour. After standardization, the time differences Δt are 0.0, 0.033, and 0.083 hours respectively. The location encoding dimension d is set to 64, and the frequency coefficient... According to the formula Calculations were performed, and the original encoding matrix size was obtained using a sine-cosine mapping, resulting in a size of 3×64. This was then achieved through the embedding matrix... (Randomly initialized, size 64×64) Projection yields a trainable position encoding matrix P, which is fused into the attention calculation formula. The resulting position fusion attention score matrix A, after softmax normalization, has a size of 3×3. This matrix assigns higher discriminative weights to rounds with larger time intervals, enabling the encoder to significantly improve the detection accuracy of cross-round topic changes in subsequent context modeling. The final generated fusion position-aware hidden layer representation has been verified to significantly improve the dialogue state discrimination ability and context sensitivity in multi-round dialogues. S2.3: Concatenate the current user query text with the historical dialogue context sequence into a joint input sequence, and perform word segmentation and vectorization processing on the joint input sequence to obtain word-level embedding sequences and dialogue round boundary markers; S2.4: Input the word-level embedding sequence into the basic semantic coding network, perform context modeling based on the multi-layer Transformer encoder, and generate an intermediate hidden layer representation sequence containing semantic information; The word-level embedding sequences obtained by word segmentation and vectorization are input into the basic semantic coding network. A multi-layer Transformer encoder structure is adopted (parameters: 12 layers, 768 hidden dimensions, and 12 attention heads) to realize the contextual dependency relationship modeling across word positions. Furthermore, through a self-attention mechanism (parameter: the key, query, and value matrices in the scaling dot product attention formula are all 64 in dimension), the semantic dependency weighting of each word vector in the embedding sequence on other word vectors is realized, and the splicing output of multi-head attention is obtained to form the first-stage intermediate representation containing global context relations; Furthermore, a position feedforward network (parameters: two-layer fully connected structure, first layer dimension is 3072, second layer dimension is 768, activation function is GELU) is used to perform non-linear mapping on the multi-head attention output sequence to enhance the feature abstraction capability of semantic representation and generate the second-stage intermediate representation; Furthermore, the input embedding sequence is added element-wise to the second-stage intermediate representation using a residual connection method, and gradient stabilization and representation standardization are achieved through layer normalization (parameter: epsilon=1e-12) to obtain the weighted fused third-stage intermediate representation; Furthermore, the self-attention, feedforward network, residual, and normalization operations are repeated for all Transformer layers to form a serialized multi-layer context encoding chain. After the 12th layer encoding is completed, the output is an intermediate hidden layer representation sequence that incorporates global semantic information, which serves as the input for subsequent positional encoding fusion processing. Through a multi-layer Transformer encoder chain, the word-level embeddings from the previous step are transformed into intermediate hidden layer representation sequences with context-dependent and global semantic information, thereby realizing deep correlation modeling of semantic features in the temporal and semantic dimensions. For example, in an enterprise intelligent customer service knowledge retrieval scenario, the current user query text is concatenated with the context of the last three rounds of historical dialogue to form a joint input sequence of length 128. After word vectorization, an embedding matrix of dimension 128×768 is obtained. This is input into a 12-layer Transformer encoder, with each layer having 12 attention heads and each head having a dimension of 64, satisfying a total dimension of 768. In the self-attention calculation, the scaled dot product attention formula is used:

[0015] in , , These are the query, key, and value matrices, respectively, with the square root of 64 serving as the scaling factor. The calculated attention weight matrix... After multiplying the resultant matrix, the multi-head output is concatenated, mapped to 768 dimensions via a feedforward network, and activated as a non-linear feature. Through a residual and normalization-stabilized training process, the final output at layer 12 is an intermediate representation sequence with a dimension of 128×768. In performance testing, the encoder's output significantly improves the semantic similarity score in the dialogue context continuity matching task, enabling it to more accurately capture the contextual dependencies of multi-turn dialogues and provide high-quality semantic input for subsequent positional encoding fusion and vector updates. S2.5: The intermediate hidden layer representation sequence is added and fused with the position encoding vector element by element to generate a context-aware semantic representation sequence that incorporates position information; S2.6: Perform pooling operation on the context-aware semantic representation sequence based on the dialogue turn boundary marker to extract the sentence vector representation of each dialogue turn, and map the sentence vector to a 768-dimensional vector space through a fully connected layer; S2.7: Perform a weighted average fusion operation on the 768-dimensional sentence vector, calculate the importance weights of each round of dialogue based on the attention mechanism, and generate the final initial semantic vector as the input to the subsequent dual-channel vector update structure.

[0016] Step S3: Construct a dual-channel vector update structure, where the first channel is a semantic consistency channel: construct a set of semantic anchor points based on key dialogue rounds within a sliding window, calculate the cosine similarity between the current semantic vector and the anchor points through a contrastive learning mechanism, and generate stability weight coefficients; the second channel is a context evolution channel: use an attention distribution change rate detection algorithm to identify semantic shift trends and output evolution signal strength values. Specifically, this includes: S3.1: Based on the initial semantic vector output by the dialogue state-aware encoder, the semantic clustering algorithm is used to perform clustering analysis on the semantic representation vectors of each round within the sliding window, identify representative core semantic nodes, and form a set of semantic anchor points as a reference benchmark for semantic consistency. Based on the semantic representation vector of the historical dialogue rounds within the sliding window as input data, the 768-dimensional semantic vectors of each round output by the dialogue state-aware encoder (parameters: pre-trained language model BERT-base, position-aware attention weight matrix) are used as the basic features for key semantic extraction. A semantic feature normalization processing method (parameters: Z-score normalization mean 0, variance 1) is adopted to unify the feature scale of semantic vectors in each round, eliminate the difference in feature dimensions between different rounds, and ensure the consistency of distance metric in clustering calculation. Furthermore, a semantic clustering algorithm (algorithm type: K-means clustering, parameter: number of clusters k = number of sliding window rounds / 2 and rounded up) is used to group the semantic representation vectors of each round within the sliding window based on their similarity, and the center vector of each cluster is obtained as a candidate core node. Furthermore, the cluster density score of each candidate core node is calculated using a semantic representativeness scoring method (the formula is based on the mean cosine similarity between the cluster center and the vector within the cluster). Furthermore, by using a representative rating sorting method (parameter: sorted in descending order of rating and taking the top m core nodes, m=3 is preferred), the semantic anchor points within the sliding window are filtered to ensure that the anchor point vectors have a stable reference benchmark role in the semantic consistency channel. By using the anchor set construction method, the set of m core node vectors obtained above is transformed into a semantic anchor set data structure, which serves as the input for subsequent cosine similarity calculation and stability weight generation, thereby establishing a reference benchmark for semantic consistency channels. For example, in a multi-turn dialogue scenario within an enterprise intelligent customer service system, the sliding window width is set to 6 turns. A dialogue state-aware encoder outputs six 768-dimensional semantic representation vectors, which are then normalized using Z-scores and input into a K-means clustering algorithm with a cluster size of 3. The center of each cluster is used as a candidate core node, and its representative score is obtained by averaging the cosine similarity of the vectors within the cluster. For example, the results are: Cluster 1: 0.92, Cluster 2: 0.88, Cluster 3: 0.85. The top three nodes with the highest scores are selected in descending order to form a semantic anchor set. This anchor set exhibits high stability in subsequent similarity matrix calculations, significantly improving the ability to maintain semantic consistency across consecutive dialogue turns without affecting the sensitivity of context evolution detection. Ultimately, this allows the retrieval system to maintain a high accuracy matching rate and contextual coherence across multiple consecutive question-and-answer turns. S3.2: Based on the cosine of the angle between each anchor point in the vector space and the current semantic vector, construct a semantic similarity matrix, and generate stability weight coefficients through normalization to measure whether the current semantics should remain stable; S3.3: Based on the attention weight matrix output by the dialogue state-aware encoder, calculate the difference in attention distribution between adjacent dialogue rounds, use the sliding window variance analysis method to identify significant change points, and output the semantic evolution signal intensity value; S3.4: The detection results are binarized and classified based on the preset evolution intensity threshold. If the evolution signal intensity value exceeds the set threshold, it is determined that there is a context evolution requirement, and the vector update mechanism is triggered. Based on the semantic evolution signal intensity value output by the context evolution channel, a dynamic threshold determination algorithm is used (parameter: evolution intensity baseline value). The context noise coefficient (κ) enables the classification and determination of evolution intensity. Furthermore, by using a normalization method (parameters: minimum value min, maximum value max), the interval standardization of the evolved signal intensity values ​​is achieved, and the normalized intensity values ​​are obtained. This is used for subsequent threshold comparisons; Furthermore, an adaptive threshold setting algorithm (parameter: base threshold) is used. Average intensity of historical dialogue ,variance ), to achieve the threshold The calculation is performed to generate an evolution judgment threshold that adapts to the characteristics of the interactive session; Furthermore, through a comparison method, the difference Δv between the normalized evolved signal intensity value and the adaptive decision threshold is calculated, and a binary classification result is generated. The formula for calculating the difference is:

[0017] Furthermore, the classification output encoding is achieved through a logic gating function (parameter: classification logic "Δv > 0"), which converts the detection result into a binary label, where label 1 indicates that there is a need for context evolution, and label 0 indicates that the current semantic state is maintained; Through the above-mentioned judgment and classification processing, the evolution signal strength value of the previous step is transformed into a judgment label for context evolution requirements, so as to achieve accurate identification of intention switching or topic jumping phenomena, and provide a clear evolution channel decision basis for the generation of dual-channel control signals. For example, in a certain enterprise intelligent customer service system, the attention change rate for five consecutive rounds of historical dialogue was calculated, and the evolution signal strength values ​​between each round were 0.15, 0.22, 0.31, 0.18, and 0.27, respectively. After normalization, the results were obtained... The base thresholds are set to 0.32, 0.47, 0.66, 0.38, and 0.58 respectively. It is 0.5, combined with the historical average intensity. Given σ² = 0.25 and variance σ² = 0.012, calculate the adaptive threshold. Where κ takes the value 0.4, the calculation result is =0.6. For each of the above rounds... and The difference operation yields Δv respectively. 0.28 0.13, 0.06, 0.22 Based on the classification logic of Δv > 0, only the third round satisfies the condition, outputting label 1 and triggering the vector update mechanism. In other rounds, label 0 is output, keeping the semantic vector unchanged. Validation results show that this decision effectively reduces unnecessary vector updates in the simulated dataset and ensures timely semantic state evolution during intent switching, significantly improving contextual adaptability and coherence in multi-turn dialogue scenarios. S3.5: Based on the stability weights of the semantic consistency channel output and the evolution signals of the context evolution channel output, a weighted fusion strategy is adopted to generate dual-channel output control parameters, which serve as the input signals of the differentiable memory gating unit to drive the state update process of the knowledge vector. The stability weight coefficients output from the semantic consistency channel and the evolution signal strength values ​​output from the context evolution channel are used as input signals to form the data starting point for dual-channel decision processing. A normalized weighted fusion algorithm (parameters: the normalization method is min-max scaling, and the initial value of the weighting factor is set based on historical retrieval performance indicators) is adopted to achieve scale unification and preliminary weighted calculation of the outputs of the two channels; Furthermore, the optimal allocation of the fusion factor is achieved through the principal component weight optimization method (parameter: the correlation matrix of stability weight coefficient and evolution signal intensity value as input), and the optimized dual-channel fusion vector is obtained. Furthermore, the dual-channel control parameters are generated through a linear combination calculation formula, as follows:

[0018] in, These are dual-channel control parameters. To integrate the weighting coefficients, For stability weighting coefficients, This represents the evolution signal strength value; Furthermore, an adaptive weight fine-tuning algorithm (parameter: step size is dynamically adjusted based on the update delay of historical data by the gating unit) is used to achieve fusion weights. Online optimization to adapt to dynamic changes in different dialogue contexts; Through the above weighted fusion and optimization processing, the output results of the two channels in the previous step are transformed into a single and differentiable dual-channel control parameter, thereby achieving high-precision driving of the input signal of the differentiable memory gating unit and improving the robustness and adaptability of knowledge vector state updates. For example, in an intelligent customer service scenario, the stability weight coefficient is calculated to be 0.82 by the semantic consistency channel, and the evolution signal strength value is detected to be 0.35 by the context evolution channel. The normalization method uses min-max scaling to map both to the [0,1] interval. After mapping... =0.82, =0.35, initial value of fusion factor w=0.6. Substitute the above values ​​into the control parameter formula: Calculated After the control parameters are input to the differentiable memory gating unit, the gating unit makes vector update decisions based on dynamically updated weights and preset threshold conditions. In another case, if the detected evolution signal strength value rises to 0.72, the optimization algorithm will fuse the weights. Adjust to 0.45, output the control parameter calculation formula. This design enhances the intensity of vector updates under high semantic evolution requirements. Execution results show that this design effectively balances the conflict between semantic consistency and contextual evolution in various dynamic scenarios, significantly improving the coherence of vector updates and the quality of retrieval matching.

[0019] like Figure 2 As shown, step S4 involves establishing a differentiable memory gating unit, receiving the stability weight coefficient and the evolution signal strength value as input parameters, and generating dynamically updated weight parameters through a sigmoid activation function. When the stability weight coefficient is greater than 0.8 and the evolution signal strength value is less than 0.2, the current semantic vector remains unchanged; otherwise, a weighted residual connection method is used to fuse the current semantic vector with the newly generated semantic vector to obtain the updated target knowledge vector. Specifically, this includes: S4.1: Construct an input vector based on stability weight coefficients and evolution signal intensity values, where stability weight coefficients are derived from the contrastive learning calculation results of the semantic consistency channel, and evolution signal intensity values ​​are derived from the attention distribution change rate detection results of the context evolution channel, in order to form the control input vector of the gating unit; The stability weight coefficients from the semantic consistency channel and the evolution signal intensity values ​​from the context evolution channel are combined using a vector assembly method (parameter: stability weight coefficients). Evolution signal strength value This enables the structured aggregation of output information from both channels; Furthermore, by using a numerical normalization method (parameter range: [0,1]), the scale of the two signals is unified, and the normalized stability weight coefficient and the evolution signal strength value are obtained, which are used to avoid the dimensional differences of different channel outputs in the gating input construction process; Furthermore, a vector concatenation algorithm is employed (parameter: concatenation order is [...]). , The two normalized signals are combined into a control input vector of length 2, and a formalized serialized representation is generated to ensure that the input vector is easy to manipulate in subsequent linear weighted mappings. Furthermore, by using an index mapping method, the control input vector is bound to the current dialogue turn identifier to achieve the distinguishability of gating inputs from different turns, which facilitates the execution of differentiated update strategies based on historical dialogue states in the memory gating unit. Furthermore, a data verification algorithm (parameter: tolerance threshold ε=1e-6) is used to detect whether there are numerical abnormalities or out-of-bounds errors in each component of the control input vector. When the detection result meets the preset range, the qualified control input vector is passed to the linear weighted processing module of the memory gating unit. By controlling the input vector construction method, the dual-channel output result of the previous step is transformed into gated input data with a standardized structure, stable values ​​and time sequence identification, thereby achieving consistency and scalability of the input interface of the differentiable memory gated unit. For example, in a multi-turn dialogue scenario for intelligent customer service, the stability weight coefficient is obtained by normalizing the cosine similarity matrix, and its value is [value missing]. The evolution signal intensity value is obtained by detecting the rate of change of attention distribution, and is set to a value of [value missing]. The normalization method is Min-Max normalization, which preserves the numerical values ​​after normalization. The concatenation algorithm generates the control input vectors sequentially. = The vector is bound to the current round number via index mapping. This forms record key-value pairs {12: After data verification, the numerical differences are within the tolerance threshold. Within the system, inputs deemed valid are sent to the entry control unit. In terms of application effectiveness, this construction process achieved a unified format for multi-turn adjacent state input interfaces in a large language model-driven dialogue system test, significantly improving the stability and computational efficiency of subsequent dynamic weight update calculations. S4.2: Perform a linear weighted transformation on the control input vector, and use a trainable weight matrix to linearly map it to generate an intermediate control signal, so as to enhance the separability and expressive power of the input features; S4.3: Based on the intermediate control signal, apply the Sigmoid activation function to generate dynamically updated weight parameters, wherein the value range of the parameter is [0,1], which is used to adjust the intensity and direction of knowledge vector updates; Based on the intermediate control signal, the Sigmoid activation function mapping algorithm (parameters: no bias term, scaling factor is 1) is used to map the linearly combined control signal to the [0,1] interval and dynamically update the weight parameters output. Furthermore, by performing numerical normalization processing at the input of the Sigmoid function (parameter: minimum value normalization method, input range mapped to [-6,6]), the dynamic range adjustment of control signals of different magnitudes is realized, and standardized input values ​​for nonlinear mapping are obtained; Furthermore, the Sigmoid function is used. The standard mathematical definition of is used to calculate and dynamically update the weight parameters using the following formula:

[0020] in, This is the normalized intermediate control signal value. is the base constant of the natural logarithm; Furthermore, by comparing the function output with a preset threshold range (parameters: lower threshold is 0.0, upper threshold is 1.0), the strength and direction control indicators of the dynamic update weights are determined, and the final dynamic update weight parameters are generated. By using the Sigmoid nonlinear compression algorithm, the intermediate control signal from the previous step is transformed into smooth and differentiable weight parameters, thereby enabling the gated unit to achieve fine adjustment capability and gradient transferability in continuous state space. For example, in multi-turn dialogue knowledge vector updates, assuming a stability weight coefficient of 0.75 and an evolution signal strength of 0.35, the intermediate control signal value of 1.25 is obtained through a linear combination operation in steps S4.1 and S4.2. This value is mapped to the [-6, 6] interval after minimum normalization, and in this example, it remains 1.25 after mapping. Substituting this input value into the Sigmoid function, the calculation is as follows:

[0021] The dynamically updated weight parameter is approximately 0.777. This weight value is used to control the fusion ratio between the current semantic vector and the newly generated semantic vector, biasing the fusion towards the newly generated part, thereby enhancing the contextual adaptability of the dialogue state vector. Under different input conditions, when the intermediate control signal value is greater than 0, the output of the sigmoid function tends to approach 1; when the value is less than 0, the output tends to approach 0, thus significantly improving the sensitivity to semantic mutations in multi-turn dialogue scenarios. S4.4: Based on the dynamically updated weight parameters and the preset threshold conditions (stability weight coefficient is greater than 0.8 and evolution signal strength value is less than 0.2), a logical judgment is made. If the conditions are met, a control instruction to keep the current semantic vector unchanged is output; otherwise, the vector update process is entered. The system receives dynamically updated weight parameters and preset threshold conditions as inputs to the logic judgment module. The dynamically updated weight parameters are derived from the Sigmoid activation output of the differentiable memory gate unit, and the preset threshold conditions are defined by the combination rules of stability weight coefficients and evolution signal strength values. A threshold comparison operation method (parameters: stability weight coefficient threshold 0.8, evolution signal strength value threshold 0.2) is adopted to realize the function of comparing the input parameters with the preset boundary one by one, and generate a logical comparison result matrix. Furthermore, by using Boolean logic operation methods (parameter: AND operation rule), a joint condition determination is achieved where the stability weight is greater than the threshold and the evolution strength value is less than the threshold, and the condition fulfillment flag data is obtained; Furthermore, a conditional branch control method (parameter: condition fulfillment flag) is adopted to realize the mapping function from the judgment result to the control signal, and to generate an execution instruction that maintains the current semantic vector or enters the vector update process; Furthermore, through a conditional coding mechanism (parameters: maintain mode coding value 0, update mode coding value 1), the control signal is standardized and a control code stream is generated that can be called by the internal state machine of the gating unit. Through a logic control algorithm, the comparison result matrix of the previous step is converted into a binary decision signal, thereby realizing the dynamic selection and execution path switching of the vector update process; For example, in a multi-turn dialogue scenario for intelligent customer service, the stability weight coefficient is 0.85, the evolution signal strength value is 0.15, and the dynamically updated weight parameters are calculated using the Sigmoid activation function. in, This is the intermediate control signal after linear weighting. Assuming the intermediate control signal is 0.7, then... The value is approximately 0.668. Through comparison, it is detected that 0.85 > 0.8 and 0.15 < 0.2, the Boolean AND operation result is true, and the condition-true flag is set to 1. Based on this flag, the logic control module outputs a hold-mode control signal, the control code stream is 0, and the instruction remains unchanged in the current 768-dimensional semantic vector. This processing method demonstrates a significant improvement in the stability of the knowledge vector during system verification, avoids invalid updates to continuous dialogue states, and ensures the semantic coherence and contextual consistency of multi-turn dialogues. S4.5: If the vector update process is entered, the current semantic vector and the newly generated semantic vector are fused using a weighted residual connection method. The fusion weight is determined by dynamically updated weight parameters to generate the updated target knowledge vector, thereby realizing the gradual evolution of knowledge representation.

[0022] Step S5: Construct a vector backtracking mechanism, storing the updated target knowledge vector and its timestamp information into a lightweight vector cache pool, and building a semantic similarity retrieval structure based on an inverted index to support recent state backtracking operations based on maximum inner product search in subsequent dialogue rounds. Specifically, this includes: S5.1: The target knowledge vector and its timestamp information after each update are encapsulated in a structured manner to form a vector state record unit containing vector identifier, vector value, timestamp, and dialogue round number, which serves as the data input unit for the lightweight vector cache pool. For the updated target knowledge vector and its timestamp information, a structured encapsulation algorithm (parameters: vector dimension = 768, timestamp precision = millisecond level) is used to realize the unified carrier construction of multidimensional numerical data and metadata; Furthermore, by using an identifier generation method (parameter: hash encoding based on SHA-256, input is vector value concatenated with timestamp), a unique vector identifier is generated, and non-repeatable ID field data is obtained; Furthermore, through the metadata binding mechanism (parameters: vector identifier, vector value, timestamp, round number), the data fields are combined in an ordered manner, and all fields are stored in a memory structure mapped to a fixed location to obtain a complete vector state tuple; Furthermore, by using a field integrity verification algorithm (parameter: CRC-32 check code, input is a byte stream combining vector values ​​and timestamps), the correctness of the encapsulation unit is checked before being written to the cache pool, and a structured record unit that passes the verification is generated. By using memory serialization, the structured record units from the previous step are transformed into data blocks that can be read and written by a lightweight cache pool, achieving the expected technical effect of fast data entry into the pool and subsequent inverted index construction. For example, in a certain enterprise's intelligent customer service system, an updated target knowledge vector is encapsulated. Its vector value is a 768-dimensional floating-point array, with a timestamp of "2024-06-18 15:23:45.123" and a dialogue round number of "". The SHA-256 hash algorithm is used to quantize the 768-dimensional floating-point array to 6 decimal places and concatenate it with a timestamp to form an input sequence, generating a 64-character hexadecimal string as a vector identifier. This identifier, along with the original vector value, timestamp, and round number, is stored sequentially in a structure field. A CRC-32 checksum is calculated on the structure byte stream, and after verification, the cache write process begins. During the write phase, a fixed memory block address is allocated to the vector state via a memory mapping API, forming a record unit that can be quickly located by the index module. In system testing, the average processing time of this encapsulation step in a multi-round session scenario remained within 0.8 milliseconds, significantly improving the I / O efficiency of the subsequent inverted index retrieval stage and ensuring the real-time performance and stability of state lookup. S5.2: Construct a lightweight vector cache pool based on the vector state recording unit, and use memory mapping to realize fast read and write access to the vector state records, so as to support the historical backtracking and semantic consistency verification of knowledge vector states in multi-turn dialogue. S5.3: Perform an inverted index construction operation on the target knowledge vector set in the lightweight vector cache pool, and generate a fast positioning index structure for semantically similar vectors based on the hash encoding of vector values ​​to improve the retrieval efficiency of subsequent maximum inner product search; S5.4: Based on the maximum inner product search algorithm, perform semantic similarity calculation on the current semantic vector and the historical target knowledge vector in the cache pool to obtain the semantically most similar historical state vector and its corresponding timestamp and dialogue round information, which is used to support context consistency verification and state lookup. S5.5: Generate a semantic consistency compensation vector based on the historical state vector with the closest semantics and its context information, and provide it as an auxiliary input to the next round of dialogue state-aware encoder to enhance the semantic coherence and context understanding ability of the system in multi-round dialogue scenarios. Based on the semantically closest historical state vector obtained by the maximum inner product search and its corresponding context information, the vector difference analysis method (parameters: historical state vector h, current semantic vector c) is used to calculate the offset between the two in the vector space. Furthermore, by using a context fragment matching algorithm (parameters: context semantic encoding, sliding window length L), the semantic overlap region between the historical context and the current dialogue is identified, and the embedded representation of the overlap region is extracted as a compensation basis. Furthermore, based on the weighted fusion algorithm (parameter: difference vector weights) Overlapping region embedding weights This achieves a linear combination of offset and overlapping region embedding, and obtains a preliminary expression of the compensation vector; Furthermore, the compensation vector is scaled by a normalization method (parameter: norm type L2) to ensure that its magnitude in the vector space matches the initial semantic vector. Furthermore, a feature gating mechanism (parameters: compensation threshold T, importance score vector) is adopted to selectively enhance or suppress the compensation vector in different semantic dimensions, and generate the final semantic consistency compensation vector. Through the above algorithm processing method, the result of the previous step is transformed into a structured compensation vector that is highly coupled with the current semantic state, thereby achieving the effect of semantic coherence enhancement on the input of the next round of dialogue state-aware encoder. For example, in a scenario involving 10 rounds of continuous customer service dialogue, in the 6th round, the similarity between the most similar historical state vector retrieved by the maximum inner product search algorithm and the current vector is: The context overlap fragment length is set to There are several tokens. The spatial offset between the two is calculated using vector difference analysis, yielding the difference vector norm as follows: Using a context fragment matching algorithm within the sliding window length Under the condition of extracting the overlapping region embedding representation, the embedding mean vector dimension is Dimensionality. A weighted fusion algorithm is adopted, and the weights of the difference vector are set. Overlapping region embedding weights The initial compensation vector after fusion is obtained, and its norm is constrained by L2 norm normalization. The application uses a feature-based gating mechanism, with the compensation threshold set to... Enhancement operations are performed on dimensions with importance scores above the threshold, while suppression operations are performed on the remaining dimensions. The resulting semantic consistency compensation vector significantly improves the encoder's ability to maintain topic continuity across rounds in the next round of input. The number of consistent term matches in the system response results during manual review increases to about twice the original number, ensuring the complete transmission of context in complex multi-round dialogues.

[0023] Step S6: Collect user response feedback signals and implicit behavior data, including page dwell time, follow-up question depth, and click sequence. Calculate the gating parameters and adjust the gradient based on the feedback signals. Update the weight parameter matrix of the memorized gating unit through online incremental learning to complete system closed-loop optimization. Specifically, this includes: S6.1: Collect user response feedback signals based on dialogue interaction log data, wherein the feedback signals include explicit ratings of the user's system response, whether follow-up questions are triggered, and whether the user jumps to other dialogue branches. Normalize the feedback signals to obtain standardized feedback intensity values. S6.2: Collect implicit user behavior data, including page dwell time, click sequence, follow-up question depth and dialogue round duration. Use behavior sequence modeling algorithm to extract temporal features from the implicit behavior data to obtain the user interaction intent evolution trajectory. Based on the collected interaction environment data, the implicit user behavior data is defined as a multi-dimensional sequence input including page dwell time, click sequence, follow-up question depth and dialogue turn duration, forming the original data matrix for time series analysis; The multidimensional sequence is time-aligned using event timestamps (parameters: global clock reference, millisecond precision) to achieve alignment mapping of different behavioral dimensions on a unified time axis. Furthermore, the time-aligned behavior sequence is segmented using a sliding window slicing algorithm (parameters: window length is 3 rounds of dialogue, window step size is 1 round), and the frequency and duration distribution of behavior events within each window are calculated. Furthermore, using a behavior sequence modeling algorithm based on Long Short-Term Memory (LSTM) network (parameters: 128 hidden units, dropout rate 0.2), nonlinear temporal features are extracted from the segmented behavior sequence to generate a high-dimensional state vector for each time window; Furthermore, by using the weighted time difference analysis method (with weights based on the importance coefficient of event type), the Euclidean distance and directional migration of the high-dimensional state vectors of adjacent windows are calculated to characterize the dynamic evolution direction and rate of user interaction intent. By combining LSTM temporal modeling and temporal difference analysis, the raw behavioral data from the previous step is transformed into user interaction intent evolution trajectory data, thereby achieving a structured and quantitative representation of implicit behavioral dynamics during multi-turn dialogues. For example, in an enterprise knowledge-based customer service system, the implicit behavioral data generated by a user during a consultation includes a page dwell time of 45 seconds, a click sequence of [knowledge item A, related question B], a follow-up question depth of 2 rounds, and a total dialogue round duration of 120 seconds. For the above data, event timestamps are recorded with millisecond precision as [1000, 45000, 46000, 120000]. After time alignment, a normalized time series is formed. The sliding window slice length is set to 3 rounds of dialogue, with a step size of 1 round. The aligned sequence is divided into several windows, and the frequency distribution of behavioral events within each window is calculated as follows: page dwell time percentage, click count percentage, and follow-up question count percentage. An LSTM model with 128 hidden units is used as input to the windowed sequence, and the output is a 128-dimensional state vector for each window. Using weighted time difference analysis, with page dwell weight set at 0.3, click weight at 0.4, and follow-up question weight at 0.3, the average change in the Euclidean distance between adjacent state vectors is calculated to be 2.35. The directional migration amount, calculated using the vector angle formula, is 0.82, indicating strong continuity of interaction intent. The final output evolution trajectory vector has a length of N=6 time window sequences, which can be directly used for constructing multimodal feedback fusion vectors in S6.3, thereby significantly improving the adaptability of gating parameter adjustment strategies to real-time user behavior patterns. S6.3: Based on standardized feedback intensity values ​​and the evolution trajectory of user interaction intent, a multimodal feedback fusion vector is constructed. The fusion vector is dynamically weighted using a gated loop unit to generate a comprehensive feedback index that reflects the quality of the current dialogue round. S6.4: Calculate the gating parameter adjustment gradient based on the comprehensive feedback index, where the gating parameter is the weight parameter matrix in the differentiable memory gating unit. Perform parameter update operation on the weight parameter matrix based on the gradient descent algorithm to improve the consistency between the knowledge vector update strategy and the user behavior pattern. S6.5: Deploy the updated weight parameter matrix to the differentiable memory gating unit, and perform the next round of knowledge vector state evolution modeling based on the latest parameters to form a closed-loop optimization structure, so as to realize the continuous adaptive optimization of the system in multi-turn dialogue scenarios.

[0024] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0025] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and rules of the present invention should be included within the scope of protection of the present invention.

Claims

1. A knowledge base intelligent retrieval method based on a large language model, characterized in that, The method comprises the following steps: S1: receiving a current user query text and historical dialogue context sequence data, wherein the dialogue context sequence comprises text content and timestamp information of at least three continuous dialogue turns; S2: constructing a dialogue state-aware encoder, inputting the current user query text and the historical dialogue context sequence data into the dialogue state-aware encoder to generate an initial semantic vector; S3: constructing a double-channel vector updating structure, wherein a first channel is a semantic consistency channel: constructing a semantic anchor set based on key dialogue turns in a sliding window, calculating the cosine similarity between the current semantic vector and the anchor points to generate a stability weight coefficient; a second channel is a context evolution channel: using an attention distribution change rate detection algorithm to identify semantic drift trends and output evolution signal strength values; S4: establishing a differentiable memory gate unit, receiving the stability weight coefficient and the evolution signal strength value as input parameters, generating a dynamic updating weight parameter, and judging based on the dynamic updating weight parameter and a preset threshold condition, if the condition is met, keeping the current semantic vector unchanged; otherwise, fusing the current semantic vector and the newly generated semantic vector to obtain an updated target knowledge vector; S5: constructing a vector backtracking mechanism, storing each updated target knowledge vector and its timestamp information in a vector cache pool, and constructing a semantic similarity retrieval structure based on an inverted index to support the nearest state backtracking operation based on the maximum inner product search in the subsequent dialogue turns.

2. The large language model-based knowledge base intelligent retrieval method according to claim 1, characterized in that, The step S5 further comprises: S6: collecting user response feedback signals and implicit behavior data, calculating a gating parameter adjustment gradient based on the feedback signals, and updating the weight parameter matrix of the memory gate unit through online incremental learning. 3.The method of claim 1, wherein, The step S1 specifically comprises: performing original text acquisition processing on the current user query text from the user interaction module to extract natural language user input content; performing retrieval operation of the historical dialogue context sequence based on the dialogue state manager to obtain text content and corresponding timestamp information of at least three continuous dialogue turns, and constructing a complete context-aware window; performing time sequence verification processing on the text content in the historical dialogue context sequence using timestamp information to form an ordered dialogue context sequence input; performing standardization preprocessing operation on the current user query text and the text content in the historical dialogue context sequence respectively to generate text representation in a unified format; based on the text representation in a unified format, performing generation operation of dialogue state identifier to form structured input data format. 4.The method of claim 3, wherein, In the step S1, the retrieval of the historical dialogue context sequence uses turn index key and timestamp key, the timestamp is recorded at millisecond level, and the sliding window algorithm with window length of 3-20 turns and window step of 1 turn is used, and the boundary verification and sorting of the window text and timestamp with a time interval threshold of 3000-60000 milliseconds are performed. 5.The method of claim 1, wherein, The step S2 specifically comprises: The basic semantic coding network is constructed based on the BERT-base pre-training language model, word-level vectorization coding is performed on the input text, and an initial embedding representation independent of context is obtained; A position-aware attention mechanism module is constructed, which generates a position encoding vector based on the dialogue turn timestamp information, and performs weighted fusion on the position encoding vector through an attention weight matrix; The current user query text and the historical dialogue context sequence are spliced into a joint input sequence, and word segmentation and vectorization processing are performed on the joint input sequence to obtain a word-level embedding sequence and a dialogue turn boundary mark; The word-level embedding sequence is input into the basic semantic coding network, context modeling is performed based on a multi-layer Transformer encoder, and an intermediate hidden layer representation sequence is generated; The intermediate hidden layer representation sequence and the position encoding vector are element-wise added and fused to generate a context-aware semantic representation sequence; The context-aware semantic representation sequence is subjected to a pooling operation based on the dialogue turn boundary mark, a sentence vector of each round of dialogue is extracted, and the sentence vector is mapped to a vector space of a preset dimension through a fully connected layer to generate a 768-dimensional sentence vector; A weighted average fusion operation is performed on the 768-dimensional sentence vector, the importance weight of each round of dialogue is calculated based on the attention mechanism, and a final initial semantic vector is generated. 6.The method of claim 1, wherein, The step S3 specifically comprises: Based on the initial semantic vector output by the dialogue state awareness encoder, a semantic clustering algorithm is used to cluster and analyze the semantic representation vectors of each round in the sliding window, identify representative core semantic nodes, and form a semantic anchor point set; Based on the cosine value between each anchor point in the vector space and the current semantic vector, a semantic similarity matrix is constructed, and a stability weight coefficient is generated through normalization processing; Based on the attention weight matrix output by the dialogue state awareness encoder, the attention distribution difference between adjacent dialogue turns is calculated, a sliding window variance analysis method is used to identify significant change points, and a semantic evolution signal strength value is output; Based on the preset evolution strength threshold, the semantic evolution signal strength value is binary classified, if the semantic evolution signal strength value exceeds the set threshold, it is determined that there is a context evolution requirement, and a vector update mechanism is triggered; Based on the stability weight coefficient output by the semantic consistency channel and the semantic evolution signal strength value output by the context evolution channel, a dual-channel output control parameter is generated by using a weighted fusion strategy.

7. The large language model-based knowledge base intelligent retrieval method according to claim 6, characterized in that, The semantic anchor point set is clustered by a K-means clustering algorithm, the number of clusters is the number of sliding window rounds divided by 2 and rounded up, the number of anchor nodes is 2-6, and the representative score is the average cosine similarity of the cluster center and the vector in the cluster. 8.The method of claim 1, wherein, The step S4 specifically comprises: An input vector is constructed based on the stability weight coefficient and the semantic evolution signal strength value to form a control input vector of the gating unit; A linear weighted transformation is performed on the control input vector, a trainable weight matrix is used for linear mapping, and an intermediate control signal is generated; A Sigmoid activation function is applied based on the intermediate control signal to generate a dynamic update weight parameter; Based on the dynamic updating weight parameter and a preset threshold condition for logical judgment, if the condition is established, a control instruction for keeping the current semantic vector unchanged is output; otherwise, a vector updating process is entered; If the vector updating process is entered, a weighted residual connection method is used to fuse the current semantic vector and the newly generated semantic vector, wherein the fusion weight is determined by the dynamic updating weight parameter, and an updated target knowledge vector is generated. 9.The method of claim 8, wherein, The preset threshold condition is set as: the stability weight coefficient is greater than 0.8 and the evolution signal strength value is less than 0.

2. 10.The method of claim 1, wherein, The step S5 specifically includes: Structural encapsulation processing is performed on the target knowledge vector after each update and its timestamp information to form a vector state record unit containing a vector identifier, a vector value, a timestamp, and a dialogue round number; A lightweight vector cache pool is constructed based on the vector state record unit, and a memory mapping method is used for fast read-write access of vector state records; An inverted index construction operation is performed on the target knowledge vector set in the lightweight vector cache pool, and a fast positioning index structure of semantic similar vectors is generated based on the hash coding of the vector value; Based on the maximum inner product search algorithm, semantic similarity calculation is performed on the current semantic vector and the historical target knowledge vector in the cache pool to obtain the most similar historical state vector and its corresponding timestamp and dialogue round information; According to the most similar historical state vector and its context information, a semantic consistency compensation vector is generated, and the semantic consistency compensation vector is provided as an auxiliary input to the next round of dialogue state perception encoder.

Images