A method and system for constructing RAG agents driven by a state-space recommendation model

Abstract

This invention belongs to the field of recommender systems and natural language processing technology, specifically relating to a method and system for constructing a RAG agent driven by a state-space recommender model. The method includes: encoding the user's historical sequence of an item recommendation system using a state-space model; extracting dynamic weight factors based on the encoded state-space model; performing multi-way retrieval from a preset external knowledge base based on the dynamic weight factors to obtain a set of candidate external knowledge documents; inputting the set of candidate external knowledge documents into a language model to generate a set of anchor item IDs; constructing a bidirectional cross-attention module to perform positive knowledge guidance and reverse purification on the user sequence behavior representation and anchor item features to obtain an external knowledge-enhanced representation; fusing the user sequence behavior representation and the external knowledge-enhanced representation to obtain the user's final representation; and performing item recommendation prediction based on the user's final representation.

Description

Technical Field

[0001] This invention belongs to the field of recommender systems and natural language processing technology, specifically relating to a method and system for constructing a RAG agent driven by a state-space recommender model. This invention relates to recommender systems, sequence modeling, and natural language processing techniques, and particularly to a dynamic intent-driven sequence recommender method and system that combines a state-space model (SSM) and retrieval-enhanced generation (RAG). Background Technology

[0002] With the widespread adoption of online services, sequence recommendation systems aim to predict a user's next item of interest by modeling their historical interaction sequences. While traditional sequence recommendation models (such as those based on RNNs, Transformers, or self-attention mechanisms) have made some progress in capturing user preferences, significant technical bottlenecks remain when faced with complex and ever-changing user intentions and massive amounts of external knowledge. 1. Long-range dependency bottleneck: Existing mainstream Transformer-based models face quadratic computational complexity when dealing with extremely long sequences, making it difficult to efficiently capture long-term interests in the early stages of the sequence.

[0003] 2. Knowledge Lag: Most recommendation models rely solely on ID co-occurrence patterns in the training data for prediction. This ID-oriented approach not only faces a serious cold start problem but also fails to perceive the latest industry trends and breaking news outside the training set, resulting in recommendation results that lack timeliness and making it difficult to discover potential long-tail items with semantic connections.

[0004] 3. Intent Misalignment and Noise Interference in Retrieval Enhancement: To incorporate external knowledge, retrieval enhancement (RAG) techniques have been introduced into the recommendation field. However, most existing RAG recommendation schemes employ static retrieval strategies, applying a uniform retrieval logic regardless of the user's current intent. This coarse-grained retrieval, lacking intent awareness, often leads to retrieval bias, introducing a large number of noisy documents unrelated to the user's core needs. Without effective denoising and purification mechanisms, this noise can mask the user's true behavioral characteristics, resulting in decreased recommendation performance.

[0005] To address the aforementioned issues, this invention proposes a method that deeply couples the internal dynamic states of a state-space model with an external knowledge retrieval system. This enables precise perception of user intent, dynamic retrieval, and efficient noise reduction fusion, resulting in a more accurate recommendation system. Summary of the Invention

[0006] To address the shortcomings of existing sequence recommendation models in interpreting short-term and long-term intentions, as well as the knowledge lag and noise interference in traditional retrieval enhancement methods, this invention provides a method and system for constructing RAG agents driven by a state-space recommendation model. This aims to achieve accurate retrieval and denoising fusion of external knowledge driven by real-time user interest weights.

[0007] To achieve the above objectives, the present invention provides the following solution: A method for constructing RAG agents driven by a state-space recommendation model, the method comprising: The user history sequence of the item recommendation system is encoded using a state-space model, and dynamic weight factors are extracted based on the encoded state-space model. Based on the dynamic weighting factor, a multi-way search is performed from a preset external knowledge base to obtain a set of candidate external knowledge documents; The candidate external knowledge document set is input into the second language model to generate an anchor item ID set. A bidirectional cross-attention module is constructed to perform positive knowledge guidance and reverse purification on the user sequence behavior representation and the anchor item features in the anchor item ID set, so as to obtain external knowledge-enhanced representation; The user sequence behavior representation is fused with the external knowledge enhancement representation to obtain the final user representation, and item recommendation prediction is performed based on the final user representation.

[0008] Preferably, the user history sequence of the item recommendation system is encoded using a state-space model, and the method for extracting dynamic weight factors based on the encoded state-space model includes: A selective state-space model architecture is used to model the user history sequence of the input item recommendation system; Based on the diagonal element features of the discretized state transition matrix of the modeled state space model, the long-term interest weights of users are obtained. Based on the modulus feature of the input projection matrix of the modeled state-space model, the user's short-term interest weights are obtained.

[0009] Preferably, the method for obtaining a candidate external knowledge document set by performing multi-way retrieval from a preset external knowledge base based on the dynamic weighting factor includes: Dual-channel knowledge retrieval is performed based on the user's historical sequence and the dynamic weighting factor; The dynamic weighting factor is used to perform a weighted summation of the user's total sequence mean vector and short-term subsequence mean vector to construct a dynamic query vector. The top-K documents are retrieved from a preset external knowledge base to obtain the first document set. The dynamic weighting factors are explicitly embedded into the prompt words of the large language model to generate a natural language query describing the user's short-term and long-term user preferences. The query is then retrieved from an external knowledge base to obtain a second document set. Perform a union operation on the first document set and the second document set and remove duplicates to obtain the final candidate external knowledge document set.

[0010] Preferably, the method of constructing a bidirectional cross-attention module to perform positive knowledge guidance and reverse purification on the user sequence behavior representation and the anchor item features in the anchor item ID set to obtain external knowledge-enhanced representations includes: ; ; ; ; in, This is the embedding vector of the currently interacting item. For user sequence behavior representation, This represents the encapsulation operation of the Mamba model. This refers to user sequence features incorporating external item trend information. This represents the activation function. It is a learnable linear projection matrix; It means and anchor point item characteristics Feature dimensions after linear transformation For the refined knowledge characteristics, This represents the semantic relevance score of the anchor item to historically interacted items. For sparse masking, It is a learnable linear projection matrix. Represents element-wise product. To enhance the representation of external knowledge, This indicates the average pooling operation.

[0011] Preferably, the user sequence behavior representation is fused with the external knowledge enhancement representation to obtain the final user representation, and the method for predicting item recommendations based on the final user representation includes: ; ; ; in, The merged user sequence representation, for The vector corresponding to the last time step serves as the user's final representation. It is a dynamically determined vector. For predicting scores, E represents the item embedding in the candidate item library, and the superscript T indicates transpose. for Recommended list, This indicates a filtering operation.

[0012] The present invention also provides a state-space recommendation model-driven RAG agent construction system, the system being used to implement the aforementioned method, the system comprising: an extraction module, a retrieval module, a generation module, a guidance module, and a recommendation module; The extraction module is used to encode the user's historical sequence of the item recommendation system using a state-space model, and extract dynamic weight factors based on the encoded state-space model. The retrieval module is used to perform multi-way retrieval from a preset external knowledge base based on the dynamic weighting factor to obtain a set of candidate external knowledge documents. The generation module is used to input the candidate external knowledge document set into the second language model to generate an anchor item ID set; The guidance module is used to construct a bidirectional cross-attention module to perform positive knowledge guidance and reverse purification on the user sequence behavior representation and the anchor item features in the anchor item ID set, so as to obtain external knowledge-enhanced representation. The recommendation module is used to fuse the user sequence behavior representation with the external knowledge enhancement representation to obtain the user's final representation, and to make item recommendation predictions based on the user's final representation.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Significantly improved intent perception accuracy: It breaks through the limitation of traditional models that rely solely on attention weights to determine interest, and directly extracts long-term and short-term intent weights with clear physical meanings from the underlying parameters of the Mamba model, making the initial intent positioning of RAG retrieval more accurate.

[0014] 2. Achieved highly robust knowledge denoising: Through the inverse sparse cross-attention mechanism, the model is endowed with the ability to self-censor, actively identifying and eliminating illusory noise generated by large models, ensuring that the final fused external knowledge is highly aligned with the user's real interests.

[0015] 3. It balances the timeliness and stability of recommendations: The bidirectional hybrid retrieval mechanism combines the stability of vector retrieval with the timeliness of large model retrieval, making up for the ID model's lag in perceiving new trends. Attached Figure Description

[0016] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a method for constructing a RAG agent driven by a state-space recommendation model according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the selective processing mechanism of the state-aware model in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the principle of the bidirectional cross-guidance and noise reduction mechanism in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] Example 1 This invention proposes a state-driven retrieval enhancement recommendation method for recommendation systems, aiming to solve problems such as inaccurate external knowledge base retrieval scope, weak long-range modeling ability of item sequences, and noise interference with sorting accuracy in scenarios where user interests change rapidly. This method aims to improve the click-through rate, conversion rate, and user satisfaction of item recommendations.

[0021] Specifically, this invention provides a method for constructing RAG agents driven by a state-space recommendation model, comprising: The user history sequence of the item recommendation system is encoded using a state-space model, and dynamic weight factors are extracted based on the encoded state-space model. Based on the dynamic weighting factor, a multi-way search is performed from a preset external knowledge base to obtain a set of candidate external knowledge documents; The candidate external knowledge document set is input into the second language model to generate an anchor item ID set. A bidirectional cross-attention module is constructed to perform positive knowledge guidance and reverse purification on the user sequence behavior representation and the anchor item features in the anchor item ID set, so as to obtain external knowledge-enhanced representation; The user sequence behavior representation is fused with the external knowledge enhancement representation to obtain the final user representation, and item recommendation prediction is performed based on the final user representation.

[0022] like Figures 1-3 As shown, the specific implementation process of the present invention is as follows: A. Extraction of short and long interest weights driven by Mamba state for user item interaction sequences This invention utilizes the hardware-aware state-space characteristics of the Mamba model. In SSM theory, the state transition matrix is ​​discretized. Controlling the hidden state The rate of decay over time and memory retention. If If the diagonal element modulus is close to 1, it means that historical information is preserved for a longer period of time, corresponding to the user's long-term interests; conversely, if the input projection matrix is ​​much smaller, it indicates that historical information is being retained for a longer period of time. Controlling the input at the current moment The amount of change to the hidden state. The larger the modulus, the stronger the impact of the current behavior on the state, corresponding to the user's short-term sudden interest. This invention explicitly extracts these two underlying mathematical parameters to adjust the RAG retrieval range, solving the problem that traditional RAGs cannot detect whether the user is currently browsing randomly (high...). (User's short-term interest weight) or purposeful purchase (high) The issue of (user long-term interest weight).

[0023] Specifically, the core recursive form of Mamba model inference is represented as follows: +

[0024] in, This represents the state vector representing the user's current interest preferences generated by the Mamba model after processing t items; This is the embedding vector of the currently interacting item. It is the discretized state transition matrix, used to characterize the strength of a user's memory retention of historical item preferences. It is the input projection matrix, used to represent the embedding vector of the currently interacting item. The degree of impact on the overall user interest. This invention defines its physical meaning through the following physical logic: In the above recursive formula, the matrix It directly affects the hidden state of the previous time step. From a cybernetics perspective, This determines the decay rate and retention strength of historical information. When When the modulus of the diagonal element is close to 1, it means Information is transmitted to the Chinese side with near-lossless transmission. The Mamba model is in long memory mode; conversely, when the module length is close to 0, historical information will be quickly forgotten.

[0025] In the user's full history sequence of the recommendation system middle, For the Lth historical sequence, first, embedding the shared items into the matrix... Mapped to embedded sequence Then input the data into the state-space model for encoding. If the state-space model generates a large amount of data for a certain item or a certain sequence... This indicates that the user's historical preferences dominate their current decision-making.

[0026] Then, extract the sequence according to the following formula. The mean of the maximum values ​​of the diagonal elements is used to map the mean of the user's long-term interest weights through a non-linear function. The higher the weight of a user's long-term interests, the more it indicates that the user's behavior is dominated by a stable long-term profile. Subsequent retrieval systems should focus on mining external knowledge that matches the user's consistent style.

[0027]

[0028]

[0029] Where L represents the total length of the user's historical interaction item sequence; This represents taking the diagonal elements of a matrix, reflecting the state's self-preservation ability; This is an indicator of the average memory strength of the entire sequence; the higher the value, the higher the overall memory retention rate. This is the long-term scaling factor, a hyperparameter used to adjust the long-term weight sensitivity; in this embodiment, it is a fixed parameter. The threshold value represents the cutoff value used to determine whether a user is in long memory mode; in this embodiment, it is a fixed parameter. The final result... This represents the weight of a user's long-term interests. The higher this value, the more the recommendation system should focus on discovering items that match the user's consistent historical habits and styles. for Activation function, through Scaling and translation of the activation function will Mapping to the (0,1) interval makes the high The value range remains sensitive.

[0030] On the other hand, in the core recursive form of Mamba model inference, the input projection matrix... Embedding vectors that directly affect the currently interacting item The projection matrix The magnitude of the modulus reflects the embedding vector of the currently interacting item. Regarding system status The impact force, the larger modulus length not only means The information in the hidden state is largely preserved, and the information at the current moment is significantly incorporated into the hidden state, which can substantially alter the state trajectory. Simultaneously, the input projection matrix... Embedding vectors that directly affect the currently interacting item The modulus of its diagonal element also affects The degree to which information is retained.

[0031] In recommender systems, this reflects the user's attention focus. When a user develops a strong interest in a novel item, the Mamba model needs to increase... To quickly capture this change in intent, the result corresponds to the user's short-term, sudden interests. This invention calculates the norm of the input matrix for short-term window content using the following formula to quantify short-term input and generate user short-term interest weights. :

[0032]

[0033] Where N represents the short-term window length, which defines the time step of the user's short-term interest; in this embodiment, it is a fixed value. This indicates the influence of users' short-term behavioral patterns within the short-term window N; This is a smoothing factor used to smooth out the effects of short-term inputs. Tanh is used for smoothing normalization to ensure... The dynamic balance. This mechanism provides a more flexible and intuitive guidance strategy for subsequent intelligent RAG intent retrieval. Furthermore, and It is not a simple binary opposition, but achieves dynamic complementarity through the selective mechanism of the Mamba model. Through constraints, it ensures that when users exhibit strong short-term exploratory behavior, the model can naturally suppress long-term historical inertia, and vice versa.

[0034] B. Intent-driven dynamic dual-track retrieval strategy Dual-channel knowledge retrieval based on dynamic weighting factors of user historical sequences and short-term and long-term interests: By using dynamic weighting factors to perform a weighted summation of the user's total sequence mean vector and short-term subsequence mean vector, a dynamic query vector is constructed. The top-K documents are then retrieved from a pre-defined external knowledge base to obtain the first document set. Dynamic weighting factors are explicitly embedded into the prompts of the large language model to generate natural language queries that describe the user's short-term and long-term preferences. These queries are then retrieved from an external knowledge base to obtain a second set of documents. Perform a union operation on the first and second document sets and remove duplicates to obtain the final candidate external knowledge document set.

[0035] Specifically as follows: Path 1: Implicit Item Retrieval Based on Weighted Vectors First, based on the embedding vector of the item the user is currently interacting with. Calculate the mean vector of short-term subsequences With the mean vector of the entire user sequence These represent the immediate demand for goods shown by users in a short period of N interactions and the long-term user profile formed throughout the entire interaction history, respectively:

[0036]

[0037] Subsequently, using the dynamic weighting factors extracted from the state-space model, the mean vector of the short-term subsequences is... and the mean vector of the entire user sequence Perform dynamic weighted fusion to construct the final query vector. And normalize it:

[0038]

[0039] The obtained normalized vector q is the final retrieval vector (i.e., dynamic query vector) that incorporates long and short-term weights. It is used to retrieve the top-K external item description documents related to the user's intent from the external knowledge base (K is the number of documents retrieved in this embodiment), denoted as the first document set. The pre-stored external knowledge base contains a massive number of candidate items, enabling fuzzy matching retrieval based on semantic similarity.

[0040] Path 2: Explicit semantic retrieval based on Prompt To translate the intent intensity perceived by the Mamba model into specific search content, this invention first uses a set operation strategy to map items in the user's historical behavior into two non-overlapping sets of keywords, accurately distinguishing between long-term habits and short-term interests. For the user's entire historical sequence... Item word frequency statistics are performed, and the top-M keywords with the highest frequency are selected as the user's long-term habits (M is the preset number of keywords to be extracted in this embodiment). On the other hand, for short-term interests, all interactive item keywords within a short-term time window N are first extracted to form a window set. Subsequently, a set difference operation is performed to obtain the short-term interest set. :

[0041] in, It is a set of long-term habits.

[0042] The extracted keyword set ( ) and weighted data ( Enter the preset template (suggested words from the large language model) to generate a natural language query, as follows: [System Command]: "You are a recommendation system intent analyst. The user's short-term interest weight is {" The user's long-term interest weight is { } [Preferred Keywords]: User short-term preference keywords: [insert] [Corresponding Item List] [User Weight] User's long-term preferred keywords: [insert] [Corresponding Item List] [User Weight] "Based on the above weighting ratios, please generate a natural language query for retrieving external knowledge bases. If short-term weights are high, please focus on retrieving the latest reviews or competitors related to short-term keywords; if long-term weights are high, please focus on retrieving classic recommendations or brand stories that fit the long-term category." Input the natural language query generated by LLM to describe the user's short-term and long-term preferences into an external knowledge base, retrieve the top-K relevant news articles, reviews, or notes, and record them as the second document set. .

[0043] Finally, the two search results are combined and deduplicated to construct the final candidate external knowledge document set. :

[0044] To bridge the semantic gap between the text space and the recommendation ID space, this invention utilizes a second language model (LLM large language model) as a bridge to integrate retrieved external knowledge (a set of candidate external knowledge documents). The data, along with the user's historical interaction sequence, is input into a second language model to infer a specific set of anchor item IDs, thereby transforming abstract knowledge documents into concrete item IDs. Specifically, leveraging the reasoning capabilities of the LLM (Local Language Model), it generates... An anchor item set ItemSet is generated that highly matches external knowledge and aligns with the user's potential interests. Finally, the anchor items are combined with ItemSet and mapped into feature vectors through a shared item embedding matrix E. , The anchor item's characteristics are represented by a vectorized representation of its ID. At this point... It contains the essence of external knowledge that has been screened and recommended by LLM.

[0045] C. Bidirectional cross-guided module Based on user sequence modeling and anchor item features, positive attention extraction (positive knowledge guidance) and reverse sparse cross-attention purification (reverse purification) are performed to obtain external knowledge-enhanced representations. Positive attention extraction is achieved through an attention mechanism, using user sequence behavior representation as Query, anchor item features as Key and Value, and calculating an attention score matrix; The reverse sparse cross-attention purification is specifically implemented through a sparse cross-attention mechanism. The anchor item features are used as the query, and the user sequence behavior representations are used as the key and value. An attention score matrix is ​​calculated. Dynamic threshold truncation is performed on each row of the matrix, retaining only the interaction relationships with attention scores higher than the set threshold, and setting the noise features with scores lower than the threshold to zero, thereby suppressing historical behavior noise that is unrelated to the anchor intent.

[0046] Specifically as follows: 1. Positive cross-attention Anchor point item features As supplementary information, it is input into the user sequence behavior representation to address the problems of cold start and behavior sparsity.

[0047] First, for the embedding vector of the currently interacting item It requires encoding using a Mamba model to extract deep characterization of the sample:

[0048] in, This represents the hidden state of the user interaction item sequence encoded using the Mamba model, i.e., the user sequence behavior representation. This represents the encapsulation operation of the Mamba model.

[0049] Subsequently, with For querying, anchor item characteristics Positive cross-attention knowledge injection for keys and values:

[0050] in, This represents the activation function. It is a learnable linear projection matrix; It means and Feature dimensions after linear transformation; Output result This represents the user sequence characteristics after external item trend information has been injected.

[0051] 2. Reverse sparse cross-attention Because of the illusion phenomenon inherent in LLM (Locally Streamed Modeling), the generated anchor items may contain interfering noise unrelated to the user's core interests. For example, a user's historical behavior might show a preference for professional-grade, high-specification electronic products, but LLM, due to this illusion, might recommend cute, entry-level models. Such inaccurate item recommendations reduce user satisfaction. Traditional direct fusion introduces this interfering noise, compromising recommendation accuracy.

[0052] This invention proposes a reverse purification strategy: allowing users' historical behavior to examine and audit external knowledge, retaining only knowledge features that can be associated with historical behavior. Firstly, using... Anchor item characteristics, for key and value. Conversely, perform reverse cross-attention calculation as the query:

[0053] in, This represents the semantic relevance score of the anchor item to historically interacted items. , It is a learnable linear projection matrix.

[0054] Subsequently, Each row (representing an anchor item) undergoes dynamic threshold truncation. The top-ranked interactions by attention score (70% in this invention) are retained, with the threshold set to the 30th percentile of the attention score distribution. The dynamic threshold is then calculated. Features below this threshold are set to zero to generate a sparse mask. :

[0055]

[0056]

[0057] in, A threshold used to truncate low-confidence associations; This represents the statistical operation function for calculating the 30th percentile. For row index, indicating the row number. A specific anchor point item generated by external knowledge; For column indexes, referring to the first column in the sequence of user historical behavior. A real interactive item; is the mask for the corresponding position; I is the identity matrix; It is a learnable linear projection matrix; Represents element-wise product; These are the refined knowledge characteristics. After this step, Only those high-confidence knowledge features verified by users' historical behavior were retained.

[0058] Finally, the user sequence behavior is characterized. Results obtained from positive cross attention Results obtained with reverse cross sparse attention By combining these elements, we obtain an enhanced representation of external knowledge:

[0059] in, This represents the average pooling operation, used to aggregate the purified knowledge features; This means that the external knowledge is enhanced and represented after noise reduction and purification.

[0060] D. Knowledge Gated Fusion Unit After the above steps, two sets of representations are obtained: one representing user behavior patterns and the other representing user behavior patterns. and external knowledge enhancement representation after denoising and purification .

[0061] By using a learnable gating network, the system dynamically determines which part of the information to rely on based on the current sequence state.

[0062] in, This indicates a splicing operation. for Activation function; It is a dynamic paranoia vector. It is a matrix that is continuously updated and optimized as the model is developed; It is a dynamically determined vector, and the dynamically determined system relies more on external knowledge recommendations in the current decision-making process. Or is it user historical inertia? This represents the degree of adoption of external knowledge.

[0063]

[0064]

[0065] Finally, the fused user sequence representation is taken. The vector corresponding to the last time step As the final user representation, a dot product is taken with the item embedding E in the candidate item library, and the superscript T denotes transpose, to calculate the predicted score. This score represents the probability, predicted by the model, that a user will interact with the corresponding item given their current intent. The score is then calculated and output. Recommended list This completes the entire technical process from implicit intent perception to displaying item recommendations. This indicates a filtering operation.

[0066] In summary, this invention discloses a method and system for constructing RAG agents driven by a state-space recommendation model, relating to the fields of recommendation systems and natural language processing. The core technical solutions of this recommendation system include state-aware weight extraction, dynamic hybrid retrieval, anchor point guidance, bidirectional guidance, and gated adaptive fusion. Specifically, state-aware weight extraction models the user input sequence and extracts the weight ratios of the user's short-term and long-term interests; dynamic hybrid retrieval, based on the extracted user interest weight ratios, combines the user input sequence to extract retrieval vectors and model prompts, thereby deriving relevant documents from an external knowledge base; anchor point guidance uses a large model to convert the derived relevant documents into item recommendations, thus generating a vectorized list; bidirectional guidance performs forward guidance and reverse denoising based on the above vectorized item list and user sequence modeling; and gated adaptive fusion further dynamically weights and fuses the pure behavioral features derived from bidirectional guidance with the purified knowledge-enhanced features. This invention can accurately perceive the user's real-time intent shifts, effectively filter out noise caused by the illusion of a large model, and significantly improve accuracy while ensuring the timeliness of recommendation results.

[0067] Example 2 Based on the same inventive concept, the present invention also provides a state-space recommendation model-driven RAG agent construction system, which is used to implement the methods described in the foregoing embodiments. The system includes: an extraction module, a retrieval module, a generation module, a guidance module, and a recommendation module. The extraction module is used to encode the user's historical sequence of the item recommendation system using a state-space model, and extract dynamic weight factors based on the encoded state-space model. The retrieval module is used to perform multi-way retrieval from a preset external knowledge base based on the dynamic weighting factor to obtain a set of candidate external knowledge documents. The generation module is used to input the candidate external knowledge document set into the second language model to generate an anchor item ID set; The guidance module is used to construct a bidirectional cross-attention module to perform positive knowledge guidance and reverse purification on the user sequence behavior representation and the anchor item features in the anchor item ID set, so as to obtain external knowledge-enhanced representation. The recommendation module is used to fuse the user sequence behavior representation with the external knowledge enhancement representation to obtain the user's final representation, and to make item recommendation predictions based on the user's final representation.

[0068] Furthermore, the user history sequence of the item recommendation system is encoded using a state-space model. Based on the encoded state-space model, methods for extracting dynamic weight factors include: A selective state-space model architecture is used to model the user history sequence of the input item recommendation system; Based on the diagonal element features of the discretized state transition matrix of the modeled state space model, the long-term interest weights of users are obtained. Based on the modulus feature of the input projection matrix of the modeled state-space model, the user's short-term interest weights are obtained.

[0069] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for constructing RAG agents driven by a state-space recommendation model, characterized in that, The method includes: The user history sequence of the item recommendation system is encoded using a state-space model, and dynamic weight factors are extracted based on the encoded state-space model. Based on the dynamic weighting factor, a multi-way search is performed from a preset external knowledge base to obtain a set of candidate external knowledge documents; The candidate external knowledge document set is input into the second language model to generate an anchor item ID set. A bidirectional cross-attention module is constructed to perform positive knowledge guidance and reverse purification on the user sequence behavior representation and the anchor item features in the anchor item ID set, so as to obtain external knowledge-enhanced representation; The user sequence behavior representation is fused with the external knowledge enhancement representation to obtain the final user representation, and item recommendation prediction is performed based on the final user representation. The method of constructing a bidirectional cross-attention module to perform positive knowledge guidance and reverse purification on the user sequence behavior representation and the anchor item features in the anchor item ID set to obtain external knowledge-enhanced representations includes: ； ； ； ； in, This is the embedding vector of the currently interacting item. For user sequence behavior representation, This represents the encapsulation operation of the Mamba model. This refers to user sequence features incorporating external item trend information. This represents the activation function. It is a learnable linear projection matrix; It means and anchor point item characteristics Feature dimensions after linear transformation For the refined knowledge characteristics, This represents the semantic relevance score of the anchor item to historically interacted items. For sparse masking, It is a learnable linear projection matrix. Represents element-wise product. To enhance the representation of external knowledge, This indicates an average pooling operation; The method of fusing the user sequence behavior representation with the external knowledge-enhanced representation to obtain the final user representation, and then performing item recommendation prediction based on the final user representation, includes: ； ； ； in, The merged user sequence representation, for The vector corresponding to the last time step serves as the user's final representation. It is a dynamically determined vector. For predicting scores, E represents the item embedding in the candidate item library, and the superscript T indicates transpose. for Recommended list This indicates a filtering operation.

2. The method according to claim 1, characterized in that, Encoding user history sequences in an item recommendation system using a state-space model, and then extracting dynamic weight factors based on the encoded state-space model, include the following methods: A selective state-space model architecture is used to model the user history sequence of the input item recommendation system; Based on the diagonal element features of the discretized state transition matrix of the modeled state space model, the long-term interest weights of users are obtained. Based on the modulus feature of the input projection matrix of the modeled state-space model, the user's short-term interest weights are obtained.

3. The method according to claim 1, characterized in that, Based on the dynamic weighting factors, the method for obtaining a candidate external knowledge document set by performing multi-way retrieval from a preset external knowledge base includes: Dual-channel knowledge retrieval is performed based on the user's historical sequence and the dynamic weighting factor; The dynamic weighting factor is used to perform a weighted summation of the user's total sequence mean vector and short-term subsequence mean vector to construct a dynamic query vector. The top-K documents are retrieved from a preset external knowledge base to obtain the first document set. The dynamic weighting factors are explicitly embedded into the prompt words of the large language model to generate a natural language query describing the user's short-term and long-term user preferences. The query is then retrieved from an external knowledge base to obtain a second document set. Perform a union operation on the first document set and the second document set and remove duplicates to obtain the final candidate external knowledge document set.

4. A RAG agent construction system driven by a state-space recommendation model, the system being used to implement the method described in any one of claims 1-3, characterized in that, The system includes: an extraction module, a retrieval module, a generation module, a guidance module, and a recommendation module; The extraction module is used to encode the user's historical sequence of the item recommendation system using a state-space model, and extract dynamic weight factors based on the encoded state-space model. The retrieval module is used to perform multi-way retrieval from a preset external knowledge base based on the dynamic weighting factor to obtain a set of candidate external knowledge documents. The generation module is used to input the candidate external knowledge document set into the second language model to generate an anchor item ID set; The guidance module is used to construct a bidirectional cross-attention module to perform positive knowledge guidance and reverse purification on the user sequence behavior representation and the anchor item features in the anchor item ID set, so as to obtain external knowledge-enhanced representation. The recommendation module is used to fuse the user sequence behavior representation with the external knowledge enhancement representation to obtain the user's final representation, and to make item recommendation predictions based on the user's final representation.

Images