A sequential recommendation method and system for predicting potential preferences of a user

Abstract

The application relates to a sequence recommendation method and system for predicting potential preferences of a user, wherein the method comprises the following steps: S1, obtaining a user historical interaction sequence, mapping the user historical interaction sequence to a continuous representation space, and obtaining an input tensor; S2, extracting a sequence state representation and a global context dependent feature of the input tensor; S3, constructing a final user interest representation based on the sequence state representation and the global context dependent feature; and S4, predicting the final user interest representation to obtain a recommended item of the user. The application can effectively recommend potential preferences of the user.

Description

Technical Field

[0001] This invention relates to the field of sequence recommendation technology, and in particular to a sequence recommendation method and system for predicting potential user preferences. Background Technology

[0002] Sequential recommendation, an important branch of recommender systems, predicts a user's next potentially interesting item or content by utilizing the temporal sequence of their historical behaviors. In this task, each user interaction (such as browsing, clicking, purchasing, rating, etc.) is considered an event in a time series. By analyzing these behavioral sequences, the system can capture dynamic patterns in user interests over time, thereby more accurately predicting users' potential future preferences.

[0003] Early research on sequence recommendation primarily relied on recurrent neural network (RNN) structures, modeling user sequences through temporal recursion. For example, GRU4Rec and GRU4Rec+ captured temporal dependencies between behaviors through gating updates, marking a significant milestone in the recommendation field. NARM, on the other hand, introduced an attention-focusing mechanism on top of RNNs to enhance the expression of users' short-term intentions. However, RNNs are inherently difficult to parallelize and lack stability for long-range dependencies, gradually revealing performance and efficiency bottlenecks. To enhance the ability to capture local behaviors, convolutional structures were introduced into sequence recommendation. For instance, Caser uses horizontal and vertical convolutions to extract local behavior fragments; NextItNet uses deep residual convolutional networks to learn long-range dependencies, demonstrating outstanding performance in short-sequence scenarios. With the development of attention mechanisms, Transformer has gradually become the mainstream framework for sequence recommendation. SASRec significantly improved recommendation performance by acquiring global high-order dependencies through a self-attention mechanism; BERT4Rec, based on bidirectional Transformer and mask prediction, constructs a more complete sequence representation; and TiSASRec introduces time interval encoding to improve the ability to express temporal relationships between behaviors. Meanwhile, new directions for model extensions are constantly emerging, such as FDSA combined with feature-level attention, S3-Rec leveraging self-supervised tasks to enhance sequence modeling capabilities, and DuoRec using contrastive learning to strengthen user preference representations. Furthermore, lightweight models like LRURec use linear recursive units to replace self-attention computation, significantly reducing the complexity of Transformers in long sequence scenarios. However, Transformer-based methods are generally limited by computational costs, resulting in low training and inference efficiency for long sequences; simultaneously, capturing local behavioral changes relies on explicit positional encoding, limiting flexibility. Recently, the rise of State Space Models (SSMs) has propelled sequence recommendation into a new stage of development. The representative Mamba4Rec, based on a linearly complex selective state space structure, performs excellently in long-range dependency modeling, but its recall rate is relatively low in short sequence scenarios, making it difficult to quickly capture local behavioral patterns and potentially exacerbating the cold-start problem under sparse data conditions.

[0004] Existing sequence recommendation models often overlook two important features present in user interaction records: local patterns and global context. As shown in the figure, local patterns refer to locally related behaviors generated by users in the short term. For example, after purchasing a mobile phone, a user may tend to consider other digital devices from the same brand, implying brand-specific characteristics; after purchasing airline tickets, a user may engage in other related behaviors such as accommodation and car rentals, reflecting a continuous local pattern; during specific holidays, users may exhibit a series of related consumption behaviors, demonstrating a time-sensitive local pattern. Global context, on the other hand, reflects long-term stable preferences and preference shifts, such as a user's overall category preferences, overall consumption cycle, and the shift in consumption interest after each consumption cycle. Current sequence recommendation models fail to capture these two typical features well, resulting in an incomplete characterization of user intent. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the sequence recommendation model in the prior art cannot extract two important features, namely local behavior patterns and global contextual dependencies, which leads to an incomplete characterization of user intent.

[0006] To address the aforementioned technical problems, this invention provides a sequence recommendation method for predicting potential user preferences, comprising:

[0007] Step S1: Obtain the user's historical interaction sequence, map the user's historical interaction sequence to a continuous representation space, and obtain the input tensor;

[0008] Step S2: Extract the sequence state representation and global context-dependent features of the input tensor;

[0009] Step S3: Construct the final user interest representation based on the sequence state representation and global context-dependent features;

[0010] Step S4: Predict the end user's interest representation to obtain the user's recommended items.

[0011] In one embodiment of the present invention, the method for mapping the user's historical interaction sequence to a continuous representation space to obtain the input tensor in step S1 includes:

[0012] A user's historical interaction sequence can be represented as:

[0013] ;

[0014] in, A representation of the user's historical interaction sequence. The first in the user's historical interaction sequence Item ID, Total number of item IDs;

[0015] The user's historical interaction sequence is embedded through the Embedding layer. Each item ID is converted into its corresponding embedding vector. :

[0016] ;

[0017] in, The embedding dimension of the Embedding layer;

[0018] To improve the embedding vector Robustness and mitigation of overfitting risk, for embedding vectors Apply Dropout operation:

[0019] ;

[0020] in, For Dropout operations;

[0021] For the embedding vector after Dropout operation Apply layer normalization to improve training stability and convergence speed:

[0022] ;

[0023] in, For layer normalization operation;

[0024] Output of formula (5) Represented as input tensor :

[0025] ;

[0026] in, To input the batch size, This represents the uniform sequence length after padding.

[0027] In one embodiment of the present invention, the method for extracting the sequence state representation of the input tensor in step S2 includes:

[0028] The input tensor is processed through the Mamba module. The output of the first linear layer is projected through two linear layers. Then, it is passed through a convolutional layer, a SiLU activation function, and a Selective SSM to obtain the first branch feature. The output of the second linear layer is then passed through the SiLU activation function to obtain the second branch feature. The first and second branch features are then multiplied by a vector and passed through another linear layer to map back to the original dimension D, ultimately yielding the initial output of the Mamba module. ;

[0029] The initial output of the Mamba module After the Dropout operation, which randomly removes some neuron activations, the following is obtained: Subsequently, residual connections are used to connect the input tensor. Compared with the output after Dropout operation Element-by-element addition yields Finally, By performing feature normalization at each position of the sequence through layer normalization, the final output of the Mamba module, i.e., the sequence state representation, is obtained. , is represented as:

[0030] ;

[0031] ;

[0032] ;

[0033] in, For Dropout operations, This is a layer normalization operation.

[0034] In one embodiment of the present invention, the method for extracting the global context-dependent features of the input tensor in step S2 includes:

[0035] The input tensor obtained from the Embedding layer is processed through the Attention module. The multi-head attention mechanism is used to map it into three sets of vectors: query, key, and value.

[0036] ;

[0037] in, For query vectors, The parameter matrix mapped to the Query space. Key is the key vector. The parameter matrix mapped to the Key space, Value is the value vector. This is the parameter matrix mapped to the Value space;

[0038] Will , , Along the hidden dimension Divided into There are 1 attention head, and each head has 1 dimension. ,at this time , , Then the attention of the bulls The calculation formula is:

[0039] ;

[0040] ;

[0041] in, This represents the output of each attention head. For splicing operations;

[0042] Bullish attention and After multiplication and dropout, the result is multiplied by the input tensor. Perform residual joins, followed by layer normalization:

[0043] ;

[0044] in, For layer normalization operation, For dropout operations;

[0045] right A two-layer feedforward network (FFN) is used to perform a pointwise nonlinear transformation on each sequence position, and then layer normalization is applied to map it back to the hidden layer dimension to obtain global context-dependent features. The formula is:

[0046] ;

[0047] in, It is a non-linear activation function. and These are the learnable weight matrices of the two-layer feedforward network FFN.

[0048] In one embodiment of the present invention, the method for constructing the final user interest representation based on the sequence state representation and global context-dependent features in step S3 includes:

[0049] Get sequence state representation and global context dependency features ,Will and The features are summed, and local behavioral patterns are extracted through local convolution operations:

[0050] ;

[0051] in, For local behavioral pattern features, This is a local convolution operation;

[0052] To adaptively select the most effective features under different scenarios, a trimodal fusion gating mechanism is constructed. First, the three types of features are... , , In the last dimension of the splicing:

[0053] ;

[0054] in, The merged user interest features;

[0055] Will The original gated signal is obtained through linear layer mapping:

[0056] ;

[0057] in, The original gating signal, This is the weight matrix of the linear layer;

[0058] The three types of features after splicing , , Average value over the sequence dimension:

[0059] ;

[0060] in, For sequence-level context representation, The length of the user interaction sequence. Let be the sequence state representation at position i. For the global context dependency feature at position i, This represents the local behavioral pattern feature at the i-th position;

[0061] Will After passing through two layers of fully connected networks, we obtain:

[0062] ;

[0063] in, This is the weight matrix for the first fully connected network, used to map the input dimension from 3D to the hidden dimension D in the first layer; This is the weight matrix for the second fully connected network, used to map the hidden representations back to the original feature dimension D; For context bias;

[0064] The original gate signal With context bias After activation by the Sigmoid function, we get:

[0065] ;

[0066] in, For gating weights, For the Sigmoid function;

[0067] Obtaining the gating weight Afterwards, based on , , Perform dynamic weighted fusion:

[0068] ;

[0069] in, This represents element-wise dot product. To enhance training stability, we will... Then, after residual connection and layer normalization:

[0070] ;

[0071] in, To represent the interests of end users, For layer normalization.

[0072] In one embodiment of the present invention, the method for predicting the user's recommended items by the final user interest representation in step S4 includes:

[0073] For any candidate item The predicted score is:

[0074] ;

[0075] in, For the first A function to predict scores for each item. For any candidate item Vector representation after embedding layer Representation of end-user interests;

[0076] Select predicted scores from all candidate items. The highest-rated item is the one recommended by the user.

[0077] In one embodiment of the present invention, steps S1 to S4 are implemented using a MAGERec network, and the loss function used when training the MAGERec network is cross-entropy loss. :

[0078] ;

[0079] in, Indicates the actual next item in the sequence. This represents the model's prediction score for the real target item. For the first The predicted score of each candidate item This is a set of candidate items.

[0080] To address the aforementioned technical problems, this invention provides a sequence recommendation system for predicting potential user preferences, comprising:

[0081] Mapping module: used to obtain the user's historical interaction sequence, map the user's historical interaction sequence to a continuous representation space, and obtain the input tensor;

[0082] Feature extraction module: used to extract the sequence state representation and global context-dependent features of the input tensor;

[0083] Construction module: used to construct the final user interest representation based on the sequence state representation and global context-dependent features;

[0084] Prediction module: used to predict the end user's interest representation to obtain the user's recommended items.

[0085] To address the aforementioned technical problems, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the sequence recommendation method for predicting potential user preferences as described above.

[0086] To address the aforementioned technical problems, the present invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the sequence recommendation method for predicting potential user preferences as described above.

[0087] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0088] The sequence recommendation method for predicting potential user preferences described in this invention constructs a MageRec model. This model structurally introduces parallel Mamba and Attention modules, and further incorporates local contextual convolution, gated fusion mechanisms, and learnable contextual biases, thereby achieving a unified representation of multi-scale sequence features. Through the stacking of multiple MageLayers (i.e., multiple Mage networks), the model can fully capture both short-term and long-term behavioral features while maintaining efficiency, enhancing robustness to noisy sequences. Experimental results show that the MageRec model outperforms mainstream sequence recommendation methods on multiple real-world datasets, verifying the complementarity of Mamba global state modeling and the attention mechanism. Further ablation experiments demonstrate that the proposed parallel Mamba-attention architecture, local convolutional modules, and contextual bias networks make crucial contributions to the final performance. Attached Figure Description

[0089] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0090] Figure 1 This is a flowchart of the method of the present invention;

[0091] Figure 2 This is an overall structural diagram of the MageRec model of this invention;

[0092] Figure 3 This is a schematic diagram of the Mamba module structure of the present invention;

[0093] Figure 4 This is a comparison chart showing the impact of different numbers of Mage network layers on the results of this invention;

[0094] Figure 5 This is a comparison chart showing the impact of different numbers of attention heads in the Attention module of this invention;

[0095] Figure 6 This is a comparison chart showing the impact of the dropout ratio on the results in the model of this invention. Detailed Implementation

[0096] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0097] Example 1

[0098] Reference Figure 1 As shown, this invention relates to a sequence recommendation method for predicting potential user preferences, comprising:

[0099] Step S1: Obtain the user's historical interaction sequence, map the user's historical interaction sequence to a continuous representation space, and obtain the input tensor;

[0100] Step S2: Extract the sequence state representation and global context-dependent features of the input tensor;

[0101] Step S3: Construct the final user interest representation based on the sequence state representation and global context-dependent features;

[0102] Step S4: Predict the end user's interest representation to obtain the user's recommended items.

[0103] The following is a detailed description of this embodiment:

[0104] Sequential recommendation is an important research area in recommender systems. Its goal is to predict a user's potential future interests by leveraging the temporal order of their historical interactions. Sequential recommendation models capture the dynamic changes in user interests by analyzing the temporal dependencies and evolutionary patterns between interactions, thus achieving more accurate personalized recommendations.

[0105] Sequential recommendation tasks aim to predict a user's next possible interaction based on the order of their historical interactions. Specifically, given a set of users... (i.e., the collection of all users) and a collection of items. (That is, the collection of all products on the website). Indicates user The interaction sequence (i.e., the set of items purchased by a user in chronological order), where The goal of sequential recommendation is to prioritize users. Interaction history Predicting users The next project to interact with .

[0106] Please see Figure 2 This embodiment constructs the MageRec model, which includes an Embedding layer, a Dropout layer, a layer normalization layer, N Mage networks, and a prediction network connected in sequence, as detailed below.

[0107] (a) Embedding layer

[0108] A user's historical interaction sequence can be represented as:

[0109]

[0110] in, A representation of the user's historical interaction sequence. The first in the user's historical interaction sequence Item ID, This represents the total number of item IDs.

[0111] The user's historical interaction sequence is embedded through the Embedding layer. Each item ID is converted into its corresponding embedding vector. :

[0112]

[0113] Since the Embedding layer is existing technology, it will not be described in detail in this embodiment.

[0114] To improve the robustness of the representation and mitigate the risk of overfitting, we first... Apply a Dropout operation to disrupt feature co-fitness:

[0115]

[0116] in, This is for Dropout operations.

[0117] Subsequently, the embedded vector after Dropout... Apply layer normalization to normalize the feature dimensions to improve training stability and convergence speed:

[0118]

[0119] in, This is a layer normalization operation.

[0120] In outputting formula (5) Represented as input tensor :

[0121]

[0122] in, To input the batch size, This represents the uniform sequence length after padding.

[0123] (ii) The Mamba module in the Mage network

[0124] Please see Figure 3 The overall structure of the Mamba Block can be viewed as a combination of "feedforward projection + SelectiveSSM + nonlinear transformation". The Mamba Block transforms the input tensor... The output of the first linear layer is projected through two linear layers. Then, it is passed through a convolutional layer, a SiLU activation function, and a Selective SSM to obtain the first branch feature. The output of the second linear layer is then passed through the SiLU activation function to obtain the second branch feature. The first and second branch features are then multiplied by a vector and passed through another linear layer to map back to the original dimension D, ultimately yielding the initial output of the Mamba module. .

[0125] It should be noted that Selective SSM is existing technology and will not be described in detail in this embodiment.

[0126] Residual connectivity and layer normalization

[0127] To stabilize training and enhance generalization ability, this embodiment uses the initial output of the Mamba module. First, a Dropout operation is performed to randomly remove some neuron activations, effectively reducing overfitting, resulting in... Subsequently, the model uses residual connections to connect the input tensor. Compared with the output after Dropout operation Element-by-element addition yields Finally, By normalizing the features at each position in the sequence using LayerNorm, the activations at different time steps maintain a stable distribution, improving the model's stability on long sequences. This yields the final output of the Mamba module, which is the sequence state representation. Assume a given initial output from a Mamba module. The specific formula is as follows:

[0128]

[0129]

[0130]

[0131] in, For Dropout operations, This is a layer normalization operation.

[0132] (III) Attention Module in Mage Network

[0133] The Attention Block employs a multi-head attention (MHA) mechanism to model global dependencies in sequence representations. This layer consists of a multi-head attention module and a feed-forward network (FFN), with a Residual-LayerNorm structure introduced after both sub-modules to enhance training stability.

[0134] First, given the input features obtained from the Embedding layer... Multi-HeadAttention maps it into three sets of vectors: Query, Key, and Value.

[0135]

[0136] in, For query vectors, The parameter matrix mapped to the Query space. Key is the key vector. The parameter matrix mapped to the Key space, Value is the value vector. This is the parameter matrix mapped to the Value space.

[0137] Will , , Along the hidden dimension Divided into There are 1 attention head, and each head has 1 dimension. ,at this time , , Then the attention of the bulls The calculation formula is:

[0138] ;

[0139] ;

[0140] in, This represents the output of each attention head. This is for splicing operations.

[0141] Bullish attention and After multiplication and dropout, the result is multiplied by the input tensor. Perform residual joins, followed by layer normalization:

[0142]

[0143] in, For layer normalization operation;

[0144] Building upon this, the Attention module further employs a two-layer feedforward network (FFN) to perform a point-by-point nonlinear transformation on each sequence position. This involves first expanding the dimensionality and then mapping it back to the hidden layer dimensions through layer normalization, thereby obtaining global context-dependent features. Its calculation form is:

[0145]

[0146] in, It is a non-linear activation function. and These are the weights of the two-layer feedforward network FFN.

[0147] (III) Local Context Gating in Mage Networks

[0148] To simultaneously capture more refined features from both sequence state representations and global context-dependent features, this embodiment designs a Local Context Gating structure in each Mamba network to adaptively fuse Mamba modules, Attention modules, and local convolutional features. This structure uses the input sequence representation... Based on this, complementary sequence information is extracted through three parallel branches, and the gating mechanism dynamically selects the features most suitable for the current context to construct a more expressive sequence representation.

[0149] (1) Local Convolutional Context

[0150] Get sequence state representation and global context dependency features ,Will and The features are summed, and local behavioral patterns are extracted through local convolution operations (Local Conv).

[0151]

[0152] in, For local behavioral pattern features, This is a local convolution operation. This local convolution operation performs convolution only along the channel dimension, allowing each hidden dimension to independently perceive a fixed-size local window, thus fully learning short-term behavioral dependencies.

[0153] Subsequently, in order to adaptively select the most effective features under different scenarios, this embodiment constructs a three-modal fusion gating mechanism. First, the three types of features... , , In the last dimension of the splicing:

[0154]

[0155] in, The merged user interest features;

[0156] Will The original gated signal is obtained through linear layer mapping:

[0157]

[0158] in, The original gating signal, Let be the weight matrix of this linear layer.

[0159] (2) Context Bias Network (BiasNet)

[0160] To enable the gating structure to dynamically adjust according to the overall context of the sequence, this embodiment further introduces a context-aware bias network (BiasNet), whose input is the concatenated three types of features. , , Average value over the sequence dimension:

[0161]

[0162] in, For sequence-level context representation, The length of the user interaction sequence. Let be the sequence state representation at position i. For the global context dependency feature at position i, Let be the local behavioral pattern feature at the i-th position.

[0163] Will After passing through two layers of fully connected networks, we obtain:

[0164]

[0165] in, This is the weight matrix for the first fully connected network, used to map the input dimension from 3D to the hidden dimension D in the first layer; This is the weight matrix for the second fully connected network, used to map the hidden representations back to the original feature dimension D; For context bias.

[0166] (3) Gating fusion

[0167] To achieve dynamic integration of multi-source features while maintaining model stability, this embodiment designs a gated fusion mechanism. The gate signal is derived from the original gate signal. With context bias After activation by the Sigmoid function, we get:

[0168]

[0169] in, For gating weights, This is the Sigmoid function.

[0170] Obtaining the gating weight Afterwards, based on , , Perform dynamic weighted fusion:

[0171]

[0172] in, This represents element-wise dot product. To enhance training stability, [the following will be used]. Then, after residual connection and layer normalization:

[0173]

[0174] in, To represent the interests of end users, For layer normalization. The result is... This is the final output of the Mage network.

[0175] It should be noted that the number of Mage networks in this embodiment can be N sequentially connected, and the specific number can be defined according to the model training situation.

[0176] (iv) Prediction Network

[0177] The prediction network computes the sequence representation and candidate items using a dot product method. The similarity score obtained after embedding is used as the final score.

[0178] For any candidate item The predicted score is:

[0179]

[0180] in, For the first A function to predict scores for each item. For any candidate item Vector representation after embedding layer This represents the interests of the end user.

[0181] (v) Loss Function

[0182] In sequence recommendation tasks, the goal of the model is to predict the next item to be interacted with based on the user's historical behavior sequence. To this end, the MAGERec model uses cross-entropy loss as the primary optimization objective to maximize the model's correct prediction probability across the entire item set. Cross-entropy loss is a widely used objective function in multi-class classification tasks, particularly suitable for full-rank prediction scenarios.

[0183] After obtaining scores through the prediction layer, the cross-entropy loss is defined as:

[0184]

[0185] in, Indicates the actual next item in the sequence. This represents the model's prediction score for the real target item. The loss function maximizes... , For the first The predicted score of each candidate item The set represents the candidate items. The advantage of the scores of all candidate items allows the model to learn a more discriminative ranking representation across the entire item space.

[0186] (vi) The comparative experiments are as follows:

[0187] (6.1) Experimental Dataset

[0188] Table 1. Statistical information of the experimental dataset

[0189]

[0190] This embodiment selects four real datasets for experimentation: Amazon-Beauty, Amazon-Clothing, Steam, and MovieLens-1M.

[0191] Amazon-Beauty: Data from Amazon product reviews, primarily covering personal care products such as makeup, skincare, and fragrance. Amazon-Clothing: Includes user interaction records for apparel products (such as shoes, clothing, and accessories).

[0192] Steam: Originating from the game distribution platform Steam, it records users' interactions with games, such as browsing and purchasing.

[0193] MovieLens-1M: Based on data from the MovieLens website, containing approximately 1 million user movie rating records.

[0194] For each user in the dataset, this embodiment sorts their interaction records according to timestamps to construct an interaction sequence. Since the interaction data contains many extremely short sequences and a large amount of noise, users and items with fewer than 5 interactions are filtered out in the experiment. Detailed statistical data summaries of each dataset after preprocessing are shown in Table 1.

[0195] (6.2) Parameter settings

[0196] This embodiment uses the open-source RecBole framework to evaluate the baseline model. To ensure fairness in the comparison results, the hyperparameters set in this embodiment are consistent with those of the relevant comparison models. All models have a hidden dimension of 64, a training batch size of 2048, an evaluation batch size of 4096, and are trained using the Adam optimizer with a uniform learning rate of 0.001. Furthermore, the same random seed was used in the experiments to ensure that all models have the same initial weights and random sampling results. For models using the Mamba architecture, common parameter settings were adopted, including a Sequential State Modeling (SSM) expansion factor of 32, a one-dimensional convolutional kernel size of 4, and a linear projection expansion factor of 2. The maximum sequence length for the Beauty, Clothing, and Steam datasets is 50, and the maximum sequence length for the ML-1M dataset is 200. The datasets are divided into training, validation, and test sets in an 8:1:1 ratio. All experiments were conducted on an Nvidia RTX3090 GPU, and the cross-entropy loss function was used uniformly.

[0197] (6.3) Comparison Model

[0198] To fully validate the effectiveness of the proposed model, this study selected several representative sequence recommendation baseline methods for comparison, covering different paradigms such as convolutional structures, recurrent structures, attention mechanisms, state-space models (SSM), and time-aware modeling. This allows for multi-dimensional evaluation of the model's performance in local pattern capture, long-term dependency modeling, and sequence context representation.

[0199] (1) Caser: It models user behavior sequences by using horizontal and vertical convolutions to capture local behavioral features and overall patterns across locations, respectively. It is the earliest representative method to introduce CNN into sequence recommendation.

[0200] (2) GRU4Rec: It uses gated recurrent units (GRU) to recursively model user behavior sequences and is a classic baseline in conversation recommendation and early sequence recommendation.

[0201] (3) SASRec: Based on the self-attention mechanism, a Transformer encoder is constructed to model the dependencies between behaviors in an adaptive way. It is the mainstream baseline for sequence recommendation.

[0202] (4) BERT4Rec: It adopts a bidirectional Transformer structure and performs self-supervised training through random masking, which can simultaneously utilize bidirectional dependency information for sequence modeling.

[0203] (5) LRURec: A sequence recommendation model optimized based on linear cyclic units and parallel scanning mechanism.

[0204] (6) TiSASRec: Introduces time interval encoding on the basic SASRec, and enables the model to better handle behavioral dependencies over different time spans by explicitly modeling the temporal relationship between behaviors.

[0205] (7) Mamba4Rec: It was the first to introduce Mamba Block into sequence recommendation tasks. It efficiently captures long-range dependencies through selective state update mechanism while maintaining linear complexity. It is an important representative of SSM in the recommendation field recently.

[0206] (8)SIGMA: Based on Mamba4Rec, by constructing forward and partially flipped reverse Mamba paths, the short-term feature extraction module FE-GRU is introduced to achieve efficient and comprehensive sequence modeling.

[0207] (6.4) Evaluation Indicators

[0208] To comprehensively evaluate the performance of the sequence recommendation model on the Top-K recommendation task, this embodiment uses three of the most commonly used metrics in recommendation systems: Hit@K, NDCG@K, and MRR@K. These metrics measure the model's prediction quality for the user's next interaction from different perspectives, effectively reflecting the accuracy of the recommendation list, the rationality of the ranking, and the user experience.

[0209] (1) Hit@K

[0210] Hit@K measures whether the model can find the user's next real interactive item in a recommendation list of length K. A value of 1 is assigned when the real item appears in the recommendation list, and 0 otherwise. The formula is as follows:

[0211]

[0212] Hit@K reflects the model's recall ability and is the most intuitive accuracy metric. The model predicts the top K recommended items for each user. For the user's next real interaction.

[0213] (2) NDCG@K (Normalized Discounted Cumulative Gain)

[0214] NDCG@K further considers the ranking quality of the recommendation results, awarding higher rewards to items that rank higher. The calculation method is as follows:

[0215]

[0216] in, Accumulate benefits for users' discounts. The number of items K is for reference. Indicates whether the recommended result for location 𝑖 is a real item (1 if yes); The DCG value represents the ideal sorting. It measures whether the model can more accurately place items that users are truly interested in at the top of the list.

[0217] (3) MRR@K (Mean Reciprocal Rank)

[0218] MRR@K measures the actual target item's ranking position in the recommended list.

[0219]

[0220] in, Indicates the number of recommendations, 1 / This indicates the position of the user's actual accessed item in the result sequence. If the item is not in the sequence, this value is infinite.

[0221] Table 2 Comparison of Experimental Results for Each Model

[0222]

[0223] (6.5) Comparison of experimental results

[0224] Table 2 shows the results of MageRec and nine representative baseline methods (including traditional neural models, CNNs, RNNs, Transformer models, and the latest state-space models Mamba4Rec and SIGMA) on four real-world datasets. The experiments were evaluated using three metrics: Hit@10, NDCG@10, and MRR@10. As shown in the table, MageRec outperforms the baseline models on all datasets and all evaluation metrics, demonstrating its consistent advantage across different scenarios. On the Amazon-Beauty dataset, where sequences are short and noisy, MageRec achieved an 8.74% improvement over the best model on the NDCG@10 metric. On the Amazon-Clothing dataset, a typical dataset with extremely short sequences and high sparseness, all models scored relatively low overall. However, MageRec showed the highest relative improvement across all datasets, with a 12.21% improvement on the NDCG@10 metric. This indicates that MageRec performs particularly well in scenarios with extremely short sequences, limited user behavior, and even significant noise. The Steam dataset features long user behavior sequences and significant fluctuations in interest over time. Overall, the performance of various models is relatively similar, with MageRec showing improvement across all three metrics. ML-1M is a long sequence dataset with lower overall sparsity and less noise. The overall performance of various models is high, with relatively narrowing performance gaps. Mamba-based models (Mamba4Rec, SIGMA) have already demonstrated leading performance in long sequences. The improvement of MageRec over SIGMA indicates that the Mamba Block in the model has a natural advantage in handling long sequences. The Attention path supplements global high-order interaction capabilities, further improving ranking quality in long sequences.

[0225] Specifically, in the experiments, traditional sequence modeling (CNN / RNN) showed stable performance but limited capabilities.

[0226] Among them, Caser (CNN) and GRU4Rec (RNN) performed the worst on all datasets, especially in sparse scenarios (such as Clothing), where they were almost unusable. Due to the structural limitations of CNN and RNN, it is difficult to model long-term dependencies, especially when facing sparse long-tail behavior, resulting in low prediction performance.

[0227] Transformer models (SASRec, BERT4Rec, TiSASRec) remain strong baselines, but they also have significant shortcomings. SASRec is a classic benchmark model in sequence recommendation, achieving above-average performance on all four datasets and serving as the primary comparison target for MageRec in this example. This type of model has a strong ability to capture global context and performs stably in medium-to-long sequence scale scenarios (Steam, ML-1M). As a past research hotspot in sequence recommendation, this type of model has undergone continuous optimization by researchers, with TiSASRec being a representative example. TiSASRec performs best or near-optimal in ML-1M (high density, strong temporal regularity), indicating that temporal encoding is very useful in stable sequences. However, its advantage is not significant in sparse data (Beauty / Clothing), highlighting the limitations of its temporal modeling capabilities in short sequences and asynchronous interaction scenarios.

[0228] The Mamba series of methods based on the State Space Model (SSM) have demonstrated excellent sequence modeling capabilities on multiple datasets. SIGMA, based on the Mamba model, serves as a strong baseline, exhibiting robust and competitive performance across various datasets. As a hybrid architecture, SIGMA's short-term feature extraction module effectively extracts features from short sequences, achieving near-optimal results on medium-to-long sequence datasets such as Beauty, Steam, and ML-1M. In particular, its Hit@10 and NDCG@10 metrics generally outperform traditional Transformer and RNN methods, validating the effectiveness of the hybrid architecture in sequence recommendation.

[0229] However, compared to SIGMA, the proposed Magecc in this embodiment achieves consistent and significant performance improvements across all datasets. The improvement is particularly pronounced on datasets with sparse sequences and highly short-term user behavior (such as Clothing), demonstrating Magecc's clear advantage in capturing local interest fluctuations and suppressing behavioral noise. In the Steam dataset with medium-length sequences, Magecc also shows steady improvements over SIGMA, indicating that the proposed local convolutional modules and context-biased networks effectively address the shortcomings of pure attention-based hybrid models in terms of temporal sensitivity and behavioral pattern alignment. It is worth noting that in the ML-1M dataset with long sequences and high behavior density, SIGMA already has strong global expressive power, but MageRec still further improves on three metrics (Hit@10 by 1.17%, NDCG@10 by 0.48%, and MRR@10 by 0.07%). This indicates that while maintaining the advantages of long-range modeling of state space structure, this model enhances its expressive power on fine-grained interest changes through the parallel Mamba-Attention mechanism, enabling it to obtain robust gains under different sequence densities and lengths.

[0230] (6.6) Ablation test

[0231] Table 3 Ablation Experiment Results for Each Module

[0232]

[0233] (6.7) The impact of different modules in the model

[0234] To verify the actual contribution of each module in MageRec, this embodiment conducted systematic ablation experiments on four datasets. The comparison models are shown below:

[0235] (1) w / o-mamba module: The Mamba module responsible for capturing long-range dependencies and sequence state evolution in the original MAGERec is completely removed, and the sequence information is only modeled by the Transformer multi-head attention layer.

[0236] (2) w / o-Attention module: The multi-head self-attention module that was originally used to model global context relationships is removed, and sequence representation only relies on the Mamba module for processing.

[0237] (3) w / o-LocalConv: The local context convolution module is removed. The model no longer models local neighborhood patterns and captures short-distance local behavioral features. It only relies on the Mamba module and the Attention module to model long-term and global dependencies, respectively.

[0238] (4) w / o-BiasNet: The BiasNet structure has been removed, meaning that the bias signal obtained from sequence context statistics is no longer provided for the gating fusion mechanism.

[0239] As shown in Table 3, all modules have a positive impact on the final performance, but their importance varies significantly across different datasets. Removing the Mamba module has a greater impact on the results for long sequences (ML-1M). Compared to removing other modules, removing the Mamba module resulted in a significant decrease in performance on ML-1M, indicating the crucial role of the state-space model in long-range dependency modeling. This is consistent with the conclusions in the comparative experiments (both the Mamba4Rec and SIGMA models achieved good results on ML-1M). Even on the short and sparse Clothing dataset, removing Mamba led to a significant degradation (NDCG@10 dropped to 0.0140), demonstrating that Mamba's global modeling ability can provide a stable representation for noisy sequences.

[0240] In contrast, the impact of removing the attention module (w / o-attention) on performance is more general, showing a significant decrease on all four datasets. This demonstrates the necessary role of self-attention mechanisms in global interest alignment and key behavior localization, proving that they remain an excellent foundational framework in the field of sequence recommendation.

[0241] The absence of local convolutional modules (w / o-conv) leads to the most significant degradation on datasets with pronounced short-term interest fluctuations. For example, in Beauty and Clothing, removing this module reduces Hit@10 to 0.0797 and 0.0301, respectively, demonstrating that convolutions effectively capture local behavioral transition patterns, which are particularly crucial for short sequences and peaks in crowded behavior. Meanwhile, in long-sequence ML-1M datasets, the absence of convolutions still results in approximately a 1.7% performance degradation, indicating that local patterns can complement global dependency modeling.

[0242] For the context bias network (BiasNet), the impact of removal is relatively mild but stable, with a performance decrease across all datasets, especially in the NDCG metric (e.g., from 0.0448 to 0.0431 in Beauty). This indicates that BiasNet makes a sustained contribution to suppressing behavioral noise and modeling local contextual differences, particularly in noisy real-world e-commerce data (Beauty, Clothing), effectively improving prediction ranking quality. The synergy of these four factors enables MageRec to achieve state-of-the-art performance across all datasets, validating the effectiveness of the parallel Mamba-Attention structure and the local-global multi-granularity modeling strategy.

[0243] Figure 4This demonstrates the impact of different numbers of MageLayer layers (i.e., Mage networks) on the experimental results. For the Amazon datasets, the number of stacked MageLayer layers (n_layers) was set to 1, 2, 3, and 4. For the Steam and ML-1M datasets, n_layers was set to 1, 2, 3, 4, and 5. The results are shown in the figure; the left vertical axis records the variation of NDCG@10, and the right vertical axis records the variation of Hit@10. The results indicate that the Beauty dataset (corresponding to...) Figure 4 (a)), Clothing dataset (corresponding to) Figure 4 (b)), ML-1M dataset (corresponding to) Figure 4 (c) The optimal value achieved when n_layers=2 indicates that shallow structures are sufficient to capture necessary long-term dependencies and global semantics. Excessive layers, on the other hand, can lead to noise accumulation and overfitting. (Steam dataset, corresponding to...) Figure 4 (d) The optimal value is achieved when n_layers=4. Steam users' behavioral sequences typically involve cross-genre games, long-term interest evolution, and phased preference changes. Deeper models are better at capturing this complex sequence structure formed by the combined effects of local patterns, long-term evolution, and contextual biases.

[0244] Figure 5 This demonstrates the impact of different numbers of attention heads in the Attention layer, using the Beauty dataset (corresponding to...). Figure 5 (a)), Clothing dataset (corresponding to) Figure 5 (b)), ML-1M dataset (corresponding to) Figure 5 (c)), Steam dataset (corresponding) Figure 5 (d) The number of attention heads (n_heads) in the experiment were 1, 2, 4, 8, 16, and 32, respectively, with the vertical axis and scale consistent with the above. The results show that the optimal results were achieved for each dataset when n_heads were 4 or 8, indicating that a moderate number of attention heads can achieve a good balance between expressive power and model stability. Compared to different numbers of MageLayer layers, the change in n_heads had a smoother impact on the model performance, indicating that the Attention module mainly plays a supplementary role in capturing global high-order interactions, rather than decisively changing the model's expressive power.

[0245] Figure 6 This demonstrates the impact of the dropout ratio on the results. The dropout_prob range in the experiment was from 0.1 to 0.5, and the vertical axis and scale of the image were consistent with those described above. The results show that when dropout_prob=0.3, Beauty (corresponding to...) Figure 6 (a) Clothing (corresponding to) Figure 6 (b) and Steam (corresponding) Figure 6 (c) Achieving the optimal result demonstrates that a moderate random deactivation mechanism can effectively alleviate overfitting in small-scale or sparse datasets, thereby improving generalization ability. ML-1M (corresponding to...) Figure 6 The optimal dropout_prob in (d) is 0.1, and the model's performance gradually decreases as the proportion increases. This is related to the large scale of the data and the more complete user behavior sequences: large-scale data has strong statistical stability and naturally has stronger generalization ability, so the model does not need to rely on a high proportion of dropout to suppress overfitting.

[0246] Example 2

[0247] This embodiment provides a sequence recommendation system for predicting potential user preferences, including:

[0248] Mapping module: used to obtain the user's historical interaction sequence, map the user's historical interaction sequence to a continuous representation space, and obtain the input tensor;

[0249] Feature extraction module: used to extract the sequence state representation and global context-dependent features of the input tensor;

[0250] Construction module: used to construct the final user interest representation based on the sequence state representation and global context-dependent features;

[0251] Prediction module: used to predict the end user's interest representation to obtain the user's recommended items.

[0252] Example 3

[0253] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the sequence recommendation method for predicting potential user preferences as described in Embodiment 1.

[0254] Example 4

[0255] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the sequence recommendation method for predicting potential user preferences as described in Embodiment 1.

[0256] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0257] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0258] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0259] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0260] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A sequence recommendation method for predicting potential user preferences, characterized in that, include: Step S1: Obtain the user's historical interaction sequence, map the user's historical interaction sequence to a continuous representation space, and obtain the input tensor; The method for mapping the user's historical interaction sequence to a continuous representation space to obtain the input tensor in step S1 includes: A user's historical interaction sequence can be represented as: ； in, A representation of the user's historical interaction sequence. The first in the user's historical interaction sequence Item ID, Total number of item IDs; The user's historical interaction sequence is embedded through the Embedding layer. Each item ID is converted into its corresponding embedding vector. : ； in, The embedding dimension of the Embedding layer; To improve the embedding vector Robustness and mitigation of overfitting risk, for embedding vectors Apply Dropout operation: ； in, For Dropout operations; For the embedding vector after Dropout operation Apply layer normalization to improve training stability and convergence speed: ； in, For layer normalization operation; Output of formula (5) Represented as input tensor : ； in, To input the batch size, The length of the padded, uniform sequence; Step S2: Extract the sequence state representation and global context-dependent features of the input tensor; The method for extracting the sequence state representation of the input tensor in step S2 includes: The input tensor is processed through the Mamba module. The output of the first linear layer is projected through two linear layers. Then, it is passed through a convolutional layer, a SiLU activation function, and a Selective SSM to obtain the first branch feature. The output of the second linear layer is then passed through the SiLU activation function to obtain the second branch feature. The first and second branch features are then multiplied by a vector and passed through another linear layer to map back to the original dimension D, ultimately yielding the initial output of the Mamba module. ; The initial output of the Mamba module After the Dropout operation, which randomly removes some neuron activations, the following is obtained: Subsequently, residual connections are used to connect the input tensor. Compared with the output after Dropout operation Element-by-element addition yields Finally, By performing feature normalization at each position of the sequence through layer normalization, the final output of the Mamba module, i.e., the sequence state representation, is obtained. , is represented as: ； ； ； in, For Dropout operations, For layer normalization operation; Step S3: Construct the final user interest representation based on the sequence state representation and global context-dependent features; Step S4: Predict the end user's interest representation to obtain the user's recommended items.

2. The sequence recommendation method for predicting potential user preferences according to claim 1, characterized in that: The method for extracting the global context-dependent features of the input tensor in step S2 includes: The input tensor is processed through the Attention module. The multi-head attention mechanism is used to map it into three sets of vectors: query, key, and value. ； in, For query vectors, The parameter matrix mapped to the Query space. Key is the key vector. The parameter matrix mapped to the Key space, Value is the value vector. This is the parameter matrix mapped to the Value space; Will , , Along the hidden dimension Divided into There are 1 attention head, and each head has 1 dimension. ,at this time , , Then the attention of the bulls The calculation formula is: ； ； in, This represents the output of each attention head. For splicing operations; Bullish attention and After multiplication and dropout, the result is multiplied by the input tensor. Perform residual joins, followed by layer normalization: ； in, For layer normalization operation, For dropout operations; right A two-layer feedforward network (FFN) is used to perform a pointwise nonlinear transformation on each sequence position, and then layer normalization is applied to map it back to the hidden layer dimension to obtain global context-dependent features. The formula is: ； in, It is a non-linear activation function. and These are the learnable weight matrices of the two-layer feedforward network FFN.

3. The sequence recommendation method for predicting potential user preferences according to claim 1, characterized in that: The method for constructing the final user interest representation based on the sequence state representation and global context-dependent features in step S3 includes: Get sequence state representation and global context dependency features ,Will and The features are summed, and local behavioral patterns are extracted through local convolution operations: ； in, For local behavioral pattern features, This is a local convolution operation; To adaptively select the most effective features under different scenarios, a three-modal fusion gating mechanism is constructed. First, the three types of features are... , , In the last dimension of the splicing: ； in, The merged user interest features; Will The original gated signal is obtained through linear layer mapping: ； in, The original gating signal, This is the weight matrix of the linear layer; The three types of features after splicing , , Average value over the sequence dimension: ； in, For sequence-level context representation, The length of the user interaction sequence. Let be the sequence state representation at position i. For the global context dependency feature at position i, This represents the local behavioral pattern feature at the i-th position; Will After passing through two layers of fully connected networks, we obtain: ； in, This is the weight matrix for the first fully connected network, used to map the input dimension from 3D to the hidden dimension D in the first layer; This is the weight matrix for the second fully connected network, used to map the hidden representations back to the original feature dimension D; For context bias; The original gate signal With context bias After activation by the Sigmoid function, we get: ； in, For gating weights, For the Sigmoid function; Obtaining the gating weight Afterwards, based on , , Perform dynamic weighted fusion: ； in, This represents element-wise dot product. To enhance training stability, we will... Then, after residual connection and layer normalization: ； in, To represent the interests of end users, For layer normalization.

4. The sequence recommendation method for predicting potential user preferences according to claim 1, characterized in that: The method for predicting the user's recommended items based on the final user interest representation in step S4 includes: For any candidate item The predicted score is: ； in, For the first A function to predict scores for each item. For any candidate item Vector representation after embedding layer Representation of end-user interests; Select predicted scores from all candidate items. The highest-rated item is the one recommended by the user.

5. The sequence recommendation method for predicting potential user preferences according to claim 1, characterized in that: The processes from steps S1 to S4 are implemented using the MAGERec network. When training the MAGERec network, the loss function used is cross-entropy loss. : ； in, Indicates the actual next item in the sequence. This represents the model's prediction score for the real target item. For the first The predicted score of each candidate item This is a set of candidate items.

6. A sequence recommendation system for predicting potential user preferences, used to implement the sequence recommendation method for predicting potential user preferences as described in any one of claims 1 to 5, characterized in that, include: Mapping module: used to obtain the user's historical interaction sequence, map the user's historical interaction sequence to a continuous representation space, and obtain the input tensor; Feature extraction module: used to extract the sequence state representation and global context-dependent features of the input tensor; Construction module: used to construct the final user interest representation based on the sequence state representation and global context-dependent features; Prediction module: used to predict the end user's interest representation to obtain the user's recommended items.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the sequential recommendation method for predicting potential user preferences as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the sequential recommendation method for predicting potential user preferences as described in any one of claims 1 to 5.

Images