A Sequential Neural Collaborative Filtering Recommendation Method Based on Self-Attention Mechanism
By combining self-attention mechanism and neural collaborative filtering model, the problem of insufficient identification of noise and long-term dependencies in existing sequential recommendation models is solved, and more accurate recommendation results are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIDIAN UNIV
- Filing Date
- 2023-03-16
- Publication Date
- 2026-06-02
AI Technical Summary
Existing sequential recommendation models cannot effectively identify noise information in users' historical behavior sequences, cannot capture long-term dependencies between users and products, and fail to make full use of global collaborative information, resulting in insufficient recommendation accuracy.
A self-attention mechanism is used to learn users’ potential preferences from the sequence of users’ historical behavior records. By combining the self-attention mechanism with a neural collaborative filtering model, long-term dependencies are captured and noise is eliminated. The product feature vectors learned by the self-attention mechanism are fused with the original feature vectors to construct a sequential recommendation framework.
It improves the accuracy of recommendations, better captures long-term and short-term dependency information between users and products, reduces noise interference, and provides more accurate recommendation results.
Smart Images
Figure CN116151928B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of recommender system technology, and in particular to a sequential neural collaborative filtering recommender method based on a self-attention mechanism. Background Technology
[0002] With the rapid development of the internet, the amount of data across various scenarios is also increasing, making it more difficult for users to find what they want online. For this reason, recommender systems are widely used in various scenarios. A key challenge in constructing a recommender system is mining users' latent interests to make more accurate recommendations. To address this problem, researchers extract users' latent interests from their historical behavior sequences, enabling serialized recommendations. Compared to traditional recommender systems, serialized recommender systems can capture users' long-term dependencies and dynamic preferences from their historical behavior records. In recent years, serialized recommendation has become a hot technology in recommender systems. Traditional serialized recommender models mainly include sequence pattern mining models, Markov chain models, and latent semantic models. Sequence pattern mining models extract user behavior patterns from their historical behavior sequences. One drawback of this model is that it generates a large number of patterns, and some unpopular items will not be recommended. Markov chain models use Markov chains to establish interaction relationships between users and items, but this model can only capture short-term dependencies, ignoring long-term dependencies. Latent semantic models can learn the latent semantics of users and items in the recommender system to make recommendations. This type of model is easily affected by scarce data and sometimes cannot make accurate recommendations.
[0003] The patent application "Serialization Recommendation Based on CNN and RNN" (patent number: 201811548205.8) from Hunan University proposes a serialization recommendation algorithm based on a combination of CNN (Convolutional Neural Network) and RNN (Recurrent Neural Network). This algorithm utilizes the local feature learning capability of CNN to capture correlations in recent historical behavior data, while simultaneously leveraging the global and sequence learning capabilities of RNN to learn the short- and long-term preferences of users' historical behavior. Finally, it uses a multilayer perceptron to predict future user behavior and provide recommendations based on the learned feature representations. Experiments show that this algorithm outperforms single CNN or RNN-based serialization recommendation methods. This invention has high application value and can be widely applied to various recommendation scenarios such as e-commerce, news portals, and entertainment. The drawback of this method is that CNN and RNN networks cannot determine the importance of different products in the sequence and fail to distinguish noise information in the sequence, thus affecting the recommendation results.
[0004] The patent application "Serialized Recommendation Method Based on Product Association Relationships" (Patent No.: 202111059171.8) from Shanghai Jiao Tong University is characterized by the following steps: Acquiring interaction data: obtaining interaction data between users and products from the network; Constructing a symbiotic relationship graph: using the interaction data to construct a symbiotic relationship graph of products, the symbiotic relationship graph being represented by an adjacency matrix of the association relationship graph; Graph convolutional network: performing graph convolution operations on the adjacency matrix of the association relationship graph to obtain the association features of products; Recommendation model training: inputting the association features of products into the recommendation model for training; Serialized recommendation: having the recommendation model output a serialized recommendation of product association relationships. The drawback of this method is that it simply uses the user's serialized data in the recommendation process, failing to consider the user's short-term preferences and not incorporating global collaborative information into the recommendation process.
[0005] The patent application from Northwestern Polytechnical University, entitled "A Serialized Recommendation Method Based on Long-Term and Short-Term Interests" (Patent No.: 202010014762.2), is characterized by the following steps: S1: Acquiring data and preprocessing it; S2: Processing all comment texts and question texts, selecting the highest-scoring words from the relevant texts of each product as features, and describing the product using the set of all features to construct a feature representation matrix for the product; S3: Constructing a vector representation of the user's purchase sequence: obtaining a vector representation of each user's purchase sequence based on the product's feature representation matrix and the user's historical purchase sequence; S4: Representing the user's long-term and short-term interest preferences respectively; S5: Obtaining the user's aggregated preference by using an attention mechanism to combine the user's long-term and short-term interest preferences; S6: Obtaining the probability of the user interacting with the product after asking a question by determining the relationship between the aggregated preference and the target product; S7: Using the cross-entropy loss function to learn the model parameters and obtain the probability of each product being purchased after the question time. The drawback of this method is that simply using sequence information for recommendation ignores global collaborative interaction information, thus affecting the accuracy of the recommendation. Summary of the Invention
[0006] The purpose of this invention is to propose a sequential neural collaborative filtering recommendation method based on a self-attention mechanism. This method uses a self-attention mechanism to learn the user's potential preferences from the user's historical behavior record sequence, obtain information-enhanced feature vectors of the recommended items, eliminate most of the noise in the sequence, and capture the long-term dependencies of the user's historical behavior record sequence.
[0007] The technical solution adopted in this invention is as follows:
[0008] This invention is a sequential neural collaborative filtering recommendation method based on a self-attention mechanism, comprising the following steps:
[0009] In recommendation scenarios, user interaction records with products are collected as initial data, and then product IDs that have interacted with users are organized into a sequence of user historical behavior records.
[0010] In recommendation scenarios, user IDs and product IDs are one-hot embedded to obtain user feature vectors and product feature vectors.
[0011] The feature vector of the product to be recommended is placed at the end of the user's historical behavior record sequence, and such user historical behavior record sequence data is input into the self-attention mechanism to learn the sequence information and obtain the feature vector of the product to be recommended with rich sequence information.
[0012] The feature vector of the product to be recommended, which has rich sequence information, along with the user feature vector and product feature vector obtained from the original embedding, are input into the serialized generalized matrix factorization part and the serialized multilayer perceptron part of the neural collaborative filtering model to complete the user's rating of the product to be recommended, and finally make a recommendation.
[0013] Furthermore, in a recommendation scenario with m users and n products, then U = {u1, u2, ..., u} m Let R = {r1, r2, ..., r} represent the user feature vector set. n Let S = {s1, s2, ..., s} represent the set of product feature vectors. k} represents a set of user history behavior record sequences, and the length of the user history behavior record sequence is k;
[0014] User i learns the sequence information of product j specifically as follows:
[0015] By placing product j at the end of the user's historical behavior record sequence, we obtain the product feature vector matrix E;
[0016] The aforementioned user historical behavior record sequence data is input into the self-attention mechanism to learn sequence information, resulting in a feature vector s of the product to be recommended with rich sequence information.
[0017] The historical behavior record sequence data is input into the self-attention mechanism to learn sequence information, specifically:
[0018] First, the product feature vector matrix E undergoes three different linear transformations to obtain Q, K, and V, as shown in the following formulas:
[0019] Q = EW Q K = EW K V = EW V (1)
[0020] Where Q represents the query feature vector matrix, K represents the keyword feature vector matrix, V represents the value feature vector matrix, and W represents the linear transformation matrix that performs a linear transformation on the input matrix E;
[0021] W Q It is a linear transformation matrix that yields the query feature vector matrix Q by performing a linear transformation on the input matrix E;
[0022] W K It is a linear transformation matrix that yields the keyword feature vector matrix K by performing a linear transformation on the input matrix E;
[0023] W V It is the linear transformation matrix of the input matrix E to obtain the eigenvector matrix V.
[0024] Then, scaled dot product attention is used to calculate the attention, and the specific calculation formula is as follows:
[0025]
[0026] Where Q represents the query feature vector matrix, K represents the keyword feature vector matrix, V represents the value feature vector matrix, and d k The size of the dimensions of the query vector, keyword vector, and value vector;
[0027] Softmax is a normalized exponential function used to obtain the weights for attention.
[0028] Furthermore, the self-attention mechanism employs a multi-head self-attention mechanism to mine the latent information of the product feature vector from multiple perspectives. The multi-head self-attention mechanism divides the original product feature vector into several low-dimensional sub-feature vectors for attention calculation, and then concatenates the various sub-feature vectors to obtain the output of the multi-head self-attention mechanism. The specific formula is as follows:
[0029] Multihead(Q,K,V) = Concat(head1,...,head h W O (3)
[0030] head i =Attention(QW i Q ,KW i K VW i V (4)
[0031] Among them, head i This represents the attention calculation result for the i-th head.
[0032] h represents the number of heads in the multi-head attention mechanism.
[0033] W i Q W i K W i V Let represent the linear transformation matrices corresponding to the query feature vector matrix, keyword feature vector matrix, and value feature vector matrix in the i-th head, respectively.
[0034] W O It is a matrix that is linearly transformed after concatenating the attention calculation results of each head;
[0035] The feature vector matrices obtained from the attention mechanisms of each head are concatenated together. The last feature vector of this matrix is the feature vector s of the recommended product with rich sequence information.
[0036] Furthermore, the length of the user historical behavior record sequence is k. The length of different user historical behavior record sequences varies, requiring a standardization of length for sequences of different lengths. Specifically:
[0037] For long sequences, select the k items with the most recent time series from the sequence to form a sequence of user historical behavior records;
[0038] For short sequences, this invention uses the number 0 to fill in the missing positions until the length of the sequence reaches k;
[0039] In the attention calculation, an additional padding mask feature vector matrix is used to eliminate the influence of the added digits 0 on the sequence information. For the attention calculation of each head, scaled dot product attention is used to learn the sequence information. The specific attention score is calculated as follows:
[0040]
[0041] Among them, Q i ,K i V i These represent the query feature vector matrix, keyword feature vector matrix, and value feature vector matrix, respectively, in the i-th head attention calculation.
[0042] softmax is a normalized exponential function that maps the result of each dimension to a value between 0 and 1.
[0043] M represents the padding mask feature vector matrix, used to address the issue of varying lengths in user historical behavior record sequences;
[0044] head i This represents the feature vector matrix of the recommended products learned by each head, which contains rich sequence information.
[0045] Furthermore, the neural collaborative filtering model includes serialized generalized matrix factorization and serialized multilayer perceptron. Specifically, for the serialized generalized matrix factorization part:
[0046] The original user feature vector and product feature vector are multiplied element-wise, and the original user feature vector and the product feature vector s with rich sequence information are multiplied element-wise. The specific formulas are as follows:
[0047] p1 = u i ⊙r j (6)
[0048] p2 = u i ⊙s (7)
[0049] Among them, u i It is the original user feature vector, r j is the original product feature vector, s is the feature vector of the product to be recommended with rich sequence information, p1 represents the latent feature vector with short-term dependency information, and p2 represents the latent feature vector with long-term dependency information from the user's historical behavior record sequence.
[0050] Finally, the two vectors p1 and p2 are combined using a linear associative method, as shown in the following formula:
[0051] p=α×p1+(1-α)×p2 (8)
[0052] Where p is a potential feature vector that simultaneously possesses short-term and long-term dependency information, and α is a hyperparameter used to control the proportion of p1 and p2 when calculating p.
[0053] Furthermore, specifically regarding the serialized multilayer perceptron portion:
[0054] First, the original user feature vector u i The original product feature vector r j It is connected to the feature vector s of the recommended product, which has rich sequence information;
[0055] Then, the concatenated feature vectors are input into a serialized multilayer perceptron, as shown in the following formula:
[0056]
[0057] q1=a1(W1 T q0+b1) (10)
[0058] ...
[0060]
[0061] Where q0 represents the user feature vector u i Product feature vector r j The result after concatenating the feature vector s of the recommended product with rich sequence information;
[0062] q1, q2, ..., q h These represent the latent feature vectors of each layer in a serialized multilayer perceptron;
[0063] W h T b h a h Let f(x), g(x), t(x), and f(x) represent the activation function, weight matrix, and bias vector of the h-th layer in the serialized multilayer perceptron, respectively.
[0064] The final q h In other words, it refers to potential feature vectors that have complex nonlinear relationships between input vectors.
[0065] Furthermore, based on the serialized generalized matrix factorization and serialized multilayer perceptron of the neural collaborative filtering model, feature vectors q with complex nonlinear information are obtained respectively. h Given an eigenvector p that has both long-term and short-term dependencies, q h Connecting p with the data, and integrating long-term and short-term dependency information and non-linear complex information between users and products, a final recommendation rating prediction is made. The specific formula is as follows:
[0066]
[0067] y i,j = sigmod(h T z) (14)
[0068] Here, z is the total feature vector that combines the feature vectors generated by serialized generalized matrix factorization and serialized multilayer perceptron. The sigmoid function maps the predicted score to between 0 and 1 and compares the final recommendation score prediction results.
[0069] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:
[0070] This invention is a sequential neural collaborative filtering recommendation method based on a self-attention mechanism. It constructs a complete sequential recommendation framework, which includes a sequence information learning part and a rating prediction part based on a neural collaborative filtering model. The sequence information learning part uses a self-attention mechanism to learn the user's historical behavior record sequence, which not only learns the user's historical preference information well, but also avoids the interference of noise in the historical behavior sequence information. The feature vector of the recommended product learned by the self-attention mechanism and the original user feature vector and product feature vector are input into the neural collaborative filtering model, which captures the long-term dependency information and short-term dependency information in the recommendation scenario, and obtains more accurate recommendation results. Attached Figure Description
[0071] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort, wherein:
[0072] Figure 1 It is a sequential neural collaborative filtering model based on the self-attention mechanism;
[0073] Figure 2 It is a sequence information learning mechanism based on self-attention. Detailed Implementation
[0074] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described embodiments are merely some embodiments of the invention, and not all embodiments. The components of the embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0075] It should be noted that the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0076] For most user history behavior records in real-world recommendation environments, each sequence contains a large amount of user history data. However, for a specific sequence, different records within the sequence have different impacts on the recommendation effect; some records have a positive impact, while others have a negative impact. For example, if a user wants to buy a new mobile phone on a shopping website, in this case, electronics from the user's history behavior record sequence are more meaningful for sequential recommendation than food, clothing, and other products. Therefore, to make more accurate recommendations, sequential recommendation systems need to identify the importance of each item in the user's history behavior record sequence and ignore noise in the user's history behavior record sequence.
[0077] This invention presents a sequential neural collaborative filtering recommendation method based on a self-attention mechanism. It proposes a sequential recommendation method that can identify and remove much noise in the user's historical behavior record sequence, while simultaneously capturing both long-term and short-term dependencies between users and items. The model consists of two main parts: a sequence information learning mechanism and a sequential neural collaborative filtering model. The proposed method is named Sequential Neural Collaborative Filtering (SNCF). In the sequence information learning part, the recommended item is placed at the end of the user's historical behavior record sequence, and the user's historical preference information is learned through a self-attention mechanism. This preference information is then incorporated into the item feature vector, resulting in an information-enhanced item feature vector with rich sequence information. This item feature vector, along with the original user and item feature vectors, is then input into the neural collaborative filtering model to obtain the recommendation result. The specific framework is as follows: Figure 1 As shown.
[0078] 1. Self-attention mechanism
[0079] Traditional neural networks used to learn sequence information always assume that all items in the sequence are equally important to the learned information. However, in the real world, the importance of different items in a sequence varies, and traditional neural networks cannot identify which item is more useful to the learned information. To solve this problem, researchers have applied attention mechanisms to sequence learning. Attention mechanisms assign a weight to each item in the sequence to distinguish the importance of different items.
[0080] An important attention mechanism computation method is scaled dot product attention. This mechanism first calculates the dot product of the query vector and the keyword vector. Then, it divides the dot product result by the dimensions of the query vector, keyword vector, and value vector, and applies the softmax function (i.e., the normalized exponential function) to obtain the attention weights. Finally, the attention mechanism multiplies the obtained weights by the value vector to obtain the output of the attention mechanism. The specific calculation formula is shown below:
[0081]
[0082] Where Q represents the query vector matrix, K represents the keyword vector matrix, V represents the value vector matrix, and d k This represents the dimension size of the query vector, keyword vector, and value vector.
[0083] Self-attention is a variant of attention mechanisms, offering significant advantages in processing serialized data. This method applies three different linear transformations to the serialized data: a query vector matrix, a keyword vector matrix, and a value vector matrix. In this way, self-attention achieves a better embedding representation of the items and global sequence information within the sequence, and can identify and remove noise from the sequence data.
[0084] 2. Sequence Information Learning
[0085] In the sequence information learning mechanism of this invention, in order to learn sequence information from the user's historical behavior record sequence, this invention places the item to be rated at the end of the user's historical behavior record sequence and inputs this sequence into the sequence learning mechanism. From the rich, denoised sequence information, a better feature vector of the item to be rated is learned. The specific learning mechanism is as follows: Figure 2 As shown.
[0086] In a recommendation scenario with m users and n products, U = {u1, u2, ..., u} m Let R = {r1, r2, ..., r} represent the user feature vector set. n} represents the set of product feature vectors, S = {s 11 ,s2,...,s k} represents a set of user history behavior record sequences, and the length of the user history behavior record sequence is k;
[0087] User i learns the sequence information of product j specifically as follows:
[0088] By placing product j at the end of the user's historical behavior record sequence, we obtain the product feature vector matrix E.
[0089] The user's historical behavior record sequence data is input into the self-attention mechanism to learn the sequence information, and a feature vector s of the product to be recommended with rich sequence information is obtained.
[0090] Specifically, firstly, the product feature vector matrix E undergoes three different linear transformations to obtain Q, K, and V, as shown in the following formulas:
[0091] Q = EW Q K = EW K V = EW V (1)
[0092] Where Q represents the query feature vector matrix, K represents the keyword feature vector matrix, V represents the value feature vector matrix, and W represents the linear transformation matrix that performs a linear transformation on the input matrix E;
[0093] W Q It is a linear transformation matrix that yields the query feature vector matrix Q by performing a linear transformation on the input matrix E;
[0094] W K It is a linear transformation matrix that yields the keyword feature vector matrix K by performing a linear transformation on the input matrix E;
[0095] W V It is the linear transformation matrix of the input matrix E to obtain the eigenvector matrix V.
[0096] Self-attention mechanisms can simultaneously calculate the weights of all dimensions in a feature vector, but they only mine latent information from one perspective. Therefore, to make more accurate recommendations, this invention uses a multi-head self-attention mechanism to mine latent information from multiple perspectives in the product feature vector. The multi-head self-attention mechanism divides the original product feature vector into several low-dimensional sub-feature vectors for attention calculation, and then concatenates these sub-vectors to obtain the output of the multi-head self-attention mechanism. The specific formula is as follows:
[0097] Multihead(Q,K,V)=Concat(head1,...,head h W O (3)
[0098] head i =Attention(QW i Q ,KW i K VW i V (4)
[0099] Among them, head iThis represents the attention calculation result for each head, where h represents the number of heads in the multi-head attention mechanism;
[0100] W i Q W i K W i V These represent the linear transformation matrices corresponding to the query feature vector matrix, keyword feature vector matrix, and value feature vector matrix in each header, respectively.
[0101] W O It is a matrix that is linearly transformed after concatenating the attention calculation results of each head;
[0102] For attention calculation of each head, this invention uses scaled dot product attention to learn sequence information. However, in reality, the lengths of different user historical behavior record sequences are different, so it is also necessary to unify the length of sequences of different lengths. For long sequences, this invention selects the k most recent items from the sequence to form the user historical behavior record sequence. For short sequences, this invention fills in missing positions with digits 0 until the sequence length reaches k. Compared with the original scaled dot product attention, in order to eliminate the influence of the added digits 0 on the sequence information, this invention uses an additional padding mask feature vector matrix in attention calculation. The specific attention score calculation is as follows:
[0103]
[0104] Among them, Q i ,K i V i These represent the query feature vector matrix, keyword feature vector matrix, and value feature vector matrix corresponding to each head i, respectively.
[0105] softmax is a normalized exponential function that maps the result of each dimension to a value between 0 and 1.
[0106] M represents the padding mask feature vector matrix, used to address the issue of varying lengths in user historical behavior record sequences;
[0107] head i This represents the feature vector matrix of the recommended products learned by each head, which contains rich sequence information.
[0108] Subsequently, the present invention concatenates the feature vector matrices obtained from each attention mechanism together, and the last feature vector of this matrix is the feature vector s of the recommended product with rich sequence information.
[0109] 3. Neural Co-filtering
[0110] After obtaining the feature vector s of the recommended product with rich sequence information, this invention combines the feature vector s of the recommended product with rich sequence information with the original user feature vector u. ii and product feature vector r j The data is input into a neural collaborative filtering model for score prediction.
[0111] The neural collaborative filtering model includes Generalized Matrix Factorization (GMF) and Multi-Layer Perceptron (MLP). Specifically, regarding the Generalized Matrix Factorization part:
[0112] The original user feature vector and product feature vector are multiplied element-wise, and the original user feature vector and the product feature vector s with rich sequence information are multiplied element-wise. The specific formulas are as follows:
[0113] p1 = u i ⊙r j (6)
[0114] p2 = u i ⊙s (7)
[0115] Among them, u i It is the original user feature vector, r j is the original product feature vector, s is the feature vector of the product to be recommended with rich sequence information, p1 represents the latent feature vector with short-term dependency information, and p2 represents the latent feature vector with long-term dependency information from the user's historical behavior record sequence.
[0116] Finally, the two vectors p1 and p2 are combined using a linear associative method, as shown in the following formula:
[0117] p=α×p1+(1-α)×p2 (8)
[0118] Where p is a potential feature vector that simultaneously possesses short-term and long-term dependency information, and α is a hyperparameter used to control the proportion of p1 and p2 when calculating p.
[0119] Corresponding to serialized generalized matrix factorization, serialized multilayer perceptron is another important component of neural collaborative filtering. Serialized multilayer perceptron is a deep neural network that can map high-dimensional feature vectors to a low-dimensional space to achieve dimensionality reduction. Serialized multilayer perceptron is good at capturing the nonlinear relationship between user feature vectors and product feature vectors.
[0120] In the method of this invention, specifically for the serialized multilayer perceptron portion:
[0121] First, the original user feature vector u i Product feature vector r j It is connected to the feature vector s of the recommended product, which has rich sequence information;
[0122] Then, the concatenated feature vectors are input into a serialized multilayer perceptron, as shown in the following formula:
[0123]
[0124] q1=a1(W1 T q0+b1) (10)
[0125] ...
[0127]
[0128] Where q0 represents the user feature vector u i Product feature vector r j The result after concatenating the feature vector s of the recommended product with rich sequence information;
[0129] q1, q2, ..., q h These represent the latent feature vectors of each layer in a serialized multilayer perceptron;
[0130] W h T b h a h Let f(x), g(x), t(x), and f(x) represent the activation function, weight matrix, and bias vector of the h-th layer in the serialized multilayer perceptron, respectively.
[0131] The final q h In other words, it refers to potential feature vectors that have complex nonlinear relationships between input vectors.
[0132] 4. Recommended output
[0133] Based on the serialized generalized matrix factorization model of the neural collaborative filtering model and the serialized multilayer perceptron, this invention obtains feature vectors q with complex nonlinear information. h Given a feature vector p that simultaneously exhibits both long-term and short-term dependencies, in the method of this invention, for the purpose of making the final recommendation score prediction, this invention will use q... h Connecting p with it to integrate short-term and long-term dependency information and non-linear complex information between users and goods, the specific formula is as follows:
[0134]
[0135] y i,j = sigmod(h T z) (14)
[0136] Here, z is the total feature vector that combines the feature vectors generated by serialized generalized matrix factorization and serialized multilayer perceptron. The sigmoid function maps the predicted score to between 0 and 1 and compares the final recommendation score prediction results.
[0137] This invention constructs a complete sequence recommendation framework, which includes a sequence information learning part and a rating prediction part based on a neural collaborative filtering model. The sequence information learning part uses a self-attention mechanism to learn the user's historical behavior record sequence, which not only learns the user's historical preference information well, but also avoids the interference of noise in the historical behavior sequence information. The information learned from the sequence information is used to enhance the product feature vector and the original user feature vector and product feature vector, which are then input into the neural collaborative filtering model to capture the long-term dependency information and short-term dependency information in the recommendation scenario, resulting in more accurate recommendation results.
[0138] Experimental instructions
[0139] The present invention conducts experiments on real-world datasets from two different domains to verify the effectiveness of the proposed method.
[0140] These two datasets are the movie dataset ML-1m and the e-commerce platform dataset Beauty.
[0141] In the experiments of this invention, to verify the effectiveness of the invented method, the entire dataset is divided into a training set, a validation set, and a test set according to the timestamps of the interaction records. This invention selects the products with which the user has most recently interacted as the test set, the next most recently interacted products as the validation set, and the remaining products as the training set.
[0142] When constructing the validation and training sets, this invention randomly selects 100 items from the items that each user has not interacted with as negative samples. When constructing the training set, this invention randomly selects 4 items from the items that each user has not interacted with as negative samples. For the sequence learning mechanism, this invention selects the 20 most recently interacted items as the user's historical behavior record sequence. For users with fewer than 20 historical records, the empty spaces are filled with the number 0. In the experiment, this invention sets the feature vector dimension to 64, the training learning rate to 0.001, and the batch size for the training process to 128.
[0143] Table 1 Model Performance Indicators
[0144]
[0145] Table 1 compares the performance of the proposed method with various state-of-the-art serialization recommendation methods in terms of HR (Hit Rate) and NDCG (Normalized Discounted Cumulative Gain). Higher HR and NDCG values indicate better performance. Compared to existing methods, the proposed method (SNCF) shows significant performance improvement on the Beauty dataset, achieving the best performance when the Top-k value is 5 and 10. On the ML-1m dataset, it achieves the best performance when the Top-k value is 10, indicating that the serialization recommendation model of this invention successfully utilizes serialization information to effectively improve recommendation accuracy. Compared to the ML-1m dataset, the Beauty dataset is sparser, suggesting that the proposed method can achieve better recommendation accuracy on relatively sparse datasets.
[0146] Compared to most existing serialization recommendation methods, the method of this invention achieves better performance, indicating that the serialization learning mechanism of this invention filters out more noise from the user's historical behavior record sequences and learns more useful information from them. Compared to the traditional NCF (Neural Collaborative Filtering) model, the serialization neural collaborative filtering model of this invention shows significant improvements on both datasets, indicating that the serialization learning mechanism of this invention improves the effectiveness of neural collaborative filtering.
[0147] In the sequence information learning part of the method of the present invention, the method of learning new product feature vectors through self-attention mechanism can also be: after processing the user's historical behavior record sequence with attention mechanism, the feature vectors in the sequence are weighted and summed to obtain the corresponding recommendation output.
[0148] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be conceived by those skilled in the art within the technical scope disclosed in the present invention without creative effort should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.
Claims
1. A sequential neural collaborative filtering recommendation method based on self-attention mechanism, characterized in that, Includes the following steps: In recommendation scenarios, user interaction records with products are collected as initial data, and then product IDs that have interacted with users are organized into a sequence of user historical behavior records. In recommendation scenarios, user IDs and product IDs are one-hot embedded to obtain user feature vectors and product feature vectors. The feature vector of the product to be recommended is placed at the end of the user's historical behavior record sequence, and such user historical behavior record sequence data is input into the self-attention mechanism to learn the sequence information and obtain the feature vector of the product to be recommended with rich sequence information. The feature vectors of the recommended products, rich in sequence information, along with the user and product feature vectors obtained from the original embedding, are input into the serialized generalized matrix factorization and serialized multilayer perceptron parts of the neural collaborative filtering model. In the recommendation scenario, there are m users and n products, then U = {u1, u2, ..., u...} m Let R = {r1, r2, ..., r} represent the user feature vector set. n Let S = {s1, s2, ..., s} represent the set of product feature vectors. k } represents a set of user historical behavior record sequences, with a sequence length of k; the neural collaborative filtering model includes serialized generalized matrix factorization and serialized multilayer perceptron. Specifically, for the serialized generalized matrix factorization part: The original user feature vector and product feature vector are multiplied element-wise, and the original user feature vector and the product feature vector s with rich sequence information are multiplied element-wise. The specific formulas are as follows: (6) (7) in, u i It is the original user feature vector. r j It is the original product feature vector. s It is a feature vector of the recommended product with rich sequence information. This represents a latent feature vector with short-term dependency information. This represents a latent feature vector containing long-term dependency information within a sequence of user historical behavior records. Finally, the two vectors above are combined using a linear associativity method. , To merge, use the following formula: (8) in, p It is a latent feature vector that simultaneously possesses both short-term and long-term dependency information, and α is used to control... and In calculation p Hyperparameters representing the proportion of time; For the serialized multilayer perceptron part, specifically: First, the original user feature vector u... i The original product feature vector r j The feature vector s of the recommended product, which has rich sequence information, is connected to it using the form of Equation (9). The specific formula is as follows: (9) Where q0 represents the user feature vector u i Product feature vector r j The result after concatenating the feature vector s of the recommended product with rich sequence information; Then, the concatenated feature vectors are input into a serialized multilayer perceptron to calculate the user's rating for the recommended products, and finally, recommendations are made.
2. The sequential neural collaborative filtering recommendation method based on self-attention mechanism according to claim 1, characterized in that: user i For goods j Sequence information learning specifically involves: Goods j By placing it at the end of the user's historical behavior record sequence, we obtain the product feature vector matrix E; The aforementioned user historical behavior record sequence data is input into a self-attention mechanism to learn sequence information, resulting in a feature vector of the product to be recommended that contains rich sequence information. s .
3. The sequential neural collaborative filtering recommendation method based on self-attention mechanism according to claim 2, characterized in that: The historical behavior record sequence data is input into the self-attention mechanism to learn sequence information, specifically: First, the product feature vector matrix E Perform three different linear transformations to obtain Q , K , V The specific formula is as follows: (1) in, Q This indicates a query feature vector matrix. K Represents the keyword feature vector matrix. V Represents the eigenvector matrix of values. W This indicates that for the input matrix E A linear transformation matrix that undergoes linear transformation; For the input matrix E The query feature vector matrix is obtained by performing a linear transformation. Q The linear transformation matrix; For the input matrix E The keyword feature vector matrix is obtained by performing a linear transformation. K The linear transformation matrix; For the input matrix E The eigenvector matrix is obtained by performing a linear transformation. V The linear transformation matrix; Then, scaled dot product attention is used to calculate the attention, and the specific calculation formula is as follows: （2） in, Q This indicates a query feature vector matrix. K Represents the keyword feature vector matrix. V Represents the eigenvector matrix of values. The size of the dimensions of the query vector, keyword vector, and value vector; Softmax is a normalized exponential function used to obtain the weights for attention.
4. The sequential neural collaborative filtering recommendation method based on self-attention mechanism according to claim 3, characterized in that: The self-attention mechanism employs a multi-head self-attention mechanism, which mines the latent information of the product feature vector from multiple perspectives. The multi-head self-attention mechanism divides the original product feature vector into several low-dimensional sub-feature vectors for attention calculation, and then concatenates the various sub-feature vectors to obtain the output of the multi-head self-attention mechanism. The specific formula is as follows: (3) (4) in, head i Indicates the first i The results of head attention calculations h This indicates the number of heads in a multi-head attention mechanism. W i Q ， W i K ，W i V They represent the first i The linear transformation matrices corresponding to the query feature vector matrix, keyword feature vector matrix, and value feature vector matrix in the header. W O It is a matrix that is linearly transformed after concatenating the attention calculation results of each head; By concatenating the feature vector matrices obtained from the attention mechanisms of each head, the last feature vector in this matrix becomes the feature vector of the recommended product, which contains rich sequence information. s .
5. The sequential neural collaborative filtering recommendation method based on self-attention mechanism according to claim 4, characterized in that, The length of the user historical behavior record sequence is k， The lengths of historical behavior record sequences vary among different users, necessitating a standardization of sequence length. Specifically: For long sequences, select the one with the closest time series from the sequence. k Each product is used to form a sequence of user history behavior records; For short sequences, this invention uses the digit 0 to fill in missing positions until the sequence length reaches a certain threshold. k ; In the attention calculation, an additional padding mask feature vector matrix is used to eliminate the influence of the added digits 0 on the sequence information. For the attention calculation of each head, scaled dot product attention is used to learn the sequence information. The specific attention score is calculated as follows: (5) in, Q i , K i , V i They represent the first i The corresponding query feature vector matrix, keyword feature vector matrix, and value feature vector matrix in head attention calculation; softmax is a normalized exponential function that maps the result of each dimension to a value between 0 and 1. M This represents the fill mask feature vector matrix, used to address the issue of varying lengths in user historical behavior record sequences; head i This represents the feature vector matrix of the recommended products learned by each head, which contains rich sequence information.
6. The sequential neural collaborative filtering recommendation method based on self-attention mechanism according to claim 1, characterized in that, Specifically, regarding the serialized multilayer perceptron portion: The concatenated feature vectors are then input into a serialized multilayer perceptron, as shown in the following formula: (10) (11) (12) q 1, q 2, … , q h These represent the latent feature vectors of each layer in a serialized multilayer perceptron; W h T , b h 、a h These represent the first and second layers of the serialized multilayer perceptron, respectively. h The activation function, weight matrix, and bias vector of the layer; The final q h In other words, it refers to potential feature vectors that have complex nonlinear relationships between input vectors.
7. The sequential neural collaborative filtering recommendation method based on self-attention mechanism according to claim 6, characterized in that, Based on the serialized generalized matrix factorization and serialized multilayer perceptron of the neural collaborative filtering model, feature vectors with complex nonlinear information are obtained respectively. q h With feature vectors that simultaneously exhibit both long-term and short-term dependencies p ,Will q h and p By connecting and integrating short-term and long-term dependency information and non-linear complex information between users and products, a final recommendation rating prediction is made. The specific formula is as follows: (13) (14) in, z It is a total feature vector that combines the feature vectors generated by serialized generalized matrix factorization and serialized multilayer perceptron. The sigmoid function maps the predicted score to between 0 and 1, and then compares the final recommendation score prediction results.