Time sequence recommendation model based on contrast enhancement time perception self-attention mechanism
By introducing a comparison-enhanced time-perceived self-attention mechanism in the sequential recommendation model, combining project time interval, item heat and user rating, the problem of existing models ignoring time information is solved, and the recommendation effect is improved, especially on sparse data sets. Performance.
Patent Information
- Application Number
- CN202410855908.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-28
- Publication Date
- 2024-12-06
AI Technical Summary
When modeling users' long-term preferences and short-term intentions, the existing sequential recommendation model ignores rich time information such as relative time intervals between projects, item popularity information, user scores and item absolute timestamps, resulting in poor recommendation results, especially Inadequate performance on sparse datasets.
A time series recommendation model (CATSASRec) based on a contrast-enhanced time-aware self-attention mechanism is proposed. Through the data preprocessing layer, embedding layer and self-attention layer, combining the time interval between items, item location information, user ratings and item heat information. , calculate new position information and attention weights, and optimize in the prediction and contrast learning layer.
Effectively model the relative time intervals between projects, item popularity information, user ratings and item absolute timestamps, improve the accuracy and personalization of recommendations, especially on sparse data sets, which outperform advanced algorithms and enhance user personality Expression and recommendation effects.
Smart Images

Figure CN119089938A_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to the technical field of sequential recommendation systems, in particular to the technical field of time series recommendation models. [Background technology]
[0002] A sequential recommendation system is a system that can predict the next item based on a sequence of user interactions through various calculations. Since the core of a sequential recommendation system is to predict whether a user will interact with a product (including clicks, ratings, and purchases), and the interaction history usually contains a variety of information, it is very important to model the user's long-term preferences and short-term intentions.
[0003] In sequential recommendation, data modeling mainly considers two aspects - time recommendation and sequential recommendation. Time recommendation focuses on modeling absolute timestamps to capture the temporal dynamics of users and items (such as a user's interest level may change in different time periods or the popularity of an item may increase or decrease over time). This model is very useful when studying the temporal changes of data sets, and can more quickly mine the dynamic changes in users' interests so as to make more accurate recommendations. Most models of sequential recommendation sort items according to the interaction timestamp, and after sorting, the timestamp is no longer used, but only focuses on sequential pattern mining to predict the next possible interaction, such as SASRec (W.-C.Kang and J.McAuley.2018. "Self-Attentive Sequential Recommendation".in Proceeding ofthe 18th IEEE International Conference onData Mining(ICDM)). In addition, there are a small number of models that introduce the use of timestamps on the basis of sequential recommendation, such as TiSASRec (which also uses the time interval between items in sequential recommendation).
[0004] Traditional sequential recommendation methods discard timestamps and only keep the order of items. In other words, in this method, the only elements that affect the next item are the position and identity of the item (for example, as long as two users have the same order, it is considered that the interaction of the two users has the same impact on the next item). On this basis, TiSASRec proposes the concept of relative time. It believes that the items with closer timestamps have a greater impact on the next item (for example, if two users have the same interaction order, and one of them has these interactions in a short time, while the other has completed these interactions in a long time, so even if the two users have the same sequential position, the interaction of the two users should have different impacts on the next item). However, the position information in sequential recommendation should not only use the position of the item in the sequence, and the value range of the item position is [1,2,...,L]. This value pattern is too local and not distinctive. This is because this position information only considers the position of the item in the current sequence, and does not consider the position of the item in the entire historical interaction sequence of the user. In addition, this position information does not consider the situation of the same timestamp at all. In fact, most of the existing shopping websites have shopping carts, and the shopping cart allows users to generate multiple shopping information at the same time (in this case, these items should have the same location information).
[0005] The attention mechanism is a mechanism that computes weights on the input to make the model more interpretable (the idea behind the attention mechanism is that the output depends on relevant, specific parts of the input). The attention mechanism usually uses scaled dot product attention and is defined as (i.e., formula (1)). Where Q, K, and V represent query, key, and value, respectively (in self-attention, these three inputs usually use the same object). At present, the attention mechanism has been incorporated into the sequential recommendation model. For example, SASRec is a model that applies the self-attention mechanism to sequential recommendation and has achieved remarkable results in the recommendation problem; DFAR (G.Lin, C.Gao, Y.Zheng, et al. 2023. "Dual-interest Factorization-headsAttention for Sequential Recommendation". in ACM) introduced contrastive learning for modeling based on the attention mechanism; CoSeRec (Z.Liu, Y.Chen, J.Li, et al. 2021. "Contrastive Self-supervised Sequential Recommendation with RobustAugmentation". in SIGIR) performed data enhancement on the self-attention mechanism and adopted different enhancement methods for sequences of different lengths; MOJITO (VATran, G.Salha-Galvan, B.Sguerra, et al. 2023. "Attention Mixtures for Time-Aware Sequential Recommendation". in Proceedings of the 46th International ACMSIGIR Conference on Research and Development in Information Retrieval) first brings the item embedding and item text embedding into the attention mechanism, and then combines the result with the original embedding as short-term interest, and combines the user embedding, item sequence embedding and predicted item embedding as long-term interest, which also has good results in recommendation problems. However, the above models only consider the time sequence of user interactions, ignoring other rich time information.
[0006] In addition, there are some time-aware models based on time-aware sequence recommendation in this field, such as Time-LSTM (Y.Zhu, H.Li, Y.Liao, et al. 2017. "What to Do Next: Modeling User Behaviors by Time-LSTM". in Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, (IJCAI)) which uses the time interval between consecutive items to model the user's sequential actions based on LSTM with time gate; SLRC (C.Wang, M.Zhang, W.Ma, et al. 2019. "Modeling Item-Specific Temporal Dynamics of Repeat Consumption for Recommender Systems". in The World Wide Web Conference) which combines Hawkes process and collaborative filtering to model the temporal dynamics of repeated consumption; CTA (J.Wu, R.Cai and H.Wang. 2020. "Déjàvu: A Contextualized Temporal Attention Mechanism for Sequential Recommendation". in Proceedings of The Web Conference 2020) uses multiple time kernel functions to measure the impact of historical operations; TiSASRec (J.Li, Y.Wang and J.McAuley.2020. "Time Interval Aware Self-Attention for Sequential Recommendation".in Proceedings of the 13th International Conference on Web Search and Data Mining (ICDM)) uses the time intervals between different items of the user, the location information of the items, and the item embedding information to model the user's sequential actions through a self-attention layer. Although these models have made some progress recently, the methods used by these models do not take into account the methods used by the present invention for timestamps. [Summary of the invention]
[0007] The purpose of the present invention is to solve the problems in the prior art and propose a temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism, which can model the relative time intervals between items, the popularity information of items, user ratings, the absolute timestamps of items, and the position information of items in the sequence, and outperforms the most advanced algorithms on both dense and sparse data sets.
[0008] To achieve the above object, the present invention is implemented through the following technical solutions:
[0009] A temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism includes a CATSASRec layer, a prediction layer and a contrastive learning layer. The CATSASRec layer is composed of a data preprocessing layer, an embedding layer and a self-attention layer arranged in sequence. After the data preprocessing layer completes data preprocessing, the embedding layer can construct an effective embedding representation of items and users and calculate attention weights according to the time interval between items, the location information of the items, the user's rating of the items and the popularity information of the items. The prediction layer and the contrastive learning layer are respectively connected to the CATSASRec layer.
[0010] Preferably, during data preprocessing, the data preprocessing layer sets U and I to represent the user set and the item set, respectively, and gives the user's action sequence for each user u∈U in the time-aware sequential recommendation setting. Absolute time series User rating sequence And the item heat sequence and Absolute time series User rating sequence And the item heat sequence Corresponding to the user's action sequence During training, the model calculates the time interval between items i and j based on the previous t items. The absolute timestamp t of item i and item j i ,t j , item popularity information h of item i and item j i ,h j and the user's rating r for item i and item j i ,r j To predict the next item.
[0011] Furthermore, for the user's action sequence For example, the training sequence of each user is converted into a fixed-length sequence s = (s1, s2, ..., s n), and n represents the maximum fixed length. If the sequence is greater than n, only the most recent n actions are considered. If the sequence is less than n, padding items 0 are added to the left until the length is n. For the absolute time series User rating sequence And the item heat sequence For example, we set a fixed length time series t = (t1, t2, ..., t n ), user rating sequence r = (r1, r2, ..., r n ) and item heat sequence h=(h1,h2,...,h n ), and the user rating sequence And the item heat sequence The fill items are all 0, and the absolute time series Adopting the user's action sequence The timestamp of the first item in the array is used as a filler item.
[0012] Furthermore, given the fixed length time series t=(t1, t2, ..., t n ), the time interval between the i-th item and the j-th item is |t i -t j |, the time interval set of user u is G under all conditions except 0 u , the minimum time interval for users is The scaled time interval is and The relationship matrix M of user u u ∈N n×n As shown in formula (2):
[0013]
[0014] Preferably, the embedding layer may create an embedding matrix M for each item I ∈R |I|×d , and d is the hidden dimension, a constant zero vector 0 is used as the embedding of the padding item, and the embedding layer can embed the embedding of the first n items and stack them together to obtain the matrix E shown in formula (3) I ∈R n×d :
[0015]
[0016] The keys and values in the self-attention mechanism contain various elements as shown in formula (4), formula (5), formula (6), formula (7) and formula (8):
[0017] P=W(E Abs +E P ) (4);
[0018]
[0019] in, is the position information in the sequence, is the absolute time of the item, W is a learnable parameter, P is the new position information embedding, and are positional embeddings in keys and values, respectively; and The two user rating matrices are respectively passed through the embedding layer to obtain the user rating embedding matrix; and The two item heat matrices are respectively passed through the embedding layer to obtain the item heat embedding matrix; and They are two relative time interval embedding matrices. The embedded matrices obtained after passing through the embedding layer are symmetric matrices and all elements on the main diagonal are zero.
[0020] Preferably, for each input sequence In terms of When the self-attention layer mixes perception and can calculate the new sequence Z = (z1, z2, ..., z n ), Each output element z i Formula (9) is used to calculate the weighted sum of the linearly transformed input elements, relative time relationship embedding, location information embedding, user rating embedding, and item heat embedding:
[0021]
[0022] Among them, W V ∈R d×d is the input projection of the value, is the time interval between item i and item j, is the rating of item j, is the location information of item j, is the item heat information of item j, each coefficient a ij All are calculated using the Softmax function through formula (10):
[0023]
[0024] Among them, e ij It is calculated using the compatible function shown in formula (11) that considers input, relative time relationship, item location information, user rating information, and item popularity information:
[0025]
[0026] Among them, W Q ∈R d×d and W K ∈R d×d are the input projection weights of query and key respectively, is the scaling factor.
[0027] Furthermore, after each time-aware attention layer, a linear transformation and ReLU activation are applied as shown in formula (12) to give the model nonlinearity:
[0028] FFN(z i )=max(0,z i W1+b1)W2+b2 (12);
[0029] in, After the self-attention layer is superimposed with the feed-forward layer, layer normalization, residual connection and dropout regularization are performed as shown in formula (13):
[0030] Z i =z i +Dropout(FFN(LayerNorm(z i ))) (13);
[0031] Assume that x is a vector containing all features of the sample, and layer normalization is defined as formula (14):
[0032]
[0033] where ⊙ is the element-wise product, μ and σ are the mean and variance of x, respectively, and α and β are the learned scaling factor and bias term, respectively.
[0034] Preferably, the prediction layer uses a latent factor model to calculate the user's preference score for item i as shown in formula (15):
[0035]
[0036] in, is the embedding of item i, Z t is a representation of the first t items given (ie, s1, s2, ..., s t ) with their time interval, position, score and heat information; the prediction layer uses negative sampling to optimize the ranking of items, for each expected positive output o i , extract a negative term Generate a pairwise preference order D = {(S u ,T u ,o,o')}, through the sigmoid function σ(x) = 1 / (1+e-x ) converts the model output score to the range (0,1), and then uses binary cross entropy as the loss function as shown in formula (16):
[0037]
[0038] Preferably, the contrastive learning layer makes the user interaction sequence go through the self-attention layer twice and the mixed representation obtained in the first pass is The mixed representation obtained in the second pass is Let the two obtained representations constitute a "positive pair", randomly select N mixed embeddings of users other than the user as negative samples and represent them as The contrast loss L is calculated using formula (17) con :
[0039]
[0040] Where τ is a hyperparameter, D is the training dataset, sim(.,.) is the similarity function, and is calculated using cosine similarity as shown in formula (18):
[0041]
[0042] Furthermore, the prediction loss L pred and contrast loss L con Combined with formula (17), the loss function is further defined as formula (19):
[0043]
[0044] in, is the set of embedding matrices, ||.|| F is the Frobenius norm.
[0045] Based on the TiSASRec model, this paper proposes a hybrid-aware self-attention model for sequential recommendation (Contrast Enhanced Time Aware Self-Attention for Sequential Recommendation, CATSASRec), and aims to effectively utilize item timestamps. The CATSASRec model can model the relative time intervals between items, item popularity information, user ratings, item absolute timestamps, and item position information in the sequence, thereby predicting future interactions. Experiments based on multiple different types of data sets have shown that the CATSASRec model is superior to the current advanced sequence models and still performs well on sparse data sets.
[0046] In general, the beneficial effects of the present invention are as follows:
[0047] 1) The present invention combines the absolute timestamp with the traditional location information to form a new location information, which can make the object representation more comprehensive;
[0048] 2) The present invention can effectively enhance the personalized representation of users by integrating rating information and item popularity into the user embedded representation;
[0049] 3) The present invention optimizes the distance between different representations of the same user by introducing a contrastive learning method.
[0050] The features and advantages of the present invention will be described in detail through embodiments in conjunction with the accompanying drawings.
Brief Description of the Drawings
[0051] Figure 1 It is a structural schematic diagram of a temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism of the present invention;
[0052] Figure 2 It is a structural schematic diagram of the self-attention layer of the temporal recommendation model based on the contrast-enhanced time-aware self-attention mechanism of the present invention;
[0053] Figure 3 is a comparison chart of learning rate parameters in an embodiment;
[0054] Figure 4 is a comparison diagram of negative pairs of contrastive learning in an embodiment;
[0055] Figure 5 is a comparison diagram of the regularization parameter λ of the embodiment;
[0056] Figure 6 β is a comparison chart of the regularization parameter β of the embodiment. [Specific implementation method]
[0057] In this embodiment, we first formulate the next item recommendation problem, and then describe our method of obtaining absolute time, obtaining item heat information, and fusing absolute time and serial position information, as well as the specific structure of the temporal recommendation model (referred to as the CATSASRec model) based on the contrast-enhanced time-aware self-attention mechanism (including personalized item heat information collection, embedding layer, time-aware and new position information self-attention block and prediction layer). The present invention embeds items and their new position information, relative time intervals, item heat information, and user rating information. By calculating the attention weights through these embeddings, the CATSASRec model will give different position information and timestamp modules to different predicted items, even if several previous items are the same.
[0058] To learn the parameters, the present invention adopts binary cross entropy plus BPR loss as the objective function. The goal of the CATSASRec model is to capture the sequence pattern and explore the influence of absolute time combined with sequence position plus relative time interval on sequence recommendation, which is an unexplored area.
[0059] As we all know, since the application of individual attributes in the model may cause noise, most models will not introduce item heat information (because item heat information will cause too much noise compared to other attributes). Since the present invention utilizes the user's individual attribute scoring information and proposes new location information, the model itself already has noise. On this basis, the present invention balances the generation of noise by introducing the individual attributes of the item (item heat information). According to experiments, the overall effect of the model after introducing item heat information is improved compared to not introducing item heat information (see the subsequent ablation experiment results for detailed effects).
[0060] See also Figure 1 The present invention is based on a temporal recommendation model of contrast-enhanced time-aware self-attention mechanism, including a CATSASRec layer, a prediction layer and a contrastive learning layer. The CATSASRec layer is composed of a data preprocessing layer, an embedding layer and a self-attention layer arranged in sequence. After the data preprocessing layer completes data preprocessing, the embedding layer can construct an effective embedding representation of items and users and calculate the attention weight according to the time interval between items, the location information of items, the user's rating of items and the popularity information of items. The prediction layer and the contrastive learning layer are connected to the CATSASRec layer respectively. The specific structure and working process of the CATSASRec model are as follows:
[0061] 1) Data preprocessing layer:
[0062] Let U and I denote the set of users and the set of items, respectively. In the time-aware sequential recommendation setting, for each user u∈U, the user’s action sequence is given as Absolute time series User rating sequence And the item heat sequence and Absolute time series User rating sequence And the item heat sequence Corresponding to the user's action sequence During training, the model calculates the time interval between items i and j based on the previous t items. The absolute timestamp t of item i and item j i ,t j , item popularity information h of item i and item j i ,hj and the user's rating r for item i and item j i ,r j To predict the next item.
[0063] Since each user training sequence The sequence lengths of the users are all inconsistent, so the present invention converts the training sequence of each user into a fixed-length sequence s=(s1, s2, ..., s n ). Where n represents the maximum fixed length. At this time, if the sequence is greater than n, we only consider the most recent n actions; if it is less than n, we add padding items 0 to the left until the length is n. Similarly, for absolute time series User rating sequence And the item heat sequence For example, the present invention sets a fixed length time series t=(t1, t2, ..., t n ), user rating sequence r = (r1, r2, ..., r n ) and item heat sequence h=(h1,h2,...,h n ). Among them, the item heat sequence Sequence with user ratings The fill items are all 0, and the absolute time series The timestamp of the first item in the user interaction sequence is used as the filler item. With this design, the value 0 will not be used as a divisor during the calculation process and the minimum timestamp in each time series can be guaranteed to remain unchanged. In this way, even if there are many short sequences, the short sequences will not be too similar due to too many filler items. The present invention models the time interval between two items in the user interaction sequence as the relationship between the two items. This is because the interaction sequence of each user is different, and some users are very frequent while others are less frequent, so we only care about the relative length of the time interval between items in each user sequence. For the time interval between items of each user, the present invention chooses to divide by the shortest time interval in the user sequence. Specifically, given a fixed-length time series t=(t1, t2, ..., t n ), then the time interval between the i-th item and the j-th item is |t i -t j |, the time interval set of user u (except 0) is G u , the minimum time interval for users is Zoomed time interval Then the relationship matrix M of user u is u ∈N n×n As shown in formula (2):
[0064]
[0065] The present invention believes that if the maximum relative time interval between two items is too large compared to other values, it will affect the accuracy of embedding other time intervals, and too large a value will not only waste a lot of time in calculation, but also cause negative effects on learnable parameters. For this reason, the present invention chooses to compress the excessively large time interval to k. Clipping the maximum interval also avoids sparse relational coding and enables the model to generalize to time intervals not seen in training. The clipping matrix is The shear operation in applies to each element In addition, the absolute time series is a time series of fixed length t=(t1,t2,...,t n )go through Obtained through simple processing; item heat sequence H u It is obtained by counting the number of occurrences of all items in the statistical data set; the fixed-length item heat sequence h = (h1, h2, ..., h n ) is the interaction sequence s=(s1,s2,...,s n ) is the number of occurrences of the corresponding item in the dataset.
[0066] 2) Embedding layer:
[0067] The embedding layer creates an embedding matrix M for the items I ∈R |I|×d , and d is the hidden dimension, and the constant zero vector 0 is used as the embedding of the padding term. The embedding lookup operation retrieves the embeddings of the first n terms and stacks them together to obtain a matrix E as shown in formula (3) I ∈R n×d :
[0068]
[0069] In addition, as shown in formulas (4) to (8), the keys and values in the self-attention mechanism contain multiple elements:
[0070] P=W(E Abs +E P ) (4);
[0071]
[0072] Among them, E P ∈R n×d is the position information in the sequence, E Abs ∈R n×d is the absolute time of the item, W is a learnable parameter, P is the new position information embedding, and are positional embeddings in keys and values, respectively;
[0073]
[0074] in, and The two user rating matrices are respectively passed through the embedding layer to obtain the user rating embedding matrix;
[0075]
[0076] in, and The two item heat matrices are respectively passed through the embedding layer to obtain the item heat embedding matrix;
[0077]
[0078] in, and They are two relative time interval embedding matrices obtained after passing through the embedding layer (they are symmetric matrices, and the elements on the main diagonal are all zero).
[0079] 3) Self-attention layer:
[0080] See also Figure 2 ,Inspired by the relative position self-attention mechanism, the present invention proposes an extension of the self-attention mechanism. The present invention takes into account that a user interaction sequence may have many instances with the same timestamp, and the model becomes self-attention without any position or relationship information. Therefore, the present invention not only considers the different time intervals between two items in the sequence, but also uses the position of the elements in the sequence, the absolute time of the items, the user's rating data on the items, and the popularity of the items for comprehensive modeling.
[0081] 3.1) Hybrid Perception Self-Attention Layer:
[0082] For each input sequence In terms of When , a new sequence Z=(z1,z2,...,z n ), and z i ∈R d Each output element z i , calculated by formula (9) as the weighted sum of the linearly transformed input elements, relative time relationship embedding, location information embedding, user rating embedding, and item heat embedding:
[0083]
[0084] Among them, W V ∈R d×d is the input projection of the value, is the time interval between item i and item j, is the rating of item j, is the location information of item j, is the item heat information of item j, each coefficient a ij All are calculated using the Softmax function through formula (10):
[0085]
[0086] Among them, e ij It is calculated using the compatible function shown in formula (11) that considers input, relative time relationship, item location information, user rating information, and item popularity information:
[0087]
[0088] Among them, W Q ∈R d×d and W K ∈R d×d are the input projection weights for query and key, respectively, and the scaling factor Used to avoid large inner product values (especially at large sizes).
[0089] 3.2) Point-wise feedforward network:
[0090] Although the hybrid perceptual attention layer of the present invention can combine all previous items, positions, ratings, heat and relative time information with adaptive weights, they are implemented through linear combinations. To this end, the present invention chooses to apply two linear transformations and a ReLU activation in the middle after each time-perceptual attention layer to give the model nonlinearity as shown in formula (12):
[0091] FFN(z i )=max(0,z i W1+b1)W2+b2 (12);
[0092] Where W1,W2∈R d×d ,b1,b2∈R d Although the linear transformations between different projects are the same, they use different parameters in each layer.
[0093] When the self-attention layer and the feedforward layer are superimposed, problems such as overfitting, unstable training process (such as gradient disappearance) and more training time will occur (see W.-C.Kang and J.McAuley.2018. "Self-Attentive Sequential Recommendation".in Proceeding of the 18th IEEE International Conference on Data Mining (ICDM)). To this end, the present invention chooses to use layer normalization, residual connection and dropout regularization technology as shown in formula (13) to solve these problems:
[0094] Z i =z i +Dropout(FFN(LayerNorm(z i ))) (13).
[0095] Layer normalization is used to normalize the input across features (i.e., zero mean and unit variance). This can be used to stabilize and accelerate the training of neural networks. Assuming x is a vector containing all the features of a sample, layer normalization is defined as follows:
[0096]
[0097] where ⊙ is the element-wise product, μ and σ are the mean and variance of x, respectively, and α and β are the learned scaling factor and bias term, respectively.
[0098] 4) Prediction layer:
[0099] After superimposing the self-attention blocks, the present invention obtains a combined representation of the item, position, score, popularity and relative time. In order to predict the next item, the present invention uses the latent factor model to calculate the user's preference score for item i as shown in formula (15):
[0100]
[0101] in, is the embedding of item i, Z t is a representation of the first t items given (ie, s1, s2, ..., s t ) with their time interval, location, rating, and popularity information.
[0102] Since user interactions are implicit data, we cannot directly optimize the preference score R i,t , and the goal of the model CATSASRec is to provide a ranked list of items, so the present invention uses negative sampling to optimize the ranking of items. For each expected positive output o i, the present invention extracts a negative term Generate a pairwise preference order D = {(S u ,T u ,o,o')}. The present invention uses the sigmoid function σ(x)=1 / (1+e -x ) converts the model output score to the range (0,1), and uses binary cross entropy as the loss function as shown in formula (16):
[0103]
[0104] 5) Contrastive learning layer:
[0105] In the present invention, Z t , because there are many mixed elements, even if the two users are not similar, it is possible that multiple elements are mixed to get Z t To solve this problem, the most intuitive idea is to match the user with other similar Z t Therefore, the present invention chooses to use contrastive learning to solve the problem. Specifically, the present invention refers to SimCGNN, by making the user interaction sequence go through the self-attention layer twice and the mixed representation obtained in the first pass is The mixed representation obtained in the second pass is The two representations are made into a "positive pair". In addition, the present invention randomly selects N mixed embeddings of users other than itself as negative samples (the representation of negative samples is ).like Figure 1 As shown, the solid double arrows are positive pairs, while the dashed double arrows are negative pairs. con It can be calculated by formula (17):
[0106]
[0107] Where τ is a hyperparameter, D is the training dataset, sim(.,.) is the similarity function, and is calculated using cosine similarity as shown in formula (18):
[0108]
[0109] Because the data from the same training batch is not comprehensive, the comparison effect is not good if only the data from the same batch is used. Therefore, we need to maintain a mixed representation database and store the "positive pairs" of each user in the memory library. When negative samples are needed, the mixed representations of other users are directly taken from the library.
[0110] 6) Model optimization:
[0111] The present invention defines the prediction loss L predand contrast loss L con , and as shown in formula (19), the two are combined to define the final loss function:
[0112]
[0113] in, is the set of embedding matrices, ||.|| F is the Frobenius norm.
[0114] 7) Experimental results and analysis:
[0115] 7.1) Dataset:
[0116] This experiment uses multiple datasets of different types, sizes, and sparsities from the Amazon platform to evaluate the CATSASRec model. These datasets all contain ratings and timestamps. This experiment treats the ratings in the dataset as implicit feedback and sorts the items according to the timestamp. Each element in the user is subtracted from the minimum timestamp in the sequence to ensure that the timestamp starts from zero. This experiment filters out cold start users and users with less than 5 interactions. This experiment uses the most recent item for testing, the second most recent item for validation, and the remaining items for training. The dataset statistics are shown in Table 1 below, and Movies_and_TV is the densest dataset (with the most average interactions and total interactions), while Gift has the least user interactions.
[0117] Table 1 Dataset statistics
[0118]
[0119] 7.2) Evaluation indicators:
[0120] This experiment uses two common Top-N indicators, Hit Rate@10 and NDCG@10, to evaluate recommendation performance. Hit@10 counts the hit rate of the top 10 items; NDCG@10 counts the top 10 items and sums up the calculated scores of each position. For each user u, this experiment randomly selects 100 negative items and sorts them with the ground-truth items. This experiment calculates Hit@10 and NDCG@10 based on the rankings of these 101 items.
[0121] 7.3) Comparison method:
[0122] This utility compares the model CATSASRec with the following models:
[0123] ① Model GRU4Rec, which uses GRU to model item sequences and implement session-based recommendations;
[0124] ② Model SASRec, all items are sorted according to timestamps and modeled using the self-attention mechanism;
[0125] ③ Model TiSASRec, a recommendation algorithm based on the self-attention mechanism, which incorporates the time interval between items in the user sequence into the attention layer;
[0126] ④ Model HTP, based on the self-attention mechanism and making full use of various time information for sequential recommendation, such as absolute time information, relative item time intervals and other time modes.
[0127] 7.4) Experimental results:
[0128] The experimental results of performance comparison between different models for prediction tasks are shown in Table 2. The bold values represent the best performance among all models, and the underlined values represent the best performance except for the best performance.
[0129] Table 2 Comparative experimental results
[0130]
[0131] From the experimental results, it can be seen that the HTP model outperforms the SASRec model and the TiSASRec model on many data sets (this shows that the self-attention mechanism can effectively improve the recommendation performance by making full use of various time information of sequential recommendation; the TiSASRec model has more disadvantages than advantages compared with the SASRec and HTP models (this shows that the self-attention mechanism, which only uses the relative time intervals of items, has good performance on individual data sets, but the overall performance is still poor); the SASRec model is significantly improved compared with the GRU4Rec model (this shows that the self-attention mechanism can effectively improve the recommendation performance).
[0132] 7.5) Ablation experiment:
[0133] This model has many components. We use ablation experiments to explore the impact of each component on the model. Among them, CATSASRec: The position information of the item is a combination of the absolute time of the item and the position information of the item in the sequence, and then combined with the heat information of the item, the time interval between items, the user's rating information, and contrastive learning for modeling; Only_position: The position information of the item only considers the position of the item in the sequence, and the rest remains unchanged; Only_AbsTime: The position information of the item only considers the absolute time of the item, and the rest remains unchanged; No_Contrast: Abandon the contrast learning module, and the rest remains unchanged; No_Hot: Abandon the heat information of the item, and the rest remains unchanged. The experimental results are shown in Table 3 below.
[0134] Table 3 Ablation experiment results
[0135]
[0136] As shown in Table 3 above, compared with location information and absolute time information, heat information and contrastive learning have good effects, and both have advantages and disadvantages, which shows that the combination of the two can more effectively improve the recommendation effect; contrastive learning is better than heat information.
[0137] 7.6) Sensitivity test:
[0138] The model parameter sensitivity test mainly includes the learning rate, the number of negative samples for contrast learning, the regularization parameter λ and the regularization parameter β. Figure 3 , Figure 4 , Figure 5 and Figure 6 It can be seen that the CATSASRec model can achieve the best effect when the learning rate is 0.0015, the number of negative samples for contrastive learning is 2, the regularization parameter λ is 0.00015, and the regularization parameter β is 0.0001.
[0139] The above embodiments are intended to illustrate the present invention, not to limit the present invention. Any solution that is a simple transformation of the present invention belongs to the protection scope of the present invention.
Claims
1. A temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism, Features: The method comprises a CATSASRec layer, a prediction layer and a contrastive learning layer. The CATSASRec layer is composed of a data preprocessing layer, an embedding layer and a self-attention layer which are arranged in sequence. After the data preprocessing layer completes data preprocessing, the embedding layer can construct an effective embedding representation of items and users and calculate the attention weight according to the time interval between items, the location information of items, the user's rating of items and the popularity information of items. The prediction layer and the contrastive learning layer are respectively connected to the CATSASRec layer.
2. The temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism as claimed in claim 1, Features: During data preprocessing, the data preprocessing layer sets U and I to represent the user set and item set respectively, and gives the user's action sequence for each user u∈U in the time-aware sequential recommendation setting Absolute time series User rating sequence And the item heat sequence and Absolute time series User rating sequence And the item heat sequence Corresponding to the user's action sequence During training, the model calculates the time interval between items i and j based on the previous t items. The absolute timestamp t of item i and item j i ,t j , item popularity information h of item i and item j i ,h j and the user's rating r for item i and item j i ,r j To predict the next item.
3. The temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism as claimed in claim 2, Features: For the user's action sequence For example, the training sequence of each user is converted into a fixed-length sequence s = (s 1 ,s 2 ,...,s n ), and n represents the maximum fixed length. If the sequence is greater than n, only the most recent n actions are considered. If the sequence is less than n, padding items 0 are added to the left until the length is n. For the absolute time series User rating sequence And the item heat sequence For each time series, we set a fixed length t = (t 1 ,t 2 ,...,t n ), user rating sequence r = (r 1 ,r 2 ,...,r n ) and item heat sequence h=(h 1 ,h 2 ,...,h n ), and the user rating sequence And the item heat sequence The fill items are all 0, and the absolute time series Adopting the user's action sequence The timestamp of the first item in the array is used as a filler item.
4. The temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism as claimed in claim 3, Features: Given the fixed length time series t=(t 1 ,t 2 ,...,t n ), the time interval between the i-th item and the j-th item is |t i -t j |, the time interval set of user u is G under all conditions except 0 u , the minimum time interval for users is The scaled time interval is and The relationship matrix M of user u u ∈N n×n As shown in formula (2):
5. The temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism as claimed in claim 1, Features: The embedding layer creates an embedding matrix M for the item I ∈R |I|×d , and d is the hidden dimension, a constant zero vector 0 is used as the embedding of the padding term, and the embedding layer can embed the embedding of the first n terms and stack them together to obtain the matrix E shown in formula (3) I ∈R n×d : The keys and values in the self-attention mechanism contain various elements as shown in formula (4), formula (5), formula (6), formula (7) and formula (8): P=W(E Abs +E P ) (4); in, is the position information in the sequence, is the absolute time of the item, W is a learnable parameter, P is the new position information embedding, and are the positional embeddings in keys and values, respectively; and The two user rating matrices are respectively passed through the embedding layer to obtain the user rating embedding matrix; and The two item heat matrices are respectively passed through the embedding layer to obtain the item heat embedding matrix; and They are two relative time interval embedding matrices. The embedded matrices obtained after passing through the embedding layer are symmetric matrices and all elements on the main diagonal are zero.
6. The temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism as claimed in claim 1, Features: For each input sequence In terms of When the self-attention layer mixes perception and can calculate the new sequence Z = (z 1 ,z 2 ,...,z n ), Each output element z i Formula (9) is used to calculate the weighted sum of the linearly transformed input elements, relative time relationship embedding, location information embedding, user rating embedding, and item heat embedding: Among them, W V ∈R d×d is the input projection of the value, is the time interval between item i and item j, is the rating of item j, is the location information of item j, is the item heat information of item j, each coefficient a ij All are calculated using the Softmax function through formula (10): Among them, e ij It is calculated using the compatible function shown in formula (11) that considers input, relative time relationship, item location information, user rating information, and item popularity information: Among them, W Q ∈R d×d and W K ∈R d×d are the input projection weights of query and key respectively, is the scaling factor.
7. The temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism as claimed in claim 6, Features: After each time-aware attention layer, a linear transformation and ReLU activation are applied as shown in Equation (12) to give the model nonlinearity: FFN(z i )=max(0,z i W 1 +b 1 )W 2 +b 2 (12); in, After the self-attention layer is superimposed with the feed-forward layer, layer normalization, residual connection and dropout regularization are performed as shown in formula (13): Z i =z i +Dropout(FFN(LayerNorm(z i ))) (13); Assume that x is a vector containing all features of the sample, and layer normalization is defined as formula (14): where ⊙ is the element-wise product, μ and σ are the mean and variance of x, respectively, and α and β are the learned scaling factor and bias term, respectively.
8. The temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism as claimed in claim 1, Features: The prediction layer uses the latent factor model to calculate the user's preference score for item i as shown in formula (15): in, is the embedding of item i, Z t is a representation of the first t items given (ie, s 1 ,s 2 ,...,s t ) with their time interval, position, score and heat information; the prediction layer uses negative sampling to optimize the ranking of items, for each expected positive output o i , extract a negative term Generate a pairwise preference order D = {(S u ,T u ,o,o')}, through the sigmoid function σ(x) = 1 / (1+e -x ) converts the model output score to the range (0,1), and then uses binary cross entropy as the loss function as shown in formula (16):
9. The temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism as claimed in claim 1, Features: The contrastive learning layer makes the user interaction sequence go through the self-attention layer twice and the mixed representation obtained in the first pass is The second mixed expression is Let the two obtained representations constitute a "positive pair", randomly select N mixed embeddings of users other than the user as negative samples and represent them as The contrast loss L is calculated using formula (17) con : Where τ is a hyperparameter, D is the training dataset, sim(.,.) is the similarity function, and is calculated using cosine similarity as shown in formula (18):
10. The temporal recommendation model based on contrast-enhanced time-aware self-attention mechanism as claimed in claim 9, Features: Let the prediction loss L pred and contrast loss L con Combined with formula (17), the loss function is further defined as formula (19): in, is the set of embedding matrices, ||.|| F is the Frobenius norm.
