Sequence recommendation method based on attention mechanism and contrast learning

Abstract

The invention discloses a sequence recommendation method based on an attention mechanism and comparative learning. The method comprises the following steps: pre-building and training a sequence recommendation model; and obtaining an interaction sequence of the user and the item, an interaction sequence of the user and the item provider, and a time point sequence, and inputting the interaction sequences into the trained sequence recommendation model to predict a next item that the user may interact with to obtain a recommendation result. The behaviors of the user at different time intervals are meticulously analyzed by introducing a time factor, so that the interest change of the user can be more accurately captured; in addition, contrast learning is introduced, so that the sensitivity and accuracy of the system can be further improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and recommendation systems, and in particular to a sequence recommendation method based on attention mechanism and contrastive learning. Background Art

[0002] Most of the recommendation systems in the prior art use a static approach to model the interaction between users and items, which means that they can only capture the general preferences of users. In reality, there are many online video platforms and music platforms where users watch various types of items. Users' viewing behavior changes over time and may be affected by the most recently watched content. The sequential recommendation system can provide items that are more in line with the user's interest preferences based on the user's viewing history and time series characteristics. At present, the recommendation system technology based on deep learning has made great progress, including neural collaborative filtering, graph convolutional neural networks, recurrent neural networks, etc. These models and structures have performed well in processing massive data, user dynamic preferences and complex interactions, and have promoted the progress and development of recommendation systems. However, in the process of modeling user behavior information, the above methods do not fully utilize auxiliary information and cannot respond to sudden changes in user preference behavior in a timely manner. In the field of sequential recommendation, how to ensure that users can see the items they like on the streaming platform instead of spending time looking for their favorite items has become an urgent problem to be solved. Therefore, it is necessary to design a sequential recommendation method that can respond to the sudden changes in user behavior preferences while further improving the accuracy of recommendations.

[0003] In today's era of information explosion, although people are constantly receiving various hot information, many of them may not be in line with the user's interests. For this reason, the recommendation system came into being, which analyzes the user's historical interaction data and accurately recommends content that suits personal preferences. At present, from video streaming services to music platforms, the application of recommendation systems has spread to many commonly used websites and applications. Especially in recent years, deep learning technologies such as GRU, LSTM, Transformer, and recurrent neural networks have been widely used in recommendation systems, significantly improving the effect of recommendations. However, these technologies have not yet been able to effectively cope with the rapid changes in user behavior preferences. For example, when users switch from watching a large number of game live broadcasts to cooking videos, the recommendation system often still recommends game content based on old viewing records and fails to adjust to the user's new interests in a timely manner.

[0004] To address this problem, traditional recommendation algorithms such as those based on Markov chains (MC) or recurrent neural networks (RNN) have their own strengths. MC assumes that users' future behavior is only related to the most recent few activities, which is suitable for situations where data is sparse; while RNN, through its complex structure, can better capture the long-term dependencies of user behavior when data is abundant. The latest SASRec model incorporates a self-attention mechanism, which can not only capture long-term behavioral dependencies, but also make accurate predictions based on a small amount of historical data, greatly improving the model's operational efficiency and its ability to parse user behavior sequences. In addition, the Transformer model based on multi-head attention significantly improves the training and inference speed through parallel computing, and optimizes the representation of the dependency relationship between users and items.

[0005] Although recommendation systems have made some progress in adapting to changes in user interests, they still face the challenge of responding quickly to dynamic changes in user interests. User interests may change rapidly due to emergencies or emerging trends, and traditional recommendation systems often fail to adapt in time because they ignore the time factor. By carefully analyzing user behavior at different time intervals, recommendation systems can more accurately capture changes in user interests. For example, the system can distinguish between viewing preferences on weekdays and weekends to provide more personalized content recommendations. Furthermore, in-depth analysis of time intervals is crucial to revealing the dynamic changes in user interests. For example, a user may frequently watch a certain type of video in a short period of time, but may then turn to completely different content. If the recommendation system can identify this pattern change in time, it can adjust the recommendation strategy more quickly to match the user's current interests. However, the recommendation methods of recommendation systems in the prior art often ignore the time factor and cannot accurately capture changes in user interests, and thus cannot accurately make recommendations. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a sequence recommendation method based on attention mechanism and contrastive learning, which improves the accuracy of user preference evolution and can recommend items that are more in line with user interest preferences.

[0007] By introducing the time factor to analyze the user's behavior at different time intervals in detail, the changes in user interests can be captured more accurately; in addition, the introduction of contrastive learning can further improve the sensitivity and accuracy of the system. By comparing the user's behavior at different times, this method can not only identify the changes in user interests, but also accurately compare the differences between the user's past and present behaviors. This analysis is not just a simple distinction between positive and negative samples, but involves deep time series analysis, which enables the model to sensitively respond to subtle changes in user interests, thereby providing more accurate content recommendations.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is:

[0009] A sequence recommendation method based on attention mechanism and contrastive learning, comprising: pre-building and training a sequence recommendation model;

[0010] The interaction sequence between users and items, the interaction sequence between users and item providers, and the time point sequence are obtained and input into the trained sequence recommendation model to predict the next item that the user may interact with and obtain the recommendation result.

[0011] The interaction sequence between users and items, the interaction sequence between users and item providers, and the time point sequence are input into the embedding layer of the sequential recommendation model, which can effectively extract the dense feature representation of items and item providers. Subsequently, the model accurately learns the user's behavior and preferences at different times through an attention layer with time bias and an attention layer with frequency information. Specifically, the attention layer with time bias dynamically adjusts the weights through the attention bias with time interval information and location information, while the attention layer with frequency information captures the periodic pattern of user behavior through time delay aggregation, thereby obtaining deep item feature representation and item provider feature representation. The deep item feature representation and item provider feature representation are then input into the feedforward neural network layer to obtain the final item feature representation and item provider feature representation. The contrastive learning module uses noise contrast estimation loss to optimize the item feature representation and item provider feature representation, and uses the semantic information between sequences as a supervisory signal to further improve the representation quality. The prediction score is calculated by the inner product of the final item feature representation and the item embedding vector, and the prediction score is converted into prediction probability using the softmax function to obtain the prediction probability of the candidate items. The prediction probabilities of the candidate items are sorted to generate the final recommendation list.

[0012] In the contrastive learning part, the model uses noise contrast estimation loss to optimize the item feature representation and item provider feature representation, and uses the semantic information between sequences as a supervisory signal to further improve the representation quality. Finally, by calculating the inner product of the item feature representation and the item embedding vector, the model can efficiently capture the relevance of candidate items, thereby generating an accurate user preference recommendation list.

[0013] An attention layer with time bias is used to learn users' behavioral preferences at different time intervals to adjust the attention weights in the recommendation model, so as to more accurately predict users' preferences and recommend items that they are more interested in. The attention layer with time bias dynamically adjusts the weights through the attention bias with time interval and location information, while the attention layer with frequency information captures the periodic patterns of user behavior through time delay aggregation.

[0014] The item sequence representation and the item provider sequence representation are cross-input into the attention layer with time bias and the attention layer with frequency information to obtain deep item feature representation and item provider feature representation.

[0015] The deep item feature representation and item provider feature representation are input into the feedforward neural network layer to obtain the final item feature representation and item provider feature representation. The prediction score is calculated by the inner product of the final item feature representation and the item embedding vector, and the softmax function is used to convert the prediction score into a prediction probability to obtain the prediction probability of the candidate item. The prediction probabilities of the candidate items are sorted to generate the final recommendation list.

[0016] Use one-hot to encode the interaction sequence between the input user and the item, the interaction sequence between the user and the item provider, and the time point sequence to obtain each item. The embedding vector representation of each project provider is Embedded vector representation of time interval information related to items Embedded vector representation of time interval information related to item providers Embedded vector representation of location information associated with items Embedded vector representation of location information associated with item providers

[0017]

[0018] in, Where t1 represents the t1th item embedding vector representation in the comprehensive item embedding vector sequence. Where t2 represents the t2th item provider embedding vector representation in the comprehensive item provider embedding vector sequence. Where t1 represents the embedding vector representation of the t1th item in the initial item embedding vector sequence. Where t2 represents the t2th item provider embedding vector representation in the initial item provider embedding vector sequence. Where t1 represents the position information related to the project. The t1th position embedding vector in the embedding vector sequence represents, Where t2 represents the t2th position embedding vector representation in the position information embedding vector sequence related to the project provider.

[0019] We use a time series with a fixed length of n. When the time series length is less than n, it is padded with the earliest time point; if the time series is greater than or equal to n, only the first n time points are retained. In addition, we model the time intervals in the interaction sequence as the temporal relationship between items. The time point sequence T of the first e interactions of user u ue =[γu,1 ,γ u,2 ,γ u,3 ,γ u,I …γ u,e ] represents the time point sequence of the uth user, u∈{1,2,3,…l}, I∈{1,2,3,…e}, e represents the length of the time point sequence, and any time point γ in each dimension of the sequence uI The first time point γ of its dimension u,1 The difference between them is (γ uI – γ u,1 ) obtain a relative time interval sequence consisting of e relative time intervals, and find the smallest relative time interval and the largest relative time interval r in the relative time interval sequence mix and r max , by calculating the difference between each time point and the initial time point, and normalizing it with an exponential decay function to obtain a relative value representing the impact of the time interval. In the normalization process, the denominator also introduces the minimum relative time interval and the maximum relative time interval in the relative time interval sequence to obtain the final time interval vector Represents the time interval vector of the uth user, u∈{1,2,3,…l},I∈{1,2,3,…e}. Specifically, the time interval vector is expressed as follows:

[0020]

[0021] It represents the rounding down operation, and λ is a decay factor that controls the speed of exponential decay.

[0022] In the attention layer with temporal bias, in order to further improve the performance of the self-attention model in capturing contextual information, we propose a novel bias adjustment mechanism based on attention weights. This mechanism is specially designed to dynamically adjust the focus of attention and its associated scope of action to adapt to the specific needs of the context. It is important to note that the evolution of user interests has obvious temporal asymmetry, which is particularly prominent in the early and late stages of user behavior. Specifically, after experiencing a certain type of content (such as a movie), a user may show extremely high interest in similar content for a short period of time, but this interest tends to gradually decay over time. Our research shows that Laplace distribution is an effective tool to describe such interest changes. Therefore, we introduce Laplace distribution bias and temporal bias in the bias strategy of the attention mechanism. To obtain the attention bias B with time interval and position information IJThis bias allows the model to take into account the temporal variation of user interests when calculating attention weights, thereby more accurately adjusting the model's response to contextual information. IJ The introduction of not only enhances the dynamic adaptability of the model, but also improves the accuracy and efficiency of the model in processing user data in different time periods. The specific calculation formula is as follows:

[0023]

[0024] Among them, J is a variable that is affected by the position information of behavior I and takes into account the bias, l is the length of the user sequence, and α is a proportional coefficient that is closely related to the distribution center, which takes a value between 0 and 1. Multiplying it by l can get the distribution center αl, which corresponds to the peak moment of user interest change; is the scale coefficient of the variance, w5 is the parameter coefficient, l is the distance variance, which controls the speed of interest decay; θ is the proportional coefficient related to the time interval, which takes a value between 0 and 1; by adjusting the parameters of the attention bias, the model can better adapt to the specific characteristics of user behavior in different application scenarios.

[0025] We design a fusion module in the corresponding Transformer, which strengthens the representation interaction between the item side and the item provider side through an attention layer with temporal bias information and an attention layer with frequency information to learn effective fusion representation. Specifically, we can get the query (Q i ), key (K i ) and value (V i ), and the query generated by the project provider (Q p ), key (K p ) and value (V p ), are obtained through their respective linear projections. How to extract a user's interest preferences from his historical activity sequence is always a complex task. This requires not only understanding the interdependence between items in a long sequence to reveal the user's long-term interests, but also capturing the changes in the user's interests in the short term.

[0026]

[0027] in, (n is the item sequence length, d is the embedding dimension) is the attention matrix used by the item-side self-attention mechanism to learn item interactions. Used to represent query parameter matrix, key parameter matrix Corresponds to Key Matrix, is the value parameter matrix It represents the value matrix. yes By calculating the query terms With each key The dot product of is used to obtain the correlation between items, which helps the model capture the association between different items. These correlations are normalized by the softmax function and converted into attention weights, and the sum of the weights is 1. These weights are then applied to the corresponding values. The sum of the two is used to generate the interaction attention matrix, which reflects the model's understanding and learning of the relationship between different item arrangements in the user-item interaction sequence.

[0028] in, (m is the length of the item provider sequence, d is the embedding dimension) is the attention matrix used by the item provider side self-attention mechanism to learn item interactions. Query matrix Used to represent query parameter matrix, key parameter matrix Corresponds to Key Matrix, is the value parameter matrix It represents the value matrix. yes By calculating the query terms With each key The dot product of is used to obtain the correlation between items, which helps the model capture the association between different items. These correlations are normalized by the softmax function and converted into attention weights, and the sum of the weights is 1. These weights are then applied to the corresponding values. The interaction attention matrix is ​​generated by summing them up. This matrix reflects the model's understanding and learning of the arrangement relationship of different item providers in the interaction sequence between users and item providers.

[0029] In the attention layer with frequency information, the traditional sequence recommendation model generally obtains the user's long-term behavior preferences based on the dependency relationship between items, which often ignores the user's periodic behavior preferences. Users may repeatedly view certain types of content or watch certain types of videos at regular intervals. In the frequency domain, the relationship between query Q and key k in the self-attention mechanism is explored through autocorrelation analysis. The method is as follows:

[0030] Ψ i =FFT -1 (FFT(Q P )⊙(FFT(k i ) * )

[0031] Ψ p =FFT -1 (FFT(Q i )⊙(FFT(kp ) * )

[0032] ⊙ represents the product of element-wise correlation, and * represents the conjugate operation. Fast Fourier Transform FFT can map the matrix from the time domain to the frequency domain. Inverse Fourier Transform FFT -1 , used to restore the results from the frequency domain to the time domain. Given a user's limited historical behavior sequence and The time delay operation is defined as On the project side and the project provider side, the embedding matrices corresponding to the time delay series are expressed as and In the frequency domain, a similar attention mechanism is performed by querying the autocorrelation of Q and key K. Since the information used to calculate the attention score comes from the same sequence, we consider it an autocorrelation operation. By querying the autocorrelation of Q and key K, we can get the most relevant time delay sequence in the frequency domain.

[0033] First, we find the past and future behavior sequences that are most relevant to the current behavior, and we use the function argTopk(Ψ i ) to find the most relevant time lag and

[0034]

[0035] argTopk(Ψ i ) function is to find the top k1 delay indices with the largest project-side autocorrelation in a set of delays. p ) function aims to find the first k2 time delay indexes with the largest autocorrelation on the project provider side in a set of time delays. The value of time delay k1 is proportional to the logarithm of the sequence length N, and г is the adjustment coefficient; the value of time delay k2 is proportional to the logarithm of the sequence length M, and ε is the adjustment coefficient.

[0036] Then, by time-delay aggregation, that is, by multiplying the most relevant sequence with its corresponding autocorrelation, we can get the deep project representation feature vector and the deep project provider representation feature vector. Normalized using the softmax function, we get the attention weights of each time-delay sequence on the project side and the project provider side. and On the project side and the project provider side, the embedding matrices corresponding to the time delay series are expressed as and

[0037]

[0038] Among them, on the project side, the most relevant time lag is The sum of the attention weights of different time delay sequences calculated by the softmax function is (n is the item sequence length, d is the embedding dimension) is the attention matrix used by the item-side attention layer with frequency information to learn item interactions.

[0039] On the project provider side, the most relevant time lag is The sum of the attention weights of different time delay sequences calculated using the softmax function is (n is the item sequence length, d is the embedding dimension) is the attention matrix used by the attention layer with frequency information on the item provider side to learn item interactions.

[0040] In the model training and optimization layer, an objective function is set for the model to optimize the model parameters, and the cross entropy loss function is selected to measure the difference between the predicted value and the true value. The formula is as follows:

[0041]

[0042] The advantages of the present invention are as follows: (1) This paper proposes a novel method to introduce time interval information into the embedding layer of the model, and the time interval vector can be obtained after processing by the embedding layer. We not only incorporate the time interval vector into the feature vector representation of the project, but also into the feature vector representation of the project provider. This improvement helps to capture the dynamic changes of user interests and thus more accurately reflect user preferences.

[0043] (2) This paper introduces an attention bias containing time interval information and position information in self-attention. This method helps the model capture sudden changes in user interests and enhances the model's adaptability and sensitivity to such changes.

[0044] (3) We propose an innovative multi-task joint optimization method that incorporates the auxiliary information of the item provider and implements positive sample enhancement through the semantic similarity of the input sequences.

[0045] (4) We conduct extensive experiments on three public datasets and the results show that our proposed sequence recommendation model outperforms existing baseline models in many aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The following is a brief description of the contents expressed in the drawings of the present invention and the symbols in the drawings:

[0047] Figure 1This is a structural principle diagram of the sequence recommendation model in the recommendation method of the present invention. DETAILED DESCRIPTION

[0048] The specific implementation of the present invention will be further explained in detail below by describing the optimal embodiment with reference to the accompanying drawings.

[0049] The sequence recommendation method based on attention mechanism and contrastive learning provided in this embodiment includes a sequence recommendation model, which inputs the interaction sequence between users and items, the interaction sequence between users and item providers, and the time point sequence into the sequence recommendation model to predict the next item that the user may interact with. The contents of the embedding layer, the attention layer with time bias, the attention layer with frequency information, the feedforward neural network layer, the contrastive learning and prediction layer included in the sequence recommendation model are as follows.

[0050] In the embedding layer, by calculating the difference between each time point and the initial time point, and normalizing it after decaying it using an exponential decay function, a relative value representing the impact of the time interval is obtained. During the normalization process, the denominator also introduces the minimum relative time interval and the maximum relative time interval in the relative time interval sequence to obtain the final time interval sequence. The corresponding embedding vector is obtained by multiplying the one-hot encoding and the embedding matrix. The multiplication of the embedding matrix is ​​the process of mapping the high-dimensional sparse one-hot encoding to a low-dimensional dense embedding vector through matrix operations. The embedding matrix (such as W1, W2, W3, X1, X2, X3, etc.) is learned by the gradient descent algorithm during the model training process. Use one-hot to encode the input user and project interaction sequence, user and project provider interaction sequence, and time point sequence to obtain each project. The embedding vector representation of each project provider is Embedded vector representation of time interval information related to items Embedded vector representation of time interval information related to item providers Embedded vector representation of location information associated with items Embedded vector representation of location information associated with item providers

[0051]

[0052] in, Where t1 represents the t1th item embedding vector representation in the comprehensive item embedding vector sequence. Where t2 represents the t2th item provider embedding vector representation in the comprehensive item provider embedding vector sequence. Where t1 represents the embedding vector representation of the t1th item in the initial item embedding vector sequence. Where t2 represents the t2th item provider embedding vector representation in the initial item provider embedding vector sequence. Where t1 represents the embedding vector of the position information related to the item. The embedding vector of the t1th position in the sequence represents, Where t2 represents the embedding vector representation of the t2th position in the embedding vector sequence of the position information related to the item provider.

[0053] We use a time series with a fixed length of n. When the time series length is less than n, it is padded with the earliest time point; if the time series is greater than or equal to n, only the first n time points are retained. In addition, we model the time intervals in the interaction sequence as the temporal relationship between items. The time point sequence T of the first e interactions of user u ue =[γ u,1 ,γ u,2 ,γ u,3 ,γ u,I …γ u,e ] represents the time point sequence of the uth user, u∈{1,2,3,…l}, I∈{1,2,3,…e}, e represents the length of the time point sequence, and any time point γ in each dimension of the sequence uI The first time point γ of its dimension u,1 The difference between them is (γ uI – γ u,1 ) Get a relative time interval sequence consisting of e relative time intervals, and find the smallest relative time interval r in the relative time interval sequence mix and the maximum relative time interval r max , by calculating the difference between each time point and the initial time point, and normalizing it with an exponential decay function to obtain a relative value representing the impact of the time interval. In the normalization process, the denominator also introduces the minimum relative time interval and the maximum relative time interval in the relative time interval sequence to obtain the final time interval vector Represents the time interval vector of the uth user, u∈{1,2,3,…l},I∈{1,2,3,…e}. Specifically, the time interval vector is expressed as follows:

[0054]

[0055] It represents the rounding down operation, and λ is a decay factor that controls the speed of exponential decay.

[0056] In the attention layer with temporal bias, in order to further improve the performance of the self-attention model in capturing contextual information, we propose a novel bias adjustment mechanism based on attention weights. This mechanism is specially designed to dynamically adjust the focus of attention and its associated scope of action to adapt to the specific needs of the context. It is important to note that the evolution of user interests has obvious temporal asymmetry, which is particularly prominent in the early and late stages of user behavior. Specifically, after experiencing a certain type of content (such as a movie), a user may show extremely high interest in similar content for a short period of time, but this interest tends to gradually decay over time. Our research shows that Laplace distribution is an effective tool to describe such interest changes. Therefore, we introduce Laplace distribution bias and temporal bias in the bias strategy of the attention mechanism. To obtain the attention bias B with time interval and position information IJ This bias allows the model to take into account the temporal variation of user interests when calculating attention weights, thereby more accurately adjusting the model's response to contextual information. IJ The introduction of not only enhances the dynamic adaptability of the model, but also improves the accuracy and efficiency of the model in processing user data in different time periods. The specific calculation formula is as follows:

[0057]

[0058] Here, the sigmoid function is a commonly used activation function, which helps to introduce nonlinearity so that the model can handle complex tasks and the output range is between 0 and 1; the tanh function performs a nonlinear transformation on the result of the linear transformation, compresses the data, and controls the value between -1 and 1. is the time interval vector, J is the biased variable affected by the location information of behavior I, l is the user sequence length, α is the proportional coefficient closely related to the distribution center, and its value is between 0 and 1. Multiplying it by l can get the distribution center αl, which usually corresponds to the peak moment of user interest change. is the scale coefficient of the variance, w5 is the learnable parameter coefficient, l is the distance variance, which controls the speed of interest decay. θ is the proportional coefficient related to the time interval, which takes a value between 0 and 1. By adjusting these parameters, we can make the model better adapt to the specific characteristics of user behavior in different application scenarios.

[0059] The representation interaction between the project side and the provider side is strengthened through the attention layer with time bias information and the attention layer with frequency information to learn effective fusion representation. Specifically, the query (Q i ), key (K i ) and value (Vi ), and the query generated by the project provider (Q p ), key (K p ) and value (V p ), are obtained through their respective linear projections. How to extract a user's interest preferences from his historical activity sequence is always a complex task. This requires not only understanding the interdependence between items in a long sequence to reveal the user's long-term interests, but also capturing the changes in the user's interests in the short term.

[0060]

[0061] in, (n is the item sequence length, d is the embedding dimension) is the attention matrix used by the item-side self-attention mechanism to learn item interactions. Used to represent query parameter matrix, key parameter matrix Corresponds to Key Matrix, is the value parameter matrix It represents the value matrix. yes By calculating the query terms With each key The dot product of is used to obtain the correlation between items, which helps the model capture the association between different items. These correlations are normalized by the softmax function and converted into attention weights, and the sum of the weights is 1. These weights are then applied to the corresponding values. The sum of the two is used to generate the interaction attention matrix, which reflects the model's understanding and learning of the relationship between different item arrangements in the user-item interaction sequence.

[0062] in, (m is the length of the item provider sequence, d is the embedding dimension) is the attention matrix used by the item provider side self-attention mechanism to learn item interactions. Query matrix Used to represent query parameter matrix, key parameter matrix Corresponds to Key Matrix, is the value parameter matrix It represents the value matrix. yes By calculating the query terms With each key The dot product of is used to obtain the correlation between items, which helps the model capture the association between different items. These correlations are normalized by the softmax function and converted into attention weights, and the sum of the weights is 1. These weights are then applied to the corresponding values. The interaction attention matrix is ​​generated by summing them up. This matrix reflects the model's understanding and learning of the arrangement relationship of different item providers in the interaction sequence between users and item providers.

[0063] In the attention layer with frequency information, the traditional sequence recommendation model generally obtains the user's long-term behavior preferences based on the dependency relationship between items, which often ignores the user's periodic behavior preferences. Users may repeatedly view certain types of content or watch certain types of videos at regular intervals. In the frequency domain, the relationship between query Q and key k in the self-attention mechanism is explored through autocorrelation analysis. The method is as follows:

[0064] Ψ i =FFT -1 (FFT(Q P )⊙(FFT(k i ) * )

[0065] Ψ p =FFT -1 (FFT(Q i )⊙(FFT(k p ) * )

[0066] ⊙ represents the product of element-wise correlation, and * represents the conjugate operation. Fast Fourier Transform FFT can map the matrix from the time domain to the frequency domain. Inverse Fourier Transform FFT -1 , used to restore the results from the frequency domain to the time domain. Given a user's limited historical behavior sequence and The time delay operation is defined as On the project side and the project provider side, the embedding matrices corresponding to the time delay series are expressed as and In the frequency domain, a similar attention mechanism is performed by querying the autocorrelation of Q and key K. Since the information used to calculate the attention score comes from the same sequence, we consider it an autocorrelation operation. By querying the autocorrelation of Q and key K, we can get the most relevant time delay sequence in the frequency domain.

[0067] First, we find the past and future behavior sequences that are most relevant to the current behavior, and we use the function argTopk(Ψ i ) to find the most relevant time lag and

[0068]

[0069] argTopk(Ψ i ) function is to find the top k1 delay indices with the largest project-side autocorrelation in a set of delays. p ) function aims to find the first k2 time delay indexes with the largest autocorrelation on the project provider side in a set of time delays. The value of time delay k1 is proportional to the logarithm of the sequence length N, and г is the adjustment coefficient; the value of time delay k2 is proportional to the logarithm of the sequence length M, and ε is the adjustment coefficient.

[0070] Then, by time-delay aggregation, that is, by multiplying the most relevant sequence with its corresponding autocorrelation, we can get the deep project representation feature vector and the deep project provider representation feature vector. We use the SoftMax function to normalize and get the attention weights of each time-delay sequence on the project side and the project provider side. and On the project side and the project provider side, the embedding matrices corresponding to the time delay series are expressed as and

[0071]

[0072] Among them, on the project side, the most relevant time lag is The sum of the attention weights of different time delay sequences calculated by the softmax function is (n is the item sequence length, d is the embedding dimension) is the attention matrix used by the item-side attention layer with frequency information to learn item interactions.

[0073] On the project provider side, the most relevant time lag is The sum of the attention weights of different time delay sequences calculated using the softmax function is (n is the item sequence length, d is the embedding dimension) is the attention matrix used by the attention layer with frequency information on the item provider side to learn item interactions.

[0074] To obtain the final item feature representation and item provider feature representation, we weighted sum the output of the attention module with time bias and the output of the attention module with frequency information.

[0075]

[0076] On the project side, using the hyperparameter β i Perform a weighted sum to combine the outputs of the two attention mechanisms. i Controls the weight distribution of the attention module with time bias information and the attention module with frequency information. iWhen β approaches 1, the model is more inclined to the feature representation output by the attention module with frequency information. i When it approaches 0, the model is more inclined to the feature representation with the output of the time bias module. The final output It is a comprehensive feature representation that combines time bias information and frequency information and has richer sequence representation capabilities.

[0077] On the project provider side, using the hyperparameter β p Perform a weighted sum to combine the outputs of the two attention mechanisms. p Controls the weight distribution of the attention module with time bias information and the attention module with frequency information. p When β approaches 1, the model prefers the feature representation output by the attention module with frequency information. p When it approaches 0, the model is more inclined to the feature representation with the output of the time bias module. The final output It is a comprehensive feature representation that combines time bias information and frequency information and has richer sequence representation capabilities.

[0078] In the feedforward neural network layer, in order to make the network have nonlinear characteristics and capture higher quality project feature representation and project provider feature representation, we use two layers of multilayer perceptrons with GELU activation function. In the feedforward neural network, each neuron in the hidden layer receives the output from the previous layer, and after linear weighted summation, it is nonlinearly transformed through the activation function, so that the model can recognize and process complex patterns and features. As a typical feedforward neural network architecture, multilayer perceptron (MLP) gradually learns the deep features of input data through the combination of multiple layers of full connection. Each layer of MLP mainly consists of two parts: linear transformation and nonlinear activation. First, the linear transformation maps the input to a new feature space, and then the linear output is nonlinearly processed through the GELU activation function to enhance the expression ability of the network. GELU can provide a smoother activation effect, especially when dealing with negative and small positive inputs. By combining the GELU activation function with MLP, the feedforward neural network not only has excellent nonlinear fitting ability, but also alleviates the gradient vanishing problem and improves the convergence speed and stability of the network. The final output Input into the feedforward neural network layer, the feedforward matrix Φ on the project side can be obtained i and the feedforward matrix Φ on the project provider side p .

[0079]

[0080] Among them, W4, W5, W6 and W7 are learnable weight matrices, and η1, η2, η3 and η4 are learnable bias vectors.

[0081] In the prediction layer, based on the historical interaction sequences between user items and the historical interaction sequences between user item providers, the user's behavior preference at time t+1 can be predicted. The feedforward matrix Φ i The feature vector of the item in row t and the item embedding vector The inner product between them can predict the correlation between candidate items and use the softmax function to convert it into the recommendation probability. The formula is as follows:

[0082]

[0083] We use self-supervised contrastive learning to learn user feature representations from both the project side and the project provider side. By combining user behavior sequences with project provider information for contrastive learning, the goal is to mine more detailed user preference and interest features.

[0084]

[0085] It is the characteristic representation of users on the project side. is the characteristic representation of the user on the project provider side, S - is a set of negative samples, represented by item providers from other users in the same batch. is the feature representation of a certain item provider in the negative sample, ∥∥ represents the Euclidean norm (L2 norm) of the vector, which is used to normalize the vector, and “·” represents the dot product of two vectors. represents the loss function for self-supervised contrastive learning.

[0086] Since negative samples are generated randomly, relying solely on self-supervised contrastive learning may cause semantically similar or identical sequences to be misclassified as negative samples. Therefore, we use semantically similar sequences as supervisory signals to further refine user feature representations. Two sequences are considered semantically similar if they reflect the same or similar user preferences. We consider sequences with the same target behavior as positive samples, while other augmented samples in the training batch are used as negative samples, thereby efficiently generating negative samples for augmented sample pairs.

[0087]

[0088] h′ u is the global vector representing the user features, h′ u,s is the feature representation of user semantic similarity sequence, S - is a set of negative samples, representing sequences with dissimilar semantics, is the feature representation of semantically dissimilar sequences in negative samples, Ъ is the temperature parameter used to control the degree of distinction between positive and negative samples in contrastive learning (the smaller Ъ, the more sensitive the model is to differences), and “·” represents the dot product of two vectors. is the loss function for semantic similarity contrastive learning, and “·” represents the dot product of two vectors.

[0089] In the model training and optimization layer, an objective function is set for the model to optimize the model parameters, and the cross entropy loss function is selected to measure the difference between the predicted value and the true value.

[0090]

[0091] Among them, y(∈R N ) is the encoding vector for the true target item in the current sequence, y j (∈(0,1)) is the value of vector y at index j if the true target item of the current sequence is v j , then y j =1, otherwise, the value of y at the index position is 0.

[0092] We use the Adam optimizer to convert the cross entropy loss function Loss Function for Self-Supervised Contrastive Learning And the loss function of supervised contrastive learning Perform joint optimization to obtain the minimum loss value A higher quality sequence recommendation can be obtained. ю is a hyperparameter that controls the proportion of contrastive learning loss.

[0093]

[0094] First, the model receives the interaction sequence between the user and the project, the interaction sequence between the user and the project provider, as well as the time point sequence and location information as input. By calculating the difference between each time point and the initial time point and normalizing it with an exponential decay function, a relative value representing the impact of the time interval is obtained. During the normalization process, the denominator also introduces the minimum relative time interval and the maximum relative time interval in the relative time interval sequence to obtain the final time interval, and uses one-hot encoding for the time interval information and the interaction sequence between the user and the project, the interaction sequence between the user and the project provider, and the location information to generate the project embedding vector Item provider embedding vector Time interval embedding vector and the embedding vector representation of the location information associated with the item Embedded vector representation of location information associated with item providers These embedding vectors are transformed into item-side feature vectors through linear transformation. and the project provider side feature vector Next, the attention bias with time interval information and position information is calculated. The model uses the time interval vector Combined with Laplace distribution, generate attention bias B with time interval information and position information ij , capturing the temporal asymmetry of user interests and dynamically adjusting the attention weights of the attention module with temporal bias information. Subsequently, these feature vectors and attention weights enter the attention module with temporal bias information to generate queries (Q i ), key (K i ) and value (V i ) matrix and query (Q p ), key (K p ) and value (V p ) matrix, the attention matrix of the project side and the project provider side is calculated by the attention module with time bias information and In this way, the interactive relationship between projects and providers is captured. At the same time, the attention module with frequency information calculates the autocorrelation relationship between the query and the key in the frequency domain through the fast Fourier transform (FFT), and generates the frequency domain attention weights on the project side and the project provider side respectively. and It is used to capture the periodic characteristics of user behavior. Then, the outputs of the attention module with time bias and the attention module with frequency information are divided according to the weight ratio β i and β p Perform weighted fusion to generate deep project-side feature representation and provider-side feature representation Next, these fused feature representations are input into a feed-forward neural network (FFN) and processed by two layers of multilayer perceptrons with GELU activation functions to generate the final item feature vector and the provider feature vector Finally, these feature representations are fed into the prediction layer, which calculates the deep item feature representation With item embedding vector The inner product of is then normalized by the softmax function to obtain the probability distribution of the user selecting the candidate item at time t+1. In addition, through the self-supervised contrastive learning module, the model uses positive and negative samples to construct a contrastive learning loss Sequences with similar semantics are used as supervisory signals to construct contrastive learning loss Capture user behavior patterns and fine-grained interest representation. Finally, the model is trained through cross entropy loss Loss function for contrastive learning with self-supervision And the loss function of supervised contrastive learning Perform joint optimization to obtain the minimum loss value Contrastive learning loss is used to jointly optimize the objective function to improve prediction performance and recommendation accuracy.

[0095] The user's behavior information, project sequence, time interval sequence and project provider sequence are input into the embedding layer of the recommendation task module. Then, the sequence representation containing project interaction information and the sequence representation containing project provider information are obtained through the attention layer with time bias and the attention layer with frequency information. The interaction between the sequence containing project information and the sequence containing project provider information is learned to obtain the final project sequence representation. In the contrastive learning part, the model uses noise contrast estimation loss to optimize the project feature representation and the project provider feature representation, and uses the semantic information between sequences as a supervision signal to further improve the representation quality.

[0096] The attention layer with time bias is used to learn the user's behavioral preferences at different time intervals to adjust the attention weights in the recommendation algorithm, so as to more accurately predict the user's preferences and recommend items that the user is more interested in. Specifically, the attention layer with time bias dynamically adjusts the weights through the attention bias with time interval and location information, while the attention layer with frequency information captures the periodic pattern of user behavior through time delay aggregation.

[0097] The item sequence representation and the item provider sequence representation are cross-input into the attention layer with time bias and the attention layer with frequency information to obtain deep item feature representation and item provider feature representation.

[0098] The item feature representation and item provider feature representation are input into the feedforward neural network layer to obtain the final item feature representation and item provider feature representation. The prediction score is calculated by the inner product of the final item feature representation and the item embedding vector, and the prediction score is converted into a prediction probability using the softmax function. The prediction probabilities of the candidate items are sorted to generate the final recommendation list.

[0099] Figure 1 The present invention proposes a sequence recommendation model based on contrastive learning and attention mechanism, the method comprising:

[0100] The interaction sequence between the user and the project and the interaction sequence between the user and the project provider are obtained as the input of the model. In the embedding layer, the time interval information, position information and sequence segmentation information are combined with the project embedding sequence and the project provider embedding sequence respectively to obtain the project embedding vector representation and the project provider embedding vector representation, and the following attention layer with time bias and attention layer with frequency information are used for calculation:

[0101] The user and item interaction sequence is input into the sequential recommendation model, and the user's interest preference representation is obtained through the attention layer with time bias and the attention layer with frequency information; the user and item provider interaction sequence is input into the sequential recommendation model, and the user preference representation containing item provider information is obtained through the attention layer with time bias and the attention layer with frequency information. The item sequence representation and the item provider sequence representation are cross-input into the attention layer with time bias and the attention layer with frequency information to obtain a deep user preference representation.

[0102] like Figure 1 As shown in the figure, the training process of the sequence recommendation model based on attention mechanism and contrastive learning is as follows:

[0103] Obtain the user's historical behavior data and its time information, and after preprocessing the location information, obtain the historical behavior data of the user's interaction with the project, the historical behavior data of the user's interaction with the project provider, time information, and location information;

[0104] The information extracted from the three real datasets TWITCH, WECHAT, and KUAIRAND is obtained, including user information, project information, project provider information, and time information, and is preprocessed to obtain the required historical interaction sequence between users and projects, historical interaction sequence between users and project providers, and time point sequence. The information extracted from the three real datasets is preprocessed, the obtained information is arranged in chronological order, and users and projects that appear less than five times are deleted. The user sequence is defined as U = [u1,u2,u3…u l ], l represents the user sequence length, and the item sequence in the user-item interaction record is defined as Among them, t n Represents the length of the item sequence, is the tth item in the user-item interaction sequence in U n The item provider sequence in the interaction record between the user and the item provider is defined as Among them, t m Represents the length of the item provider sequence, P is the tth item in the interaction sequence between the user and the item provider in U m An item provider that interacts with the user.

[0105] The time sequence T of the first e interactions of user u ue =[γ u,1 ,γ u,2 ,γ u,3 ,γ u,I …γ u,e] represents the time point sequence of the uth user, u∈{1,2,3,…l}, I∈{1,2,3,…e}, e represents the length of the time point sequence, and any time point γ in each dimension of the sequence uI The first time point γ of its dimension u,1 The difference between them is (γ uI – γ u,1 ) obtain a relative time interval sequence consisting of e relative time intervals, and find the smallest relative time interval and the largest relative time interval r in the relative time interval sequence mix and r max , by calculating the difference between each time point and the initial time point, and normalizing it with an exponential decay function to obtain a relative value representing the impact of the time interval. In the normalization process, the denominator also introduces the minimum relative time interval and the maximum relative time interval in the relative time interval sequence to obtain the final time interval vector Represents the time interval vector of the uth user, u∈{1,2,3,…l},I∈{1,2,3,…e}. Specifically, the time interval vector is expressed as follows:

[0106]

[0107] represents the floor operation, and λ is a decay factor that controls the speed of exponential decay. The exp function effectively captures the weakening of correlation when the time interval increases in the form of exponential decay.

[0108] Use one-hot to encode the interaction sequence between the input user and the item, the interaction sequence between the user and the item provider, and the time point sequence to obtain each item. The embedding vector representation of each project provider is Embedded vector representation of time interval information related to items Embedded vector representation of time interval information related to item providers Embedded vector representation of location information associated with items Embedded vector representation of location information associated with item providers Combine these embedding vectors to get the item embedding vector Item embedding matrix H i , item provider embedding vector and the project provider embedding matrix H p(In the embedding layer, the corresponding embedding vector is obtained by multiplying the one-hot encoding and the embedding matrix. The multiplication of the embedding matrix is ​​the process of mapping the high-dimensional sparse one-hot encoding into a low-dimensional dense embedding vector through matrix operations. The embedding matrix (such as W1, W2, W3, X1, X2, X3, etc.) is learned by the gradient descent algorithm during the model training process.)

[0109] For the item embedding matrix H i and the project provider embedding matrix H p By performing linear transformation, we can obtain the query generated by the project side (Q i ), key (K i ) and value (V i ), and the query generated by the project provider (Q p ), key (K p ) and value (V p ).

[0110] We propose a novel bias adjustment mechanism based on attention weights. This mechanism is specifically designed to dynamically adjust the focus of attention and its associated scope of action to adapt to the specific needs of the context. It is important to note that the evolution of user interests has a clear temporal asymmetry, which is particularly evident in the early and later stages of user behavior. Specifically, after experiencing a certain type of content (such as a movie), a user may show extremely high interest in similar content for a short period of time, but this interest tends to gradually decay over time. Our research shows that the Laplace distribution is an effective tool to describe such interest changes. Therefore, we introduce the Laplace distribution bias and time bias in the bias strategy of the attention mechanism. To obtain the attention bias B with time interval and position information IJ This bias allows the model to take into account the temporal variation of user interests when calculating attention weights, thereby more accurately adjusting the model's response to contextual information. IJ The introduction of not only enhances the dynamic adaptability of the model, but also improves the accuracy and efficiency of the model in processing user data in different time periods. The specific calculation formula is as follows:

[0111]

[0112] Here, the sigmoid function is a commonly used activation function, which helps to introduce nonlinearity so that the model can handle complex tasks and the output range is between 0 and 1; the tanh function performs a nonlinear transformation on the result of the linear transformation, compresses the data, and controls the value between -1 and 1. is the time interval vector, J is the biased variable affected by the location information of behavior I, l is the length of the user sequence, α is the proportional coefficient closely related to the distribution center, which takes a value between 0 and 1. Multiplying it by l can get the distribution center αl, which usually corresponds to the peak moment of user interest change. V d T is the scale coefficient of the variance, w5 is the learnable parameter coefficient, l is the distance variance, which controls the speed of interest decay. θ is the proportional coefficient related to the time interval, which takes a value between 0 and 1. By adjusting these parameters, we can make the model better adapt to the specific characteristics of user behavior in different application scenarios.

[0113] On the project side, the query vector matrix Q p , value vector matrix V i and the key vector matrix K i and attention bias B with time interval and position information IJ Inputting them together into the attention layer with time bias can obtain the attention weights related to the time information on the item side.

[0114]

[0115] in, (n is the item sequence length, d is the embedding dimension) is the attention matrix used by the item-side self-attention mechanism to learn item interactions. Used to represent query parameter matrix, key parameter matrix Corresponds to Key Matrix, is the value parameter matrix It represents the value matrix. yes By calculating the query terms With each key The dot product of is used to obtain the correlation between items, which helps the model capture the association between different items. These correlations are normalized by the softmax function and converted into attention weights, and the sum of the weights is 1. These weights are then applied to the corresponding values. The sum of the two is used to generate the interaction attention matrix, which reflects the model's understanding and learning of the relationship between different item arrangements in the user-item interaction sequence.

[0116] in, (m is the length of the item provider sequence, d is the embedding dimension) is the attention matrix used by the item provider side self-attention mechanism to learn item interactions. Query matrix Used to represent query parameter matrix, key parameter matrix Corresponds to Key Matrix, is the value parameter matrix It represents the value matrix. yes By calculating the query terms With each key The dot product of is used to obtain the correlation between items, which helps the model capture the association between different items. These correlations are normalized by the softmax function and converted into attention weights, and the sum of the weights is 1. These weights are then applied to the corresponding values. The interaction attention matrix is ​​generated by summing them up. This matrix reflects the model's understanding and learning of the arrangement relationship of different item providers in the interaction sequence between users and item providers.

[0117] Traditional sequential recommendation models usually rely on dependencies between items to capture users' long-term behavioral preferences. However, this approach often ignores the periodic behavior patterns of users. Many users may repeatedly view certain types of content or watch certain types of videos at specific time intervals, and the periodic characteristics of this behavior are crucial to the accuracy of recommendations. To address this problem, we can explore the relationship between query (Q) and key value (k) through autocorrelation analysis in the frequency domain. The specific steps are as follows:

[0118] Frequency domain analysis uses Fourier Transform to convert time series signals into frequency domain, thereby revealing the potential periodicity and frequency components in the signal. In the frequency domain attention mechanism of the present invention, the user's behavior sequence is first converted into a frequency domain representation through Fast Fourier Transform (FFT). This step can convert time series data into frequency components, so that behaviors at different times can be directly compared and analyzed in the frequency domain.

[0119] Ψ i =FFT -1 (FFT(Q P )⊙(FFT(k i ) * )

[0120] Ψ p =FFT -1 (FFT(Q i )⊙(FFT(k p ) * )

[0121] ⊙ represents the product of element-wise correlation, and * represents the conjugate operation. Fast Fourier Transform FFT can map the matrix from the time domain to the frequency domain. Inverse Fourier Transform FFT -1 , used to restore the results from the frequency domain to the time domain. Given a user's limited historical behavior sequence and The time delay operation is defined as

[0122] The corresponding embedding matrix is ​​represented as and In the frequency domain, a similar attention mechanism is performed by querying the autocorrelation of Q and key K. Since the information used to calculate the attention score comes from the same sequence, we consider it an autocorrelation operation. By querying the autocorrelation of Q and key K, we can get the most relevant time delay sequence in the frequency domain.

[0123] Second, we need to identify the past and future behavior sequences that are most relevant to the current behavior. By applying the function argTopk(Ψ i ), we can find the most relevant time delay ξ i , these time delays represent the periodic characteristics of user behavior. Next, through time aggregation, we softmax these selected most relevant sequences and then combine them with the corresponding key values ​​( or ) is multiplied to obtain the deep item feature representation and the deep item provider representation vector.

[0124]

[0125] argTopk(Ψ i ) function is to find the top k1 delay indices with the largest project-side autocorrelation in a set of delays. p ) function aims to find the first k2 time delay indexes with the largest autocorrelation on the project provider side in a set of time delays. The value of time delay k1 is proportional to the logarithm of the sequence length N, and r is the adjustment coefficient; the value of time delay k2 is proportional to the logarithm of the sequence length M, and j is the adjustment coefficient.

[0126]

[0127] Among them, on the project side, the most relevant time lag is The sum of the attention weights of different time delay sequences calculated by the softmax function is (n is the item sequence length, d is the embedding dimension) is the attention matrix used by the item-side attention layer with frequency information to learn item interactions.

[0128] On the project provider side, the most relevant time lag is The sum of the attention weights of different time delay sequences calculated using the softmax function is (n is the item sequence length, d is the embedding dimension) is the attention matrix used by the attention layer with frequency information on the item provider side to learn item interactions.

[0129] In order to obtain the final item feature representation and item provider feature representation, we perform a weighted summation of the time domain attention module and the frequency domain attention module. This method not only enhances the model's ability to capture user periodic behaviors, but also improves the overall performance of the recommendation system.

[0130]

[0131] On the project side, using the hyperparameter β i Perform a weighted sum to combine the outputs of the two attention mechanisms. i Controls the weight distribution of the attention module with time bias information and the attention module with frequency information. i When β approaches 1, the model is more inclined to the feature representation output by the attention module with frequency information. i When it approaches 0, the model is more inclined to the feature representation with the output of the time bias module. The final output It is a comprehensive feature representation that combines time bias information and frequency information and has richer sequence representation capabilities.

[0132] On the project provider side, using the hyperparameter β p Perform a weighted sum to combine the outputs of the two attention mechanisms. p Controls the weight distribution of the attention module with time bias information and the attention module with frequency information. p When β approaches 1, the model prefers the feature representation output by the attention module with frequency information. p When it approaches 0, the model is more inclined to the feature representation with the output of the time bias module. The final output It is a comprehensive feature representation that combines time bias information and frequency information and has richer sequence representation capabilities.

[0133] In order to make the network have nonlinear characteristics and capture higher quality project feature representation and project provider feature representation, we use two layers of multilayer perceptrons with GELU activation function. In a feedforward neural network, each neuron in the hidden layer receives the output from the previous layer, and after linear weighted summation, it is nonlinearly transformed through the activation function, so that the model can recognize and process complex patterns and features. As a typical feedforward neural network architecture, multilayer perceptron (MLP) gradually learns the deep features of input data through a combination of multiple layers of full connection. Each layer of MLP mainly consists of two parts: linear transformation and nonlinear activation. First, the linear transformation maps the input to a new feature space, and then the linear output is nonlinearly processed through the GELU activation function to enhance the expression ability of the network. GELU can provide smoother activation effects, especially when dealing with negative and small positive inputs. By combining the GELU activation function with MLP, the feedforward neural network not only has excellent nonlinear fitting ability, but also alleviates the gradient vanishing problem and improves the convergence speed and stability of the network. The final output Input into the feedforward neural network layer, the feedforward matrix Φ on the project side can be obtained i and the feedforward matrix Φ on the project provider side p .

[0134]

[0135] Among them, W4, W5, W6 and W7 are learnable weight matrices, and η1, η2, η3 and η4 are learnable bias vectors.

[0136] Based on the historical interaction sequences between user items and the historical interaction sequences between user item providers, the user's behavior preferences at time t+1 can be predicted. The feedforward matrix Φ i The feature vector of the item in row t and the item embedding vector The inner product between them can predict the correlation between candidate items and convert it into probability form through the softmax function, as shown in the following formula:

[0137]

[0138] An objective function is set for the sequence recommendation model of the present invention to optimize the model parameters, and a cross entropy loss function is selected to measure the difference between the predicted value and the true value.

[0139]

[0140] Where y∈R N is the encoding vector for the true target item in the current sequence, y j∈(0,1) is the value of vector y at index j if the true target item of the current sequence is v j , then y j =1, otherwise, the value of y at the index position is 0; the cross entropy loss function is optimized by Adam. Loss Function for Self-Supervised Contrastive Learning And the loss function of supervised contrastive learning Perform joint optimization to obtain the minimum loss value Then the optimized model parameters are obtained.

[0141] We use the Adam optimizer to convert the cross entropy loss function Loss Function for Self-Supervised Contrastive Learning And the loss function of supervised contrastive learning Perform joint optimization to obtain the minimum loss value A higher quality sequence recommendation can be obtained. ю is a hyperparameter that controls the proportion of contrastive learning loss.

[0142]

[0143] Obviously, the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, they are all within the protection scope of the present invention.

Claims

1. A sequence recommendation method based on attention mechanism and contrastive learning, characterized by: include: Pre-build and train sequence recommendation models; The interaction sequence between users and items, the interaction sequence between users and item providers, and the time point sequence are obtained and input into the trained sequence recommendation model to predict the next item that the user may interact with and obtain the recommendation result.

2. The sequence recommendation method based on attention mechanism and contrastive learning as claimed in claim 1, characterized in that: The interaction sequence between users and items, the interaction sequence between users and item providers, and the time point sequence are input into the embedding layer of the sequential recommendation model to extract dense feature representations of items and item providers. Subsequently, the model uses a time-biased attention layer and a frequency-biased attention layer to accurately learn the user's behavior and preferences at different times. The time-biased attention layer dynamically adjusts the weights through the attention bias with time interval information and location information, and the frequency-biased attention layer captures the periodic pattern of user behavior through time-delayed aggregation, thereby obtaining deep item feature representation and item provider feature representation. Then, the deep project feature representation and project provider feature representation are input into the feed-forward neural network layer to obtain the final project feature representation and project provider feature representation; The contrastive learning module uses noise contrast estimation loss to optimize the item feature representation and item provider feature representation, and uses the semantic information between sequences as a supervisory signal to improve the representation quality; the prediction score is calculated by the inner product of the final item feature representation and the item embedding vector, and the softmax function is used to convert the prediction score into a prediction probability to obtain the prediction probability of the candidate item, and the prediction probabilities of the candidate items are sorted to generate the final recommendation list.

3. The sequence recommendation method based on attention mechanism and contrastive learning as claimed in claim 2, characterized in that: An attention layer with time bias is used to learn users' behavioral preferences at different time intervals to adjust the attention weights in the recommendation model, so as to more accurately predict users' preferences and recommend items that they are more interested in. The attention layer with time bias dynamically adjusts the weights through the attention bias with time interval and location information, and the attention layer with frequency information captures the periodic patterns of user behavior through time delay aggregation.

4. The sequence recommendation method based on attention mechanism and contrastive learning as claimed in claim 3, characterized in that: The item sequence representation and the item provider sequence representation are cross-input into the attention layer with time bias and the attention layer with frequency information to obtain deep item feature representation and item provider feature representation.

5. A sequence recommendation method based on attention mechanism and contrastive learning as described in any one of claims 1 to 4, characterized in that: Input the deep project feature representation and project provider feature representation into the feed-forward neural network layer to obtain the final project feature representation and project provider feature representation; The prediction score is calculated by the inner product of the final item feature representation and the item embedding vector, and the softmax function is used to convert the prediction score into a prediction probability to obtain the prediction probability of the candidate item. The prediction probabilities of the candidate items are sorted to generate the final recommendation list.

6. The sequence recommendation method based on attention mechanism and contrastive learning as claimed in claim 5, characterized in that: The attention layer with time bias dynamically adjusts the weights through the attention bias with time interval and position information, which includes the introduction of Laplace distribution bias and time bias in the bias strategy of the attention mechanism To obtain the attention bias B with time interval and position information IJ ;attention Bias B IJ The calculation includes: Among them, J is a variable that is affected by the position information of behavior I and takes into account the bias, l is the length of the user sequence, and α is a proportional coefficient that is closely related to the distribution center, which takes a value between 0 and 1. Multiplying it by l can get the distribution center αl, which corresponds to the peak moment of user interest change; is the scale coefficient of the variance, w5 is the parameter coefficient, is the distance variance, which controls the speed of interest decay; θ is the proportional coefficient related to the time interval, which takes a value between 0 and 1; by adjusting the parameters of attention bias, the model can better adapt to the specific characteristics of user behavior in different application scenarios.

7. The sequence recommendation method based on attention mechanism and contrastive learning as claimed in claim 6, characterized in that: When training the sequence recommendation model, the model parameters are optimized through the objective function, in which the cross entropy loss function is selected to measure the difference between the predicted value and the true value. Where y∈R N is the encoding vector for the true target item in the current sequence, y j ∈(0,1) is the value of vector y at index j if the true target item of the current sequence is v j , then y j =1, otherwise, the value of y at the index position is 0; the cross entropy loss function is optimized by Adam. Loss Function for Self-Supervised Contrastive Learning And the loss function of supervised contrastive learning Perform joint optimization to obtain the minimum loss value Then the optimized model parameters are obtained.

8. The sequence recommendation method based on attention mechanism and contrastive learning as claimed in claim 6, characterized in that: On the project side, the query vector matrix Q p , value vector matrix V i and the key vector matrix K i and attention bias B with time interval and position information IJ Inputting them together into the attention layer with time bias can obtain the attention weights related to the time information on the project side: On the project provider side, the query vector matrix Q i , value vector matrix V p and the key vector matrix K p and attention bias B with time interval and position information IJ Inputting them together into the attention layer with time bias can obtain the attention weights related to time information on the project provider side;

Images