A sequence recommendation method and system based on contrastive learning

Abstract

The present invention discloses a sequence recommendation method based on contrastive learning, comprising the following steps: S1, collecting historical interaction behavior sequence data between users and items, encoding them according to an embedding layer, and obtaining initial embedding representations of users and items with interaction records with users; S2, inputting the initial embedding representations into a filter group for filtering, and obtaining denoised embedding representations; obtaining user intention information through a clustering algorithm based on the behavior sequence data; S3, using a sequence encoder based on a feedforward neural network in combination with the denoised embedding representation as a first training input; combining the user intention information with the output sequence representation of the sequence encoder through a contrastive self-supervisory algorithm, and using the first training input and the second training input for training a sequence recommendation model at the same time; obtaining a prediction score, and finally performing sequence prediction.

Description

Technical Field

[0001] The present invention relates to the field of sequence recommendation, and in particular to a sequence recommendation method and system based on contrastive learning. Background Art

[0002] As an important information filtering method in today's society, the recommendation system has received extremely high attention. The essence of the recommendation system is to match user and project features through various methods such as matrix decomposition, deep learning, and autoencoders. It also explores users' personalized preferences and needs based on their interest classification labels, historical behavior records, and other user personal information, and provides customized services.

[0003] Sequential recommendation systems require the collection, processing, and storage of large amounts of user and item data. This data necessarily includes timestamps and may include other information such as user history, interests, and preferences, as well as item attributes, categories, and tags. Recommendation systems require preprocessing, feature extraction, and representation learning for this data to facilitate subsequent use in recommendation models. The recommendation model is the core of the recommendation system, mining the implicit information within the data to predict user preferences or ratings for different items. However, existing sequential recommendation methods still face several challenges. First, sequential recommendation algorithms require sufficient training data to support model training. However, user behavior data is typically sparse, meaning that most items lack user behavior records. This can lead to insufficient data during model training, compromising model accuracy and generalization performance. Second, existing sequential recommendation models often use stacked self-attention layers. Excessive self-attention layers can lead to excessive model parameters and excessive sensitivity to noise in the raw data, potentially causing overfitting and instability.

[0004] Therefore, there is an urgent need to solve the technical problems of insufficient training data for recommendation models and overfitting and instability of pricing models in existing sequential recommendation methods. Summary of the Invention

[0005] The present invention provides a sequence recommendation method and system based on contrastive learning, which are used to solve the technical problems of insufficient training data for recommendation models and overfitting and instability of recommendation models in existing sequence recommendation methods.

[0006] To achieve the above object, the present invention provides a sequence recommendation method based on contrastive learning, comprising the following steps:

[0007] S1. Collect historical interaction behavior sequence data between users and items, encode it according to the embedding layer, and obtain the initial embedding representation of users and items with which users have interaction records;

[0008] S2. Input the initial embedding representation into the filter bank for filtering to obtain a denoised embedding representation; obtain user intent information through a clustering algorithm based on the behavior sequence data;

[0009] S3. Use a sequence encoder based on a feedforward neural network combined with a denoising embedding representation as the first training input; combine the user intent information with the output sequence representation of the sequence encoder through a comparative self-supervised algorithm as the second training input; use the first training input and the second training input simultaneously for training the sequence recommendation model; obtain the prediction score and finally perform sequence prediction.

[0010] Preferably, S1 includes: collecting historical interaction behavior sequence data between users and projects, encoding through a pre-trained embedding layer, mapping the high-dimensional representation of users and projects into low-dimensional dense vector representations, thereby converting each historical behavior into a vector representation, and then obtaining the initial embedding representation of users and projects with which users have interaction records through splicing operations.

[0011] Preferably, S2 further includes: after obtaining the denoised embedded representation, performing data augmentation on the denoised embedded representation: sampling a negative item in each time step and each sequence, and comparing the negative item with the positive item; the positive item is an item with which the user has interacted, and the negative item is an item with which the user has not interacted;

[0012] After obtaining user intent information, data enhancement is performed on the behavior sequence data corresponding to the user intent information: the behavior sequence data corresponding to the user intent information is processed using self-supervised learning or data enhancement methods such as insertion or replacement to construct positive and negative samples. Positive samples are products that the user has actually interacted with, and negative samples are products that the user has not interacted with.

[0013] Preferably, the method further comprises: after collecting historical interaction behavior sequence data between the user and the project, first preprocessing the data:

[0014] Denote users and items by U and I respectively, u∈U represents a user, i∈I represents an item; each user u has a sequence of items that he interacts with in chronological order During preprocessing, if the length of the item sequence is greater than L, the most recent L interactions in the item sequence are considered and the item sequence is truncated to a length of L; if the sequence length is less than L, preset padding items are added to the sequence until the length of the sequence reaches L.

[0015] Preferably, the filter bank is composed of a plurality of automatically learnable filters, and for a given input item representation matrix F at the lth layer l , F l ∈R n×d , first perform fast Fourier transform along the dimension of the project, F lConvert to frequency domain signal X l , then multiply the filter W, W∈C n×d To modulate the spectrum:

[0016]

[0017] Where ⊙ is an element-wise multiplication, and the filter W represents an automatically learnable filter. is the frequency domain signal obtained after denoising.

[0018] Preferably, in S2, obtaining user intention information through a clustering algorithm based on the behavior sequence data includes:

[0019] The user's corresponding behavior sequence data is used as the supervision signal, and all sequences are encoded by the sequence encoder. Encode and output the user's intention representation sequence through aggregation method

[0020] The clustering algorithm is used to process the output user intention representation to obtain the user's potential intention distribution function Q(c , ):

[0021]

[0022] Among them, P θ (c , |S u ) indicates that user u belongs to user intention c in the case of behavior sequence S i probability;

[0023] Get the user intention distribution function Q(c , ) and then average the representations in the cluster by average pooling to obtain the center of the cluster, that is, the representation of the user's potential intention c , .

[0024] Preferably, in the sequence encoder with the feedforward neural network as the main body, the output representation of the sequence encoder at layer l is It can be expressed as:

[0025]

[0026]

[0027] Among them, W1, b1, W2, and b2 are training parameters, W1 represents the learnable weight of the first fully connected layer in the sequence mixer, b1 represents the learnable bias of the first fully connected layer in the sequence mixer, W2 represents the learnable weight of the second fully connected layer in the sequence mixer, b2 represents the learnable bias of the second fully connected layer in the sequence mixer, and FFN represents feedforward neural network; is the matrix representing the input items of the filtered layer l; ReLU is the nonlinear activation function. Layernorm represents the layer normalization operation; Dropout represents the dropout operation.

[0028] Preferably, the representation C of the user's potential intention is obtained , Then, the mutual information between the behavior sequence and its corresponding potential intention is maximized to derive the interaction probability representation P between the user and an item. θ (S u ,c i ) and optimize the objective function for:

[0029]

[0030]

[0031] Among them, sim represents the inner product operation; exp represents the exponential function; P θ (c i )P θ (S u |c u ) indicates that user u has a behavior sequence S and a behavior sequence belonging to user intention c at the same time i The probability of the case.

[0032] Preferably, after deriving the probability representation of the interaction between the user and a certain item and the optimization objective function, the method further includes:

[0033] According to the principle of contrastive self-supervised learning, data enhancement is performed on the training sequence in each batch-size to create two front views and Optimize the model and optimize the loss function for:

[0034]

[0035]

[0036]

[0037] Among them, c represents the set of users with the same intention;

[0038] Then, through a pre-calculation method based on item similarity, the most similar items are selected for insertion or replacement to achieve data augmentation. At the same time, considering the correlation between the two behavior sequences and maximizing the mutual information in contrastive learning, the behavior sequences are encoded and another set of loss functions is constructed:

[0039]

[0040]

[0041]

[0042] in, Sequence representation generated under negative view;

[0043] In the last prediction layer of the model, the user’s interest score P for item i in step (t+1) is calculated:

[0044]

[0045] Among them, a i is the vector representation of item i, H t is the output of the sequence encoder in step t;

[0046] Finally, a multi-task joint loss function is used Train and optimize model parameters:

[0047]

[0048] Among them, the parameters β and d regulate the degree of sequence-intention contrast learning task and sequence-sequence contrast learning task respectively; is the binary cross entropy loss function of the sequence encoder without incorporating user intent.

[0049] The present invention also provides a sequence recommendation system based on contrastive learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method of the present invention are implemented.

[0050] The present invention has the following beneficial effects:

[0051] The present invention addresses the problem of noise information such as user malicious behavior noise, item noise, context noise, etc. that may exist in sequence recommendation. By using a self-learning filter group, a filtering operation is performed to reduce the noise in the initial embedding representation that affects the performance of the sequence recommendation model, and obtain a denoised embedding representation. Such processing improves the robustness of the model to noise and redundant information. The present invention uses the behavioral sequence data corresponding to all users as a supervisory signal, and learns the potential intentions of different users and their correlations through a clustering algorithm. The clustering algorithm adopted by the present invention can help group users into groups with similar behavioral patterns and interest preferences. By clustering users into different groups, the diversity of users' behaviors and interests can be better understood. This learning process helps to better understand users' behavioral patterns and preferences, and helps to make personalized recommendations. The present invention improves the sequence encoder based on the self-attention mechanism, uses a multi-layer perceptron architecture for modeling, and combines the denoised embedding representation for model training. By optimizing the model parameters, the present invention can more accurately capture temporal dynamics and sequential patterns. Combining the user's potential intentions learned by the clustering algorithm, a semantic-aware contrastive learning mechanism is introduced. By comparing the differences between positive and negative samples, the correlation and similarity between positive and negative samples can be identified, thereby improving the understanding and modeling of the user's true intentions.

[0052] In addition to the above-described objects, features and advantages, the present invention has other objects, features and advantages. The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0054] Figure 1 This is a flow chart of a sequence recommendation method according to a preferred embodiment of the present invention.

[0055] Figure 2 It is a framework diagram of the noise filtering part of the preferred embodiment of the present invention.

[0056] Figure 3 4 is a framework diagram of user intention backup according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0057] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered by the claims.

[0058] In a preferred embodiment of the present invention, a sequence recommendation method based on contrastive learning is provided, comprising the following steps:

[0059] S1. Collect historical interaction behavior sequence data between users and items, encode it according to the embedding layer, and obtain the initial embedding representation of users and items with which users have interaction records;

[0060] In a preferred embodiment of the present invention, after collecting the historical interaction behavior sequence data between users and projects, the data is first preprocessed:

[0061] Denote users and items by U and I respectively, u∈U represents a user, i∈I represents an item; each user u has a sequence of items that he interacts with in chronological order During preprocessing, if the length of the item sequence is greater than ), the most recent ) interactions in the item sequence are considered and the item sequence is truncated to length ); if the sequence length is less than ), preset padding items are added to the sequence until the sequence length reaches ).

[0062] In a preferred embodiment of the present invention, the padding item is to fill a sequence of insufficient length with '0' elements, and then perform masking processing to reduce the adverse effects of the 0 elements.

[0063] S1 includes: collecting historical interaction behavior sequence data between users and projects, encoding it through a pre-trained embedding layer, mapping the high-dimensional representations of users and projects into low-dimensional dense vector representations, thereby converting each historical behavior into a vector representation, and then obtaining the initial embedding representation of users and projects with which users have interaction records through splicing operations.

[0064] In the preferred embodiment of the present invention, see Figure 2 The initialization embedding layer model is a pre-trained embedding representation model using Word2Vec technology. The user interaction item sequence data is converted into a continuous vector representation (matrices) so that the neural network can process it.

[0065] S2. Input the initial embedding representation into the filter bank for filtering to obtain the denoised embedding representation; see Figure 3 ,According to the behavior sequence data, the user intention information is obtained through the clustering algorithm;

[0066] In S2: After obtaining the denoised embedding representation, data augmentation is performed on the denoised embedding representation: a negative item is sampled at each time step and each sequence and compared with the positive item; the positive item is the item that the user has interacted with, and the negative item is the item that the user has not interacted with;

[0067] In a preferred embodiment of the present invention, softmax technology is used for data enhancement. When calculating the conditional probability distribution of each time step and each sequence, only positive items and sampled negative items are considered, while all unsampled negative items are not considered.

[0068] After obtaining user intent information, data enhancement is performed on the behavior sequence data corresponding to the user intent information: the behavior sequence data corresponding to the user intent information is processed using self-supervised learning or data enhancement methods such as insertion or replacement to construct positive and negative samples. Positive samples are products that the user has actually interacted with, and negative samples are products that the user has not interacted with.

[0069] In a preferred embodiment of the present invention, a sequence data noise filtering module is constructed, given a digital sequence {x n}, the discrete signal in the time domain can be converted into spectrum information through the one-dimensional discrete Fourier transform method:

[0070]

[0071] Given a one-dimensional discrete Fourier transform X k , the signal in the frequency domain can be converted back to the signal in the time domain by inverse discrete Fourier transform, and the original sequence {x n}:

[0072]

[0073] Fast Fourier transform and inverse fast Fourier transform are algorithms corresponding to the efficient calculation of discrete Fourier transform and inverse discrete Fourier transform, respectively.

[0074] The filter bank is composed of multiple filters that can be automatically learned. For a given input item representation matrix F at the lth layer, l ∈R n×d , first perform fast Fourier transform along the dimension of the project, F l Convert to frequency domain signal X l , then multiply the filter W∈C n×d To modulate the spectrum:

[0075]

[0076] Where ⊙ is an element-wise multiplication, and the filter W represents an automatically learnable filter. is the frequency domain signal obtained after denoising.

[0077] In a preferred embodiment of the present invention, the sequence encoder f θ (·) When the user’s potential intention information is not included, for a given sequence S u Encode and output user interest representation H at all position steps u =f θ (S # ),in Represents the user’s intention at position t. The optimal encoder parameter θ is obtained through model training to maximize the log-likelihood function of the next item expected at all position steps in the given sequence.

[0078] In S2, user intention information is obtained through clustering algorithm based on behavior sequence data, including:

[0079] The user's corresponding behavior sequence data is used as the supervision signal, and all sequences are encoded by the sequence encoder. Encode and output the user's intention representation sequence through aggregation method

[0080] The clustering algorithm is used to process the output user intention representation to obtain the user's potential intention distribution function Q(c i ):

[0081]

[0082] Get the user intention distribution function Q(9 i ) and then average the representations in the cluster by average pooling to obtain the center of the cluster, which is the representation of the user's potential intention C i .

[0083] Get the representation C of the user's potential intention i After that, the mutual information between the behavior sequence and its corresponding potential intention is maximized, and the interaction probability representation P of the user and an item can be derived. θ (S u ,c i ) and optimize the objective function for:

[0084]

[0085]

[0086] In a preferred embodiment of the present invention, the model is based on the K potential intentions learned from Rewrite the optimization objective function of maximizing the log-likelihood function of the expected next item at all position steps of the given sequence as:

[0087]

[0088] Since the variable θ is related to the user intention c i are all missing values, so they need to be handled through a generalized expectation maximization framework. According to the generalized expectation maximization framework, it is also necessary to maximize the above formula by maximizing the lower bound function. Assume that the intention c follows the distribution Q(c) and satisfies the conditions:

[0089] ∑ c Q(Ci )=1&Q(C i )>0,

[0090] Then the lower bound function of the above optimization objective function can be constructed:

[0091]

[0092] Combined with Jensen's inequality, the performance function of the evaluation model can be derived as follows:

[0093]

[0094] in, In order to reduce the computational complexity of the loss function, we can only consider the lower bound of each last step (i.e., the tth step), and the model will remove the summation operation on the t variables.

[0095] S3. Use a sequence encoder based on a feedforward neural network combined with a denoising embedding representation as the first training input; combine the user intent information with the output sequence representation of the sequence encoder through a comparative self-supervised algorithm as the second training input; use the first and second training inputs simultaneously for training the sequence recommendation model; obtain the prediction score and finally perform sequence prediction.

[0096] In the sequence encoder with feedforward neural network as the main body, the output representation of the sequence encoder of layer l is It can be expressed as:

[0097]

[0098]

[0099] Among them, W1, b1, W2, and b2 are trainable parameters, W1 represents the learnable weight of the first fully connected layer in the sequence mixer, b1 represents the learnable bias of the first fully connected layer in the sequence mixer, W2 represents the learnable weight of the second fully connected layer in the sequence mixer, b2 represents the learnable bias of the second fully connected layer in the sequence mixer, and FFN represents a feedforward neural network; is the matrix representing the filtered input items of the lth layer.

[0100] In the preferred embodiment of the present invention, it is assumed that a recommendation system has a set of users and items, denoted by U and I respectively, where u∈U represents a user and i∈I represents an item. The number of users and items are denoted by |U| and |I| respectively. For the initial item matrix M I ∈R |I|×d, this paper uses an embedding layer to map the high-dimensional ont-hot representation of user and item IDs into low-dimensional dense vector representations and concatenate them together. Given an item sequence of length n, find the item elements that interact with the user from the sequence dataset and construct the initial sequence embedding matrix k∈R n×d , combined with the position encoding matrix P∈R n×d To capture the position information of each item in the sequence to enhance the input representation of the item sequence, the two embedding matrices are summed to obtain the complete sequence representation k I ∈R n×d That is, the output representation of the initial embedding layer is obtained. Then in the filter layer, the input item representation matrix F of the given layer l is l ∈R n×d (When l=0, this paper sets F A =k I ), filter the filter layer and get Input to the sequence encoder and get That is, the output of the sequence encoder at layer l.

[0101] After deriving the probability representation of the interaction between the user and an item and the optimization objective function, it also includes:

[0102] According to the principle of contrastive self-supervised learning, data enhancement is performed on the training sequence in each batch-size to create two front views and Optimize the model and optimize the loss function for:

[0103]

[0104]

[0105]

[0106] Among them, c represents the set of users with the same intention;

[0107] Then, through a pre-calculation method based on item similarity, the most similar items are selected for insertion or replacement to achieve data augmentation. At the same time, considering the correlation between the two behavior sequences and maximizing the mutual information in contrastive learning, the behavior sequences are encoded and another set of loss functions is constructed:

[0108]

[0109]

[0110]

[0111] in, Sequence representation generated under negative view;

[0112] In the last prediction layer of the model, the user’s interest score for item i in step (t+1) is calculated:

[0113]

[0114] Among them, a i is the vector representation of item i, H t is the output of the sequence encoder in step t;

[0115] Finally, a multi-task joint loss function is used Train and optimize model parameters:

[0116]

[0117] Among them, the parameters β and γ regulate the degree of sequence-intention contrast learning task and sequence-sequence contrast learning task respectively; is the binary cross entropy loss function of the sequence encoder without incorporating user intent.

[0118] In a preferred embodiment of the present invention, for the construction of the loss function:

[0119] The model is trained using a sequence encoder based on a feedforward neural network combined with a denoising embedding representation as the first training input. The following parts are used to train the model:

[0120] The sequence recommendation model is used to model the sequence pattern and predict the user's next behavior. In this case, a sequence encoder f can be defined θ (·), which is a sequence S u (S u represents the interaction item sequence of user u, n is the number of interactive items) and outputs the user interest representation H at all position steps in the sequence u =f θ (S u ),in Represents the user’s interest at position t. The goal is to find the optimal encoder (model) parameter θ to maximize the log-likelihood function of the next item expected at all position steps of the given sequence, as shown in the following formula:

[0121]

[0122] This is equivalent to minimizing the adaptive binary cross entropy loss function, as shown below:

[0123]

[0124]

[0125] in, represents the user's interest in location C, and represents the target item i of user u t and the vector representation of all items that have not been interacted with by u, v is the sigmoid function, and N refers to the mini-batch size of the sequence model input (i.e., batchsize, the number of samples included in each training batch).

[0126] The contrastive self-supervised algorithm combines user intent information with the output sequence representation of the sequence encoder as the second training input to train the model. The following parts are included:

[0127] Assume that user behavior sequences can reflect K implicit intentions through clustering algorithms Then the probability of a user interacting with an item can be expressed as P θ (S u ), the optimization objective function can be rewritten as θ * :

[0128]

[0129]

[0130] Among them, E(c) represents the expected value of the random variable c, where c represents the representation of the user's intention. That is, the expectation of the input data encoding. Since the variable θ is related to the user's intention c i are all missing values, so we need to use the EM algorithm to solve the above equation. The E-step of the EM algorithm starts with an initial guess of the model parameters θ and estimates the missing variables c i The expected value of the M-step is when there is a variable c i Based on the value of , maximize the parameter θ. Repeat this iterative process until the likelihood value can no longer be increased.

[0131] According to the EM algorithm, we need to maximize the above formula by maximizing the lower bound function. Assume that the intention c follows the distribution Q(c), and ∑ c Q(c i )=1&Q(c i )>0, then the following formula can be constructed as θ * The lower bound function of :

[0132]

[0133] According to Jensen's inequality, we can deduce the result:

[0134]

[0135] In order to reduce the computational complexity of the loss function, we can only consider the lower bound of each last step (i.e., step C). Thus, we can remove a summation operation and obtain the maximization formula:

[0136]

[0137] In order to learn the intention distribution Q(c), the encoder Encode and output representation Then perform Kmeans clustering on these representations to obtain Q(c i )=P θ (c i |S # ), as shown below:

[0138]

[0139] The representations in the cluster are averaged by average pooling, and the center of the cluster can be used as the intention c i Characterization of C i .

[0140] After obtaining the intention distribution Q(c), we need to calculate P θ (S u ,c i ), defined as follows:

[0141]

[0142] Among them, h u It is sequence S u The maximization formula is equivalent to the following minimization formula, which aims to maximize the mutual information between a certain behavior sequence and its corresponding potential intention (MIM principle):

[0143]

[0144] Among them, sim represents the inner product. According to the principle of contrastive self-supervised learning, for each sequence trained in batch-size, data enhancement is required to create two front views. and And optimize the loss function as shown below:

[0145]

[0146]

[0147] Among them, c u Is the intentioni The vector representation of c neg is all the intents in a given batch. However, directly optimizing the above formula may lead to the problem of false negative samples, that is, treating the same intent of different users as negative samples. To solve this problem, the loss function is finally modified as follows:

[0148]

[0149] in, represents a set of users with the same intent. Therefore, during the iterative execution of the EM-Step, the intent distribution Q(c) and model parameters θ are continuously updated.

[0150] In addition to maximizing the mutual information between the above sequence and the intention, it is also necessary to consider the correlation between the two behavior sequences and the maximization of the mutual information in contrastive learning. u , and the transformation function set G, two positive sample views can be constructed as shown in the following formula:

[0151]

[0152] Among them, V is a transformation function sampled from the transformation set G, which generally includes mask, crop, reorder, etc. However, applying these operations to sequence data is likely to cause deviation or loss of original information. Therefore, this paper adds a pre-calculation method based on item similarity, selecting the most similar items for insertion or replacement operations to achieve data enhancement. Generally speaking, views within the same sequence are defined as positive pairs (PositivePairs), while views from different sequences are regarded as negative pairs (NegativePairs). After encoding the behavior sequence under the positive view, its representation is obtained. Finally, the loss function is constructed according to the idea of ​​InfoNCE algorithm, as shown in the following formula:

[0153]

[0154]

[0155] in, Sequence representation generated under negative view.

[0156] The preferred embodiment of the present invention addresses the problem of noise information such as user malicious behavior noise, item noise, context noise, etc. that may exist in sequence recommendation. By using a self-learning filter group, a filtering operation is performed to reduce the noise in the initial embedding representation that affects the performance of the sequence recommendation model, and obtain a denoised embedding representation. Such processing improves the robustness of the model to noise and redundant information. The present invention uses the behavioral sequence data corresponding to all users as a supervisory signal, and learns the potential intentions of different users and their correlations through a clustering algorithm. The clustering algorithm adopted by the present invention can help group users into groups with similar behavioral patterns and interest preferences. By clustering users into different groups, the diversity of user behaviors and interests can be better understood. This learning process helps to better understand user behavioral patterns and preferences, and helps to make personalized recommendations. The present invention improves the sequence encoder based on the self-attention mechanism, uses a multi-layer perceptron architecture for modeling, and combines the denoised embedding representation for model training. By optimizing the model parameters, the present invention can more accurately capture temporal dynamics and sequential patterns. Combining the user's potential intentions learned by the clustering algorithm, a semantic-aware contrastive learning mechanism is introduced. By comparing the differences between positive and negative samples, the correlation and similarity between positive and negative samples can be identified, thereby improving the understanding and modeling of the user's true intentions.

[0157] A preferred embodiment of the present invention further provides a sequence recommendation system based on contrastive learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the system implements the steps of the method of the present invention. The system has the same beneficial effects as the method of the present invention.

[0158] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A sequence recommendation method based on contrastive learning, characterized in that: The following steps are involved: S1. Collect historical interaction behavior sequence data between users and items, encode it according to the embedding layer, and obtain the initial embedding representation of users and items with which users have interaction records; S2. Input the initial embedding representation into a filter bank for filtering to obtain a denoised embedding representation; obtain user intent information through a clustering algorithm based on the behavior sequence data; S3. Using a sequence encoder based on a feedforward neural network in combination with the denoised embedding representation as a first training input; combining the user intent information with the output sequence representation of the sequence encoder through a comparative self-supervised algorithm as a second training input; using the first training input and the second training input simultaneously for training a sequence recommendation model; obtaining a prediction score, and finally performing sequence prediction; The filter bank is composed of a plurality of automatically learnable filters, and for a given input item representation matrix F at the lth layer l , F l ∈R n×d , first perform fast Fourier transform along the dimension of the project, F l Convert to frequency domain signal X l , then multiply the filter W, W∈C n×d To modulate the spectrum: Where ⊙ is an element-wise multiplication, and the filter W represents an automatically learnable filter. is the frequency domain signal obtained after denoising; In S2, obtaining user intention information based on the behavior sequence data through a clustering algorithm includes: The user's corresponding behavior sequence data is used as the supervision signal, and all sequences are encoded by the sequence encoder. Encode and output the user's intention representation sequence through aggregation method The clustering algorithm is used to process the output user intention representation to obtain the user's potential intention distribution function Q(c i ): Among them, P θ (c i |S u ) indicates that user u belongs to user intention c in the case of behavior sequence S i probability; Get the user intention distribution function Q(c i ) and then average the representations in the cluster by average pooling to obtain the center of the cluster, that is, the representation of the user's potential intention c i ; In the sequence encoder based on the feedforward neural network, the output representation of the sequence encoder at layer l is It can be expressed as: Among them, W1, b1, W2, and b2 are training parameters, W1 represents the learnable weight of the first fully connected layer in the sequence mixer, b1 represents the learnable bias of the first fully connected layer in the sequence mixer, W2 represents the learnable weight of the second fully connected layer in the sequence mixer, b2 represents the learnable bias of the second fully connected layer in the sequence mixer, and FFN represents feedforward neural network; is the input item representation matrix of the filtered lth layer; ReLU is the nonlinear activation function, Layernorm represents the layer normalization operation; Dropout represents the dropout operation.

2. The sequence recommendation method based on contrastive learning according to claim 1, characterized in that S1 includes: collecting historical interaction behavior sequence data between users and projects, encoding it through a pre-trained embedding layer, mapping the high-dimensional representation of users and projects into low-dimensional dense vector representations, thereby converting each historical behavior into a vector representation, and then obtaining the initial embedding representation of users and projects with which users have interaction records through splicing operations.

3. The sequence recommendation method based on contrastive learning according to claim 2, characterized in that Also included in S2: After obtaining the denoised embedding representation, data augmentation is performed on the denoised embedding representation: a negative item is sampled at each time step and each sequence, and the negative item is compared with the positive item; the positive item is the item that the user has interacted with, and the negative item is the item that the user has not interacted with; After obtaining the user intent information, data enhancement is performed on the behavior sequence data corresponding to the user intent information: the behavior sequence data corresponding to the user intent information is processed using self-supervised learning or insertion or replacement data enhancement methods to construct positive and negative samples. Positive samples are products that the user has actually interacted with, and negative samples are products that the user has not interacted with.

4. The sequence recommendation method based on contrastive learning according to claim 1, characterized in that The method further includes: after collecting historical interaction behavior sequence data between the user and the project, first preprocessing the data: Denote users and items by U and I respectively, u∈U represents a user, i∈I represents an item; each user u has a sequence of items that he interacts with in chronological order During preprocessing, if the length of the item sequence is greater than L, the most recent L interactions in the item sequence are considered and the item sequence is truncated to a length of L; if the sequence length is less than L, preset padding items are added to the sequence until the length of the sequence reaches L.

5. The sequence recommendation method based on contrastive learning according to claim 1, characterized in that Get the representation C of the user's potential intention i Then, the mutual information between the behavior sequence and its corresponding potential intention is maximized to derive the interaction probability representation P between the user and an item. θ (S u ,c i ) and optimize the objective function for: Among them, sim represents the inner product operation; exp represents the exponential function; P θ (c i )P θ (S u |c i ) indicates that user u has a behavior sequence S and a behavior sequence belonging to user intention c at the same time i The probability of the case.

6. The sequence recommendation method based on contrastive learning according to claim 5, characterized in that After deriving the probability representation of the interaction between the user and an item and the optimization objective function, it also includes: According to the principle of contrastive self-supervised learning, data enhancement is performed on the training sequence in each batch-size to create two front views and Optimize the model and optimize the loss function for: Among them, c represents the set of users with the same intention; Then, through a pre-calculation method based on item similarity, the most similar items are selected for insertion or replacement to achieve data augmentation. At the same time, considering the correlation between the two behavior sequences and maximizing the mutual information in contrastive learning, the behavior sequences are encoded and another set of loss functions is constructed: in, Sequence representation generated under negative view; In the last prediction layer of the model, the user’s interest score P for item i in step (t+1) is calculated: Among them, a i is the vector representation of item i, H t is the output of the sequence encoder in step t; Finally, a multi-task joint loss function is used Train and optimize model parameters: Among them, the parameters β and γ regulate the degree of sequence-intention contrast learning task and sequence-sequence contrast learning task respectively; is the binary cross entropy loss function of the sequence encoder without incorporating user intent.

7. A sequential recommendation system based on contrastive learning, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the method according to any one of claims 1 to 6 are implemented when the processor executes the computer program.

Images