Sequence recommendation method based on contrastive learning and variational autoencoder

Abstract

The application discloses a sequence recommendation method based on contrast learning and variational autoencoder, which comprises the following steps: generating sequence embedding according to the historical interaction sequence of a user; performing time-frequency domain conversion on an original data branch; performing data sliding enhancement on a data enhancement branch; performing model level enhancement processing on data according to a random deep enhancement strategy; encoding the data of the original data branch and the enhanced data branch respectively according to a Transformer encoder, and obtaining the mean and variance of the data of each data stream branch; maximizing positive and negative sample losses according to a contrast learning strategy; generating a new item sequence according to a shared Transformer decoder; and constructing a sequence recommendation model based on contrast learning and variational autoencoder. The application takes the historical interaction sequence of a user as input, processes the model by fusing the contrast learning paradigm and the variational autoencoder, generates an item of interest of the user, has better recommendation quality, and improves recommendation accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of recommendation system, and particularly relates to a sequence recommendation method based on contrast learning and variational autoencoder. BACKGROUND

[0002] With the rapid development of the digital era, the amount of information on the Internet has increased dramatically, and the difficulty of finding specific information has also increased, and the problem of information overload has become increasingly prominent. Under this background, how to effectively filter out the information that is really valuable to users from the vast data sea becomes particularly critical. The recommendation system accurately selects the content that users may be interested in from a large amount of data by analyzing the user's preferences. The recommendation system provides significant convenience in many fields such as e-commerce, video streaming media and news media, and improves the user experience. For enterprises, the recommendation system improves user satisfaction and retention rate through personalized recommendation mechanism, and promotes the promotion effect and profit growth.

[0003] Currently, sequence recommendation is an important direction of recommendation system research. Sequence recommendation can predict user's future behavior and preference by analyzing user's historical behavior sequence, and then accurate recommendation results can be generated. In recent years, with the continuous development of sequence recommendation, many researchers have proposed some high-performance sequence recommendation models. However, in real application scenarios, the following problems still affect the recommendation performance: First, data sparsity and cold start problem are common challenges in recommendation systems, especially in sequence recommendation tasks. Data sparsity refers to the fact that user-item interaction data is very sparse, that is, most users only interact with a small number of items. The cold start problem refers to the fact that when new users or new items are added, the recommendation system cannot give accurate recommendations due to the lack of sufficient historical data. Existing recommendation system models often cannot make effective recommendations when facing the above two situations, and this problem has become a long-term technical difficulty in recommendation tasks. Second, personalized and diverse recommendation. Existing recommendation models cluster users according to similar categories to enable collaborative learning between users, which can alleviate the sparsity problem but at the cost of personalized performance. Since user behavior is full of uncertainty, it is difficult for the recommendation system to predict the impact of the user in the real world, resulting in interaction behavior that does not follow the prediction model based on historical interaction. Single recommendation based on user historical interaction information not only easily leads to over-recommendation, but also cannot solve the uncertainty problem. Third, the noise problem in user historical interaction sequence has been seriously ignored. The recommendation model constructed solely based on historical interaction information has certain defects, because there are many noise factors in historical interaction information that affect the accuracy of the recommendation model, such as user's unintentional clicks, accidental touches, etc. However, existing recommendation methods mostly ignore the processing of original interaction data, resulting in the input of original data with noise into the model. As the model processes the original data, the influence of noise information on the performance of the model is inevitably enlarged, so that the recommendation system cannot make high-quality recommendations for users. SUMMARY

[0004] To solve the above problems, the purpose of the present application is to provide a sequence recommendation method based on contrastive learning and variational autoencoder, which uses user's historical interaction sequence as input, and generates items of interest to the user through model processing that combines contrastive learning paradigm and variational autoencoder.

[0005] Based on the principle of variational inference, the present application deduces the single-variable ELBO (Evidence Lower Bound) in VAE into a double-variable ELBO, and naturally introduces contrastive learning into the double-variable ELBO. Through the double-pipe structure, the contrastive loss is reconstructed, and the mutual information maximization item generated after optimization can effectively optimize the contrastive learning.

[0006] To achieve the above object of the application, the application adopts the following technical solutions:

[0007] A sequence recommendation method based on contrastive learning and variational autoencoder, comprising the following steps:

[0008] S1. Generating sequence embedding according to the historical interaction sequence of the user;

[0009] S2. Performing time-frequency domain conversion on the original data branch according to the FFT algorithm;

[0010] S3. Performing data sliding enhancement on the data enhancement branch according to the EMA algorithm;

[0011] S4. Performing model layer enhancement processing on the data processed in step S3 according to the random deep enhancement strategy;

[0012] S5. According to the Transformer encoder, respectively encoding the data of the original data branch processed in step S2 and the enhanced data branch processed in step S4 to obtain the mean and variance of the respective data stream branch data;

[0013] S6. According to the contrastive learning strategy, maximizing the positive and negative sample loss;

[0014] S7. Generating a new item sequence according to the shared Transformer decoder;

[0015] S8. Constructing a sequence recommendation model based on contrastive learning and variational autoencoder.

[0016] Further, the above step S1 comprises:

[0017] Set the user set as The item set is The historical interaction sequence of each user u is Wherein, Indicates the item interacted by the user u at time t; T indicates the length of the user interaction sequence; the CBOW is used to learn the item embedding, which is represented as:

[0018]

[0019] Wherein, Indicates the embedding vector of the target item x t ; Indicates the average value of the context item vector; N indicates the total number of items;

[0020] By maintaining a learnable item embedding matrix The items are mapped to a low-dimensional dense space, Wherein, |N| indicates the total number of items; d indicates the dimension of the item feature embedding;

[0021] For user interaction sequences longer than T, the strategy of retaining the most recent T interaction sequences is adopted.

[0022] For user interaction sequences with a length less than T, pad the leftmost portion of the sequence with tokens to make its length reach T; Among them, ◇ represents the learnable mark of filling; represents the kth historical interaction item of user u;

[0023] Given a T-length user-item interaction sequence, we maintain a learnable item embedding matrix Apply the embedding lookup operation to initially construct the embedding matrix Where [d] represents the item embedding dimension; The embedding vector representation of the interaction items of user u at the tth step;

[0024] The embedding layer is defined as:

[0025]

[0026] Where P represents the learnable position encoding matrix; p n[d] represents the learnable positional encoding embedding of n steps; d represents the dimension of the positional encoding embedding, which is the same as the item embedding dimension.

[0027] Furthermore, the above step S2 includes:

[0028] For a conversion from the time domain E l The t-th dimension feature representation of the item Where s∈[0,N-1] represents the number of items and the convolution in the time domain is defined as:

[0029]

[0030] in, s represents the sequence length; After Fourier transformation, Represented in the frequency domain

[0031]

[0032] in, Represents the mth weight number of the weight matrix of the filtering algorithm on the t-dimensional feature; Represents the time domain E l The mth digital mapping representation on the tth dimension feature in the frequency domain;

[0033] According to the above formula (4), it is proved that Among them, * represents circular convolution in the time domain; ⊙ represents element-by-element multiplication in the frequency domain;

[0034] Get U and Multiplication is equivalent to time domain E l Using convolution kernel Circular convolution.

[0035] Furthermore, the above step S3 includes:

[0036] Input a user-item historical interaction sequence processed by the embedding layer in step S1 Interaction sequence value at time t Enhanced by the EMA algorithm:

[0037]

[0038] Among them, γ∈(0,1) represents the smoothing factor, which controls the contribution of the historical interaction sequence value to the EMA value at the current time t;

[0039] For a user's historical interaction sequence When k = h, the contribution of the previous sequence value to the current sequence value is less than a specific threshold ∈, and the moving window size S is approximately determined at this time; the formula is described as follows:

[0040]

[0041] Here, h∈[0,t] represents a certain interaction moment when the set threshold is met. At this time, the influence of the sequence value before moment h on the current sequence value is approximately equal to 0, thereby determining the window size S=h.

[0042] Furthermore, the above step S4 includes:

[0043] Use binomial distribution to represent each depth vector d={d1,d2,…,d L}, where d i Indicates whether the i-th layer is retained The probability of the depth vector is expressed using a binomial distribution:

[0044]

[0045] Where d represents the depth vector; p i represents the probability that the i-th layer is retained; (1-p i ) represents the probability of each layer being discarded; the whole process follows the parameter p ibinomial distribution; by the above formula (8), input a sequence processed by the EMA algorithm in step S3, and obtain the data distribution processed by the random depth reinforcement strategy coding

[0046] Further, the above step S5 includes:

[0047] Using multi-head self-attention as the encoder of VAE, the posterior distribution q φ (z|x) is inferred by encoding the embedding layer data

[0048] Assuming that the variational approximation posterior distribution q φ (z|x) is subject to a diagonal covariance multivariate Gaussian distribution of z For the user interaction sequence q φ (z|x) is expressed as:

[0049]

[0050] wherein, and are two transformer-based encoders in VAE, responsible for estimating the mean μ and variance σ representing the posterior distribution;

[0051] The Gaussian distribution fitted by the neural network is obtained The posterior distribution q φ (z|x) represented by the Gaussian distribution is obtained

[0052] Specifically, the multi-head self-attention receives input from the embedding layer The formula of the multi-head self-attention mechanism is as follows:

[0053]

[0054] wherein, d represents the dimension of the lookup vector; represents a learnable weight matrix; using f(x) = max(0, x) as a nonlinear activation function, while applying residual connection and Dropout to stack multi-head self-attention layers to form an encoding module; by inputting the embedding matrix, q φ (z|x) represented by the mean μ and variance σ is obtained.

[0055] Further, the above step S6 includes:

[0056] By maximizing the InfoNCE loss function in contrastive learning The lower bound of mutual information between z and under the joint distribution is as follows:

[0057]

[0058] wherein, with denote the positive, negative samples of the sample z i , respectively; τ denotes the temperature coefficient.

[0059] Further, the step S7 described above comprises:

[0060] The encoder reparameterized hidden variable z is sampled from the variational distribution q φ (z|x) and the original data x is reconstructed, and the reconstruction term is:

[0061]

[0062] wherein, z i = μ i + σ i ⊙ ∈, through the decoding reconstruction process, the possible interaction item of the user at T+1 step is predicted;

[0063] The target loss function is obtained as:

[0064]

[0065] wherein, is the reconstruction loss, and represents the negative log-likelihood of the generated data x under the conditional distribution q φ (z|x) of the latent variable z; is the divergence loss;

[0066] The analytical solution can be directly calculated as:

[0067]

[0068] The model parameters are optimized by minimizing the loss function formula (16), and the next time interaction item of the user is predicted and generated.

[0069] Further, the step S8 described above comprises:

[0070] The double pipeline contrastive variational autoencoder has input and latent variable wherein, t depends on z, depends on and z and are independent, thus, there are

[0071] The double pipeline contrastive variational inference is naturally applied in the double pipeline contrastive variational autoencoder, and the complete loss function of the double pipeline contrastive variational autoencoder is obtained:

[0072]

[0073] wherein beta represents a hyperparameter.

[0074] The application also provides a sequence recommendation system based on contrast learning and variational autoencoder, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the sequence recommendation method of the application when executing the computer program.

[0075] Due to the adoption of the technical solutions described above, the application has the following advantages:

[0076] The sequence recommendation method based on contrast learning and variational autoencoder can make personalized recommendations for users according to their historical interaction sequences, the VAE can learn the latent representation space of the user interaction items according to the historical interaction items of each user, the high-dimensional user-item sequence is mapped into a low-dimensional latent space, so that the model learns the structure and features of the single user interaction item, thereby making high-quality recommended items for the user; the VAE has the feature of predicting the uncertainty of the input data, which can alleviate the uncertainty problem when the user interacts with the item; the contrast learning is integrated into the VAE model with bivariate ELBO to form a double-pipeline structure, improve the continuity of the latent space in the VAE model, and solve the problem of mode collapse and the problem of being unable to alleviate data sparsity and cold start when the VAE faces the long tail problem; the contrast learning encourages similar samples to be close to each other in the latent space, so that the samples in the latent space are more continuous, thereby making the interpolation operation in the latent space more meaningful; the contrast learning encourages the generated samples to be more uniformly distributed in the latent space rather than excessively concentrated near a few modes, which helps the VAE to reduce the occurrence of mode collapse.

[0077] The sequence recommendation method based on contrast learning and variational autoencoder can effectively alleviate the problems of data sparsity, user behavior uncertainty and cold start by fusing contrast learning and variational autoencoder, and has better recommendation quality; the contrast learning paradigm is naturally integrated into the VAE to solve the mode collapse problem of the VAE, and the contrast samples are introduced to help the model generate more accurate recommendation results; for the noise problem existing in the original data, the sequence data is converted to the frequency domain, then the adaptive filtering algorithm is used to attenuate the noise, and finally converted back to the time domain; two kinds of enhancement methods with universality and more suitable for sequence data enhancement are used, so that the contrast learning of the double-pipeline structure has difference, the contrast learning framework adapts to the recommendation task, can be efficiently performed, and thus the DCCVAE model is obtained, and the recommendation accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1is a flow chart of a sequence recommendation method based on contrastive learning and variational autoencoder according to the present application;

[0079] Figure 2 is an architecture diagram of a recommendation framework of a DCCVAE model. DETAILED DESCRIPTION

[0080] The technical solutions of the present application are further described in detail below with the aid of the accompanying drawings and examples.

[0081] As shown in Figure 1 , 2 a sequence recommendation method based on contrastive learning and variational autoencoder comprises the following steps:

[0082] S1, generating a sequence embedding according to the historical interaction sequence of the user; in this embodiment, the pre-trained embedding vector of the historical interaction item is obtained by using Word2Vec (w2v); including:

[0083] Set the user set as the item set as the historical interaction sequence of each user u as wherein, denotes the item interacted by the user u at time t; T denotes the length of the user interaction sequence; the CBOW (Continuous Bag of Words) is used to learn the item embedding representation:

[0084]

[0085] wherein, denotes the embedding vector of the target item x t ; denotes the average value of the context item vector; N denotes the total number of items;

[0086] By maintaining a learnable item embedding matrix Q, the item is mapped to a low-dimensional dense space, wherein, |N| denotes the total number of items; d denotes the dimension of the item feature embedding;

[0087] For the user interaction sequence with a length greater than T, the strategy of retaining the last T interaction sequences is adopted

[0088] For the user interaction sequence with a length less than T, fill the mark at the leftmost side of the sequence to make its length reach T; wherein, represents the filled learnable mark; denotes the item interacted by the user u at time k;

[0089] Given a user history item interaction sequence of T-length, an item embedding matrix is learned by maintaining a learnable item embedding matrix The embedding matrix is initially constructed by applying embedding lookup operations Where [d] represents the item embedding dimension; represents the vector representation of the item interacted by the user at step t, which is d-dimensional;

[0090] The form of the embedding layer is defined as:

[0091]

[0092] Where P represents a learnable position encoding matrix used to obtain the position relationship between the interaction items; p n[d] represents the n-step learnable position encoding embedding; d represents the dimension of the position encoding embedding, which is the same as the item embedding dimension, so as to facilitate the addition operation at the element level;

[0093] S2, according to the FFT (Fast Fourier Transform) algorithm, the time-frequency domain conversion is carried out on the original data branch, and the user-item interaction sequence in the time domain space is converted into the user-item representation in the frequency domain space, which is used for denoising processing to ensure the reliability of the data; including:

[0094] For the t-th dimensional feature representation of a project from the converted time domain E l Where s∈[0,N-1] represents the number of projects, and the convolution in the time domain is defined as:

[0095]

[0096] Where, s represents the sequence length; After Fourier transform, the frequency domain representation is

[0097]

[0098] Where, represents the m-th weight number on the t-dimensional feature in the weight matrix of the filtering algorithm; represents the m-th digital mapping representation on the t-dimensional feature from the time domain E l in the frequency domain;

[0099] According to the above formula (4), it is proved that Where * represents the circular convolution in the time domain; and represents the element-wise multiplication operation in the frequency domain;

[0100] U in the frequency domain and​ Multiplication is equivalent to the time domain E l Using convolution kernel The circular convolution kernel is symmetrical in all directions and has a large receptive field, which can more effectively extract key information in the sequence and reduce information loss and redundancy.

[0101] S3, according to the EMA (Exponential Moving Average) algorithm, the data enhancement branch is enhanced by sliding data; including:

[0102] Input a user-item historical interaction sequence processed by the embedding layer in step S1 The interaction sequence value at time t Enhanced by the EMA algorithm:

[0103]

[0104] Where, γ∈(0,1) represents a smoothing factor, controlling the contribution of historical interaction sequence values to the current EMA value at time t; to understand the influence of the γ parameter on the EMA enhancement strategy, further expand formula (5), we have:

[0105]

[0106] For a user historical interaction sequence When k=h, the contribution of previous sequence values to the current sequence value is less than a certain threshold ∈ (usually ∈ is very small), at this time the moving window size S is approximately determined; The formula is described as follows:

[0107]

[0108] Where, h∈[0,t] represents a certain interaction time that meets the set threshold, at this time the influence of sequence values before h steps on the current sequence value is approximately equal to 0, thus determining the window size S=h;

[0109] S4, according to the random depth enhancement strategy, the data processed in step S3 is enhanced at the model level; In the encoding stage of the second pipeline branch, the historical interaction items are randomly encoded by different depths of transformer, so that the data is different from the first pipeline branch, achieving the purpose of enhancement; The encoding process of different depths is considered during encoding, and the overall characteristics of the data are fully considered; Deeper network will encode more abstract item potential representation, and shallower neural network will encode detailed item potential representation, both of which will not affect the overall trend and important features of the data; Specifically:

[0110] The probability of each depth vector d = {d1, d2, …, di, …, dD} is represented using a binomial distribution, where di L i represents whether the i-th layer is retained Each depth vector is independently random, so the retention or discard of each layer is considered as an independent Bernoulli experiment, and therefore the probability of the depth vector is represented using a binomial distribution:

[0111]

[0112] where d represents the depth vector; p i represents the probability that the i-th layer is retained; (1-p i ) represents the probability that each layer is discarded; the entire process follows a binomial distribution with parameter p i Through the above formula (8), an input sequence after the EMA algorithm enhancement processing in step S3 is obtained, and a data distribution after random depth enhancement strategy coding processing is obtained represents the mean, represents the variance;

[0113] S5, according to the Transformer encoder, respectively, the data of the original data branch processed in step S2 and the enhanced data branch processed in step S4 are encoded to obtain the mean and variance of each data stream branch data; including:

[0114] Use Multi-Head Self-Attention as the encoder of VAE (Variational AutoEncoder), by encoding the embedding layer data, the posterior distribution q φ (z|x) is inferred.

[0115] Assume that the variational approximation posterior distribution q φ (z|x) is subject to a diagonal covariance multivariate Gaussian distribution of z For the user interaction sequence q φ (z|x) is expressed as:

[0116]

[0117] where,

[0118]

[0119] where, and ​respectively, are two transformer-based encoders in VAE, i.e., a first transformer encoder and a second transformer encoder, responsible for estimating the mean μ and variance σ of the represented posterior distribution;

[0120] obtained by fitting a Gaussian distribution by a neural network the represented posterior distribution q φ (z|x);

[0121] Specifically, the multi-head self-attention receives input from the embedding layer Learn the long-distance dependency of the item, solve the long-distance attention forgetting and parameter sharing problem; the formula of the multi-head self-attention mechanism is as follows:

[0122]

[0123] Where d represents the dimension of the lookup vector, which is used to control the attention score in a proper range, so that it has better gradient properties; represents a learnable weight matrix; using f(x)=max(0,x) as a nonlinear activation function, while applying residual connection and Dropout to stack multi-head self-attention layers to form an encoding module; by input the embedding matrix of the input φ (z|x);

[0124] S6, according to the contrast learning strategy, maximize the positive and negative sample loss; including:

[0125] maximize the InfoNCE loss function in contrast learning the joint distribution of z and the lower bound of mutual information, the formula is as follows:

[0126]

[0127] Where, and respectively represent the positive and negative samples of the sample z i ; τ represents the temperature coefficient, which is used to adjust the scale of similarity;

[0128] S7, generate a new item sequence according to the shared Transformer decoder; including:

[0129] Also use Transformer, and stack multi-head self-attention layers in it, the model structure of the decoder is the same as that of the Transformer encoder; the decoder samples the reparameterized latent variable z in the variational distribution q φ (z|x) and reconstructs the original data x, the reconstruction term is:

[0130]

[0131] where z i = μ i + σ i ⊙ ∈, the expectation is difficult to calculate directly, so approximate this expectation; assume that from q φ (z|x) extract T samples, denoted as z = {z1, z2…z T}, the expectation is expressed as where T is the length of the user interaction sequence, and the sample is calculated by sampling from the variational distribution q φ (z|x) to approximate the expectation The average value of the log p(x|z) corresponding to these samples; through the reconstruction process of the decoder, the possible interaction items of the user at the next time are predicted;

[0132] At this time, the loss function is obtained But in the optimization process of the model, it is usually through the optimization algorithm such as gradient descent to minimize the loss function, not gradient ascent, so the target loss function is further rewritten as:

[0133]

[0134] where, is the reconstruction loss, which represents the negative log-likelihood of generating data x under the conditional distribution q φ (z|x) of latent variable z; is the divergence loss;

[0135] The analytical solution can be calculated directly:

[0136]

[0137] Optimize the model parameters by minimizing the loss function formula (16), and generate the next interaction item for the user;

[0138] S8, constructing a sequence recommendation model DCCVAE based on contrastive learning and variational autoencoder;

[0139] Integrate the user sequence interaction relationship while incorporating contrastive learning and variational autoencoder to obtain the sequence recommendation model DCCVAE; Specifically:

[0140] The dual-channel contrastive variational autoencoder (Dual-Channel Contrastive Varuatuonal Autoencoder, referred to as DCCVAE) has input and latent variable where x depends on z, depends on and z and are independent, respectively, so that The derivation of the Dual-Channel Contrastive Inference (DCCVI) is as follows:

[0141] First, we give the expression:

[0142]

[0143] Introducing and the log-likelihood function, we have by Jensen's inequality (for a concave function log):

[0144]

[0145]

[0146] The third term on the right side of equation (18) is further written as:

[0147]

[0148] From equation (18) and equation (19), we have

[0149]

[0150] where

[0151] and are the reconstruction term and the KL divergence, respectively;

[0152] is the mutual information (MI) term about

[0153] However, this term involves the calculation of the real prior distribution p(z) and which are complex and difficult to express analytically, so we let q(z)≈p(z)and to greatly simplify the calculation process; thus,

[0154] The MI term is replaced with Under the real data distribution, we have where denotes the mutual information of z and

[0155] ​​The InfoNCE loss function in the contrastive learning is maximized by being naturally introduced The joint distribution of z and The lower bound of mutual information is as follows:

[0156]

[0157] wherein, and respectively represent the positive and negative samples of the sample z i ; τ is a temperature coefficient for adjusting the scale of the similarity;

[0158] Based on the above theoretical derivation, the DCCVI is obtained and is naturally applied in the DCCVAE, and the complete loss function of the DCCVAE is obtained by combining formula (16) and formula (17):

[0159]

[0160] wherein, β represents a hyperparameter for adjusting the weight of the InfoNCE loss in the total loss, so as to ensure that the relative importance of each loss term in the training process is reasonably balanced;

[0161] According to the obtained DCCVAE model, the recommendation is performed.

[0162] As Figure 2 shown, the recommendation framework of the above DCCVAE model is as follows:

[0163] The original data stream pipeline branch is first processed by the embedding layer, and then is transmitted into the FFT denoising module, and after denoising, is encoded by the first Transformer encoder;

[0164] Then, after the enhanced data stream pipeline branch is processed by the embedding layer, it is first transmitted through the EMA data enhancement module, and then is transmitted through the random depth enhancement module and is encoded by the second Transformer encoder; the two branches are encoded by using different encoders;

[0165] Then, the two data stream branches are re-parameterized to obtain the latent representation of the corresponding vector, at this time, through the contrastive learning, the similar items are as close as possible, and the dissimilar items are as far as possible, so as to maximize the similarity;

[0166] Then, the two data stream branches are transmitted into the decoder for decoding operation, and in the decoding, the same decoder is shared;

[0167] Finally, the items that the user may be interested in are reconstructed, and are sorted by scores and then are recommended to the user.

[0168] The above merely describes preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A sequence recommendation method based on contrastive learning and variational autoencoder, characterized by: It includes the following steps: Step 1: Generate sequence embedding based on the user's historical interaction sequence; Step 2: According to the FFT algorithm, the sequence embedding is transformed into the time-frequency domain through the original data branch; for a sequence from the time domain E l The t-th dimension feature representation of the item The convolution in the time domain is defined as: in, T represents the sequence length; After Fourier transformation, Represented in the frequency domain in, Represents the mth weight number of the weight matrix of the filtering algorithm on the t-dimensional feature; Represents the time domain E l The mth digital mapping representation on the tth dimension feature in the frequency domain; According to the above formula (4), it is proved that Among them, * represents circular convolution in the time domain; ⊙ represents element-by-element multiplication in the frequency domain; Get U and Multiplication equals time domain E l Using convolution kernel Circular convolution of Step 3: Perform data sliding enhancement on the sequence embedding through the data enhancement branch according to the EMA algorithm; Step 4: Perform model-level enhancement on the data processed in step 3 according to the random depth enhancement strategy; including: Use binomial distribution to represent each depth vector θ={θ1,θ2,…,θ L }, where θ i Indicates whether the i-th layer is retained, that is, The probability of the depth vector is expressed using a binomial distribution: Among them, θ represents the depth vector; p b represents the probability that the bth layer is retained; (1-p b ) represents the probability of being discarded; the whole process follows the parameter p b Binomial distribution; through the above formula (8), input a sequence enhanced by the EMA algorithm in step 3, and obtain the data distribution after random depth enhancement strategy encoding Step 5: Encode the original data branch processed in step 2 and the enhanced data branch processed in step 4 according to the Transformer encoder to obtain the mean and variance of the data of each data stream branch. Step 6: Maximize the loss of positive and negative samples according to the contrastive learning strategy; Step 7: Generate a new item sequence based on the shared Transformer decoder; including: From the variational distribution q φ (z|x) samples the reparameterized latent variable z in the encoder and reconstructs the original data x, the reconstruction term for: Among them, z i =μ i +σ i ⊙ρ, predicting the user's next possible interaction item through the decoding reconstruction process; The target loss function is obtained as: in, is the reconstruction loss, which represents the latent variable z in the variational distribution q φ The negative log-likelihood of generating data x under (z|x); is the divergence loss; The analytical solution can be directly calculated: Optimize the model parameters by minimizing the loss function formula (16) and generate the next interactive items for the user; Step 8: Naturally apply dual-pipeline contrastive variational inference to the dual-pipeline contrastive variational autoencoder to obtain the complete loss function of the dual-pipeline contrastive variational autoencoder.

2. The sequence recommendation method based on contrastive learning and variational autoencoder according to claim 1, characterized in that: The step 1 comprises: Set the user collection to The item set is I={i1,i2,…,i |I| }, each user u k The historical interaction sequence of in, Represents user u k The mth interaction item in the historical interaction sequence of ; S represents the number of elements in the user interaction sequence; CBOW is used to learn the item embedding representation: in, Represents the target item x m Embedding vector of represents the average value of the context item vector, and x' represents other items in the same sequence; By maintaining a learnable item embedding matrix Map items into a low-dimensional dense space, Where d represents the dimension of item feature embedding; Set the maximum length of the user's historical interaction sequence to T. For user interaction sequences longer than T, the strategy of retaining the most recent T interaction sequences is adopted, that is, For user interaction sequences with a length less than T, fill the leftmost tag of the sequence to make its length reach T, that is, Among them, ◇ represents the learnable mark of filling; Given a user-item interaction sequence of length T, we maintain a learnable item embedding matrix Apply the embedding lookup operation to initially construct the embedding matrix Among them, d represents the dimension of item feature embedding, Represents user u k The embedding vector representation of the mth interaction item; The embedding layer is defined as: Where P represents the learnable position encoding matrix; p n，[d] represents the learnable positional encoding embedding of n steps; d represents the dimension of the item feature embedding.

3. The sequence recommendation method based on contrastive learning and variational autoencoder according to claim 1, characterized in that: The step 3 comprises: Input a user history interaction sequence processed by the embedding layer in step 1 The mth interaction sequence value Enhanced by the EMA algorithm: Among them, γ∈(0,1) represents the smoothing factor, which controls the contribution of the historical interaction sequence value to the EMA value of the current m-th interaction item; To understand the impact of the γ parameter on the EMA enhancement strategy, we further expand formula (5) to: For a user's historical interaction sequence When a=h, the contribution of the previous sequence value to the current sequence value is less than the threshold β. At this time, the moving window size C is approximately determined; the formula is described as follows: Here, h∈[0,t] represents the hth interaction point when the set threshold is met. At this time, the influence of the sequence value before h on the current sequence value is approximately equal to 0, thereby determining the window size to be C=h.

4. The sequence recommendation method based on contrastive learning and variational autoencoder according to claim 1, characterized in that: The step 5 comprises: Using multi-head self-attention as the encoder of VAE, the posterior distribution q is inferred by encoding the embedding layer data. φ (z|x); Assume that the variational approximation posterior distribution q φ (z|x) follows a multivariate Gaussian distribution with diagonal covariance of z For user interaction sequences q φ (z|x) is expressed as: in, in, and They are two transformer-based encoders in VAE, responsible for estimating the mean μ and variance σ of the posterior distribution; Get the Gaussian distribution fitted by the neural network The posterior distribution q represented by φ (z|x); Multi-head self-attention receives input from the embedding layer The formula of the multi-head self-attention mechanism is as follows: Where d represents the dimension of item feature embedding; Represents a learnable weight matrix; uses f(x) = max(0,x) as a nonlinear activation function, and applies residual connections and Dropout stacking multi-head self-attention layers to form an encoding module; by input The embedding matrix of q is represented by mean μ and variance σ φ (z|x).

5. The sequence recommendation method based on contrastive learning and variational autoencoder according to claim 1, characterized in that: The step 6 comprises: Maximizing the InfoNCE loss function in contrastive learning Under the joint distribution z and The lower bound of mutual information is as follows: in, represents z i positive samples of ; τ represents the temperature coefficient.

6. The sequence recommendation method based on contrastive learning and variational autoencoder according to claim 1, characterized in that: The step 8 comprises: The dual-pipeline contrast variational autoencoder has input With latent variables Among them, x depends on z, Depends on And z and are independent of each other, so we have The complete loss function of the dual-pipeline contrast variational autoencoder: Here, β represents a hyperparameter.

7. A sequence recommendation system based on contrastive learning and variational autoencoder, characterized by: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the sequence recommendation method according to any one of claims 1 to 6 are implemented.

Images