Contrast learning depolarization intelligent recommendation method based on diffusion model

Abstract

The invention provides a diffusion model-based comparative learning depolarization intelligent recommendation method, which comprises the following steps of: S1, extracting prejudice article characteristics, article popularity characteristics and article semantic characteristics from an interaction graph of user articles, and combining the prejudice article characteristics, the article popularity characteristics and the article semantic characteristics of individuals and neighbor nodes to obtain a recommendation result; coding the representation of the user and the article by adopting hypergraph convolution to obtain a coded embedding; s2, the encoded embedding is input into a diffusion model, and the diffusion model outputs final outputs of different embedding; then, carrying out aggregation operation on the final outputs of different embedding to obtain user article interaction embedding representation; and calculating scores of the interactive items recommended by the user based on the user item interaction embedding representation, and arranging the scores in a descending order to generate a recommendation list of related candidate items. According to the method, the multilateral complex relation can be captured, and more randomness and diversity are added in the graph convolutional neural network by introducing the diffusion model.

Description

Technical Field

[0001] The present invention relates to the field of intelligent recommendation, and in particular to a contrastive learning debiasing intelligent recommendation method based on a diffusion model. Background Art

[0002] With the problem of information overload in online applications brought about by the development of the Internet, high-quality personalized recommendation systems have become an indispensable key role in daily life. Recommendation systems generally improve the quality of recommendation results by mining historical behavioral data of user-item interactions. Most of them use graph convolutional neural networks to enhance the embedded representation of user-item interactions and capture collaborative signals for recommendation. In deep learning-based recommendation systems, the use of neural network learning can effectively represent complex user behaviors and item features, explore user interests and provide accurate recommendations. Graph representation learning can more effectively capture multi-hop connections and adjacent node information between user items, and learn the representation of users and items through graph neural networks.

[0003] Most existing recommendation systems based on graph neural networks make node embeddings similar after multi-layer convolution operations are enhanced, and the node representations become less distinguishable. This results in the model being unable to effectively capture the personalized differences between users and items, affecting the performance and accuracy of recommendations. In graph-based recommendation models, attention mechanisms and convolutional neural network layers are often added to aggregate neighbor node information to update node representations, making them more and more similar. In certain cases, they will tend to be consistent, resulting in serious node embedding smoothing problems. Summary of the invention

[0004] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a contrastive learning debiasing intelligent recommendation method based on a diffusion model.

[0005] In order to achieve the above-mentioned object of the present invention, the present invention provides a contrastive learning debiasing intelligent recommendation method based on a diffusion model, comprising the following steps:

[0006] S1, extract biased item features, item popularity features and item semantic features from the user-item interaction graph, combine the biased item features, item popularity features and item semantic features of the individual and neighboring nodes, and then use hypergraph convolution to encode the representations of the user and item to obtain the encoded embedding, which includes the user's embedded representation x and the item's embedded representation y;

[0007] Three different token information, personal bias information, semantic information, and popular item information, are considered and injected into the hypergraph convolution operation as learnable perturbations to decouple the popularity bias and user individual bias in user-item interactions and generate debiased user and item representations.

[0008] S2, input the encoded embedding into the diffusion model, and the diffusion model outputs the final output of different embeddings; then the final output of the different embeddings is aggregated to obtain the user-item interaction embedding representation; then the score of the user-recommended interactive item is calculated based on the user-item interaction embedding representation, and the score is arranged in descending order to generate a recommendation list of related candidate items.

[0009] Preferably, S1 comprises the following steps:

[0010] S1-1, convert the user-item interaction graph information into an embedded representation of the user-item interaction, and derive the historical interaction sequence H between user u and item i u,i , determine the user u who interacts with the current item i, and then obtain the item's embedding representation y through uniformly distributed random sampling, where the item's embedding value is randomly generated within the uniform distribution range;

[0011] S1-2, in the historical behavior sequence of user-item interaction, according to the items interacted with the current user, the embedding vectors of all the interacted items are integrated to obtain the user embedding representation x;

[0012] S1-3, the embedded representation of items that interact with users is integrated. It separates the user bias in the item embedding representation based on the item bias characteristics, item popularity characteristics and item semantic information in the user interaction process, fits the biased interaction, and performs self-attention mechanism enhancement to obtain the final user embedding feature representation, and obtains the event sequence representation (x, y) of the user-item interaction characteristics.

[0013] Preferably, the calculation formulas for the item embedding representation y and the user embedding representation x are as follows:

[0014]

[0015] Among them, x is the user embedding representation;

[0016] y is the embedding representation of the item;

[0017] y i is the embedding representation of the i-th item, and is the item that user u interacts with;

[0018] H u,i is the historical interaction sequence between user u and item i;

[0019] r u is the embedding vector of individual user biases in the original user-item interaction graph;

[0020] ||r u ‖ is r u The module length;

[0021] sign() is the sign function;

[0022] ⊙ is the Hadamard product symbol;

[0023] Hgc(·) is a hypergraph convolution, which multiplies the normalized user embedding with the item embedding, combines the information of the two, and passes the updated embedding representation as input through the encoder, fusing the user embedding and the adjacency matrix to generate the final user embedding representation;

[0024] is the adjacency matrix;

[0025] Encoder() is a hypergraph convolutional coding representation.

[0026] Preferably, the diffusion process of the diffusion model includes a forward process and a reverse process. In the forward process, Gaussian noise is gradually added to destroy the user's interaction history. In this process, noise limits are set to retain user personalized information. In the reverse process, the original interaction data is gradually restored from the interaction history destroyed by noise through a parameterized neural network, and the restored interaction probability is used to sort and recommend non-interacted items.

[0027] In the forward diffusion process, the interaction z between the currently selected user and a set of items is defined u =(x,y) u =[(x,y0),(x,y1),…,(x,y I )], initialize the diffusion process γ0 = z u By gradually introducing Gaussian noise in S steps, taking s as the index, the whole process is parameterized, and noise is gradually introduced into the user-item interaction, so that the original interaction evolves into a noisy state.

[0028]

[0029] Among them, δ(γ s |γ s-1 ) is a forward process that destroys the user's interaction history by gradually adding Gaussian noise.

[0030] is a Gaussian distribution;

[0031] s∈{0,…,S} is the current state γ at each time step s-1 By adding noise, it is transferred to the state γ s Here the noise follows a Gaussian distribution The mean is The variance is μ s I. Introducing two hyperparameters and The noise in the current state is controlled by a linear noise scheduler, and noise is gradually added to mimic the randomness in user-item interactions.

[0032]

[0033] Among them, γ s is the state of the current step in diffusion;

[0034] γ0 is the state of the initial step;

[0035] is the scaling factor, indicating the intensity of added noise;

[0036] It controls the scale of Gaussian noise added in each step (i.e., s steps), and its value range is (0,1);

[0037] ε is standard Gaussian noise, which obeys a Gaussian distribution with a mean of 0 and a covariance matrix of the identity matrix I

[0038] Perform a linear noise schedule for the current state.

[0039]

[0040] in, It controls the scale of Gaussian noise added in each step (i.e., s steps), and its value range is (0,1);

[0041] The hyperparameter c∈[0,1] controls the noise scale, and the two hyperparameters η max and η min Represents the upper and lower bounds for adding noise.

[0042] In the back diffusion process, the current noise state is eliminated and the initial state is restored, so that the diffusion model can effectively capture the subtle changes in the complex generation process. s Initially, the denoising transition step gradually restores the user-item interactions.

[0043]

[0044] Among them, p θ (γ s-1 |γ s ) represents the reverse process in diffusion, predicting the conditional probability distribution of the previous state given the current state. θ (γ s ,s) is the mean Σ θ (γ s ,s) is the covariance matrix, which are the mean and covariance predicted by the parameterized neural network based on the current state and time step.

[0045] Represents Gaussian distribution, and the content in brackets specifically demonstrates how noise is gradually added under different conditions and finally conforms to Gaussian distribution.

[0046] γ s-1 represents the state of the previous time step;

[0047] χ θ (γ s ,s) and ∑ θ (γ s ,s) represent the mean and covariance of the predicted Gaussian distribution, respectively, generated by two neural networks with learnable parameters.

[0048] Preferably, the diffusion model further comprises diffusion optimization:

[0049] To effectively guide the backward graph diffusion process to learn θ, the Evidence Lower Bound (ELBO) of the negative log-likelihood of user interactions is maximized. This is achieved by optimizing the log-likelihood function of the latent variable model, maximizing the log-likelihood of the result to enhance the generative ability of the model and maintain the stability and diversity of the model.

[0050] definition Reflects the model training objectives at different time steps during the diffusion model optimization process:

[0051]

[0052] in, represents the loss function at each time step;

[0053] is the noise scale parameter for time step s, controlling the amount of noise added at each step;

[0054] is the cumulative noise ratio parameter, which represents the cumulative noise ratio from the initial state to the current time step s;

[0055] ||||2 is the two-norm;

[0056] Based on the current state s Prediction of the initial state γ0.

[0057] Preferably, the scoring formula for calculating the user-recommended interactive items is as follows:

[0058] β= <x||μ·r u ,y||μ·r i > (14)

[0059] Among them, β represents the score of the interactive items recommended by the user;

[0060] <·,·> is the inner product operation;

[0061] || is the concatenation operation;

[0062] <x||μ·r u ,y||μ·r i >When calculating, first concatenate each part and then perform inner product operation;

[0063] x and y represent the user embedding representation and item embedding representation respectively;

[0064] μ represents a hyperparameter that adjusts the influence of popularity bias;

[0065] r u and r i denote the bias vectors of users and items respectively.

[0066] Preferably, during the aggregation process, a gradient optimization operation is performed to obtain a debiased embedding representation; wherein the loss function of the gradient optimization operation is:

[0067] Based on the historical behavior data of the user's previous interactions and the correlation between the user-item interactions in the interaction graph, the item embeddings that the user interacts with are obtained.

[0068]

[0069] in, is the embedding matrix of items that users have interacted with in the past;

[0070] is the embedding matrix of items that users may interact with in the future;

[0071] The score of the user's recommended items is obtained by the inner product of the user embedding representation containing only the items that the user interacted with in the past and the item representation that the user may interact with, and the cross entropy loss function is calculated:

[0072]

[0073] Among them, W1 T is the transpose of W1, which is the parameter weight matrix for calculating the cross entropy loss function;

[0074] σ is the activation function.

[0075] Based on the cross-entropy loss function, the loss gradient is adjusted by combining parameter weights and regularization, so that it can learn more accurate user-item interaction relationships that are not biased towards popular items or user personal preferences, reducing the risk of one-sided predictions of user-interaction items due to personal bias.

[0076] Preferably, it also includes: setting a parameter matrix Z, and adjusting the parameter matrix Z by gradient calculation to reduce the influence of the bias error, which is expressed as follows:

[0077]

[0078] in, T is the transpose symbol;

[0079] [·,·] is a concatenation symbol;

[0080] x and y represent the user embedding representation and item embedding representation respectively.

[0081] Preferably, the method further includes: S3, performing hypergraph convolution enhancement on the encoded embedding by using hypergraph convolution, enhancing signal propagation through hypergraph convolution enhancement, and hypergraph convolution can effectively select corresponding rows in hyperedges and effectively learn context information in hyperedges; reduce computational complexity, obtain user embedding representations and item embedding representations optimized by hypergraph convolution; then calculate the scores of the user recommended interactive items, and sort the scores in descending order to generate a recommendation list of related candidate items.

[0082] Preferably, the hypergraph convolution enhancement includes:

[0083] S2-1, extract feature information through multi-layer convolution propagation to enhance user embedding representation:

[0084]

[0085] Among them, M l is the enhanced output of the lth convolutional layer;

[0086] is the ELU activation function;

[0087] D -1 / 2 BD -1 / 2 is the symmetric normalized adjacency matrix;

[0088] B is the bipartite graph adjacency matrix;

[0089] D represents a diagonal matrix;

[0090] W2 is the parameter weight matrix of convolution propagation;

[0091] S2-2, using hypergraph convolution to model the contextual representation of users, by learning the contextual representation from the user hypergraph in the hypergraph, enhancing the signal propagation in the convolution process, adjusting the critical matrix and feature matrix according to the introduced hypergraph information, and selecting the specific rows and columns corresponding to the user;

[0092]

[0093] Among them, ψu is the hyperedge of user u;

[0094] T is the transpose symbol;

[0095] Norm is the L2 regularization calculation;

[0096] d is the dimension in the self-attention mechanism;

[0097] is the ELU activation function;

[0098] Q, K, and V represent the query, key, and value matrices in the attention mechanism, respectively;

[0099] W Q , W K is the corresponding parameter weight matrix;

[0100] Φ is a hyperparameter that controls the degree of attention;

[0101] S2-3, aggregate multiple subspace information through multi-head attention mechanism:

[0102]

[0103] Among them, W3 is the weight of each head in the multi-head attention mechanism;

[0104] k is the value of the number of attention heads;

[0105] n is the total number of attention heads;

[0106] It represents the result of input embedding layer l after being processed by different numbers of attention heads;

[0107] S2-4, after average pooling and normalization, the user embedding representation and item embedding representation optimized by hypergraph convolution are obtained;

[0108]

[0109] Among them, x is the user embedding representation optimized by hypergraph convolution;

[0110] M i It is an already processed embedding;

[0111] i∈H u,i The embedded selection belongs to the user-item interaction sequence;

[0112] || is the absolute value symbol;

[0113] H u,i Represents the historical interaction sequence between user u and item i;

[0114] y is the embedding representation of the item optimized by hypergraph convolution;

[0115] y i is the embedding representation of the i-th item, and is the item that user u interacts with;

[0116] sign() is the sign function;

[0117] ⊙ is the Hadamard product symbol;

[0118] ||r u || is the modulus length, This calculation is a regularization calculation.

[0119] Preferably, it also includes: using a contrastive self-supervised learning method to perform cross-view contrastive learning enhancement on the multi-feature embedding representation of user-item interaction, learning personalized features of different nodes to prevent bias; and using a hard negative supervision contrastive learning loss optimization strategy to select samples with different but similar labels within the current range as negative samples.

[0120] First, a sample selection strategy based on random probability is designed to calculate the distribution probability (0-1) of positive and negative samples in the sample set. If the sample label belongs to the current sample set, It is selected as a positive sample; otherwise, it is regarded as a negative sample; and multiple different negative sample selection structures are set in the same input sample space to assist in the selection of difficult negative samples in the contrastive learning framework;

[0121]

[0122] Among them, p(z + ) is the probability distribution of positive samples under different conditions;

[0123] z + 、z - They are positive samples and negative samples respectively;

[0124] z is the sample;

[0125] p U (z - ) is the probability distribution of negative samples under different conditions in the sample set;

[0126] Represents the probability distribution of selecting positive samples;

[0127] Represents the probability distribution of selecting negative samples;

[0128] Represent the positive sample set and the negative sample set respectively;

[0129] Under the current negative sample selection structure, the selection of difficult negative samples is performed: after the anchor point is determined, the cosine similarity between the anchor point and the sample is calculated according to the sample space set, the difficult negative sample set is constructed, and the negative probability distribution p of the difficult negative sample selection is obtained. HU (z - ):

[0130]

[0131] in, is to select the current limited range, that is, HU Negative sample expectation;

[0132] Ψ HU It is the specific range of the current condition sample selection;

[0133] exp() is an exponential operation with a natural base as the base;

[0134] w is the sample label;

[0135] is a set of sample labels;

[0136] d(z,ξ) is the similarity between the currently selected node (sample z) and the anchor point;

[0137] To control the threshold of sampling hardness;

[0138] In the sample space combined with the same anchor point, the method of selecting samples similar to the anchor point but of different types is used to construct a set of difficult negative samples and obtain the corresponding negative probability distribution:

[0139]

[0140] Among them, p S (z - ) is in Negative probability distribution of negative sample selection under conditions;

[0141] The label information of the sample w z And the label information w of the anchor ξ are comprehensively considered; finally, the above two difficult negative sample sets are combined to construct a difficult negative sample selection set based on the contrastive learning framework; this set selects negative samples within the specified range and meets two conditions: similar to the anchor point but with different label types, and meets the selection conditions within the specified selection range.

[0142]

[0143] Among them, p HS (z -) is the negative probability distribution of difficult negative samples that meet the conditions;

[0144] Ψ HS , S , HU They represent different limiting ranges in the sample selection process.

[0145] The contrast loss function of the difficult negative sample is:

[0146]

[0147] Among them, d(·,·) represents the similarity;

[0148] n is the number of negative samples;

[0149] k is the negative sample index;

[0150] Indicates difficult negative samples that fall within the current limited range;

[0151] In addition, in order to effectively balance the user's biased features on items, the features of items that the user is currently interacting with and the features of items that the user may interact with in the future, and prevent the problem of non-smooth node embedding, the weighting scheme is changed to balance the gradient and reduce the impact of popular items on the user's true preferences. At this time, the loss function is expressed as follows:

[0152]

[0153] Among them, λ is a hyperparameter that balances the weight of the item embedding matrix that interacts with the user;

[0154] λ1 is used for the embedding matrix of items that the user has interacted with;

[0155] ⊙ is the Hadamard product symbol;

[0156] η is a weight vector used to weight the losses of different items;

[0157] Θ is the regularization coefficient that controls the regularization weight;

[0158] is the previous loss function;

[0159] A collection of users;

[0160] H u,i is the historical interaction sequence between user u and item i;

[0161] In summary, due to the adoption of the above technical solutions, the present invention addresses the problem of over-smoothing of node embeddings in graph neural network recommendation systems. The application of hypergraph convolutional networks in encoding user and item embedding representations is studied, aiming to mine complex interaction information between users and items. This method can not only capture multilateral complex relationships, but also add more randomness and diversity to graph convolutional neural networks by introducing a diffusion model. The use of the diffusion model not only retains the diversity of the original data, but also enhances the differentiation of the embedding representation by introducing random noise. This differentiated representation helps to enhance the accuracy of user preference modeling and provides a basis for us to explore debiased embedding representations.

[0162] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0163] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0164] Figure 1 This is the DDCLRec framework diagram of the present invention.

[0165] Figure 2 This is a schematic diagram of the performance comparison of DDCLRec compared with EEDN and CaDRec in Precision@k, Recall@k and NDCG@k on four datasets including ML-1M and Yelp2018 in the sparse user-item interaction graph scenario; Figure 2 (a) is ML-1M, Figure 2 (b) is Yelp2018, Figure 2 (c) Yelp2018, Figure 2 (c) Douban-book, Figure 2 (d) is Foursquare.

[0166] Figure 3 This is a schematic diagram showing the performance comparison of DDCLRec with EEDN and CaDRec in terms of Precision@k, Recall@k and NDCG@k on four real datasets such as ML-1M and Yelp2018 in the random embedding vector scenario; Figure 3 (a) is Precision@5, Figure 3 (b) is Precision@10, Figure 3 (c) is Precision@20, Figure 3 (d) is Recall@5, Figure 3 (e) is Recall@10, Figure 3(f) is Recall@20, Figure 3 (g) is NDCG@5, Figure 3 (h) is NDCG@10, Figure 3 (i) is NDCG@20.

[0167] Figure 4 This is a schematic diagram analyzing the changes in the Precision@k indicator of DDCLRec compared with other models on four real datasets such as ML-1M and Yelp2018. Figure 4 (a) is ML-1M, Figure 4 (b) is Yelp2018, Figure 4 (c) is Douban-book, Figure 4 (d) is Foursquare.

[0168] Figure 5 is to analyze λ1 and τ as well as λ2 and τ in the data set ML - Schematic diagram of the impact of 1M on the Recall@k and NDCG@k indicators of DDCLRec. Figure 5 (a) is λ1 and τ in ML - Recall@10 on 1M, Figure 5 (b) is the Recall@20 of λ1 and τ on ML-1M, Figure 5 (c) is the NDCG@10 of λ1 and τ on ML-1M, Figure 5 (d) is the NDCG@20 of λ1 and τ on ML-1M, Figure 5 (e) is the Recall@10 of λ2 and τ on ML-1M, Figure 5 (f) is λ2 and τ in ML - Recall@20 on 1M, Figure 5 (g) is λ2 and τ in ML - NDCG@10 on 1M, Figure 5 (h) is λ2 and τ in ML - NDCG@20 on 1M. DETAILED DESCRIPTION

[0169] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0170] Compared with traditional recommendation methods, the recommendation system based on graph neural network can better capture the complex interaction relationship between users and items. It uses the rich information of the interaction graph structure to model the complex interaction relationship between users and items, optimizes the embedding coding representation of users and items, effectively handles the problem of data sparsity, and enhances the interpretability of the recommendation system. However, excessive use of graph neural networks will also be accompanied by problems such as high computational complexity, data sparsity, and interpretability, and there are potential challenges such as building high-quality interaction graphs.

[0171] The introduction of contrastive learning has greatly increased the inclusiveness of recommendation methods, and it is more versatile than any task. It reduces the reliance on labeled data, improves the representation ability of the model, and enhances the robustness of the model. It has the advantages of lightweight models and flexible designs, reduces cost consumption, and improves the generalization ability of the model. However, contrastive learning relies on the selection of negative samples. Improper selection strategies will affect the learning characteristics of the model, and there are high requirements for the selection of negative samples.

[0172] As an effective paradigm for recommendation enhancement, self-supervised learning can train a large amount of unlabeled data and improve recommendation performance. The input data itself is supervised, which is beneficial to various types of downstream tasks.

[0173] In recent years, with the continuous improvement of comparative self-supervised recommendation methods, new recommendation methods are no longer limited to recommendation accuracy, but better promote the diversity of recommendations. Hypergraph convolutional networks can express richer relationship information through hyperedges, effectively process multi-type data relationships, and capture high-order relationships between nodes.

[0174] In order to better solve the problem of over-smoothing of node embeddings in recommendation systems, a diffusion-based debiased contrastive learning for recommendation method DDCLRec (Diffusion-based debiased contrastive learning for recommendation) was proposed by combining the advantages of diffusion model and contrastive learning to selectively retain more valuable embedding representation information, improve recommendation performance and promote recommendation diversity.

[0175] The user-item interaction process is affected by many factors such as popularity and exposure, and the recommendation effect is affected by the differences in rating standards of different users. Some users will give high scores, some will give low scores, and some will give medium scores. This difference will interfere with the accurate capture of the true semantics of the item. At the same time, different users have different focuses on specific dimensions, which will widen the semantic distribution of items and expand the differences. In actual recommendation scenarios, user interest bias will also fluctuate due to factors such as the accumulation effect of historical behaviors and short-term interest hotspots, which will increase the interference with the true semantics of items and put forward higher requirements for the robustness of recommendation methods. However, the user interest bias factor can be used as a learnable perturbation, which can serve as an important basis for the change in the semantic distribution of items, and make corresponding strategies based on this change, which is more conducive to improving the robustness of recommendation methods. Under the interaction of hypergraph convolution enhancement and diffusion model simulation, we can learn important features more effectively, reconsider the generation of user-item embedding representations, balance user interest biases in the recommendation process, update item embeddings by balancing gradients through different weighting schemes, and use regularization to further balance embedding updates, prevent popular items from occupying too large a weight in the embedding space, and tap into users' true preferences.

[0176] The key symbols used in the present invention are shown in Table 1.

[0177] Table 1 Mathematical symbols

[0178]

[0179]

[0180] The task is formulated as follows: Input: user-item interaction graph, Output: Use the denoised user-item interaction embedding representation to generate a recommendation list containing relevant candidate items for the user, and predict the items that the user ultimately wants to interact with based on the user's preferences.

[0181] For the user-item interaction graph, the biased item features of individuals and neighboring nodes, item popularity features, and item semantic features are combined to obtain a multi-feature interaction representation of users on items. Hypergraph convolution is used to encode the representation of users and items to obtain the enhanced embedding representation of user-item interaction. The diffusion model is introduced to incorporate multiple semantic information and bias features to enhance the embedding representation of users and items and model user preferences. Secondly, the process of signal propagation is enhanced by using hypergraph convolution to effectively learn the contextual information in the hyperedge. Hypergraph convolution is combined with the self-attention mechanism to assist information diffusion and aggregate the semantic information of neighboring nodes. Finally, the multi-feature embedding representation of user-item interaction is enhanced by cross-view contrastive learning through contrastive self-supervised learning methods. A hard negative supervised contrastive learning loss optimization strategy is adopted to combine multi-label cross entropy loss and NCE loss as the objective function to better utilize contextual information.

[0182] The overall framework of the proposed DDCLRec is as follows Figure 1 As shown: For the user-item interaction graph, the hypergraph convolution is used to encode the representation of users and items by combining the biased item features of individuals and neighboring nodes, item popularity features and item semantic features. Three different token information, personal bias information, semantic information, and popular item information are considered and injected into the hypergraph convolution operation as learnable perturbations to decouple the popularity bias and user individual bias in user-item interactions and generate debiased user and item representations. The diffusion model is introduced to incorporate a variety of semantic information and bias features to enhance the embedded representation of users and items. The diffusion process first proceeds forward, gradually adding Gaussian noise to destroy the user's interaction history. In this process, noise limits are set to retain user personalized information. In the reverse process, the original interaction data is gradually restored from the interaction history destroyed by noise through a parameterized neural network, and the restored interaction probability is used to sort and recommend items that have not been interacted with.

[0183] Secondly, the process of signal propagation is enhanced by using hypergraph convolution to effectively learn the contextual information in the hyperedge. The slicing operation is used to enable the hypergraph convolution to effectively select the corresponding rows in the hyperedge, reducing the computational complexity. The attention mechanism is injected into the convolution process as a perturbation to select valid neighbor nodes, assist in information diffusion and aggregate the semantic information of adjacent nodes. The multi-head hypergraph convolution network layer learns features in multiple subspaces to generate diversified representations, and finally learns the contextual information through average pooling and normalization until the user representation.

[0184] Finally, by contrasting the self-supervised learning method, the multi-feature embedding representation of user-item interactions is enhanced by cross-view contrastive learning, and personalized features of different nodes are learned to prevent bias. The hard negative supervised contrastive learning loss optimization strategy is adopted, and samples with different but similar labels in the current range are preferentially selected as negative samples, and the multi-label cross entropy loss and NCE loss are combined as the objective function to learn the characteristics of real data, distinguish real data from noise data, and improve the generalization ability and robustness of the model.

[0185] 1. Multi-feature interactive representation learning

[0186] 1.1 Extracting Multi-feature Embedding Representations of User-item Interactions

[0187] According to the interaction information of different users on the interaction graph, the obtained information is converted into entity embedding representation and relationship embedding representation, and the biased item features, item popularity features and item semantic features of the user nodes and neighbors on the graph are embedded to construct a multi-feature interaction embedding representation. The user-item interaction graph information is converted into an embedding representation of the user-item interaction, and the historical interaction sequence H between user u and item i is derived. u,i , determine which users u the current item i interacts with. Obtain the item embedding representation y through uniformly distributed random sampling, where the item embedding value is randomly generated within the uniform distribution range. In the historical behavior sequence of the current user-item interaction, according to the items interacting with the current user, the embedding vectors of all interacting items are integrated to obtain the user embedding representation x. The embedding representation of the items interacting with the current user is to separate the user bias in the item embedding representation based on the item bias characteristics, item popularity characteristics and item semantic information in the user interaction process, fit the biased interaction, and perform self-attention mechanism enhancement to obtain the final user embedding feature representation, and obtain the event sequence representation (x, y) that enriches the user-item interaction characteristics.

[0188]

[0189] Among them, x is the user embedding representation;

[0190] y is the embedding representation of the item;

[0191] y i is the embedding representation of the i-th item, and is the item that user u interacts with;

[0192] H u,i is the historical interaction sequence between user u and item i;

[0193] ||r u || is the modulus length, This calculation is a regularized calculation;

[0194] sign() is, for example, sign(x). If x is positive, it outputs 1; if x is negative, it outputs -1; if x is 0, it outputs 0. The purpose is to extract the sign information of x and perform element-wise multiplication with the following content.

[0195] ⊙ is element-wise multiplication, Hadamard product. The corresponding elements in two vectors or matrices are multiplied, and the result is a new vector or matrix, each element of which is the product of the corresponding elements of the original two vectors or matrices.

[0196] Hgc(·) multiplies the normalized user embedding with the item embedding, combines the information of the two, and passes the updated embedding representation as input through the encoder to fuse the user embedding and the adjacency matrix to generate the final user embedding representation.

[0197] r u is the embedding vector of individual user biases in the original user-item interaction graph;

[0198] is the adjacency matrix;

[0199] Encoder() is a hypergraph convolutional encoding representation injected with a self-attention mechanism.

[0200] In order to better model user preferences, semantic information is introduced to separate item bias. A preference evaluation is set for the current user and calculated by the inner product of the user and the item.

[0201] α=<x,y> (3)

[0202] Among them, x is the embedded representation of the user, y is the representation of the item interacted with, and <> is the inner product operation.

[0203] 1.2 Probabilistic Diffusion Paradigm of Interaction

[0204] For the obtained event sequence representation (x, y) containing rich user-item interaction features, a diffusion model is introduced to enhance user preference modeling and reduce the impact of irrelevant features in the recommendation process. The collaborative signal of user-item interaction is unified with the feature information. By destroying the original user-item interaction, iterative learning is used to restore the initial state through probability diffusion, and iterative denoising training is used to integrate the information into the user-item interaction embedding representation to reduce the negative impact of noise features. The diffusion process is introduced into the user-item interaction graph. First, the original user-item interaction graph is destroyed by gradually introducing Gaussian noise, gradually destroying the interaction between users and items, and simulating the negative impact of noise features. Secondly, in the reverse process, we focus on learning and denoising the damaged graph connection structure, aiming to gradually refine the damaged interaction information to restore the original interaction between users and items.

[0205] In the forward diffusion process, the interaction z between the currently selected user and a set of items is defined u =(x,y) u =[(x,y0),(x,y1),…,(x,y I )], initialize the diffusion process γ0 = z u By gradually introducing Gaussian noise in S steps, taking s as the index, the whole process is parameterized, and noise is gradually introduced into the user-item interaction, so that the original interaction evolves into a noisy state.

[0206]

[0207] Among them, δ(γ s |γ s-1 ) is a forward process that destroys the user's interaction history by gradually adding Gaussian noise.

[0208] is a Gaussian distribution;

[0209] s∈{0,…,S} is the current state γ at each time step s-1 By adding noise, it is transferred to the state γ s Here the noise follows a Gaussian distribution The mean is The variance is μ s I. Introducing two hyperparameters and The noise in the current state is controlled by a linear noise scheduler, and noise is gradually added to mimic the randomness in user-item interactions.

[0210]

[0211] Among them, γ s is the state of the current step in diffusion;

[0212] γ0 is the state of the initial step;

[0213] is the scaling factor, indicating the intensity of added noise;

[0214] It controls the scale of Gaussian noise added in each step (i.e., s steps), and its value range is (0,1);

[0215] ε is standard Gaussian noise, which obeys a Gaussian distribution with a mean of 0 and a covariance matrix of the identity matrix I

[0216] Perform a linear noise schedule for the current state.

[0217]

[0218] in, It controls the scale of Gaussian noise added in each step (i.e., s steps), and its value range is (0,1);

[0219] The hyperparameter c∈[0,1] controls the noise scale, and the two hyperparameters η max and η min Represents the upper and lower bounds for adding noise.

[0220] In the back diffusion process, the current noise state is eliminated and the initial state is restored, so that the diffusion model can effectively capture the subtle changes in the complex generation process. s Initially, the denoising transition step gradually restores the user-item interactions.

[0221]

[0222] Among them, p θ (γ s-1 |γ s ) represents the reverse process in diffusion, predicting the conditional probability distribution of the previous state given the current state. θ (γ s ,s) is the mean Σ θ (γ s ,s) is the covariance matrix, which are the mean and covariance predicted by the parameterized neural network based on the current state and time step.

[0223] Represents Gaussian distribution, and the content in brackets specifically demonstrates how noise is gradually added under different conditions and finally conforms to Gaussian distribution.

[0224] γ s-1 represents the state of the previous time step;

[0225] χ θ (γ s ,s) and ∑ θ (γ s ,s) represent the mean and covariance of the predicted Gaussian distribution, respectively, generated by two neural networks with learnable parameters.

[0226] 1.3 Diffusion Optimization

[0227] To effectively guide the reverse graph diffusion process to learn θ, the Evidence Lower Bound (ELBO) of the negative log-likelihood of user interactions is maximized. This is achieved by optimizing the log-likelihood function of the latent variable model, maximizing the log-likelihood of the result to enhance the generative ability of the model and maintain the stability and diversity of the model. Definition Reflects the model training objectives at different time steps s during the diffusion model optimization process. It is optimized at different stages according to the changes in different step numbers to effectively obtain necessary information from noise.

[0228]

[0229] in, is the KL divergence between the denoised distribution predicted by the model and the true posterior distribution;

[0230] is the reconstruction term, which measures the ability of the model to reconstruct the original data under a given state. It calculates the expected log-likelihood value of the initial state under the conditions of the current time step.

[0231] It is the denoising match term, which describes the denoising transition step and how to gradually restore the user's interaction history in the reverse process. It measures the KL divergence between the denoised distribution predicted by the model and the true posterior distribution at each time step;

[0232] logp(γ0) represents the log-likelihood of the original user interaction data, which measures the model's ability to recover the original user interaction data in the reverse process;

[0233] p θ (γ0|γ1) is the reverse process in diffusion, which predicts the conditional probability distribution of the previous state given the current state.

[0234] D KL (δ(γ s-1 |γ s ,γ0)||p θ (γ s-1 |γ s )) is a measure of the true posterior distribution δ(γ s-1 |γ s ,γ0) and the posterior distribution p predicted by the model θ (γ s-1 |γ s )

[0235] δ(γ s-1 |γ s ,γ0) represents the true posterior distribution, which means that in the diffusion process, given the current state γ s and the initial state γ0, the previous state γ s-1 The true distribution of is defined by gradually adding Gaussian noise;

[0236] γ0 represents the initial state;

[0237] p θ (γ s-1 |γs ) represents the reverse process in diffusion, predicting the conditional probability distribution of the previous state given the current state.

[0238] When s = 0,

[0239] s is the time step;

[0240] γ s is the state under the current step size.

[0241] During the diffusion process, Gaussian distribution reflects the reverse update of the entire process, ensuring that the reverse sampling in each step can be completely restored to the state before the introduction of noise, and finding complete user-item interaction information from the noise. By adjusting the parameters to change the accuracy of the reverse update, the mean is used to calculate the current diffusion state and the initial diffusion state, ensuring the effectiveness of the reverse recovery while ensuring the diversity of the data.

[0242]

[0243] Among them, ω(γ s ,γ0,s) is the mean expression for each step in the reverse process, describing how to calculate the mean of the next step given the current noise state, the original state and the time s.

[0244] ω θ (γ s ,s) is to facilitate the understanding of the transition process of the reverse calculation process. The final change is to push Formula 11 to be close to Formula 10;

[0245] is the noise scale parameter for time step s, controlling the amount of noise added at each step;

[0246] is the cumulative noise ratio parameter, which represents the cumulative noise ratio from the initial state to the current time step s;

[0247] s is the time step;

[0248] ω is given the current state γ s and the mean of the state at the previous time step predicted under the initial state.

[0249] Based on the current state s Prediction of the initial state γ0. Receives the current state and the embedding of the time step as input and outputs the prediction of the initial state γ0.

[0250] Calculate the inverse distribution probability from time step s to s-1 under the current diffusion state. Optimize the KL divergence to measure the approximate distribution of the inverse probability distribution, and use the Bayesian rule to repair the expression of the inverse probability distribution.

[0251]

[0252] Among them, δ(γ s-1 ∣γ s ,γ0) represents how to estimate the conditional probability distribution of the previous state given the current state and the initial state.

[0253] ∝ means proportional, that is, the probability density on the right is proportional to the conditional probability on the left.

[0254] Represents a Gaussian distribution (normal distribution).

[0255] ω(γ s ,γ0,s) represents the mean expression of each step in the reverse process, describing how to calculate the mean of the next step given the current noise state, the original state and the time s.

[0256] I represents the identity matrix.

[0257] variance Add randomness to the sampling process to ensure the diversity of generated data.

[0258] When the initial state is transformed to a certain time step state, the model learns to capture the complex dynamic process. In order to effectively improve the training efficiency, prevent the instability of the introduced noise, and prevent the model from falling into the local minimum and affecting the final training effect. Simplify the state change of the reverse process, ensure the training efficiency and stability, and instantiate it through the multi-layer perceptron (MLP) And based on the step size s of γ s The state predicts the initial state γ0 and defines the loss function for each time step.

[0259]

[0260] Among them, |||| 2 is the two-norm;

[0261] Based on the current state s Prediction of the initial state γ0. Receives the current state and the embedding of the time step as input and outputs the prediction of the initial state γ0.

[0262] 1.4 Debiasing Optimization

[0263] In order to better adjust the embedding representation of users and items during the training process, the introduction of popularity bias effectively improves the learning ability of the model. The popularity bias of users and items is added during the training process, but only the representation of users and items is considered during the test phase. Combined with the interactive item popularity feature information and interactive item bias information of the user context, the user's embedding representation is enriched and the score of the interactive items recommended to the user is set.

[0264] β= <x||μ·r u ,y||μ·r i 〉 (14)

[0265] Among them, β represents the score of the interactive items recommended by the user;

[0266] <·,·〉 is the inner product operation;

[0267] || is the concatenation operation;

[0268] <x||μ·r u ,y||μ·r i >When calculating, first concatenate each part and then perform inner product operation;

[0269] x and y represent the user embedding representation and item embedding representation respectively;

[0270] μ represents a hyperparameter that adjusts the influence of popularity bias; the level of influence of changing popularity.

[0271] r u and r i denote the bias vectors of users and items respectively.

[0272] Based on the historical behavior data of the user's previous interactions and the correlation between the user-item interactions in the interaction graph, the item embeddings that the user interacts with are obtained.

[0273]

[0274] in, is the embedding matrix of items that users have interacted with in the past;

[0275] is the embedding matrix of items that users may interact with in the future.

[0276] The score of the user's recommended items is obtained by the inner product of the user embedding representation that only contains the items that the user has interacted with in the past and the representation of the items that the user may interact with, and the cross entropy loss function is calculated.

[0277]

[0278] in, is the transpose of W1, which is the parameter weight matrix for calculating the cross entropy loss function;

[0279] σ is the activation function.

[0280] Based on the cross entropy loss function, the loss gradient is adjusted by combining parameter weights and regularization, so that it can learn more accurate user-item interactions that are not biased towards popular items or user personal preferences, reducing the risk of one-sided prediction of user-interactive items due to personal bias. A gradient update parameter weight matrix Z is set, and the parameters are adjusted through partial derivative operations and specific optimization algorithms such as gradient descent. The overall loss function is minimized to improve the accuracy of model recommendations, so that the model can adapt to data characteristics while reducing popularity bias. The parameter matrix adjustment process is divided into three categories: the existing user-item interaction in the current interaction scenario, the possible user-item interaction in the future interaction scenario, and the non-existent interaction. The embedding matrix of items that may interact with users in the future is introduced to correct possible bias in the future. The hyperparameter weight matrix Z is used to compare and analyze the impact of bias in different interaction scenarios, and the parameter matrix Z is adjusted through gradient calculation to reduce the impact of bias error, making the model more robust.

[0281]

[0282] 2. Contextual Representation Contrast Enhancement

[0283] 2.1 Hypergraph Convolution Enhancement

[0284] The attention mechanism is used as a perturbation injected into the hypergraph convolution operation to learn features in multiple subspaces, effectively select neighbor nodes, and consider context and sequence context during the propagation process.

[0285] Graph convolution is used to enhance user embedding. First, feature information is extracted through multi-layer convolution propagation to enhance user embedding representation.

[0286]

[0287] in, is the enhanced output of the lth convolutional layer;

[0288] is the ELU activation function;

[0289] D -1 / 2 BD -1 / 2 The symmetric normalized adjacency matrix is ​​calculated;

[0290] B is the bipartite graph adjacency matrix, If the same user and item i f and item i ginteraction, then the matrix b f,g The position is 1, otherwise it is 0.

[0291] represents a diagonal matrix.

[0292] is the parameter weight matrix of convolution propagation.

[0293] Use hypergraph convolution to model the contextual representation of users. By learning contextual representation from user hypergraphs in the hypergraph, signal propagation in the convolution process is enhanced. The critical matrix and feature matrix are adjusted according to the introduced hypergraph information, and specific rows and columns corresponding to users can be selected. Hypergraph convolution is combined with attention mechanism to enhance the ability to select effective neighbors, improve information diffusion in hypergraph neural networks, and consider the multi-layer meaning of structure and order to promote different representations related to the context.

[0294]

[0295] Among them, ψ u is the hyperedge of user u;

[0296] T is the transpose symbol;

[0297] Norm is the L2 regularization calculation;

[0298] d is the dimension in the self-attention mechanism, which is the dimension of the query matrix Q and the key matrix K, and is used for the calculation of the self-attention mechanism.

[0299] is the ELU activation function;

[0300] Q, K, and V represent the query, key, and value matrices in the attention mechanism, respectively;

[0301] W Q , W K is the corresponding parameter weight matrix;

[0302] Φ is a hyperparameter that controls the degree of attention.

[0303] Then, multiple subspace information is aggregated through a multi-head attention mechanism.

[0304]

[0305] Among them, W3 is the weight of each head in the multi-head attention mechanism;

[0306] k is the value of the number of attention heads;

[0307] n is the total number of attention heads;

[0308] It represents the result of input embedding layer l after being processed by different numbers of attention heads;

[0309] Then, after average pooling and normalization, the user embedding representation optimized by hypergraph convolution is obtained.

[0310]

[0311] M i Embeddings that have been processed previously;

[0312] i∈H u,i The embedded selection belongs to the user-item interaction sequence;

[0313] || is the absolute value symbol;

[0314] H u,i Represents the historical interaction sequence between user u and item i;

[0315] The hypergraph convolution enhancement process is optimized through a multi-label cross entropy loss function.

[0316]

[0317] in, is the true label vector of user u’s interaction with item 1;

[0318] is the true label vector of user u’s interaction with item N; and each dimension of the vector is equal to 1 when the user has visited the item; otherwise, it is 0.

[0319] α u,1 , α u,N They are the model's predicted scores for user u on items 1 to n, and each represents the user's predicted score for the item.

[0320] σ is the sigmoid activation function.

[0321] 2.2 Comparative Self-Supervised Optimization

[0322] In the self-supervised recommendation system, the introduction of contrastive learning framework and the combination of difficult negative sample sampling strategy can significantly improve the learning performance. A self-supervised recommendation method based on difficult negative sample sampling is adopted. This method makes full use of label information and the selection of difficult negative samples during training to optimize the learning process of the model. First, a sample selection strategy based on random probability is designed to calculate the distribution probability (0~1) of positive samples and negative samples in the sample set. In this strategy, if the sample label belongs to the current sample set, It is selected as a positive sample; otherwise, it is regarded as a negative sample. Furthermore, multiple different negative sample selection structures are set in the same input sample space to assist in the selection of difficult negative samples in the contrastive learning framework. The similarity between samples is calculated through auxiliary functions, and the final loss function is inferred based on this, thereby optimizing the learning process of the model. This process is similar to unsupervised contrastive learning, where negative samples are randomly selected in the entire input space.

[0323]

[0324] Among them, p(z + ) is the probability distribution of positive samples under different conditions;

[0325] z + 、z - They are positive samples and negative samples respectively;

[0326] z is the sample;

[0327] p U (z - ) is the probability distribution of negative samples under different conditions in the sample set;

[0328] Represents the probability distribution of selecting positive samples;

[0329] Represents the probability distribution of selecting negative samples;

[0330] Represent the positive sample set and the negative sample set respectively;

[0331] Under the current negative sample selection structure, the difficult negative samples are selected. After the anchor point is selected, the cosine similarity between the anchor point and the sample is calculated according to the sample space set, the difficult negative sample set is constructed, and the negative probability distribution p of the difficult negative sample selection is obtained. HU (z - ).

[0332]

[0333] in, is to select the current limited range, that is, HU Negative sample expectation.

[0334] Ψ HU It is the specific range of sample selection for the current condition.

[0335] exp() is an exponential operation with a natural base as the base;

[0336] w is the sample label;

[0337] is a set of sample labels;

[0338] d(z,ξ) is the similarity between the currently selected node (sample z) and the anchor point;

[0339] The threshold for controlling sampling hardness.

[0340] In the sample space combined with the same anchor point, a method is used to select samples that are similar to the anchor point but of different types to construct a set of difficult negative samples and obtain the corresponding negative probability distribution.

[0341]

[0342] Among them, p S (z - ) is in Negative probability distribution of negative sample selection under conditions.

[0343] The label information of the sample w z And the label information w of the anchor ξ Finally, the above two difficult negative sample sets are combined to construct a difficult negative sample selection set based on the contrastive learning framework. This set selects negative samples within the specified range and meets two conditions: similar to the anchor point but with different label types, and meets the selection conditions within the specified selection range.

[0344]

[0345] Among them, p HS (z - ) is the negative probability distribution of difficult negative samples that meet the conditions;

[0346] Ψ HS , S , HU They represent different limiting ranges in the sample selection process.

[0347] Based on the above difficult negative sample selection preparation, a difficult negative sample contrast loss function is established, allowing a sample selection set that meets the above two requirements to be included in the calculation, maximizing the similarity between the anchor point and the positive sample, while minimizing the similarity between the anchor point and the negative sample, bringing similar samples closer and pushing dissimilar samples away in the current sample space.

[0348]

[0349] Among them, d(·,·) represents the similarity;

[0350] n is the number of negative samples;

[0351] k is the negative sample index;

[0352] Indicates difficult negative samples that fall within the current limited range;

[0353] In order to effectively balance the user's biased features on items, the features of items that the user is currently interacting with and the features of items that the user may interact with in the future, and prevent the problem of non-smooth node embedding, the gradient is balanced by changing the weighting scheme to reduce the impact of popular items on the user's true preferences.

[0354]

[0355] Among them, λ is a hyperparameter that balances the weight of the item embedding matrix that interacts with the user;

[0356] λ1 is used for the embedding matrix of items that the user has interacted with;

[0357] ⊙ is element-wise multiplication, Hadamard product. The corresponding elements in two vectors or matrices are multiplied, and the result is a new vector or matrix, each element of which is the product of the corresponding elements of the original two vectors or matrices.

[0358] η is a weight vector used to weight the losses of different items;

[0359] Θ is the regularization coefficient that controls the regularization weight;

[0360] is the previous loss function;

[0361] A collection of users;

[0362] H u,i is the historical interaction sequence between user u and item i;

[0363] For the embedding matrix of items that the user may interact with in the future, λ2 is used.

[0364] 3. Experimental Analysis

[0365] Experiments are conducted on different datasets to evaluate the performance of DDCLRec by comparing with various state-of-the-art recommendation methods, aiming to answer the following research questions:

[0366] RQ1: How does DDCLRec perform when competing with different types of recommendation methods?

[0367] RQ2: How does DDCLRec perform in alleviating the data sparsity problem?

[0368] RQ3: How effective is DDCLRec in alleviating noise issues?

[0369] RQ4: How robust is the recommendation of DDCLRec?

[0370] RQ5: What are the contributions of different key modules in DDCLRec to the overall performance?

[0371] RQ6: What is the impact of the hyperparameters set in DDCLRec on the performance indicators Recall and NDCG?

[0372] 3.1 Dataset

[0373] To evaluate the effectiveness of DDCLRec, experiments are conducted on four public datasets collected from different real-life platforms to reflect real-life scenarios: ML-1M, Yelp2018, Douban-book, Foursquare, and they differ in size and sparsity. The dataset statistics are shown in Table 3.

[0374] Table 3. Statistics of experimental datasets

[0375]

[0376] 3.2 Evaluation indicators

[0377] For performance evaluation, three representative indicators: Precision@k, Recall@k and NDCG@k are used to measure the performance of the proposed DDCLRec. For the top k recommendations, the value of k is set to 5, 10, 20. The method DDCLRec is implemented using PyTorch and its performance is compared with various baseline methods using official or third-party codes. The calculation formulas of Precision, Recall and NDCG are as follows:

[0378] 1) Recall rate reflects how much information the user is interested in that is perceived by us. R(u) represents the Top-k recommendation list given to the user based on the user's behavior on the training set; T(u) represents the set of items actually selected by the user after the system recommends items to the user.

[0379]

[0380] 2) Precision: R(u) represents the Top-k recommendation list given to the user based on the user's behavior on the training set; T(u) represents the set of items actually selected by the user after the system recommends items to the user.

[0381]

[0382] 3) CG (Cumulative Gain), which accumulates the relevance scores of each recommendation result in the recommendation list; however, it is possible that a list with high scores is ranked low and a list with low scores is ranked high.

[0383]

[0384] 4) DCG (Discounted Cumulative Gain) introduces position influence factors based on CG and "discounts" the recommendation effects of lower-ranked recommendation results.

[0385]

[0386] 5) NDCG (Normalized Discounted Cumulative Gain). To evaluate the recommendation system, the recommendation lists of all users in the test set should be evaluated as a whole, and the evaluation scores of recommendation lists of different users should be normalized. IDCG refers to the list of the best recommendation results returned by the recommendation system for a certain user.

[0387]

[0388] 3.3 Baseline Methods

[0389] The performance of DDCLRec is compared with 16 existing popular recommendation system methods such as LightGCN and AutoCF on Recall@5, @10, @20 and NDCG@5, @10, @20 of four datasets including ML-1M and Yelp2018.

[0390] Graph Convolutional Neural Networks:

[0391] LightGCN: Better representation learning and model training using lightweight convolutional graph encoders.

[0392] LCFN: A triple training framework based on self-supervised learning that integrates users’ social information, enhances the learning of multi-view encoders, and leverages self-supervisory signals generated by other users to iteratively optimize representation learning in recommender systems.

[0393] Figure contrast learning:

[0394] AutoCF: It can learn generative self-supervised learning to perform automatic data augmentation and improve the representation ability of collaborative filtering models.

[0395] ·HCCF: Hypergraph-enhanced cross-view contrastive learning structure to jointly capture local and global collaborations.

[0396] NCL: Incorporating potential neighbors in structural and semantic space into comparison pairs to enhance the performance of graph collaborative filtering.

[0397] SGL: Generates multiple views and maximizes the similarity between different views of the same node to enhance representation learning.

[0398] XSimGCL: Enhances generative views with noise-based embeddings.

[0399] LightGCL: A simplified graph contrastive learning paradigm by leveraging singular value decomposition for contrastive enhancement.

[0400] Debiasing Deep Learning:

[0401] DICE: Causal inference separates user interests and herd behavior and learns independent representations.

[0402] InvCF: Discovers disentangled representations that are not affected by changes in popularity distribution, reflecting latent preferences and popularity semantics.

[0403] STaTRL: Transformer captures long-range dependencies in user check-in sequences.

[0404] Diffusion Model:

[0405] DiffRec: The denoising process learns the generative process of user interactions.

[0406] Ease over-smoothing:

[0407] IMP-GCN: Interest-Aware Message Passing Graph Convolutional Network.

[0408] GDE: A simple and effective encoder for graph denoising.

[0409] EEDN: Enhanced Encoder-Decoder Network that combines hybrid hypergraph convolutions to enhance the aggregation of graph convolution steps.

[0410] CaDRec: A hypergraph convolution operation that introduces structural and sequential context to select valid neighbors and alleviate over-smoothing.

[0411] 3.4 Comparative Analysis Experiments with Recommender System Baseline Methods (RQ1)

[0412] Table 4 shows the Recall@k and NDCG@k performance evaluation of DDCLRec compared with 16 baseline methods such as LightGCN and AutoCF on four real datasets ML-1M, Yelp2018, Douban-book, and Foursquare, where the k value is set to 5, 10, and 20.

[0413] Table 4 Comparison of the best Recall@k and NDCG@k performance of the proposed DDCLRec and 16 baseline methods on four datasets including ML-1M and Yelp2018.

[0414]

[0415]

[0416] The experimental results in Table 4 show that DDCLRec outperforms other baselines in all cases, which verifies the effectiveness of introducing diffusion model to enhance embedding representation and hard negative supervision contrastive learning optimization strategy. The diversity of evaluation datasets varies depending on the sparsity of interaction graphs, knowledge graph features, and recommendation scenarios. The results demonstrate the versatility and flexibility of DDCLRec. Overall, the progress of DDCLRec can be attributed to two aspects:

[0417] 1) A hypergraph convolutional network is used to encode and embed users and items to capture complex multilateral relationship structures. A diffusion model is introduced to add random noise, retain and enhance the diversity of the original data, and make the embedding representation more differentiated. This further improves the modeling ability of user preferences and promotes the derivation of debiased recommendations.

[0418] 2) The self-supervisory signal of hard negative sample contrast learning is used to optimize the embedding representation of users and items, and combined with the multi-label cross entropy loss optimization rate, the model's ability to perceive user preferences and the distinguishability in the embedding space is enhanced, enriching the contextual features.

[0419] According to the content of Table 4, DDCLRec has achieved better performance than most recommendation methods on four real datasets such as ML-1M and Yelp2018. For example, for the dataset ML-1M, DDCLRec has improved by about 11.2% on Recall@20 and about 7% on NDCG@20 compared with LightGCN. Compared with AutoCF, it has improved by about 27.5% on Recall@20 and about 22.4% on NDCG@20. For the dataset Douban-book, DDCLRec has improved by about 42.4% on Recall@20 and about 56.3% on NDCG@20 compared with LightGCN. Compared with AutoCF, it has improved by about 54.2% on Recall@20 and about 57.9% on NDCG@20. It has also achieved good performance on the other two datasets. It proves the effectiveness of combining hypergraph convolutional coding with diffusion model to solve problems such as popularity bias in recommendation systems. Existing recommendation methods mainly use the related items in the current user's historical behavior data for collaborative filtering and calculation recommendations, often ignoring the correlation information of many edge sets. However, DDCLRec's advantages on some data sets are not obvious, indicating that the noise introduced by the diffusion model affects the analysis of individual user biases, resulting in distortion of user-item interaction data, and the selection strategy of difficult negative samples may not be perfect enough, resulting in user interest bias. DDCLRec still has a lot of room for improvement in these aspects.

[0420] 3.5 Experiments to alleviate data sparsity analysis (RQ2)

[0421] In order to explore the robustness of the model in data sparse scenarios, a sparsity experiment is conducted by randomly deleting 10% of the interaction edges in the user-item interaction graph. The edge information of the user-item interaction graph in the current scenario is reduced by 10%. In the absence of sufficient interaction information, the performance of DDCLRec and other different recommendation methods in this scenario is evaluated, with special attention paid to the performance of handling the sparsity problem of user data in the absence of sufficient interaction information. Figure 2 As shown in Figure 2, Precision@k, Recall@k and NDCG@k indicators are used, where the k value is set to 10, 20.

[0422] exist Figure 2In the scenario of medium-sparse user-item interaction information graph, DDCLRec has certain advantages over EEDN and CaDRec in Precision@10, @20, Recall@10, @20 and NDCG@10, @20. The experimental results prove the superiority of DDCLRec in alleviating the problem of data sparsity. The advantage of DDCLRec in alleviating data sparsity is more prominent on the dataset ML-1M, but weaker on Yelp2018. The possible reason is that Yelp2018 contains more complex nodes and interaction edge information. The lack of a certain proportion of user-item interaction edge information leads to a greater impact of individual user bias, and the lack of recommendation basis causes bias problems. This shows that DDCLRec is not very good in dealing with complex network node mining and training of a large amount of unlabeled data, and there is still a lot of room for improvement. At the same time, it is also observed that the lower the completeness of the interaction graph, the lower the performance. Slightly reducing the completeness of the interaction graph is also conducive to alleviating the overfitting problem and improving the recommendation performance. This shows that a reasonable selection of the interaction information ratio and learning rate of the interaction graph during training is more conducive to improving the generalization ability of the model.

[0423] 3.6 Mitigation noise analysis experiment (RQ3)

[0424] Noise in recommendations may distort user-item interaction data, leading to recommendation bias, popularity bias and other issues. High noise levels may reduce the signal-to-noise ratio in the user-item interaction graph, reduce recommendation quality and stability. After the hypergraph convolution encodes the user-item interaction, a 10% random scaling tensor is added to the output of the encoder to introduce noise so that it can be included in the recommendation basis. The robustness of the recommendation method is verified through performance evaluation. Figure 3 As shown in Figure 2, Precision@k, Recall@k and NDCG@k metrics are used, where the k value is set to 5, 10, and 20.

[0425] exist Figure 3In the figure, the horizontal axis is different datasets, and the vertical axis is Precision@5, @10, @20, Recall@5, @10, @20 and NDCG@5, @10, @20. For the existing embedding representation set, adding a certain proportion of random scaling tensors to it will lead to recommendation bias when incorporating it into the recommendation basis. In the current noise scenario, DDCLRec has advantages on four datasets such as ML-1M and Yelp2018, proving the effectiveness of DDCLRec in alleviating noise perturbations. It shows the necessity of combining hypergraph convolutional coding and diffusion model of DDCLRec, which can effectively alleviate the problem of user interest bias in noise perturbation scenarios. Among them, the advantage on the ML-1M dataset is more obvious, while the advantage on the Foursquare dataset is weaker. It may be because there are more user nodes and fewer item nodes in the Foursquare dataset, and the node interaction information is less. In the process of storing hyperedge information in complex hypergraph convolution, adding noise is more likely to lead to user interest bias, causing the user's true preference to be covered by popular items, making it difficult to discover the user's true interest. This shows that in terms of alleviating noise disturbance, DDCLRec still has a lot of room for improvement in improving encoding methods and embedding vector calculation methods.

[0426] 3.7 Robustness Analysis Experiment (RQ4)

[0427] In the evaluation of recommendation systems, Precision is an indicator used to measure the accuracy of recommendation results. It is defined as the proportion of truly relevant items in the recommended items. High Precision means that the proportion of relevant items in the recommendation results of the recommendation system is high, indicating that the accuracy of the recommendation system is high. It generally decreases with the increase of the K value of Top-K. The effectiveness of the recommendation strategy can be judged by the stability of its change process. K is set to (1, 5, 10, 20) and multiple experiments are conducted on four data sets such as ML-1M and Yelp2018. The changes in Precision of different models are analyzed. Figure 4 shown.

[0428] exist Figure 4It is observed that on four real datasets such as ML-1M and Yelp2018, as the K value of Top-K increases, the Precision of different models is gradually decreasing. In the initial stage, when the K value is small, the Precision of CaDRec in some datasets is relatively high, but as the K value increases, the Precision of DDCLRec appears to be more stable compared to EEDN and CaDRec, especially on the datasets Yelp2018 and Foursquare. This shows that DDCLRec can still maintain a high accuracy in a longer recommendation list, and can effectively mine the real interests of users even in a longer recommendation list. It proves that DDCLRec can maintain good recommendation quality and robustness when facing more recommendation options. However, the advantage of DDCLRec on the dataset ML-1M is not obvious. Compared with other datasets, ML-1M has fewer nodes and interaction information, which shows that there is still a lot of room for improvement in the strategy of effectively mining user interest preferences in the process of hypergraph convolution and multi-layer graph neural network propagation.

[0429] 3.8 Ablation Experiment (RQ5)

[0430] DDCLRec has advantages over other baseline methods in both Recall@k and NDCG@k performance indicators. The superior results of DDCLRec can be attributed to the following factors:

[0431] A hypergraph convolutional network is used to encode and embed users and items to capture complex multiple relationships. A diffusion model is introduced to preserve and enhance the diversity of the original data, distinguish the embedding representation, strengthen the user preference modeling, and achieve debiased recommendation. In addition, the hard negative supervision contrastive learning loss optimization and multi-label cross entropy loss optimization strategy are combined to optimize the embedding representation, enrich the contextual features, and enhance the model's ability to identify user preferences in the embedding space.

[0432] To study the effectiveness of the key components of DDCLRec, ablation studies are performed on three model variants, as shown in Table 5. The two DDCLRec method variants are as follows:

[0433] 1) DR: Only hypergraph convolutional encoding embedding representation is used to perform multi-layer graph neural network propagation enhancement. The diffusion model is not introduced, and the hard negative contrast learning loss optimization strategy is abandoned.

[0434] 2) DDR: Use hypergraph convolutional encoding to embed representation, introduce diffusion model, and perform multi-layer graph neural network propagation enhancement. Abandon the hard negative contrast learning loss optimization strategy.

[0435] 3) DCLR: Use hypergraph convolutional coding embedding representation to perform multi-layer graph neural network propagation enhancement, and adopt the strategy of discarding hard negative contrast learning loss optimization. No diffusion model is introduced.

[0436] The comparative analysis experiments of Recall@k and NDCG@k indicators of DDCLRec and three method variants on four real datasets such as ML-1M are shown in Table 5, where the value of k is set to 5, 10, and 20.

[0437] Table 5 Evaluation results of Recall@k and NDCG@k indicators of DDCLRec and three method variants on four real datasets

[0438]

[0439] In Table 5, DDCLRec has superior performance in Recall@5, @10, @20 and NDCG@5, @10, @20 indicators of four datasets including ML-1M and Yelp2018. This shows the necessity of combining hypergraph convolutional coding and diffusion model, verifies the importance of combining hard negative contrast learning loss optimization strategy with multi-label cross entropy loss optimization strategy for user preference evaluation, and proves that DDCLRec is reasonable and effective in solving problems such as popularity bias in recommendation systems.

[0440] 3.9 Hyperparameter Analysis Experiment (RQ6)

[0441] DDCLRec has three important hyperparameters. The important weight parameters λ1, λ2 and hard negative sampling depth τ in the loss optimization process. The effects of different combinations of λ1 and τ and different combinations of λ2 and τ on the recommendation performance are studied from (10 -3 ,10 -2 ,10 -1 ), (0.35, 0.4, 0.46, 0.5, 0.55) and (0.3, 0.35, 0.41, 0.45, 0.5). Taking ML-1M data as an example, we explore the effects of hyperparameters on the Recall@k and NDCG@k indicators of DDCLRec. Figure 5 As shown in the figure, the value of k is set to 10 and 20.

[0442] exist Figure 5 (a) and Figure 5 In (b), it is found that when τ is 10 -1 When λ1 is set to 0.5, DDCLRec has the best performance. When the values ​​of the two parameters are changed, the performance is significantly reduced. Figure 5 (c) and Figure 5 In (d), it is found that when τ is 10 -2, when λ2 is 0.41, DDCLRec has the best performance. According to the experimental results, DDCLRec is sensitive to τ. When λ1 is appropriately increased, the convergence is faster and helps to improve the model performance, while too high λ2 will affect the performance. This shows that the participation and change of τ can greatly improve the recommendation performance, but the overall excellent performance also requires λ1 and λ2 to balance. It proves the necessity of hard negative contrast learning for optimizing recommendation methods. Therefore, according to the consideration of weight parameters in different ranges, an effective weighting scheme is synthesized, τ, λ 1和 λ2 is set to 10 -1 , 0.5, 0.41, τ, λ1 and λ2 are set to 10 respectively -2 , 0.46, and 0.41, which can make DDCLRec achieve better performance.

[0443] In summary, the present invention proposes a contrastive learning debiased recommendation method DDCLRec based on a diffusion model. First, the user-item interaction is encoded by combining hypergraph convolutional networks with contextual information to capture the multi-sided complex relationship between users and items, and the valuable information is effectively transferred to the modeling process of user-item interaction. At the same time, the diffusion model is introduced to retain data diversity, so that the embedding representation can be differentiated, enhance user preference modeling and derive debiased representation. Secondly, the study adopts different combinations of regularization and weighting schemes to enhance the embedding representation of users and items through the self-supervisory signal of hard negative supervised contrastive learning, further improve the ability of the method in user preference learning, as well as the performance in user preference perception and embedding space distinguishability, and enrich the method's understanding of contextual features. In order to verify the effectiveness and feasibility of the method, multiple comparative analysis experiments were carried out on four real datasets such as ML-1M and Yelp2018, and DDCLRec was compared with 16 existing popular recommendation methods. Experimental results show that in terms of recommendation system performance evaluation, the DDCLRec proposed on the ML-1M dataset outperforms the latest CaDRec method by 0.8% to 1.4% in Recall@k and 0.5% to 1.1% in NDCG@k, demonstrating the accuracy and superiority of DDCLRec. In the scenario of sparse user-item interaction graphs, DDCLRec has certain advantages in Precision@k, Recall@k, and NDCG@k on four real datasets such as ML-1M and Yelp2018, proving that DDCLRec can better recommend items to users in dealing with sparsity problems. In scenarios with noise, DDCLRec

[0444] Precision@k, Recall@k and NDCG@k on four real datasets including ML-1M and Yelp2018 all outperform the existing popular recommendation methods EEDN and CaDRec; in terms of recommendation stability, as the list of recommended items increases, DDCLRec performs more stably, proving that DDCLRec is more stable and effective in scenarios with a large number of item recommendations; in terms of ablation analysis and hyperparameter sensitivity, the effects of different hyperparameters on Recall@k and NDCG@k of DDCLRec are explored, proving the necessity of introducing a diffusion model on DDCLRec to enhance the hypergraph convolutional encoding embedding representation, as well as the importance of hard negative supervision contrastive learning loss optimization strategy. The above experiments fully demonstrate the advantages of DDCLRec in solving key problems faced by recommendation systems and highlight the effectiveness of DDCLRec.

[0445] However, DDCLRec still has a lot of room for improvement, especially in dealing with long-tail items and user bias. Effectively distinguishing between users' real preference items and popular items is crucial to alleviating user interest bias. In addition, the application of contrastive learning in recommendation systems can improve the accuracy and efficiency of recommendation systems, which requires combining user interaction modeling and contrastive embedding enhancement. In terms of scalability, DDCLRec is not much different from other models. Although DDCLRec is also based on a lightweight framework for recommendation, it also faces increased memory requirements and time when the data set is larger, and its advantage in scalability is not obvious. Future recommendation research should not only focus on accuracy and diversity, but also take scalability as a key feature. In the context of the era of massive information overload on the Internet, recommendation methods are no longer limited to the accuracy of personalized recommendations, but also focus on the scalability of recommendation methods, effectively saving time and space resources, and are more conducive to the development of recommendation system research. Therefore, developing lightweight recommendation methods and exploring effective recommendation methods to remove bias to tap users' real preferences are crucial for the high-quality development of recommendation systems.

[0446] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A contrastive learning debiasing intelligent recommendation method based on diffusion model, characterized in that: The following steps are involved: S1, extract biased item features, item popularity features and item semantic features from the user-item interaction graph, combine the biased item features, item popularity features and item semantic features of the individual and neighboring nodes, and then use hypergraph convolution to encode the representations of the user and item to obtain the encoded embedding, which includes the user's embedded representation x and the item's embedded representation y; S2, input the encoded embedding into the diffusion model, and the diffusion model outputs the final output of different embeddings; then the final output of the different embeddings is aggregated to obtain the user-item interaction embedding representation; then the score of the user-recommended interactive item is calculated based on the user-item interaction embedding representation, and the score is arranged in descending order to generate a recommendation list of related candidate items.

2. According to claim 1, a contrastive learning debiasing intelligent recommendation method based on diffusion model is characterized in that: S1 includes the following steps: S1-1, convert the user-item interaction graph information into an embedded representation of the user-item interaction, and derive the historical interaction sequence H between user u and item i u,i , determine the user u who interacts with the current item i, and then obtain the item's embedding representation y through uniformly distributed random sampling, where the item's embedding value is randomly generated within the uniform distribution range; S1-2, in the historical behavior sequence of user-item interaction, according to the items interacted with the current user, the embedding vectors of all the interacted items are integrated to obtain the user embedding representation x; S1-3, the embedded representation of items that interact with users is integrated. It fits biased interactions based on item bias features, item popularity features, and item semantic information during the user interaction process to obtain the final user embedded feature representation and the event sequence representation (x, y) of the user-item interaction features.

3. According to claim 2, a contrastive learning debiasing intelligent recommendation method based on diffusion model is characterized in that: The calculation formulas for item embedding representation y and user embedding representation x are as follows: Among them, x is the user embedding representation; y is the embedding representation of the item; y i is the embedding representation of the i-th item, and is the item that user u interacts with; H u,i is the historical interaction sequence between user u and item i; r u is the embedding vector of individual user biases in the original user-item interaction graph; ||r u ‖ is r u The module length; sign() is the sign function; ⊙ is the Hadamard product symbol; Hgc(·) is the hypergraph convolution; is the adjacency matrix; Encoder() is a hypergraph convolutional coding representation.

4. According to the diffusion model-based contrastive learning debiasing intelligent recommendation method of claim 1, characterized in that: The diffusion process of the diffusion model includes a forward process and a reverse process. In the forward process, Gaussian noise is gradually added to destroy the user's interaction history. In the reverse process, the original interaction data is gradually restored from the interaction history destroyed by noise through a parameterized neural network, and the restored interaction probability is used to sort and recommend non-interacted items.

5. According to the diffusion model-based contrastive learning debiasing intelligent recommendation method of claim 1, characterized in that: The diffusion model also includes diffusion optimization: definition Reflects the model training objectives at different time steps during the diffusion model optimization process: in, represents the loss function at each time step; is the noise scale parameter for time step s, controlling the amount of noise added at each step; is the cumulative noise ratio parameter, which represents the cumulative noise ratio from the initial state to the current time step s; || ||2 is the two-norm; Based on the current state s Prediction of the initial state γ0.

6. The method for contrastive learning debiasing intelligent recommendation based on diffusion model according to claim 1, characterized in that: The formula for calculating the score of user-recommended interactive items is as follows: β= <x||μ·r u ,y||μ·r i 〉 (14) Among them, β represents the score of the interactive items recommended by the user; 〈·,·> is the inner product operation; || is the concatenation operation; <x||μ·r u ,y||μ@r i >When calculating, first concatenate each part and then perform inner product operation; x and y represent the user embedding representation and item embedding representation respectively; μ represents a hyperparameter that adjusts the influence of popularity bias; r u and r i denote the bias vectors of users and items respectively.

7. The method for contrastive learning debiasing intelligent recommendation based on diffusion model according to claim 1, characterized in that: During the aggregation process, a gradient optimization operation is performed to obtain a debiased embedding representation; The loss function of the gradient optimization operation is: Based on the historical behavior data of the user's previous interactions and the correlation between the user-item interactions in the interaction graph, the item embeddings that the user interacts with are obtained. in, is the embedding matrix of items that users have interacted with in the past; is the embedding matrix of items that users may interact with in the future; The score of the user's recommended items is obtained by the inner product of the user embedding representation containing only the items that the user interacted with in the past and the item representation that the user may interact with, and the cross entropy loss function is calculated: Among them, W1 T is the transpose of W1, which is the parameter weight matrix for calculating the cross entropy loss function; σ is the activation function.

8. The contrastive learning debiasing intelligent recommendation method based on diffusion model according to claim 7, characterized in that: Also includes: Set a parameter matrix Z and adjust the parameter matrix Z through gradient calculation to reduce the impact of bias error, as shown below: in, T is the transpose symbol; [·,·] is a concatenation symbol; x and y represent the user embedding representation and item embedding representation respectively.

Images