Contrastive learning debiasing intelligent recommendation method based on diffusion model

Abstract

The application provides a contrast learning and deviation-eliminating intelligent recommendation method based on a diffusion model, which comprises the following steps: S1, extracting biased item features, item popularity features and item semantic features from a user-item interaction graph, combining the biased item features, the item popularity features and the item semantic features of individuals and neighbor nodes, and then encoding the representations of users and items by using a hypergraph convolution to obtain embedded representations; S2, inputting the embedded representations into a diffusion model, and outputting final outputs of different embedded representations by the diffusion model; then performing an aggregation operation on the final outputs of the different embedded representations to obtain a user-item interaction embedded representation; and then calculating scores of user-recommended interaction items based on the user-item interaction embedded representation, and generating a recommendation list of relevant candidate items in descending order of the scores. The method can not only capture multi-edge complex relationships, but also increase randomness and diversity in a graph convolutional neural network by introducing a diffusion model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent recommendation, and in particular to a contrast learning debiasing intelligent recommendation method based on a diffusion model. BACKGROUND

[0002] With the problem of information overload in online applications brought by the development of the Internet, a high-quality personalized recommendation system has become an indispensable key role in daily life. The recommendation system generally improves the high-quality results of recommendation by mining the historical behavior data of user-item interaction, and most of them use graph convolutional neural networks to enhance the embedding representation of user-item interaction and capture collaborative signals for recommendation. In the recommendation system based on deep learning, the use of neural networks can effectively learn complex user behavior and item feature representation to explore user interest and provide accurate recommendations. Graph representation learning can more effectively capture multi-hop connections and adjacent node information between users and items, and learn user and item representations through graph neural networks.

[0003] Most existing recommendation systems based on graph neural networks make node embeddings similar after multi-layer convolution operations, reducing the discrimination of node representations. This leads to the model's inability to effectively capture the personalized differences between users and items, affecting the performance and accuracy of recommendations. In graph-based recommendation models, attention mechanisms and convolutional neural network layers are often used to aggregate neighbor node information to update node representations, making them more and more similar, and in some cases, they tend to be consistent, causing serious node embedding smoothing problems. SUMMARY

[0004] The present application aims to at least solve the technical problems existing in the prior art, and particularly innovatively proposes a contrast learning debiasing intelligent recommendation method based on a diffusion model.

[0005] In order to achieve the above-mentioned purpose of the present application, the present application provides a contrast learning debiasing intelligent recommendation method based on a diffusion model, comprising the following steps:

[0006] S1, extracting biased item features, item popularity features, and item semantic features from the interaction graph of users and items, combining the biased item features, item popularity features, and item semantic features of individuals and neighbor nodes, and then using hypergraph convolution to encode the representations of users and items to obtain encoded embeddings, the encoded embeddings including user embedding representation x and item embedding representation y;

[0007] Among them, three different token information personal bias information, semantic information, and popular item information are considered as learnable perturbations and injected into the hypergraph convolution operation, decoupling the popularity bias and individual bias in user-item interaction to generate debiased user and item representations.

[0008] S2: Input the encoded embedding into the diffusion model, which outputs the final output of different embeddings. The final output of different embeddings is then aggregated to obtain the user-item interaction embedding representation. The scores of the user-recommended interactive items are then calculated based on the user-item interaction embedding representation, and the scores are sorted in descending order to generate a recommendation list of relevant candidate items.

[0009] Preferably, S1 comprises the following steps:

[0010] S1-1, convert the user-item interaction graph information into an embedded representation of the user-item interaction, and derive the historical interaction sequence H between user u and item i u,i , determine the user u that interacts with the current item i, and then obtain the item's embedding representation y through uniformly distributed random sampling, where the item's embedding value is randomly generated within the uniform distribution range;

[0011] S1-2, in the historical behavior sequence of user-item interactions, based on the items that the current user interacts with, the embedding vectors of all interacting items are integrated to obtain the user embedding representation x;

[0012] S1-3, the embedded representation of items interacting with users, is based on the item bias characteristics, item popularity characteristics, and item semantic information during the user interaction process. It separates the user bias in the item embedding representation, fits the biased interaction, and performs self-attention mechanism enhancement to obtain the final user embedding feature representation, obtaining the event sequence representation (x, y) of the user-item interaction characteristics.

[0013] Preferably, the calculation formulas for the item embedding representation y and the user embedding representation x are as follows:

[0014]

[0015] Among them, x is the user embedding representation;

[0016] y is the embedding representation of the item;

[0017] y i is the embedding representation of the i-th item, and is the item that user u interacts with;

[0018] H u,i is the historical interaction sequence between user u and item i;

[0019] r u is the embedding vector of the user's individual bias in the original user-item interaction graph;

[0020] ||r u ‖ is r u Length of the module;

[0021] sign() is the sign function;

[0022] ⊙ is the Hadamard product symbol;

[0023] Hgc(·) is the hypergraph convolution, which combines the information of the normalized user embedding and the item embedding by multiplying them together, taking the updated embedding representation as input through the encoder, fusing the user embedding and the adjacency matrix, and generating the final user embedding representation;

[0024] is the adjacency matrix;

[0025] Encoder() is the hypergraph convolution encoding representation.

[0026] Preferably, the diffusion process of the diffusion model includes a forward process and a reverse process, in the forward process, Gaussian noise is gradually added to destroy the interaction history of the user, and in this process, the noise limit is set to retain the personalized information of the user. In the reverse process, the original interaction data is gradually recovered from the interaction history destroyed by noise through a parameterized neural network, and the interaction probability is recovered for ranking and recommending uninteracted items.

[0027] In the forward diffusion process, the interaction z u = (x, y) u = [(x, y0), (x, y1), …, (x, y I )] between the currently selected user and a set of items is defined, and the diffusion process is initialized as γ0= z u The entire process is parameterized by gradually introducing Gaussian noise on S steps, indexed by s, to gradually introduce noise on the user-item interaction, so that the original interaction evolves into a state containing noise.

[0028]

[0029] wherein δ(γ s |γ s-1 ) is the destruction of the user's interaction history by gradually adding Gaussian noise in the forward process.

[0030] is a Gaussian distribution;

[0031] s∈{0,…,S} is the current state γ s-1 is transferred to state γ s by adding noise. Here the noise follows a Gaussian distribution with mean and variance μ s ·I. Two hyperparameters and The current state of noise is controlled by a linear noise scheduler, and noise is gradually added to simulate the randomness in user-item interactions.

[0032]

[0033] where γ s is the state of the current step in diffusion;

[0034] γ0is the state of the initial step;

[0035] is a scaling factor representing the intensity of added noise;

[0036] is the Gaussian noise scale added at each step (i.e., s step), which takes a value in the range (0, 1);

[0037] εis a standard Gaussian noise, following a Gaussian distribution with mean 0 and covariance matrix I

[0038] Linear noise scheduling is performed for the current state.

[0039]

[0040] where, is the Gaussian noise scale added at each step (i.e., s step), which takes a value in the range (0, 1);

[0041] The hyperparameter c∈[0,1] controls the noise scale, and the two hyperparameters η max and η min represent the upper and lower bounds of added noise.

[0042] In the reverse diffusion process, the current noise state is eliminated and the initial state is restored, making the diffusion model effectively capture subtle changes in complex generation processes. Starting from the current state γ s , the denoising transition step gradually restores user-item interactions.

[0043]

[0044] where p θ (γ s-1 |γ s ) represents the reverse process in diffusion, predicting the conditional probability distribution of the previous state given the current state. χ θ (γ s ,s) is the mean Σ θ (γ s ,s) is the covariance matrix, both of which are predicted by the parameterized neural network according to the current state and the time step.

[0045] represents the Gaussian distribution, and the content in the parentheses specifically demonstrates the step-by-step addition of noise under different states to finally conform to the Gaussian distribution.

[0046] γ s-1 represents the state of the previous time step;

[0047] χ θ (γ s ,s) and ∑ θ (γ s ,s) are the mean and covariance representing the predicted Gaussian distribution, respectively, generated by two neural networks with learnable parameters.

[0048] Preferably, the diffusion model further comprises diffusion optimization:

[0049] To effectively guide the learning of θ in the reverse graph diffusion process, the evidence lower bound (ELBO) of the negative log-likelihood of user interaction is maximized. By optimizing the log-likelihood function of the latent variable model, the log-likelihood of the maximization result is maximized to enhance the generative ability of the model and maintain the stationarity and diversity of the model.

[0050] Definition Reflects the model training objective at different time steps s in the diffusion model optimization process:

[0051]

[0052] wherein, represents the loss function at each time step;

[0053] is the noise proportion parameter at time step s, which controls the amount of noise added at each step;

[0054] is the cumulative noise proportion parameter, representing the cumulative noise proportion from the initial state to the current time step s;

[0055] |||2 is the two-norm;

[0056] is the prediction based on the existing state γ s s.

[0057] Preferably, the rating formula for recommending interaction items to users is as follows:

[0058] β = <x||μ·r u ,y||μ·r i > (14)

[0059] wherein, β represents the rating of the recommended interaction item to the user;

[0060] <·,·> is an inner product operation;

[0061] || is a concatenation operation;

[0062] x||μ·r u y||μ·r i > is calculated by first performing concatenation and then performing inner product operation;

[0063] x and y represent user embedding representation and item embedding representation, respectively;

[0064] μ represents a hyperparameter for adjusting the influence of popularity bias;

[0065] r u and r i represent bias vectors of users and items, respectively.

[0066] Preferably, in the aggregation process, a gradient optimization operation is performed to obtain a debiased embedding representation; wherein the loss function of the gradient optimization operation is:

[0067] According to the historical behavior data of past user interactions and the relevance of user-item interactions in the interaction graph, the item embedding of the user interaction is obtained

[0068]

[0069] wherein, is the embedding matrix of the past user interaction items;

[0070] is the embedding matrix of the items that the user may interact with in the future;

[0071] The cross-entropy loss function is calculated by only including the inner product of the user embedding representation of the past user interaction items and the item representation that the user may interact with to obtain the score of the recommended item of the user:

[0072]

[0073] wherein, T is the transpose of W1, and W1 is the parameter weight matrix for calculating the cross-entropy loss function;

[0074] σ is an activation function.

[0075] Based on the cross-entropy loss function, the loss gradient is adjusted by combining the parameter weight and regularization, so that it can learn more accurate and unbiased user-item interaction relationships that are not biased towards popular items or personal preferences of the user, and reduce the risk of one-sided prediction of user interaction items caused by personal bias.

[0076] Preferably, further comprising: setting a parameter matrix Z, and adjusting the parameter matrix Z through gradient calculation to reduce the influence of bias error, represented as follows:

[0077]

[0078] wherein, T is the transpose symbol;

[0079] [·,·] is the concatenation symbol;

[0080] x, y represent user embedding representation and item embedding representation respectively.

[0081] Preferably, further comprising: S3, using hypergraph convolution to enhance the hypergraph convolution of the encoded embedding, enhancing signal propagation through hypergraph convolution enhancement, and hypergraph convolution can effectively select corresponding rows in the hyperedge and effectively learn the context information in the hyperedge; reduce the computational complexity. Get the user embedding representation and the embedding representation of the item optimized by the hypergraph convolution; then calculate the score of the user recommendation interaction item, and arrange the score in descending order to generate a recommendation list of related candidate items.

[0082] Preferably, the hypergraph convolution enhancement comprises:

[0083] S2-1, enhance the user embedding representation by extracting feature information through multi-layer convolution propagation:

[0084]

[0085] wherein, M l is the output of the lth convolution layer after enhancement;

[0086] is the ELU activation function;

[0087] D -1 / 2 BD -1 / 2 is a symmetric normalized adjacency matrix;

[0088] B is a bipartite graph adjacency matrix;

[0089] D represents a diagonal matrix;

[0090] W2 is the parameter weight matrix of convolution propagation;

[0091] S2-2, model the context representation of the user using hypergraph convolution, enhance the signal propagation in the convolution process by learning the context representation from the user hyperedge in the hypergraph, and adjust the critical matrix and the feature matrix according to the introduced hypergraph information, select the specific row and column corresponding to the user;

[0092]

[0093] wherein, ψu is the hyperedge of user u;

[0094] T is the transpose symbol;

[0095] Norm is the L2 regularization calculation;

[0096] d is the dimension in the self-attention mechanism;

[0097] is the ELU activation function;

[0098] Q, K, V represent the query, key and value matrices in the attention mechanism, respectively;

[0099] W Q , W K is the corresponding parameter weight matrix;

[0100] Φ is a hyperparameter that controls the degree of attention;

[0101] S2-3, multiple subspace information is aggregated through the multi-head attention mechanism:

[0102]

[0103] where W3 is the weight of each head in the multi-head attention mechanism;

[0104] k is the value of the number of attention heads;

[0105] n is the total number of attention heads;

[0106] represents the result of the input embedding after the lth layer is processed through different attention heads;

[0107] S2-4, after average pooling and normalization, the user embedding representation and the item embedding representation optimized by the hypergraph convolution are obtained;

[0108]

[0109] where x is the user embedding representation optimized by the hypergraph convolution;

[0110] M i is the processed embedding;

[0111] i∈H u,i is the selection of the embedding belongs to the user-item interaction sequence;

[0112] || is the absolute value symbol;

[0113] H u,i represents the historical interaction sequence of user u and item i;

[0114] y is an embedding representation of the item optimized by hypergraph convolution;

[0115] y i is an embedding representation of the ith item, and is an item interacted with the user u;

[0116] sign() is a sign function;

[0117] is a Hadamard product symbol;

[0118] ||r u || is a module length, This calculation is a regularization calculation.

[0119] Preferably, it further comprises: using a contrast self-supervised learning method, enhancing cross-view contrast learning of multi-feature embedding representations of user-item interactions, learning personalized features of different nodes to prevent bias; and using a hard negative supervision contrast learning loss optimization strategy, selecting samples with different labels but similar samples within the current range as negative samples.

[0120] First, a sample selection strategy based on random probability is designed to calculate the distribution probability (0-1) of positive samples and negative samples in the sample set. If the sample label belongs to the current sample set , it is selected as a positive sample; otherwise, it is considered as a negative sample; and multiple different negative sample selection structures are set in the same input sample space to assist difficult negative sample selection in the contrast learning framework.

[0121]

[0122] wherein p(z + ) is the probability distribution of the positive sample under different conditions;

[0123] z + , z - are positive samples and negative samples, respectively;

[0124] z is a sample;

[0125] p U (z - ) is the probability distribution of the negative sample under different conditions in the sample set;

[0126] represents the probability distribution of selecting a positive sample;

[0127] represents the probability distribution of selecting a negative sample;

[0128] represent the positive sample set and the negative sample set, respectively;

[0129] Under the current negative sample selection structure, the difficult negative sample selection work is carried out: after the anchor point is determined, the cosine similarity between the anchor point and the sample is calculated according to the sample space set, the difficult negative sample set is constructed, and the negative probability distribution p of the difficult negative sample selection is obtained. HU (z - ):

[0130]

[0131] in, Is to select the current limited range, that is, Ψ HU Negative sample expectation;

[0132] Ψ HU It is the specific range of the current condition sample selection;

[0133] exp() is an exponential operation with a natural base as the base;

[0134] w is the sample label;

[0135] is a set of sample labels;

[0136] d(z,ξ) is the similarity between the currently selected node (sample z) and the anchor point;

[0137] To control the threshold of sampling hardness;

[0138] In the sample space combined with the same anchor point, a method is used to select samples that are similar to the anchor point but of different types to construct a set of difficult negative samples and obtain the corresponding negative probability distribution:

[0139]

[0140] Among them, p S (z - ) is in Negative probability distribution of negative sample selection under conditions;

[0141] The label information of the sample w z And the label information w of the anchor ξ are comprehensively considered; finally, the above two difficult negative sample sets are combined to construct a difficult negative sample selection set based on the contrastive learning framework; this set selects negative samples within the specified range that meet two conditions: similar to the anchor point but with different label types, and meet the selection conditions within the specified selection range.

[0142]

[0143] Among them, p HS (z -) is the negative probability distribution of difficult negative samples that meet the conditions;

[0144] Ψ HS Ψ S Ψ HU respectively represent different limited ranges in the sample selection process.

[0145] The contrastive loss function of the difficult negative sample is:

[0146]

[0147] Where d(·,·) represents the similarity;

[0148] n is the number of negative samples;

[0149] k is the negative sample index;

[0150] represents the difficult negative sample belonging to the current limited range;

[0151] In addition, in order to effectively balance the user's bias feature of the item, the user's current interaction item feature and the user's future possible interaction item feature, and prevent the problem of non-smooth node embedding. By changing the weighting scheme to balance the gradient, the recommendation of the user's true preference is reduced because of the high popularity of the item. At this time, the loss function is represented as follows:

[0152]

[0153] Where λ is a hyperparameter for balancing the weight of the item embedding matrix interacted with the user;

[0154] λ1 is used for the item embedding matrix interacted by the user;

[0155] ⊙ is the Hadamard product symbol;

[0156] η is a weight vector used to weight the loss of different items;

[0157] Θ is a regularization coefficient to control the regularization weight;

[0158] is the loss function in the front;

[0159] is the user set;

[0160] H u,i is the historical interaction sequence of user u and item i;

[0161] In summary, by adopting the technical scheme, the application aims at the problem of node embedding over-smoothing existing in the graph neural network recommendation system.

[0162] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and / or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0163] The above and / or additional aspects and advantages of the application will become apparent and be readily understood by considering the following detailed description, from which the singular aspects become apparent.

[0164] Figure 1 is a DDCLRec framework diagram of the application.

[0165] Figure 2 is a performance comparison diagram of Precision@k, Recall@k and NDCG@k of DDCLRec, EEDN and CaDRec in a sparse user-item interaction graph scenario on four data sets of ML-1M, Yelp2018, etc. Figure 2 (a) is ML-1M, Figure 2 (b) is Yelp2018, Figure 2 (c) is Yelp2018, Figure 2 (c) is Douban-book, Figure 2 (d) is Foursquare.

[0166] Figure 3 is a performance comparison diagram of Precision@k, Recall@k and NDCG@k of DDCLRec, EEDN and CaDRec in a random embedding vector scenario on four real data sets of ML-1M, Yelp2018, etc. Figure 3 (a) is Precision@5, Figure 3 (b) is Precision@10, Figure 3 (c) is Precision@20, Figure 3 (d) is Recall@5, Figure 3 (e) is Recall@10, Figure 3(f) is Recall@20, Figure 3 (g) is NDCG@5, Figure 3 (h) is NDCG@10, Figure 3 (i) is NDCG@20.

[0167] Figure 4 is a plot of the Precision@k index changes of DDCLRec over other models on four real datasets, ML-1M, Yelp2018, etc. Figure 4 (a) is ML-1M, Figure 4 (b) is Yelp2018, Figure 4 (c) is Douban-book, Figure 4 (d) is Foursquare.

[0168] Figure 5 is a plot of the impact of λ1 and τ and λ2 and τ on the Recall@k and NDCG@k indexes of DDCLRec on the dataset ML - 1M. Figure 5 (a) is λ1 and τ on Recall@10 on ML - 1M, Figure 5 (b) is λ1 and τ on Recall@20 on ML-1M, Figure 5 (c) is λ1 and τ on NDCG@10 on ML-1M, Figure 5 (d) is λ1 and τ on NDCG@20 on ML-1M, Figure 5 (e) is λ2 and τ on Recall@10 on ML-1M, Figure 5 (f) is λ2 and τ on Recall@20 on ML - 1M, Figure 5 (g) is λ2 and τ on NDCG@10 on ML - 1M, Figure 5 (h) is λ2 and τ on NDCG@20 on ML - 1M. DETAILED DESCRIPTION

[0169] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein the same or like reference numerals and characters throughout the figures denote the same elements or elements with the same functions. The embodiments described below are merely exemplary for the purpose of explanations and are not to be understood as limiting the present application.

[0170] Compared with traditional recommendation methods, recommendation systems based on graph neural networks can better capture complex user-item interaction relationships. By utilizing the rich information of the interaction graph structure, the complex interaction relationships between users and items are modeled, the embedding encoding representation of users and items is optimized, the data sparsity problem is effectively addressed, and the interpretability of the recommendation system is enhanced. However, excessive use of graph neural networks also brings high computational complexity, data sparsity, and interpretability issues, and there are potential challenges in constructing high-quality interaction graphs.

[0171] The introduction of contrastive learning greatly increases the inclusiveness of recommendation methods, and is more general for any task. It reduces the dependence on labeled data, improves the representation ability of the model, and enhances the robustness of the model. It has the advantages of lightweight model and flexible design, reduces cost consumption, and improves the generalization ability of the model. However, contrastive learning relies on the selection of negative samples, and improper selection strategies can affect the learning of features by the model, and there are high requirements for the selection of negative samples.

[0172] Self-supervised learning as an effective paradigm for recommendation enhancement can train a large amount of unlabeled data and improve recommendation performance. The input data itself is supervised, which is beneficial for various types of downstream tasks.

[0173] In recent years, with the continuous improvement of contrastive self-supervised recommendation methods, new recommendation methods are no longer limited to recommendation accuracy, but better promote the diversity of recommendations. Hypergraph convolutional networks can express more rich relationship information through hyperedges, effectively handle multi-type data relationships, and capture high-order relationships between nodes.

[0174] To better solve the problem of node embedding over-smoothing in recommendation systems, combining the advantages of diffusion models and contrastive learning, selectively retaining more valuable embedding representation information, improving recommendation performance while promoting recommendation diversity, a diffusion model-based contrastive learning debiased recommendation method DDCLRec (Diffusion-based debiased contrastive learning for recommendation) is proposed.

[0175] The user-item interaction process is affected by factors such as popularity, exposure, and the like, and the recommendation effect is affected by the difference in different user rating standards. Some users will give high scores, some users will give low scores, and some users will give medium scores. This difference will interfere with the accurate capture of the real semantics of the item. At the same time, different users have different concerns in certain dimensions, which further widens the semantic distribution of the item and expands the difference. In the actual recommendation scene, the user interest bias will fluctuate due to factors such as historical behavior accumulation effect and short-term interest hotspot, which will further interfere with the real semantics of the item, and higher requirements are put forward for the robustness of the recommendation method. However, the user interest bias factor can be used as a learnable disturbance, which can be used as an important basis for the change of the semantic distribution of the item, and the corresponding strategy is made according to the change, which is more conducive to improving the robustness of the recommendation method. Under the simulation of the interaction of the hypergraph convolution enhancement and diffusion model, the important features can be learned more effectively, and the generation of the user-item embedding representation is reconsidered, the user interest bias in the recommendation process is balanced, the gradient is balanced through different weighting schemes, the item embedding is updated, the regularization is further used to balance the update of the embedding, and the popular items are prevented from occupying too large weight in the embedding space, so that the real preference of the user is mined.

[0176] The key symbols used in the application are shown in Table 1.

[0177] Table 1 Mathematical symbols

[0178]

[0179]

[0180] The task is as follows: input: user-item interaction graph, output: use the denoised user-item interaction embedding representation, generate a recommendation list containing relevant candidate items for the user, and predict the item that the user ultimately wants to interact with according to the user's preference.

[0181] For the user-item interaction graph, the multi-feature interaction representation of the user to the item is obtained by combining the individual and neighbor node bias item features, item popularity features and item semantic features. The representation of the user and the item is encoded using hypergraph convolution to obtain an enhanced embedding representation of the user-item interaction. A diffusion model is introduced to incorporate various semantic information and bias features to enhance the embedding representation of the user and the item, and to model the user preference. Secondly, the process of signal propagation is enhanced using hypergraph convolution to effectively learn the context information in the hyperedge. The hypergraph convolution is combined with the self-attention mechanism to assist information diffusion and aggregate the semantic information of the neighboring nodes. Finally, the multi-feature embedding representation of the user-item interaction is enhanced through cross-view contrastive learning by comparing the self-supervised learning method. The multi-label cross-entropy loss and NCE loss are combined as the objective function to better utilize the context information.

[0182] The overall framework of the proposed DDCLRec is shown in Figure 1 For the user-item interaction graph, the multi-feature interaction representation of the user to the item is obtained by combining the individual and neighbor node bias item features, item popularity features and item semantic features. The representation of the user and the item is encoded using hypergraph convolution to obtain an enhanced embedding representation of the user-item interaction. A diffusion model is introduced to incorporate various semantic information and bias features to enhance the embedding representation of the user and the item, and to model the user preference. Secondly, the process of signal propagation is enhanced using hypergraph convolution to effectively learn the context information in the hyperedge. The hypergraph convolution is combined with the self-attention mechanism to assist information diffusion and aggregate the semantic information of the neighboring nodes. Finally, the multi-feature embedding representation of the user-item interaction is enhanced through cross-view contrastive learning by comparing the self-supervised learning method. The multi-label cross-entropy loss and NCE loss are combined as the objective function to better utilize the context information.

[0183] Secondly, the process of signal propagation is enhanced using hypergraph convolution to effectively learn the context information in the hyperedge. The slicing operation is used to effectively select the corresponding row in the hyperedge, reducing the computational complexity. The attention mechanism is injected into the convolution process as a perturbation to select effective neighbor nodes, assist information diffusion and aggregate the semantic information of the neighboring nodes. The multi-head hypergraph convolution network layer learns features in multiple subspaces to generate diversified representations, and finally learns the context information through user representation, average pooling and normalization.

[0184] Finally, by comparing the self-supervised learning method, the multi-feature embedding representation of user-item interaction is enhanced by cross-view contrast learning, and the personalized features of different nodes are learned to prevent bias. The hard negative supervision contrast learning loss optimization strategy is adopted, and the samples with different labels but similar samples in the current range are selected as negative samples in priority, and the multi-label cross-entropy loss and NCE loss are combined as the objective function to learn the features of the real data, distinguish real data and noise data, and improve the generalization ability and robustness of the model.

[0185] 1. Multi-feature interaction representation learning

[0186] 1.1 Extracting multi-feature embedding representation of user-item interaction

[0187] According to the interaction information of different users on the interaction graph, the obtained information is converted into entity embedding representation and relationship embedding representation, and the bias item feature, item popularity feature and item semantic feature of the user node and neighbor on the graph are embedded to construct a multi-feature interaction embedding representation. The user-item interaction graph information is converted into the embedding representation of the user-item interaction, and the historical interaction sequence H u,i of the user u and the item i is determined. The embedding representation y of the item is obtained by uniform distribution random sampling, wherein the embedding value of the item is randomly generated in the uniform distribution range. In the historical behavior sequence of the current user-item interaction, according to the item interacted with the current user, the embedding vectors of all interaction items are integrated to obtain the user embedding representation x. The embedding representation of the item interacted with the current user is separated from the user bias in the item embedding representation according to the item bias feature, item popularity feature and item semantic information in the user interaction process, the biased interaction is fitted, and the self-attention mechanism is enhanced to obtain the final user embedding feature representation, and the event sequence representation (x, y) of rich user-item interaction features is obtained.

[0188]

[0189] Wherein, x is the user embedding representation;

[0190] y is the embedding representation of the item;

[0191] y i is the embedding representation of the i-th item, and is the item interacted with the user u;

[0192] H u,i is the historical interaction sequence of the user u and the item i;

[0193] ||r u || is the module length, This calculation is a regularization calculation;

[0194] sign() is, for example, sign(x), which outputs 1 if x is positive, -1 if x is negative, and 0 if x is 0, aiming to extract the sign information of x and perform element-level multiplication with the following content.

[0195] ⊙ is element-level multiplication, Hadamard product. The corresponding elements in two vectors or matrices are multiplied, and the result is a new vector or matrix, whose each element is the product of the corresponding elements of the original two vectors or matrices.

[0196] Hgc(·) is to multiply the normalized user embedding with the item embedding, combine the information of the two through the encoder to update the embedding representation as input, fuse the user embedding and the adjacency matrix, and generate the final user embedding representation.

[0197] r u is the embedding vector of the user individual bias of the original user-item interaction graph;

[0198] is the adjacency matrix;

[0199] Encoder() is a hypergraph convolution encoding representation with self-attention mechanism injection.

[0200] In order to better model the user preference, semantic information is introduced to separate the item bias. A preference evaluation is set for the current user, which is calculated by the inner product of the user and the item.

[0201] α = <x, y> (3)

[0202] Where x is the embedding representation of the user, y is the representation of the item interacted with, and <> is the inner product operation.

[0203] 1.2 Interaction probability diffusion paradigm

[0204] A diffusion model is introduced for the obtained event sequence representation (x, y) containing rich user-item interaction features, to enhance user preference modeling and reduce the influence of irrelevant features in the recommendation process. The collaborative signal of user-item interaction and feature information are unified. By destroying the original user-item interaction, the initial state is restored through probability diffusion by iterative learning, and the information is integrated into the user-item interaction embedding representation through iterative denoising training, to reduce the negative impact of noise features. The diffusion process is introduced to the user-item interaction graph. First, the original user-item interaction graph is destroyed by gradually introducing Gaussian noise, gradually destroying the interaction between users and items, and simulating the negative impact of noise features. Secondly, in the reverse process, focus on learning and denoising the damaged graph connection structure, aiming to gradually refine the damaged interaction information to restore the original interaction between users and items.

[0205] In the forward diffusion process, define the interaction z between the currently selected user and a set of items u = (x, y) u = [(x, y0), (x, y1), …, (x, y I )], initialize the diffusion process γ0= z u The entire process is parameterized by introducing Gaussian noise step by step on S steps, and noise is introduced step by step on the user-item interaction, so that the original interaction evolves into a state containing noise.

[0206]

[0207] where δ(γ s |γ s-1 ) is the destruction of the user's interaction history by gradually adding Gaussian noise in the forward process.

[0208] is a Gaussian distribution;

[0209] s∈{0,…,S} is the current state γ s-1 is transferred to state γ s by adding noise. Here the noise is subject to Gaussian distribution with mean and variance μ s ·I. Two hyperparameters and are introduced to control the noise in the current state by a linear noise scheduler, and noise is added step by step to simulate the randomness in user-item interaction.

[0210]

[0211] where γ s is the state of the current step in diffusion;

[0212] γ0is the state of the initial step;

[0213] is a scaling factor representing the strength of noise added;

[0214] is the scale of Gaussian noise added at each step (i.e., s step), which takes a value in the range (0, 1);

[0215] εis a standard Gaussian noise, subject to Gaussian distribution with mean 0 and covariance matrix I

[0216] Linear noise scheduling is performed for the current state.

[0217]

[0218] where, is the Gaussian noise scale controlling the addition of each step (i.e., s-step), which is in the range of (0, 1);

[0219] The hyper-parameter c e [0, 1] controls the noise scale, and the two hyper-parameters max and min represent the upper and lower bounds of the added noise.

[0220] In the backward diffusion process, the current noise state is eliminated and the initial state is restored, making the diffusion model effectively capture subtle changes in the complex generation process. Starting from the current state s , the denoising transition step gradually restores the user-item interaction.

[0221]

[0222] where, p θ (γ s-1 |γ s ) represents the reverse process in diffusion, predicting the conditional probability distribution of the previous state given the current state. χ θ (γ s , s) is the mean θ (γ s , s) is the covariance matrix, both of which are predicted by the parameterized neural network according to the current state and time step.

[0223] represents a Gaussian distribution, and the contents in the brackets specifically demonstrate that the final noise added step by step under different states conforms to the Gaussian distribution.

[0224] γ s-1 represents the state of the previous time step;

[0225] χ θ (γ s , s) and θ (γ s , s) are the mean and covariance of the predicted Gaussian distribution, respectively, generated by two neural networks with learnable parameters.

[0226] 1.3 Diffusion Optimization

[0227] To effectively guide the learning of the reverse graph diffusion process, maximize the evidence lower bound (ELBO) of the negative log-likelihood of user interaction. This is achieved by optimizing the log-likelihood function of the latent variable model, maximizing the log-likelihood of the result to enhance the generative ability of the model, and maintaining the stationarity and diversity of the model. Define Reflects the model training objective at different time steps s in the diffusion model optimization process. According to the change of different steps, it is optimized at different stages, and the necessary information is effectively obtained from the noise.

[0228]

[0229] wherein, is the KL divergence between the model predicted denoising distribution and the true posterior distribution;

[0230] is the reconstruction term, which measures the ability of the model to reconstruct the original data under a given state, and the calculation is the log-likelihood expectation value of the initial state under the current time step condition;

[0231] is the denoising matching term, which describes the denoising overstep step, and describes how to gradually restore the user's interaction history in the reverse process. Measure the KL divergence between the model predicted denoising distribution and the true posterior distribution at each time step;

[0232] logp(γ0) represents the log-likelihood of the original user interaction data, which measures the model's recovery of the original user interaction data in the reverse process;

[0233] p θ (γ0|γ1) is the reverse process in diffusion, which predicts the conditional probability distribution of the previous state given the current state.

[0234] D KL (δ(γ s-1 |γ s ,γ0)||p θ (γ s-1 |γ s )) is the difference between the true posterior distribution δ(γ s-1 |γ s ,γ0) and the model predicted posterior distribution p θ (γ s-1 |γ s );

[0235] δ(γ s-1 |γ s ,γ0) represents the true posterior distribution, which refers to the true distribution of the previous state γ s given the current state γ s-1 and the initial state γ0 in the diffusion process, and gradually adds Gaussian noise to define;

[0236] γ0 represents the initial state;

[0237] p θ (γ s-1 |γs ) represents the reverse process in diffusion, predicting the conditional probability distribution of the previous state given the current state.

[0238] When s = 0,

[0239] s is the time step;

[0240] γ s is the state at the current step.

[0241] The Gaussian distribution in the diffusion process embodies the reverse update of the entire process, ensuring that the reverse sampling in each step can completely restore the state before introducing noise, finding complete user-item interaction information from noise. By adjusting the parameters to change the precision of the reverse update, the mean is used to calculate the current diffusion state and the initial diffusion state, ensuring the effectiveness of the reverse recovery while ensuring data diversity features.

[0242]

[0243] where ω(γ s ,γ0,s) is the expression for calculating the mean of each step in the reverse process, describing how to calculate the mean of the next step given the current noise state, the original state and the time s.

[0244] ω θ (γ s ,s) is a transition process for calculating the reverse process for easy understanding, and the final change is to push formula 11 to approach formula 10;

[0245] is the noise proportion parameter at time step s, which controls the amount of noise added in each step;

[0246] is the cumulative noise proportion parameter, indicating the cumulative noise proportion from the initial state to the current time step s;

[0247] s is the time step;

[0248] ω is the mean of the state at the previous time step given the current state γ s and the initial state.

[0249] is the prediction of the initial state γ0 based on the existing state γ s . The current state and the time step embedding are received as input, and the initial state γ0 is output.

[0250] The reverse probability distribution of time step s to s-1 is calculated in the current diffusion state. The KL divergence is optimized to measure the approximation of the reverse probability distribution, and the expression of the reverse probability distribution is repaired using the Bayesian rule.

[0251]

[0252] where δ(γ s-1 | γ s , γ0) represents how to estimate the conditional probability distribution of the previous state given the current state and the initial state.

[0253] ∝ represents proportional to, that is, the probability density on the right side is proportional to the conditional probability on the left side.

[0254] represents a Gaussian distribution (normal distribution).

[0255] ω(γ s , γ0, s) represents the expression of the mean value of each step in the reverse process, which describes how to calculate the mean value of the next step given the current noise state, the original state and the time s.

[0256] I represents a unit matrix.

[0257] Variance Randomness is added to the sampling process to ensure the diversity of generated data.

[0258] When the initial state is transformed into a state at a certain time step, it is the model learning to capture complex dynamic processes. In order to effectively improve the training efficiency, prevent the introduction of noise instability, and avoid the model falling into local minimum to affect the final training effect. Simplify the state change of the reverse process, ensure the training efficiency and stability, and instantiate based on the γ s state of step s to predict the initial state γ0, and define the loss function of each time step.

[0259]

[0260] where ||| 2 is the two norm;

[0261] is the prediction of the initial state γ0based on the existing state γ s . The current state and the embedding of the time step are received as input, and the initial state γ0is output.

[0262] 1.4 De-bias optimization

[0263] In order to better adjust the embedding representation of users and items in the training process, the introduction of popularity bias effectively improves the learning ability of the model. The popularity bias of users and items is added in the training process, but only the representation of users and items is considered in the test stage. The interaction item popularity feature information and the interaction item bias information of the user context are combined to enrich the embedding representation of the user, and the score of recommending the interaction item to the user is set.

[0264] β = <x||μ·r u ,y||μ·r i > (14)

[0265] Wherein, β represents the score of recommending the interaction item to the user;

[0266] <·,·> is the inner product operation;

[0267] || is the concatenation operation;

[0268] <x||μ·r u ,y||μ·r i > is calculated by first concatenating each other and then performing the inner product operation;

[0269] x and y represent the embedding representation of the user and the embedding representation of the item respectively;

[0270] μ represents the hyperparameter for adjusting the influence of popularity bias; change the influence level of popularity.

[0271] r u and r i respectively represent the bias vector of the user and the item.

[0272] According to the historical behavior data of the user's past interaction and the correlation of the user-item interaction in the interaction graph, the embedding of the item interacted with the user is obtained

[0273]

[0274] Wherein, is the embedding matrix of the past user interaction item;

[0275] is the embedding matrix of the item that the user may interact in the future.

[0276] The score of recommending the item to the user is obtained by only containing the inner product of the user embedding representation of the past user interaction item and the user interaction item, and the cross entropy loss function is calculated.

[0277]

[0278] Wherein, is the transpose of W1, W1 is the parameter weight matrix for computing the cross-entropy loss function;

[0279] σ is an activation function.

[0280] Based on the cross-entropy loss function, the loss gradient is adjusted by combining the parameter weight and regularization, so that it can learn more accurate user-item interaction relationship without being biased to popular items or personal preferences of users, and reduce the risk of one-sided prediction of user interaction items caused by personal bias. A gradient update parameter weight matrix Z is set, and the parameters are adjusted by using specific gradient descent optimization algorithm through partial derivative operation, to minimize the overall loss function and improve the model recommendation accuracy, so that the model can adapt to the data characteristics while reducing popularity bias. The parameter matrix adjustment process is divided into three categories: the first is the existing user-item interaction in the current interaction scenario, the second is the possible user-item interaction in the future interaction scenario, and the third is the non-interaction case. The embedding matrix of the item that may interact with the user in the future is introduced to correct the bias that may occur in the future. The influence of bias in different interaction scenarios is compared and analyzed through the hyperparameter weight matrix Z, and the influence of bias error is reduced by adjusting the parameter matrix Z through gradient calculation, so that the model has better robustness.

[0281]

[0282] 2. Context representation contrast enhancement

[0283] 2.1 Hypergraph convolution enhancement

[0284] The attention mechanism is used as a perturbation injection hypergraph convolution operation to learn features in multiple subspaces and effectively select neighbor nodes, considering context and sequential context in the propagation process.

[0285] The user embedding is enhanced by using graph convolution, which first extracts feature information by multi-layer convolution propagation to enhance the user embedding representation.

[0286]

[0287] wherein, is the output of the lth convolution layer after enhancement;

[0288] is an ELU activation function;

[0289] D -1 / 2 BD -1 / 2 is the symmetric normalized adjacency matrix;

[0290] B is the bipartite graph adjacency matrix, if the same user and item i f and item i gInteraction, b in the matrix f,g 1 if the position is 1, otherwise 0.

[0291] represents a diagonal matrix.

[0292] is the parameter weight matrix of convolution propagation.

[0293] The context representation of the user is modeled using hypergraph convolution, the signal propagation in the convolution process is enhanced by learning the context representation from the user hyperedge in the hypergraph, and the critical matrix and feature matrix are adjusted according to the introduced hypergraph information, so that the specific row and column corresponding to the user can be selected. The combination of hypergraph convolution and attention mechanism enhances the ability to select effective neighbors, improves the information diffusion in the hypergraph neural network, and considers the multi-layer meaning of structure and order to promote different representations related to the context.

[0294]

[0295] where ψ u is the hyperedge of user u;

[0296] T is the transpose symbol;

[0297] Norm is the L2 regularization calculation;

[0298] d is the dimension in the self-attention mechanism, which is the dimension of the query matrix Q and the key matrix K, used for the calculation of the self-attention mechanism.

[0299] is the ELU activation function;

[0300] Q, K, V represent the query, key and value matrices in the attention mechanism, respectively;

[0301] W Q , W K is the corresponding parameter weight matrix;

[0302] Φ is a hyperparameter that controls the degree of attention.

[0303] Then, multiple subspace information is aggregated through the multi-head attention mechanism.

[0304]

[0305] where W3 is the weight of each head in the multi-head attention mechanism;

[0306] k is the value of the number of attention heads;

[0307] n is the total number of attention heads;

[0308] represents the input embedding of the 1th layer after being processed by different attention heads;

[0309] Then, the user embedding representation optimized by the hypergraph convolution is obtained through average pooling and normalization.

[0310]

[0311] M i Embedding processed previously;

[0312] i∈H u,i is the selection of the embedding belongs to the user-item interaction sequence;

[0313] || is the absolute value symbol;

[0314] H u,i represents the historical interaction sequence of user u and item i;

[0315] The hypergraph convolution enhancement process is optimized by a multi-label cross-entropy loss function.

[0316]

[0317] wherein, is the true label vector of user u interacting with item 1;

[0318] is the true label vector of user u interacting with item N; and when the user has accessed the item, each dimension of the vector is equal to 1; otherwise, it is 0.

[0319] α u,1 , α u,N are the predicted scores of the model for user u interacting with item 1 to item n, each representing the predicted score of the user for the item.

[0320] σ is a sigmoid activation function.

[0321] 2.2 Contrastive self-supervised optimization

[0322] In a self-supervised recommendation system, introducing a contrastive learning framework combined with a difficult negative sample sampling strategy can significantly improve learning performance. A self-supervised recommendation method based on difficult negative sample sampling is adopted, which fully utilizes the selection of label information and difficult negative samples in the training process to optimize the learning process of the model. First, a sample selection strategy based on random probability is designed to calculate the distribution probability (0-1) of positive samples and negative samples in the sample set. In this strategy, if the sample label belongs to the current sample set is selected as a positive sample; otherwise, it is regarded as a negative sample. Further, a plurality of different negative sample selection structures are set in the same input sample space to assist the selection of difficult negative samples in the contrast learning framework. The similarity between samples is calculated by an auxiliary function, and the final loss function is inferred accordingly to optimize the learning process of the model. This process is similar to unsupervised contrast learning, in which negative samples are randomly selected in the entire input space.

[0323]

[0324] wherein p(z + ) is the probability distribution of the positive sample under different conditions;

[0325] z + , z - are positive samples and negative samples, respectively;

[0326] z is a sample;

[0327] p U (z - ) is the probability distribution of the negative sample under different conditions in the sample set;

[0328] represents the probability distribution of selecting a positive sample;

[0329] represents the probability distribution of selecting a negative sample;

[0330] represent the positive sample set and the negative sample set, respectively;

[0331] Under the current negative sample selection structure, the selection of difficult negative samples is performed. After the anchor point is selected, the cosine similarity between the anchor point and the samples in the sample space set is calculated, a difficult negative sample set is constructed, and the negative probability distribution p HU (z - ) of the difficult negative sample selection is obtained.

[0332]

[0333] wherein, is the expected value of selecting a negative sample within the current limited range, i.e., Ψ HU .

[0334] Ψ HU is the specific range of the current conditional sample selection.

[0335] exp() is the exponential operation with the natural base;

[0336] w is the sample label;

[0337] is a set of sample labels;

[0338] d(z,ξ) is the similarity between the current selected node (sample z) and the anchor point;

[0339] is the threshold value of the control sampling hardness.

[0340] In the sample space combined with the same anchor point, the method of selecting samples similar to the anchor point but of different types is used to construct a difficult negative sample set and obtain the corresponding negative probability distribution.

[0341]

[0342] where p S (z - ) is the negative probability distribution of the negative sample selection under the condition

[0343] The label information w z of the sample and the label information w ξ of the anchor point are comprehensively considered. Finally, the two difficult negative sample sets described above are combined to construct a difficult negative sample selection set based on the contrast learning framework. This set selects negative samples within a specified range, satisfying two conditions: similar to the anchor point but with different label types, and satisfying the selection condition within the specified selection range

[0344]

[0345] where p HS (z - ) is the negative probability distribution of the difficult negative sample satisfying the condition;

[0346] Ψ HS , Ψ S , Ψ HU respectively represent different limited ranges in the sample selection process.

[0347] Based on the above difficult negative sample selection preparation, the difficult negative sample contrast loss function is established, allowing the sample selection set satisfying the above two requirements to join the calculation, maximizing the similarity between the anchor point and the positive sample, while minimizing the similarity between the anchor point and the negative sample, and in the current sample space, similar samples are pulled closer and dissimilar samples are pushed farther apart.

[0348]

[0349] where d(·,·) represents the similarity;

[0350] n is the number of negative samples;

[0351] k is the negative sample index; ​

[0352] represents the difficult negative samples belonging to the current limited range;

[0353] In order to effectively balance the user's bias feature of the item, the user's current interaction item feature and the user's future possible interaction item feature, prevent the problem of non-smooth node embedding. By changing the weighting scheme to balance the gradient, the recommendation of the user's real preference is reduced because of the high popularity of the item.

[0354]

[0355] where λ is a hyperparameter balancing the weight of the item embedding matrix interacted with the user;

[0356] λ1 is used for the item embedding matrix interacted by the user;

[0357] ⊙ is an element-wise multiplication, Hadamard product. The corresponding elements in two vectors or matrices are multiplied, and the result is a new vector or matrix, each element of which is the product of the corresponding elements of the original two vectors or matrices.

[0358] η is a weight vector used to weight the loss of different items;

[0359] Θ is a regularization coefficient to control the regularization weight;

[0360] is the loss function in the front;

[0361] is the user set;

[0362] H u,i is the historical interaction sequence of the user u and the item i;

[0363] λ2 is used for the item embedding matrix that the user may interact with in the future.

[0364] 3. Experimental analysis

[0365] Experiments are conducted on different data sets by comparing with various state-of-the-art recommendation methods to evaluate the performance of DDCLRec, and the goal is to answer the following research questions:

[0366] ·RQ1: How does DDCLRec perform when competing with different types of recommendation methods?

[0367] ·RQ2: How does DDCLRec perform in alleviating the problem of data sparsity?

[0368] ·RQ3: How effective is DDCLRec in alleviating the noise problem?

[0369] ·RQ4: How is the recommendation robustness of DDCLRec?

[0370] • RQ5: What is the contribution of different key modules in DDCLRec to the overall performance?

[0371] • RQ6: How do the hyperparameters set in DDCLRec affect the changes in performance metrics Recall and NDCG?

[0372] 3.1 Datasets

[0373] To evaluate the effectiveness of DDCLRec, experiments were conducted on four public datasets collected from different real-life platforms to reflect real-life scenarios: ML-1M, Yelp2018, Douban-book, and Foursquare, which differ in size and sparsity, and the dataset statistics are shown in Table 3.

[0374] Table 3 Statistics of experimental datasets

[0375]

[0376] 3.2 Evaluation metrics

[0377] For performance evaluation, three representative metrics were used: Precision@k, Recall@k, and NDCG@k to measure the performance of the proposed DDCLRec. The top-k recommendations, with k values set to 5, 10, and 20. The method DDCLRec was implemented using PyTorch, and its performance was compared with various baseline methods using official or third-party code. The calculation formulas for Precision, Recall, and NDCG are as follows:

[0378] 1) Recall (Recall rate) size reflects how much information users are interested in being perceived, R(u) represents the Top-k recommendation list made for the user based on the user's behavior on the training set; T(u) represents the set of items actually selected by the user after the system recommends items to the user.

[0379]

[0380] 2) Precision (Precision), R(u) represents the Top-k recommendation list made for the user based on the user's behavior on the training set; T(u) represents the set of items actually selected by the user after the system recommends items to the user.

[0381]

[0382] 3) CG (Cumulative Gain), the relevance score of each recommendation result in the recommendation list is accumulated; however, it is possible that in a list, high scores are placed in low positions, and low scores are placed in high positions.

[0383]

[0384] 4) DCG (Discounted Cumulative Gain) introduces position influence factors based on CG and "discounts" the recommendation effects of lower-ranked recommendation results.

[0385]

[0386] 5) NDCG (Normalized Discounted Cumulative Gain): To evaluate a recommendation system, the recommendation lists of all users in the test set must be evaluated holistically, and the evaluation scores of recommendation lists for different users must be normalized. IDCG refers to the list of best recommendations returned by the recommendation system for a particular user.

[0387]

[0388] 3.3 Baseline Methods

[0389] The performance of DDCLRec is compared with 16 existing popular recommendation system methods such as LightGCN and AutoCF on Recall@5, @10, @20 and NDCG@5, @10, @20 of four datasets including ML-1M and Yelp2018.

[0390] Graph Convolutional Neural Networks:

[0391] LightGCN: Better representation learning and model training using lightweight convolutional graph encoders.

[0392] LCFN: A triple training framework based on self-supervised learning that integrates users’ social information, enhances the learning of multi-view encoders, and leverages self-supervisory signals generated by other users to iteratively optimize representation learning in recommendation systems.

[0393] Graph contrast learning:

[0394] AutoCF: It can learn generative self-supervised learning to automatically enhance data and improve the representation ability of collaborative filtering models.

[0395] ·HCCF: Hypergraph-enhanced cross-view contrastive learning structure to jointly capture local and global collaborations.

[0396] NCL: Incorporating potential neighbors in structural and semantic spaces into comparison pairs to enhance the performance of graph collaborative filtering.

[0397] • SGL: generates multiple views and maximizes the similarity between different views of the same node, enhancing representation learning.

[0398] • XSimGCL: uses noise-based embedding enhancement to generate views.

[0399] • LightGCL: a simplified graph contrastive learning paradigm that enhances contrast through singular value decomposition.

[0400] Debiasing deep learning:

[0401] • DICE: causal inference separates users' interests and conformity behavior, learning independent representations.

[0402] • InvCF: uncovers decoupled representations that are not affected by changes in popularity distribution, reflecting latent preferences and popularity semantics.

[0403] • STaTRL: Transformer captures long-range dependencies in user check-in sequences.

[0404] Diffusion model:

[0405] • DiffRec: denoising process learns the generative process of user interactions.

[0406] Mitigating over-smoothing:

[0407] • IMP-GCN: interest-aware message passing graph convolutional network.

[0408] • GDE: simple and effective graph denoising encoder.

[0409] • EEDN: enhanced encoder-decoder network that combines hybrid hypergraph convolution to enhance the aggregation of graph convolution steps.

[0410] • CaDRec: introduces hypergraph convolution operations that select effective neighbors by incorporating structure and sequence context, mitigating over-smoothing.

[0411] 3.4 Comparative analysis experiment with baseline methods of recommendation system (RQ1)

[0412] In Table 4, the Recall@k and NDCG@k performance evaluations of DDCLRec compared with 16 baseline methods such as LightGCN, AutoCF, etc. on four real datasets ML-1M, Yelp2018, Douban-book, Foursquare are shown. The k value is set to 5, 10, 20.

[0413] Table 4 shows the best performance comparison of Recall@k and NDCG@k of DDCLRec and 16 baseline methods on four datasets ML-1M, Yelp2018, etc.

[0414]

[0415]

[0416] The experimental results in Table 4 show that DDCLRec outperforms other baselines in all cases, which verifies the effectiveness of introducing diffusion model enhanced embedding representation and hard-negative supervised contrastive learning optimization strategy. The diversity of the evaluation dataset varies with the interaction graph sparsity, knowledge graph features, and recommendation scenarios. The research results prove the universality and flexibility of DDCLRec. Overall, the progress made by DDCLRec can be attributed to two aspects:

[0417] 1) The hypergraph convolutional network is used to encode the embedding representation of users and items to capture complex multi-edge relationship structures. The diffusion model is introduced to add random noise, preserve and enhance the diversity of the original data, so that the embedding representation has greater differentiation. The modeling ability of user preferences is further improved, and the derivation of the de-biased recommendation is promoted.

[0418] 2) The embedding representation of users and items is optimized by the self-supervised signal of hard-negative sample contrastive learning, and is combined with the multi-label cross-entropy loss optimization rate to enhance the perception ability of the model to user preferences and the distinguishability in the embedding space, and enrich the context features.

[0419] According to the content of Table 4, it can be seen that DDCLRec achieves better performance than most recommendation methods on four real datasets such as ML-1M, Yelp2018, etc. For example, for the dataset ML-1M, DDCLRec improves LightGCN by about 11.2% in Recall@20 and about 7% in NDCG@20. Compared with AutoCF, it improves by about 27.5% in Recall@20 and about 22.4% in NDCG@20. For the dataset Douban-book, DDCLRec improves LightGCN by about 42.4% in Recall@20 and about 56.3% in NDCG@20. Compared with AutoCF, it improves by about 54.2% in Recall@20 and about 57.9% in NDCG@20. Good performance is also achieved on the other two datasets. It confirms the effectiveness of combining hypergraph convolution coding with diffusion model to solve popularity bias and other problems in recommendation system. Existing recommendation methods mainly use the associated items in the current user historical behavior data for collaborative filtering to calculate recommendations, often ignoring a lot of edge set relevance information. But the advantage of DDCLRec on some datasets is not obvious, indicating that the noise introduced by the diffusion model affects the analysis of user individual bias, leading to distortion of user-item interaction data, and the selection strategy of difficult negative samples may not be perfect enough, leading to user interest bias. DDCLRec still has a lot of room for improvement in these aspects.

[0420] 3.5 Analysis experiment of alleviating data sparsity (RQ2)

[0421] To explore the robustness of the model in the data sparse scene, the user-item interaction graph is randomly deleted with a 10% proportion of interaction edges for sparsity experiment. According to a 10% proportion, the edge information of the current user-item interaction graph is reduced. In the absence of sufficient interaction information, the performance of DDCLRec and other different recommendation methods in this scenario is evaluated, especially the performance of handling user data sparsity problems in the absence of sufficient interaction information as shown in Figure 2 Precision@k, Recall@k and NDCG@k indicators are used, where k is set to 10, 20.

[0422] In Figure 2In the scenario of sparse user-item interaction information graph, DDCLRec has certain advantages over EEDN and CaDRec in Precision@10, @20, Recall@10, @20 and NDCG@10, @20. The experimental results prove the superiority of DDCLRec in alleviating the problem of data sparsity. The advantage of DDCLRec in alleviating data sparsity is more prominent on the dataset ML-1M, but the advantage on Yelp2018 is weaker. The possible reason is that Yelp2018 contains more complex node and interaction edge information, and the removal of a certain proportion of user-item interaction edge information leads to greater user individual bias. The bias problem caused by less recommendation basis shows that DDCLRec is not very good at dealing with complex network node mining and large amounts of unlabeled data training, and there is still a lot of room for improvement. At the same time, it is also observed that the lower the completeness of the interaction graph, the lower the performance. Slightly reducing the completeness of the interaction graph is also beneficial to alleviate the overfitting problem and improve the recommendation performance, which shows that reasonable selection of interaction information ratio and learning rate of interaction graph in the training process is more conducive to improving the generalization ability of the model.

[0423] 3.6 Noise alleviation analysis experiment (RQ3)

[0424] Noise in recommendation may distort user-item interaction data, leading to recommendation bias, popularity bias and other problems. High noise level may reduce the signal-to-noise ratio in the user-item interaction graph, and reduce the recommendation quality and stability. After the hypergraph convolution encodes the user-item interaction, a 10% random scaling tensor is added to the output of the encoder to introduce noise, which is included in the recommendation basis. The robustness of the recommendation method is verified through performance evaluation as shown in Figure 3 Precision@k, Recall@k and NDCG@k indicators are used, where k is set to 5, 10 and 20.

[0425] In Figure 3In the figure, the horizontal axis is different data sets, and the vertical axis is Precision@5, @10, @20, Recall@5, @10, @20 and NDCG@5, @10, @20. For the existing embedding representation set, a certain proportion of random scaling tensors are added, which will cause the recommendation to deviate when the recommendation basis is included. In the current noise scene, DDCLRec has an advantage on four data sets such as ML-1M, Yelp2018, etc., which proves the effectiveness of DDCLRec in alleviating noise disturbance. It shows the necessity of the combination of hypergraph convolution coding and diffusion model of DDCLRec, which can effectively alleviate the user interest bias problem in the noise disturbance scene. Among them, the advantage on the ML-1M data set is more obvious, while the advantage on the Foursquare data set is weak. It may be because there are more user nodes and fewer item nodes in the Foursquare data set, and the node interaction information is less. In the process of storing hyperedge information in complex hypergraph convolution, adding noise is more likely to cause user interest bias, so that the real preference of the user is covered by the high popularity of the item, and it is difficult to discover the real interest of the user. It shows that in terms of alleviating noise disturbance, DDCLRec still has a lot of room for improvement in improving the coding method and embedding vector calculation method.

[0426] 3.7 Robustness analysis experiment (RQ4)

[0427] In the recommendation system evaluation, Precision is used to measure the accuracy of the recommendation result, defined as the proportion of truly relevant items in the recommended items. High Precision indicates that the proportion of relevant items in the recommendation result of the recommendation system is high, indicating that the accuracy of the recommendation system is high, which generally decreases with the increase of the K value of Top-K. The stability of its change process can be used to judge the effectiveness of the recommendation strategy. Set K to (1, 5, 10, 20) and perform multiple experiments on four data sets such as ML-1M, Yelp2018, etc. The Precision changes of different models are analyzed as shown in Figure 4

[0428] In Figure 4 ​It is observed that the precision of different models gradually decreases with the increase of K value of Top-K on four real datasets such as ML-1M, Yelp2018, etc. In the initial stage, the precision of CaDRec is higher in some datasets, but as K value increases, the precision of DDCLRec changes more stably compared with EEDN and CaDRec, especially on datasets Yelp2018 and Foursquare. This shows that DDCLRec can maintain high accuracy in longer recommendation lists, and can effectively mine the real interests of users even in longer recommendation lists. It proves that DDCLRec can maintain good recommendation quality and robustness when facing more recommendation options. However, the advantage of DDCLRec on dataset ML-1M is not obvious, and the node and interaction information of ML-1M is less than that of other datasets, which shows that there is still a lot of room for improvement in the strategy of effectively mining user interest preferences in the process of supergraph convolution and multi-layer graph neural network propagation.

[0429] 3.8 Ablation experiments (RQ5)

[0430] DDCLRec outperforms other baseline methods in terms of Recall@k and NDCG@k performance indicators. The superior results of DDCLRec can be attributed to the following factors:

[0431] The supergraph convolution network is used to encode and embed users and items to capture complex multi-relation. The diffusion model is introduced to preserve and enhance the diversity of the original data, distinguish embedded representations, strengthen user preference modeling, and achieve unbiased recommendation. In addition, the optimization strategy of hard negative supervised contrastive learning loss and multi-label cross-entropy loss is combined to optimize embedded representations, enrich context features, and enhance the model's ability to recognize user preferences in the embedding space.

[0432] To study the effectiveness of the key components of DDCLRec, three model variants are used for ablation study, as shown in Table 5. Two DDCLRec method variants are as follows:

[0433] 1) DR: Only use supergraph convolution to encode embedded representations and perform multi-layer graph neural network propagation enhancement. Do not introduce diffusion model and discard hard negative contrastive learning loss optimization strategy.

[0434] 2) DDR: Use supergraph convolution to encode embedded representations, introduce diffusion model, and perform multi-layer graph neural network propagation enhancement. Discard the hard negative contrastive learning loss optimization strategy.

[0435] 3) DCLR: using hypergraph convolutional coding embedding representation, multi-layer graph neural network propagation enhancement, taking hard negative contrastive learning loss optimization strategy. No diffusion model is introduced.

[0436] The comparative analysis experiment of Recall@k and NDCG@k indexes of DDCLRec and three method variants on four real data sets such as ML-1M is shown in Table 5, where the value of k is set to 5, 10, 20.

[0437] Table 5 Evaluation results of Recall@k and NDCG@k indexes of DDCLRec and three method variants on four real data sets

[0438]

[0439] In Table 5, DDCLRec has superior performance in Recall@5, @10, @20 and NDCG@5, @10, @20 indexes on four data sets such as ML-1M, Yelp2018. It shows the necessity of combining hypergraph convolutional coding and diffusion model, verifies the importance of combining hard negative contrastive learning loss optimization strategy and multi-label cross-entropy loss optimization strategy when evaluating user preferences, and proves that DDCLRec is reasonable and effective for solving popularity bias and other problems in recommendation systems.

[0440] 3.9 Hyperparameter analysis experiment (RQ6)

[0441] DDCLRec has three important hyperparameters. Important weight parameters λ1, λ2 and hard negative sampling depth τ in the loss optimization process. The influence of different combinations of λ1 and τ and different combinations of λ2 and τ on the recommendation performance is studied, respectively from (10 -3 ,10 -2 ,10 -1 ), (0.35, 0.4, 0.46, 0.5, 0.55) and (0.3, 0.35, 0.41, 0.45, 0.5) range of τ, λ1 and λ2, taking ML-1M data as an example, the influence of hyperparameters on Recall@k and NDCG@k indexes of DDCLRec is shown in Figure 5 , where the value of k is set to 10, 20.

[0442] In Figure 5 (a) and Figure 5 (b), it is found that when τ takes the value of 10 -1 , λ1 takes the value of 0.5, DDCLRec has the best performance, and the performance decreases significantly when the values of the two parameters are changed. In Figure 5 (c) and Figure 5 (d), it is found that when τ takes the value of 10 -2, λ2 is 0.41, the DDCLRec performance is optimal. According to the experimental results, it is found that DDCLRec is sensitive to tau, when appropriately increasing lambda1, the convergence is faster and helps to improve the performance of the model, and too high lambda2 will affect the performance, which shows that the participation and change of tau can greatly improve the recommendation performance, but the overall excellent performance also needs lambda1 and lambda2 to balance. It is proved that the necessity of hard negative contrastive learning for the optimization of the recommendation method. Therefore, according to the consideration of the weight parameters in different ranges, the effective weighting scheme is synthesized, tau, lambda 1和 λ2 is 10 -1 , 0.5, 0.41, tau, lambda1 and lambda2 are respectively set to 10 -2 , 0.46, 0.41, so that DDCLRec can achieve better performance.

[0443] In summary, the application proposes a contrastive learning debiasing recommendation method DDCLRec based on a diffusion model. First, the hypergraph convolution network is combined with the context information to capture the multi-edge complex relationship between users and items to encode the user-item interaction, and effectively transmit the valuable information to the modeling process of the user-item interaction. At the same time, the diffusion model is introduced to retain the data diversity, so that the embedding representation is differentiated, the user preference modeling is enhanced, and the debiased representation is derived. Secondly, different combinations of regularization and weighting schemes are used, the self-supervised signal of hard negative supervised contrastive learning is enhanced to enhance the embedding representation of users and items, further improve the ability of the method in user preference learning, and the performance of user preference perception and embedding space distinction, and enrich the understanding of the method to the context features. In order to verify the effectiveness and feasibility of the method, a plurality of comparative analysis experiments are carried out on four real data sets such as ML-1M, Yelp2018, and the DDCLRec is compared with 16 existing popular recommendation methods. The experimental results show that: in the performance evaluation of the recommendation system, the proposed DDCLRec is 0.8% to 1.4% better than the latest CaDRec method in the Recall@k index on the ML-1M data set, and 0.5% to 1.1% better than the latest CaDRec method in the NDCG@k index, which highlights the accuracy and superiority of DDCLRec; in the scene of sparse user-item interaction graph, DDCLRec has certain advantages in Precision@k, Recall@k and NDCG@k on four real data sets such as ML-1M, Yelp2018, which proves that DDCLRec can better recommend items for users in the treatment of sparsity problem; in the scene containing noise, DDCLRec

[0444] Precision@k, Recall@k and NDCG@k on four real datasets, ML-1M, Yelp2018, etc., are superior to existing popular recommendation methods EEDN and CaDRec; in terms of recommendation stability, DDCLRec performs more stably as the recommended item list increases, proving that DDCLRec is more stable and effective in the context of a large number of item recommendations; in terms of ablation analysis and hyperparameter sensitivity, the influence of different hyperparameters on Recall@k and NDCG@k of DDCLRec is explored, proving the necessity of introducing a diffusion model to enhance the hypergraph convolutional embedding representation of DDCLRec, and the importance of the hard negative supervised contrastive learning loss optimization strategy. The above experiments fully demonstrate the advantages of DDCLRec in solving key problems faced by recommendation systems, highlighting the effectiveness of DDCLRec.

[0445] However, there is still much room for improvement in DDCLRec, especially in handling long-tail items and user bias. Effectively distinguishing between user's real preference items and popular items is crucial to alleviate user interest bias. In addition, the application of contrastive learning in recommendation systems can improve the accuracy and efficiency of recommendation systems, which requires the combination of user interaction modeling and contrastive embedding enhancement. In terms of scalability, DDCLRec is not much different from other models, although DDCLRec is also based on a lightweight framework for recommendation, but in the case of larger datasets, it also faces the increase of memory requirements and time, and the advantage in scalability is not obvious. Future recommendation research should not only focus on accuracy and diversity, but also take scalability as a key feature. In the era of massive information overload on the Internet, recommendation methods are no longer limited to the accuracy of personalized recommendations, but also focus on the scalability of recommendation methods to effectively save time and space resources, which is more conducive to the development of recommendation system research. Therefore, developing lightweight recommendation methods and exploring effective bias-removing recommendation methods to mine user's real preferences are crucial for the high-quality development of recommendation systems.

[0446] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A contrastive learning debiasing intelligent recommendation method based on diffusion model, characterized by: The following steps are involved: S1, extracts item bias features, item popularity features, and item semantic features from the user-item interaction graph, combines the item bias features, item popularity features, and item semantic features of the individual and neighboring nodes, and then uses hypergraph convolution to encode the representation of the user and item to obtain the encoded embedding, which includes the user's embedded representation and item embedding representations ; S2: The encoded embedding is input into the diffusion model, which outputs the final output of the different embeddings. The final output of the different embeddings is then aggregated to obtain a user-item interaction embedding representation. The scores of the user-recommended interactive items are then calculated based on the user-item interaction embedding representation, and the scores are sorted in descending order to generate a recommendation list of relevant candidate items. The diffusion process of the diffusion model includes a forward process and a backward process. In the forward process, Gaussian noise is gradually added to destroy the user's interaction history. In the backward process, the original interaction data is gradually restored from the noise-corrupted interaction history through a parameterized neural network. The restored interaction probability is used to sort and recommend non-interacted items. The diffusion model also includes diffusion optimization: definition Reflects the diffusion model optimization process at different time steps The following model training objectives: ， in, represents the loss function at each time step; is the time step The noise scale parameter controls the amount of noise added at each step; is the cumulative noise ratio parameter, which represents the noise from the initial state to the current time step The cumulative noise ratio of is the two-norm; Based on the current status For the initial state predictions; During the aggregation process, a gradient optimization operation is performed to obtain a debiased embedding representation; the loss function of the gradient optimization operation is: Based on the historical behavior data of the user's previous interactions and the correlation between the user-item interactions in the interaction graph, the item embeddings that the user interacts with are obtained. : ， in, is the embedding matrix of items that users have interacted with in the past; is the embedding matrix of items that users may interact with in the future; The score of the user's recommended items is obtained by the inner product of the user embedding representation containing only the items that the user has interacted with in the past and the item representation that the user may have interacted with, and the cross entropy loss function is calculated: ， in, yes The transpose of is the parameter weight matrix for calculating the cross entropy loss function; is the activation function.

2. The method for contrastive learning debiasing intelligent recommendation based on diffusion model according to claim 1, characterized in that: S1 includes the following steps: S1-1, convert the user-item interaction graph information into the embedded representation of the user-item interaction, and derive the user With items Historical interaction sequence , determine the current item Interacting users , and then obtain the embedding representation of the item by uniformly distributed random sampling , where the embedding values ​​of items are randomly generated within a uniform distribution; S1-2, in the historical behavior sequence of user-item interaction, according to the items interacted with the current user, the embedding vectors of all interacting items are integrated to obtain the user embedding representation ; S1-3, the embedding representation of items interacting with users is integrated. It is based on the biased features, popularity features and semantic information of items in the user interaction process, fitting biased interactions to obtain the final user embedding feature representation and obtain the event sequence representation of user-item interaction features. .

3. The method for contrastive learning debiasing intelligent recommendation based on diffusion model according to claim 2, characterized in that: Item embedding representation , user embedding representation The calculation formula is as follows: ， ， in, is the embedded representation of the user; is the embedding representation of the item; For the The embedding representation of items is related to the user Interactive projects; For users With items Historical interaction sequences; is the embedding vector of the user's individual bias in the original user-item interaction graph; for Length of the module; is a symbolic function; is the Hadamard product symbol; It is hypergraph convolution; is the adjacency matrix; is the hypergraph convolutional coding representation.

4. The method for contrastive learning debiasing intelligent recommendation based on diffusion model according to claim 1, characterized in that: The formula for calculating the score of user-recommended interactive items is as follows: ， in, Indicates the rating of the user's recommended interactive items; is the inner product operation; is the splicing operation; When calculating, first perform splicing and then perform inner product operation; 、 Represent user embedding representation and item embedding representation respectively; represents the hyperparameter that regulates the influence of popularity bias; and Represent the bias features of users and items respectively.

5. The method for contrastive learning debiasing intelligent recommendation based on diffusion model according to claim 1, characterized in that: Also includes: Set a parameter matrix , and adjust the parameter matrix by gradient calculation To reduce the impact of bias error, it is expressed as follows: ， in, is the transpose symbol; It is a splicing symbol; 、 represent user embedding representation and item embedding representation respectively.

Images