Confounding factor removing multi-behavior recommendation method based on causal variation inference

Abstract

The invention discloses a causal variation inference-based confounding factor-removing multi-behavior recommendation method, which comprises the following steps of: encoding potential uncertainty in multi-behavior interaction by adopting a variation graph automatic encoder so as to capture heterogeneity among different behaviors; in order to efficiently deduce potential confounding factors, a confounding factor reasoning mechanism is designed, and the potential confounding factors are generated through variational reasoning. In a forward diffusion stage of the conditional diffusion module, the model gradually injects noise into potential variables of users and articles to simulate dynamic evolution of user preferences along with time and different situations. In the back diffusion stage, the model uses the deduced confounding factors and combines causal reasoning to guide the denoising process, the influence of potential confounding factors is relieved, and the real causal effect of multi-behavior interaction is captured. Experiments are carried out on two public data sets by aiming at ten recommendation algorithms of four different research emphasis, and experimental results show that the performance of the method is superior to that of existing algorithms of the same type.

Description

Technical Field

[0001] This invention relates to the field of multi-behavior recommendation technology, and mainly to a multi-behavior recommendation method based on causal variational inference to remove confounding factors. Background Technology

[0002] Traditional recommender systems primarily focus on modeling single behaviors, typically relying on implicit feedback data such as clicks or purchases, failing to fully capture users' personalized preferences across different contexts. In reality, user behavior is far more complex than a single action, often manifesting as multiple interaction types (such as browsing pages, adding to favorites, adding to cart, etc.). These multi-behavioral interactions provide richer insights into user preferences, revealing underlying needs. Therefore, multi-behavioral recommendation has become a key research focus. By analyzing the relationships between different behaviors, these systems can better capture user interests and preferences, thereby improving the effectiveness and accuracy of recommendations.

[0003] Existing research on multi-behavior recommendation mainly explores the dependence and heterogeneity of different user behaviors. For example, graph neural network methods capture complex user-item interactions by constructing multi-behavior graphs. Contrastive learning is also increasingly being applied to multi-behavior recommendation, aiming to address data sparsity and behavioral differences by constructing positive and negative sample pairs. However, these methods often ignore the influence of confounding factors on user multi-behavior interactions. Explicit confounding factors (such as popularity bias) will bias recommendations towards items with high click-through rates, while exposure bias will increase the visibility of certain items due to previous recommendations or external factors. Latent confounding factors (such as contextual influence and social relationships) are particularly difficult to identify or quantify

[47] . These latent confounding factors can distort the model's understanding of user preferences, leading to recommendations that do not match the user's actual interests, thereby weakening the model's ability to accurately capture personalized needs. Figure 2 As shown, users exhibited preferences for electronic products by browsing game consoles, saving laptops, and adding smartphones to their shopping carts. However, influenced by contextual factors (such as the Olympic atmosphere), users also browsed badminton rackets. Furthermore, social influences (such as friends' interest in table tennis) prompted users to save table tennis rackets, even though the users had no personal interest in the sport. These confounding factors introduced spurious correlations, masking users' true preferences.

[0004] In recent years, confounding factors have significantly hindered recommender systems from accurately capturing personalized user preferences, prompting researchers to develop methods to mitigate their impact. Mainstream approaches typically rely on domain experts to identify specific confounding factors and design targeted recommender models accordingly. For explicit confounding factors: adversarial training strategies are used to address the long-tail phenomenon (popularity bias), improving model performance by leveraging adversarial networks to learn implicit association structures from feedback data. Furthermore, segmentation bias models are introduced to handle sparse click data, while off-line estimators based on inverse bias scores (IPS) are used to reduce the impact of exposure bias. For latent confounding factors: causal inference provides a powerful tool to reveal the true relationship between user behavior and recommendation outcomes. By using backdoor adjustment operations, researchers can simulate user behavior without removing confounding factors, thereby estimating the true causal effect of recommendations.

[0005] While existing causal inference methods have made some progress in mitigating the influence of confounding factors, significant challenges remain. First, in multi-behavior recommendation scenarios, one of the core challenges is how to effectively infer the potential confounding factors influencing different types of user interactions. Current research often neglects the necessity of identifying and modeling these potential confounding factors, which, if not properly addressed, can lead to biased recommendation results. Second, due to the heterogeneity of user behavior, effectively eliminating the influence of potential confounding factors on multi-behavior interactions becomes crucial. In multi-behavior recommendation scenarios, the challenge lies in dealing with the interference caused by potential confounding factors. These factors can severely distort user preference modeling, making it difficult for the model to accurately capture users' true preferences. Summary of the Invention

[0006] Objective: To address the problems existing in the aforementioned background technologies, this invention proposes a causal variational inference method (CVID) suitable for decontaminating multi-behavior recommendation. CVID effectively handles confounding factors in multi-behavior recommendation scenarios using causal inference, ensuring that the model learns the true causal effects between user multi-behavior interactions and items. CVID comprises four key components: a variational graph autoencoder, used to capture the complex relationships between multiple behaviors and capture potential uncertainties, generating multi-behavior latent variables; a confounding factor inference module, which uses variational inference and the latent variables generated by the variational graph autoencoder to infer potential confounding factors, laying the foundation for the subsequent conditional diffusion process; a conditional diffusion module, which combines the inferred potential confounding factors and uses backdoor adjustment techniques in causal inference to recover the original latent variables from noise, removing interference caused by potential confounding factors; and finally, a lightweight graph convolution module is used to aggregate multi-behavior representations, thereby generating accurate recommendation results.

[0007] To effectively infer potential confounding factors influencing different types of user interactions, this invention constructs a causal graph to describe causal relationships and performs causal reasoning on the impact of confounding factors in multi-behavioral scenarios. For example... Figure 3 As shown, each node is a causal variable, and the directed edges in the nodes are causal paths, where A→B means that A directly affects B.

[0008] The cause-effect graph contains five causal nodes: C, I, T, A, and Y. C represents confounding factors, including but not limited to the popularity of the item, the user's social influence, and contextual factors. I represents the interaction between the user and the item, T represents the user's target behavioral interaction (purchasing behavior), A represents the user's auxiliary behavioral interaction (browsing, clicking, saving, adding to cart), and Y represents the model's prediction.

[0009] Confounding Factors C → User-Item Interaction I: Confounding factors directly influence the user's interaction behavior I between the user and the item, which can be represented as P(I|C). For example, the popularity of an item may lead users to interact more with popular items than with items that truly match their preferences. Furthermore, unobserved confounding factors such as social attributes and contextual factors can influence the decision-making process; for instance, during the Olympics, a person who doesn't usually exercise might browse table tennis rackets.

[0010] User item interaction I → Target behavior interaction T and auxiliary behavior interaction A: The user's overall interaction behavior I further determines the more specific target behavior T and auxiliary behavior A. For example, a user's browsing behavior may ultimately lead to a purchase or adding certain items to their shopping cart. This shows that user item interaction is the foundation of target behavior and auxiliary behavior.

[0011] Target behavior interaction T and auxiliary behavior interaction A → Predicted result Y: The multi-behavior recommendation model predicts the final user preference by analyzing the user's target behavior T and auxiliary behavior A, denoted as P(Y|T,A).

[0012] Confounding Factors C → Predicted Result Y: Confounding factors not only directly affect the user's interaction behavior I with the item, but also directly affect the model's prediction Y. For example, certain popularity factors may cause the model to be more inclined to recommend popular items, ignoring the user's true preferences.

[0013] In multi-behavior recommendation scenarios, confounding factors C influence users' various behavioral interactions I and the model's final prediction Y, forming a backdoor path from I to Y: I←C→Y. For example, contextual factors may cause users to interact multiple times with items that do not match their actual preferences. If only P(Y|I) is optimized, the model may use this interaction information caused by confounding factors for prediction during training, leading to misjudgments. As a result, the model may over-recommend certain popular items while ignoring the user's true preferences, reducing recommendation accuracy and user satisfaction.

[0014] To address this issue, we not only improve model performance by eliminating the influence of confounding factors, but also optimize the recommendation system using causal inference methods to ensure that the model learns the true causal relationships under the influence of confounding factors. Specifically, we capture the stable causal relationship between user-item interaction I and user preference Y by optimizing Pθ(Y|do(I)) instead of Pθ(Y|I). In causal inference, the backdoor adjustment controls the confounding factor C through do-operations, making I independent of its parent node C, thereby estimating the causal effect of user behavior on the outcome, i.e., the impact of the user's true preference on the recommendation result. This not only cuts off the backdoor path I←C→Y, but also ensures that the model can learn the causal effect of real user interaction on user behavior by optimizing Pθ(Y|do(I)). In this way, we can effectively eliminate the interference of confounding factors on the model's prediction, thereby significantly improving its accuracy and robustness.

[0015] Therefore, this invention proposes a variational graph autoencoder (VGAE) and a confounding factor inference module. The VGAE models the potential uncertainty in user behavior through variational inference, effectively capturing the dependencies between multiple behaviors and alleviating the data sparsity problem in multi-behavior recommendation scenarios. The confounding factor inference module, based on the output of the VGAE, generates potential confounding factors through variational inference, laying the foundation for the subsequent conditional diffusion process. By simultaneously addressing the complexity of multi-behavior interactions and the interference of confounding factors, the model ensures that the learned user behavior representation is decoupled from confounding effects. To effectively eliminate the influence of potential confounding factors on multi-behavior interactions, this invention introduces a conditional diffusion module. This module combines the latent variables generated by the VGAE with the latent confounding variables generated by the confounding factor inference module. During forward diffusion, noise is gradually added to simulate the dynamic evolution of user preferences over time and in different contexts. During backward diffusion, the inferred potential confounding factors are combined, and the backdoor adjustment technique in causal inference is used to recover the original latent variables from the noise, enabling the model to block the causal path I←C→Y. This ensures that user behavior I is independent of confounding factors C, removing interference from potential confounding factors and more accurately estimating P(Y|do(I)). The conditional diffusion module effectively decouples the influence of user behavior from confounding factors, ensuring that the learned causal effects reflect the true relationship between user behavior and item interactions. Finally, a lightweight graph convolution module iteratively aggregates user and item representations through multi-layer graph convolutions to generate the final recommendation result. A multi-head attention mechanism is used to weight and aggregate latent variables of multiple behaviors, further emphasizing the heterogeneity between different user behaviors, thereby improving the accuracy of recommendations.

[0016] Technical solution: To achieve the above objectives, the technical solution adopted by this invention is as follows:

[0017] A multi-behavior recommendation method based on causal variational inference to remove confounding factors includes the following steps:

[0018] Step S1: Construct a multi-behavior recommendation model by importing multi-behavior type interaction data of users and projects and converting it into a graph structure.

[0019] Step S2: Use variational graph autoencoder to generate latent variables for multiple behaviors to capture the complex interaction relationships and heterogeneity of behaviors between users and items, and explicitly model the potential uncertainties in the interaction.

[0020] Step S3: Generate potential confounding factors through the confounding factor inference module to lay the foundation for the subsequent conditional diffusion process;

[0021] Step S4: Based on the conditional diffusion model, noise is gradually introduced during the forward diffusion process to simulate the dynamic evolution of user preferences in different contexts. During the backward diffusion process, the inferred potential confounding factors are combined, and the backdoor adjustment technique in causal inference is used to recover the original potential variables from the noise and remove the interference caused by potential confounding factors.

[0022] Step S5: Aggregate multi-behavior embeddings using a lightweight graph convolutional network, leverage multi-head attention mechanism to weigh the contribution of different behaviors to user preferences, and generate the final user and item embedding representations through multi-layer convolution.

[0023] Step S6: Train and optimize the multi-behavior recommendation model. Adjust the loss functions of VGAE, confounding factor inference module and conditional diffusion module through joint optimization strategy to ensure the improvement of model performance.

[0024] Step S7: The evaluation module is used to evaluate the performance of the trained model using predetermined evaluation metrics. The multi-behavior recommendation model predicts the probability of user interaction with each item on the target behavior (purchase), and ranks all recommended items based on the interaction probability, selecting the top N items with the highest probability to form a sorted recommendation list.

[0025] Based on the above technical solution, the present invention can be further improved as follows:

[0026] As an embodiment of the present invention, the process of step S1 includes:

[0027] S1-1. Define the user set U = {u1,...,u...} m ,...,u M} Item set I = {i1,...,i n ,...,i N}, where M and N are the number of users and items, indexed by m and n;

[0028] S1-2. Define a multi-behavior set B = {v, f, c, p} (browsing, favorites, adding to cart, and purchasing) indexed by b. Simultaneously, consider the behavior to be predicted (purchasing) as the target behavior t, and other behaviors (browsing, favorites, adding to cart) as auxiliary behaviors b′;

[0029] S1-3, Define the multi-behavior user-item interaction graph as G. b =(V,E) b ), where V represents the set of nodes including users and projects, E b This represents an edge set that includes multiple similar user behaviors;

[0030] S1-4. Define a tensor To reflect the user's multiple behavioral interaction types, if user u m ∈U and item i n If ∈I interacts under behavior type b∈B, then It is set to 1.

[0031] It should be understood that multi-behavioral interaction data of users is introduced and defined, as well as a tensor to reflect the multi-behavioral types of user interactions. This prepares for capturing users' personalized preferences in the future.

[0032] In the above embodiments, by importing multi-behavioral interaction data of users and items and converting it into a graph structure, it is easier to capture rich contextual information between users and items in the future, thereby encoding embeddings of users and items with high-order connectivity, which significantly improves the accuracy of multi-behavioral recommendations.

[0033] As an embodiment of the present invention, the process of step S2 includes:

[0034] S2-1, In the four behavioral interaction diagrams (G v G f G c G p The variational graph autoencoder (GCN) is used to generate latent variables for multiple behaviors, capturing the complex interaction relationships and behavioral heterogeneity between users and items, and explicitly modeling the potential uncertainties in the interaction. The GCN calculates the mean μ and standard deviation σ of each node to represent the latent distribution of each node.

[0035] μ b =GCN mean (A b E)

[0036] logσ b =GCN lstd (A b E)

[0037] In GCN, 'b' represents the graph convolution operation, 'b' represents the multi-row interaction type, 'E' represents the initial embedding of the user and item generated through the initial embedding layer, and 'A' represents the initial embedding of the user and item. b is the adjacency matrix corresponding to the multi-row interaction graph; by performing a convolution operation on each node, the encoder generates the mean μ and standard deviation μ of each node, thereby representing the distribution of each node in the latent space.

[0038] S2-2, In order to generate the latent variable z b Using the reparameterization technique, the latent variable z b Represented as:

[0039] z b =μ b+σ b ⊙∈

[0040] in The reconstructed multi-row adjacency matrix is ​​given by φ, where φ is the activation function Sigmoid, and z... b ·z b T This indicates the calculation of similarity among latent variables.

[0041] S2-3, Latent variable z generated by the encoder b The decoder reconstructs the user-item interaction matrix by utilizing the node relationships in the graph structure. The reconstruction process is as follows:

[0042]

[0043] in The reconstructed multi-row adjacency matrix is ​​given by φ, where φ is the activation function Sigmoid, and z... b ·z b T This indicates the calculation of the similarity of latent variables.

[0044] S2-4. The variational graph autoencoder optimizes the model using the lower bound of evidence. The optimization process is as follows:

[0045]

[0046] Where D KL KL divergence is used to measure the posterior distribution q of the latent variable. φ The difference between (z|X) and the prior distribution p(z), where σ is the standard deviation and μ is the mean; reconstruction loss. Adjacency matrix for measuring reconstruction With the true adjacency matrix A b The difference between them is that u and i represent users and projects, and n represents the dimension of the latent variable;

[0047] S2-5. By combining the KL divergence and reconstruction loss, the loss of the variational graph autoencoder is calculated as follows:

[0048]

[0049] in For the loss of the variational autoencoder, The reconstruction loss is β, and the equilibrium KL divergence is β. Hyperparameters.

[0050] It should be understood that by generating latent variables with multiple behaviors through variational graph autoencoders, it is possible to capture the complex interaction relationships and behavioral heterogeneity between users and items. Variational graph autoencoders model latent uncertainties by calculating the mean and log-standard deviation of nodes, and combine KL divergence and reconstruction loss to optimize the model, thereby accurately learning the multi-behavioral relationships between users and items.

[0051] In the above embodiments, a variational graph autoencoder is applied to four behavioral interaction graphs to generate multi-behavioral latent variables. By explicitly modeling the uncertainty in the interactions, the model's ability to capture the complexity and heterogeneity of multi-behavioral data is improved. This method effectively improves the accuracy of multi-behavioral recommendations by optimizing KL divergence and reconstruction loss.

[0052] As an embodiment of the present invention, the process of step S3 includes:

[0053] S3-1, Using an encoder on the latent variable z b Encode to generate the mean of confounding factors. and standard deviation The expression is as follows:

[0054]

[0055] S3-2, By analyzing the latent variable z b Using reparameterization techniques, initial potential confounding factors are generated to model latent factors such as user context and social influence, as shown in the following expression:

[0056]

[0057] in ∈ represents the initial potential confounding factor, ∈ represents random noise, and ⊙ represents the element-wise multiplication operation, which multiplies the standard deviation with the noise element-wise by click, thereby capturing the confounding factor.

[0058] S3-3 uses a decoder to map the latent variables to a low-dimensional space. After processing with the parameter matrix W and a nonlinear activation function, it further generates the final confounding latent variables, as shown in the following expression:

[0059]

[0060] in For the final potential confounding factors, Initial potential confounding factors, where φ is the activation function Sigmoid.

[0061] S3-4. The confounding factor inference engine optimizes the model through the lower bound of evidence. The optimization process is as follows:

[0062]

[0063] Where D KL KL divergence is used to measure the posterior distribution of latent variables. With prior distribution The difference between them, where σ is the standard deviation and μ is the mean; reconstruction loss Measuring the final potential confounding factors Initial potential confounding factors The difference between them, where i represents the item, The loss of the confounding factor inferrer is β, where β is the equilibrium KL divergence. hyperparameters

[0064] It should be understood that by encoding latent variables to generate the mean and standard deviation of confounding factors, and using reparameterization techniques to generate latent confounding factor variables, it is possible to effectively model latent factors such as user context and social influence. The confounding factor inference engine uses a decoder to map latent variables to a low-dimensional space and optimizes the model through lower bounds of evidence, thereby effectively capturing and controlling the impact of latent factors on recommendation results.

[0065] In the above embodiments, by encoding latent variables to generate latent variables of confounding factors, the confounding factor inferencer can effectively infer latent confounding factors in multi-behavior recommendation scenarios. This method, through the combination of lower bounds of evidence, reparameterization techniques, and decoder structures, enables the model to more accurately reflect users' true preferences and improves the accuracy of multi-behavior recommendations.

[0066] As an embodiment of the present invention, the process of step S4 includes:

[0067] SS4-1, In the forward propagation process of the conditional diffusion model, the model is progressively fed into the latent variable z generated by the variational graph autoencoder. b Add noise to generate noisy latent variables at different time steps t. The expression is as follows:

[0068]

[0069] Where t∈{1,...,T} is the time step, b is the multi-row interaction type, N is a Gaussian distribution, I is the identity matrix, and α t It is a scheduling parameter related to time step t. This represents the cumulative noise figure from time step 0 to time step t. For noisy latent variables, These are potential variables at time step 0;

[0070] S4-2. By recursively adding noise, noisy latent variables are gradually generated, as shown in the following expression:

[0071]

[0072] S4-3. In the reverse denoising process, the conditional diffusion model incorporates confounding factors. Using backdoor adjustment techniques in causal inference to recover the original latent variables from noise. To ensure that confounding factors are eliminated during the denoising process to accurately capture users' true preferences, the expression is as follows:

[0073]

[0074] Where p θ Represents the reverse process. These are the potential variables for denoising generated at the last time step in the reverse process. Let N be the latent variable representing the confounding factors, N be a Gaussian distribution, t be the time step, and μ be the latent variable representing the confounding factors. θ Σθ is a function for parameterizing the neural network, where Σθ is the covariance matrix used to control the uncertainty of the Gaussian distribution;

[0075] S4-4. The loss function of the conditional diffusion model is used to optimize the model's performance in the denoising process. By minimizing the difference between the noise predicted by the model and the actual noise, the model can learn more accurately how to remove noise, thereby effectively restoring the real interaction relationship between users and items. The expression is as follows:

[0076]

[0077] in This represents the error between the noise ∈ predicted by the model and the actual noise ∈ θ. This indicates that the expected value is being calculated. Loss due to conditioned diffusion, For the latent variables of the final time step, The multiple rows of output for conditional diffusion are latent variables.

[0078] Understandably, the conditional diffusion model progressively adds noise to latent variables during the forward pass to generate noisy latent variables across multiple time steps, thereby introducing dynamic changes in multi-behavioral interactions. In the reverse denoising process, latent variables with confounding factors are incorporated, and the denoising process is guided by backdoor adjustment techniques from causal inference, reducing the interference of confounding factors on user behavior prediction. Ultimately, the model is optimized by minimizing the noise prediction error, enabling it to accurately reconstruct the true interaction relationship between users and items.

[0079] In the above embodiments, the conditional diffusion model effectively captures the dynamic evolution of multi-behavioral data by adding noise forward and denoising backward. Combined with a confounding factor removal process, it ensures the model accurately recovers latent variables, avoiding interference from potential confounding factors on recommendation accuracy. Through loss function optimization, the model significantly improves denoising performance, making the multi-behavioral recommendation system more accurate in capturing users' true preferences.

[0080] As an embodiment of the present invention, the process of step S5 includes:

[0081] S5-1. In multi-behavior recommendation scenarios, different behaviors have different weights for users. A multi-head attention mechanism is used to weight and aggregate these behaviors, enabling the model to better understand the impact of various user behaviors on their final preferences. The expression is as follows:

[0082]

[0083] Where B represents the number of multiple actions. For conditional diffusion, the output consists of multiple rows of latent variables. The numerical function ensures that the weights are positive, thus improving stability. The attention score of behavior b in attention head h determines the degree to which the behavior contributes to the final aggregated representation. This involves weighted summation of latent variables for different behaviors using attention weights. `concat` is the concatenation operation, and `z` represents the final concatenated multi-behavior fusion representation.

[0084] S5-2, Lightgcn aggregates the embedded representations of users and items through multiple iterations, capturing high-order collaborative filtering signals in the interaction graph layer by layer, thereby more accurately characterizing the relationship between user preferences and item features. The expression is as follows:

[0085]

[0086] Where l is the number of graph convolutional layers, z i (l) z is a latent variable for the project when the number of graph convolutional layers is l. u (l+1) Let N be the latent variable for the user when the number of graph convolutional layers is l+1. u N i A collection of users and projects The normalization factor is used to balance the influence of node degree;

[0087] S5-3. Aggregate the user and item embeddings generated at each layer in Lightgcn to obtain the latent variables of the final user and item, as shown in the following expression:

[0088]

[0089] Where L is the total number of graph convolutional layers, E u E i For the final embedded representation of users and items used to calculate user preferences, z u l ,z i l These are the latent variables for users and items when the number of graph convolutional layers is l;

[0090] S5-4. User preference prediction for items is achieved by calculating the inner product of user embedding and item embedding:

[0091]

[0092] in E represents the probability of interaction between user u and item i under behavior b. u E i This is an embedded representation of users and items ultimately used to calculate user preferences.

[0093] It should be understood that in multi-behavior recommendation scenarios, different behaviors have varying impacts on user preferences. Weighted aggregation of multi-behavior embeddings using a multi-head attention mechanism can provide a better understanding of the role of each behavior in user preferences. LightGCN iteratively aggregates user and item embeddings layer by layer, capturing high-order collaborative filtering signals in the interaction graph, and generates the final latent variables for users and items through average aggregation. Ultimately, user preference prediction for items can be achieved by calculating the inner product of user and item embeddings.

[0094] In the above embodiments, a multi-head attention mechanism is used to weighted aggregate multiple behavior embeddings, enabling the model to accurately grasp the relative impact of different behaviors on user preferences. Through high-order collaborative filtering signals aggregated layer by layer using LightGCN, the model better characterizes the relationship between users and items, achieving accurate preference prediction. This method has a significant effect on improving recommendation accuracy.

[0095] As an embodiment of the present invention, the process of step S6 includes:

[0096] S6-1. Bayesian Personalized Ranking (BPR) is an optimization method specifically designed for implicit feedback recommendation tasks. BPR learns model parameters by maximizing the preference differences between interacted and uninteracted items, thereby effectively improving the accuracy and personalization of recommendations.

[0097]

[0098] Where lnσ(y u,i+ -y u,i -) Calculate user u against positive sample i + The relative preference between the negative sample i and the negative sample i-, λ‖θ‖ 2 This is the regularization term. S6-2, the loss function expression for multi-behavior recommendation is as follows:

[0099]

[0100] in

[0101] For the loss of a multi-behavior recommendation model, For the loss of the variational autoencoder, It is a loss due to conditioned diffusion. λ is the loss inferred from confounding factors, and λ is a hyperparameter.

[0102] It should be understood that Bayesian Personalized Ranking (BPR) is an optimization method for implicit feedback recommendation tasks. By maximizing the difference in user preferences for interacted and uninterrupted items, BPR can effectively improve the personalization and accuracy of recommendations. In multi-behavior recommendation scenarios, the loss function combined with the optimization objective of BPR further enhances the model's understanding of user behavior preferences, making the recommendation results more in line with the user's actual needs.

[0103] In the above embodiments, by applying the BPR optimization method, the model can effectively distinguish the differences in user preferences for interacted and uninterrupted items, improving the accuracy of multi-behavior recommendations. Combined with the multi-behavior loss function, the model performs even better in capturing users' personalized preferences, achieving more accurate and personalized recommendation results.

[0104] As an embodiment of the present invention, the process of step S7 includes:

[0105] S7-1. Normalized depreciation cumulative gain measures the quality of the recommendation list, considering the relevance of items in the list and their position in the list. Higher weights are given to items that rank higher and have higher relevance. Normalized depreciation cumulative gain is used as the evaluation metric, and the expression is as follows:

[0106]

[0107] Where N represents the top N items in the recommendation list, DCG takes into account the positional relationship between the sorting, and IDCG is the best sorting result for the current sorting; the higher the value of NDCG, the better the performance of the recommendation system.

[0108] S7-2. Hit Rate (HR) measures the accuracy of recommendations. Specifically, it measures the percentage of recommendations that contain at least one item that a user is actually interested in. The expression for Hit Rate as an evaluation metric is as follows:

[0109]

[0110] NumbersofHits@10 represents the number of users whose items from the test set appear in the Top-N recommendation list, and TsetSets represents the total number of users in the test set.

[0111] It should be understood that Normalized Discounted Cumulative Gain (NDCG) and Hit Rate (HR) are commonly used evaluation metrics in multi-behavioral recommendation systems. NDCG considers the relevance and ranking of items in the recommendation list, assigning higher weights to highly relevant items ranked higher; a higher NDCG value indicates better performance of the recommendation system. Hit Rate, on the other hand, measures the accuracy of the recommendation list, reflecting the proportion of items in the list that the user is actually interested in, thus providing a comprehensive evaluation of the recommendation effect.

[0112] In the above embodiments, using NDCG and HR as evaluation metrics can effectively assess the quality and accuracy of the recommendation list. NDCG, by combining item ranking and relevance weights, ensures that high-quality items are displayed first; HR reflects the degree to which the recommendation list matches the user's actual needs, which helps optimize the accuracy and user satisfaction of the multi-behavior recommendation system.

[0113] Beneficial effects:

[0114] This invention integrates variational inference, confounding factor inference, conditional diffusion model, and graph convolutional network to effectively remove confounding factors in multi-behavioral interactions, thereby improving recommendation accuracy and model robustness. Experiments were conducted on two public datasets against 11 recommendation algorithms from four different research focuses, including the algorithm of this invention. The experimental results show that the performance of this invention is superior to existing algorithms of the same type. Attached Figure Description

[0115] Figure 1 This is a schematic diagram of the system architecture in this invention;

[0116] Figure 2 This is a diagram illustrating multi-behavior interactions;

[0117] Figure 3 It is a multi-behavior cause-effect graph;

[0118] Figure 4 This is a schematic diagram of an ablation experiment analysis;

[0119] Figure 5 This is a schematic diagram of multi-behavioral data ablation experiment analysis;

[0120] Figure 6 This is a diagram illustrating the effects of confounding factors;

[0121] Figure 7 This is a graph showing the experimental analysis of attention hyperparameters;

[0122] Figure 8 This is an experimental analysis diagram of conditional diffusion hyperparameters;

[0123] Figure 9 This is an experimental analysis diagram of the hyperparameters of the optimization penalty term; Detailed Implementation

[0124] The present invention will be further described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0125] Research work related to this invention:

[0126] Multi-behavior recommendation

[0127] Multi-behavior recommendation, by analyzing various user interactions (such as browsing, favorites, adding to cart, and purchasing), enriches users' personalized preferences from multiple perspectives and has attracted widespread attention from researchers. Previous multi-behavior recommendation models mainly used matrix factorization to capture user preferences. For example, IMPGCN constructs and decomposes multiple matrices, including non-behavior-specific user matrices, single-behavior user matrices, and combined behavior-specific topic matrices, to learn implicit embeddings and model user preferences by estimating users' interests in various topics. Recently, some researchers have attempted to develop neural network models to capture diverse interactions between users and items. For example, EHCF captures complex relationships between different behaviors by constructing correlation predictions of heterogeneous behaviors, while using a non-sampling strategy to optimize the model so that it can effectively learn parameters from the entire heterogeneous dataset. However, these models do not encode high-order collaborative information for heterogeneous user-item interactions. To address this issue, GHCF incorporates node (user and item) representations and relational embeddings into multi-relation predictions to explore high-order heterogeneous user-item interactions.

[0128] Cause and effect recommendation

[0129] In recommender systems, confounding factors can cause bias and reduce the model's ability to predict user preferences, thus affecting the accuracy of recommendation results. Confounding factors can be categorized into observable (e.g., item popularity and user attribute information) and unobservable (e.g., user social influence and contextual factors). Causal inference, by removing these confounding factors, has been widely applied in recommender systems. Some works employ inverse propensity score (IPS) to reweight observed data to mitigate selection and exposure bias. However, the IPS method performs poorly when handling complex high-order interaction information, and its performance is highly dependent on the accurate estimation of propensity scores. To address this issue, some researchers use causal graphs to incorporate confounding factors into the data generation process, mitigating bias through causal inference. While causal graphs can intuitively capture causal relationships, their construction relies on reasonable assumptions and may be limited when facing unknown or difficult-to-observe confounding factors. To address these challenges, front-door and back-door adjustment techniques have been introduced to cut causal paths to remove confounding effects. CausalD proposed a causal multi-teacher distillation framework, using front-door adjustment to effectively remove unobservable confounding factors from the model. However, the effectiveness of front-door and back-door adjustments relies on an explicit causal structure and may not be able to handle all complex causal relationships. Counterfactual reasoning eliminates the influence of confounding factors by estimating the possible outcomes under unobserved circumstances. However, its high computational complexity and data requirements limit its widespread application.

[0130] Recommendation based on diffusion model

[0131] By integrating the diffusion process into recommender systems, the forward noise addition and backward denoising processes of the diffusion model are utilized to simulate the dynamic propagation of user preferences, effectively capturing the characteristics of user preferences changing over time, thereby significantly improving the accuracy of recommendations. For example, DiffuRec was the first to apply the diffusion model to recommender systems, better understanding user intent by fusing target item embeddings to generate representations of historical interaction items. DiffuASR proposed a diffusion-based pseudo-sequence generation framework, generating enhanced sequences from the original data and designing classifier-guided and classifier-free diffusion strategies, making the enhanced sequences generated by the diffusion model more consistent with actual user preferences. DiffGT proposed a diffusion graph converter model for Top-k recommendations, integrating the graph Transformer with a linear attention module during the backward diffusion process to reverse the effects of early noise addition and recover true user preferences. DiffKG proposed a knowledge graph diffusion model, combining generative diffusion models with data augmentation paradigms to achieve robust knowledge graph representation learning and effectively improving the model's recommendation performance through contrastive learning. Finally, DiffRec proposed a diffusion recommendation model with two variants. L-DiffRec significantly reduces model parameters and memory costs when predicting large-scale projects by combining clustering and diffusion, while T-DiffRec captures temporal information in the interaction sequence through a simple and effective time-aware reweighting strategy.

[0132] Recommendation based on variational encoder

[0133] In recommender systems, variational autoencoders (VAEs) effectively capture complex patterns and relationships in user behavior by encoding user-item interaction data into a low-dimensional latent space. VAEs learn the latent distribution of data by maximizing the likelihood function of observed data, thus significantly enhancing the ability to model latent user preferences. For example, RecVAE, by embedding VAEs within a collaborative filtering framework, successfully alleviates the data sparsity problem by modeling the implicit distribution of user preferences, generating more robust user representations and significantly improving recommendation performance. Furthermore, NSVAE, based on VAE prediction of user preferences, designs a self-supervised task, considering both local and global user preferences, further optimizing recommendation performance. However, variational graph autoencoders (VGAEs) exhibit even more significant advantages when processing graph-structured data. Combining graph convolutional networks (GCNs) with variational inference, VGAEs not only extend the capabilities of VAEs but also more effectively capture higher-order relationships between users and items. For example, GraphVAE extends VAEs to graph data, using GCNs as encoders to aggregate information about node neighbors, thereby generating richer node representations. Based on this, NGCF-VGAE further integrates GCN and variational inference, significantly improving the quality of user and item embeddings, demonstrating stronger robustness in handling sparse and noisy interactive data, and enhancing the model's generalization ability under different data distributions.

[0134] This invention provides a multi-behavior recommendation method based on diffusion contrastive learning, combined with... Figure 1 It includes the following steps:

[0135] Step S1: Construct a multi-behavior recommendation model by importing multi-behavior interaction data of users and projects, and converting it into a graph structure. For example... Figure 2 The diagram illustrates a multi-behavior interaction, where a user engages in a series of multi-behavior interactions with the project in a real-world scenario. Therefore, given a user set U = {u1,...,u...} m ,...,u M} Item set I = {i1,...,i n ,...,i N Let M and N be the number of users and items, respectively, and be indexed by m and n. Introduce and define a multi-behavior set B = {v, f, c, p} (browsing, favorites, adding to cart, and purchasing) indexed by b. Simultaneously, the behavior to be predicted (purchasing) is considered the target behavior. Other behaviors (browsing, saving, adding to cart) are considered auxiliary behaviors b′. This invention defines a multi-behavior user-item interaction graph as G. b =(V,E) b ), where V represents the set of nodes including users and projects, E bThis represents a set of edges that include multiple similar user behaviors. Furthermore, this invention defines a tensor. To reflect the user's multiple behavioral interaction types, if user u m ∈U and item i n If ∈I interacts under behavior type b∈B, then It is set to 1.

[0136] Step S2: In multi-behavior recommendation scenarios, users interact with items through various behaviors (such as browsing, favorites, adding to cart, and purchasing). These behaviors are often influenced by confounding factors (such as user context and social influence), resulting in complex dependencies and potential uncertainties. Traditional graph convolutional neural networks rely on multi-level neighbor aggregation operations when processing these interactions, making it difficult to fully capture the diversity and uncertainty in user behavior. Therefore, this paper introduces a variational graph autoencoder (VGAE), which explicitly models the potential uncertainties in user-item interactions, better capturing the complex patterns in multi-behavior interactions. The VGAE consists of an encoder and a decoder.

[0137] Encoder: The encoder encodes the multi-behavior interaction graph between users and items using a graph convolutional network (GCN), generating latent variables to capture the complex relationships between users and items. Given a multi-behavior user-item interaction graph G = (V, E), the VGAE encoder calculates the mean μ of the nodes using a GCN. b and standard deviation σ b , used to represent the potential distribution of each node:

[0138] μ b =GCN mean (A b E)

[0139] logσ b =GCN log_std (A b E)

[0140] Where GCN represents graph convolution operation, b represents multi-behavior interaction type, E represents the initial embedding of user and item generated through the initial embedding layer, and A... b The adjacency matrix corresponds to the multi-row interaction graph. By performing a convolution operation on each node, the encoder generates the mean μ and standard deviation μ of each node, thus representing the distribution of each node in the latent space. This is to generate the latent variable z. b VGAE uses the reparameterization technique to transform the latent variable z b Represented as:

[0141] z b =μ b +σb ⊙∈

[0142] Where ∈~N(0,I) represents noise sampled from the standard normal distribution, and ⊙ represents element-wise product.

[0143] Decoder: Based on the latent variable z generated by the encoder b The decoder reconstructs the user-item interaction matrix by utilizing the node relationships in the graph structure. The reconstruction process is as follows:

[0144]

[0145] Here, φ is the activation function. Through the inner product operation, the decoder can capture the implicit association between users and items. VGAE optimizes the model using the Evidence Lower Bound (ELBO):

[0146]

[0147] The KL divergence measures the difference between the posterior distribution qφ(z|X) and the prior distribution p(z) of the latent variables. Reconstruction loss. Adjacency matrix for measuring reconstruction With the true adjacency matrix A b The difference between them. By combining KL divergence and reconstruction loss, the optimization objective of VGAE is as follows:

[0148]

[0149] Step S3: In multi-behavior recommendation scenarios, users' diverse interactive behaviors are often affected by various confounding factors. These factors include not only "explicit confounding factors" such as popularity and exposure confounding factors that can be manually identified, but also "latent confounding factors" such as contextual influences and social relationships that are difficult to identify or measure. These unobserved factors, as confounding factors, may obscure the user's true intentions, causing the model to reduce its ability to predict user preferences.

[0150] To address this issue, this paper designs a confounding factor inference module. This module is based on the latent variable z generated by VGAE. b Inferring confounding factors These latent variables are used to model and control for the impact of unobserved external factors on user behavior, thereby reducing their interference with the recommender system. Compared to VGAE, which encodes graph structure information in user-item interactions, the confounding factor inferencer generates unobserved confounding factors. Although both use variational inference techniques, the confounding factor inferencer, by generating unobserved confounding factors, helps the model remove the interference of these confounding factors on user behavior prediction through causal inference, ensuring the elimination of backdoor paths I←C→Y in the multi-behavior causal graph, and improving the accuracy and robustness of the recommender system.

[0151] First, use an encoder on the latent variable z b Encode the data to generate the mean of the confounding factors. and standard deviation The encoder's main task is to extract hidden confounding information from user behavior and interaction data, and generate latent distribution parameters using the following formula:

[0152]

[0153] in and Let represent the mean and standard deviation of the confounding factors, and φ be the activation function. and For encoder parameters, z b The multi-row latent variables are the output of the variational graph autoencoder. By analyzing the latent variable z... b Using reparameterization techniques, latent confounding variables are generated to model hidden factors such as user context and social influence.

[0154]

[0155] in For the final potential confounding factors, The initial latent confounding factors are represented by φ, which is the sigmoid activation function. The latent confounding variables generated through the aforementioned reparameterization technique are then mapped to a low-dimensional space using a decoder. After processing by the parameter matrix W and a nonlinear activation function, the final latent confounding variables are generated. These latent variables represent unobserved external confounding factors such as user context, social influence, and temporal popularity, and are used in the model's causal inference process.

[0156]

[0157] in For the final potential confounding factors, The initial potential confounding factor is φ, which is the sigmoid activation function. The confounding factor inference engine optimizes the model using a lower bound on the evidence:

[0158]

[0159] Where D KL KL divergence is used to measure the posterior distribution of latent variables. With prior distribution The difference between them, where σ is the standard deviation and μ is the mean; reconstruction loss Measuring the final potential confounding factors Initial potential confounding factors The difference between them, where i represents the item, The loss of the confounding factor inferrer is β, where β is the equilibrium KL divergence. Hyperparameters.

[0160] Step S4: In multi-behavior recommendation scenarios, users' various interactive behaviors (such as context, social influence, and temporal popularity) are often interfered with by multiple confounding factors, blurring the user's true personalized preferences. To address this issue, this paper proposes a conditional diffusion model, which uses latent variables z generated by VGAE. b Furthermore, the confounding factor zc is introduced into the reverse denoising step of the diffusion process. By adjusting the backdoor, user behavior and confounding factors are effectively decoupled, and the causal path I←C→Y is effectively blocked, thereby avoiding the interference of confounding factors on the recommendation results.

[0161] In the forward denoising process, the model uses latent variables generated by VGAE to progressively introduce noise, generating a series of noisy latent variables to simulate the influence of confounding factors on user preferences during item interactions. Next, the reverse denoising process restores the user's original latent variables through progressive denoising and incorporates latent confounding factors generated by the confounding factor inferrer, providing causal intervention during the denoising process. This process ensures that the model can effectively control confounding factors, thereby accurately capturing the true causal effect of user interaction behavior I on outcome Y, significantly improving the prediction accuracy and robustness of the recommendation system.

[0162] Forward noisy addition process: During the forward diffusion process, the model is gradually fed into the latent variable z generated by VGAE. b Add noise to generate noisy latent variables at different time steps t.

[0163]

[0164] Where t∈{1,...,T} is the time step, b is the multi-row interaction type, N is a Gaussian distribution, I is the identity matrix, and α t It is a scheduling parameter related to time step t. This represents the cumulative noise figure from time step 0 to time step t. For noisy latent variables, These are the latent variables at time step 0. By recursively adding noise, the model progressively generates noisy latent variables:

[0165]

[0166] Inverse denoising process: In the inverse denoising process, the conditional diffusion model recovers the original latent variable z from the noise by incorporating the latent variable zc of confounding factors and using the backdoor adjustment technique in causal inference. b This ensures that confounding factors are eliminated during the denoising process to prevent interference with user behavior prediction, thereby accurately capturing users' true preferences.

[0167]

[0168] Where p θ Represents the reverse process. These are the potential variables for denoising generated at the last time step in the reverse process. Let N be the latent variable representing the confounding factors, N be a Gaussian distribution, t be the time step, and μ be the latent variable representing the confounding factors. θ Σθ is a function for parameterizing the neural network, where Σθ is the covariance matrix used to control the uncertainty of the Gaussian distribution.

[0169] Optimization: The loss function of the conditional diffusion model is used to optimize the model's performance during the denoising process; by minimizing the difference between the noise predicted by the model and the actual noise, the model can learn more accurately how to remove noise, thereby effectively restoring the real interaction relationship between users and items.

[0170]

[0171] in The noise ∈ predicted by the model is equal to the noise ∈ in the actual model. θ The error between them This indicates that the expected value is being calculated. Loss due to conditioned diffusion, For the latent variables of the final time step, The multiple rows of output for conditional diffusion are latent variables.

[0172] The conditional diffusion model combines latent variables generated by VGAE and latent variables generated by a confounding factor inferrer to model multi-behavioral interactions and confounding factors in multi-behavioral recommendation scenarios. It progressively simulates the evolution of user preferences through forward diffusion, and during back diffusion, it incorporates the inferred latent confounding factors and uses backdoor adjustment techniques from causal inference to recover the original latent variables from noise, removing interference caused by latent confounding factors.

[0173] Step S5: LightGCN is a simplified graph convolutional neural network that captures users' true preferences by removing complex nonlinear activation functions and relying solely on multi-layer iterative aggregation of user-item embeddings. Compared to traditional GCN models, LightGCN has lower computational complexity and can more effectively preserve collaborative filtering information between users and items. It samples latent variables generated from different behaviors during the backpropagation process of the conditional diffusion model. By weighted aggregation of multi-behavioral latent variables, LightGCN further generates the final user embedding and item embedding E through multi-layer graph convolution operations. u E i It is used for recommendation tasks in multi-behavior scenarios.

[0174] However, in multi-behavior recommendation scenarios, different behaviors have different weights for users. Therefore, for multi-behavior embeddings (such as browsing, favorites, adding to cart, purchasing, etc.), a multi-head attention mechanism is used to weight and aggregate them, enabling the model to better understand the impact of various user behaviors on their final preferences.

[0175]

[0176] Where B represents the number of multiple actions. For conditional diffusion, the output consists of multiple rows of latent variables. The numerical function ensures that the weights are positive, thus improving stability. The attention score of behavior b in attention head h determines the degree to which the behavior contributes to the final aggregated representation. This involves weighted summation of latent variables for different behaviors using attention weights. `concat` is the concatenation operation, and `z` represents the final concatenated multi-behavior fusion representation. LightGCN iteratively aggregates the embedded representations of users and items, capturing higher-order collaborative filtering signals in the interaction graph layer by layer, thereby more accurately characterizing the relationship between user preferences and item features.

[0177]

[0178] Where l is the number of graph convolutional layers, z i (l) z is a latent variable for the project when the number of graph convolutional layers is l. u (l+1) Let N be the latent variable for the user when the number of graph convolutional layers is l+1. u N i A collection of users and projects The normalization factor is used to balance the influence of node degree. The user and item embeddings generated at each layer in Lightgcn are averaged and aggregated to obtain the latent variables for the final user and item:

[0179]

[0180] Where L is the total number of graph convolutional layers, E u E i For the final embedded representation of users and items used to calculate user preferences, Let be the latent variables for users and items when the number of graph convolutional layers is l. Ultimately, user preference prediction for items is achieved by calculating the inner product of the user embedding and the item embedding:

[0181]

[0182] in E represents the probability of interaction between user u and item i under behavior b. u E i This is an embedded representation of users and items ultimately used to calculate user preferences.

[0183] Step S6: Bayesian Personalized Ranking (BPR) is an optimization method specifically designed for implicit feedback recommendation tasks. BPR learns model parameters by maximizing the preference difference between interacted and uninteracted items, thereby effectively improving the accuracy and personalization of recommendations.

[0184]

[0185] Where σ is the sigmoid function, and λ‖θ‖ 2 This is the regularization term, representing the predicted rating of user u for positive sample items. A joint learning strategy is used to optimize DCMB, and the training process is shown in Algorithm 1:

[0186]

[0187] in For the loss of a multi-behavior recommendation model, For the loss of the variational autoencoder, It is a loss due to conditioned diffusion. λ is the loss inferred from confounding factors, and λ is a hyperparameter.

[0188]

[0189]

[0190] Step S7: The evaluation module is used to assess the performance of the trained model using predetermined evaluation metrics. Specifically, the multi-behavior recommendation model mainly predicts the probability of user interaction with each item on the target behavior (purchase), and ranks all potential recommendation items based on the interaction probability, selecting the top N items with the highest probability to form a sorted recommendation list, i.e., the TOP-N recommendation strategy. This invention uses the hit rate (HR) and normalized discounted cumulative gain (NDCG) from the TOP-N recommendation strategy as evaluation metrics.

[0191] Normalized depreciation cumulative gain measures the quality of the recommendation list. It considers the relevance of items in the list and their position in the list, giving higher weights to items that rank higher and are more relevant. The normalized depreciation cumulative gain is used as the evaluation metric, and the expression is as follows:

[0192]

[0193] DCG considers the positional relationships between rankings, while IDCG represents the optimal ranking result for the current ranking. A higher NDCG value indicates better recommendation system performance.

[0194] Hit Rate (HR) measures the accuracy of recommendations. Specifically, it measures the percentage of recommendations that contain at least one item that a user is actually interested in. The expression for Hit Rate as a metric is as follows:

[0195]

[0196] NumbersofHits@N represents the number of users whose items from the test set appear in the Top-N recommendation list, and TsetSets represents the total number of users in the test set.

[0197] Experimental Analysis

[0198] To evaluate the effectiveness of the multi-behavior recommendation model proposed in this invention, a series of experiments were designed to address the following research questions:

[0199] Experimental data

[0200] The effectiveness of the present invention model was evaluated on two publicly available multi-behavioral datasets, Taobao and Tmall. Table 1 shows the data distribution of the three datasets.

[0201] Table 1 Experimental Dataset

[0202]

[0203] Taobao: This dataset was collected from Taobao, one of China's largest e-commerce platforms, and includes 47,894 users, 39,493 items, and 1,952,931 interactions.

[0204] Tmall: This dataset was collected from Tmall, one of China's largest e-commerce platforms, and includes 17,345 users, 35,920 items, and 1,451,219 interactions.

[0205] Baseline model

[0206] For a more comprehensive evaluation, this invention was compared with recommendation models based on graph neural networks, multi-behavior recommendation models, diffusion-based recommendation models, and causal learning-based recommendation models.

[0207] Recommendation models based on graph neural networks

[0208] ·NGCF: By propagating embeddings on the user-item graph and injecting cooperative signals during embedding propagation, this method enhances the ability to capture complex user-item relationships and simulates higher-order connectivity.

[0209] LightGCN: Removes the feature transformations and non-linear activation functions commonly used in traditional graph convolutional networks.

[0210] Multi-behavior recommendation

[0211] • EHCF: This method uses a novel non-negative sampling transfer learning approach to perform collaborative filtering on multi-behavioral user interaction data, thereby capturing fine-grained relationships between users and items.

[0212] ·GHCF: Based on multi-behavior interaction, behavior weights are introduced into graph convolution for iterative aggregation.

[0213] • MBGCN: Introduces a heterogeneous graph that integrates multiple behaviors, enabling user embeddings to take into account the weights of specific behaviors, thereby achieving behavior-based embedding propagation.

[0214] • MBGMN: Employs a meta-learning framework for multi-behavioral relation learning, promoting embedding propagation in graph neural networks.

[0215] Diffusion-based recommendation model

[0216] • DiffuRec: The first attempt to apply a diffusion model to sequential recommendation for item representation construction and uncertainty injection.

[0217] • DiffuASR: It uses a diffusion model to generate pseudo-sequences in sequential recommendations, filling the gap between discrete-valued item identifiers and the generated real-valued matrices.

[0218] Recommendation models based on causal learning

[0219] • PDA: Removes confounding popularity bias during training and adjusts the popularity bias in the recommendation score as needed during inference.

[0220] • DecRs: Eliminate the confounding effects of imbalanced item distribution on user representation and predicted scores through causal modeling and backdoor adjustment.

[0221] CausalD: By adjusting the front door, we estimate the causal impact of a user’s historical behavior on their next action to address the problem of unobserved confounding factors.

[0222] Performance Comparison

[0223] The table below shows the experimental performance of the proposed invention and the baseline on two datasets. The proposed invention categorizes the baseline models into graph neural network (GNN)-based recommendation models, multi-behavior recommendation models, diffusion-based recommendation models, and causal learning-based recommendation models. The length N of the truncated recommendation list used in the evaluation metrics is 10.

[0224] Table 2 compares the experimental performance of this invention on the BeiBei, Tmall, and Taobao datasets.

[0225]

[0226] Table 2 shows that the model of this invention significantly outperforms the baseline models on both datasets. Specifically:

[0227] The performance gap between CVID and GNN-based models (NGCF, LightGCN): GNN models typically rely on a single user-item interaction graph for embedding propagation, limiting their ability to capture complex dependencies in multi-behavioral interactions. In contrast, CVID introduces a Variational Graph Autoencoder (VGAE) to model multi-behavioral interaction data, generating latent variables for various user behaviors. This not only effectively alleviates the data sparsity problem but also explicitly models the potential uncertainty in user behavior, more accurately capturing the heterogeneity between different behaviors. Therefore, CVID outperforms traditional GNN models in personalized recommendation accuracy.

[0228] Performance gap between CVID and multi-behavior recommendation models (EHCF, GHCF, MBGCN, MBGMN): CVID outperforms existing multi-behavior recommendation models in handling potential confounding factors. While multi-behavior models like EHCF and GHCF can capture dependencies between different user behaviors to some extent, they fail to effectively address the interference of potential confounding factors. CVID applies variational inference through its confounding factor inference module to generate potential confounding factors (such as contextual influences and social relationships) in multi-behavior interactions, enabling the model to effectively infer and control the interference of these confounding factors. Furthermore, CVID uses a conditional diffusion module to intervene causally during backdiffusion, blocking the interference paths of confounding factors. This allows CVID to eliminate the influence of potential confounding factors in complex multi-behavior interaction scenarios, significantly improving system performance.

[0229] The performance gap between CVID and diffusion-based recommendation methods (DiffuRec, DiffuASR): While diffusion-based recommendation methods like DiffuRec and DiffuASR improve recommendation performance by generating representations of users' historical interactions, they do not specifically address confounding factors in multi-behavioral scenarios. CVID, on the other hand, combines conditional diffusion models and causal inference, simulating the dynamic evolution of user preferences during forward diffusion and introducing inferred confounding factors for guidance during backward denoising. This design effectively decouples user behavior from potential confounding factors, enabling the model to better capture true user preferences, thereby improving the accuracy and robustness of recommendations.

[0230] Performance Gap Between CVID and Causal Inference-Based Recommendation Models (PDA, CausalD, DecRs): While causal recommendation models such as PDA, CausalD, and DecRs address explicit confounding factors through causal intervention, especially CausalD which handles unobserved confounding factors through front-door adjustment, their adaptability to complex multi-behavioral scenarios is limited. CVID, leveraging its confounding factor inference and conditional diffusion modules, effectively infers and controls potential confounding factors across multiple user behaviors, ensuring that the causal relationship between user behavior and item interactions is unaffected by confounding factors. Furthermore, CVID blocks the causal path between user behavior and confounding factors through causal intervention in backdiffusion. Compared to PDA, CausalD, and DecRs, CVID demonstrates stronger recommendation accuracy and robustness in multi-behavioral scenarios, especially in complex real-world environments.

[0231] System module ablation experiment

[0232] To investigate the different contributions of the design modules in this invention, ablation studies were conducted on two datasets, resulting in the following three variations:

[0233] • w / o VGAE: Replace the variational graph autoencoder with GCN.

[0234] • w / o CI: Remove confounding factor inference module.

[0235] • w / o CD: Removes the conditional diffusion module.

[0236] Figure 4 The results of the ablation experiment show that:

[0237] (1) Comparing CVID with the w / o VGAE model reveals that VGAE more effectively captures the potential relationships between user behaviors through variational inference, solving the problems of behavioral heterogeneity and data sparsity. Removing VGAE significantly reduces the model's ability to capture personalized preferences, highlighting the important role of this module in improving recommendation accuracy.

[0238] (2) Comparing CVID with the w / o CI model highlights the importance of the confounding factor inference module in controlling potential confounding factors in multi-behavioral interactions. Without this module, the model cannot mitigate the interference of confounding factors, leading to a significant decrease in accuracy. The confounding factor inference module enables CVID to handle complex user behaviors and significantly improves model performance.

[0239] (3) A comparison of CVID with the w / o CD model demonstrates the crucial role of the conditional diffusion module in enhancing model robustness. This module simulates the dynamic evolution of user preferences and applies causal interventions to block confounding factors. Without this module, the model struggles to cope with changes in user preferences and remove confounding factors, thus weakening its accuracy and robustness.

[0240] Multi-behavioral data ablation experiment

[0241] To verify the different contributions of various user behaviors to the recommender system, we conducted a multi-behavior ablation study on the Tmall dataset with the most multi-behavior interactions. The results are as follows: Figure 5 As shown, we gradually removed browsing, favorites, adding to cart, and purchasing behaviors from the model and analyzed the specific impact of these changes on the recommendation performance.

[0242] The results show that deleting purchases significantly reduces model performance, highlighting its crucial role in predicting user preferences. Purchase behavior directly reflects the user's final decision-making process and is key to capturing personalized preferences. Browsing behavior also has a significant impact, although its weight is less than that of purchase behavior in real-world scenarios, it provides rich interaction data reflecting user preferences. In contrast, the impact of adding items to favorites and carts is relatively small because while these behaviors indicate purchase intention to some extent, they do not necessarily translate into actual purchases, hence their lower weight. In conclusion, compared to single-behavior recommendation systems, multi-behavior recommendation systems enhance the understanding of user preferences and improve model effectiveness by analyzing diverse user interactions from multiple perspectives.

[0243] Study on the effect of confounding factors in alleviating the symptoms

[0244] To explore the effectiveness of CVID in mitigating confounding factors, we conducted an in-depth analysis on the Tmall dataset. First, by calculating the number of multi-behavioral interactions for each item, we labeled items with high interaction frequency as high-exposure items and constructed the high-exposure subset Tmall-C. Next, we conducted comparative experiments on the Tmall-C dataset, focusing on evaluating their recommendation accuracy when handling popular items and their ability to mitigate confounding factors.

[0245] Experimental results are as follows Figure 6 As shown, CVID's performance on the Tmall-C dataset slightly decreases compared to the original dataset, indicating that high-exposure items introduce some confounding noise. This decrease may be due to the popularity bias of high-exposure items, which shifts user interactions, making it more difficult for the model to capture true preferences. However, despite the performance drop, CVID still outperforms PDA, demonstrating its robustness and accuracy in dealing with complex confounding factors. CVID's confounding factor inference module, combined with a conditional diffusion model, effectively decouples user behavior from confounding factors, thus maintaining high performance. By blocking the backdoor path I←C→Y, CVID ensures a more accurate causal relationship between user behavior and item interactions, making it more resilient to the influence of item popularity.

[0246] Hyperparameter experimental analysis

[0247] We further analyzed the impact of different hyperparameters on the performance of the CVID model. Specific factors analyzed included the number of attention heads in the attention mechanism, the steps and noise scale in the conditional diffusion process, and the penalty term in the optimization. All experiments were conducted with a fixed learning rate and batch size to quantitatively analyze the hit rate (HR).

[0248] Attention hyperparameter experimental analysis

[0249] Figure 7 This study illustrates the impact of the number of attention heads on model performance. The results show that setting the number of attention heads to 8 achieves the optimal balance between capturing diverse behavioral patterns and maintaining computational efficiency. Using 8 attention heads, the model effectively captures complex user-item interactions, significantly improving HR and NDCG on the Taobao and Tmall datasets. While increasing the number of attention heads to 16 or 32 yields some benefits, it also leads to a significant increase in computational cost without a substantial performance improvement. Therefore, using 8 attention heads achieves the optimal balance between accuracy and efficiency.

[0250] Experimental analysis of conditional diffusion hyperparameters

[0251] We examined the impact of diffusion step size and noise scale in the conditional diffusion module on model performance, such as Figure 8 As shown, for the Taobao dataset, a diffusion step size of 80–100 yields the best results, while for the Tmall dataset, a diffusion step size of 50–80 yields the best results. A moderate diffusion step size can effectively simulate the dynamic evolution of user preferences, enabling the model to capture true preferences in multi-behavioral scenarios. Initial performance improves with increasing noise scale, demonstrating the effectiveness of the forward noise process; however, excessively high noise scales hinder the model's ability to recover user preferences. The optimal noise scale is 0.2, ensuring that the model accurately captures true causal effects while mitigating confounding factors.

[0252] Optimize the hyperparameters of the penalty term.

[0253] We examined the impact of the penalty term coefficient in optimization on model performance, such as Figure 9 As shown. For the Tmall dataset, λ1(VGAE) takes values ​​in the range of 1e. -3 -1e -4 The optimal performance is achieved when the conditional diffusion occurs between 1 and 2, balancing the dependencies between multiple behaviors and avoiding overfitting. Similarly, λ² (conditional diffusion) takes values ​​between 1 and 2. -4 When the time interval is between these parameters, noise can be effectively controlled, enhancing the simulation of user preference evolution. Finally, λ3 (inference of confounding factors) takes a value in the range of 1e. -4 The optimal performance is achieved, ensuring accurate decoupling of user behavior from confounding factors and avoiding the introduction of excessive noise. In the experiment, we analyzed the influence of hyperparameters λ1, λ2, and λ3. We observed that when λ1 = 1e... -2 When λ² = 1e, the model performs optimally. This value effectively prevents overfitting and allows the model to learn more stable latent representations. λ² is responsible for mitigating the effects of latent confounding factors. Our experiments show that when λ² = 1e -3When λ2 is too large or too small, the instability of the diffusion process leads to a decrease in model performance. When λ3 = 1e... -4 When λ3 is at its optimal value, the model performs best. A larger value for λ3 leads to a more dominant confounding factor inference module, causing the model to focus excessively on modeling confounding factors while neglecting to capture real changes in user preferences, negatively impacting performance. Therefore, we fixed λ3 at this optimal value and further analyzed λ1 and λ2. The results confirmed the applicability of these hyperparameters: when λ1 = 1e -2 ,λ2=1e -3 In this way, CVID not only effectively avoids overfitting, but also significantly reduces the impact of potential confounding factors, achieving optimal model performance.

[0254] Summarize

[0255] This invention proposes a causal variational inference model (CVID) for deconfounding multi-behavior recommendation, aiming to address the problem of confounding factors in multi-behavior recommendation scenarios. CVID combines causal inference with a diffusion model and introduces a confounding factor inference mechanism and a conditional diffusion module to mitigate the impact of confounding factors on recommendation accuracy. Specifically, the conditional diffusion model combines latent variables generated by a variational graph autoencoder (VGAE) with inferred confounding factors. During forward diffusion, noise is gradually added to simulate the dynamic evolution of user preferences; during backward denoising, the inferred confounding factors guide the denoising process, effectively blocking causal paths and ensuring that the model learns the true causal relationship between user behavior and item interaction. Furthermore, the multi-head attention mechanism in LightGCN performs weighted aggregation of multi-behavior data, improving the model's ability to capture the contribution of different behaviors to user preferences. Extensive experimental results show that CVID consistently outperforms various baseline models on multiple public multi-behavior datasets, demonstrating its robustness and superior recommendation accuracy.

[0256] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A multi-behavior recommendation method based on causal variational inference to remove confounding factors, characterized in that, Includes the following steps: Step S1: Construct a multi-behavior recommendation model by importing multi-behavior type interaction data of users and projects and converting it into a graph structure. Step S2: Use variational graph autoencoder to generate latent variables for multiple behaviors to capture the complex interaction relationships and heterogeneity of behaviors between users and items, and explicitly model the potential uncertainties in the interaction. Step S3: Generate potential confounding factors through the confounding factor inference module to lay the foundation for the subsequent conditional diffusion process; Step S4: Based on the conditional diffusion model, noise is gradually introduced during the forward diffusion process to simulate the dynamic evolution of user preferences in different contexts. During the backward diffusion process, the inferred potential confounding factors are combined, and the backdoor adjustment technique in causal inference is used to recover the original potential variables from the noise and remove the interference caused by potential confounding factors. Step S5: Aggregate multi-behavior embeddings using a lightweight graph convolutional network, leverage multi-head attention mechanism to weigh the contribution of different behaviors to user preferences, and generate the final user and item embedding representations through multi-layer convolution. Step S6: Train and optimize the multi-behavior recommendation model. Adjust the loss functions of VGAE, confounding factor inference module and conditional diffusion module through joint optimization strategy to ensure the improvement of model performance. Step S7: The evaluation module is used to evaluate the performance of the trained model using predetermined evaluation metrics. Multi-behavior recommendation models predict the probability of user interaction with each item in a target behavior, and then rank all recommended items based on the interaction probability, selecting the top N items with the highest probability to form a sorted recommendation list.

2. The method according to claim 1, characterized in that, Step S1 includes the following sub-steps: S1-1. Define the user set U = {u1,...,u...} m ,...,u M } Item set I = {i1,...,i n ,...,i N }, where M and N are the number of users and items, indexed by m and n; S1-2. Define a set of multiple behaviors B = {v, f, c, p} and index it by behavior type b, where v, f, c, and p represent different behaviors; at the same time, consider the purchase behavior to be predicted as the target behavior. Other behaviors are considered auxiliary behaviors b′; S1-3, Define the multi-behavior user-item interaction graph as G. b =(V,E) b ), where V represents the set of nodes including users and projects, E b This represents an edge set that includes multiple similar user behaviors; S1-4. Define a tensor This reflects the user's multi-type interaction behavior, where M, N, and B represent the number of users, items, and behaviors, respectively. If user u m ∈U and item i n If ∈I interacts under behavior type b∈B, then the multi-behavior type interaction tensor... It is set to 1.

3. The method according to claim 2, characterized in that, Step S2 includes the following sub-steps: S2-1, In the four behavioral interaction diagrams (G v G f G c G p The variational graph autoencoder is used to generate latent variables with multiple behaviors to capture the complex interaction relationships and heterogeneity of behaviors between users and items, and to explicitly model the potential uncertainties in the interaction; the variational graph autoencoder calculates the mean μ of the nodes through GCN. b and standard deviation σ b , used to represent the potential distribution of each node: m b =GCN(A b ,E) s b =GCN(A b ,E) Where GCN represents graph convolution operation, b represents multi-behavior interaction type, E represents the initial embedding of user and item generated through the initial embedding layer, and A... b The adjacency matrix corresponds to the multi-row interaction graph; by performing a convolution operation on each node, the encoder generates the mean μ and standard deviation μ of each node, thereby representing the distribution of each node in the latent space; S2-2. To generate the latent variable z, the reparameterization technique is used to represent the latent variable z as: z b =μ b +s b ⊙∈ Where ⊙ represents element-wise multiplication, and the standard deviation μ is multiplied... b Multiplying the noise by each element helps the model learn diversity while ensuring the smoothness of the latent variables and avoiding overfitting. S2-3. Based on the latent variable z generated by the encoder, the decoder reconstructs the user-item interaction matrix by utilizing the node relationships in the graph structure. The reconstruction process is as follows: in The reconstructed multi-row adjacency matrix is ​​given by φ, where φ is the activation function Sigmoid, and z... b ·z b T This indicates the calculation of similarity among latent variables; S2-4. The variational graph autoencoder optimizes the model using the lower bound of evidence. The optimization process is as follows: Where D KL KL divergence is used to measure the posterior distribution q of the latent variable. φ The difference between (z|X) and the prior distribution p(z), where σ is the standard deviation and μ is the mean; reconstruction loss. Adjacency matrix for measuring reconstruction With the true adjacency matrix A b The difference between them is that u and i represent users and projects, and n represents the dimension of the latent variable; S2-5. By combining the KL divergence and reconstruction loss, the loss of the variational graph autoencoder is calculated as follows: in For the loss of the variational autoencoder, The reconstruction loss is β, and the equilibrium KL divergence is β. Hyperparameters.

4. The method according to claim 3, characterized in that, Step S3 includes the following sub-steps: S3-1. Using an encoder on the latent variable z b Encode the data to generate the mean of the confounding factors. and standard deviation The expression is as follows: in and Let represent the mean and standard deviation of the confounding factors, and φ be the activation function. and For encoder parameters, z b The multi-row output of the variational graph autoencoder represents latent variables. S3-2, By analyzing the multi-behavior latent variable z output by the variational graph autoencoder. b Using reparameterization techniques, initial potential confounding factors are generated to model latent factors such as user context and social influence, as shown in the following expression: in ∈ represents the initial potential confounding factor, ∈ represents random noise, and ⊙ represents the element-wise multiplication operation, which multiplies the standard deviation with the noise element-wise by click, thereby capturing the confounding factor; S3-3. Using a decoder, the latent variables are mapped to a low-dimensional space. After processing by the parameter matrix W and a nonlinear activation function, the final confounding latent variables are generated, as shown in the following expression: in For the final potential confounding factors, Initial potential confounding factors, where φ is the activation function Sigmoid; S3-4. The confounding factor inference engine optimizes the model through the lower bound of evidence. The optimization process is as follows: Where D KL KL divergence is used to measure the posterior distribution of latent variables. With prior distribution The difference between them, where σ is the standard deviation and μ is the mean; reconstruction loss Measuring the final potential confounding factors Initial potential confounding factors The difference between them, where i represents the item, The loss of the confounding factor inferrer is β, where β is the equilibrium KL divergence. Hyperparameters.

5. The method according to claim 4, characterized in that, Step S4 includes the following sub-steps: S4-1. In the forward propagation process of the conditional diffusion model, the model is progressively guided towards the latent variable z generated by the variational graph autoencoder. b Add noise to generate noisy latent variables at different time steps t. The expression is as follows: Where t ∈{1,....,T} α is the time step, b is the multi-row interaction type, N is a Gaussian distribution, I is the identity matrix, and α is the time step. t It is a scheduling parameter related to time step t. This represents the cumulative noise figure from time step 0 to time step t. For noisy latent variables, These are potential variables at time step 0; S4-2. By recursively adding noise, noisy latent variables are gradually generated, as shown in the following expression: S4-3. In the reverse denoising process, the conditional diffusion model incorporates confounding factors. Using backdoor adjustment techniques in causal inference to recover the original latent variables from noise. To ensure that confounding factors are eliminated during the denoising process to accurately capture users' true preferences, the expression is as follows: Where p θ Represents the reverse process. These are the potential variables for denoising generated at the last time step in the reverse process. Let N be the latent variable representing the confounding factors, N be a Gaussian distribution, t be the time step, and μ be the latent variable representing the confounding factors. θ Σθ is a function for parameterizing the neural network, where Σθ is the covariance matrix used to control the uncertainty of the Gaussian distribution; S4-4. The loss function of the conditional diffusion model is used to optimize the model's performance in the denoising process. By minimizing the difference between the noise predicted by the model and the actual noise, the model can learn more accurately how to remove noise, thereby effectively restoring the real interaction relationship between users and items. The expression is as follows: in The noise ∈ predicted by the model is equal to the noise ∈ in the actual model. θ The error between them This indicates that the expected value is being calculated. Loss due to conditioned diffusion, For the latent variables of the final time step, The multiple rows of output for conditional diffusion are latent variables.

6. The method according to claim 5, characterized in that, Step S5 includes the following sub-steps: S5-1. In multi-behavior recommendation scenarios, different behaviors have different weights for users. A multi-head attention mechanism is used to weight and aggregate these behaviors, enabling the model to better understand the impact of various user behaviors on their final preferences. The expression is as follows: Where B represents the number of multiple actions. For conditional diffusion, the multiple rows of output are latent variables. The positive function ensures that the weights are positive, thus improving stability; The attention score of behavior b in attention head h determines the degree of contribution of that behavior to the final aggregated representation; This involves weighted summation of latent variables for different behaviors using attention weights; concat is the concatenation operation, and z is the final concatenated multi-behavior fusion representation; S5-2, Lightgcn aggregates the embedded representations of users and items through multiple iterations, capturing high-order collaborative filtering signals in the interaction graph layer by layer, thereby more accurately characterizing the relationship between user preferences and item features. The expression is as follows: Where l is the number of graph convolutional layers, z i (l) z is a latent variable for the project when the number of graph convolutional layers is l. u (l+1) Let N be the latent variable for the user when the number of graph convolutional layers is l+1. u N i A collection of users and projects The normalization factor is used to balance the influence of node degree; S5-3. Aggregate the user and item embeddings generated at each layer in Lightgcn to obtain the latent variables of the final user and item, as shown in the following expression: Where L is the total number of graph convolutional layers, E u E i For the final embedded representation of users and items used to calculate user preferences, z u l ,z i l These are the latent variables for users and items when the number of graph convolutional layers is l; S5-4. User preference prediction for items is achieved by calculating the inner product of user embedding and item embedding: in E represents the probability of interaction between user u and item i under behavior b. u E i This is an embedded representation of users and items ultimately used to calculate user preferences.

7. The method according to claim 6, characterized in that, Step S6 includes the following sub-steps: S6-1. The Bayesian Personalized Ranking (BPR) method is used to optimize the multi-behavior recommendation model. By maximizing the preference difference between interacted and uninteracted items, the model parameters are learned, thereby effectively improving the accuracy and personalization of the recommendations. in Calculate user u against positive sample i + The relative preference between the negative sample i and the negative sample i-, λ‖θ‖ 2 It is a regularization term; S6-2, The loss function expression for multi-behavior recommendation is as follows: in For the loss of a multi-behavior recommendation model, For the loss of the variational autoencoder, It is a loss due to conditioned diffusion. λ is the loss inferred from confounding factors, and λ is a hyperparameter.

8. The method according to claim 7, characterized in that, Step S7 includes the following sub-steps: S7-1. Normalized depreciation cumulative gain measures the quality of the recommendation list, considering the relevance of items in the list and their position in the list. Higher weights are given to items that rank higher and have higher relevance. Normalized depreciation cumulative gain is used as the evaluation metric, and the expression is as follows: Where N represents the top N items in the recommendation list, DCG takes into account the positional relationship between sorts, and IDCG is the best sort result for the current sort; the higher the value of NDCG, the better the performance of the recommendation system. S7-2. Hit Rate (HR) measures the accuracy of recommendations; that is, what percentage of the recommended list contains at least one item that a user is actually interested in. The expression for Hit Rate as an evaluation metric is as follows: NumbersofHits@N represents the number of users whose items from the test set appear in the Top-N recommendation list, and TsetSets represents the total number of users in the test set.

Images