A multi-behavior-aware graph contrast recommendation method

Abstract

The application discloses a multi-behavior perception graph contrast recommendation method, comprising the following steps: (1) representation learning on an original view; (2) representation learning on a noise-free view; (3) multi-behavior perception modeling, containing latent behavior modeling, E-step behavior representation learning and M-step behavior contrast learning; and (4) multi-task learning. The application well models various behaviors of a user, thereby more effectively capturing the interaction relationship between the user and an item, and enabling the representation to be further improved. Meanwhile, the noise problem in the contrast learning is solved, and the representation learned by the recommendation method of the application is more effective.

Description

Technical Field

[0001] This invention belongs to the field of personalized recommendation, and in particular relates to a graph comparison recommendation method with multi-behavior awareness. Background Technology

[0002] In recent years, recommender systems (RS) have become an important tool in web applications, helping users quickly discover relevant content from vast amounts of online information. These systems provide personalized recommendations based on user interests, such as recommending products on social networking sites, short video platforms, and shopping apps. Collaborative filtering (CF) is one of the most commonly used methods in recommendation, leveraging the preferences of similar users to recommend new items to a given user.

[0003] Recent studies have demonstrated the significant success of Graph Contrast Learning (GCL) methods in collaborative filtering recommendations. The core idea of ​​contrastive learning (CL) is to minimize (maximize) the mutual information of positive (negative) pairs, thereby improving the quality of representations used for downstream tasks. By combining GNNs with CL, recommendation methods based on Graph Contrast Learning (GCL) have emerged, providing an effective solution to the data sparsity problem and better modeling user interests: for example, methods SGL and SimGCL augment graph structures by employing random insertion and deletion operations to generate different views for contrastive learning.

[0004] However, such random augmentation may lose important information, potentially exacerbating the sparsity problem of inactive users. Therefore, CGI selectively removes edges / nodes to construct a more practical contrastive view. Compared to SGL, the NCL method treats similar semantic prototypes and structural neighbors in the embedding space as positive views. HCCF captures local and global synergies through a cross-view contrastive learning architecture augmented by a hypergraph. SimGCL experimentally determined that data augmentation contributes minimally to SGL, thus suggesting adding noise at each embedding layer to generate positive instances. LightGCL utilizes Singular Value Decomposition (SVD) to generate contrastive views for global collaborative relationship modeling.

[0005] While these methods have achieved satisfactory performance, they typically represent user behavior as two values ​​(0 for no interaction, 1 for interaction), limiting their ability to capture the diversity of interactions such as "like," "dislike," or "comment." This hinders their ability to model complex relationships and complex user item interactions in real-world recommendation scenarios.

[0006] The main disadvantages of existing technologies are as follows: 1) First, these methods typically represent user behavior as 0 (no interaction) or 1 (interaction), without exploring the diversity of user-item interactions, which limits their ability to capture the various user interaction patterns that exist in the real world. In real-world recommendation scenarios, user-item interactions can involve various types of user behavior, such as "likes," "dislikes," and "comments."

[0007] 2) Secondly, existing GCL-based methods may introduce additional noisy interactions or discard important information during the data augmentation process, thus limiting the applicability and potential of contrastive learning. Summary of the Invention

[0008] This invention provides a multi-behavior-aware graph comparison recommendation method, which improves the recommendation effect by modeling multi-behavior interactions and generating noise-free comparison views.

[0009] A multi-behavior-aware graph comparison recommendation method includes: (1) Representation learning on the original view First, user and item representations are embedded. Then, an L-layer GNN is used to propagate and aggregate user and item representations on the original interaction matrix. Finally, the output representations of all GNN layers are summed to obtain the final user and item representations, and dot products are used to predict user representations. i and items j Interaction probability between Construct the main recommendation task loss based on the optimization objective; (2) Representation learning on noiseless views First, a noise-free method is designed to augment the original graph and obtain a noise-free contrast view. Then, an L-layer GNN is used to propagate and aggregate the node representations on the noise-free view. Finally, the GCL loss for users and items is constructed by comparing the original view and the noise-free view. (3) Multi-behavior perception modeling First, model the potential behavior: assuming it exists. K One latent behavioral variable Determine the interaction probability between users and items, then rewrite the objective function and optimize the objective by constructing a lower bound function; Then, E-step behavior representation learning is performed: the interaction probability is obtained by the dot product between the user and item representations learned from the original view. And based on the interaction probability Divide user-item interaction into K Each behavior type, and thus the interaction. and behavior The representation of; Finally, M-step behavior contrastive learning is performed: assuming the prior distribution of behaviors is uniform, given... of The conditional distribution follows an isotropic Gaussian distribution, thus yielding the loss function for Behavior Contrastive Learning (BCL). (4) Multi-task learning The model is trained by using a multi-task training method, and the main recommendation task, GCL task, and BCL task are jointly optimized. (5) Application In practical applications, user and item representations are obtained through model training. When recommending items to a user, the inner product of the user's representation with that of all items in the item pool is calculated to obtain the probability of interaction. The items corresponding to the top k items with the highest probability are selected and recommended to the user.

[0010] In step (1), the L-layer GNN is used to propagate and aggregate the representations of users and items on the original interaction matrix, specifically including: The l-th GNN layer is defined as: in, and The output of the l-th layer represents user i and item j; using and Go to initialization and ; It is the LeakyReLU activation function; It is a standardized interaction matrix.

[0011] Interaction probability Described as: The loss for constructing the main recommendation task is: in, and J represents the prediction score for a pair of positive and negative items for user i, J is the number of correct predictions, and M is the number of users.

[0012] The noise-free comparison view obtained in step (2) is as follows: in, and It is a learnable embedding of user i and item j; This represents the cosine similarity function.

[0013] The L-layer GNN is used to propagate and aggregate the node representations on the noiseless view, specifically as follows: in, and It is a representation of users and items learned from a noise-free view; , These represent user i and item j, respectively.

[0014] Construct the GCL loss for users and items, specifically as follows: in, , It is the representation of user i and item j learned from the original view; , It is the representation of user i and item j learned from a noise-free view; sim() is the similarity function, i.e., the dot product; Indicates the temperature coefficient; and These are the GCL losses for users and items, respectively.

[0015] In step (3), the specific process of modeling potential behaviors is as follows: Assume it exists K Latent behavioral variables To determine the probability of interaction between users and items, the objective function is rewritten as follows: By constructing a lower bound function and maximizing it, firstly, we have: Assume that the behavioral variable follows a The distribution of, among which and Then, based on Jensen's inequality, for the right-hand side of the above equation, we have: in, Indicates proportionality; when When the inequalities are equal, a lower bound is found.

[0016] Based on interaction probability Divide user-item interaction into K Each behavior type, and thus the interaction. and behavior The representation is as follows: according to User-item interaction Divided into K Individual behavior types: in, , Representing interaction The behavior type is obtained by acquiring the behavior type of each interaction in a batch, thus obtaining the behavior. The distribution within a batch is represented as follows: To obtain a representation of the behavior, the interaction is first defined. The representation is as follows: in, It is interaction The representation of; and These are the original view representations of user i and item j, respectively; based on By classifying behaviors The interaction representation is averaged to calculate the behavior. The representation of is defined as follows: in, Indicates behavior The expression .

[0017] The specific process of performing M-step behavioral contrastive learning is as follows: The behavior distribution function was estimated by performing E-step behavior representation learning. And received a representation of the behavior. Assuming the prior distribution of behavior is uniform, given... of The conditional distribution follows an isotropic Gaussian distribution, then... Rewritten as follows: in, and They are interaction and behavior The representation is to minimize the following loss function: Where sim() is a similarity function; the loss function for Behavior Contrast Learning (BCL) is optimized as follows: in It is an interactive representation from a noise-free view, and It is a representation of behavior from the original view.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1) Compared with existing methods, this invention solves the noise problem in contrastive learning, making the representation learned by this method more effective.

[0019] 2) Compared with existing methods, this invention models a variety of user behaviors very well (previous methods only distinguish between interaction and no interaction, while this invention can capture the type of interaction, such as liking, blocking, commenting, etc.), thereby more effectively capturing the interaction relationship between users / items and further improving the representation.

[0020] 3) Compared with existing methods, the present invention significantly improves the performance of collaborative filtering recommendation systems. Attached Figure Description

[0021] Figure 1 This is a framework diagram of a multi-behavior-aware graph comparison recommendation method of the present invention; Figure 2 This is a flowchart of a multi-behavior-aware graph comparison recommendation method according to the present invention. Detailed Implementation

[0022] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the embodiments described below are intended to facilitate the understanding of the present invention and do not constitute any limitation thereof.

[0023] This invention proposes a multi-behavior-aware graph contrastive recommendation method (MBGCR), which effectively models the diversity of user behavior within a generalized expectation-maximization (EM) framework. Specifically, a latent variable is introduced to represent different user behaviors, and the distribution function of the behavior variable is learned by capturing the interest levels between users and items (E-step). Then, using the learned behaviors, the representations of users and items are refined through contrastive behavior learning, thereby maximizing the consistency between user-item interactions and corresponding behavior variables (M-step). Furthermore, to generate a better informative view of the GCL, the weights of the original edges are considered without affecting nodes / edges, ensuring that the contrastive view is noise-free. Finally, a multi-task learning strategy is designed to optimize the model for effective representation learning.

[0024] MBGCR's model structure is as follows Figure 1 As shown. Initially, it performs graph contrastive learning (GCL) between representations obtained from the original view and the generated noise-free view. Then, E-steps and M-steps are performed iteratively to estimate behavioral variables. C Distribution function on Q ( c ), and the model parameters θ Optimization is performed. In the E-step, it estimates the distribution. Q( c ), and calculate the interaction probability. To obtain a representation of the behavior. In the M-step, it uses an estimation-based approach. Q ( c Behavioral contrastive learning (BCL) is used to optimize model parameters. θ In each iteration, Q ( c )and θ They will all be updated accordingly.

[0025] Problem Definition: In the recommendation scenario, this invention uses and Let M and N represent the sets of users and items, respectively, where M and N are the number of users and items, respectively. User-item interaction matrix. R M*N This represents the observed user-item interaction, where if the user With items If it has been interacted with, it is an element. ,on the contrary Then it is 0. For convenience, use (u i , v j Let be the interaction between user i and item j. Given the items that user i has interacted with, the goal of recommendation is to predict the items that user i may visit in the future, which can be formalized as: in It has parameters The mapping function, This indicates the item that user i is most likely to visit next. Let each item represent an item the user has interacted with. The optimization objective can then be expressed as finding the optimal parameters that maximize the log-likelihood function of a given user's interactions. 1) Representation learning on the original view In collaborative filtering, both users and items are associated with learnable embedding vectors. R M*d and R N*d They are related, where d is the embedding dimension. Given user i and item j, their embeddings are represented as follows: and To learn valuable contextual information from the original relationships, we use an L-layer GNN to propagate and aggregate user / item representations across the interaction matrix. Specifically, the l-th GNN layer is defined as follows: in and Let l represent the l-th level output representation of user i and item j. We use and Go to initialization and . It is the LeakyReLU activation function. This is a standardized interaction matrix. We sum the output representations of all GNN layers to obtain the final user (item) representation, and use the dot product to predict the interaction probability between user i and item j. The description is as follows: We put By applying this to formula (2), we obtain the specific optimization objective: This is equivalent to minimizing the loss of the following BPR: in and Let J represent the prediction score for a pair of positive and negative items for user i, and J be the number of correct predictions. The specific process is as follows: Figure 2 As shown in (a).

[0026] 2) Representation learning on noiseless views Existing GCL-based methods typically generate contrastive views by perturbing nodes or edges. However, this approach risks introducing harmful noise or discarding crucial information, hindering representation learning. We argue that the information in the original view is valuable and should not be discarded or altered. Therefore, we propose a noise-free method to augment the original graph and obtain an informative contrastive view. It is important to emphasize that our method does not perturb edges and nodes, but only involves augmenting the weights of the original edges, i.e., the observed interactions, ensuring that the generated view is noise-free and informative. To effectively augment edge information, we utilize trainable user and item embeddings to generate corresponding edge weights, which can be dynamically adjusted for better graph contrastive learning. Specifically, we construct a noise-free contrastive view. as follows: (7) in and It is a learnable embedding of user i and item j. Let represent the cosine similarity function. Using formula (7), a noise-free view can be obtained. This setting improves the generated comparison view. The reliability of the graph is high, and it can model and capture user-item relationships well. The propagation and aggregation of the graph on the generated learnable view are defined as follows: in and It is a representation of users and items learned from a noise-free view. ( Let be the representation of user i (item j). Then, by comparing the original view and the noisy tree view, a contrastive loss for the graph is designed, defined as follows: in, , ( , ) is the representation of user i and item j learned from the original view (noise-free view). sim() is the similarity function (dot product). This represents the temperature coefficient. and This refers to GCL loss for both users and items. The specific process is as follows: Figure 2 As shown in (b).

[0027] 3) Multi-behavior perception modeling To address multi-behavior modeling, this invention proposes a generalized expectation-maximization (EM) method, which iteratively optimizes the objective function and guarantees convergence. Its core idea is to optimize the objective function from model parameters... θ The initial estimation begins by estimating the missing behavioral variables in the E-step. C Iterative refinement is then carried out. Once it has... C The value of is updated by maximizing the equation. θ In the M-step, repeat this iterative process until the probability no longer increases. By using the EM framework, missing behavioral variables can be handled effectively. C The resulting incompleteness allows us to study the underlying behavior and model parameters. θ Make a reliable estimate.

[0028] I. Latent Behavior Modeling. To incorporate latent behaviors into user-item interaction modeling, it is assumed that... K Latent behavioral variables Determine the interaction probability between the user and the item, and then rewrite the objective function in equation (2) as follows: However, optimizing this objective remains challenging. To address this issue, this invention employs a novel approach by constructing a lower bound function and maximizing it. First, we have: In mathematics, it is assumed that behavioral variables follow a... The distribution of, among which and Then, based on Jensen's inequality, for the right-hand side of equation (11), we have: in Indicates "proportional". When When the inequalities are equal, the equations are equal. Now, a lower bound for equation (10) has been found. However, it cannot be directly optimized because... The unknown is the problem. Therefore, this invention uses the expectation-maximization algorithm to solve the unknown problem by alternately optimizing the model between behavior representation learning (E-step) and behavior contrastive learning (M-step).

[0029] II. Step E: Behavior representation learning. To estimate the behavior distribution... Based on equations (3) and (4), this invention obtains the interaction probability by the dot product between the user and item representations learned from the original view. . This can be viewed as the degree of interest of user i in item j. Then, the min-max normalization method is used to normalize all interaction probabilities to promote differentiation on the same scale, as defined below: Intuitively, different user behaviors correspond to different levels of interest in items. Therefore, this invention can be based on interaction probabilities. User interactions are categorized into different behavior types, or interest levels. Specifically, this invention considers different interest levels when assigning different behavior types. For example, "like" behavior corresponds to a higher interest level compared to "dislike" behavior. Then, based on... User-item interaction Divided into K Individual behavior types: in, Representing interaction The behavior type. Now, we have obtained the behavior type for each interaction in a batch. Of course, the behavior can be obtained. The distribution within a batch is represented as follows: (15) To obtain a representation of the behavior, the interaction is first defined. The representation is as follows: (16) in, It is interaction The expression . and These are the original view representations of user i and item j, respectively. Based on It can be achieved by classifying behaviors The interaction representation is averaged to calculate the behavior. The representation of is defined as follows: (17) in, Indicates behavior The expression .

[0030] III. M-step: Behavior contrastive learning. The behavior distribution function is estimated through the E-step. And received a representation of the behavior. To maximize the objective function in equation (12), it is also necessary to calculate... Assuming the prior distribution of behavior is uniform, given... of The conditional distribution follows an isotropic Gaussian distribution, and then we can... Rewritten as follows: in, and They are interaction and behavior The representation of . Based on equations (12), (15), and (18), maximizing the lower bound of equation (12) is equivalent to minimizing the following loss function: Where sim() is a similarity function (e.g., dot product). Equation (19) has a similar structure to Equation (9), which aims to maximize the mutual information between two view representations, while Equation (19) maximizes the mutual information between an interaction and its corresponding behavior type. Formally, given representations Z and H learned from the original view and the noise-free view, the corresponding interaction and behavior representations are first obtained by calculating Equations (16) and (17). Then the loss function of Behavior Contrastive Learning (BCL) is optimized as follows: in It is an interactive representation from a noise-free view, and It represents the behavior from the original view. The specific process is as follows: Figure 2 As shown in (c).

[0031] 4) Multi-task learning This invention employs a multi-task training method to train the model, and jointly optimizes the main recommended task-equation (6), GCL task-equation (9), and BCL task-equation (20). Specifically, the proposed model is trained simultaneously as follows: in, , and These are the weights for graph contrast loss, behavior contrast loss, and regularization loss, respectively. The specific process is as follows: Figure 2 As shown in (d).

[0032] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-behavior-aware graph comparison recommendation method, characterized in that, include: (1) Representation learning on the original view First, embed the representations of users and items; Then, an L-layer GNN is used to propagate and aggregate user and item representations on the original interaction matrix; finally, the output representations of all GNN layers are summed to obtain the final user and item representations, and dot products are used to predict users. i and items j Interaction probability between ; Construct the main recommendation task loss based on the optimization objective; (2) Representation learning on noiseless views First, we design a noise-free method to enhance the original image and obtain a noise-free comparison view; Then, an L-layer GNN is used to propagate and aggregate the node representations on the noiseless view; finally, the GCL loss for users and items is constructed by comparing the original view and the noiseless view; the resulting noiseless comparison view is as follows: in, and It is a learnable embedding of user i and item j; Represents the cosine similarity function; (3) Multi-behavior perception modeling First, model the potential behavior: assuming it exists. K One latent behavioral variable Determine the interaction probability between users and items, then rewrite the objective function and optimize the objective by constructing a lower bound function; Then, E-step behavior representation learning is performed: the interaction probability is obtained by the dot product between the user and item representations learned from the original view. And based on the interaction probability Divide user-item interaction into K Each behavior type, and thus the interaction. and behavior The representation of; Finally, M-step behavior contrastive learning is performed: assuming the prior distribution of behaviors is uniform, given... of The conditional distribution follows an isotropic Gaussian distribution, thus yielding the loss function for Behavior Contrastive Learning (BCL). (4) Multi-task learning The model is trained by using a multi-task training method, and the main recommendation task, GCL task, and BCL task are jointly optimized. (5) Application In practical applications, user and item representations are obtained through model training. When recommending items to a user, the inner product of the user's representation with that of all items in the item pool is calculated to obtain the probability of interaction. The items corresponding to the top k items with the highest probability are selected and recommended to the user.

2. The multi-behavior-aware graph comparison recommendation method according to claim 1, characterized in that, In step (1), the L-layer GNN is used to propagate and aggregate the representations of users and items on the original interaction matrix, specifically including: The l-th GNN layer is defined as: in, and This represents the output of the l-th GNN layer for user i and item j; using and Go to initialization and ; It is the LeakyReLU activation function; It is a standardized interaction matrix.

3. The multi-behavior-aware graph comparison recommendation method according to claim 1, characterized in that, The L-layer GNN is used to propagate and aggregate the node representations on the noiseless view, specifically as follows: in, and It is a representation of users and items learned from a noise-free view; , These represent user i and item j, respectively.

4. The multi-behavior-aware graph comparison recommendation method according to claim 3, characterized in that, Construct the GCL loss for users and items, specifically as follows: in, , It is the representation of user i and item j learned from the original view; , It is the representation of user i and item j learned from a noise-free view; sim() is the similarity function, i.e., the dot product; Indicates the temperature coefficient; and These are the GCL losses for users and items, respectively; M is the number of users, and N is the number of items.

5. The multi-behavior-aware graph comparison recommendation method according to claim 1, characterized in that, In step (3), the specific process of modeling potential behaviors is as follows: Assume it exists K Latent behavioral variables Determine the probability of interaction between users and items, and rewrite the objective function as follows: By constructing a lower bound function and maximizing it, firstly, we have: Assume that the behavioral variable follows a The distribution of, among which and Then, based on Jensen's inequality, for the right-hand side of the above equation, we have: in, Indicates proportionality; when When the inequalities are equal, a lower bound is found.

6. The multi-behavior-aware graph comparison recommendation method according to claim 5, characterized in that, Based on interaction probability Divide user-item interaction into K Each behavior type, and thus the interaction. and behavior The representation is as follows: according to User-item interaction Divided into K Individual behavior types: in, , Representing interaction The behavior type is obtained by acquiring the behavior type of each interaction in a batch, thus obtaining the behavior. The distribution within a batch is represented as follows: To obtain a representation of the behavior, the interaction is first defined. The representation is as follows: in, It is interaction The representation of; and These are the original view representations of user i and item j, respectively; based on By classifying behaviors The interaction representation is averaged to calculate the behavior. The representation of is defined as follows: in, Indicates behavior The expression .

7. The multi-behavior-aware graph comparison recommendation method according to claim 6, characterized in that, The specific process of performing M-step behavioral contrastive learning is as follows: The behavior distribution function was estimated by performing E-step behavior representation learning. And received a representation of the behavior. Assuming the prior distribution of behavior is uniform, given... of The conditional distribution follows an isotropic Gaussian distribution, then... Rewritten as follows: in, and They are interaction and behavior The representation is to minimize the following loss function: Where sim() is a similarity function; the loss function for Behavior Contrast Learning (BCL) is optimized as follows: in It is an interactive representation from a noise-free view, and It is a representation of behavior from the original view.

Images