A collaborative filtering information recommendation system based on graph-based hybrid message passing mechanism

Abstract

The present invention provides a collaborative filtering information recommendation system based on a hybrid message passing mechanism of a graph. The system of the present invention decouples the user-project information bipartite heterogeneous graph into two homogeneous graphs of user and project information through an explicit message passing mechanism of decoupling heterogeneous objects, and combines it with a complementary implicit message passing method to form a hybrid message passing mechanism for the user-project information bipartite heterogeneous graph, thereby improving the single graph convolutional message passing mechanism of the undecoupled user-project information representation. The mechanism is used to optimize the learning process of the low-dimensional vector representation of each node in the training process of the graph neural network collaborative filtering algorithm, thereby improving the recall rate (recall), normalized discounted cumulative gain (NDCG) and other indicators of the information recommendation results.

Description

Technical Field

[0001] The present invention relates to a collaborative filtering information recommendation system based on a graph-based hybrid message passing mechanism, and belongs to the field of deep learning and recommendation systems. Background Art

[0002] Recommendation system is one of the important Internet information services for dealing with information overload and performing personalized data screening. A complete recommendation system is usually divided into recall, rough sorting, fine sorting, and re-ranking. The recall stage usually involves initially screening out hundreds or thousands of candidate items from tens of thousands of recommendation candidates for the next more fine-grained recommendation sorting stage or directly screening the candidates with the highest matching scores in the recall as information for recommendation. Many models have been proposed to deal with the personalized recommendation recall problem, among which collaborative filtering is one of the most classic and effective methods. Collaborative filtering is a matching process, specifically, the process of measuring the user's interest in project information. In recent years, graph neural networks have become the latest recommendation system method. Many scholars have used graph neural networks to improve traditional collaborative filtering methods, modeling the multi-order relationship between users and project information through graph convolution, thereby effectively improving the matching performance in the recall process of the recommendation system.

[0003] At present, graph convolutional neural networks are mainly divided into two categories: frequency domain graph convolutional models and spatial domain graph convolutional models. In the field of recommendation systems, the frequency domain graph convolutional model converts graph information into graph signal processing through graph Fourier transform, and directly designs a series of filters of different frequencies to process the frequency domain signals on the bipartite graph, and finally obtains the node representation, such as GF-CF and other models; while the spatial domain graph convolutional model directly uses the regularized bipartite graph adjacency matrix information to iteratively propagate the neighbor node information of each node to the corresponding node, and then aggregates the information to obtain the final node representation, such as NGCF, LightGCN and other models. The core of the graph convolutional neural network lies in its message passing mechanism for aggregating neighbor information, which forms the final node representation by continuously aggregating information from the neighborhood.

[0004] The data faced by recommendation systems is often sparse. The current implementation of collaborative filtering recall algorithms based on graph neural network modeling usually constructs a bipartite graph based on the user-item information items in the data set, and captures multi-layer node information through iterative graph convolution, which alleviates the problem of sparse node data to a certain extent. The core of graph convolutional neural networks lies in their message passing mechanism for aggregating neighbor information. For each node, when it is a source node, the range of the graph convolutional message passing mechanism to propagate aggregated neighbor nodes is limited. There is a maximum hop range that is artificially limited as a hyperparameter of the neural network model. Through continuous iterative propagation, the information from the neighborhood of the source node is aggregated. When the number of iterations of the graph convolution operator is n, the information of the neighboring nodes separated by n hops from the source node will be captured by the model and participate in the formation of the final source node representation. In current applications, although graph convolutional neural networks that explicitly limit the number of graph convolution iterations have achieved good results to a certain extent, they are limited by the number of model message passing. The message passing range in the learning process of each node representation is explicitly limited, which limits the model's ability to learn and express the entire graph information to a certain extent. At the same time, the graph convolutional neural network model has an over-smoothing problem, that is, when the graph convolution operator iterates to a certain number of times, the final representation of each node will tend to converge to a fixed vector representation, and the node representations obtained after iteration n and iteration n+1 are the same. In view of the above phenomena and problems, it can be seen that the scheme for selecting the number of explicit message passing in the current graph convolutional neural network message passing mechanism has some oversimplified problems and cannot fully mine the graph information.

[0005] At the same time, the data graph used in the collaborative filtering setting of graph neural networks is a heterogeneous graph. There are two types of heterogeneous objects in the graph, namely user and project information. User and project information are represented as ID features and abstracted as low-dimensional vector representations. In the process of learning the representation of each node, the graph convolutional neural network incorporates the representation learning of user and project information into the same graph convolution process, indiscriminately aggregates the neighborhood information of each hop of the source node to form the final node representation, and finally uses the inner product between the user and project information representations to calculate the final user and project information matching score, performs matching on user and project information, and thus guides the selection of recommended project information. In a bipartite graph with only user and project information interaction behaviors, when the source node type is determined, the neighbor node types of odd-numbered hops are all different from the source node type, while the neighbor node types of even-numbered hops are the same as the source node type. However, the current graph convolution does not distinguish between different types of hop information in the explicit message passing of representation learning, but simply aggregates heterogeneous hop information into the final node representation to establish a connection between user and item information representations. In this way, the learned node representation does not decouple heterogeneous information, and the learned representation information is relatively noisy. Summary of the invention

[0006] Purpose of the invention: In view of several defects of the message passing mechanism in the existing graph convolutional neural network, the present invention provides a collaborative filtering information recommendation system based on a hybrid message passing mechanism of a graph. The present invention decouples the user-project information bipartite heterogeneous graph into two homogeneous graphs of user and project information by means of an explicit message passing mechanism that decouples heterogeneous objects. At the same time, it is combined with a supplementary implicit message passing method to form a hybrid message passing mechanism for the user-project information bipartite heterogeneous graph, which improves the single graph convolutional message passing mechanism of the undecoupled user-project information representation. The mechanism is used to optimize the learning process of the low-dimensional vector representation of each node in the training process of the graph neural network collaborative filtering algorithm, and improves the recall rate (recall), normalized discounted cumulative gain (NDCG) and other indicators of the information recommendation results. The system of the present invention includes a data preprocessing module, a graph neural collaborative filtering module of a hybrid message passing mechanism, and a user and project information matching score prediction module. Through the collaborative processing of data information by each module, the system can effectively and personalizedly recommend project information, and is suitable for a variety of project information recommendation scenarios.

[0007] The system performs the following steps:

[0008] Step 1: extract the user and project information data to be recommended from the database, process the data through the data preprocessing module, obtain the adjacency matrix of the user and project information interaction, the initial user vector representation matrix and the project information vector representation matrix, and then pass the obtained data information to the graph neural collaborative filtering module of the hybrid message passing mechanism;

[0009] Step 2: The graph neural collaborative filtering module of the hybrid message passing mechanism establishes an explicit message passing mechanism to decouple heterogeneous objects;

[0010] Step 3, the graph neural collaborative filtering module of the hybrid message passing mechanism establishes an implicit message passing mechanism for approximating convergent representation;

[0011] Step 4: Mix the mechanisms established in Step 2 and Step 3 to obtain a collaborative filtering information recommendation model under the hybrid message passing mechanism of the graph. The model processes the input information as a graph neural collaborative filtering module of the hybrid message passing mechanism to obtain a new vector representation matrix about the user and a new vector representation matrix about the project information.

[0012] Step 5: Pass the new vector representation matrix about the user and the new vector representation matrix about the project information obtained in step 4 into the user and project information matching score prediction module, calculate the matching score between each user and each project information, and finally recommend the top k project information with the highest matching score for each user as candidate items to the user.

[0013] In step 1, the user and project information data are historical information in the database. The data preprocessing module will filter out users whose total amount of historical interactive project information is less than a threshold (generally 10), and then construct the adjacency matrix A of the user and project information interaction. The data preprocessing module will use the Xavier initialization method to initialize the user's vector representation matrix E U The vector representation matrix E with the project information I , E I 、E U The vector dimension of each row is dim, E U The number of matrix rows is equal to the number of users, E I The number of rows is equal to the number of item information:

[0014]

[0015] In formula (1), R is the interaction matrix between user and item information, and the size of the interaction matrix is ​​N U ×N I , where N U Represents the number of users, N I Represents the number of project information. If user u interacts with project information i, then the element R in the uth row and ith column of the R matrix ui The value is 1, otherwise it is 0; R T is the transposed matrix of R, 0 is a zero matrix whose elements are all 0, and the size of the entire adjacency matrix A is (N U +N I )×(N U +N I ), and there is no self-loop, the R matrix uniquely corresponds to a user and item information interaction graph G.

[0016] Step 2 includes: The graph convolution layer and aggregation layer in graph collaborative filtering are defined as follows:

[0017]

[0018] Formula (2) represents the graph convolution operation. In formula (2), D is the degree matrix of the adjacency matrix A, which is used to regularize the adjacency matrix for graph convolution operation. E(n) is a low-dimensional vector representation matrix representing the n-th layer user and project information. (0) is the 0th-layer user and item information representation matrix, Represents a vector concatenation operation;

[0019] Formula (3) represents the graph aggregation operation, where E is the low-dimensional vector representation matrix of the final user and item information, and l represents the number of graph convolution iterations;

[0020] When using an adjacency matrix without self-loops, information passing is defined as follows:

[0021]

[0022] In formula (4) and formula (5), They are the node representation of node u obtained after n+1 rounds of iteration and the node representation of node i obtained after n+1 rounds of iteration respectively;

[0023] N(u) and N(i) represent the one-hop neighborhood of user node u and the one-hop neighborhood of item information node i, respectively; d u d t d i d v They represent the degrees of nodes u, t, i, and v in the user-item information interaction graph G respectively;

[0024] The explicit message passing process with the user as the source node is decoupled as follows:

[0025]

[0026] in is the regularized adjacency matrix, represents the initial representation of the user source node;

[0027] According to formula (7), the initial representations of the decoupled user jump and item information jump are:

[0028] They represent the user jump representation and item information jump representation obtained after the n+1th iteration, respectively. is the vector representation of layer 0, The vector representation matrix E of the user initialized by the data preprocessing module U Initialize.

[0029] In step 2, the explicit message passing mechanism for decoupling heterogeneous objects adopts an aggregation scheme that averages the user jump representations of each layer and the item information jump representations of each layer to obtain the final comprehensive user jump representation and item information jump representation:

[0030]

[0031] Where E uu In the case where the user is the source node, the user jump corresponding to the user last provides the representation for the source node; E ui In the case where the user is the source node, the corresponding user's project information jumps to the representation provided by the source node at the end; l1 represents the number of user jump information transmission iterations, and l2 represents the number of project information jump information transmission iterations;

[0032] Get the final matching score:

[0033]

[0034] Among them, E u 、E i They represent the final user representation and item information representation after splicing, E uu 、E ui are the user hop representation and item information hop representation decoupled in the user representation, E iu 、E ii are the user jump representation and item information jump representation decoupled in the item information representation; Formula (12) calculates the vector inner product to obtain the matching score y between the kth user and the jth item information kj , represents the transpose of the kth user representation, e ij Represents the information of the j-th item.

[0035] Step 3 includes: Formula (2) is simplified to Formula (13):

[0036]

[0037] In formula (13), E (0) represents the user and project information representation matrix of the 0th layer, that is, the initialized node representation, E (n +1) is the node representation obtained after n rounds of convolution. express The element in the i-th row and j-th column of the matrix, d i d j Respectively represent the degree of the i-th node and the degree of the j-th node;

[0038] For the two types of heterogeneous nodes, user and item information, their convolutional convergence representation is defined as:

[0039]

[0040] e in formula (15) u 、e i They are the node representation of the final convergence of user u and the node representation of the final convergence of item information i; They are the node representations of user node u obtained in the l+1th round of iteration and the node representations of project information node i obtained in the l+1th round of iteration; e u 、e t 、e i 、e v They represent the final converged representations of nodes u, t, i, and v respectively;

[0041] For the decoupled graph message passing, according to formula (7) and formula (8), each node will learn two types of node representations for the final concatenation of inner products to calculate the matching score. The convergence representation of the nodes of the decoupled graph convolution is uniformly defined as follows:

[0042]

[0043] Where E ego Indicates the source node matrix that needs to be calculated; is the square of the normalized adjacency matrix; e ego represents the source node representation that needs to be calculated, and N(ego) represents the one-hop neighborhood of the ego node after sampling. Depending on the specific project information recommended, the number of one-hop neighbors of different ego nodes is different. Therefore, for the convenience of calculation, N is sampled for each ego node. sample Neighbors make up N(ego), e t represents one of the one-hop neighbor nodes of the source node, d ego d t They represent the degree of the source node and the degree of the source node’s neighbor node t respectively.

[0044] In step 3, a negative logarithmic loss function is finally introduced to model the implicit message passing mechanism:

[0045]

[0046] Where L implicit represents the loss function, U and I are the sets of user and item information nodes respectively, ln represents the logarithmic function, and σ(·) represents the sigmoid activation function.

[0047] Step 4 includes:

[0048] For the item information representation, the nodes with even hops constitute the representation of each hop of the item information. In the process of negative example synthesis, the observed item information interacted with the user is first taken as the positive example, and N items are randomly selected. neg The item information that is not in the neighborhood of the positive example user is used as the candidate for the initial negative example. The jump representation of each negative example item information is mixed with the corresponding jump representation of the positive example item information to obtain the item information jump negative example representation of the enhanced negative example item information:

[0049]

[0050] Where i represents the positive example item information number, j represents the randomly sampled negative example item information number, and α is a random number sampled from 0 to 1 according to a uniform distribution. denote the representation of the 2nth hop of the i-th item information node and the representation of the 2nth hop of the j-th item information node, respectively, and N denotes a natural number set;

[0051] For the item information representation, the nodes with odd hops constitute the representation of each hop of the item information. In the process of negative example synthesis, the item information that has been observed to interact with the user is first taken as the positive example, and N nodes are randomly selected. neg The item information that is not in the neighborhood of the positive user is used as the candidate for the initial negative example. The user jump representation of each negative item information is mixed with the corresponding user jump representation of the positive item information to obtain the user jump negative example representation of the enhanced negative item information:

[0052]

[0053] Where i represents the positive example item information number, j represents the randomly sampled negative example item information number, and α is a random number sampled from 0 to 1 according to a uniform distribution. denote the representation of the 2n+1th hop of the i-th item information node and the representation of the 2n+1th hop of the j-th item information node, respectively, and N denotes a natural number set;

[0054] The representation of each item information jump of the enhanced item information negative example is inner-producted with the final representation of the item information jump of the positive user. In each item information jump, the negative item information jump representation with the highest inner-product score is selected as the jump representation corresponding to the synthetic negative example, and finally the averaging is used to synthesize into a strong item information jump negative example; for user jump information, the representation of each user jump of the enhanced item information negative example is inner-producted with the final representation of the user jump of the positive user. In each user jump, the negative user jump representation with the highest inner-product score is selected as the jump representation corresponding to the synthetic negative example, and finally the averaging is used to synthesize into a strong user jump negative example. After obtaining the two types of negative example representations of strong item information jump and strong user jump of project information, vector concatenation is performed in the same order of user jump and item information jump to obtain the final vector representation of the synthetic negative example.

[0055] In step 4, the BPR loss function L is used BPR Train the collaborative filtering information recommendation model based on the graph-based hybrid message passing mechanism:

[0056] L BPR =-∑ u∈U ∑ i∈N(u) ∑ j∈N(u) ln(σ(y ui -y uj ))+β*||E (0) || 2 (twenty two)

[0057] In formula (22), U is the user set, N(u) represents the one-hop neighbor set of node u, and y ui is the matching score between user u node and item information i node, y uj is the matching score of the user u node and the synthetic item information negative example j node, E (0) represents the initialized node representation, β is the coefficient of l2 regularization, ln is the logarithmic function, σ(·) is the sigmoid activation function, ||·|| 2 is the function for calculating the l2 norm;

[0058] Combining the BPR loss function of formulas (19) and (21) and the negative logarithmic loss function of the implicit message passing mechanism, the final loss function L of the model is defined as follows:

[0059] L=L BPR +γ*L implicit (twenty three)

[0060] where γ is a coefficient that controls the strength of the implicit message passing mechanism.

[0061] In step 5, the graph neural collaborative filtering model trained by the loss function of formula (23) constitutes a graph neural collaborative filtering module of a hybrid message passing mechanism. This module will generate a new user vector representation matrix and a new project information vector representation matrix, and then pass them into the user and project information matching score prediction module. The user and project information matching score prediction module calculates the matching score between each user and each project information based on the passed information, and finally recommends the top k project information with the highest matching score for each user as the final recommendation result to the corresponding user.

[0062] Beneficial effects: In the existing collaborative filtering recall model based on graph neural network, different types of heterogeneous object messages are not decoupled in the process of processing heterogeneous graphs, so there is a certain degree of information loss in the node representation learning process. The present invention designs a graph neural network that explicitly decouples heterogeneous object message transmission for the user-item information bipartite graph of collaborative filtering. Compared with the traditional scheme, the explicit decoupled message transmission mechanism does not change the time complexity of the original graph convolutional neural network algorithm, and improves the recall rate (recall), normalized discounted cumulative gain (NDCG) and other indicators of the final recall result. At the same time, by decoupling heterogeneous object representation learning, it provides explainability for the representation learning process of the recommendation system. In addition, the introduction of implicit message transmission mechanism complements the explicit message transmission mechanism, alleviates the potential information loss caused by the existing graph neural network due to the limited message transmission range, so that the graph neural network model can learn better node representation faster. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0064] Figure 1 It is a bipartite graph of user-item information.

[0065] Figure 2 It is a bipartite graph with user type nodes as source nodes and a decoupled graph.

[0066] Figure 3 It is a bipartite graph and a decoupling graph with the project information type node as the source node.

[0067] Figure 4 It is a schematic diagram of the inner product represented by splicing decoupling.

[0068] Figure 5 It is a schematic diagram of the negative example flow of the project information jump representation of the synthesized project information.

[0069] Figure 6 It is a module principle diagram of the collaborative filtering information recommendation system of the present invention. DETAILED DESCRIPTION

[0070] The present invention mainly improves the message passing mechanism in graph neural networks, and learns node representation by combining explicit message passing mechanism with implicit message passing mechanism. The following is a detailed description of the graph collaborative filtering information recommendation method of the hybrid message passing mechanism:

[0071] 1. Explicit messaging mechanism for decoupling heterogeneous objects:

[0072] In the graph collaborative filtering problem of the present invention, only the interaction information of user and project information is involved, wherein the user and project information are represented as ID features and abstracted as low-dimensional latent vector representations, without introducing other user attribute features and project information attribute features. The bipartite graph recording the user-project information interaction information with a certain user as the source node is specifically as follows: Figure 1 As shown, Figure 1 In this case, two methods of drawing the user-project information bipartite graph are shown. The edges between the user set and the project information set represent the interaction between the corresponding users and the project information. Nodes in the same type set do not have any interaction behavior. The bipartite graph is centered on the source node and can be abstracted as a graph with high-order connectivity. The neighbor nodes of each hop are of the same type, and the heterogeneous types of the corresponding nodes alternate between user type and project information type as the number of hops increases.

[0073] Taking the spatial domain graph convolutional neural network commonly used in collaborative filtering problems in recommendation systems as an example, when the number of aggregation hops is limited to 3, Figure 1The representation of the source node in the final representation will be aggregated from the node information from the project information hop-user hop-project information hop-source node. As the limit on the number of aggregation hops changes, the scope of information that the source node can obtain will also change accordingly.

[0074] In the recall process of the recommendation system, the user-item information interaction data involved corresponds to a large-scale sparse graph, and each sparse graph corresponds to an adjacency matrix A:

[0075]

[0076] In formula (1), R is the interaction matrix of user-item information, and the size of the interaction matrix is ​​N U ×N I , where N U Represents the number of users, N I Represents the number of project information. If user u interacts with project information i, then the element R in the uth row and ith column of the R matrix ui The value is 1, otherwise it is 0; R T is the transposed matrix of R, 0 is a zero matrix whose elements are all 0, and the size of the entire adjacency matrix A is (N U +N I )×(N U +N I ), and there is no self-loop, the R matrix uniquely corresponds to a user-item information interaction graph G.

[0077] The graph convolutional layer and aggregation layer in graph collaborative filtering are defined as follows:

[0078]

[0079] Formula (2) represents the graph convolution operation. In formula (2), D is the degree matrix of the adjacency matrix A, which is used to regularize the adjacency matrix for the graph convolution operation. (n) is a low-dimensional vector representation matrix representing the n-th layer of user and item information, E (0) is the user and item information representation matrix at the 0th level, E U 、E I They are the vector representation matrix of the user and the vector representation matrix of the project information initialized by the data preprocessing module. Represents a vector concatenation operation;

[0080] Formula (3) represents the graph aggregation operation, where E is the low-dimensional vector representation matrix representing the final user-item information, l represents the number of graph convolution iterations, and the aggregation operation averages the representation obtained in each iteration with the representation of the original source node to obtain the final node representation.

[0081] The above is the representation obtained by calculating all nodes in parallel through matrix operations after one transmission. Specifically, when using an adjacency matrix without self-loops, the information transmission of each layer of each node of the heterogeneous object is defined as follows:

[0082]

[0083] In formula (4) and formula (5), They are the node representations of u and i nodes obtained in the n+1th round of iterations. The operation process of this representation is to obtain the representations obtained in the nth round of iterations of the one-hop neighbor and perform weighted summation to obtain the final information. N(u) and N(i) represent the one-hop neighborhood of user node u and the one-hop neighborhood of project information node i respectively; d u d t d i d v They represent the degrees of nodes u, t, i, and v in the user-item information interaction bipartite graph G, and their specific values ​​can be obtained through the degree matrix D. From formula (4) and formula (5), we can see that the representation of the node will continuously obtain new hop information through iterative aggregation to update the node representation.

[0084] It is known that in the user-item information bipartite graph, the node types of two adjacent hops are heterogeneous. In order to distinguish the information of different types of objects in explicit message passing, taking the case where the source node is of user type as an example, the decoupled explicit message passing process is defined as follows:

[0085]

[0086]

[0087] Formula (6) is the regularized adjacency matrix. In order to achieve the aggregation of different types of nodes, represents the initial representation of the user source node. According to formula (7), the initial representations of the decoupled user jump and item information jump can be obtained as They represent the user jump representation and item information jump representation obtained after the n+1th iteration respectively. is the vector representation of layer 0, The vector representation matrix E of the user initialized by the data preprocessing module U Initialize.

[0088] like Figure 2 , Figure 3As shown in the figure, in the transmission aggregation process with user type and project information type nodes as source nodes, the decoupled message passing mechanism can be seen as decoupling the original user-project information bipartite graph into a homogeneous graph about user and project information, and a virtual node needs to be added to the decoupled graph of different types from the source node to collect messages, and then perform ordinary graph convolution message passing aggregation in the final homogeneous graph to obtain two types of final representations. Each type of node can be disassembled into two types of representations, corresponding to two decoupled graphs.

[0089] The aggregation scheme adopted by the decoupled explicit message passing mechanism is to obtain the final comprehensive user jump representation and item information jump representation by averaging the user jump representation of each layer and the item information jump representation of each layer:

[0090]

[0091] E in formula (10) uu 、E ui In the case of the user as the source node, the corresponding user jump and item information jump are finally provided for the source node. l1 represents the number of user jump information transmission, and l2 represents the number of item information jump information transmission iterations. In this information recommendation system, the values ​​of l1 and l2 are 3. The derivation of the rest only requires replacing the first subscript u of the elements in formula (7), formula (8), and formula (10) with i, that is, the calculation formula corresponding to the case where the source node is the item information type, and the values ​​of l1 and l2 parameters are consistent.

[0092] The way that the graph neural network that has not been decoupled finally calculates a pair of specific user and project information is to perform the inner product of the representation of the user and project information to obtain a matching score. In the case of decoupled explicit message passing, it is necessary to perform vector concatenation of the user jump representation of the user node and the project information node and the project information jump representation to obtain the final user and project information representation. In this way, in the process of performing the inner product of the concatenated vectors, the neighborhood similarity of nodes of the same type is matched at each dimension level, instead of mixing the information of nodes of different types in each dimension as in the process of performing the inner product of the user and project information representation that has not been decoupled. Figure 4 This is a schematic diagram of the inner product of the concatenated representation after decoupling. Different nodes perform inner products on the same type of information representation in this inner product mode, which can also be regarded as matching the representations of the corresponding decoupling graphs to obtain the final matching score:

[0093]

[0094] In formula (11), E u 、E i represents the final user and project information after splicing, E uu 、Eui are the user hop representation and item information hop representation decoupled in the user representation, E iu 、E ii They are the user hop representation and item hop representation decoupled from the item information representation, Represents the vector concatenation operation. Formula (12) calculates the vector inner product to obtain the matching score y between the kth user and the jth item information kj , represents the transpose of the kth user representation, e ij Represents the information of the j-th item.

[0095] 2. Implicit message passing mechanism for approximate convergence representation:

[0096] The graph convolutional neural network captures high-order collaborative signals during the iteration process. The iteration process of Formula 2 can be simplified to

[0097] Formula 13:

[0098]

[0099] In formula (13), E(0) represents the initial node representation, and after n rounds of convolution, E(n+1) is obtained. When n approaches infinity, formula (14) shows that the power of the regularization matrix in formula (13) tends to a fixed matrix. express The element in the i-th row and j-th column of the matrix, d i d j Represent the degrees of the i-th and j-th nodes respectively. When the graph convolutional neural network is iterated to a certain number, the final representation of each node will tend to converge to a fixed vector representation. The node representations obtained after iteration l and iteration l+1 are the same. For the two types of heterogeneous nodes, user and project information, their convolutional convergence representations are defined as:

[0100]

[0101] e in formula (15) u 、e i They are the node representation of the final convergence of user u and the node representation of the final convergence of item information i; are the node representation of user node u obtained in the l+1th round of iteration and the node representation of project information node i obtained in the l+1th round of iteration respectively; N(u) and N(i) in formula (16) represent the one-hop neighborhood of user node u and the one-hop neighborhood of project information node i respectively; e u 、e t 、e i 、e v represents the final converged representation of nodes u, t, i, v; du d t d i d v They respectively represent the degrees of nodes u, t, i, and v in the bipartite graph G of user-item information interaction; the final representation of each node convolution convergence and its corresponding weight coefficient are uniquely determined by its convergent neighborhood representation.

[0102] For the decoupled graph message passing, according to formula (7) and formula (8), each node will learn two types of node representations for the final concatenation of inner products to calculate the matching score, and the initial representations of the two types of nodes are derived from the randomly initialized representation E(0). Therefore, the converged representation of the nodes of the decoupled graph convolution can be uniformly defined as follows:

[0103]

[0104] E in formula (17) ego Indicates the source node matrix that needs to be calculated and is used to uniformly refer to different types of nodes. is the square of the normalized adjacency matrix. Formula (18) is a refinement of Formula (17), specifically for the actual calculation of each node, e ego represents the source node representation that needs to be calculated, and N(ego) represents the one-hop neighborhood of the ego node after sampling. Depending on the specific project information recommended, the number of one-hop neighbors of different ego nodes is different. Therefore, for the convenience of calculation, N is sampled for each ego node. sample Neighbors make up N(ego), e t represents one of the one-hop neighbor nodes of the source node, d ego d t Represent the degrees of the source node and the source node neighbor node t respectively. Formula 18 defines the final representation of the convolution convergence of each node in the decoupled graph convolution and its corresponding weight coefficient.

[0105] In order to break through the limitation of explicitly set message passing hops in the message passing mechanism, the present invention introduces an implicit message passing mechanism. By maximizing the cosine similarity between the source node and all its neighbor nodes, an implicit approximation mechanism is introduced in the learning process. Finally, a negative logarithmic loss function is introduced to model the implicit message passing mechanism:

[0106]

[0107] In formula (19), L implicit represents the loss function, U and I are the user and item information sets respectively, ln represents the logarithmic function, and σ(·) represents the sigmoid activation function.

[0108] 3. Collaborative filtering information recommendation model under hybrid message passing mechanism

[0109] In the graph neural network collaborative filtering recall algorithm, it is necessary to match one or more pairs of negative examples for each pair of observed user-item information pairs. The MixGCF negative sampling method in the reference (Huang, Tinglin, et al. "MixGCF: An Improved Training Method for Graph Neural Network-based Recommender Systems." (2021).) synthesizes negative examples of user jump representation and negative examples of item information jump representation for explicitly decoupling the item information of the message. Figure 5 The negative example process of item information skip representation for synthesizing item information is introduced in.

[0110] First, for the item information representation, the nodes with even hops constitute the representation of the item information hops. In the process of negative example synthesis, the observed item information interacted with the user is first taken as the positive example, and N items are randomly selected. neg The item information that is not in the neighborhood of the positive example user is used as the candidate for the initial negative example. The jump representation of each negative example item information is mixed with the corresponding jump representation of the positive example item information to obtain the item information jump negative example representation of the enhanced negative example item information:

[0111]

[0112] In formula (20), i represents the positive example item information number, j represents the randomly sampled negative example item information number, α is a random number sampled from 0 to 1 according to a uniform distribution, and 2n limits the number of aggregated hops to an even number of hops (item information hops). represents the representation of the 2nth hop of the i-th and j-th item information nodes, where N represents a set of natural numbers;

[0113] For the item information representation, the nodes with odd hops constitute the representation of each hop of the item information. In the process of negative example synthesis, the item information that has been observed to interact with the user is first taken as the positive example, and N nodes are randomly selected. neg The item information that is not in the neighborhood of the positive user is used as the candidate for the initial negative example. The user jump representation of each negative item information is mixed with the corresponding user jump representation of the positive item information to obtain the user jump negative example representation of the enhanced negative item information:

[0114]

[0115] In formula (21), i represents the positive example item information number, j represents the randomly sampled negative example item information number, and α is a random number sampled from 0 to 1 according to a uniform distribution. denote the representation of the 2n+1th hop of the i-th item information node and the representation of the 2n+1th hop of the j-th item information node, respectively, and N denotes a natural number set;

[0116] The representation of each item information jump of the enhanced item information negative example is inner-producted with the final representation of the item information jump of the positive user. In each item information jump, the negative item information jump representation with the highest inner-product score is selected as the jump representation corresponding to the synthetic negative example, and finally the averaging is used to synthesize into a strong item information jump negative example; for user jump information, the representation of each user jump of the enhanced item information negative example is inner-producted with the final representation of the user jump of the positive user. In each user jump, the negative user jump representation with the highest inner-product score is selected as the jump representation corresponding to the synthetic negative example, and finally the averaging is used to synthesize into a strong user jump negative example. After obtaining the two types of negative example representations of strong item information jump and strong user jump of project information, vector concatenation is performed in the same order of user jump and item information jump to obtain the final vector representation of the synthetic negative example.

[0117] The BPR loss function is usually used to train the model:

[0118] L BPR =-∑ u∈U ∑ i∈N(u) ∑ j∈N(u) ln(σ(y ui -y uj ))+β*||E (0) || 2 (twenty two)

[0119] In formula (22), U is the user set, N(u) represents the one-hop neighbor set of node u, and y ui is the matching score between user u node and item information i node, y uj is the matching score of the user u node and the synthetic item information negative example j node, E (0) represents the initialized node representation, β is the coefficient of l2 regularization, ln is the logarithmic function, σ(·) is the sigmoid activation function, ||·|| 2 is the function that calculates the 12-norm.

[0120] Combining the BPR loss function of formulas (19) and (22) and the negative logarithmic loss function of the implicit message passing mechanism, the final loss function of the model is defined as follows:

[0121] L=L BPR +γ*L implicit (twenty three)

[0122] In formula (23), γ is the coefficient that controls the strength of the implicit message passing mechanism.

[0123] 4. Collaborative Filtering Information Recommendation System

[0124] like Figure 6 As shown in the figure, the module principle diagram of the recommendation system of the collaborative filtering information recommendation model based on the graph hybrid message passing mechanism is presented. The system includes a data preprocessing module, a graph neural collaborative filtering module based on the hybrid message passing mechanism, and a user and project information matching score prediction module.

[0125] The processing flow of the information recommendation system for personalized information recommendation is as follows:

[0126] First, the user and project information data to be recommended are extracted from the database and processed by the data preprocessing module, and then the obtained data information is passed to the graph neural collaborative filtering module of the hybrid message passing mechanism. The user and project information data are historical information in the database. The data preprocessing module will filter out users whose total historical interactive project information is less than 10, and then construct the user-project information adjacency matrix A. The data preprocessing module will use the Xavier initialization method to initialize the user's vector representation matrix E U The vector representation matrix E with the project information I , E I 、E U The vector dimension of each row is dim, E U The number of matrix rows is equal to the number of users, E I The number of rows is equal to the number of item information.

[0127] The graph neural collaborative filtering information recommendation model with hybrid message passing mechanism is used as a graph neural collaborative filtering module to process the information output from the data preprocessing module to obtain a new vector representation matrix about the user and a new vector representation matrix about the project information. The new vector representation matrix of the user and the new vector representation matrix of the project information will be passed to the user and project information matching score prediction module, which calculates the matching score of each user and each project information. Finally, the information recommendation system will recommend the top k project information with the highest matching score for each user as candidate results to the user.

[0128] Table 1 shows the parameter selection and recommendation result indicators for the information recommendation system for three types of project information, where recall represents the recall rate, and NDCG represents the normalized discounted cumulative gain. This indicator is calculated based on recommending 20 project information to each user; the Yelp2018 dataset provides information about restaurants and bars visited by Yelp website users in 2018, the Gowalla dataset is information about users and their historical check-in locations provided by Stanford University, and the Amazon-book dataset is the browsing and purchase records of users and related books crawled from the Amazon website. This information recommendation system method processes the above datasets and recommends 20 project information that may be of interest to users based on the information provided, such as recommending restaurants and bars based on Yelp2018 data, recommending check-in locations based on Gowalla data, or recommending books to users based on Amazon-book data.

[0129] Table 1

[0130] Parameters_Dataset Yelp2018 Gowalla Amazon-book dim 128 128 128 <![CDATA[l1]]> 3 3 3 <![CDATA[l2]]> 3 3 3 β 0.001 0.001 0.001 γ 0.05 1.0 0.1 <![CDATA[N sample ]]> 800 50 200 <![CDATA[N neg ]]> 64 64 16 <![CDATA[N U ]]> 31668 29858 52643 <![CDATA[N I ]]> 38048 40981 91599 recall 7.48％ 19.40％ 5.02％ NDCG 6.16％ 16.33％ 2.35％

[0131] The present invention provides a collaborative filtering information recommendation method and system based on a hybrid message passing mechanism of a graph. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A collaborative filtering information recommendation system based on a hybrid message passing mechanism of a graph, characterized in that: It includes a data preprocessing module, a graph neural collaborative filtering module with a hybrid message passing mechanism, and a user and project information matching score prediction module; The system performs the following steps: Step 1: extract the user and project information data to be recommended from the database, process the data through the data preprocessing module, obtain the adjacency matrix of the user and project information interaction, the initial user vector representation matrix and the project information vector representation matrix, and then pass the obtained data information to the graph neural collaborative filtering module of the hybrid message passing mechanism; Step 2: The graph neural collaborative filtering module of the hybrid message passing mechanism establishes an explicit message passing mechanism to decouple heterogeneous objects; Step 3, the graph neural collaborative filtering module of the hybrid message passing mechanism establishes an implicit message passing mechanism for approximating convergent representation; Step 4: Mix the mechanisms established in Step 2 and Step 3 to obtain a collaborative filtering information recommendation model under the hybrid message passing mechanism of the graph. The model processes the input information as a graph neural collaborative filtering module of the hybrid message passing mechanism to obtain a new vector representation matrix about the user and a new vector representation matrix about the project information. Step 5: The new vector representation matrix of the user and the new vector representation matrix of the project information obtained in step 4 are passed to the user and project information matching score prediction module to calculate the matching score between each user and each project information, and finally the top k project information with the highest matching score for each user are recommended to the user as candidate items; In step 1, the user and project information data are historical information in the database. The data preprocessing module will filter out users whose total amount of historical interactive project information is less than the threshold, and then construct the adjacency matrix A of the user and project information interaction. The data preprocessing module will use the Xavier initialization method to initialize the user's vector representation matrix E U The vector representation matrix E with the project information I , E I 、E U The vector dimension of each row is dim, E U The number of matrix rows is equal to the number of users, E I The number of rows is equal to the number of item information: In formula (1), R is the interaction matrix between user and item information, and the size of the interaction matrix is ​​N U ×N I , where N U Represents the number of users, N I Represents the number of project information. If user u interacts with project information i, then the element R in the uth row and ith column of the R matrix ui The value is 1, otherwise it is 0; R T is the transposed matrix of R, 0 is a zero matrix whose elements are all 0, and the size of the entire adjacency matrix A is (N U +N I )×(N U +N I ), and there is no self-loop, the R matrix uniquely corresponds to a user and project information interaction graph G; Step 2 includes: The graph convolution layer and aggregation layer in graph collaborative filtering are defined as follows: Formula (2) represents the graph convolution operation. In formula (2), D is the degree matrix of the adjacency matrix A, which is used to regularize the adjacency matrix for the graph convolution operation. (n) is a low-dimensional vector representation matrix representing the n-th layer of user and item information, E (0) is the 0th-layer user and item information representation matrix, Represents a vector concatenation operation; Formula (3) represents the graph aggregation operation, where E is the low-dimensional vector representation matrix of the final user and item information, and l represents the number of graph convolution iterations; When using an adjacency matrix without self-loops, information passing is defined as follows: In formula (4) and formula (5), They are the node representation of node u obtained after n+1 rounds of iteration and the node representation of node i obtained after n+1 rounds of iteration respectively; N(u) and N(i) represent the one-hop neighborhood of user node u and the one-hop neighborhood of item information node i, respectively; d u d t d i d v They represent the degrees of nodes u, t, i, and v in the user-item information interaction graph G respectively; The explicit message passing process with the user as the source node is decoupled as follows: in is the regularized adjacency matrix, represents the initial representation of the user source node; According to formula (7), the initial representations of the decoupled user jump and item information jump are: They represent the user jump representation and item information jump representation obtained after the n+1th iteration, respectively. is the vector representation of layer 0, The vector representation matrix E of the user initialized by the data preprocessing module U Initialize.

2. The system according to claim 1, characterized in that In step 2, the explicit message passing mechanism for decoupling heterogeneous objects adopts an aggregation scheme that averages the representations of user jumps at each layer and item information jumps at each layer to obtain the final comprehensive user jump representation and item information jump representation: Where E uu In the case where the user is the source node, the user jump corresponding to the user last provides the representation for the source node; E ui In the case of the user as the source node, the corresponding user's project information jumps to the representation provided by the source node at the end, l1 represents the number of user jump information transmission iterations, and l2 represents the number of project information jump information transmission iterations; Get the final matching score: Among them, E u 、E i They represent the final user representation and item information representation after splicing, E uu 、E ui are the user hop representation and item information hop representation decoupled in the user representation, E iu 、E ii are the user jump representation and item information jump representation decoupled in the item information representation; Formula (12) calculates the vector inner product to obtain the matching score y between the kth user and the jth item information kj , represents the transpose of the kth user representation, e ij Represents the information of the j-th item.

3. The system according to claim 2, characterized in that Step 3 includes: Formula (2) is simplified to Formula (13): In formula (13), E (n+1) is the node representation obtained after n rounds of convolution. express The element in the i-th row and j-th column of the matrix, d i ,d j Respectively represent the degree of the i-th node and the degree of the j-th node; For the two types of heterogeneous nodes, user and item information, their convolutional convergence representation is defined as: e in formula (15) u 、e i They are the node representation of the final convergence of user u and the node representation of the final convergence of item information i; They are the node representations of user node u obtained in the l+1th round of iteration and the node representations of project information node i obtained in the l+1th round of iteration; e u 、e t 、e i 、e v They represent the final converged representations of nodes u, t, i, and v respectively; For the decoupled graph message passing, according to formula (7) and formula (8), each node will learn two types of node representations for the final concatenation of inner products to calculate the matching score. The convergence representation of the nodes of the decoupled graph convolution is uniformly defined as follows: Where E ego Indicates the source node matrix that needs to be calculated; is the square of the normalized adjacency matrix; e ego represents the source node representation that needs to be calculated, N(ego) represents the one-hop neighborhood after sampling of the node ego, and the number of samples is N sample , e t represents one of the one-hop neighbor nodes of the source node, d ego d t They represent the degree of the source node and the degree of the source node’s neighbor node t respectively.

4. The system according to claim 3, characterized in that In step 3, a negative logarithmic loss function is finally introduced to model the implicit message passing mechanism: Where L implicit represents the loss function, U and I are the sets of user and item information nodes respectively, ln represents the logarithmic function, and σ(·) represents the sigmoid activation function.

5. The system according to claim 4, characterized in that Step 4 includes: For the item information representation, the nodes with even hops constitute the representation of the item information hops. In the process of negative example synthesis, the observed item information interacted with the user is first taken as the positive example, and N items are randomly selected. neg The item information that is not in the neighborhood of the positive example user is used as the candidate for the initial negative example. The jump representation of each negative example item information is mixed with the corresponding jump representation of the positive example item information to obtain the item information jump negative example representation of the enhanced negative example item information: Where i represents the positive example item information number, j represents the randomly sampled negative example item information number, and α is a random number sampled from 0 to 1 according to a uniform distribution. denote the representation of the 2nth hop of the i-th item information node and the representation of the 2nth hop of the j-th item information node, respectively, and N denotes a natural number set; For the item information representation, the nodes with odd hops constitute the representation of each hop of the item information. In the process of negative example synthesis, the item information that has been observed to interact with the user is first taken as the positive example, and N nodes are randomly selected. neg The item information that is not in the neighborhood of the positive user is used as the candidate for the initial negative example. The user jump representation of each negative item information is mixed with the corresponding user jump representation of the positive item information to obtain the user jump negative example representation of the enhanced negative item information: Where i represents the positive example item information number, j represents the randomly sampled negative example item information number, and α is a random number sampled from 0 to 1 according to a uniform distribution. denote the representation of the 2n+1th hop of the i-th item information node and the representation of the 2n+1th hop of the j-th item information node, respectively, and N denotes a natural number set; The representation of each item information jump of the enhanced item information negative example is inner-producted with the final representation of the item information jump of the positive user. In each item information jump, the negative item information jump representation with the highest inner-product score is selected as the jump representation corresponding to the synthetic negative example, and finally the averaging is used to synthesize into a strong item information jump negative example; for user jump information, the representation of each user jump of the enhanced item information negative example is inner-producted with the final representation of the user jump of the positive user. In each user jump, the negative user jump representation with the highest inner-product score is selected as the jump representation corresponding to the synthetic negative example, and finally the averaging is used to synthesize into a strong user jump negative example. After obtaining the two types of negative example representations of strong item information jump and strong user jump of project information, vector concatenation is performed in the order of user jump and item information jump to obtain the final vector representation of the synthetic negative example.

6. The system according to claim 5, characterized in that In step 4, the BPR loss function L is used BPR Train the collaborative filtering information recommendation model based on the graph-based hybrid message passing mechanism: L BPR =-∑ u∈U ∑ i∈N(u) ∑ j∈N(u) ln(σ(y ui -y uj ))+β*||E (0) || 2 (22) In formula (22), U is the user set, N(u) represents the one-hop neighbor set of node u, and y ui is the matching score between user u node and item information i node, y uj is the matching score of the user u node and the synthetic item information negative example j node, E (0) represents the initialized node representation, β is the coefficient of l2 regularization, ln is the logarithmic function, σ(·) is the sigmoid activation function, ||·|| 2 is the function for calculating the l2 norm; Combining the BPR loss function of formulas (19) and (21) and the negative logarithmic loss function of the implicit message passing mechanism, the final loss function L of the model is defined as follows: L=L BPR +γ*L implicit (23) where γ is a coefficient that controls the strength of the implicit message passing mechanism.

7. The system according to claim 6, characterized in that In step 5, the graph neural collaborative filtering model trained by the loss function of formula (23) constitutes a graph neural collaborative filtering module of a hybrid message passing mechanism. This module will generate a new user vector representation matrix and a new project information vector representation matrix, and then pass them into the user and project information matching score prediction module. The user and project information matching score prediction module calculates the matching score between each user and each project information based on the passed information, and finally recommends the top k project information with the highest matching score for each user as the final recommendation result to the corresponding user.

Images