A Recommendation Method and System Based on De-Smoothed Graph Convolutional Neural Network

Abstract

The present invention proposes a recommendation method and system based on a de-smoothed graph convolutional neural network, comprising the following steps: obtaining initial embedding vectors of user nodes and item nodes from a user-item interaction graph as training samples; performing a smoothed convolutional aggregation operation on the training samples through a multi-layer graph convolutional neural network model based on a de-smoothed module to obtain user embedding vectors and item embedding vectors of each order; respectively performing layer combination on the obtained user embedding vectors and item embedding vectors of each order, and performing an inner product operation on the final user embedding vectors and item embedding vectors obtained after layer combination to obtain the predicted scores of the training samples; training the multi-layer graph convolutional neural network model based on the de-smoothed module by using BPR-Loss to obtain a trained recommendation model. The de-smoothed module of the present invention can help graph nodes maintain their diversity and prevent the degradation of the feature diversity of nodes.

Description

Technical Field

[0001] The present invention belongs to the technical field related to recommendation, and particularly relates to a recommendation method and system based on a de-smoothed graph convolutional neural network. Background Art

[0002] The statements in this part only provide background technical information related to the present invention, and do not necessarily constitute prior art.

[0003] With the rapid explosion of information on various services and platforms (such as e-commerce, short videos, etc.), personalized recommendation technology aimed at alleviating the problem of information overload is developing rapidly and has been widely applied. Due to its effectiveness in the real world, both the industrial community and the academic community are very concerned about this field. The main challenge of the recommendation system is to learn the embedding representations of users and items from the interactions between users and items. The interaction between users and items can be regarded as a bipartite graph structure. Due to the advantages of graph neural networks in learning graph-structured data, the recommendation system is using the graph neural network architecture to model the recommendation task from the perspective of the graph.

[0004] The Graph Convolution Network (GCN) is a graph neural network based on graph topology that uses convolutional operators to transfer and aggregate information. GCN has the natural ability to establish correlations between users, items, and related features by performing message-passing on the graph. For the recommendation system, specifically, neighborhood embeddings are aggregated into the target node through message-passing, and this operation is iteratively executed multiple times. In this way, the high-order structured information and multi-hop correlations between graph nodes are encoded in the corresponding embeddings, further enhancing the expressive ability of the embedding vectors of user and item nodes. Compared with traditional models, these GCN-based recommendation models have stronger performance.

[0005] Although graph neural networks have achieved great success in the recommendation system, there is still a serious problem with the current GCN: as the number of layers increases, the model performance degrades severely. This problem is widely considered to be caused by over-smoothing. Because graph convolution operations tend to mix the features of connected nodes through message-passing, when graph convolution operations are iterated multiple times, the features between interconnected nodes will become indistinguishable. This phenomenon is called the over-smoothing of graph neural networks. Considering that deeper neural networks usually exhibit better expressive and reasoning abilities. Therefore, how to de-smooth the graph convolutional network and improve the accuracy of the recommendation model is the problem to be solved currently. Summary of the Invention

[0006] To overcome the deficiencies of the above-mentioned existing technologies, the present invention provides a recommendation method and system based on a de-smoothed graph convolutional neural network. The adopted smoothing module realizes de-smoothing only by multiplying the mask matrix with the feature vector. On the one hand, the matrix multiplication operation will not become a burden on model training; on the other hand, it can effectively solve the over-smoothing problem existing in the graph convolutional neural network of the recommendation system and improve the accuracy of the recommendation model.

[0007] To achieve the above object, one or more embodiments of the present invention provide the following technical solutions: A recommendation method based on a de-smoothed graph convolutional neural network, comprising the following steps:

[0008] Step 1: Obtain the initial embedding vectors of user nodes and item nodes through the user-item interaction graph as training samples;

[0009] Step 2: Perform a smoothing convolution aggregation operation on the training samples through a multi-layer graph convolutional neural network model based on the de-smoothed module to obtain user embedding vectors and item embedding vectors of each order;

[0010] Step 3: Perform layer combination on the obtained user embedding vectors and item embedding vectors of each order respectively, and perform an inner product operation on the final user embedding vector and item embedding vector obtained after layer combination to obtain the predicted score of the training sample;

[0011] Step 4: Use BPR-Loss to train the multi-layer graph convolutional neural network model based on the de-smoothed module to obtain a trained recommendation model.

[0012] The second part of the present invention provides a recommendation system based on a de-smoothed graph convolutional neural network, comprising:

[0013] An acquisition module: Obtain the initial embedding vectors of user nodes and item nodes through the user-item interaction graph;

[0014] A convolution aggregation module: Perform a smoothing convolution aggregation operation on the training samples through a multi-layer graph convolutional neural network model based on the de-smoothed module to obtain user embedding vectors and item embedding vectors of each order;

[0015] A calculation module: Perform layer combination on the obtained user embedding vectors and item embedding vectors of each order respectively, and perform an inner product operation on the final user embedding vector and item embedding vector obtained after layer combination to obtain the predicted score of the training sample;

[0016] A model training module: Use BPR-Loss to train the multi-layer graph convolutional neural network model based on the de-smoothed module to obtain a trained recommendation model.

[0017] The third aspect of the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps described in the above method.

[0018] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and computer instructions stored on the memory and running on the processor. When the computer instructions are run by the processor, the steps described in the above method are completed.

[0019] The above one or more technical solutions have the following beneficial effects:

[0020] The de-smoothing module of the present invention realizes de-smoothing only by multiplying the mask matrix with the feature vector. On the one hand, matrix multiplication operations will not become a burden on model training; on the other hand, it can effectively solve the over-smoothing problem existing in the graph convolutional neural network of the recommendation system and improve the accuracy of the recommendation model.

[0021] The de-smoothing module of the present invention can help graph nodes maintain their diversity and prevent the degradation of node feature diversity. Among them, the skip convolution operation can directly backpropagate the gradient, thereby preventing gradient disappearance and excessive attenuation of model weights.

[0022] The de-smoothing module of the present invention has strong generalization. This module can be applied to most graph convolutional neural network models and can improve GCN-based methods in various graph-based tasks.

[0023] The present invention designs two de-smoothing strategies: Uniform sampling strategy: Randomly select nodes to skip the convolution operation and directly output their input feature vectors. The uniform sampling strategy tends to randomly select nodes to skip the convolution operation. In addition, it can also be regarded as a data augmentation technique because it can generate different deformed input data for each intermediate layer. Biased sampling strategy: It selects nodes to skip the convolution operation based on the degree of the nodes. For the GCN model, the biased sampler will prevent the disappearance of the diversity of high-degree nodes to maintain the stability of model learning.

[0024] The advantages of the additional aspects of the present invention will be partially given in the following description, partially will become obvious from the following description, or will be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The specification drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0026] Figure 1 It is a flowchart of a recommendation method based on a de-smoothing graph convolutional neural network in Embodiment 1 of the present invention;

[0027] Figure 2 It is the construction structure diagram of the graph data in the first embodiment of the present invention;

[0028] Figure 3 It is the structure diagram of the de-smoothed graph convolutional neural network model in the first embodiment of the present invention;

[0029] Figure 4 It is the schematic diagram of de-smoothed convolutional node aggregation in the first embodiment of the present invention. Detailed implementation manners

[0030] It should be noted that the following detailed description is exemplary and is intended to provide further illustration of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0031] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary embodiments according to the present invention.

[0032] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0033] Embodiment 1

[0034] As Figures 1-4 shown, this embodiment discloses a recommendation method based on a de-smoothed graph convolutional neural network, including the following steps:

[0035] Step 1: Obtain the initial embedding vectors of user nodes and item nodes as training samples through the user-item interaction graph;

[0036] Step 2: Perform smooth convolutional aggregation operations on the training samples through a multi-layer graph convolutional neural network model based on a de-smoothed module to obtain user embedding vectors and item embedding vectors of each order;

[0037] Step 3: Combine the user embedding vectors and item embedding vectors of each order respectively, perform an inner product operation on the final user embedding vector and item embedding vector obtained after the layer combination, and obtain the prediction score of the training sample;

[0038] Step 4: Use BPR-Loss to train the multi-layer graph convolutional neural network model based on the de-smoothed module to obtain a trained recommendation model.

[0039] In this embodiment, the data of user and item interactions (such as clicks, browsing, or purchases) are used to construct a user-item interaction graph where represents the node set, including user nodes d = {1, …, N u}, and the item nodes where N u is the number of users, and N v is the number of items. ε represents the edge set. If user u d has an interaction record (click, browse, purchase, etc.) with item i v , then the edge (u d , i v ) ∈ ε. It means there is an undirected edge connecting node u d and node i v .

[0040] The adjacency matrix is defined as If user u d has an edge with item i v , then its element A dv = 1; otherwise, A dv = 0.

[0041] Then D is defined as the degree matrix, which is a diagonal matrix, and the elements on the diagonal are the number of non-zero elements in the corresponding row of the adjacency matrix A dv .

[0042] In step 1 of this embodiment, the entities (users and items) in the user-item interaction graph are mapped to vectors using one-hot encoding, which is expressed as:

[0043]

[0044] where h is the embedding size. Here and are the initial embedding vectors of the user node and the item node respectively. The number of users u is N, and the number of items i is M.

[0045] Specifically, for example, by simply encoding the user and item nodes in the user-item bipartite graph in Figure 2 , the initial embedding vectors of each node are obtained as follows:

[0046]

[0047] In step 2 of this embodiment, after obtaining the initial embedding vectors of the user nodes and the item nodes through step 1, based on the rules of the graph neural network message passing method, the nodes are aggregated through the de-smoothing module, which mainly includes the construction of message passing and the aggregation of node embeddings. This step shows the process of learning the first-order embedding vectors and the high-order embedding vectors extended from the first-order embeddings.

[0048] Among them, for the construction of message passing, for the connected user-item pair (u, i), we define the message passing from i to u as:

[0049] M u←i = f(e u , e i , d ui , P) (3)

[0050] Among them, M u←i represents the message passing from item i to user u, e u and e i are the embedded input vectors of the user and the item respectively. d ui is the attenuation coefficient that controls the propagation on (u, i). Here, d ui is set to the graph Laplacian norm: N u and N i represent the first-order neighbor sets of user u and item i respectively, that is, the set of nodes directly connected to this node. P is the mask matrix corresponding to the message passing process.

[0051] In this embodiment, f(·) is defined as:

[0052]

[0053] Among them, I is the identity matrix with the same dimension as matrix P.

[0054] In this embodiment, we provide the matrix form implemented by the de-smoothed module to facilitate the implementation and introduction of the model. Let the embedding matrix of the 0th layer be: Then the matrix operation of the de-smoothed module in the graph convolutional neural network is defined as follows:

[0055]

[0056] Among them, represents the input embedded vector of the lth layer, is the obtained output embedded vector of the (l + 1)th layer. By performing the de-smoothed convolution aggregation operation on the embedded vector of the lth layer node, the embedded vector of the (l + 1)th layer is obtained. is the symmetric normalized form after adding self-connections to the adjacency matrix A. Among them, D is the degree matrix of the adjacency matrix A, and I is the identity matrix with the same dimension.

[0057] For the corresponding mask matrix P, as Figure 4 shown, assuming a deep GCN contains l layers, for each de-smoothed graph convolutional layer, the de-smoothed module generates a mask matrix The generated mask matrix is a diagonal matrix, and its diagonal consists of ρ×N zeros and (1 - ρ)×N ones, where ρ is the sampling rate and N is the dimension of the corresponding mask matrix P (l) of P

[0058] P ii = 1 indicates that the node performs a standard graph convolution operation: P ii = 0 indicates that node i is a sampled node, that is, this node is selected to skip the convolution operation and directly output its embedding vector: Whether to skip the convolution for a node is selected through the mask matrix, which can represent the embedding vectors of users or items of each order, that is, it represents e (0) e (1) ...e (l) .

[0059] The mask matrix P (l) is determined by the sampling strategy and the sampling rate ρ. This embodiment proposes two sampling strategies: Uniform Sampling strategy: Select ρ×N nodes, and each node has the same probability of 1 / N. Biased Sampling strategy: Sample the "skipped nodes" according to the degree of the nodes. The sampling weight of v i is: where D ii is the degree of the current node, is the sum of the diagonal elements of the degree matrix. That is, the proportion of the degree of the current node among the degrees of all nodes. Biased sampling is adopted because nodes with higher degrees are more likely to be affected by over-smoothing in deep GCN. Therefore, the higher the degree of a node, the greater its sampling weight, and the more likely it is to be selected as a sampled node.

[0060] In this embodiment, the first-order embedding vector of a node is obtained by aggregating the initial embedding vectors of user and item nodes. The aggregation function is defined as:

[0061]

[0062] where respectively represent the embedding vectors of user u and item i after the first-order node message passing and aggregation.

[0063] In this embodiment, to improve the first-order embedding, the high-order embedding vector is obtained by iteratively performing the de-smoothing convolution operation multiple times, which is defined as follows:

[0064]

[0065] where They respectively represent the embedding vectors of user u and item i after the l-th order node message passing and aggregation.

[0066] In step 2 of this embodiment, by performing a de-smoothed convolutional aggregation operation on the graph nodes, the embedding vectors of users and items at each layer are obtained.

[0067] In step 3 of this embodiment, after passing through the l-th layer de-smoothed graph convolutional module, the l-th layer embedding vectors of users and items can be obtained: and

[0068] The embedding vectors obtained for each layer are combined to form the final user representation vector and the final item representation vector:

[0069]

[0070] where l represents the number of layers, and there are a total of L layers. They respectively represent the embedding vectors of the user and item of the finally obtained l-th layer. α l is the hyperparameter of the l-th layer. α l >0 represents the importance of the layer embedding in forming the final embedding representation, which is equivalent to an attention mechanism. To prevent the algorithm from becoming too complex, setting α l to 1 / (L + 1) usually brings good performance. Therefore, this embodiment does not design a special component to optimize α l , so as to avoid unnecessarily complicating the model.

[0071] The inner product operation of the final user embedding vector and the final item embedding vector obtained through the above steps is defined as follows:

[0072]

[0073] where the inner product score y u ′ i represents the probability that user u interacts with item i. y u ′ j represents the probability that user u interacts with item j.

[0074] In step 4 of this embodiment, we use the Bayesian Personalized Ranking (BPR) loss to update the model parameters. The Bayesian Personalized Ranking loss considers the relative order between the observed and unobserved user-item interactions, and encourages the prediction of the observed entries to be higher than their corresponding unobserved entries. The objective function is as follows:

[0075]

[0076] where O = {(u, i, j)|(u, i) ∈ R+ , where \((u, j) \in R\) -} is paired training data: among which, \(R\) + is the observed interaction, and \(R\) - is the unobserved interaction. \(\sigma\) is the activation function, set to sigmoid. \(\lambda \|E\) (0) \| 2 is the added L2 regularization term, where \(\lambda\) controls the strength of L2 regularization, and \(E\) (0) represents the trainable model parameters. The training parameters of this model are only the embeddings of the 0th layer, that is, \(\Theta=\{E\) (0) \}\). The Adam optimizer is used in mini-batch mode for the prediction algorithm and to update the algorithm parameters.

[0077] Example 2

[0078] The purpose of this example is to provide a computing device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above method are implemented.

[0079] Example 3

[0080] The purpose of this example is to provide a computer-readable storage medium.

[0081] A computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the above method are executed.

[0082] Example 4

[0083] The purpose of this example is to provide a recommendation system based on a de-smoothed graph convolutional neural network, including:

[0084] An acquisition module: obtaining the initial embedding vectors of user nodes and the initial embedding vectors of item nodes as training samples through a user-item interaction graph;

[0085] A convolutional aggregation module: performing a smoothed convolutional aggregation operation on the training samples through a multi-layer graph convolutional neural network model based on a de-smoothed module to obtain user embedding vectors and item embedding vectors of each order;

[0086] A calculation module: respectively performing layer combination on the obtained user embedding vectors and item embedding vectors of each order, and performing an inner product operation on the final user embedding vectors and item embedding vectors obtained after layer combination to obtain the prediction scores of the training samples;

[0087] A model training module: training a multi-layer graph convolutional neural network model based on a de-smoothed module using BPR-Loss to obtain a trained recommendation model.

[0088] In the devices of the foregoing Second, Third, and Fourth Embodiments, the steps involved correspond to those of the First Method Embodiment. For specific implementation manners, reference may be made to the relevant description part of the First Embodiment. The term "computer-readable storage medium" should be understood to include a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and cause the processor to execute any method in the present invention.

[0089] Those skilled in the art should understand that the foregoing modules or steps of the present invention can be implemented by a general computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device for execution by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0090] Although the specific implementation manners of the present invention have been described above in conjunction with the accompanying drawings, this is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present invention.

Claims

1. A recommendation method based on a de-smoothed graph convolutional neural network, characterized in that It includes the following steps: Step 1: Obtain the initial embedding vectors of user nodes and item nodes as training samples through the user-item interaction graph; Step 2: Perform a smoothed convolutional aggregation operation on the training samples through a multi-layer graph convolutional neural network model based on a de-smoothed module to obtain the user vector representations and item vector representations of the multi-layer graph convolutional neural network of each layer of the de-smoothed module; In Step 2, a de-smoothed module is added to each layer in the graph convolutional neural network model. The de-smoothed module multiplies the initial embedding vectors of user nodes and item nodes with a mask matrix respectively to select whether to activate the convolutional operation for the nodes; For each de-smoothed graph convolutional layer, the de-smoothed module generates a masking matrix , and the generated masking matrix is a diagonal matrix, the diagonal of which consists of zeros and ones, where is the sampling rate and N is the dimension of the corresponding masking matrix ; Indicates that the node performs a standard graph convolution operation: ; Indicates that node i is a sampled node, that is, this node is selected to skip the convolution operation and directly output its embedding vector: , and a mask matrix is used to select whether the node skips the convolution, can represent the embedding vectors of users or items of each order, that is, it represents , ... ; among them, represents the input embedding vector of the l-th layer, is the symmetric normalization form after adding self-connections to the adjacency matrix A of the user-item interaction graph, is the identity matrix; Step 3: Combine the user vector representations and item vector representations of each layer respectively, and perform operations based on the combined user vector representations and item vector representations to obtain the predicted scores of the training samples; Step 4: Use BPR-Loss to train the multi-layer graph convolutional neural network model based on the de-smoothed module to obtain a trained recommendation model.

2. The recommendation method based on the de-smoothed graph convolutional neural network according to claim 1, wherein In Step 1, use one-hot encoding to obtain the initial embedding vectors of user nodes and item nodes based on the user-item interaction graph.

3. The recommendation method based on a de-smoothed graph convolutional neural network according to claim 1, characterized in that The sampling strategy is a uniform sampling strategy or a biased sampling strategy. The uniform sampling strategy selects nodes, and each node has the same probability , being the sampling rate; the biased sampling strategy samples the skipped nodes.

4. The recommendation method based on a de-smoothed graph convolutional neural network according to claim 1, wherein, Construction of message passing in the multi-layer graph convolutional neural network model: For a user-item pair, define the message passing from item to user as: Among them, represents the message passing from item i to user u, and are the embedding input vectors of the user and the item, is the attenuation coefficient that controls the propagation on (u, i), and is set to the graph Laplacian norm: , and represent the first-order neighbor sets of user u and item i respectively, is the corresponding mask matrix.

5. The recommendation method based on a de-smoothed graph convolutional neural network according to claim 4, characterized in that, Obtain high-order embedding vectors through the de-smoothed module in the multi-layer graph convolutional neural network model, which is defined as follows: Among them, , respectively represent the embedding vectors of user u and item i after the l-th order node message passing and aggregation.

6. The recommendation method based on a de-smoothed graph convolutional neural network according to claim 1, characterized in that Train the recommendation model using Bayesian personalized ranking loss, and the objective function is: Among them, are paired training data, is the observed interaction, is the unobserved interaction, controls the strength of regularization.

7. A recommendation system based on a de-smoothed graph convolutional neural network, characterized in that It includes: Acquisition module: Obtain the initial embedding vectors of user nodes and item nodes as training samples through the user-item interaction graph; Convolutional aggregation module: Perform a de-smoothed convolutional aggregation operation on the training samples through a multi-layer graph convolutional neural network model based on a de-smoothed module to obtain user embedding vectors and item embedding vectors of each order; Specifically, a de-smoothed module is added to each layer in the graph convolutional neural network model. The de-smoothed module multiplies the initial embedding vectors of user nodes and item nodes with a mask matrix respectively to select whether to activate the convolutional operation for the nodes; For each de-smoothed graph convolutional layer, the de-smoothed module generates a mask matrix , and the generated mask matrix is a diagonal matrix, the diagonal of which consists of zeros and ones, where is the sampling rate and N is the dimension of the corresponding mask matrix ; Indicates that the node performs a standard graph convolution operation: ; Indicates that node i is a sampled node, that is, this node is selected to skip the convolution operation and directly output its embedding vector: , and a mask matrix is used to select whether the node skips the convolution, which can represent the embedding vectors of users or items of each order, that is, it represents , ... ; among them, represents the input embedding vector of the l-th layer, is the symmetric normalization form after adding self-connections to the adjacency matrix A, is the identity matrix; Calculation module: Combine the user embedding vectors and item embedding vectors of each order obtained respectively, and perform an inner product operation on the final user embedding vectors and item embedding vectors obtained after combination to obtain the predicted scores of the training samples; Model training module: Use BPR-Loss to train the multi-layer graph convolutional neural network model based on the de-smoothed module to obtain a trained recommendation model.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps in a recommendation method based on a de-smoothed graph convolutional neural network as described in any one of claims 1-6.

9. A processing device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in a recommendation method based on a de-smoothed graph convolutional neural network as described in any one of claims 1-6.

Images