A pruning method and device for a graph neural network recommendation model and a storage medium

Abstract

The application discloses a kind of prunning methods, devices and storage medium for graph neural network recommendation model, it is related to model prunning and computer deep learning field.Given user set article interaction information, the user-article bipartite graph constructed, prunning algorithm will calculate edge reliability, structure importance and information quality in the process of model training, to produce the score of each edge in user-article bipartite graph, and it is pruned, to reduce the scale of graph, to accelerate model training and improve the accuracy of model prediction, and the recommendation model based on pruned graph neural network is recommended for user article.The application can delete the edge of smaller contribution in the training process of recommendation model based on graph neural network, optimize graph structure, make information transmission more effective while speed up the speed of aggregation.

Description

Technical Field

[0001] This invention relates to the fields of model pruning and computer deep learning, and particularly to a pruning method, apparatus and storage medium for graph neural network recommendation models. Background Technology

[0002] Graph neural networks have shown significant effectiveness in recommender systems due to their ability to capture high-order collaborative signals. Existing recommendation models based on graph neural networks typically include the following steps: 1) constructing a bipartite graph based on user-item interactions; 2) applying a multi-layer graph neural network to the constructed graph to learn user and item embedding vectors; 3) generating predictions based on the similarity of the embedding vectors.

[0003] While graph neural networks (GNNs) have demonstrated effectiveness in recommender systems, they also introduce significant computational burdens. Domain aggregation, a core component, requires repeated computation of information propagation among numerous neighboring nodes in each batch, becoming a major efficiency bottleneck. For example, the classic LightGCN architecture saw a six-fold increase in computational demands due to the introduction of GNNs. This is primarily because parallelizing GNNs on large graphs is difficult. These issues hinder the practical deployment of GNN-based recommender systems.

[0004] To overcome this efficiency limitation, several strategies have been proposed: some recent works suggest using simplified modules to approximate graph neural networks. While efficient, these methods are limited by specific design architectures and perform worse than recent state-of-the-art graph neural network-based techniques. Another strategy is to use knowledge distillation to transfer knowledge from graph neural network-based methods to non-graph neural network methods. However, this approach only accelerates the inference process and still results in extended training time due to the need to train a graph neural network-based teacher model. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a pruning method, apparatus, and storage medium for graph neural network recommendation models. Pruning is performed during the training process of the recommendation model, thereby improving the efficiency of model training and the accuracy of model prediction, and achieving more efficient and accurate user item recommendations.

[0006] The objective of this invention is achieved through the following technical solution: Firstly, this invention provides a pruning method for graph neural network recommendation models, the method comprising the following steps:

[0007] (1) Obtain the user set, the item set, and the user's interest information on the items, and construct a user-item bipartite graph with users and items as nodes and interest information as edges;

[0008] (2) Calculate the structural importance of edges in the user-item bipartite graph based on the data of interest;

[0009] (3) Randomly initialize the embedding vectors of users and items, and set the pruning rate and number of iterations.

[0010] (4) Train a recommendation model based on graph neural networks and update the embedding vectors of users and items;

[0011] (5) Based on the updated embedding vectors, the final embedding vectors of users and items are obtained by merging; the reliability and information quality of the edges in the user-item bipartite graph are calculated.

[0012] (6) Based on the structural importance, reliability and information quality of the edges in the user-item bipartite graph, delete the edges in the user-item bipartite graph according to the set pruning rate, update the user-item bipartite graph, and complete one iteration;

[0013] (7) Repeat steps (4)-(6) for iterative updates until the number of iterations is reached, and obtain the user-item bipartite graph and the embedding vectors of users and items after iteration, which are used to recommend items to users based on the recommendation model of the pruned graph neural network.

[0014] Furthermore, the structural importance of edge (u,i) in the user-item bipartite graph. The calculation method is as follows:

[0015] 1) Construct the adjacency matrix

[0016] 2) Calculate A 2 The diagonal matrix Λ;

[0017] 3) Define the structural importance matrix B = A 3 -AΛ-ΛA+A;

[0018] 4) Define B ui This represents the value in the u-th row and i-th column of the structural importance matrix B;

[0019] 5) Structural importance S I (u,i)=B ui ;

[0020] 6) Calculate the structural importance of all edges in edge set E to obtain set S. I ={S I (u,i)|(u,i)∈E};

[0021] 7) Calculate the structural importance S of edge (u,i) I (u,i) and normalize it:

[0022] Furthermore, the user's final embedding vector e u The final embedding vector e of the item i The calculation method is as follows:

[0023] 1) Calculate the embedding vectors of users and items after passing through the l-th layer of the graph neural network: N u Let d represent the set of neighbors of user u. u For the degree of user u, N i d represents the set of neighbors of item i; i Let i be the degree of item i;

[0024] 2) The final embedding vectors for users and items are defined as follows: L represents the number of layers in the graph neural network.

[0025] Furthermore, the edge reliability S of edge (u,i) in the user-item bipartite graph R The method for calculating (u,i) is as follows: calculate the co-similarity of the embedding vectors of the two nodes connected by edge (u,i), i.e., the edge reliability.

[0026] Furthermore, the information quality of the edge (u,i) in the user-item bipartite graph. The calculation method is as follows:

[0027] 1) Define the information quality from user u to item i Information quality from item i to user u

[0028] 2) Define the information quality of edge (u,i)

[0029] 3) Calculate the information quality of all edges in edge set E to obtain set S. Q ={S Q (u,i)|(u,i)∈E};

[0030] 4) Calculate the information quality S of edge (u,i) Q (u,i) and normalize it:

[0031] Furthermore, the score of each edge in the edge set E of the user-item bipartite graph is calculated. Where the weight exponents α1 and α2 are hyperparameters, S R (u,i) represents the edge reliability between user u and item i. To determine the structural importance of edges after normalization, To improve the information quality of the normalized edges, the edges with the lowest scores [|E|*p%] are deleted according to the pruning rate to obtain a new edge set E′, and the user-item bipartite graph is updated.

[0032] Secondly, the present invention also provides a pruning device for a graph neural network recommendation model, comprising a memory and one or more processors, wherein the memory stores executable code, and wherein the processor, when executing the executable code, implements the pruning method for a graph neural network recommendation model.

[0033] Thirdly, the present invention also provides a computer-readable storage medium having a program stored thereon, characterized in that, when the program is executed by a processor, it implements the aforementioned pruning method for a graph neural network recommendation model.

[0034] Fourthly, the present invention also provides a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the aforementioned pruning method for a graph neural network recommendation model.

[0035] The beneficial effects of this invention are as follows: According to the method of this invention, edges with smaller contributions can be removed during the training process of a recommendation model based on a graph neural network, the graph structure can be optimized, information transmission can be made more effective, and the aggregation speed can be accelerated, thereby achieving more efficient and accurate user item recommendations. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of a pruning method for a graph neural network recommendation model provided by the present invention;

[0038] Figure 2 This is a structural diagram of a pruning device for a graph neural network recommendation model provided by the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are merely illustrative and not intended to limit the invention.

[0040] To illustrate the specific implementation of this invention in detail, the Amazon-book dataset will be used as an example. This dataset contains 52,643 users, 91,599 items (in this invention, items are books on the Amazon platform), and 2,984,108 interaction records between users and books on the platform, such as user clicks on books of interest. Figure 1 As shown, the present invention provides a method comprising the following steps:

[0041] 1. Given a user set U, an item set I, and a click-through rate matrix. Based on actual needs or experience, set the pruning ratio p% = 50%, the number of iterations T = 1000, where the user set U contains 52643 elements, the item set I contains 91599 items, and the click-through rate matrix R contains 2984108 interaction records.

[0042] 2. Construct a vertex set V = U∪I containing the user set and the item set, and an edge set E = {(u,i)|u∈U,i∈I,R} containing the click relationships between users and items. ui =1}, and the user-item bipartite graph G(V,E), R ui This is the value in the u-th row and i-th column of the click-through rate matrix R. A value of 1 indicates that user u clicked on item i, and a value of 0 indicates that user u did not click on item i. For user u ∈ U, the set of neighbors of user u is defined as N. u ={i|R ui =1}, the degree d of user u u =|N u |;For item i∈I, define the neighbor set of item i as N. i ={u|R ui =1}, the degree d of item i i =|N i |;

[0043] 3. Calculate the structural importance of each edge (u,i) in set E.

[0044] The method for calculating structural importance is as follows: first construct an adjacency matrix. Calculate A 2 The diagonal matrix Λ is then used to calculate the structural importance matrix B = A. 3 -AΛ-ΛA+A, yielding the structural importance S. I (u,i)=B ui B ui Let S be the value in the u-th row and i-th column of the structural importance matrix B, representing the number of quadrilaterals that edge (u,i) can form with other edges in the user-item bipartite graph G(V,E). We need to calculate the structural importance of all edges in edge set E to obtain set S. I ={SI (u,i)|(u,i)∈E}, then the structural importance S of edge (u,i) I (u,i) is normalized:

[0045] 4. The embedding vector of user u is represented as follows: The embedding vector of item i is represented as Randomly initialize the embedding vectors for all users and items, with each embedding vector having a size of 64;

[0046] 5. Set the size of each batch to 2048, and one training cycle contains... For each batch, a recommendation model based on a graph neural network is trained once using the user-item bipartite graph G(V,E), and the user embedding vectors are updated. Embedding vectors of items

[0047] In an embodiment of the present invention, the LightGCN model is used to train and update the embedding vector of user u. Embedding vector of item i The LightGCN described uses a 3-layer network. Specifically, the LightGCN model is used to train and update the user's embedding vectors. Embedding vectors of items The prior art in this field will not be described in detail here.

[0048] 6. Based on the updated user embedding vector and item embedding vector, calculate the user's final embedding vector e. u The final embedding vector e of the item i ;

[0049] The calculation method is as follows: First, calculate the embedding vectors of users and items after passing through different layers of the graph neural network: Next, calculate the final embedding vectors for users and items:

[0050] 7. Based on the user's final embedding vector e u The final embedding vector e of the item i Calculate the edge reliability S for each edge (u,i) in set E. R (u,i);

[0051] The edge reliability is defined as the co-similarity of the final embedding vectors of the two nodes connected by edge (u,i), calculated using the following formula:

[0052]

[0053] 8. Based on the user's final embedding vector e u The final embedding vector e of the item i Calculate the information quality of each edge (u,i) in set E.

[0054] Information quality The calculation method is as follows: based on the information quality from user u to item i Information quality from item i to user u Calculate the information quality of all edges in E Get set S Q ={S Q (u,i)|(u,i)∈E}; then the information quality S of edge (u,i) Q (u,i) is normalized:

[0055] 9. Calculate the score based on the formula for edge (u,i). The weight indices α1 and α2 are hyperparameters. Based on experience, α1 = 0.75 and α2 = 0.25 are set to calculate the score of each edge in set E.

[0056] 10. Delete the edges with the lowest scores of [|E|*p%], which in this invention are [2984108*50%] edges, to obtain a new edge set E′, and update the user-item bipartite graph G(V,E′);

[0057] 11. Iterate through steps 5-10 1000 times. Finally, use the user-item bipartite graph G(V,E′) and the embedding vectors of users and items after 1000 iterations for prediction in the LightGCN model to achieve item recommendation for users.

[0058] 12. Test Results: The LightGCN model after applying the method of this invention was tested on the test set of the Amazon-book dataset. Compared with the original LightGCN model, the accuracy was improved by 7.7% and the training speed was improved by 88%.

[0059] Corresponding to the aforementioned embodiment of a pruning method for graph neural network recommendation models, the present invention also provides an embodiment of a pruning device for graph neural network recommendation models.

[0060] See Figure 2The present invention provides a pruning device for a graph neural network recommendation model, comprising a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it implements a pruning method for a graph neural network recommendation model as described in the above embodiment.

[0061] The pruning device for graph neural network recommendation models provided by this invention can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 2 The diagram shown illustrates a hardware structure of any data processing-capable device, including the pruning device for a graph neural network recommendation model provided by this invention. Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0062] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0063] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0064] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a pruning method for a graph neural network recommendation model as described in the above embodiments.

[0065] The computer-readable storage medium can be an internal storage unit of any data processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0066] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned pruning method for a graph neural network recommendation model.

[0067] The above embodiments are used to explain and illustrate the present invention, but not to limit the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims shall fall within the protection scope of the present invention.

Claims

1. A pruning method for recommendation models of graph neural networks, characterized in that, The method includes the following steps: (1) Obtain the user set, the item set, and the user's interest information on the item, and construct a user-item bipartite graph with user A and item B as nodes and interest information as edges; (2) Calculate the structural importance of edges in the user-item bipartite graph based on the data of interest; edges in the user-item bipartite graph Structural importance The calculation method is as follows: 1) Construct the adjacency matrix ; 2) Calculation diagonal matrix ; 3) Define the structural importance matrix ; 4) Definition Structural importance matrix The value in row 0 and column 0; 5) Structural importance ; 6) Calculate edge sets The structural importance of all edges in the set is obtained. ; 7) Calculate the edges Structural importance And perform normalization: ; (3) Randomly initialize the embedding vectors of users and items, and set the pruning rate and number of iterations. (4) Train a recommendation model based on graph neural networks and update the embedding vectors of users and items; (5) Based on the updated embedding vectors, the final embedding vectors of the users are obtained by merging them. The final embedding vector of the item ; Calculate the reliability and information quality of edges in a user-item bipartite graph; The edge of the user-item bipartite graph Edge reliability The calculation method is as follows: calculate the edge The cosine similarity of the embedding vectors of the two connected nodes, i.e., the edge reliability. ; The edge of the user-item bipartite graph Information quality The calculation method is as follows: 1) Define user to items Information quality ,thing To users Information quality ; For users The embedding vector representation, For users The degree, For items The embedding vector representation, For items The degree; 2) Define edges Information quality ; 3) Calculate the edge set The information quality of all edges in the set is obtained. ; 4) Calculate the edges Information quality And perform normalization: ; (6) Based on the structural importance, reliability and information quality of the edges in the user-item bipartite graph, delete the edges in the user-item bipartite graph according to the set pruning rate, update the user-item bipartite graph, and complete one iteration; (7) Repeat steps (4)-(6) for iterative updates until the number of iterations is reached, and obtain the user-item bipartite graph and the embedding vectors of users and items after iteration, which are used to recommend items to users based on the recommendation model of the pruned graph neural network.

2. The pruning method for a graph neural network recommendation model according to claim 1, characterized in that, The user's final embedding vector The final embedding vector of the item The calculation method is as follows: 1) Calculate after the first... User and item embedding vectors from a layered graph neural network: , ; Indicates user The neighborhood group, For users The degree, Represents items The set of neighbors; For items The degree; 2) The final embedding vectors for users and items are defined as follows: , L represents the number of layers in the graph neural network.

3. The pruning method for a graph neural network recommendation model according to claim 1, characterized in that, Calculate the score of each edge in the edge set E of the user-item bipartite graph. Among them, the weight index For hyperparameters, For the edge reliability between user u and item i, To determine the structural importance of edges after normalization, To determine the information quality of the normalized edges; remove the edges with the lowest scores based on the pruning rate. By adding edges, we obtain a new edge set. Update user-item bipartite graph.

4. A pruning device for a graph neural network recommendation model, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the processor executes the executable code, it implements a pruning method for a graph neural network recommendation model as described in any one of claims 1-3.

5. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements a pruning method for a graph neural network recommendation model as described in any one of claims 1-3.

6. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements a pruning method for a graph neural network recommendation model as described in any one of claims 1-3.

Images