A decentralized federated graph neural network recommendation method for local bipartite graphs

Abstract

The present invention proposes a decentralized federated graph neural network recommendation method for a local bipartite graph. The implementation steps are as follows: building a decentralized federated recommendation system; clustering clients based on communication distance; constructing a local bipartite graph for each client in the cluster; obtaining a training data set and a test data set; the client initializes a local federated recommendation model and iteratively trains it; and obtaining the user's recommendation results for items. In the present invention, the client negotiates with the cluster head and the global aggregation node to be responsible for aggregating model parameters within and between clusters. If the cluster head or global aggregation node goes offline, a new election can be performed to prevent model training failure. A privacy-preserving intersection method is implemented between the clients to obtain the intersection set of items with the clients in the cluster. A connection with a client vertex on the left and an item vertex on the right is added to the original connection graph with client vertices on the left and item vertices interacting with the client on the right, making the local bipartite graph denser and obtaining higher recommendation accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of federated recommendation technology and relates to a decentralized federated graph neural network recommendation method for a local bipartite graph. Background Art

[0002] Recommender systems, as data-driven applications, centrally train recommendation models by collecting user personal information and interaction records (browsing, ratings, etc.), capturing user interests and preferences, and generating recommendations for them. User-item interaction records form a bipartite graph structure. Graph neural networks can effectively capture high-order similarities between user and item nodes in the user-item bipartite graph for model training. They map pre-existing features describing users (or items), such as IDs and attributes, to obtain user (or item) embeddings for training recommendation models. However, training data often exists in multiple data sources and contains sensitive personal information. Protecting data privacy has become a global consensus and trend, with numerous regulations, such as the EU's General Data Protection Regulation (GDPR), being introduced.

[0003] For these reasons, McMahan et al. proposed the concept of federated learning in 2017, balancing efficiency and security. Today, combining graph neural networks with federated learning frameworks to train high-quality centralized recommendation models based on decentralized user data is becoming increasingly common. However, researchers have found that a completely neutral, "trusted" central server is difficult to identify, and the server's need to communicate with all nodes creates a high communication bottleneck and makes performance difficult to guarantee. Therefore, in federated recommendation systems, improving the accuracy of recommendation models and minimizing the risk of privacy leaks are pressing challenges for achieving efficient federated recommendation systems.

[0004] For example, the patent application with application publication number CN 113420232 A and titled “A Method for Federated Recommendation of Graph Neural Networks for Privacy Protection” discloses a graph recommendation training method based on federated learning. The main steps of the method are: (1) the server maintains a global item table; initializes the global weight and item embedding matrix and distributes them to the clients participating in the training; (2) after each client obtains the initialized global weight and item embedding matrix, it uses local data in parallel to train the recommendation model based on the graph neural network; (3) after each client completes a round of local training, it homomorphically encrypts the local gradient and item embedding matrix and uploads it to the central server; (4) after receiving the uploaded homomorphically encrypted local gradient and item embedding matrix, the server aggregates them using the FedAvg algorithm to generate a new global weight, updates the weighted average weight and global item embedding, and distributes them to each client; (5) after each client receives the aggregated global weight and global item embedding matrix, it decrypts and reassigns them, and then trains the local recommendation system model in parallel; (6) steps (3) to (5) are repeated until the training ends at the preset threshold, and each client outputs its own final prediction result. The shortcomings of this method are: since the client only holds a local bipartite graph of itself and the items it has interacted with, the graph is too sparse, resulting in low recommendation accuracy for user training; in addition, the central server will instantly receive thousands of model parameters trained by the client during model aggregation. Once the central server fails, the entire training process will be forced to terminate, leading to training failure and other problems. Summary of the Invention

[0005] The purpose of the present invention is to address the shortcomings of the above-mentioned technologies and propose a decentralized federated graph neural network recommendation method for local bipartite graphs, which is used to solve the technical problems in the existing technology of low recommendation accuracy due to the excessive sparseness of local bipartite graphs and poor recommendation reliability due to the existence of a central server.

[0006] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0007] (1) Build a decentralized federated recommendation system:

[0008] The build includes M clients and N items to be rated decentralized federated recommendation system, each client u m Hold Z m Items to be rated and corresponding labels Among them, M≥100, u m Indicates the mth client, N≥100, t n Indicates the nth item to be rated, Z m≥50,t m,z Indicates u m The zth item to be rated, y m,z Indicates t m,z Corresponding labels;

[0009] (2) Clustering clients based on communication distance:

[0010] Based on the communication distance, M clients Divided into S clusters Each cluster I s Clients negotiate to elect the cluster head node τ s , S clusters The cluster head nodes negotiate to elect the global aggregation node τ; where S≥5, represents the sth cluster, I s express The total number of clients included in I s ≥10, c s,i express The i-th client in ;

[0011] (3) Construct a local bipartite graph of each client in the cluster:

[0012] For each cluster Every two clients c s,i 、c s,j Collection of items waiting to be rated Perform privacy protection intersection and get client c s,i with c s,j The intersection of common items And construct each cluster Each client c s,i 、c s,i Holding Z m Items to be rated t s,i,z They are the left vertex and the right vertex respectively, with c s,i With t s,i,z The connection line and the c s,j and The c of each item's edge s,i Local bipartite graph G s,i ;

[0013] (4) Obtain training and test datasets:

[0014] Each client c s,i Get the client c s,i ID, rating item t s,i,z ID, t s,i,z Label ys,i,z and the local bipartite graph G s,i As c s,i Z s,i Scoring data, and then more than half of all scoring data form the training data set The remaining scoring data is used to form the test data set

[0015] (5) The client initializes the local federated recommendation model W s,i , and iteratively train it:

[0016] (5a) Each client c s,i Initialize the local federated recommendation model W consisting of a sequentially connected embedding layer, K stacked lightweight graph convolutional layers LGC, a merging layer, and F stacked fully connected layers. s,i , the inter-cluster aggregation round is l, the maximum inter-cluster aggregation round is L, L ≥ 15, the intra-cluster aggregation round is α, the maximum intra-cluster aggregation round is P, P ≥ 5, the local federated recommendation model of the lth inter-cluster aggregation and the αth intra-cluster aggregation The model parameters are And let l = 0;

[0017] (5b) Let α = 0;

[0018] (5c) The training dataset B training data are randomly selected with replacement as each client c s,i Local federated recommendation model W s,i The embedding layer converts the client c s,i ID, rating item t s,i,z The ID is converted into a d-dimensional user embedding vector Item embedding vector K LGC layers are based on the local bipartite graph G s,i right and Perform convolution aggregation; the merging layer embeds the user vector after convolution aggregation and item embedding vectors Perform feature fusion; F stacked fully connected layers perform feature fusion on client u according to the embedding vector of feature fusion s,i Item t s,i,b Perform rating prediction and get the predicted rating

[0019] (5d) Using the predicted score labels and its corresponding true rating label y s,i,b Computing local recommendation models The MSE loss value And using the stochastic gradient descent algorithm, by Model parameters Update to obtain the model parameters of the αth intra-cluster training in the lth inter-cluster aggregation And upload to the cluster head τ s ;

[0020] (5e) Cluster head τ s Model parameters Aggregate to get model parameters Distribute to clusters Internal client c s,i , and judge whether α=P is established. If so, get the cluster The federated recommendation model with the lth inter-cluster aggregation and the Pth intra-cluster aggregation Model parameters Cluster head τ s Model parameters that will be aggregated After uploading to the global aggregation node τ, the user c in the cluster s,i with c s,j Transfer user embedding vectors to each other and and execute step (5f); otherwise, set α = α + 1 and execute step (5c);

[0021] (5f) Global aggregation node τ for model parameters Perform aggregation and set the aggregation model parameter ω l Distributed to each client c s,i Then, we get the global l-th inter-cluster aggregation federated recommendation model W l The model parameter ω l , judge whether l=L is true, if so, get the trained federated recommendation model W l , otherwise l=l+1, execute step (5b);

[0022] (6) Obtain the user's recommendation results for items:

[0023] The test dataset As a trained local recommendation model The input is forward propagated to obtain the test data set Client u s,i Item t s,i,o Prediction score of

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] 1. This invention performs a privacy-preserving intersection on the sets of to-be-rated items held by every two clients in each cluster, and constructs a local bipartite graph with the multiple to-be-rated items held by each client in each cluster as left and right vertices, respectively, and the lines connecting one client and the to-be-rated items, and the lines connecting another client and each item in the intersection result as edges. This avoids the sparseness of the existing local bipartite graphs, which only use the lines connecting one client and the to-be-rated items as edges, and effectively improves the accuracy of recommendations.

[0026] 2. In the present invention, the client negotiates and elects the cluster head node and the global aggregation node to be responsible for aggregating the model parameters within and between clusters. Once the cluster head or the global aggregation node goes offline, the election can be re-performed, avoiding the defect of the existing technology that the central server training fails due to the large number of clients, and effectively improving the reliability of the recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart for implementing the present invention. DETAILED DESCRIPTION

[0028] The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0029] Reference Figure 1 , the present invention comprises the following steps:

[0030] Step 1) Build a decentralized federated recommendation system:

[0031] The build includes M clients and N items to be rated decentralized federated recommendation system, each client u m Hold Z m Items to be rated and corresponding labels Among them, M≥100, u m Indicates the mth client, N≥100, t n Indicates the nth item to be rated, Z m ≥50,t m,z Indicates u m The zth item to be rated, y m,z Indicates t m,z Corresponding labels;

[0032] In this embodiment, M=943, N=1682, Z m The value is 100.

[0033] Step 2) Cluster the clients based on communication distance:

[0034] Based on the communication distance, M clients Divided into S clusters Each cluster I s Clients negotiate to elect the cluster head node τ s , S clusters The cluster head nodes negotiate to elect the global aggregation node τ; where S≥5, represents the sth cluster, I s express The total number of clients included in I s ≥10, c s,i express The i-th client in ;

[0035] In this embodiment, M clients are grouped based on the communication distance. Divided into S clusters The k-means prototype clustering algorithm is used, and the specific implementation process is as follows:

[0036] Step 2a) All clients are D2D users, capable of sending and receiving signals and having automatic routing (message forwarding) capabilities;

[0037] Step 2b) Store all D2D users in the collection In , S D2D users are randomly selected, where S is the number of clusters to be divided, and the D2D user location coordinates are used as the initial mean vector;

[0038] Step 2c) Determine the user set of the sth D2D cluster For the mth D2D user u m , calculate its distance to S mean vectors respectively;

[0039] Step 2d) Select user u m The nearest mean vector is used to determine the cluster label corresponding to the mean vector, and the corresponding value is used as the cluster number it selects. m into this cluster;

[0040] Step 2e) For each cluster, calculate a new mean vector and determine whether the new mean vector is equal to the mean vector of the current cluster. If not, replace the original mean vector with the new mean vector. If they are equal, the value of the original mean vector remains unchanged.

[0041] Step 2f) Repeat the above steps 2b) to 2e) until the mean vectors of all clusters are equal to the mean vectors calculated in the previous round, and the clustering result is obtained; in this embodiment, the number of clusters S is set to 5, and the total number of clients in each cluster is set to I s The value of is 300;

[0042] Step 3) Construct a local bipartite graph for each client in the cluster:

[0043] For each cluster Every two clients c s,i 、c s,j Collection of items waiting to be rated Execute the privacy-preserving intersection method to obtain client c s,i with c s,j The intersection of common items And construct each cluster Each client c s,i 、c s,i Holding Z m Items to be rated t s,i,z They are the left vertex and the right vertex respectively, with c s,i With t s,i,z The connection line and the c s,j and The c of each item's edge s,i Local bipartite graph G s,i ;

[0044] In this embodiment, for each cluster Every two clients c s,i 、c s,j Collection of items waiting to be rated The privacy-preserving intersection method may be implemented by using a privacy-preserving intersection method based on RSA blind signatures, a privacy-preserving intersection method based on OT protocols, or a privacy-preserving intersection method based on homomorphic encryption. In this embodiment, the privacy-preserving intersection method based on RSA blind signatures is adopted because it has higher privacy. The specific implementation process is as follows:

[0045] Step 3a)c s,i Generate two large prime numbers p and q, Choose any a that satisfies a and (p-1)*(q-1) are relatively prime, choose r that satisfies (a*r)mod((p-1)*(q-1))=1, and send a, To client c s,j ;

[0046] Step 3b)c s,i with c s,j Negotiate two hash functions With H2:{0,1} * →{0,1} θ , where θ is the safety parameter, It's a model The integer domain of ;

[0047] Step 3c) s,i For each item ts,i,z ID, calculate And further calculate the hash value k s,i,z =H2(t s ' ,i,z ), get c s,i A hash result set of all encrypted IDs held

[0048]

[0049] Step 3d)c s,j For each item t s,j,z ID, calculate in To client c s,i ;

[0050] Step 3e)c s,i calculate Get Collection send Collection and c s,i hold Gather to c s,j ;

[0051] Step 3f)c s,j calculate Get Collection Calculate intersection and send Gather to c s,i ;

[0052] Step 3g)c s,i Calculate the intersection item ID set

[0053] Step 4) Get the training and test datasets:

[0054] Each client c s,i Get the client c s,i ID, rating item t s,i,z ID, t s,i,z Label y s,i,z and the local bipartite graph G s,i As c s,i Z s,i Then more than half of all the scoring data are used to form the training data set The remaining scoring data is used to form the test data set

[0055] The dataset used in this embodiment is the MovieLen 100K user and item recommendation dataset. In this dataset, the number of users is M = 943, the number of items is N = 1682, and the number of user ratings for items is 100,000. In this embodiment, the Z held by each client in the cluster is set to s,i There are 100 scoring data.

[0056] Step 5) The client initializes the local federated recommendation model W s,i , and iteratively train it:

[0057] Step 5a) Each client c s,i Initialize the local federated recommendation model W consisting of a sequentially connected embedding layer, K stacked lightweight graph convolutional layers LGC, a merging layer, and F stacked fully connected layers. s,i , the inter-cluster aggregation round is l, the maximum inter-cluster aggregation round is L, L ≥ 15, the intra-cluster aggregation round is α, the maximum intra-cluster aggregation round is P, P ≥ 5, the local federated recommendation model of the lth inter-cluster aggregation and the αth intra-cluster aggregation The model parameters are And let l = 0;

[0058] In this embodiment, the local recommendation model includes K=3 lightweight graph convolutional layers LGC and F=3 fully connected layers. The input of the first fully connected layer is 128 and the output is 64. The input of the second fully connected layer is 64 and the output is 32. The input of the third fully connected layer is 32 and the output is 1. The maximum intra-cluster aggregation round is P=5, and the maximum inter-cluster aggregation round is L=25.

[0059] Step 5b) set α=0;

[0060] Step 5c) The training dataset B training data are randomly selected with replacement as each client c s,i Local federated recommendation model W s,i The embedding layer converts the client c s,i ID, rating item t s,i,z The ID is converted into a d-dimensional user embedding vector Item embedding vector K LGC layers are based on the local bipartite graph G s,i right and Perform convolution aggregation; the merging layer embeds the user vector after convolution aggregation and item embedding vectors Perform feature fusion; F stacked fully connected layers perform feature fusion on client c according to the embedding vector of feature fusion s,i Item t s,i,bThe score of

[0061]

[0062] In this embodiment, B=32, the embedding layer converts the client ID and the rated item ID into a 64-dimensional user embedding vector and an item embedding vector; the first LGC layer is based on the local bipartite graph G s,i For 64-dimensional and Perform convolution aggregation, the aggregation formula is:

[0063]

[0064]

[0065] in, Represents client c s,i In a local bipartite graph G s,i Every item connected in Is client c s,i In a local bipartite graph G s,i The number of connected items in It is item t s,i,b In a local bipartite graph G s,i The number of connected clients, Indicates item t s,i,b In a local bipartite graph G s,i Each user in the cluster connected to Is client c s,g In a local bipartite graph G s,i The number of items connected in the , after aggregation, the 64-dimensional aggregated user embedding vector is obtained and item embedding vectors Similarly, the second and third LGC layers are based on the local bipartite graph G s,i For 64-dimensional and Convolution aggregation is performed again to obtain 64 dimensions and With 64-dimensional and The merging layer performs the sum operation on the convolutional user embedding vector and the item embedding vector to obtain and Concatenating user embedding vectors Item embedding vector Get a 128-dimensional vector, and perform client c on the 128-dimensional embedding vector of feature fusion through three fully connected layers s,i Item t s,i,b The ratings are predicted.

[0066] Step 5d) Use the predicted score labels and its corresponding true rating label y s,i,b Computing local recommendation models The MSE loss value And using the stochastic gradient descent algorithm, by Model parameters Update to obtain the model parameters of the αth intra-cluster training in the lth inter-cluster aggregation And upload to the cluster head τ s ;

[0067] cluster Local recommendation model in The MSE loss value and model parameters The update formulas are:

[0068]

[0069]

[0070] Where ∑ represents a summation operation, and η>0 represents a learning rate. In this example, η=0.01.

[0071] Step 5e) Cluster head τ s Model parameters Aggregate to get model parameters Distribute to clusters Internal client c s,i , and judge whether α=P is established. If so, get the cluster The federated recommendation model with the lth inter-cluster aggregation and the Pth intra-cluster aggregation Model parameters Cluster head τ s Model parameters that will be aggregated After uploading to the global aggregation node τ, the user c in the cluster s,i with c s,j Transfer user embedding vectors to each other and and execute step (5f); otherwise, set α = α + 1 and execute step (5c);

[0072] Cluster head node τ s Model parameters The formula for aggregation is as follows:

[0073]

[0074] In this embodiment, S=5, I s The value is 300.

[0075] Step 5f) Global aggregation node τ for model parameters Perform aggregation and set the aggregation model parameter ω l Distributed to each client c s,i Then, we get the global l-th inter-cluster aggregation federated recommendation model W l The model parameter ω l , judge whether l=L is true, if so, get the trained federated recommendation model W l , otherwise l=l+1, execute step (5b);

[0076] Global aggregation node τ on model parameters The formula for aggregation is as follows:

[0077]

[0078] Step 6) Get the user's recommendation results for the item:

[0079] The test dataset As a trained local recommendation model The input is forward propagated to obtain the test data set Client u s,i Item t s,i,o Prediction score of

Claims

1. A decentralized federated graph neural network recommendation method for local bipartite graphs, characterized in that: The steps include: (1) Build a decentralized federated recommendation system: The build includes M clients and N items to be rated decentralized federated recommendation system, each client u m Hold Z m Items to be rated and corresponding labels Among them, M≥100, u m Indicates the mth client, N≥100, t n Indicates the nth item to be rated, Z m ≥50,t m,z Indicates u m The zth item to be rated, y m,z Indicates t m,z Corresponding labels; (2) Clustering clients based on communication distance: Based on the communication distance, M clients Divided into S clusters Each cluster I s Clients negotiate to elect the cluster head node τ s , S clusters The cluster head nodes negotiate to elect the global aggregation node τ; where S≥5, represents the sth cluster, I s express The total number of clients included in I s ≥10, c s,i express The i-th client in ; (3) Construct a local bipartite graph of each client in the cluster: For each cluster Every two clients c s,i 、c s,j Collection of items waiting to be rated Perform privacy protection intersection and get client c s,i with c s,j The intersection of common items And construct each cluster Each client c s,i 、c s,i Holding Z m Items to be rated t s,i,z They are the left vertex and the right vertex respectively, with c s,i With t s,i,z The connection line and the c s,j and The line connecting each item is the edge c s,i Local bipartite graph G s,i ; (4) Obtain training and test datasets: Each client c s,i Get the client c s,i ID, rating item t s,i,z ID, t s,i,z Label y s,i,z and the local bipartite graph G s,i As c s,i Z s,i Then more than half of all the scoring data are used to form the training data set The remaining scoring data is used to form the test data set (5) The client initializes the local federated recommendation model W s,i , and iteratively train it: (5a) Each client c s,i Initialize the local federated recommendation model W consisting of a sequentially connected embedding layer, K stacked lightweight graph convolutional layers LGC, a merging layer, and F stacked fully connected layers. s,i , the inter-cluster aggregation round is l, the maximum inter-cluster aggregation round is L, L ≥ 15, the intra-cluster aggregation round is α, the maximum intra-cluster aggregation round is P, P ≥ 5, the local federated recommendation model of the lth inter-cluster aggregation and the αth intra-cluster aggregation The model parameters are And let l = 0; (5b) Let α = 0; (5c) The training dataset B training data are randomly selected with replacement as each client c s,i Local federated recommendation model W s,i The embedding layer converts the client c s,i ID, rating item t s,i,z The ID is converted into a d-dimensional user embedding vector Item embedding vector K LGC layers are based on the local bipartite graph G s,i right and Perform convolution aggregation; the merging layer embeds the user vector after convolution aggregation and item embedding vectors Perform feature fusion; F stacked fully connected layers perform feature fusion on client u according to the embedding vector of feature fusion s,i Item t s,i,b Perform rating prediction and get the predicted rating (5d) Using the predicted score labels and its corresponding true rating label y s,i,b Computing local recommendation models The MSE loss value And using the stochastic gradient descent algorithm, by Model parameters Update to obtain the model parameters of the αth intra-cluster training in the lth inter-cluster aggregation And upload to the cluster head τ s ; (5e) Cluster head τ s Model parameters Aggregate to get model parameters Distribute to clusters Internal client c s,i , and judge whether α=P is established. If so, get the cluster The federated recommendation model W with the lth inter-cluster aggregation and the Pth intra-cluster aggregation s l,P Model parameters Cluster head τ s Model parameters that will be aggregated After uploading to the global aggregation node τ, the user c in the cluster s,i with c s,j Transfer user embedding vectors to each other and and execute step (5f); otherwise, set α = α + 1 and execute step (5c); (5f) Global aggregation node τ for model parameters Perform aggregation and set the aggregation model parameter ω l Distributed to each client c s,i Then, we get the global l-th inter-cluster aggregation federated recommendation model W l The model parameter ω l , judge whether l=L is true, if so, get the trained federated recommendation model W l , otherwise l=l+1, execute step (5b); (6) Obtain the user's recommendation results for items: The test dataset As a trained local recommendation model The input is forward propagated to obtain the test data set Client u s,i Item t s,i,o Prediction score of 2. The decentralized federated graph neural network recommendation method based on local bipartite graph expansion according to claim 1 is characterized in that: The M clients are grouped based on the communication distance described in step (2). Divided into S clusters The k-means prototype clustering algorithm is used.

3. The decentralized federated graph neural network recommendation method based on local bipartite graph expansion according to claim 1 is characterized in that: For each cluster described in step (3) Every two clients c s,i 、c s,j Collection of items waiting to be rated To perform privacy-preserving intersection, a privacy-preserving intersection method based on RSA blind signature, a privacy-preserving intersection method based on OT protocol, or a privacy-preserving intersection method based on homomorphic encryption can be used.

4. The decentralized federated graph neural network recommendation method based on local bipartite graph expansion according to claim 1 is characterized in that: The cluster described in step (5d) Local recommendation model in The MSE loss value And model parameters The update formulas are: Where ∑ represents the summation operation and η>0 represents the learning rate.

5. The decentralized federated graph neural network recommendation method based on local bipartite graph expansion according to claim 1 is characterized by: The cluster head node τ in step (5e) s Model parameters The formula for aggregation is as follows:

6. The decentralized federated graph neural network recommendation method based on local bipartite graph expansion according to claim 1 is characterized by: The global aggregation node τ described in step (5f) is the model parameter The formula for aggregation is as follows:

Images