Graph neural network enhanced combined personalized federal recommendation method

Abstract

The invention discloses a graph neural network enhanced combined personalized federal recommendation method, and belongs to the crossing field of federal learning and recommendation systems. The problem that the personalized recommendation effect is limited due to the fact that hidden information among users cannot be fully utilized due to privacy protection in an existing recommendation method is solved. The method comprises the following steps: constructing a user relation graph on a server by using locally updated project embedding without accessing a user interaction record; personalized items are embedded in different devices for local fine tuning, and global item embedding is obtained through calculation between user specific item embedding; the sparse global view is encouraged to save the communication cost of federated learning, two regularizers are used to ensure complementation of the two views, and finally, the weight is dynamically adjusted to realize combination personalization; a differential privacy technology is adopted, and privacy protection based on data locality is realized according to a data locality principle. According to the invention, on the premise of ensuring privacy security, the personalized performance and communication efficiency of the recommendation system are improved.

Description

Technical Field

[0001] This invention relates to the field of federated learning and recommendation systems, and in particular to a combined personalized federated recommendation method enhanced by graph neural networks. Background Technology

[0002] Recommender systems are a core technology for alleviating information overload. With the development of internet technology, the amount of data on the internet is growing exponentially, and users' demand for personalized information is becoming increasingly urgent. Recommender systems build user profiles by analyzing users' historical interaction data, thereby customizing personalized recommendations. Currently, recommender systems are widely used in e-commerce, video streaming, news, and other fields, becoming a key force driving the upgrade of digital services. Traditional recommender systems require uploading user data to servers, posing a risk of privacy leaks. However, the EU's General Data Protection Regulation (GDPR) and increased public awareness of privacy protection have promoted the application of federated learning in the recommender field. Federated learning supports local model training without uploading raw data to servers, effectively protecting user data privacy. Meanwhile, graph neural networks, due to their ability to model complex relationships, have become an important tool in recommender systems.

[0003] Patent CN118332165A discloses a privacy-preserving federated learning method for personalized recommendation systems. This method involves the user locally initializing their own feature vector, and a trusted third party initializing and distributing the item matrix. The user then locally updates the vector, calculates local gradients, and requests noise parameters from the trusted third party. The user adds noise to the local gradient and sends it to the server, which aggregates the noisy local gradients and feeds back the global noise gradient. Finally, the user denoises to obtain the true gradient and updates the item embeddings. After successful training, a recommendation list is generated locally. However, this method struggles to meet practical application requirements when dealing with common problems in personalized recommendation such as cold start and data sparsity. Furthermore, it fails to consider implicit relationships between items or differentiated user preferences for the same item, making it difficult to accurately characterize the matching relationship between item features and user preferences.

[0004] Some recommendation methods based on graph neural networks have failed to fully integrate the synergistic benefits of social networks and knowledge graphs, and have not explored the gains of both on recommendation performance. Although they have paid attention to the propagation of preferences among users or knowledge graph information, there is still a problem of insufficient feature mining, which has failed to achieve optimal recommendation results. Summary of the Invention

[0005] In view of the above problems, the main objective of this invention is to propose a graph neural network-enhanced combined personalized federated recommendation method to solve the aforementioned technical problems.

[0006] This invention proposes a graph neural network-enhanced combined personalized federated recommendation method, the method comprising the following steps: Step 1: During the initialization phase, the server initializes the sparse global item embedding. Each client initializes its local recommendation model parameters. ,in Embed module parameters for users, Embed module parameters for local projects. The parameters for the scoring function module are used, and the server initializes the user-specific item embedding. And assign a value ; Step 2, Federated Training Phase: The following sub-steps are executed iteratively: 2a. In each round, the server randomly selects a subset of clients from all clients. Embed the current global project Distribute to clients within the collection; 2b. Client receives Then, based on local user and project interaction data With the optimization objective of combining personalized loss functions, the local model parameters are updated using the stochastic gradient descent algorithm. The loss function includes recommendation task loss, personalization regularization loss, and consistency regularization loss. 2c. The client embeds the updated local project. Uploaded to the server, local differential privacy protection is achieved by adding Laplace noise perturbation before uploading; 2d. The server receives data from each client. Then, the cosine similarity of item embeddings among clients is calculated, and a user relationship graph is constructed based on the similarity adaptive threshold. ; 2e. The server aggregates the item embeddings in the user relationship graph using a graph neural network aggregation mechanism, and updates the user-specific item embeddings. and global project embedding Global project embedding is achieved through Norm constraints remove redundant information to achieve sparsity; Step 3, Recommendation Generation Stage: After multiple rounds of iterative training, the client combines local user embeddings... Updated local project embedding and server distribution Through the scoring function module Calculate the user's predicted preference rating for uninterrupted items, and generate a personalized recommendation list by sorting the ratings.

[0007] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention constructs a two-way personalization mechanism, which combines local project embedding and global project embedding updated by server aggregation through project embedding to realize the combination and personalization of project information; (2) The present invention adds two regularizers, one to make the global item embedding remain sparsity to reduce communication overhead, and the other to ensure that there is a difference between the global item embedding and the item embedding; (3) A method is proposed that can identify user associations under the federated recommendation framework and realize the construction of user relationship graphs through shared item embeddings under privacy protection; an aggregation mechanism based on graph neural network is designed to generate user-adaptive item embeddings to enhance personalized modeling capabilities and mine implicit association information between items. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating the steps of a graph neural network-enhanced ensemble personalized federated recommendation method proposed in this invention. Figure 2 This is a diagram illustrating the overall framework of a graph neural network-enhanced combined personalized federated recommendation method proposed in this invention. Detailed Implementation

[0009] See Figure 1 This embodiment provides a graph neural network-enhanced combined personalized federated recommendation method, which includes the following steps: Step 1: During the initialization phase, the server initializes the sparse global item embedding. Each client initializes its local recommendation model parameters. ,in Embed module parameters for users, Embed module parameters for local projects. The parameters for the scoring function module are used, and the server initializes the user-specific item embedding. And assign a value .

[0010] Step 2, Federated Training Phase: The following sub-steps are executed iteratively: 2a. In each round, the server randomly selects a subset of clients from all clients. Embed the current global project Distribute to clients within the collection; 2b. Client receives Then, based on local user and project interaction data With the optimization objective of combining personalized loss functions, the local model parameters are updated using the stochastic gradient descent algorithm. The loss function includes recommendation task loss, personalization regularization loss, and consistency regularization loss. See Figure 2 The combined personalized total loss function is: ; in Recommend a loss function for binary cross-entropy. To vary with the number of training iterations Dynamically adjusted personalized regularization weights, with values ​​ranging from [value range missing]. ( (the maximum value constant of the weights) For consistency regularization weights, To constrain local project embedding and user-specific project embedding The regularization term for similarity, For weight adjustment parameters; By gradually increasing the regularization weights, the transition from fully personalized to a combination of personalized information about the project is achieved. It may contain some redundant information. use norm to promote The sparsity can be removed. This reduces redundant information and helps reduce communication costs between the server and the client; The item embedding is set as a shared component, and its main responsibility is to convey common knowledge across users. This means that all clients rely on the same globally updated item embedding to capture general trends and common features throughout the system. Meanwhile, the user embedding and rating function modules are kept local to each client, using local data for personalized maintenance and updates to more accurately reflect each user's unique preferences and behavioral patterns. For this purpose, the following objective function is used: ; Furthermore, the binary cross-entropy recommendation loss function used in the combined personalized total loss function has the following formula: ; in Indicates user With the project Whether interactive discussion took place is important, as this pertains to a typical recommendation task with implicit feedback. Therefore, when the user... With the project Interaction was conducted. ,otherwise, , It is the result of model prediction. Indicates the first The collection of projects in a client where the user has had positive interactions with the project. Indicates the first A set of negative samples randomly sampled from projects in which the client has never interacted; In particular, in order to build efficiently The set, denoted as: (This likely refers to a collection of items that were not interacted with.) ; in Represents a complete collection of projects. It is the first A set of user interaction items was used, and then a sampling ratio was set based on the number of observed interactions. Negative samples are uniformly drawn from the sample and obtained. ; Furthermore, the update method for the parameters of the local recommendation model follows the formula: ; in yes Proceed to the first Next update It's the learning rate. It's about the loss function. For model parameters The gradient of the model is given by the formula, which means that in each iteration, the model parameters are updated based on the gradient information of the current parameters, thereby gradually reducing the loss of the model and improving the recommendation performance. When updating the parameters of the local recommendation model, in order to personalize the project while retaining shared project information, the following steps are taken: To learn about sharing information, and in the corresponding... Use on a client To capture specific to the Project information for each user, and then through... and Summation is used to personalize the project's combinations; additionally, to ensure... and In the The project information learned on each client may differ; the following formula is used to force differentiation between them: ; 2c. The client embeds the updated local project. Uploaded to the server, local differential privacy protection is achieved by adding Laplace noise perturbation before uploading; See Figure 1 With the addition of Laplace noise perturbation, the following formula exists: ; in For noise intensity parameters, and ; The introduction of noise makes it impossible for the server to accurately infer updated project information solely by monitoring changes in the embedding vector. Furthermore, the noise intensity increases... As the data size increases, the privacy protection capability will also be enhanced accordingly. Therefore, it can be dynamically adjusted according to the needs of privacy protection to achieve a balance between privacy protection and recommendation accuracy.

[0011] 2d. The server receives data from each client. Then, the cosine similarity of item embeddings among clients is calculated, and a user relationship graph is constructed based on the similarity adaptive threshold. ; See Figure 1 The adaptive threshold is determined as follows: ; in For the client and Project embedding cosine similarity, This represents the mean similarity among all clients. This is the similarity threshold scaling factor. For the adjacency matrix elements of the user relationship graph, cosine similarity is used as a similarity measure between item embeddings, and the following formula exists: ; in, and They are the first The and the first Project embedding for each client, specifically, by calculating a similarity matrix. mean As a reference standard, the threshold is dynamically adjusted based on this mean to ensure that the selected neighbors have a high degree of similarity; Furthermore, the construction of a user relationship graph specifically includes: The user relationship graph constructed based on the similarity relationships between different users is defined as follows: For user sets, Given the set of edges connecting users, the adjacency matrix of this graph satisfies: When the user With users When there are edges ,otherwise .

[0012] 2e. The server aggregates the item embeddings in the user relationship graph using a graph neural network aggregation mechanism, and updates the user-specific item embeddings. and global project embedding Global project embedding is achieved through Norm constraints remove redundant information to achieve sparsity.

[0013] See Figure 2 The specific calculation process of the graph neural network aggregation mechanism is as follows: ; ; in Embed a collection of projects uploaded by the client. For the user relationship graph adjacency matrix, The number of convolutional layers is set to [value]. , Adjacency matrix The degree matrix, Embed a matrix for the aggregated user-specific items. The Middle Action is .

[0014] Step 3, Recommendation Generation Phase: After multiple rounds of iterative training, the client combines local user embeddings... Updated local project embedding and server distribution Through the scoring function module Calculate the user's predicted preference rating for uninterrupted items, and generate a personalized recommendation list by sorting the ratings.

[0015] In step 3, the scoring function module contains the following formula: ; The Sigmoid activation function is used to... To process, Indicates that for the first Personalized embedding of individual project combinations It is a scoring function module, and .

[0016] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are presented sequentially as indicated by the arrows, they are not necessarily executed in that order. Unless otherwise expressly stated herein, there is no strict order constraint on the execution of the steps, and other orders may be used. In addition, at least some steps in each embodiment may include multiple sub-steps or stages, which do not need to be completed simultaneously, can be executed at different times, and the execution order does not need to proceed sequentially, but can be performed alternately or in turn with other steps or at least a portion of their sub-steps or stages.

[0017] In the detailed description of this specification, when terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples" are used, their core meaning is that the technical features, structural designs, material selections, or functional characteristics mentioned in the specific description of the corresponding embodiment or example are all included within the protection and coverage of at least one embodiment or example of the present invention. It should be noted that the illustrative descriptions of the above-mentioned terms in this specification do not necessarily refer to the same set of embodiments or the same specific example. The technical content referred to by different terms may correspond to different implementation scenarios, but all fall within the scope of the technical concept of the present invention. Furthermore, the specific features, structures, materials, or characteristics involved in the above description are not limited to a single embodiment or example, but can be combined and matched in a reasonable and effective manner in any one or more embodiments or examples according to the needs of actual application scenarios, thereby forming more diversified implementation forms.

[0018] The embodiments described above only illustrate some implementations of the present invention. Although the descriptions are detailed, they do not constitute a limitation on the scope of protection of this invention. For those skilled in the art, equivalent modifications and partial improvements made based on an understanding of the core concept of this invention all fall within the scope of protection of this invention.

Claims

1. A graph neural network-enhanced combinatorial personalized federated recommendation method, characterized in that, The method includes the following steps: Step 1: During the initialization phase, the server initializes the sparse global item embedding. Each client initializes its local recommendation model parameters. ,in Embed module parameters for users. Embed module parameters for local projects. The parameters for the scoring function module are used, and the server initializes the user-specific item embedding. And assign a value ; Step 2, Federated Training Phase: The following sub-steps are executed iteratively: 2a. In each round, the server randomly selects a subset of clients from all clients. Embed the current global project Distribute to clients within the collection; 2b. Client receives Then, based on local user and project interaction data With the optimization objective of combining personalized loss functions, the local model parameters are updated using the stochastic gradient descent algorithm. The loss function includes recommendation task loss, personalization regularization loss, and consistency regularization loss. 2c. The client embeds the updated local project. Uploaded to the server, local differential privacy protection is achieved by adding Laplace noise perturbation before uploading; 2d. The server receives data from each client. Then, the cosine similarity of item embeddings among clients is calculated, and a user relationship graph is constructed based on the similarity adaptive threshold. ; 2e. The server aggregates the item embeddings in the user relationship graph using a graph neural network aggregation mechanism, and updates the user-specific item embeddings. and global project embedding Global project embedding is achieved through Norm constraints remove redundant information to achieve sparsity. Step 3, Recommendation Generation Stage: After multiple rounds of iterative training, the client combines local user embeddings... Updated local project embedding and server distribution Through the scoring function module Calculate the user's predicted preference rating for uninterrupted items, and generate a personalized recommendation list by sorting the ratings.

2. The graph neural network-enhanced combined personalized federated recommendation method according to claim 1, characterized in that, In step 2b, the combined personalized total loss function is: ； in Recommend a loss function for binary cross-entropy. To vary with the number of training iterations Dynamically adjusted personalized regularization weights, with values ​​ranging from [value range missing]. ( (the maximum value constant of the weights) For consistency regularization weights, To constrain local project embedding and user-specific project embedding The regularization term for similarity, For weight adjustment parameters; By gradually increasing the regularization weights, the transition from fully personalized to a combination of personalized information about the project is achieved. It may contain some redundant information. use norm to promote The sparsity can be removed. This reduces redundant information and helps reduce communication costs between the server and the client; The method for updating the parameters of the local recommendation model is based on the following formula: ； in yes Conduct the first Next update It's the learning rate. It's about the loss function. For model parameters The gradient of the model is given by the formula, which means that in each iteration, the model parameters are updated based on the gradient information of the current parameters, thereby gradually reducing the loss of the model and improving the recommendation performance.

3. The graph neural network-enhanced combined personalized federated recommendation method according to claim 1, characterized in that, In step 2c, a Laplace noise perturbation is added, and the following formula exists: ； in For noise intensity parameters, and ; The introduction of noise makes it impossible for the server to accurately infer updated project information solely by monitoring changes in the embedding vector. Furthermore, the noise intensity increases... As the data size increases, the privacy protection capability will also be enhanced accordingly. Therefore, it can be dynamically adjusted according to the needs of privacy protection to achieve a balance between privacy protection and recommendation accuracy.

4. The graph neural network-enhanced combined personalized federated recommendation method according to claim 1, characterized in that, The adaptive threshold is determined in step 2d as follows: ； in For the client and Project embedding cosine similarity, This represents the mean similarity among all clients. This is the similarity threshold scaling factor. For the adjacency matrix elements of the user relationship graph, cosine similarity is used as a similarity measure between item embeddings, and the following formula exists: ； in, and They are the first The and the first Project embedding for each client; specifically, by calculating a similarity matrix. mean As a reference standard, the threshold is dynamically adjusted based on this mean to ensure that the selected neighbors have a high degree of similarity; Furthermore, the construction of a user relationship graph specifically includes: The user relationship graph constructed based on the similarity relationships between different users is defined as follows: For user sets, Given the set of edges connecting users, the adjacency matrix of this graph satisfies: When the user With users When there are edges ,otherwise .

5. The graph neural network-enhanced combined personalized federated recommendation method according to claim 1, characterized in that, The specific calculation process of the graph neural network aggregation mechanism in step 2e is as follows: ； ； in Embed a collection of projects uploaded by the client. For the user relationship graph adjacency matrix, The number of convolutional layers is set to [value]. , Adjacency matrix The degree matrix, Embed a matrix for the aggregated user-specific items. The Middle Action is .

6. The graph neural network-enhanced combined personalized federated recommendation method according to claim 1, characterized in that, In step 3, the scoring function module contains the following formula: ； The Sigmoid activation function is used to... To process, Indicates that for the first Personalized embedding of individual project combinations It is a scoring function module, and .

7. The graph neural network-enhanced combined personalized federated recommendation method according to claim 2, characterized in that, The binary cross-entropy recommendation loss function has the following formula: ； in Indicates user With the project Whether interactive discussion took place is important, as this pertains to a typical recommendation task with implicit feedback. Therefore, when the user... With the project Interaction was conducted. ,otherwise, , It is the result of model prediction. Indicates the first The collection of projects in a client where the user has had positive interactions with the project. Indicates the first A set of negative samples randomly sampled from projects where no client interaction has occurred.

Images