Token chain aggregation and mask privacy protection recommendation method

Abstract

This invention relates to the field of system privacy protection technology, specifically to a token chain aggregation and mask privacy protection recommendation method. Addressing the issues of lack of trust in the central server and single point of failure in existing architectures, it adopts a peer-to-peer network framework with clients as organizations, coupled with a token control mechanism, to achieve fully decentralized chain aggregation, enhancing system robustness. Simultaneously, to address the privacy leakage problem caused by plaintext gradient transmission, it introduces a pairwise seed-based masking technique, adding cancelable noise masks during the gradient calculation stage to ensure that even if intercepted during transmission or maliciously analyzed by neighboring nodes, the true gradient information cannot be reconstructed. Furthermore, to address the problem of star network communication congestion, this invention utilizes a chain aggregation strategy, controlling the orderly transmission and accumulation of gradients through tokens, significantly reducing instantaneous network bandwidth pressure and improving communication efficiency, thus providing a secure and efficient solution for multi-party collaborative modeling.

Description

Technical Field

[0001] This invention relates to the field of system privacy protection technology, specifically to a token chain aggregation and masked privacy protection recommendation method. Background Technology

[0002] Recommender systems, as a core technology for solving information overload and achieving personalized services, have been widely applied in various fields such as e-commerce, social networks, healthcare, and financial services. For example, in the medical field, if different hospitals can jointly use patient medical records to train recommendation models, the accuracy of assisted diagnosis and treatment plan recommendations will be greatly improved. In the financial field, if different banks or financial institutions can share customer credit behavior data, the accuracy of wealth management product recommendations and risk control capabilities will be significantly improved. With the advancement of big data technology, cross-organizational data collaboration has become a key driver for improving the level of industry intelligence.

[0003] However, with the promulgation and implementation of laws and regulations such as the Personal Information Protection Act (PIPL) and the EU's General Data Protection Regulation (GDPR), data privacy protection has become an inviolable red line. Traditional recommendation systems are typically based on centralized matrix factorization (MF), requiring the uploading of user historical behavior data (such as ratings, medical records, and transaction logs) scattered across various organizations (such as hospitals and banks) to a central server for unified storage and model training. This centralized processing method not only faces significant legal compliance risks, but also poses a catastrophic risk of massive leaks of sensitive data (such as patient privacy and asset information) should the central server be attacked or internal data misuse occur. Therefore, how to break down "data silos" and achieve efficient cross-organizational collaboration while ensuring that data does not leave local storage has become a key issue that urgently needs to be addressed.

[0004] The shortcomings and deficiencies of existing products:

[0005] Traditional federated learning architectures suffer from trust issues and single points of failure: Most current federated learning (FL) systems employ a client-server star topology. This architecture heavily relies on a central server for parameter aggregation, requiring all participating organizations to unconditionally trust this central server. If the central server is maliciously controlled or has security vulnerabilities, attackers can easily tamper with the model or implement a denial-of-service attack. Furthermore, the central server constitutes a single point of failure (SPoF); if it fails, the entire cross-organizational collaborative network will immediately collapse.

[0006] Plaintext gradient transmission poses a privacy risk: While federated learning avoids the transmission of raw data, existing research shows that gradients during model updates carry a wealth of data distribution information. Without additional protection, attackers (even malicious central servers) can use gradient inversion attacks to precisely recover the original interaction data (such as which items a user rated) from the uploaded plaintext gradients, rendering privacy protection ineffective.

[0007] Communication bottlenecks and inefficiencies under high concurrency: In large-scale collaborative networks across organizations, if a star topology is used, all clients need to concurrently connect to the central server to upload parameters, which can easily cause network congestion at the central node and lead to a sharp increase in communication latency. While existing secure aggregation protocols can improve security, they often have extremely high computational and communication complexity, making it difficult to meet the real-time and high-frequency update requirements of recommendation systems. Summary of the Invention

[0008] The purpose of this invention is to provide a token chain aggregation and masked privacy-preserving recommendation method, aiming to provide a privacy-preserving recommendation system based on federated matrix factorization, so as to realize cross-organizational secure collaborative recommendation in a decentralized server environment.

[0009] To achieve the above objectives, this invention provides a token chain aggregation and masked privacy-preserving recommendation method, comprising the following steps:

[0010] Step 1: Architect a peer-to-peer network system without a central server and consisting of multiple clients, and perform system initialization;

[0011] Step 2: Construct a joint matrix factorization model and train the model locally;

[0012] Step 3: Gradient masking;

[0013] Step 4: Token-controlled chain aggregation;

[0014] Step 5: Global update and distribution;

[0015] Step 6: Iteration Termination and Recommendation Generation.

[0016] Optionally, the execution process of step 1 includes the following steps:

[0017] Step 1.1: P2P network setup and key negotiation;

[0018] Step 1.2: Constructing the logical chain topology and client set;

[0019] Step 1.3: Initialize tokens and model parameters.

[0020] Optionally, the joint matrix factorization model simultaneously integrates user-item rating information and user-item type preference information. In step 2, each client... Based on local user-project rating data and user-type preference matrix Using the current version's global project feature matrix and type features Calculate the regularized least squares loss function with respect to the item feature matrix. Local gradient And about the preference matrix Local gradient Update the local user feature matrix .

[0021] Optionally, in step 3, a mask is generated using a preset seed to obfuscate the local gradient and obtain the masked gradient. and :

[0022]

[0023]

[0024] in, This refers to the set of clients participating in the chain aggregation in this round. and It is a pseudo-random generator used to generate random matrices with the same dimension as the gradient.

[0025] Optionally, during the execution of step 4, the mask gradient is accumulated sequentially by passing the token on the logical chain. The starting client accumulates the mask gradient and updates the token, and then sends it to the intermediate client on the chain. The intermediate client calculates and updates the token and passes it to the next node. This process ensures that each node can only see the mask gradient of the preceding node.

[0026] Optionally, in step 5, the gradient descent algorithm update formula uses the Barzilai-Borwein method to adaptively calculate the learning rate. .

[0027] Optionally, step 6 terminates the iteration when the training reaches the model accuracy or the upper limit threshold, for each client. Obtaining optimal model parameters The algorithm calculates a rating matrix and outputs a recommended list of items based on their rating values.

[0028] This invention provides a token chain aggregation and masked privacy protection recommendation method. Addressing the issues of lack of trust in the central server and single point of failure in existing architectures, it adopts a peer-to-peer network framework with clients as organizations, coupled with a token control mechanism, to achieve fully decentralized chain aggregation, enhancing system robustness. Simultaneously, to address the privacy leakage problem caused by plaintext gradient transmission, this invention introduces a pairwise seed-based masking technique, adding cancelable noise masks during the gradient calculation stage to ensure that even if intercepted during transmission or maliciously analyzed by neighboring nodes, the true gradient information cannot be reconstructed. Furthermore, to address the congestion problem in star network communication, this invention utilizes a chain aggregation strategy, controlling the orderly transmission and accumulation of gradients through tokens, significantly reducing instantaneous network bandwidth pressure and improving communication efficiency, thus providing a secure and efficient solution for multi-party collaborative modeling. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a schematic diagram of the system architecture in the token chain aggregation and masked privacy protection recommendation method of the present invention.

[0031] Figure 2 This is a schematic diagram of the training process of the method model of the present invention.

[0032] Figure 3 This is a schematic diagram of the chain aggregation and token circulation process of the method of the present invention. Detailed Implementation

[0033] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0034] This invention provides a token chain aggregation and masked privacy-preserving recommendation method, comprising the following steps:

[0035] Step 1: Architect a peer-to-peer network system without a central server and consisting of multiple clients, and perform system initialization;

[0036] Step 2: Construct a joint matrix factorization model and train the model locally;

[0037] Step 3: Gradient masking;

[0038] Step 4: Token-controlled chain aggregation;

[0039] Step 5: Global update and distribution;

[0040] Step 6: Iteration Termination and Recommendation Generation.

[0041] Please see Figures 1 to 3 The following provides further explanation with reference to specific embodiments and execution steps:

[0042] Step 1 is the system initialization process, which specifically includes the following steps:

[0043] (1) P2P network topology and key negotiation

[0044] The overall system architecture is as follows Figure 1 As shown, upon system startup, all clients connect to the network and establish peer-to-peer (P2P) connections. All clients are paired up and negotiate a pairwise random seed using a secure key exchange protocol (such as the Diffie-Hellman protocol). (in = (where i and j represent the i-th and j-th clients, respectively). This process only needs to be executed once, seed. It is stored locally in encrypted form and used to generate masks in subsequent training rounds.

[0045] (2) Logical chain topology and client set construction

[0046] The system constructs a unidirectional logical chain topology based on client IDs. Assume the system has... Each client (organization) is assigned a logical index. And specify its predecessor node (Predecessor, ) and successor node (Successor, The index number of each client constitutes the client set, denoted as . .

[0047] (3) Initialization of tokens and model parameters

[0048] The system generates a unique authorization token. The token is a data packet containing metadata, primarily including: the current training epoch. Participating in the Wheel-chain aggregation client collection Digital signatures used to verify identity and gradient accumulator and In the initial state, tokens are allocated to the first client in the logical chain (i.e., the node of Client 1), marking the readiness for the first round of training.

[0049] Specifically, to enable adaptive BB step size updates later, the client (the node responsible for global updates) needs to initialize the historical parameter cache upon termination: recording the model parameters from the previous round. , and the global gradient of the previous round , .

[0050] Step 2 is the process of training the local model. At the start of each training round, the client holding the token acts as the starting node and initiates training. Each client... Based on local user-project rating data and user-type preference matrix Using the current version's global project feature matrix and type features Calculate the regularized least squares loss function with respect to Local gradient And about Local gradient Simultaneously, update the local user feature matrix. (No upload required, privacy protected).

[0051] In step 2, to address the data sparsity problem and fully utilize multi-source heterogeneous data, this invention constructs a joint matrix factorization model that simultaneously integrates user-item rating information and user-item type preference information. Assume the system consists of... The system consists of participating organizations (i.e., clients, such as different hospitals or banks), and has a total of [number missing] participants. Projects and Project type. Record the first... The number of users owned by each client is To uncover potential feature associations, matrix factorization is employed to map high-dimensional sparse matrices to a low-dimensional latent space. The specific factorization strategy is as follows:

[0052] (1) User-project interaction modeling: The first Interaction rating matrix of individual clients Decomposed into user feature matrix With project feature matrix The product of you and the target is .

[0053] (2) User-type preference modeling: The interaction frequency matrix of users to item types Decomposed into a shared user latent feature matrix Project type feature matrix The product of , i.e., the fitting objective is .

[0054] In order to collaboratively train the above model parameters within the federated learning framework , and The present invention constructs the following regularized least squares loss function:

[0055]

[0056] in, Denotes the Frobenius norm; , , This is a regularization parameter used to prevent the model from overfitting; Indicates matrix transpose; This is a projection operator used to process unobserved terms in a sparse matrix. Specifically, for any matrix... and observation index set , The definition is as follows:

[0057]

[0058] This ensures that the optimization process only performs error calculations based on actual user interactions.

[0059] To solve the aforementioned non-convex objective function, this invention employs an adaptive gradient descent algorithm based on the step size of the Barzilai-Borwein (BB) method for iterative optimization. The specific update formulas for each parameter are as follows (where... (representing the learning rate)

[0060] (1) Update user latent characteristics (executed locally):

[0061] because Only involving the first Each client's private data is updated independently locally:

[0062]

[0063] in, , These represent the reconstruction errors of the rating matrix and the type matrix, respectively.

[0064] (2) Globally shared feature update (federated aggregation):

[0065] Project Features and type features These are global parameters shared by all clients. During federated training, each client calculates its local gradient and then updates the global parameters through a secure aggregation mechanism.

[0066]

[0067] Through the above iterative process, the system is able to prevent the leakage of the original data of each client. and Under the premise of [the above conditions], collaborative learning yields accurate recommendation model parameters.

[0068] Local gradient and The calculation formula is as follows:

[0069]

[0070]

[0071] Step 3 is the gradient masking process, on the client side. Do not send the raw gradient directly and Instead, it uses a pre-set seed to generate a mask to obfuscate the gradient and obtain the masked gradient. and :

[0072]

[0073]

[0074] in, This refers to the set of clients participating in the chain aggregation in this round. and It is a pseudo-random generator used to generate random matrices with the same dimension as the gradient.

[0075] The chain aggregation process for token control in step 4 is as follows:

[0076] The mask gradient is accumulated sequentially by passing tokens along the logical chain:

[0077] 4.1 Initial Client (Client 1): Accumulate mask gradients and update tokens

[0078] calculate: .

[0079] renew:

[0080] Token Send it to the next client on the chain.

[0081] 4.2 Intermediate Client ): Receive tokens sent by the preceding node ,calculate

[0082] (1) Regarding project characteristics Accumulated mask gradient:

[0083] (2) Regarding type characteristics Accumulated mask gradient:

[0084] Update token:

[0085]

[0086] And Pass it to the next node.

[0087] Note: This process means that each node can only see the mask gradient sum of its predecessor nodes, and cannot extract the specific details. and .

[0088] Step 5 is the global update and distribution process, during which:

[0089] End Client ): Receive token Then, obtain the final accumulated value:

[0090]

[0091]

[0092] According to mathematical principles,

[0093]

[0094] All masks cancel each other out. Therefore, the client directly obtains the true global gradient sum at the end:

[0095]

[0096] End the client to update model parameters using global gradients , and aggregated global gradients , Broadcast to all clients.

[0097] The specific steps for the global update based on the step size of the Barzilai-Borwein (BB) method in step 5 are as follows:

[0098] (1) Calculate the difference between variables (only when hour)

[0099] Calculate the change in parameters:

[0100] Calculate the gradient change:

[0101] (2) Calculate the adaptive step size :

[0102] Calculate based on BB step size formula for... learning rate ( Similarly):

[0103]

[0104] in This represents the inner product of matrices. If... Then use the preset initial learning rate. .

[0105] (3) Perform global parameter update:

[0106]

[0107]

[0108] (4) Update cache and distribution:

[0109] The current gradient and parameters Store it in the history cache for use in the next round of BB step size calculation. Finally, update the... , It is distributed to all clients via broadcast.

[0110] Step 6 is the process of iteration termination and recommendation generation. Iteration terminates when the training reaches the model accuracy or the iteration upper limit threshold. (Each client...) Obtaining optimal model parameters The system calculates a rating matrix using these values ​​and then sorts the items according to their ratings to output a recommended list. The specific calculation formula is as follows:

[0111] (1) Iteration termination condition:

[0112]

[0113] in, This represents the total number of observations from all clients. This is the model accuracy threshold. The upper limit threshold for iteration, It means "or".

[0114] (2) Calculate the rating matrix: .

[0115] Furthermore, this invention also selected three small, medium, and large movie datasets that are widely used for evaluating recommendation algorithms in the examples for comparison and illustration. Specific information is shown in Table 1.

[0116] Table 1. Evaluation Dataset for Recommendation Algorithms

[0117]

[0118] The specific implementation steps of the embodiment are as follows:

[0119] Initialization phase:

[0120] The system sets the regularization parameters of the model. , , Initial learning rate Model accuracy threshold and maximum number of iterations The specific parameter settings are shown in Table 2. Initialize the global model parameters. and Build a logical index set containing all clients. Generate authorization token .

[0121] All clients connect to the network and establish peer-to-peer (P2P) connections. Clients pair up and negotiate a pairwise random seed using a secure key exchange protocol (such as the Diffie-Hellman protocol). (in = Meanwhile, each client Initialize the local user feature matrix .

[0122]

[0123] Table 2 Hyperparameter Settings

[0124] Model training phase:

[0125] The client holding the token acts as the starting node and initiates the training signal. The client received the first Tokens sent by clients Using the current version of the project feature matrix and type feature matrix Complete the following calculations:

[0126] (1) Update the local user feature matrix:

[0127] (2) Calculate about Local gradient and about Local gradient : ,

[0128] in, , .

[0129] Then, the local gradient mask is processed:

[0130]

[0131]

[0132] in, This refers to the set of clients participating in the chain aggregation in this round. and For pseudo-random generators, the seed is expanded to a random matrix with the same dimension as the gradient.

[0133] Client implement

[0134] Accumulated gradient: , ,

[0135] Update token:

[0136] and pass it on to the successor node (Client) ).

[0137] The above operations are executed sequentially on the logical chain formed by the clients, until the last client completes gradient accumulation to obtain the global aggregate gradient. and The system then updates the project matrix and type matrix, completing the current training round. The system selects the user set for the next round of training, iterating until the termination condition is met, at which point the training phase ends. Each training round uses 5-fold cross-validation to evaluate the model's prediction performance on the test set. Evaluation metrics include root mean square error (RMSE) and mean absolute error (MAE), defined as follows:

[0138] ,

[0139] in, Indicates the number of observations in the test set; and users respectively For the project The actual rating and the predicted rating.

[0140] Training completion phase:

[0141] After the training phase is complete, the optimal model parameters are output. Each client is allocated Perform the calculation locally: The system obtains the user's predicted ratings for all items, and selects the top K items with the highest predicted ratings (items the user has not interacted with) as the user's recommendation list, thus completing the recommendation task.

[0142] Finally, the method of this invention (Ours) is compared with the Federated Matrix Factorization (FedMF) scheme in plaintext and the Federated Matrix Factorization (DPFedMF) scheme based on differential privacy. The comparison results of 5-fold cross-validation are shown in Table 3, where the privacy budget of DPFedMF is 1. The comparison results show that the prediction accuracy of the method of this invention is comparable to that of the method trained in plaintext, and significantly better than the federated learning method based on local differential privacy. Furthermore, the method of this invention requires more than 50% fewer iterations than the comparison schemes, which helps to save the communication overhead caused by frequent interactions in federated learning.

[0143] Table 3 Comparison of Model Prediction Accuracy

[0144]

[0145] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A token chain-based aggregation and masked privacy-preserving recommendation method, characterized in that, Includes the following steps: Step 1: Architect a peer-to-peer network system without a central server and consisting of multiple clients, and perform system initialization; Step 2: Construct a joint matrix factorization model and train the model locally; Step 3: Gradient masking; Step 4: Token-controlled chain aggregation; Step 5: Global update and distribution; Step 6: Iteration Termination and Recommendation Generation.

2. The token chain aggregation and masked privacy-preserving recommendation method as described in claim 1, characterized in that, The execution process of step 1 includes the following steps: Step 1.1: P2P network setup and key negotiation; Step 1.2: Constructing the logical chain topology and client set; Step 1.3: Initialize tokens and model parameters.

3. The token chain aggregation and masked privacy-preserving recommendation method as described in claim 2, characterized in that, The joint matrix factorization model simultaneously integrates user-item rating information and user-item type preference information. In step 2, each client... Based on local user-project rating data and user-type preference matrix Using the current version's global project feature matrix and type features Calculate the regularized least squares loss function with respect to the item feature matrix. Local gradient And about the preference matrix Local gradient Update the local user feature matrix .

4. The token chain aggregation and masked privacy-preserving recommendation method as described in claim 3, characterized in that, In step 3, a mask is generated using a preset seed to obfuscate the local gradient, resulting in the masked gradient. and : in, This refers to the set of clients participating in the chained aggregation in this round. and It is a pseudo-random generator used to generate random matrices with the same dimension as the gradient.

5. The token chain aggregation and masked privacy-preserving recommendation method as described in claim 4, characterized in that, During the execution of step 4, the mask gradient is accumulated sequentially by passing the token on the logical chain. The starting client accumulates the mask gradient and updates the token, and then sends it to the intermediate client on the chain. The intermediate client calculates and updates the token and passes it to the next node. This process ensures that each node can only see the mask gradient of the preceding node.

6. The token chain aggregation and masked privacy-preserving recommendation method as described in claim 5, characterized in that, In step 5, the gradient descent algorithm update formula uses the Barzilai-Borwein method to adaptively calculate the learning rate. .

7. The token chain aggregation and masked privacy-preserving recommendation method as described in claim 6, characterized in that, Step 6: The iteration terminates when the training reaches the model accuracy or the upper limit threshold. (For each client) Obtaining optimal model parameters The algorithm calculates a rating matrix and outputs a recommended list of items based on their rating values.

Images