A privacy protection personalized federated learning method based on contribution degree relationship graph model

Abstract

This invention proposes a privacy-preserving personalized federated learning method based on a contribution graph model. Based on a collaborative training aggregation mechanism with dynamic contribution, the server dynamically weights and aggregates client parameters according to their contributions, generating a personalized global model for each client. A contribution graph model is created by fusing data scale, model accuracy, parameter similarity, and feature similarity. Clients train their local models, incorporating differential privacy into the parameter transmission process. Noise injection ensures that the data transmitted by the client in each round of communication satisfies differential privacy constraints. The privacy loss is calculated based on the number of samples used by the client during local training, obtaining an upper bound for the privacy loss in parameter communication. By establishing a contribution graph model to achieve collaborative training among clients and combining differential privacy to protect client privacy data, this method effectively solves the challenge of balancing privacy protection and performance in personalized federated learning under heterogeneous data environments.

Description

Technical Field

[0001] This invention relates to the field of personalized federated learning technology, and in particular to a privacy-preserving personalized federated learning method based on a contribution graph model. Background Technology

[0002] Data heterogeneity is the most common heterogeneity problem in federated learning, referring to the non-independent, identically distributed (Non-IID) nature of the data from participating parties. Its essence stems from the multi-source heterogeneous characteristics of distributed data acquisition environments. Federated learning is a distributed machine learning framework that aims to enable multiple participants to collaboratively train a globally shared model while protecting their local data privacy, thus breaking down data silos. Personalized federated learning achieves customized modeling for clients through methods such as meta-learning or clustering. However, federated learning still faces the challenge of balancing privacy protection and model performance in heterogeneous data environments.

[0003] To address these issues, we designed a privacy-preserving personalized federated learning method based on a contribution graph model. Summary of the Invention

[0004] The purpose of this invention is to address the performance degradation and lack of privacy protection in federated learning under heterogeneous data environments in existing technologies, and to propose a privacy-preserving personalized federated learning method based on a contribution graph model.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A privacy-preserving personalized federated learning method based on a contribution graph model, comprising the following:

[0007] A collaborative training aggregation mechanism based on dynamic contribution is proposed, in which the server dynamically weights and aggregates client parameters according to contribution, generating a personalized global model for each client;

[0008] By integrating data scale, model accuracy, parameter similarity, and feature similarity, a contribution relationship graph model is created.

[0009] The client trains a local model and introduces differential privacy into the parameter transmission process. Noise injection is used to ensure that the data transmitted by each client in each round of communication satisfies the differential privacy constraint. The privacy loss is calculated based on the number of samples used by the client in local training, and the upper bound of the privacy loss in parameter communication is obtained.

[0010] To further optimize the above scheme, a collaborative training aggregation mechanism based on dynamic contribution is proposed as follows:

[0011] Construct a fully connected topology network, with all clients acting as nodes. Establish bidirectional transmission channels through directed edges, and assign dynamically updated contribution values ​​to each edge to quantify the strength of cooperation between clients.

[0012] During the model aggregation phase, the server dynamically weights and aggregates client parameters based on contribution, generating a personalized global model for each client, expressed as follows:

[0013]

[0014] In the formula, w i Indicates client U i A personalized global model, where N represents the number of clients, R ij Indicates client U j For client U i Contribution, w j Indicates client U j The local model.

[0015] Further optimization of the above scheme involves creating a contribution relationship graph model by integrating data scale, model accuracy, parameter similarity, and feature similarity; among which,

[0016] The data size is expressed by the following formula:

[0017]

[0018] In the formula, p i Indicates client U i The proportion of the local dataset size to the global sample size, with each client possessing a local dataset D. i , let d i =|D i | represents dataset D i The sample size included;

[0019] The accuracy of the model is expressed by the following formula:

[0020]

[0021] In the formula, a i This indicates that the client U is being reflected. i The model's accuracy in fitting its own data distribution, Acc i Indicates client U i The accuracy of the local model;

[0022] The parameter similarity is expressed by the following formula:

[0023]

[0024] In the formula, w represents the similarity of parameters that measure the distribution of client-side data. i Indicates client U i The local model, w j Indicates client U j The local model;

[0025] The feature similarity is expressed by the following formula:

[0026]

[0027] In the formula, This indicates that the client's local model performs inference and observes the feature similarity of the inference results, c i and c j They represent client U respectively i and client U j The training hyperparameters of the local model, D g This represents the public sample set owned by the server.

[0028] The service dynamically constructs a contribution graph model R based on data scale, model accuracy, parameter similarity, and feature similarity. The optimization objective for the client is:

[0029]

[0030] Where γ1, γ2, and γ3 are all weighting coefficients, p j Indicates client U j The proportion of the local dataset size to the global sample size, a j This indicates that the client U is being reflected. j The model accuracy, which measures the fit to its own data distribution, is represented by R, which denotes the contribution graph model. When the number of clients is N, R ij Indicates client U j For client U i The degree of contribution;

[0031] After obtaining the local model parameters uploaded by the client in each round of communication, the server optimizes the contribution relationship graph model and aggregates the server model.

[0032] To further optimize the above scheme, the client trains a local model, and the optimization objective expression is:

[0033]

[0034] Among them, L i Let F represent the client's loss function. i D represents the local experience loss in traditional federated learning. i,j Represents the local dataset Di The j-th data point, E represents the number of local iterations on the client, B represents the batch size for training the model on the client; EB represents the number of samples used by the client in local training, λ is the balancing parameter, w represents the training model, w i Indicates client U i Personalized global model;

[0035] Through local training optimization on the client side, the client adjusts the local model to minimize the task loss of the local model on the local dataset, and the direction of the local model is consistent with the direction of the global model.

[0036] To further optimize the above scheme, differential privacy is introduced into the parameter transmission process. Noise injection ensures that the transmitted data of each client in each round of communication satisfies differential privacy constraints. The process is as follows:

[0037] In each round of communication, the client injects Gaussian noise that conforms to the (ε,δ)-differential privacy constraint before uploading parameters, making the parameter distributions generated by any adjacent datasets indistinguishable and resisting member inference privacy attacks. The local model noise addition process is expressed by the following formula:

[0038]

[0039] in, Indicates client U i Gaussian noise in the local model Indicates client U i Local model parameters, Indicates the disturbed client U i The local model parameters; the server's aggregation process is represented as:

[0040]

[0041] Among them, w i,t This indicates the client U after introducing differential privacy. i A personalized global model, where N represents the number of clients. This indicates that after introducing differential privacy, the client U j For client U i Contribution Indicates the disturbed client U j Local model parameters, Indicates client U j Local model parameters, Indicates client U j Gaussian noise in the local model;

[0042] To determine the noise scale, client U iThe local model parameters are pruned using a maximum pruning threshold C as a constraint. This operation is denoted as w. i =w i / max(1,||w i || / C);

[0043] Based on this, the parameter sensitivity is calculated as follows:

[0044]

[0045] Among them, D i With D′ i For adjacent datasets that differ by only one data record, L i Denotes the client's loss function, D i,j With D′ i,j Represents the neighboring dataset D in the local dataset. i With D′ i,j The j-th data; and This represents the local model in adjacent datasets that differ by only one data record; d i Represents dataset D i The number of samples included, w represents the training model.

[0046] Based on the differential privacy Gaussian mechanism, in a single round of communication satisfying (ε,δ)-differential privacy, the expression for the noise scale added by the client to the parameters is:

[0047]

[0048] Where, σ i Indicates client U i The noise scale added to the parameters, ε represents the privacy budget, and δ is the tolerance for a low probability of privacy leakage.

[0049] Further optimization of the above scheme, specifically the calculation of the privacy loss based on the number of samples used by the client during local training to obtain the upper bound of the privacy loss in parameter communication, includes:

[0050] The client trains locally using multiple rounds of stochastic gradient descent. When the training between two rounds of communication does not utilize all local data samples, privacy loss is estimated by considering privacy sampling amplification. When correcting the noise scale added to the parameters by the client, the privacy parameters that are satisfied are:

[0051]

[0052] The corrected expression for the noise scale added to the parameters by the client is as follows:

[0053]

[0054] In the above formula, ε' and δ' represent privacy parameters, ε represents the global privacy budget given by the client in a single round of communication, the client performs E iterations locally, the batch data size used in each iteration is B, EB represents the number of samples used by the client in local training, and the total privacy loss of the client after multiple rounds of communication is affected by the data sampling rate q = EB / d. i ;

[0055] Client U i Add noise to the local model according to the noise scale shown in the above formula and send it to the server, so that the model parameters satisfy (ε,δ)-differential privacy in each round of communication. Then, after T rounds of communication, obtain the upper bound of the privacy loss in parameter communication, that is, the total privacy loss of the client is expressed as:

[0056]

[0057] Where c0 is a constant;

[0058] The total privacy loss of the client after T rounds of communication is affected by the data sampling rate q and the number of communication rounds T.

[0059] Compared with existing technologies, the advantages of this invention are as follows: This invention achieves fine-grained client-side collaborative training by establishing a contribution graph model, while simultaneously protecting the privacy data of each client through differential privacy, effectively solving the privacy protection problem of personalized federated learning in heterogeneous data environments. The establishment of the contribution graph model frees clients from the limitations of global model non-convergence caused by federated averaging or cross-class knowledge barriers caused by client clustering, thus benefiting all participating clients. Gaussian-based differential privacy provides mathematically provable privacy protection for the training system, improves defense against member inference attacks, accurately calculates privacy losses, saves privacy budget, and improves model usability, outperforming existing benchmark methods in both privacy and model performance tradeoffs. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of a privacy-preserving personalized federated learning method based on a contribution graph model proposed in this invention.

[0061] Figure 2 This is a schematic diagram of the model layout for a privacy-preserving personalized federated learning method based on a contribution graph model proposed in this invention. Detailed Implementation

[0062] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0063] Reference Figure 1 This embodiment proposes a privacy-preserving personalized federated learning method based on a contribution graph model, which includes the following:

[0064] A collaborative training aggregation mechanism based on dynamic contribution is proposed, in which the server dynamically weights and aggregates client parameters according to contribution, generating a personalized global model for each client.

[0065] Consider a typical federated learning scenario, which includes a cloud server and N clients {U} i Given a dataset D, where i = 1, 2, ..., N, each client has a local dataset D. i , let d i =|D i | Represents the local dataset D i The sample size included. Addressing the knowledge-sharing limitations of client-side clustering methods in traditional federated learning frameworks, this embodiment proposes a collaborative training mechanism based on a dynamic contribution graph model. Figure 2 A schematic diagram of a model containing 6 clients (U1-U6) is shown.

[0066] A collaborative training aggregation mechanism based on dynamic contribution is proposed as follows:

[0067] A fully connected topology network is constructed, with all clients acting as nodes. Bidirectional transmission channels are established through directed edges, and each edge is assigned a dynamically updated contribution value to quantify the strength of cooperation among clients. The contribution value is essentially driven by the similarity of data distribution among clients. For client pairs with highly similar data distributions, their mutual contributions are significantly enhanced; conversely, the contributions between clients with high data heterogeneity are correspondingly diminished.

[0068] During the model aggregation phase, the server dynamically weights and aggregates client parameters based on contribution, generating a personalized global model for each client, expressed as follows:

[0069]

[0070] In the formula, w i Indicates client U i A personalized global model, where N represents the number of clients, R ij Indicates client U j For client U i Contribution, w j Indicates client U j The local model.

[0071] This mechanism breaks the hard boundary constraints of traditional intra-class aggregation through fully connected topology, making the training process more flexible and allowing clients to benefit globally.

[0072] First, consider how the server maintains the contribution relationship graph model R. Since the server cannot access the client's local data to obtain relevant data distribution information, it cannot directly calculate the data distribution similarity between clients. Therefore, other metrics are needed for measurement. This embodiment considers multiple metrics to comprehensively evaluate the contribution between clients. Specifically:

[0073] By integrating data scale, model accuracy, parameter similarity, and feature similarity, a contribution relationship graph model is created.

[0074] Dataset size p i This represents the proportion of the client's local dataset to the total global sample size. In heterogeneous data environments, the size of clients' local datasets often varies significantly, making it reasonable to consider clients with larger datasets as more reliable collaborators. Therefore, larger clients can be assigned a higher contribution score when evaluating contributions.

[0075] The data size is expressed by the following formula:

[0076]

[0077] Model accuracy a i Local model accuracy reflects the client model's ability to fit its own data distribution and can be obtained during model training with low computational complexity; therefore, this metric is included in the evaluation scope. However, it should be noted that in federated learning with heterogeneous data, local model accuracy may be misleading and incomparable due to local overfitting or data distribution shifts, leading to misjudgments of contribution. Therefore, this metric should not account for too much weight in the overall contribution evaluation.

[0078] Model accuracy is expressed by the following formula:

[0079]

[0080] In the formula, a i This indicates that the client U is being reflected. i The model's accuracy in fitting its own data distribution, Acc i Indicates client U i The accuracy of the local model.

[0081] Parameter similarity of local models When client-side local data distributions are similar, their models tend to converge to neighboring directions in the parameter space under the same optimization objective. In this case, cosine similarity can effectively capture the consistency of parameter directions. However, for clients with significantly different data distributions, their parameter update directions often show significant deviations. Therefore, it is reasonable to use the cosine similarity of local model parameters to measure the similarity of client-side data distributions. The core basis is that the direction of model parameters implicitly contains data distribution characteristics.

[0082] Parameter similarity is expressed by the following formula:

[0083]

[0084] In the formula, w represents the similarity of parameters that measure the distribution of client-side data. i Indicates client U i The local model, w j Indicates client U j The local model.

[0085] Feature similarity of local models Compared to parameter similarity of local models, feature similarity is less affected by local training hyperparameters (such as optimizer type and batch size), making it reasonable to use for evaluating the similarity of client data distributions. In real-world environments, it is common for servers to possess a small batch of common samples; therefore, these common samples can be used to perform inference on the client's local model and observe the similarity of the inference results.

[0086] Feature similarity is expressed by the following formula:

[0087]

[0088] In the formula, This indicates that the client's local model performs inference and observes the feature similarity of the inference results, c i and c j They represent client U respectively i and client U j The training hyperparameters of the local model, D g This represents the public sample set owned by the server.

[0089] The service dynamically constructs a contribution graph model R based on data scale, model accuracy, parameter similarity, and feature similarity. The optimization objective for the client is:

[0090]

[0091] Where γ1, γ2, and γ3 are weighting coefficients, γ1 + γ2 + γ3 = 1, and are adjusted according to actual conditions; p j Indicates client Uj The proportion of the local dataset size to the global sample size, a j This indicates that the client U is being reflected. j The model accuracy, which measures the fit to its own data distribution, is represented by R, which denotes the contribution graph model. When the number of clients is N, R ij Indicates client U j For client U i The degree of contribution.

[0092] Since the parameter similarity and inference similarity of the local model are strongly correlated, they are combined into a new similarity metric. This indicates the multiple similarity between two client parameters, all of which are s. ij This forms a similarity matrix s.

[0093] The first two terms in the above formula bring the contribution weights close to the normalized ratio of dataset size to local model accuracy, while the third term encourages the contribution weights to maintain consistency with the similarity of the client models. The constraints ensure that the collaboration weights for each client follow a normalized probability distribution, avoiding unbounded weights, and also restrict the weights to be non-negative. For ease of calculation, the above formula can be rearranged into the following form:

[0094]

[0095] Among them, R i This is the reorganized representation of the contribution relationship graph model; p represents the proportion of the client's local dataset size to the global sample size, a represents the model accuracy reflecting the client's ability to fit its own data distribution, and s i For client U j The similarity matrix is ​​calculated; T represents the number of server communications, and r represents the contribution between clients. Optimizing the contribution graph model using this method can achieve personalized model aggregation.

[0096] After obtaining the local model parameters uploaded by the client in each round of communication, the server optimizes the contribution relationship graph model and aggregates the server model.

[0097] The client trains a local model and introduces differential privacy into the parameter transmission process. Noise injection is used to ensure that the data transmitted by each client in each round of communication satisfies the differential privacy constraint. The privacy loss is calculated based on the number of samples used by the client in local training, and the upper bound of the privacy loss in parameter communication is obtained.

[0098] The objective expression for training a local model on the client side is:

[0099]

[0100] Among them, L i Let F represent the client's loss function. i D represents the local experience loss in traditional federated learning. i,j Represents the local dataset D i The j-th data point, E represents the number of local iterations on the client, B represents the batch size for training the model on the client; EB represents the number of samples used by the client in local training, λ is the balancing parameter, w represents the training model, w i Indicates client U i A personalized global model.

[0101] Through client-side local training optimization, the first term in the above equation represents the client adjusting the local model to minimize the task loss on the local dataset, and the second term is a regularization term that forces the direction of the local model to be consistent with the direction of the global model, avoiding the drift problem. When λ = 0, client-side local model training degenerates into traditional pure local optimization.

[0102] Considering the widespread privacy risks associated with parameter transmission in federated learning, this embodiment introduces differential privacy into the parameter transmission process. Noise injection ensures that the data transmitted by each client in each round of communication satisfies differential privacy constraints. Specifically, the process is as follows:

[0103] In each round of communication, the client injects Gaussian noise that conforms to the (ε,δ)-differential privacy constraint before uploading parameters, making the parameter distributions generated by any adjacent datasets indistinguishable and resisting member inference privacy attacks. The local model noise addition process is expressed by the following formula:

[0104]

[0105] in, Indicates client U i Gaussian noise in the local model Indicates client U i Local model parameters, Indicates the disturbed client U i The local model parameters; the server's aggregation process is represented as:

[0106]

[0107] Among them, w i,t This indicates the client U after introducing differential privacy. i A personalized global model, where N represents the number of clients. This indicates that after introducing differential privacy, the client U j For client U i Contribution Indicates the disturbed client U j Local model parameters, Indicates client U j Local model parameters, Indicates client U j The local model has Gaussian noise.

[0108] To determine the noise scale, client U i The local model parameters are pruned using a maximum pruning threshold C as a constraint. This operation is denoted as w. i =w i / max(1,||w i || / C);

[0109] Based on this, the parameter sensitivity is calculated as follows:

[0110]

[0111] Among them, D i With D′ i For adjacent datasets that differ by only one data record, L i Denotes the client's loss function, D i,j With D′ i,j Represents the neighboring dataset D in the local dataset. i With D′ i,j The j-th data; and This represents the local model in adjacent datasets that differ by only one data record; d i Represents dataset D i The number of samples included, w represents the training model.

[0112] In this embodiment, privacy protection is provided for the local data and model updates of the participants through model noise addition and privacy budget control mechanisms. The following analysis of privacy guarantees for personalized federated learning is conducted from the perspective of each round of communication and global training.

[0113] Based on the differential privacy Gaussian mechanism, in a single round of communication satisfying (ε,δ)-differential privacy, the expression for the noise scale added by the client to the parameters is:

[0114]

[0115] Where, σ i Indicates client U i The noise scale added to the parameters, ε represents the privacy budget, and δ is the tolerance for a low probability of privacy leakage, indicating that a certain degree of non-compliance with differential privacy is acceptable.

[0116] Client U iPerturbed local model parameters uploaded in a single communication All satisfy (ε,δ)-differential privacy. The privacy loss is calculated based on the number of samples used by the client during local training, and the upper bound of the privacy loss in parameter communication is obtained, including:

[0117] The client trains locally using stochastic gradient descent over multiple rounds. When the training between two rounds of communication does not utilize all local data samples, the sensitivity of the model parameters defined on the local dataset is Δs. i =2C / d i The model parameters satisfy (ε',δ')-differential privacy in a single round of communication. Considering privacy sampling amplification to estimate privacy loss, and correcting the noise scale added to the parameters by the client, the privacy parameters satisfied are:

[0118]

[0119] The corrected expression for the noise scale added to the parameters by the client is as follows:

[0120]

[0121] In the above formula, ε' and δ' represent privacy parameters, ε represents the privacy budget, i.e., the global privacy budget given by the client in a single round of communication; the client performs E iterations locally, with a batch data size of B used in each iteration, and EB represents the number of samples used by the client in local training. The total privacy loss of the client after multiple rounds of communication is affected by the data sampling rate q = EB / d. i .

[0122] Client U i Add noise to the local model according to the noise scale shown in the above formula and send it to the server, so that the model parameters satisfy (ε,δ)-differential privacy in each round of communication. Then, after T rounds of communication, obtain the upper bound of the privacy loss in parameter communication, that is, the total privacy loss of the client is expressed as:

[0123]

[0124] Where c0 is a constant;

[0125] The total privacy loss of the client after T rounds of communication is affected by the data sampling rate q and the number of communication times T. The experiment shows that the subsampling privacy amplification effect only takes effect on the privacy loss after T rounds of communication when the sampling rate q < 0.618. Thanks to the matrix accounting and subsampling privacy amplification, a tighter privacy loss boundary is obtained than that of traditional serial combination and advanced combination.

[0126] This embodiment further verifies the effectiveness of the personalized federated learning method based on the contribution graph model by combining it with real-world applications.

[0127] Experimental environment setup:

[0128] All experiments were run on a Windows 11 computer equipped with an Intel Core i5-12400F, an NVIDIA RTX 4070 GPU, and 32GB of RAM. The software environment was built using Python 3.10 and PyTorch 2.0.1.

[0129] To ensure the reliability of the experimental results, this embodiment uses several datasets commonly used in federated learning research, including MNIST, Fashion-MNIST, and CIFAR10. These three datasets are widely recognized image classification benchmark datasets in computer vision. The MNIST and Fashion-MNIST datasets contain 60,000 training samples and 10,000 test samples, respectively, in the form of 28×28 pixel single-channel grayscale images. The CIFAR10 dataset contains 50,000 training samples and 10,000 test samples, in the form of 32×32 pixel three-channel RGB images. The training model architecture uses a two-dimensional convolutional neural network, containing two convolutional layers with a kernel size of 5×5 (using ReLU activation and max pooling operations, respectively), and three fully connected layers (with 120, 84, and 10 neurons respectively). The basic training parameters are set as follows: 40 local training iterations, learning rate 0.01, batch size 64, weight coefficients γ1 = 0.3, γ2 = 0.1, γ3 = 0.6, and balancing parameter λ = 0.5.

[0130] The experiment employed three data partitioning strategies: independent and identically distributed (IID) partitioning, ill-conditioned non-independent and identically distributed (IID) partitioning, and Dirichlet distribution-based partitioning. IID partitioning indicates that the training data is uniformly distributed across all clients; ill-conditioned non-independent and identically distributed partitioning means that each client contains data for only n specific classes, with the remaining classes completely missing (denoted as PD-n below); Dirichlet distribution-based partitioning more closely resembles the disordered distribution in the real world and is controlled by the hyperparameter 'a', with smaller 'a' indicating a more skewed data distribution (denoted as DD-a below). These partitioning schemes cover a continuous spectrum of data distributions from ideal equilibrium to true disorder, effectively validating the performance of personalized federated learning methods in heterogeneous data environments.

[0131] The baseline algorithms for comparison include: (1) Local: The client only relies on local data to complete the model training, without a federated aggregation process; (2) FedAvg: The classic federated learning method, which adopts federated average aggregation; (3) FedProx: Based on FedAvg, a proximal regularization term is introduced to prevent the local model update from deviating too far from the global model, so as to improve the convergence stability in heterogeneous data environments; (4) CFL: A personalized federated learning method based on model gradient information for client clustering; (5) Per-FedAvg: A personalized federated learning method based on meta-learning; (6) DP-FedAvg: A federated learning algorithm based on differential privacy, which integrates a Gaussian noise injection mechanism in the FedAvg framework; (7) DP-SCAFFOLD: A privacy-preserving federated learning algorithm based on the SCAFFOLD algorithm, which adds control variables to reduce user bias and thus adapt to heterogeneous data environments, while also using differential privacy for privacy protection. All algorithms used the same client data partitioning when participating in the comparison. Each group of experiments was repeated under three independent random seeds, and the data mean was used as the final statistical result.

[0132] Three heterogeneous data distribution scenarios were constructed: Independent and Identically Distributed (IID), Ill-conditioned Non-Independent and Identically Distributed (PD-2), and Dirichlet Distribution (DD-0.1). Experiments were conducted on the MNIST, Fashion-MNIST, and CIFAR-10 datasets, with 10 clients participating in training for each dataset and 50 rounds of federated communication. All comparison algorithms used the same dataset partitioning. For the differential privacy-enhancing algorithm, a global privacy budget ε = 10 and a pruning threshold C = 5 were uniformly set. Statistical results are shown in Table 1, comparing the accuracy of each algorithm model under different datasets and data distribution conditions.

[0133] Table 1. Comparison of accuracy (%) of various algorithm models under different datasets and data distributions.

[0134]

[0135] Table 1 shows the classification accuracy of each algorithm across datasets and scenarios, with the personalized federated learning method using the average accuracy of the personalized model. CRFL represents the base version of DPCRFL without integrated differential privacy mechanism, i.e., the control group with ablation differential privacy module, designed to observe the performance of the personalized federated learning algorithm proposed in this chapter and the impact of privacy protection mechanisms on model performance.

[0136] As shown in Table 1, among algorithms without differential privacy protection, CRFL achieved the highest accuracy in most cases, with an average of 7.53% and 7.76% higher accuracy than non-personalized federated learning FedAvg and FedProx, and an average of 1.05% and 2.06% higher accuracy than personalized federated learning CFL and Per-FedAvg. This indicates that personalized federated learning methods based on contribution graph models have a significant performance improvement over existing methods.

[0137] It should be noted that any parts not covered in this invention are the same as or can be implemented using existing technology. The above description is merely a preferred embodiment of this invention, but the scope of protection of this invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in this invention, based on the technical solution and inventive concept of this invention, should be covered within the scope of protection of this invention.

Claims

1. A privacy-preserving personalized federated learning method based on a contribution graph model, characterized in that, include: A collaborative training aggregation mechanism based on dynamic contribution is proposed, in which the server dynamically weights and aggregates client parameters according to contribution, generating a personalized global model for each client; By integrating data scale, model accuracy, parameter similarity, and feature similarity, a contribution relationship graph model is created. The client trains a local model and introduces differential privacy into the parameter transmission process. Noise injection is used to ensure that the data transmitted by each client in each round of communication satisfies the differential privacy constraint. The privacy loss is calculated based on the number of samples used by the client in local training, and the upper bound of the privacy loss in parameter communication is obtained.

2. The privacy-preserving personalized federated learning method based on a contribution graph model according to claim 1, characterized in that, The proposed collaborative training aggregation mechanism based on dynamic contribution is as follows: Construct a fully connected topology network, with all clients acting as nodes. Establish bidirectional transmission channels through directed edges, and assign dynamically updated contribution values ​​to each edge to quantify the strength of cooperation between clients. During the model aggregation phase, the server dynamically weights and aggregates client parameters based on contribution, generating a personalized global model for each client, expressed as follows: In the formula, w i Indicates client U i A personalized global model, where N represents the number of clients, R ij Indicates client U j For client U i Contribution, w j Indicates client U j The local model.

3. The privacy-preserving personalized federated learning method based on a contribution graph model according to claim 1, characterized in that, A contribution graph model is created by integrating data scale, model accuracy, parameter similarity, and feature similarity; among which, The data size is expressed by the following formula: In the formula, p i Indicates client U i The proportion of the local dataset size to the global sample size, with each client possessing a local dataset D. i , let d i =|D i | represents dataset D i The sample size included; The accuracy of the model is expressed by the following formula: In the formula, a i This indicates that the client U is being reflected. i The model's accuracy in fitting its own data distribution, Acc i Indicates client U i The accuracy of the local model; The parameter similarity is expressed by the following formula: In the formula, w represents the similarity of parameters that measure the distribution of client-side data. i Indicates client U i The local model, w j Indicates client U j The local model; The feature similarity is expressed by the following formula: In the formula, This indicates that the client's local model performs inference and observes the feature similarity of the inference results, c i and c j They represent client U respectively i and client U j The training hyperparameters of the local model, D g This represents the public sample set owned by the server. The service dynamically constructs a contribution graph model R based on data scale, model accuracy, parameter similarity, and feature similarity. The optimization objective for the client is: Where γ1, γ2, and γ3 are all weighting coefficients, p j Indicates client U j The proportion of the local dataset size to the global sample size, a j This indicates that the client U is being reflected. j The model accuracy, which measures the fit to its own data distribution, is represented by R, which denotes the contribution graph model. When the number of clients is N, R ij Indicates client U j For client U i The degree of contribution; After obtaining the local model parameters uploaded by the client in each round of communication, the server optimizes the contribution relationship graph model and aggregates the server model.

4. The privacy-preserving personalized federated learning method based on a contribution graph model according to claim 1, characterized in that, The client trains a local model, and the training optimization objective expression is: Among them, L i Let F represent the client's loss function. i D represents the local experience loss in traditional federated learning. i,j Represents the local dataset D i The j-th data point, E represents the number of local iterations on the client, B represents the batch size for training the model on the client; EB represents the number of samples used by the client in local training, λ is the balancing parameter, w represents the training model, w i Indicates client U i Personalized global model; Through local training optimization on the client side, the client adjusts the local model to minimize the task loss of the local model on the local dataset, and the direction of the local model is consistent with the direction of the global model.

5. The privacy-preserving personalized federated learning method based on a contribution graph model according to claim 1, characterized in that, The introduction of differential privacy into the parameter transmission process, through noise injection, ensures that the transmitted data of each client in each round of communication satisfies differential privacy constraints. The process is as follows: In each round of communication, the client injects Gaussian noise that conforms to the (ε,δ)-differential privacy constraint before uploading parameters, making the parameter distributions generated by any adjacent datasets indistinguishable and resisting member inference privacy attacks. The local model noise addition process is expressed by the following formula: in, Indicates client U i Gaussian noise in the local model Indicates client U i Local model parameters, Indicates the disturbed client U i The local model parameters; the server's aggregation process is represented as: Among them, w i,t This indicates the client U after introducing differential privacy. i A personalized global model, where N represents the number of clients. This indicates that after introducing differential privacy, the client U j For client U i Contribution Indicates the disturbed client U j Local model parameters, Indicates client U j Local model parameters, Indicates client U j Gaussian noise in the local model; To determine the noise scale, client U i The local model parameters are pruned using a maximum pruning threshold C as a constraint. This operation is denoted as w. i =w i / max(1,||w i || / C); Based on this, the parameter sensitivity is calculated as follows: Among them, D i With D' i For adjacent datasets that differ by only one data record, L i Denotes the client's loss function, D i,j With D' i,j Represents the neighboring dataset D in the local dataset. i With D' i,j The j-th data; and This represents the local model in adjacent datasets that differ by only one data record; d i Represents dataset D i The number of samples included, w represents the training model. Based on the differential privacy Gaussian mechanism, in a single round of communication satisfying (ε,δ)-differential privacy, the expression for the noise scale added by the client to the parameters is: Where, σ i Indicates client U i The noise scale added to the parameters, ε represents the privacy budget, and δ is the tolerance for a low probability of privacy leakage.

6. The privacy-preserving personalized federated learning method based on a contribution graph model according to claim 5, characterized in that, The step of calculating the privacy loss based on the number of samples used by the client during local training, and obtaining the upper bound of the privacy loss in parameter communication, includes: The client trains locally using multiple rounds of stochastic gradient descent. When the training between two rounds of communication does not utilize all local data samples, privacy loss is estimated by considering privacy sampling amplification. When correcting the noise scale added to the parameters by the client, the privacy parameters that are satisfied are: The corrected expression for the noise scale added to the parameters by the client is as follows: In the above formula, ε' and δ' represent privacy parameters, ε represents the privacy budget, the client performs E iterations locally, the batch data size used in each iteration is B, EB represents the number of samples used by the client in local training, and the total privacy loss of the client after multiple rounds of communication is affected by the data sampling rate q = EB / d. i ; Client U i Add noise to the local model according to the noise scale shown in the above formula and send it to the server, so that the model parameters satisfy (ε,δ)-differential privacy in each round of communication. Then, after T rounds of communication, obtain the upper bound of the privacy loss in parameter communication, that is, the total privacy loss of the client is expressed as: Where c0 is a constant; The total privacy loss of the client after T rounds of communication is affected by the data sampling rate q and the number of communication rounds T.

Images