Privacy-enhanced adaptive clustering federated learning method and system for heterogeneous resources

Abstract

The present invention relates to a privacy-enhanced adaptive clustering federated learning method and system for heterogeneous resources, and belongs to the technical field of federated learning. It includes: (1) clustering of participants based on uplink latency; clustering heterogeneous devices using different clustering standards; (2) local training within clusters with similar computing power; clustering users according to device computing power to optimize training efficiency; (3) secure cluster model aggregation based on spectral embedding; using soft clustering based on gradient vector similarity to allow user models to be aggregated into multiple cluster models, and using multiple cluster models to update local models. The present invention ensures privacy and confidentiality by shuffling the user's gradient vector during the clustering process. In addition, the method uses homomorphic encryption and bilinear aggregate signatures to verify user identity and protect gradient sharing.

Description

Technical Field

[0001] The present invention relates to a privacy-enhanced adaptive clustering federated learning method and system for heterogeneous resources, and belongs to the technical field of federated learning. Background Art

[0002] Thanks to the empowerment of big data, deep learning research has made significant progress. However, in the real world, data is often distributed among different entities (such as smart devices or organizations). Due to privacy issues and data protection regulations, it is difficult for participants to aggregate scattered data for model training. Federated Learning (FL) is an emerging machine learning paradigm that can train models on multiple devices without sharing original data. Federated Learning has been applied to multiple industries, including medical image analysis and autonomous driving.

[0003] In federated learning research, resource heterogeneity is an inevitable problem. First, the clients participating in federated learning are usually highly heterogeneous in terms of hardware, which leads to relative variations in computing speed between clients. Assuming that each client runs Round local update stochastic gradient descent (SGD) with a batch size of Therefore, for The number of local iterations for a client with local samples is , which may vary greatly between clients. The relative variation in client computation speed also exacerbates the heterogeneity of the number of local SGD iterations, especially for synchronous federated learning, which often leads to a lag effect (especially on large-scale heterogeneous devices). Asynchronous federated learning solves this problem, but more frequent communication between relatively more powerful devices and servers may cause server crashes. Another unavoidable and practical problem is data heterogeneity. In actual scenarios, the data of different participants is likely to be non-independent and identically distributed (Non-IID) or unbalanced, which greatly affects the training accuracy of the final model. Current studies have shown that adding proximal terms to the model can effectively reduce the model convergence offset caused by heterogeneous data training. However, imposing constraints on local models will lead to slower model convergence and insufficient utilization of local data. In particular, when applied to areas where there is no obvious difference in distribution, privacy models tend to show significant performance degradation.

[0004] To solve the above problems, researchers adopted a clustering-based solution based on FL. Clustering algorithms help group participants according to their characteristics or data distribution, ensuring that clients within a cluster exhibit similar resource attributes. However, clustering-based FL also has some problems:

[0005] (1) Clustering clients based on a single criterion. For example, when the data is not independent and identically distributed, clustering clients based solely on computing power may improve efficiency but fail to optimize model quality.

[0006] (2) Similarities between clients are leaked to the service provider, which can be further used to infer the privacy of other users.

[0007] (3) The number of clusters as a hyperparameter will affect model performance, and static configuration cannot adapt to changes in resources during training. Summary of the invention

[0008] In view of the shortcomings of the prior art, the present invention proposes a privacy-enhanced adaptive clustering federated learning method for heterogeneous resources, aiming to solve the technical problems of lagging effect and model degradation caused by resource heterogeneity in clustering federated learning and leakage of user similarity during the clustering process. The method of the present invention adopts an adaptive clustering strategy to cluster heterogeneous devices by applying different clustering criteria, separating training efficiency from model performance. Specifically, in the local training stage, users are clustered according to the computing power of the device to optimize the training efficiency. In the model aggregation stage, soft clustering based on gradient vector similarity is adopted to allow user models to be aggregated into multiple clustering models, and these multiple clustering models are used to update the local model. For privacy issues, the method of the present invention integrates homomorphic encryption and bilinear aggregate signatures to ensure the confidentiality and integrity of data.

[0009] In the present invention, the local training data stored on the user device are all non-independent and identically distributed, and there are differences in network environment and computing power between different devices. The present invention uses uplink time to quantify the difference, and the uplink time refers to the duration of the device from local training to parameter uploading. The core idea is that the impact of clustering based on a single standard on training efficiency and model performance is coupled. The sensitivity of training efficiency and model performance to different stages is different. Therefore, the effect can be decoupled by adopting different clustering standards at different stages. In order to decouple the effects discussed above, different characteristics of heterogeneous resources are integrated at each stage of FL. Before local training, the present invention clusters participants based on uplink time to minimize idle waiting time. In order to improve the overall model performance, the present invention adopts a spectral embedding method to allow user models to participate in model aggregation of multiple clusters through adaptive affinity budgets. In addition, the present invention disrupts the gradients submitted to the computing server to enhance privacy and prevent CSP from identifying the similarity of customers.

[0010] The technical solution of the present invention is:

[0011] Privacy-enhanced adaptive clustering federated learning method for heterogeneous resources, including:

[0012] (1) Clustering of participants based on uplink latency; clustering heterogeneous devices using different clustering criteria;

[0013] (2) Local training within clusters with similar computing power: cluster users according to their device computing power to optimize training efficiency;

[0014] (3) Secure cluster model aggregation based on spectral embedding: soft clustering based on gradient vector similarity is adopted to allow user models to be aggregated into multiple cluster models, and multiple cluster models are used to update the local model.

[0015] Preferably, according to the present invention, the clustering of participants based on uplink delay includes:

[0016] The key generation center KGC selects security parameters , using the key generation function Generate asymmetric key pairs for the cloud service provider CSP and all users in the Paillier cryptosystem respectively and ; All users share a pair of keys ;

[0017] KGC broadcast bilinear aggregate signature parameters and curve hash functions , where G1 and G2 are two prime orders on any elliptic curve. The added cyclic group of It is the elemental level The multiplicative cyclic group of Group The generator of are two points on the curve, ,and is a bilinear map;

[0018] KGC uses a key generation function Generate a signing key pair for the CSP , KGC generates a signature key pair for the service server BS , KGC generates a signature key pair for each user in , is the number of participants in this round of training; BS sends initialization parameters to each participant and unique identifier , To initialize the gradient, Indicates the identity of each participant.

[0019] Preferably, according to the present invention, local training within a cluster with similar computing power includes:

[0020] At the beginning of each round of training, the BS receives the uplink latency from the devices participating in the round, including the participants Time to complete local training and the communication delay with the BS ;

[0021] BS first uses the K-means clustering algorithm to divide the devices into Classes: ,in, It is the class with the shortest latency. This is the class with the longest delay; every time interval , the server aggregates the local models received from all devices to complete a round of training;

[0022] Select a subset of the local dataset to train the local model and get the local gradient as follows:

[0023] ▽ ;

[0024] in, is the local loss function, Indicates the number of rounds, represents the local learning rate; represents the model parameters uploaded by user i in round r, Similarly, ▽ is the Laplace operator;

[0025] Encode the encrypted gradient vector as an integer as follows:

[0026] ;

[0027] in, Indicates the encoding accuracy, Indicates the encoding length;

[0028] Using CSP's public key encryption , as shown below:

[0029] ;

[0030] in, Indicates the party using the public key of the CSP for encryption Uploaded gradient refers to the encryption function in Plliar, Generate data tuples ,in, is the timestamp;

[0031] calculate Digital signature , and Send to BS.

[0032] Preferably, according to the present invention, the secure cluster model aggregation based on spectrum embedding includes:

[0033] BS receives encrypted gradient and the corresponding signature After that, verification is performed, and the verification equation is:

[0034] ;

[0035] if Failed to pass verification, The uploaded data will be rejected;

[0036] if Can pass verification, BS randomly selects blinding factor , get the ciphertext set of blinded gradients , as shown below:

[0037] ;

[0038] in, Represents the blinding factor encrypted using the public key of the CSP The ciphertext, then BS selects a permutation function :

[0039] ;

[0040] Will gather Disrupt, get ,in, , Indicates the current computational domain;

[0041] BS sets the perturbed gradient vector and the corresponding signature Send to CSP;

[0042] CSP decrypts the blurred gradient vector with a valid signature:

[0043] ;

[0044] in, is the private key of CSP, is the decryption function, represents a blurred gradient vector with a valid signature;

[0045] CSP calculates the gradient vector The similarity matrix , as shown below:

[0046] in, is the Gaussian radial basis function, Respectively represent the participants The fuzzy gradient vector with a valid signature is the scale parameter that determines the width of the RBF;

[0047] The similarity is quantized by a Gaussian radial basis function between the perturbed gradients as follows:

[0048] ;

[0049] in, Respectively represent the participants The similarity of is an exponential function;

[0050] CSP performs spectral clustering on the gradient vectors to obtain cluster labels, indicating the clusters they belong to;

[0051] Then, the cluster label matrix is ​​calculated using the spectral embedding method ;

[0052] Distance budget for The average of the distances to all clusters; if ,but Assigned to cluster , ; CSP clusters the label matrix and Send to BS;

[0053] BS uses the following formula to calculate the set Unblinding:

[0054] ;

[0055] cluster The aggregation formula inside is:

[0056] ;

[0057] in, Is the current round Middle Cluster The cluster model, It is a cluster The number of participating devices;

[0058] For clusters Members of The real subscript of ; BS will Distribute to clients , ;

[0059] user Verify the received clustering gradient vector and decrypt the verified clustering gradient vector:

[0060] ;

[0061] Update the local model using the average of the verified parameters as follows:

[0062] .

[0063] Preferably, according to the present invention, CSP performs spectral clustering on the gradient vector to obtain cluster labels; comprising:

[0064] First, construct the degree matrix ;

[0065] Then, calculate the Laplacian matrix ,right Normalized to get:

[0066] ;

[0067] calculate The characteristic matrix :

[0068] ;

[0069] in, is the number of clusters;

[0070] Finally, Use K-means algorithm to cluster the sample set and get the cluster label set .

[0071] According to the preferred embodiment of the present invention, the clustering label matrix is ​​calculated by using the spectral embedding method ;include:

[0072] Calculate the similarity matrix The characteristic matrix :

[0073]

[0074] and standardize;

[0075] For each cluster , calculate the cluster center ,in, ;

[0076] Calculate the cluster label matrix , cluster label matrix The matrix elements are , express With cluster center distance.

[0077] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a privacy-enhanced adaptive clustering federated learning method for heterogeneous resources when executing the computer program.

[0078] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a privacy-enhanced adaptive clustering federated learning method for heterogeneous resources.

[0079] Privacy-enhanced adaptive clustering federated learning system for heterogeneous resources, including:

[0080] The participant clustering module is configured to: cluster participants based on uplink latency; cluster heterogeneous devices using different clustering criteria;

[0081] The local training module is configured to: local training within clusters with similar computing power; clustering users according to device computing power to optimize training efficiency;

[0082] The security cluster model aggregation module is configured as follows: security cluster model aggregation based on spectral embedding; using soft clustering based on gradient vector similarity to allow user models to be aggregated into multiple cluster models, and using multiple cluster models to update the local model.

[0083] The beneficial effects of the present invention are:

[0084] 1. The method of the present invention clusters devices according to computing power to alleviate resource heterogeneity before local training. This solution implements spectral embedding technology in the model aggregation stage to enhance model performance.

[0085] 2. The present invention ensures privacy and confidentiality by shuffling the user's gradient vectors during the clustering process. In addition, the method uses homomorphic encryption and bilinear aggregate signatures to verify user identities and protect gradient sharing. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 It is a flow chart of the privacy-enhanced adaptive clustering federated learning method for heterogeneous resources of the present invention;

[0087] Figure 2This is a comparison chart of the model performance of the method of the present invention and other algorithms under different data distributions on the MNIST dataset;

[0088] Figure 3 A comparison chart of the model performance of the method of the present invention and other algorithms under different data distributions on the Fashion-MNIST dataset;

[0089] Figure 4 This is a comparison chart of the model performance of the method of the present invention and other algorithms under different data distributions on the CIFAR-10 dataset;

[0090] Figure 5 This is a comparison chart of the model performance of the method of the present invention and other algorithms under different data distributions on the CIFAR-100 dataset. DETAILED DESCRIPTION

[0091] The present invention will be further defined below in conjunction with the accompanying drawings and embodiments, but is not limited thereto.

[0092] Example 1

[0093] Privacy-enhanced adaptive clustering federated learning method for heterogeneous resources, including:

[0094] (1) Clustering of participants based on uplink latency; clustering heterogeneous devices using different clustering criteria;

[0095] (2) Local training within clusters with similar computing power: cluster users according to their device computing power to optimize training efficiency;

[0096] (3) Secure cluster model aggregation based on spectral embedding: soft clustering based on gradient vector similarity is adopted to allow user models to be aggregated into multiple cluster models, and multiple cluster models are used to update the local model.

[0097] Example 2

[0098] The privacy-enhanced adaptive clustering federated learning method for heterogeneous resources described in Example 1 is different in that:

[0099] like Figure 1 As shown, the participants are clustered based on the uplink delay; including:

[0100] The key generation center KGC selects security parameters , using the key generation function Generate asymmetric key pairs for the cloud service provider CSP and all users in the Paillier cryptosystem respectively and ; All users share a pair of keys ;

[0101] KGC broadcast bilinear aggregate signature parameters and curve hash functions , where G1 and G2 are two prime orders on any elliptic curve. The added cyclic group of It is the elemental level The multiplicative cyclic group of Group The generator of are two points on the curve, ,and is a bilinear map;

[0102] KGC uses a key generation function Generate a signing key pair for the CSP , KGC generates a signature key pair for the service server BS , KGC generates a signature key pair for each user in , is the number of participants in this round of training; BS sends initialization parameters to each participant and unique identifier , To initialize the gradient, Indicates the identity of each participant.

[0103] The key generation center KGC is an independent trusted institution responsible for the generation, distribution and management of key pairs. The cloud service provider CSP uses a clustering method to organize users according to the private similarity of the gradients uploaded by the BS, thereby realizing the aggregation of the clustering model. Users represent edge devices with limited computing power and non-independent and identically distributed data, and interact specifically with the service provider to upload local gradients and receive aggregated results. The business server BS clusters devices according to computing power, confuses and shuffles the encrypted gradients, and propagates the aggregated results to users.

[0104] Local training within a cluster with similar computing power; including:

[0105] At the beginning of each round of training, the BS receives the uplink latency from the devices participating in the round, including the participants Time to complete local training and the communication delay with the BS ;

[0106] BS first uses the K-means clustering algorithm to divide the devices into Classes: ,in, It is the class with the shortest latency. This is the class with the longest delay; every time interval , the server aggregates the local models received from all devices to complete a round of training;

[0107] Select a subset of the local dataset to train the local model and get the local gradient as follows:

[0108] ▽ ;

[0109] in, is the local loss function, Indicates the number of rounds, represents the local learning rate; represents the model parameters uploaded by user i in round r, Similarly, ▽ is the Laplace operator;

[0110] Encode the encrypted gradient vector as an integer as follows:

[0111] ;

[0112] in, Indicates the encoding accuracy, Indicates the encoding length;

[0113] Using CSP's public key encryption , as shown below:

[0114] ;

[0115] in, Indicates the party using the public key of the CSP for encryption Uploaded gradient refers to the encryption function in Plliar, Generate data tuples ,in, is the timestamp;

[0116] calculate Digital signature , and Send to BS.

[0117] Security cluster model aggregation based on spectral embedding; including:

[0118] The privacy-enhanced spectral clustering based on similarity is implemented as Algorithm 1, as shown in Table 1.

[0119] Table 1 Algorithm 1;

[0120]

[0121] BS receives encrypted gradient and the corresponding signature After that, verification is performed, and the verification equation is:

[0122] ;

[0123] if Failed to pass verification, The uploaded data will be rejected;

[0124] if Can pass verification, BS randomly selects blinding factor , get the ciphertext set of blinded gradients , as shown below:

[0125] ;

[0126] in, Represents the blinding factor encrypted using the public key of the CSP The ciphertext, then BS selects a permutation function :

[0127] ;

[0128] Will gather Disrupt, get ,in, , Indicates the current computational domain;

[0129] BS sets the perturbed gradient vector and the corresponding signature Send to CSP;

[0130] CSP decrypts the blurred gradient vector with a valid signature:

[0131] ;

[0132] in, is the private key of CSP, is the decryption function, represents a blurred gradient vector with a valid signature;

[0133] CSP calculates the gradient vector The similarity matrix , as shown below:

[0134] in, is the Gaussian radial basis function, Respectively represent the participants The fuzzy gradient vector with a valid signature is the scale parameter that determines the width of the RBF;

[0135] The similarity is quantified by a Gaussian radial basis function (RBF) between the perturbed gradients as follows:

[0136] ;

[0137] in, Respectively represent the participants The similarity of is an exponential function;

[0138] CSP performs spectral clustering on the gradient vectors to obtain cluster labels, indicating the cluster they belong to (Lines 2-8 in Algorithm 1);

[0139] Then, the cluster label matrix is ​​calculated using the spectral embedding method ;

[0140] In order for the gradient vector to participate in the model aggregation of multiple clusters, Distance budget for The average of the distances to all clusters; if ,but Assigned to cluster , ; CSP clusters the label matrix and Send to BS;

[0141] BS uses the following formula to calculate the set Unblinding:

[0142] ;

[0143] In this phase, the same weight is assumed according to the number of samples of each device to avoid weight errors. BS aggregates the local models within the perturbed clusters in the similarity-based privacy-enhanced spectral clustering algorithm 2. Cluster The aggregation formula inside is:

[0144] ;

[0145] in, Is the current round Middle Cluster The cluster model, It is a cluster The number of participating devices;

[0146] In order to correctly distribute the aggregate gradients, BS needs to recover the subscripts of the cluster labels. Members of The real subscript of ; BS will Distribute to clients , ;

[0147] user Verify the received clustering gradient vector and decrypt the verified clustering gradient vector:

[0148] ;

[0149] Update the local model using the average of the verified parameters as follows:

[0150] .

[0151] CSP performs spectral clustering on the gradient vectors to obtain cluster labels; including:

[0152] First, construct the degree matrix ;

[0153] Then, calculate the Laplacian matrix ,right Normalized to get:

[0154] ;

[0155] calculate The characteristic matrix :

[0156] ;

[0157] in, is the number of clusters;

[0158] Finally, Use K-means algorithm to cluster the sample set and get the cluster label set .

[0159] Calculate the cluster label matrix using spectral embedding method ;include:

[0160] Calculate the similarity matrix The characteristic matrix :

[0161] ;

[0162] and standardize;

[0163] For each cluster , calculate the cluster center ,in, ;

[0164] Calculate the cluster label matrix , cluster label matrix The matrix elements are , express With cluster center distance.

[0165] The scheme of the present invention is evaluated using the datasets MNIST, Fashion-MNIST, CIFAR-10 and CIFAR100. MNIST, CIFAR10 and Fashion-MNIST all contain 10 categories for classification. CIFAR-100 is expanded to 100 categories based on CIFAR-10, and each category has 600 color images. Dirichlet distribution (denoted as ) to simulate the non-independent and identically distributed situation between the client local data sets, and the smaller Indicates greater heterogeneity.

[0166] To evaluate the effect of the number of clusters on model accuracy and convergence speed, experiments were conducted on the MNIST and CIFAR10 datasets. Figure 2 This is a comparison chart of the model performance of the method of the present invention and other algorithms under different data distributions on the MNIST dataset; Figure 3 A comparison chart of the model performance of the method of the present invention and other algorithms under different data distributions on the Fashion-MNIST dataset; Figure 4 This is a comparison chart of the model performance of the method of the present invention and other algorithms under different data distributions on the CIFAR-10 dataset; Figure 5 This is a comparison chart of the model performance of the method of the present invention and other algorithms under different data distributions on the CIFAR-100 dataset. Under the setting of , the effect of changing the number of clusters on the model accuracy. It can be seen that when When constant, the final model accuracy increases as The increase first increases and then decreases. The highest accuracy is achieved when the number of clusters is too small. If the number of clusters is too small, the heterogeneity is reduced. The values ​​within each cluster remain high, resulting in a decrease in model accuracy. Conversely, if the number of clusters is too large, the number of devices within each cluster is insufficient. This reduces the amount of device-specific data available for parameter updates, affecting the final training accuracy. and When is varied simultaneously, the final model accuracy shows a similar trend. The increase in will result in fewer gradient updates in a single cycle, thus reducing the In addition, the increase It can improve the communication efficiency of devices and thus improve the accuracy of the model. When fixed, The change of does not significantly affect the accuracy of the final model, which is consistent with the theoretical analysis.

[0167] A multi-layer perceptron architecture with 2 hidden layers is used for MNIST and Fashion-MNIST. The number of neurons in the layers is 200 and 100, respectively. For CIFAR-10 and CIFAR-100, a convolutional neural network model consisting of two 5 × 5 convolutional layers, each followed by two 2 × 2 pooling layers, a fully connected layer of 512 units activated by ReLu, and a softmax layer is used.

[0168] To simulate the heterogeneity of devices among users, we first set the maximum number of local training rounds. For each user, from Random selection within range A number is used to simulate the client device uplink time, recorded as Then, the number of local training rounds for each user is calculated as In the present invention, the setting , Distributed in The learning rate for different data sets is different. The learning rate on MNIST is , and the learning rate η on CIFAR is 0.001. The simulation has 100 users participating in the federated learning training.

[0169] Example 3

[0170] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the privacy-enhanced adaptive clustering federated learning method for heterogeneous resources described in Example 1 or 2 are implemented.

[0171] Example 4

[0172] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the privacy-enhanced adaptive clustering federated learning method for heterogeneous resources described in Example 1 or 2.

[0173] Example 5

[0174] Privacy-enhanced adaptive clustering federated learning system for heterogeneous resources, including:

[0175] The participant clustering module is configured to: cluster participants based on uplink latency; cluster heterogeneous devices using different clustering criteria;

[0176] The local training module is configured to: local training within clusters with similar computing power; clustering users according to device computing power to optimize training efficiency;

[0177] The security cluster model aggregation module is configured as follows: security cluster model aggregation based on spectral embedding; using soft clustering based on gradient vector similarity to allow user models to be aggregated into multiple cluster models, and using multiple cluster models to update the local model.

Claims

1. A privacy-enhanced adaptive clustering federated learning method for heterogeneous resources, characterized in that: include: (1) Clustering of participants based on uplink latency; clustering heterogeneous devices using different clustering criteria; (2) Local training within clusters with similar computing power; Cluster users according to device computing power to optimize training efficiency; (3) Security cluster model aggregation based on spectral embedding; Adopt soft clustering based on gradient vector similarity, allowing user models to be aggregated into multiple cluster models, and use multiple cluster models to update the local model; Local training within a cluster with similar computing power; including: At the beginning of each round of training, the BS receives the uplink latency from the devices participating in the round, including the participants U i Time t to complete local training i and the communication delay D with the BS i ; BS first uses the K-means clustering algorithm to divide the devices into τ classes according to the uplink delay: {K1, ..., K τ }, where K1 is the class with the shortest delay, K τ This is the class with the longest latency. Every time interval R, the server aggregates the local models received from all devices to complete a round of training. U i Select a subset of the local dataset to train the local model and get the local gradient as follows: Where l(·) is the local loss function, r is the number of rounds, and η is the local learning rate; represents the model parameters uploaded by user i in round r, Similarly, ▽ is the Laplace operator; Encode the encrypted gradient vector as an integer as follows: Among them, ρ represents the coding accuracy, and m represents the coding length; U i Using CSP's public key encryption As shown below: in, represents the gradient uploaded by participant i encrypted with the public key of CSP, Refers to the encryption function in Plliar, U i Generate data tuples Among them, TS is the timestamp; U i calculate Digital signature and will Send to BS.

2. The privacy-enhanced adaptive clustering federated learning method for heterogeneous resources according to claim 1, characterized in that: Participant clustering based on uplink latency; including: The key generation center KGC selects the security parameter κ and uses the key generation function KeyGen(κ) to generate asymmetric key pairs (pk c ,sk c ) and (pk u ,pk u ); all users share a pair of keys (pk u ,sk u ); KGC broadcasts bilinear aggregate signature parameters (e, g1, g2, G1, G2) and curve hash function H(·): {0, 1} * →G1, where G1 and G2 are two prime order q-added cyclic groups on any elliptic curve, G T is a multiplicative cyclic group of prime order q, g1 and g2 are generators of groups G1 and G2 respectively; let P and Q be two points on the curve, b∈Z q , and e∶G1×G2→G T is a bilinear map; KGC uses the key generation function KeyGen(κ) to generate a signature key pair (Ψ c ,λ c ), KGC generates a signature key pair for the service server BS (Ψ b ,λ b ), KGC generates a signature key pair for each user (Ψ i ,λ i ), where 1≤i≤n, n is the number of participants in this round of training; BS sends an initialization parameter ω to each participant init and unique identifier UID i ∈[1,n],ω init To initialize the gradient, UID i ∈[1,n] represents the identity of each participant.

3. The privacy-enhanced adaptive clustering federated learning method for heterogeneous resources according to claim 1, characterized in that: Security cluster model aggregation based on spectral embedding; including: BS receives encrypted gradient and the corresponding signature After that, verification is performed, and the verification equation is: if Failed to pass verification, U i The uploaded data will be rejected; if Can be verified, BS randomly selects the blinding factor μ r , get the ciphertext set of blinded gradients As shown below: in, represents the blinding factor μ encrypted using the public key of CSP r , then BS chooses a permutation function π(·): Will gather Disrupt, get Where j = π(i)∈U, U represents the current computational domain; BS sets the perturbed gradient vector and the corresponding signature Send to CSP; CSP decrypts the blurred gradient vector with a valid signature: Among them, sk c is the private key of CSP, Dec() is the decryption function, represents a blurred gradient vector with a valid signature; CSP calculates the similarity matrix S of the gradient vector B as follows: S←GaussianRBF(B i′ ,B j ,c),1≤i′,j≤n; Among them, GaussianRBF() is the Gaussian radial basis function, B i′ ,B j , respectively represent the fuzzy gradient vector with valid signature of participant i′, j; γ is the scale parameter that determines the width of RBF; The similarity is quantized by a Gaussian radial basis function between the perturbed gradients as follows: Among them, ρ i′,j Respectively represent the similarity of participants i′, j, exp() is an exponential function; CSP performs spectral clustering on the gradient vectors to obtain cluster labels, indicating the clusters they belong to; Then, the cluster label matrix is ​​calculated using the spectral embedding method B j Distance budget For B j The average of the distances to all clusters; if Then B j Assigned to cluster C k , CSP clusters the label matrix and Send to BS; BS uses the following formula to calculate the set Unblinding: Cluster C k The aggregation formula inside is: in, is the cluster C in the current round r k The cluster model of ||C k || is cluster C k The number of participating devices; For cluster C k Members of The real subscript is l m =π(l t ) -1 ; BS will θ k Distribute to clients user Verify the received clustering gradient vector and decrypt the verified clustering gradient vector: Update the local model using the average of the verified parameters as follows:

4. The privacy-enhanced adaptive clustering federated learning method for heterogeneous resources according to claim 1, characterized in that: CSP performs spectral clustering on the gradient vectors to obtain cluster labels; including: First, construct the degree matrix D = diagg∑ j S i′j ); Then, calculate the Laplace matrix L = D – S and normalize L to get: Calculate L norm The characteristic matrix U of U=eigenvectors(L norm )[:,-c∶] Where c is the number of clusters; Finally, the K-means algorithm is used to cluster the sample set U to obtain the cluster label set C.

5. The privacy-enhanced adaptive clustering federated learning method for heterogeneous resources according to claim 4, characterized in that: Calculate the cluster label matrix using spectral embedding method include: Calculate the feature matrix E of the similarity matrix S: E = eigenvectors(S)[:,-c∶] and standardize; For each cluster k, calculate the cluster center δ k =Mean(E norm [labels==k]), where labels∈C; Calculate the cluster label matrix Cluster label matrix The matrix element is a jk , a jk Indicates B j and cluster center δ k distance.

6. Privacy-enhanced adaptive clustering federated learning system for heterogeneous resources, characterized by: include: The participant clustering module is configured to: cluster participants based on uplink latency; cluster heterogeneous devices using different clustering criteria; The local training module is configured as follows: local training within clusters with similar computing power; Cluster users according to device computing power to optimize training efficiency; The security cluster model aggregation module is configured to: aggregate security cluster models based on spectrum embedding; Adopt soft clustering based on gradient vector similarity, allowing user models to be aggregated into multiple cluster models, and use multiple cluster models to update the local model; Local training within a cluster with similar computing power; including: At the beginning of each round of training, the BS receives the uplink latency from the devices participating in the round, including the participants U i Time t to complete local training i and the communication delay D with the BS i ; BS first uses the K-means clustering algorithm to divide the devices into τ classes according to the uplink delay: {K1, ..., K τ }, where K1 is the class with the shortest delay, K τ This is the class with the longest latency. Every time interval R, the server aggregates the local models received from all devices to complete a round of training. U i Select a subset of the local dataset to train the local model and get the local gradient as follows: Where l(·) is the local loss function, r is the number of rounds, and η is the local learning rate; represents the model parameters uploaded by user i in round r, Similarly, ▽ is the Laplace operator; Encode the encrypted gradient vector as an integer as follows: Among them, ρ represents the coding accuracy, and m represents the coding length; U i Using CSP's public key encryption As shown below: in, represents the gradient uploaded by participant i encrypted with the public key of CSP, Refers to the encryption function in Plliar, U i Generate data tuples Among them, TS is the timestamp; U i calculate Digital signature and will Send to BS.

Images