Decentralized adaptive federated learning method based on mixed privacy calculation

Abstract

The invention belongs to the technical field of artificial intelligence, and discloses a decentration adaptive federal learning method based on mixed privacy computing, which comprises the following steps that: firstly, a publisher registers, publishes a training task and performs evaluation task classification, and the evaluation task classification comprises that the publisher constructs an evaluation classifier; the participant obtains a training task and an evaluation classifier, and performs data quality evaluation and layering based on the evaluation classifier; the participant carries out local model training and carries out hierarchical encryption on an upper-layer model and a lower-layer model; the model after hierarchical encryption is subjected to hierarchical aggregation, an upper layer aggregation model and a lower layer aggregation model are obtained, the upper layer aggregation model and the lower layer aggregation model are fused, a global model is obtained, and experiments prove that the federal learning privacy protection and attack defense capabilities are remarkably enhanced while the training efficiency and the model precision are improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a decentralized adaptive federated learning method based on hybrid privacy computing. Background Technology

[0002] In recent years, the rapid advancement of big data and computing power has driven the development of machine learning, significantly impacting fields such as chemical engineering and medicine. However, traditional machine learning relies on centralized training data on a central server, which suffers from limited computing resources, high time costs, and risks of data privacy breaches. To address this, Google proposed federated learning technology in 2016. Through a distributed architecture, models are trained locally and only parameters are uploaded to the server for aggregation, effectively balancing data sharing and privacy protection. However, this technology still faces privacy protection deficiencies and data quality risks. Existing privacy protection mechanisms (such as differential privacy and homomorphic encryption) suffer from high computational overhead or insufficient compatibility. Therefore, a more efficient and secure optimized federated learning solution is urgently needed to address these challenges.

[0003] Since the introduction of federated learning technology, it has been extensively studied by academia and industry. However, the technology still has several key defects that need to be addressed: (1) Insufficient communication efficiency - Federated learning relies on frequent interaction between the server and the client to update model information, resulting in a heavy burden on communication bandwidth, especially creating a barrier to participation for users with abundant computing resources but limited network conditions; (2) Inadequate privacy protection - Although the "data does not leave the local area" interaction mechanism is adopted, it still faces security threats such as poisoning attacks and inference attacks in actual deployment. Moreover, there is a lack of effective means to verify the client's behavioral motivation and the credibility of the central server. The privacy protection mode that only relies on model parameter updates has significant vulnerabilities; (3) Lack of incentive mechanism - Federated learning relies on a large number of clients for collaborative training, but the existing technology lacks efficient incentive strategies, making it difficult to attract a sufficient scale of high-quality data to participate, resulting in limited performance of the final model. Although current research has proposed some improvement schemes in differential privacy and model compression, there are still technical problems in federated learning such as low communication efficiency, inadequate privacy protection, and lack of incentive mechanism. Summary of the Invention

[0004] The purpose of this invention is to provide a decentralized adaptive federated learning method based on hybrid privacy computing to solve the problems of low communication efficiency and inadequate privacy protection in existing federated learning methods.

[0005] To address the aforementioned technical issues, this invention proposes a decentralized adaptive federated learning method based on hybrid privacy computing, which significantly enhances privacy protection and attack defense capabilities in federated learning.

[0006] This invention proposes a decentralized adaptive federated learning method based on hybrid privacy computation, comprising the following steps: S1: The publisher registers, publishes training tasks, and classifies evaluation tasks; The assessment task classification includes the publisher constructing an assessment classifier; S2: Participants obtain training tasks and evaluation classifiers, and perform data quality assessment and stratification based on the evaluation classifiers; S3: Participants train local models and perform layered encryption on upper and lower layer models; S4: Perform layered aggregation on the encrypted model to obtain an upper-layer aggregated model and a lower-layer aggregated model. Aggregate the upper-layer aggregated model with the lower-layer aggregated model, upload the aggregation result to the blockchain, and complete one round of federated learning iteration. Repeat S2~S4 until the model performance reaches the relevant system indicators.

[0007] Furthermore, the specific steps in S1 include: S1.1: The publisher prepares identity information and submits it through the registration interface of the blockchain network; The identity information includes an identity identifier, a public key, and the dataset held by the individual. S1.2: The publisher releases the training task to the blockchain in the form of a smart contract; S1.3: The publisher utilizes its own sample dataset D Construct the classifier parameter set C ; S1.4: The publisher randomly searches the classifier parameter set. C The initial parameters are obtained from the dataset, an initial classifier is constructed based on the initial parameters, and the sample dataset is used. D The initial classifier is trained to obtain the evaluation classifier; S1.5: The publisher uploads the trained evaluation classifier and its labels to the blockchain via a smart contract.

[0008] Furthermore, the specific steps of S2 include: S2.1: The participants obtain the training task and the evaluation classifier, and calculate the data quality evaluation score, which is obtained by weighting the classifier accuracy and the amount of data; S2.2: The K-means clustering algorithm is used to divide the participants into upper and lower training sets with the highest and lowest evaluation scores as the cluster centers, and the hierarchical results are stored in the blockchain.

[0009] Furthermore, the layered encryption of the upper and lower layer models in S3 specifically includes: S3.1: Encrypt the upper-layer model using the PHS2 encryption method; S3.2: Use LAD encryption to encrypt the lower-level model.

[0010] Furthermore, in S3.1, the upper-layer model is encrypted using the PHS2 encryption method as follows: S3.1.1: Define the lower-level user set as ， ,in For the lower-level user set Specific users in the, For lower-level user sets The Middle B A lower-level user; S3.1.2: Secret Distributor Randomly select lower-level participants to form a collaborative decryption set. ,in ,in For collaborative decryption set The specific participants in the process, For collaborative decryption set The Middle t Each participating party N B Secret distributors for lower-level users ∈ Collect collaborative decryption sets The public key set of members ,in For collaborative decryption set The public keys corresponding to each participating party. Represents a collaborative decryption set Lidi t The public keys corresponding to each participant, and the collaborative decryption set Member identification set ,in For collaborative decryption set The identity identifiers corresponding to each participating party. For the first to participate in collaborative decryption t The identity of each member ID ; S3.1.3: Secret Distributor Four coprime large prime numbers are randomly selected. And calculate the plaintext encryption key parameters. , , , and The specific calculation formulas are shown in equations (1)-(5): (1) (2) (3) (4) (5) in: for Paillier Encrypted modular multiplication base, To generate auxiliary numbers for encryption, It is the least common multiple of Euler's totient function. for Paillier Auxiliary decryption function, This is the auxiliary number for the inverse of the private key. It is the largest prime number. It is the second largest prime number. It is the third largest prime number. It is the fourth largest prime number. for , yes The input variables of the function; use Paillier Encrypted modular multiplication base N Encryption generator auxiliary number g Euler's least common multiple , Paillier Auxiliary decryption function Private key inverse auxiliary number calculate Paillier The public and private key pairs are calculated using the formulas shown in equations (6) and (7): (6) (7) in: for Paillier public key pair ， for Paillier Private key pair; S3.1.4: Utilizing coprime large prime numbers calculate Public-private key pairs S3.1.5: In the... t After completing the local training, the secret distributor Get model update parameters and utilize Paillie public key Update parameters of the model Encrypt; S3.1.6: Secret Distributor Generate the first share generator polynomial The specific calculation formula is as follows: (8) (9) in: For share encryption modular multiplication parameters, Generate a polynomial for the first share. Generate a polynomial for the second share. Generate polynomial coefficients for the first share. Generate polynomial coefficients for the second share. t For training rounds; S3.1.7: Collaborative Decryption Set Each member According to the label Obtain the secret partition value The specific calculation formula is as follows: (10) In the formula: For secret partitioning value, Member identification through After mapping, substitute into the polynomial Obtain the median value. The Lagrange difference coefficient, The first secure hash function, For members Identity identifier, For multiplication, For members Belongs to the collaborative decryption set , For traversal Excluding current members k All other users besides; S3.1.8: Secret Distributor With collaborative decryption set Members negotiate to generate random numbers and , Using random numbers Construct a mask matrix The secret partition value obtained in S3.1.7 With mask matrix Adding or subtracting the elements in the array yields the encrypted secret partition value. The specific calculation formula is as follows: (11) In the formula: The encrypted secret split value, For secret partitioning value, The sum of the upper triangular parts of the mask matrix, The lower triangular sum of the mask matrix, The indices of the elements in the mask matrix. The indices of the elements in the mask matrix; S3.1.9: Constructing a set of random numbers Calculate the collaborative decryption set Each member Decryption share The specific calculation formula is as follows: (12) in: For the first k Decryption share parameters generated by each collaborative decryption member The second secure hash function, For members Identity identifier, As auxiliary parameters, The encryption key parameter for the secret share; S3.1.10: Decryption share obtained based on S3.1.9 Computational collaborative decryption set Each member First verification parameter First verification parameter The calculation formula is (14), and the second verification parameter is... The calculation formula is (13), and the generated , Stored on the blockchain; (13) (14) In the formula: for , For the first k Decryption share parameters generated by each collaborative decryption member The public key used to encrypt the secret share. For the first k A random number negotiated between the collaborative decryption members and the secret distributor. As the first verification parameter, The encrypted secret segment value, For XOR operation, For hash functions, For polynomials Coefficients of real terms; S3.1.11: Decrypting a set collaboratively Randomly select a member and identified as To collaboratively decrypt leaders, a collaborative decryption set. Each member Receive decryption share Then, based on the generated Determine whether the data is stored on the blockchain. If the function relationship shown in equation (15) is satisfied, then calculate the encrypted secret segmentation value. Collaborative decryption set Each member Will Handed over to the leaders who collaborated on declassification Recover and decrypt the private key ; (15).

[0011] Furthermore, in S3.2, the LAD method is used to encrypt the lower-level model as follows: S3.2.1: Participants use local datasets to train the model and obtain local model parameters. And calculate the parameters to update the gradient. The specific calculation formula is as follows: (16) In the formula: These are the aggregated model parameters. For the first The model parameters obtained after local training on each node Update the gradient for the parameters; S3.2.2: Update the gradient of the parameters. Perform gradient clipping to obtain the clipped gradient. The specific calculation formula is as follows (17): (17) In the formula: Update the gradient for the parameters. The gradient after clipping. The cropping threshold, The L2 norm of the original gradient. This is the cropping scaling factor; S3.2.3: Calculate the test accuracy difference of the model in the current round. and with preset threshold Compare and calculate the noise scaling factor. k ; S3.2.4: According to the noise scaling factor kDifferential privacy sensitivity Generate adaptive noise and add it to the clipped gradient. The gradient is obtained after privacy protection.

[0012] Furthermore, in S3.2.4, the differential privacy sensitivity... The specific calculation formula is as follows: (18) In the formula: For differential privacy sensitivity, For users Local dataset, For users Local dataset, To and Data sets that differ by only one record m For the upper bound, For dataset The reciprocal of the sample size For the dataset , The relevant quantity is taken as the maximum value. For dataset Summing of samples, a For parameters Below, regarding the loss function In data The parameter when taking the maximum value. For dataset Sample size Iterate through all adjacent dataset pairs ( , ),Pick The maximum value.

[0013] Furthermore, the specific steps of S4 include: S4.1: Upper-level participants adopt FedAvg The algorithm determines the aggregation weight based on the aggregation weight and the size of the node data, and uses the aggregation weight of the upper layer to aggregate the upper layer model; S4.2: Lower-level participants utilize ReLU The function calculates the cosine similarity between the current node's model update and the previous global model, and then... ReLU The cosine similarity result obtained from the function calculation is used as the weight of the lower-level aggregation parameter, and then the weights of the users in the lower-level training set are calculated. ; S4.3: User weights in the lower-level training set obtained based on S4.2 Aggregate the lower-level models; S4.4: Perform global model aggregation based on the aggregation results of the upper-level model and the aggregation results of the lower-level model.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a decentralized adaptive federated learning method based on hybrid privacy computing. By employing an evaluation-layered training strategy, an evaluation classifier is constructed to assess the quality of user-owned data. Differential encryption measures and aggregation schemes are applied to users at different levels, better protecting user data security while maintaining model training accuracy. The method employs differentiated encryption measures for different user levels, with upper-level users using a secondary threshold secret sharing mechanism. encryption( Lower-level users employ local adaptive differential privacy. This invention aims to protect user data security while reducing encryption overhead. By combining traditional federated learning with blockchain technology, it stores federated learning training information and hash values ​​of important parameters through smart contracts. Based on the characteristics of blockchain being difficult to tamper with and traceable, users can verify the hash value to determine whether the information has been tampered with. At the same time, by utilizing the decentralized nature of blockchain, it can reduce some of the centralized impact of federated learning. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a diagram of a decentralized adaptive federated learning architecture based on hybrid privacy computing; Figure 2 This is a flowchart of the training process for the algorithm (DAFL-HPC) proposed in this application; Figure 3 It is a Paillier flow encryption diagram based on secondary threshold secret sharing; Figure 4 The test results of the abnormal data models in the algorithm proposed in this application, where the participants respectively account for 30%, 45%, and 60% of the data. (a) represents the model test results when the participating party contains 30% abnormal data, (b) represents the model test results when the participating party contains 45% abnormal data, and (c) represents the test results of the model proposed in this application when the participating party contains 45% abnormal data. Figure 5 These are the accuracy test results of the algorithm proposed in this application on various datasets under different privacy budgets; (a) represents the accuracy test results of the proposed algorithm on the MNIST dataset under different privacy budgets, (b) represents the accuracy test results of the proposed algorithm on the FashionMNIST dataset, and (c) represents the accuracy test results of the proposed algorithm on the KMNIST dataset. Figure 6 This is a comparison of the security and accuracy of the algorithm proposed in this application with traditional differential privacy and label smoothing regularization; (a) shows the accuracy comparison results of traditional differential privacy, label smoothing regularization, and the algorithm proposed in this application on the MNIST dataset; (b) shows the accuracy comparison results of traditional differential privacy, label smoothing regularization, and the algorithm proposed in this application on the FashionMNIST dataset; (c) shows the accuracy comparison results of traditional differential privacy, label smoothing regularization, and the algorithm proposed in this application on the KMNIST dataset; (d) shows the F1 comparison results of traditional differential privacy, label smoothing regularization, and the algorithm proposed in this application on the MNIST dataset; (e) shows the F1 comparison results of traditional differential privacy, label smoothing regularization, and the algorithm proposed in this application on the FashionMNIST dataset; and (f) shows the F1 comparison results of traditional differential privacy, label smoothing regularization, and the algorithm proposed in this application on the KMNIST dataset.

[0017] Figure 7 These are the accuracy, F1 score, and time test results of the proposed algorithm under different stratification ratios; where (a) is the accuracy result of the proposed algorithm on the MNIST dataset; (b) is the accuracy result of the proposed algorithm on the FashionMNIST dataset; (c) is the accuracy result of the proposed algorithm on the KMNIST dataset; (d) is the F1 score change result of the proposed algorithm on the MNIST dataset under member inference attack; (e) is the F1 score change result of the proposed algorithm on the FashionMNIST dataset under member inference attack; and (f) is the F1 score change result of the proposed algorithm on the KMNIST dataset under member inference attack. Figure 8The figures show the comparison results of the poisoning attack accuracy of the proposed algorithm with three other algorithms when there is no abnormal data among the participants; (a) shows the accuracy experimental results of the proposed algorithm with Multi-Krum, TrimmedMean, and FedAvg on the MNIST dataset under the condition of no abnormal data; (b) shows the accuracy experimental results of the proposed algorithm with Multi-Krum, TrimmedMean, and FedAvg on the FashionMNIST dataset under the condition of no abnormal data; (c) shows the accuracy experimental results of the proposed algorithm with Multi-Krum, TrimmedMean, and FedAvg on the KMNIST dataset under the condition of no abnormal data. Figure 9 The following are comparisons of the accuracy of the proposed algorithm with three other algorithms in poisoning attacks when the participants have anomalous data: (a) shows the accuracy of the proposed algorithm with Multi-Krum, Trimmed Mean, and FedAvg on the MNIST dataset with anomalous data; (b) shows the accuracy of the proposed algorithm with Multi-Krum, Trimmed Mean, and FedAvg on the FashionMNIST dataset with anomalous data; and (c) shows the accuracy of the proposed algorithm with Multi-Krum, Trimmed Mean, and FedAvg on the KMNIST dataset with anomalous data. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] This implementation method is based on a decentralized adaptive federated learning approach using hybrid privacy computing. The specific steps of this implementation are as follows: S1: The publisher registers and publishes training tasks and performs evaluation task classification. Evaluation task classification includes the publisher constructing an evaluation classifier. In some specific implementations, S1 specifically includes: S1.1: The publisher prepares identity information and submits it through the registration interface of the blockchain network; Identity information includes identity identifier, public key, and dataset held by the individual; S1.2: The publisher releases the training task to the blockchain in the form of a smart contract; S1.3: The publisher uses its own small sample dataset D Construct the classifier parameter set C These are the basic input conditions for the algorithm to run; S1.4: The publisher randomly searches the classifier parameter set. C The initial parameters are obtained from the dataset, an initial classifier is constructed based on the initial parameters, and the sample dataset is used. D The initial classifier is trained to obtain the evaluation classifier, which is used to prepare for subsequent evaluations. S1.5: The publisher uploads the trained evaluation classifier and its labels to the blockchain via a smart contract.

[0020] Please see Figure 1 In one feasible implementation, this application utilizes blockchain technology (FISCO BCOS consortium blockchain) and smart contracts written in Solidity to build a decentralized federated learning framework. It integrates differential privacy and homomorphic encryption based on cryptography to achieve hybrid privacy computing. For complex data scenarios, it designs a hierarchical federated learning model with hierarchical encryption and aggregation. Then, it integrates decentralized architecture and privacy computing to form a decentralized adaptive federated learning scheme based on hybrid privacy computing.

[0021] S2: Participants obtain training tasks and evaluation classifiers, and perform data quality assessment and stratification based on the evaluation classifiers; In some specific implementations, S2 specifically includes: S2.1: The participants obtain the training task and the evaluation classifier, and calculate the data quality evaluation score, which is obtained by weighting the classifier accuracy and the amount of data; S2.2: The K-means clustering algorithm is used to divide the participants into upper and lower training sets with the highest and lowest evaluation scores as the cluster centers, and the hierarchical results are stored in the blockchain. S3: Participants train local models and perform layered encryption on upper and lower layer models; This application is based on an evaluation-based hierarchical training strategy, which separates the data held by the participants. Since the data after evaluation separation by the upper-level participants is closer to the data expected by the publisher, data privacy should be protected from being stolen by adversaries while ensuring training accuracy. Meanwhile, the lower-level participants should increase their contribution to the model while ensuring data security.

[0022] In some specific implementations, S3 specifically includes: The upper-level participants first randomly select collaborative decryption members from the lower-level user set and obtain their public keys and identity identifiers. Then, they generate Paillier-encrypted public-private key pairs and secret share encryption key pairs. The parameters of the locally trained model are updated using Paillier public keys. The Paillier private key is generated through Shamir secret sharing and multinomial splitting. The mask matrix is ​​generated using Diffie-Hellman technology to encrypt the shares. The decrypted shares are calculated and stored in the blockchain. Finally, the decryption leader aggregates the shares to recover the private key. Lower-level participants first perform L2 norm clipping on the gradient updates of their local models, calculate the dataset sensitivity, then dynamically adjust the noise parameters based on the difference in global model accuracy, and add adaptive noise to the clipped gradients to satisfy differential privacy; finally, the participants upload the encrypted parameters to the blockchain via smart contracts.

[0023] In some specific implementations, the layered encryption of the upper and lower layer models in S3 specifically includes: S3.1: Encrypt the upper-layer model using the PHS2 encryption method. See [link / reference]. Figure 2 ; In this application, the PHS2 encryption method uses the Diffie-Hellman technique to obtain a mask matrix, which is then used to encrypt the original secret. Secret shares are obtained through secret sharing, and these shares are then distributed through threshold secret sharing. This dual-threshold secret sharing mechanism ensures that restoring the secret requires all disclosed secret shares, thus enhancing the security of the secret recovery phase. S3.2: Use the LAD method to encrypt the lower-level model.

[0024] In S3.1, the upper-layer model is encrypted using the PHS2 encryption method: During encryption, the upper-layer participants use their local private dataset to train the local model and perform Paillier encryption based on a quadratic threshold secret sharing mechanism, specifically: S3.1.1: Define the lower-level user set as ， ,in For the lower-level user set Specific users in the, For lower-level user sets The Middle B A lower-level user; S3.1.2: Secret Distributor Randomly select lower-level participants to form a collaborative decryption set. ,in ,in For collaborative decryption set The specific participants in the process, For collaborative decryption set The Middle t Each participating party N B Secret distributors for lower-level users ∈ Collect collaborative decryption sets The public key set of members ,in For collaborative decryption set The public keys corresponding to each participating party. Represents a collaborative decryption set Lidi t The public keys corresponding to each participant, and the collaborative decryption set Member identification set ,in For collaborative decryption set The identity identifiers corresponding to each participating party. For the first to participate in collaborative decryption t The identity of each member ID ; S3.1.3: Secret Distributor Four coprime large prime numbers are randomly selected. Calculate plaintext encryption key parameters , , , and The specific calculation formulas are shown in equations (1)-(5): (1) (2) (3) (4) (5) in: for Paillier Encrypted modular multiplication base, To generate auxiliary numbers for encryption, It is the least common multiple of Euler's totient function. for Paillier Auxiliary decryption function, This is the auxiliary number for the inverse of the private key. It is the largest prime number. It is the second largest prime number. It is the third largest prime number. It is the fourth largest prime number. for , yes The input variables of the function; use Paillier Encrypted modular multiplication base N Encryption generator auxiliary number g Euler's least common multiple , Paillier Auxiliary decryption function Private key inverse auxiliary number calculate Paillier The public and private key pairs are calculated using the formulas shown in equations (6) and (7): (6) (7) in: for Paillier public key pair ， for Paillier Private key pair.

[0025] S3.1.4: Calculation Public-private key pairs S3.1.5: In the... t After completing the local training, the secret distributor Get model update parameters and utilize Paillie public key Encryption is performed, and the specific encryption process is as follows: Figure 3 As shown; S3.1.6: Secret Distributor Generate the first share generator polynomial The specific calculation formula is as follows: (8) (9) in: For share encryption modular multiplication parameters, Generate a polynomial for the first share. Generate a polynomial for the second share. Generate polynomial coefficients for the first share. Generate polynomial coefficients for the second share; S3.1.7: Collaborative Decryption Set Each member According to the label Obtain the secret partition value The specific calculation formula is as follows: (10) In the formula: For secret partitioning value, Member identification through After mapping, substitute into the polynomial Obtain the median value. The Lagrange difference coefficient, The first secure hash function, For members Identity identifier, For multiplication, For members Belongs to the collaborative decryption set , For traversal Excluding current members k All other users besides; S3.1.8: Secret Distributor With collaborative decryption set Members negotiate to generate random numbers and , Using the random number Construct a mask matrix The secret partition value obtained in S3.1.7 With mask matrix Adding or subtracting the elements in the array yields the encrypted secret partition value. The specific calculation formula is as follows: (11) In the formula: The encrypted secret split value, For secret partitioning value, The sum of the upper triangular parts of the mask matrix, The lower triangular sum of the mask matrix, The index of an element in the mask matrix; S3.1.9: Using random numbers Construct a set of random numbers Calculate the collaborative decryption set Each member Decryption share The specific calculation formula is as follows: (12) in: For the first k Decryption share parameters generated by each collaborative decryption member The second secure hash function, For members Identity identifier, As auxiliary parameters, The encryption key parameter for the secret share; S3.1.10: Decryption share obtained based on S3.1.9 Computational collaborative decryption set Each member First verification parameter First verification parameter The calculation formula is (14), and the second verification parameter is... The calculation formula is (13), and the generated , Stored on the blockchain; (13) (14) In the formula: for , For the first k Decryption share parameters generated by each collaborative decryption member The public key used to encrypt the secret share. For the first k A random number negotiated between the collaborative decryption members and the secret distributor. As the first verification parameter, The encrypted secret segment value, For XOR operation, For hash functions, For polynomials Coefficients of real terms; S3.1.11: Decrypting a set collaboratively Randomly select a member and identified as To collaboratively decrypt leaders, a collaborative decryption set. Each member Receive decryption share Then, based on the generated Determine whether the data is stored on the blockchain. If the function relationship shown in equation (15) is satisfied, then calculate the encrypted secret segmentation value. Collaborative decryption set Give a member Will Handed over to the leaders who collaborated on declassification Recover and decrypt the private key .

[0026] (15) S3.2: Use the LAD method to encrypt the lower-level model.

[0027] In S3.2, the encryption of the lower-level model using the LAD method is specifically as follows: S3.2.1: Participants use local datasets to train the model and obtain local model parameters. And calculate the parameters to update the gradient. The specific calculation formula is as follows: (16) In the formula: These are the aggregated model parameters. For the first The model parameters obtained after local training on each node Update the gradient for the parameters; S3.2.2: Update the gradient of the parameters Perform gradient clipping to obtain the clipped gradient. The specific calculation formula is as follows (17) (17) In the formula: Update the gradient for the parameters. The gradient after clipping. The cropping threshold, The L2 norm of the original gradient. This is the cropping scaling factor; S3.2.3: Calculate the test accuracy difference of the model in the current round. and with preset threshold Compare and calculate the noise scaling factor. k ; S3.2.4: According to the noise scaling factor k Differential privacy sensitivity Generate adaptive noise and add it to the clipped gradient. The gradient is obtained after privacy protection.

[0028] In S3.2.4, the differential privacy sensitivity... The specific calculation formula is as follows: (18) In the formula: For differential privacy sensitivity, For users Local dataset, For users Local dataset, To and Data sets that differ by only one record m For the upper bound, For dataset The reciprocal of the sample size For the dataset ， The relevant quantity is taken as the maximum value. For dataset Sum of samples, a For parameters Below, regarding the loss function In data The parameter when taking the maximum value. For dataset Sample size Iterate through all adjacent dataset pairs ( ， ),Pick The maximum value.

[0029] S4: Perform layered aggregation on the encrypted model to obtain an upper-layer aggregated model and a lower-layer aggregated model. Aggregate the upper-layer aggregated model with the lower-layer aggregated model, upload the aggregation result to the blockchain, and complete one round of federated learning iteration. Repeat S2~S4 until the model performance reaches the relevant system indicators.

[0030] In some specific implementations, S4 specifically includes: S4.1: Upper-level participants adopt FedAvg The algorithm determines the aggregation weight based on the aggregation weight and the size of the node data, and uses the aggregation weight of the upper layer to aggregate the upper layer model; S4.2: Lower-level participants utilize ReLU The function calculates the cosine similarity between the current node's model update and the previous global model, and then... ReLU The cosine similarity result obtained from the function calculation is used as the weight of the lower-level aggregation parameter, and then the weights of the users in the lower-level training set are calculated. S4.3: User weights in the lower-level training set obtained from S4.2 S4.4: Aggregate the lower-level models; Based on the aggregation results of the upper-level models and the lower-level models, perform global model aggregation.

[0031] In this implementation, the publisher itself has a small amount of data, but due to issues such as data volume and computing power, it cannot undertake the training task on its own. Therefore, it publishes the task to the blockchain and entrusts others to complete the training.

[0032] The tester performs model accuracy tests on the global model after model aggregation. If the model accuracy fails to converge or the global training rounds have not met the publisher's requirements, the test model is redistributed to the participants for training. Once the model accuracy converges or the required number of global training rounds have been reached, the publisher obtains the model required by the publisher, completing the federated learning model training. The tester uploads relevant information about each test result to the blockchain for storage.

[0033] The evaluation classifier is trained by the publisher using its own limited data, with the expectation that participants will complete the training task using the same data distribution. Participants then use the evaluation classifier locally to perform the evaluation stratification.

[0034] The upper training set is where participants use an evaluation classifier to perform evaluation stratification, assigning participants with evaluation values ​​higher than a threshold to the upper training set. After evaluation stratification, the data distribution of the upper participants more closely resembles the data distribution desired by the publisher.

[0035] The lower training set is where participants use an evaluation classifier to perform evaluation stratification, and participants whose evaluation values ​​are below a threshold are assigned to the lower training set.

[0036] Participants, commissioned by the publisher, evaluate and stratify the data using an evaluation classifier. Due to differences in data distribution, value, and model contribution among participants at different levels, different privacy protection methods are applied to the local model training results based on the participant's level. Simultaneously, important model training and encrypted parameter information are uploaded to the blockchain for storage via smart contracts. Participant roles include validators and workers, both of whom participate in model training. Furthermore, validators are responsible for checking data consistency.

[0037] The blockchain primarily stores the identity information of all nodes, including identity identifiers, node public keys, the size of the dataset held, user evaluation values, and other identity information, as well as important information for data encryption and model aggregation testing. When a node receives relevant submitted information, it can compare and verify the information submitted by the user with the information stored in the blockchain to prevent malicious tampering during information transmission.

[0038] Smart contracts are automatically invoked during different federated learning and training phases when predefined conditions are met, and relevant information is uploaded to the blockchain for storage in the form of transactions.

[0039] Experimental simulation: This experiment was conducted on Ubuntu 22.04.5 LTS, with an NVIDIA GeForce RTX 4060 Laptop GPU, an Intel(R) Core(TM) i7-14650HX CPU, and 16.0 GB of RAM. The system was built using Python 3.8.3 and PyTorch 1.8.1+cu111, with the blockchain constructed using the FISCOBCOS chain. Smart contracts were written in Solidity version 0.4.24. Ten participants were involved in training the federated learning model, with each participant performing 7 rounds of training locally, for a total of 40 rounds of global model training.

[0040] The experiment in this application simulates that although honest participants have no intention of attacking the model, their private data may unintentionally contain some abnormal data such as feature distribution differences and mislabeled data, which will have a certain negative impact on the training of the model.

[0041] This application uses MNIST, FashionMNIST, and KMNIST as participant data. To simulate honest participants containing anomalous data, feature discrimination, label flipping, and low-quality operations were performed on the original data. Feature discrimination was achieved by mixing the training data into the SVHN dataset; mislabeled data was achieved by making some data labels mismatched with the images; and low-quality data was achieved by adding noise, blurring, and adding, cropping, and flipping images. Figure 4 (a)-(c) represent the model test results when honest participants contain 30%, 45%, and 60% of outlier data, respectively.

[0042] from Figure 4 The test results show that even if honest participants have no intention of attacking the model, the training accuracy of the model continuously decreases as the proportion of anomalous data increases, indicating that anomalous data will continuously impact the model. As the amount of anomalous data from participants increases, reaching approximately 60% anomalous data, Figure 4 In (a), the model accuracy on the MNIST dataset decreased to 0.7, while... Figure 4 (b) Datasets FashionMNIST and Figure 4 (c) The training accuracy of the KMNIST dataset consistently dropped to around 0.6, and the presence of outliers also consistently affected model convergence. This indicates that the data processing of the algorithm proposed in this application is consistent with the assumption that outliers from participating parties affect model training.

[0043] This application considers 45% of the data from honest participants to be anomalous. First, it discusses and analyzes the privacy budget in the DAFL-HPC algorithm. The training accuracy of the model under different privacy budgets is shown in Figures 5(a)-(c). Figure 5 It can be seen that the three datasets, under different privacy budgets, are similar to... Figure 5 In the model accuracy comparison, all models showed some improvement, and with the continuous increase in privacy budgets, model training became more stable. Figure 6 (a) The DAFL-HPC algorithm was tested on the MNIST dataset with privacy budgets of 0.05, 0.1, and 0.5. While the model training was not robust enough with a privacy budget of 0.05, requiring more iterations to converge compared to other privacy budgets, ... Figure 5 (b) FashionMNIST and Figure 5 (c) The KMNIST dataset was tested with privacy budgets of 0.1, 0.3 and 0.5. Although the model still fluctuated slightly with a privacy budget of 0.1 on both datasets, the adaptive local differential privacy in the proposed algorithm DAFL-HPC is a post-processing mechanism that can adjust the noise in a timely manner to prevent excessive noise from affecting the training.

[0044] To address member inference attacks launched by external adversaries against federated learning, this paper assumes that the adversary can intercept the results of each global model update and knows the model's construction details. The adversary can obtain prior knowledge by sending queries to the target model and construct attack data by building a shadow model to implement the member inference attack. However, the external adversary does not know the scale of noise addition each time. Under a privacy budget of 0.1, the proposed DAFL-HPC algorithm is compared with existing algorithms targeting inference attacks, traditional differential privacy, and label smoothing regularization. The accuracy and F1 score results are as follows: Figure 6 As shown.

[0045] exist Figure 6 middle, Figure 6 (a)(b)(c) represent a comparison of the accuracy of the three datasets MNIST, FashionMNIST, and KMNIST on traditional differential privacy, label smoothing regularization, and the DAFL-HPC algorithm proposed in this application, respectively. Figure 6 In (a)(b)(c), the DAFL-HPC algorithm proposed in this application is superior to the differential privacy and label smoothing regularization algorithms. Differential privacy causes a decrease in accuracy due to perturbation parameters. Although label smoothing has a smaller impact, it continuously and slowly reduces the model accuracy due to the presence of abnormal data among the participants, thereby affecting the model convergence. Figure 6(e), (f), and (g) represent the F1 scores of the three datasets MNIST, FashionMNIST, and KMNIST in the face of member inference attacks, respectively, using traditional differential privacy, label smoothing regularization, and the DAFL-HPC algorithm proposed in this application. Figure 6 (e)(g) shows that, in most cases, Figure 6 (e)MNIST and Figure 6 (g) On the KMNIST dataset, the F1 score of the proposed DAFL-HPC algorithm is very similar to that of other algorithms, while... Figure 6 (f) On FashionMNIST, in the worst case, the difference between DAFL-HPC in this application and the other two algorithms is within 0.1. This phenomenon is due to the fact that the upper-level participants in the algorithm of this application use PHS2 encryption, which does not perturb the parameters compared with LAD. However, in general, DAFL-HPC is comparable to traditional differential privacy and label smoothing regularization in resisting member inference attacks, while significantly improving model accuracy and enhancing model utility.

[0046] This application employs a hierarchical training strategy, setting α as the hierarchical ratio. This α represents the proportion of participants in each round of the global model that use PHS2 encryption, while the remaining participants (1-α) use Local Adaptive Differential Privacy (LAD). The combined result primarily reflects the utility of the global model. This application sets α=0.3, α=0.4, and α=0.5 respectively to test the model's accuracy, F1 score under member inference attacks, and time. Figure 7 As shown in the results, Figure 7 (a), (b), and (c) represent the accuracy results of the DAFL-HPC algorithm on the three datasets MNIST, FashionMNIST, and KMNIST, respectively. As the stratification ratio α increases, the model's accuracy decreases. At a stratification ratio α = 0.5, Figure 7 (a) MNIST and Figure 7 (c) The KMNIST dataset also maintains an accuracy of around 0.6, while Figure 7 (b) The accuracy of the FashionMNIST dataset is also above 0.5. However, as the stratification ratio increases, the time required decreases. Figure 7 (h)(i)(j) represent the global training time of the DAFL-HPC algorithm on the three datasets MNIST, FashionMNIST, and KMNIST, respectively. It is easy to see that for every 0.1 increase in the stratification ratio, the training time decreases by 100 seconds (s). However, in the face of member inference attacks... Figure 7(e)(f)(g) represent the changes in F1 scores of the DAFL-HPC algorithm on the three datasets MNIST, FashionMNIST and KMNIST, respectively, in the face of member inference attacks. The increase in the stratification ratio also means the increase in mixing strength. From the results, the increase in the stratification ratio will have a better resistance to member inference attacks in most cases.

[0047] This application considers both scenarios with and without anomalous data from participating parties, using label flipping attack as the primary means of poisoning attack. The proposed algorithm is compared in accuracy with existing common anti-poisoning attack algorithms Multi-Krum, Trimmed Mean, and FedAvg aggregation. With a scenario of 10 participants, including 2 malicious poisoners, the accuracy results for scenarios with and without anomalous data are shown in the following figures. Figure 8 and Figure 9 As shown.

[0048] Figure 8 This indicates that, under conditions free of outlier data, the traditional aggregation method FedAvg is susceptible to poisoning, while the proposed DAFL-HPC algorithm, after employing privacy protection techniques, [does not benefit from this]. Figure 8 (a) MNIST, Figure 8 (b) FashionMNIST and Figure 8 (c) On the KMNIST dataset, compared to Multi-Krum and Trimmed Mean without privacy protection techniques, the model accuracy did not decrease significantly, remaining within 0.1 of the previous level; while Figure 9 This indicates that, under conditions where participating parties contain anomalous data, the DAFL-HPC algorithm proposed in this application, compared with Multi-Krum, Trimmed Mean, and FedAvg, in... Figure 9 (a) MNIST, Figure 9 (b) The accuracy is improved on the FashionMNIST and KMNIST datasets (Figure 10(c)). On the MNIST dataset (Figure 10(a)), the DAFL-HPC algorithm proposed in this application improves the accuracy by 0.3 compared with other algorithms. Figure 9 (b) The accuracy of the FashionMNIST and (c) KMNIST datasets is improved by more than 0.2. In addition, the anomalous data contained in the honest participants will affect the processing of poisoned data by Multi-Krum and Trimmed Mean. The presence of anomalous data will cause the aggregation direction of the model to deviate from the normal direction. The DAFL-HPC algorithm proposed in this application can ensure that the training is directed towards the target expected by the publisher because the publisher provides a credible sample set.

[0049] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A decentralized adaptive federated learning method based on hybrid privacy computing, characterized in that, Includes the following steps: S1: The publisher registers, publishes training tasks, and classifies evaluation tasks; The assessment task classification includes the publisher constructing an assessment classifier; S2: Participants obtain training tasks and evaluation classifiers, and perform data quality assessment and stratification based on the evaluation classifiers; S3: Participants train local models and perform layered encryption on upper and lower layer models; S4: Perform layered aggregation on the encrypted model to obtain an upper-layer aggregated model and a lower-layer aggregated model. Aggregate the upper-layer aggregated model with the lower-layer aggregated model, upload the aggregation result to the blockchain, and complete one round of federated learning iteration. Repeat S2~S4 until the model performance reaches the relevant system indicators.

2. The decentralized adaptive federated learning method based on hybrid privacy computing according to claim 1, characterized in that, S1 specifically includes: S1.1: The publisher prepares identity information and submits it through the registration interface of the blockchain network; The identity information includes an identity identifier, a public key, and the dataset held by the individual. S1.2: The publisher releases the training task to the blockchain in the form of a smart contract; S1.3: The publisher utilizes its own sample dataset D Construct the classifier parameter set C ; S1.4: The publisher randomly searches the classifier parameter set. C The initial parameters are obtained from the dataset, an initial classifier is constructed based on the initial parameters, and the sample dataset is used. D The initial classifier is trained to obtain the evaluation classifier; S1.5: The publisher uploads the trained evaluation classifier and its labels to the blockchain via a smart contract.

3. The decentralized adaptive federated learning method based on hybrid privacy computing according to claim 1, characterized in that, S2 specifically includes: S2.1: The participants obtain the training task and the evaluation classifier, and calculate the data quality evaluation score, which is obtained by weighting the classifier accuracy and the amount of data; S2.2: The K-means clustering algorithm is used to divide the participants into upper and lower training sets with the highest and lowest evaluation scores as the cluster centers, and the hierarchical results are stored in the blockchain.

4. The decentralized adaptive federated learning method based on hybrid privacy computing according to claim 1, characterized in that, The layered encryption of the upper and lower layer models in S3 specifically includes: S3.1: Encrypt the upper-layer model using the PHS2 encryption method; S3.2: Use LAD encryption to encrypt the lower-level model.

5. The decentralized adaptive federated learning method based on hybrid privacy computing according to claim 4, characterized in that, The encryption of the upper-layer model using the PHS2 encryption method in S3.1 is specifically as follows: S3.1.1: Define the lower-level user set as ， ,in For the lower-level user set Specific users in the, For lower-level user sets The Middle B A lower-level user; S3.1.2: Secret Distributor Randomly select lower-level participants to form a collaborative decryption set. ,in ,in For collaborative decryption set The specific participants in the process, For collaborative decryption set The Middle t Each participating party N B Secret distributors for lower-level users ∈ Collect collaborative decryption sets The public key set of members ,in For collaborative decryption set The public keys corresponding to each participating party. Represents a collaborative decryption set Lidi t The public keys corresponding to each participant, and the collaborative decryption set Member identification set ,in For collaborative decryption set The identity identifiers corresponding to each participating party. For the first to participate in collaborative decryption t The identity of each member ID ; S3.1.3: Secret Distributor Four coprime large prime numbers are randomly selected. And calculate the plaintext encryption key parameters. , , , and The specific calculation formulas are shown in equations (1)-(5): （1） （2） （3） （4） （5） in: for Paillier Encrypted modular multiplication base, To generate auxiliary numbers for encryption, It is the least common multiple of Euler's totient function. for Paillier Auxiliary decryption function, This is the auxiliary number for the inverse of the private key. It is the largest prime number. It is the second largest prime number. It is the third largest prime number. It is the fourth largest prime number. for , yes The input variables of the function; use Paillier Encrypted modular multiplication base N Encryption generator auxiliary number g Euler's least common multiple , Paillier Auxiliary decryption function Private key inverse auxiliary number calculate Paillier The public and private key pairs are calculated using the formulas shown in equations (6) and (7): （6） （7） in: for Paillier public key pair ， for Paillier Private key pair; S3.1.4: Utilizing coprime large prime numbers calculate Public-private key pairs S3.1.5: In the... t After completing the local training, the secret distributor Get model update parameters and utilize Paillie public key Update parameters of the model Encrypt; S3.1.6: Secret Distributor Generate the first share generator polynomial The specific calculation formula is as follows: （8） （9） in: For share encryption modular multiplication parameters, Generate a polynomial for the first share. Generate a polynomial for the second share. Generate polynomial coefficients for the first share. Generate polynomial coefficients for the second share. t For training rounds; S3.1.7: Collaborative Decryption Set Each member According to the label Obtain the secret partition value The specific calculation formula is as follows: （10） In the formula: For secret partitioning value, Member identification through After mapping, substitute into the polynomial Obtain the median value. The Lagrange difference coefficient, The first secure hash function, For members Identity identifier, For multiplication, For members Belongs to the collaborative decryption set , For traversal Excluding current members k All other users besides; S3.1.8: Secret Distributor With collaborative decryption set Members negotiate to generate random numbers and , Using random numbers Construct a mask matrix The secret partition value obtained in S3.1.7 With mask matrix Adding or subtracting the elements in the array yields the encrypted secret partition value. The specific calculation formula is as follows: （11） In the formula: The encrypted secret split value, For secret partitioning value, The sum of the upper triangular parts of the mask matrix, The lower triangular sum of the mask matrix, The indices of the elements in the mask matrix. The indices of the elements in the mask matrix; S3.1.9: Constructing a set of random numbers Calculate the collaborative decryption set Each member Decryption share The specific calculation formula is as follows: ，（12） in: For the first k Decryption share parameters generated by each collaborative decryption member The second secure hash function, For members Identity identifier, As auxiliary parameters, The encryption key parameter for the secret share; S3.1.10: Decryption share obtained based on S3.1.9 Computational collaborative decryption set Each member First verification parameter First verification parameter The calculation formula is (14), and the second verification parameter is... The calculation formula is (13), and the generated , Stored on the blockchain; （13） （14） In the formula: for , For the first k Decryption share parameters generated by each collaborative decryption member The public key used to encrypt the secret share. For the first k A random number negotiated between the collaborative decryption members and the secret distributor. As the first verification parameter, The encrypted secret split value, For XOR operation, For hash functions, For polynomials Coefficients of real terms; S3.1.11: Decrypting a set collaboratively Randomly select a member and identified as To collaboratively decrypt leaders, a collaborative decryption set. Each member Receive decryption share Then, based on the generated Determine whether the data is stored on the blockchain. If the function relationship shown in equation (15) is satisfied, then calculate the encrypted secret segmentation value. Collaborative decryption set Each member Will Handed over to the leaders who collaborated on declassification Recover and decrypt the private key ; （15）。 6. The decentralized adaptive federated learning method based on hybrid privacy computing according to claim 4, characterized in that, The encryption of the lower-level model using the LAD method in S3.2 is specifically as follows: S3.2.1: Participants use local datasets to train the model and obtain local model parameters. And calculate the parameters to update the gradient. The specific calculation formula is as follows: （16） In the formula: These are the aggregated model parameters. For the first The model parameters obtained after local training on each node Update the gradient for the parameters; S3.2.2: Update the gradient of the parameters. Perform gradient clipping to obtain the clipped gradient. The specific calculation formula is as follows (17): （17） In the formula: Update the gradient for the parameters. The gradient after clipping. The cropping threshold, The L2 norm of the original gradient. This is the cropping scaling factor; S3.2.3: Calculate the test accuracy difference of the model in the current round. and with preset threshold Compare and calculate the noise scaling factor. k ; S3.2.4: According to the noise scaling factor k Differential privacy sensitivity Generate adaptive noise and add it to the clipped gradient. The gradient is obtained after privacy protection.

7. The decentralized adaptive federated learning method based on hybrid privacy computing according to claim 6, characterized in that, In S3.2.4, the differential privacy sensitivity... The specific calculation formula is as follows: （18） In the formula: For differential privacy sensitivity, For users Local dataset, For users Local dataset, To and Data sets that differ by only one record m For the upper bound, For dataset The reciprocal of the sample size For the dataset , The relevant quantity is taken as the maximum value. For dataset Summing of samples, a For parameters Below, regarding the loss function In data The parameter when taking the maximum value. For dataset Sample size Iterate through all adjacent dataset pairs ( , ),Pick The maximum value.

8. The decentralized adaptive federated learning method based on hybrid privacy computing according to claim 1, characterized in that, S4 specifically includes: S4.1: Upper-level participants adopt FedAvg The algorithm determines the aggregation weight based on the aggregation weight and the size of the node data, and uses the aggregation weight of the upper layer to aggregate the upper layer model; S4.2: Lower-level participants utilize ReLU The function calculates the cosine similarity between the current node's model update and the previous global model, and then... ReLU The cosine similarity result obtained from the function calculation is used as the weight of the lower-level aggregation parameter, and then the weights of the users in the lower-level training set are calculated. ; S4.3: User weights in the lower-level training set obtained based on S4.2 Aggregate the lower-level models; S4.4: Perform global model aggregation based on the aggregation results of the upper-level model and the aggregation results of the lower-level model.

Images