A lightweight and trusted federated learning method for edge scenarios

Abstract

The present invention discloses a lightweight trusted federated learning method for edge scenarios, comprising: (1) a terminal device trains a model locally using a private dataset and outputs a training result; (2) the training result is encrypted and uploaded to an edge server; (3) the edge server calculates the average value outmean in one dimension; (4) based on the calculated outmean, the edge server broadcasts the result to each terminal device to help the terminal device complete subsequent training; at the same time, the edge server also calculates the KL divergence using the outmean and the distillation temperature T to guide the training of global model parameters; (5) the edge server uploads the updated global model parameters to a cloud server for aggregate update; (6) the cloud server verifies whether the performance of the trained model meets the standard requirements. If so, the training ends; otherwise, the global model parameters are sent to each edge server to start the next round of training. The present invention can effectively improve the training efficiency and accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a lightweight, trusted federated learning method in edge scenarios. Background Art

[0002] In recent years, the integration of federated learning and big data has promoted the development of various industries, such as healthcare and autonomous driving. However, with the explosive growth of terminal devices, learning has also led to problems such as huge communication delays and data privacy leaks (Lin, Hui et al. "A Secure Federated Learning Mechanism for Data Privacy Protection." 2021 20th International Conference on Ubiquitous Computing and Communications (IUCC / CIT / DSCI / SmartCNS) (2021): 25-31).

[0003] To address communication latency, edge computing extends cloud servers to form a service-supporting edge close to end devices. Edge computing is typically located close to end devices, making it easy to deploy and enabling rapid responses to user requests. Federated learning and edge computing transform traditional training models, reducing the load on cloud servers and improving training efficiency. However, while horizontal federated learning uses local model gradients from different end devices to update the global model, the heterogeneity of end devices and data can affect the accuracy of the global model, resulting in limitations in the trained model. For example, small end devices cannot train large models (Bhayo, Jalal et al. “An Efficient Counter-Based DDoS Attack Detection Framework Leveraging Software Defined IoT (SD-IoT).” IEEE Access 8 (2020): 221612-221631). Furthermore, horizontal federated learning trains the same model in edge scenarios and is subject to more attacks, such as generative adversarial attacks and poisoning attacks. Furthermore, most federated learning methods use gradients as transmission objects, posing a serious threat to data privacy.

[0004] Data privacy leakage is mainly divided into data leakage based on model output and data leakage based on gradient update. For the model gradient update of federated learning, Li et al. (Yoshida, Kota et al. "Model Reverse-Engineering Attack against Systolic-Array-Based DNN Accelerator Using Correlation Power Analysis." IEICE Trans. Fundam. Electron. Commun. Comput. Sci. 104-A (2021): 152-161) proposed a DLG attack to achieve gradient at minimal cost by simulating and restoring the original data set. As for model output, Olatunji et al. (Olatunji, Iyiola E. et al. "Membership Inference Attack on Graph Neural Networks." 2021 Third IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) (2021): 11-20) proposed a model reverse attack to obtain sensitive information about training data by optimizing the difference between vectors. Diao et al. (Diao, Enmao et al. "HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients." ArXiv abs / 2010.01264(2020):n.pag) proposed a membership inference attack that infers whether a sample is in a dataset by comparing the output of the model before and after an update. Lai et al. (Sui, Dianbo et al. "FedED: Federated Learning via Ensemble Distillation for Medical Relation Extraction." EMNLP(2020)) proposed an edge intelligence collaborative privacy protection scheme for smart healthcare. This scheme divides the model into two parts for training and reduces communication overhead.

[0005] In terms of privacy protection, Kumari et al. (Bonawitz, Keith et al. "Practical Secure Aggregation for Privacy-Preserving Machine Learning." Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (2017): n.pag) proposed a secure authentication scheme based on elliptic curve cryptography for IoT and cloud servers, which is more robust, efficient, and secure against various known attacks. Bonawitz et al. (Benhamouda, Fabrice et al. "On the Local Leakage Resilience of Linear Secret Sharing Schemes." Journal of Cryptology 34 (2018): n.pag) proposed a multi-party secure aggregation scheme (SMPC) for federated learning. This scheme uses secret sharing to form a double-masked perturbation model gradient, addressing the gradient leakage problem during transmission. However, for each round of training, the terminal device must communicate with the edge server five times, which significantly increases communication overhead. In addition, high-dimensional gradient transmission consumes huge bandwidth, which is disastrous for hundreds of terminal devices in edge scenarios.

[0006] In addition, Wang et al. (Zhao, Yang et al. "Local Differential Privacy-Based Federated Learning for Internet of Things." IEEE Internet of Things Journal 8 (2020): 8836-8853) proposed PPFLEC, which places dual masks on the edge and cloud, reducing the number of entities involved in aggregation. However, it still requires multiple rounds of communication and uploading data from the terminal device to the edge server, increasing the risk of data leakage. Zhao et al. (Wang, Yichuan et al. "A trusted recommendation scheme for privacy protection based on federated learning." CCF Trans. Netw. 3 (2020): 218-228) proposed a federated differential privacy scheme that incorporates noise perturbations into the model gradients. This prevents attackers from recovering the terminal data through the gradients, but the choice of noise is a tricky issue. If the noise is too loud, the gradient will be reduced, but if the noise is too quiet, more privacy is required, and gradient leakage may still occur. Wang et al. (Zhou, Yuhao et al. "Communication-Efficient Federated Learning With Compensated Overlap-FedAvg." IEEE Transactions on Parallel and Distributed Systems 33(2020):192-205) proposed a model compression method based on the FedAvg architecture, which hides the original gradient to a certain extent, protects data privacy, and reduces data transmission and communication overhead. However, model compression may have an uncertain impact on accuracy, which is unacceptable to some people. When devices and data are heterogeneous, Sui et al. (Learning Multiple Layers of Features from Tiny Images, Alex Krizhevsky, 2009) proposed FedED, which combines knowledge distillation to improve model accuracy. However, it is based on traditional cloud servers and is not suitable for edge scenarios. At the same time, it requires the server to disclose the training dataset during model distillation, which poses the following risk of privacy leakage.

[0007] As can be seen, although the above work solves some problems, it is necessary to balance model accuracy, communication overhead, and data privacy. This has a significant impact on the training efficiency and accuracy of the model. Summary of the Invention

[0008] The purpose of this invention is to provide a lightweight and trusted federated learning method in edge scenarios, which can effectively improve the training efficiency and accuracy of the model.

[0009] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0010] A lightweight and trusted federated learning method for edge scenarios includes the following steps:

[0011] (1) The terminal device e in the third layer of the cloud-edge-end three-layer federated learning architecture i Obtain the assigned model M from the edge server in the second layer of the cloud-edge-end three-layer federated learning architecture i , and then use the private dataset to locally train the model M i Perform training and output training results O i ;

[0012] (2) Using iterative masking scheme to train the results O i Encrypt and upload to the edge server;

[0013] (3) The edge server performs training on the result O i Decrypt, and then for the training result O i , the edge server uses the following formula to calculate the average value outmean in one dimension:

[0014]

[0015] Where |I| is the number of terminal devices under the edge server;

[0016] (4) Based on the calculated outmean, the edge server broadcasts the result to each terminal device to help the terminal device complete subsequent training; at the same time, the edge server also uses the outmean and distillation temperature T to calculate the KL divergence to guide the global model parameter training. The specific guidance of the global model parameter training process is as follows:

[0017] (a) The edge server uses the local dataset D p Train a model and get an output O global , when the distillation temperature is 1, cross entropy is used to calculate its own loss. The calculation formula is as follows:

[0018]

[0019] Where Hard Loss represents the loss of the edge server itself, K represents the total number of categories, c krepresents the probability of the kth category in the data label, q k represents the probability of the kth classification, which is calculated by the following formula:

[0020]

[0021] Where, O global_k Represents output O global The kth component of global_j Represents output O global The jth component of ;

[0022] (b) The edge server calculates the probability distribution at distillation temperature T = t using the following formula:

[0023]

[0024] (c) The edge server calculates the probability distribution of the kth category at the distillation temperature T using the following formula:

[0025]

[0026] In the formula, outmean k Represents the kth component of the average value outmean, outmean j The jth component representing the mean outmean;

[0027] (d) The edge server uses the following formula to calculate the distillation loss Soft Loss :

[0028]

[0029] (e) The edge server combines its own loss and distillation loss to calculate the total loss function using the formula:

[0030] Loss global =(1-α)Hard Loss +αSoft Loss

[0031] Where α is the loss adjustment factor of the edge server, ranging from 0 to 1;

[0032] (f) The edge server performs gradient updates on the training model based on the calculated total loss to obtain the updated global model parameters ω u ;

[0033] (5) The edge server will update the global model parameters ω u The data is uploaded to the cloud server in the first layer of the cloud-edge-end three-layer federated learning architecture. The cloud server uses average weights to aggregate and update the global model parameters. The update formula is as follows:

[0034]

[0035] Where |U| represents the number of edge clusters;

[0036] (6) The cloud server verifies whether the performance of the training model meets the standard requirements based on the global model parameters obtained by cloud aggregation. If yes, the training ends; otherwise, the global model parameters are sent to each edge server to start the next round of training and execute step (7);

[0037] (7) Terminal equipment e i Using the distillation temperature T and outmean, calculate the model M i distillation losses;

[0038] (8) Terminal equipment e i Combine the self-loss and distillation loss as the loss function of this round Loss i , and then according to the loss function Loss i Take the partial derivative of the model parameters to get the gradient G i ;

[0039] (9) Terminal equipment e i According to the gradient G i Use backpropagation to update the parameters, end this round of model training, obtain the updated model, and then return to step (2).

[0040] The step (1) is specifically as follows: the terminal device e i Get the assigned model M from the edge server i , and using private dataset D i Training is performed locally, where D i ={(x i1 ,y i1 ),(x i2 ,y i2 )…(x id ,y id )}, d is the dataset D i The length of the private data set D i Divide into training data sets and test dataset Among them, using the training data set The characteristics of x i As Model M i Input, after forward propagation, the training result O is obtained i , by comparing with the data label y i Perform cross entropy calculation to obtain its own loss function, the calculation formula is as follows:

[0041]

[0042] Where q ik Indicates e i The probability of the kth category, q ik is calculated as follows:

[0043]

[0044] Where, O ik Represents the training result O i The kth component of ij Represents the training result O i The jth component of .

[0045] Specifically, in step (7), the calculation model M i The distillation loss process is as follows:

[0046] (g) Calculate the probability distribution of the local model at the distillation temperature T using the following formula:

[0047]

[0048] Where q ik_T Indicates e i The probability of the kth category at the distillation temperature T;

[0049] (h) The probability distribution of the polymerization results at the distillation temperature T is calculated using the following formula:

[0050]

[0051] (i) Calculate the distillation loss using the following formula:

[0052]

[0053] (j) represents the distillation loss of the i-th terminal device, which is calculated by the cross entropy of the probability of the aggregation result and the probability of the model at the distillation temperature T.

[0054] Furthermore, in step (8), the terminal device e is calculated using the following formula: i The loss function of this round is Loss i :

[0055]

[0056] Where, α i Represents the loss adjustment factor, which ranges from 0 to 1 and is a balance between the self-loss and the distillation loss.

[0057] Furthermore, in step (9), the following formula is used to update the parameters:

[0058] ω i =ω i -ηG i

[0059] Where, ω i It is e i The model parameters on , η is the learning rate, and its value range is greater than 0.

[0060] Specifically, the step (2) includes the following steps:

[0061] (2a) The edge server records the numbers of the terminal devices participating in this training, which are recorded as set I. Generate two pairs of public and private keys based on the parameter pp: KA.gen() represents the public-private key generation algorithm; for edge servers, the public-private key pair is generated using the parameter pp.

[0062] (2b) The edge server for each terminal device e in set I i , use the DH key exchange algorithm to obtain an encrypted symmetric key s u,i , the formula is as follows:

[0063]

[0064] By s u,i The symmetry of s u,i =s i,u , the key is used for encrypted transmission of the model result mask;

[0065] (2c) Before the first round of training, each terminal device in set I randomly generates a random number seed seed i And use the pseudo random number generator PRG to generate a random vector b i , which is then expanded into vector b prai , with output The same size as the output mask, the formula is as follows:

[0066]

[0067] Where, Indicates the results obtained in the first round of training;

[0068] (2d) Mask the result b prai and Add together to get And use s u,iAs the key, and then use the encryption algorithm AE.enc() to encrypt, and then the encrypted training result O i Upload to the edge server, the encryption formula is as follows:

[0069]

[0070] Furthermore, in step (3), the decryption process of the edge server is: the edge server obtains e i,u Then use the symmetric key s u,i Decryption is performed using the decryption algorithm AE.dec(). The decryption formula is as follows:

[0071]

[0072] In the tth round of training, the calculation formula of the mask result is as follows:

[0073]

[0074] Where, Represents the terminal device e in round t i The results of the training, Represents the mask result of the (t-1)th round.

[0075] Compared with the prior art, the present invention has the following beneficial effects:

[0076] Based on the cloud-edge-terminal architecture, the present invention constructs a multi-model federated learning architecture that can improve the efficiency of model training. At the same time, by designing a lightweight iterative masking scheme at the edge, the present invention can not only reduce communication overhead and solve problems such as data silos and significant communication delays, but also ensure data privacy and reliability. In addition, the present invention also uses different combinations to implement multi-model federated training and transfers knowledge through integrated distillation, solving the problem of low training volume and improving the accuracy caused by device and data heterogeneity in edge scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 This is a schematic diagram of the model used in the embodiment of the present invention.

[0078] Figure 2 Schematic diagram of the federated learning architecture adopted in the embodiment of the present invention.

[0079] Figure 3 Schematic diagram of training global model parameters in an embodiment of the present invention.

[0080] Figure 4 A schematic diagram of the loss and accuracy of the CIFAR10 dataset using Resnet18 as the local model and Resnet20 as the global model in this embodiment.

[0081] Figure 5 This figure shows the loss and accuracy of the CIFAR10 dataset using Resnet18 and Vgg16 as local models and Resnet20 as the global model in this embodiment.

[0082] Figure 6 This is a diagram showing the loss and accuracy of the CIFAR10 dataset when using Resnet18, VGG16, and VGG19 as local models and Resnet20 as the global model under ELTFL.

[0083] Figure 7 2 is a comparison chart of accuracy on the CIFAR10 dataset (ELTFL in the figure represents an embodiment of the present invention).

[0084] Figure 8 A diagram showing the loss and accuracy of the CIFAR100 dataset using Resnet18 as the local model and Resnet20 as the global model under ELTFL.

[0085] Figure 9 This figure shows the loss and accuracy of the CIFAR100 dataset using Resnet18 and Vgg16 as local models and Resnet20 as the global model under ELTFL.

[0086] Figure 10 This figure shows the loss and accuracy of the CIFAR100 dataset using Resnet18, Vgg16, and Resnet8 as local models and Resnet20 as the global model under ELTFL.

[0087] Figure 11 This figure shows the loss and accuracy of the CIFAR100 dataset using Resnet8, Vgg16, and ShuffleV1 as local models and Resnet20 as the global model under ELTFL.

[0088] Figure 12 2 is a comparison chart of accuracy on the CIFAR100 dataset (ELTFL in the figure represents an embodiment of the present invention).

[0089] Figure 13 Schematic diagram of the time overhead on the CIFAR10 dataset (ELTFL in the figure represents an embodiment of the present invention).

[0090] Figure 14 Schematic diagram of the time overhead on the CIFAR100 dataset (ELTFL in the figure represents an embodiment of the present invention).

[0091] Figure 15Schematic diagram of a successful DLG attack.

[0092] Figure 16 Schematic diagram of a failed DLG attack.

[0093] Figure 17 Schematic diagram of DLG’s attack success rate on four schemes. DETAILED DESCRIPTION

[0094] The present invention will be further described below with reference to the accompanying drawings and examples. The embodiments of the present invention include but are not limited to the following examples.

[0095] Example

[0096] This embodiment provides an efficient and lightweight trusted federated learning solution that is suitable for edge scenarios and can effectively improve the training efficiency and accuracy of the model. Figure 1 As shown in the figure, the terminal device performs model training locally. Edge aggregation involves the terminal and the cloud aggregating models from different edge clusters. Specifically, after receiving the distributed model at the edge (edge ​​server), the terminal device trains the model locally using a private dataset. After multiple iterations of training, masks are added, and then the gradient updates are uploaded to the edge server. After unmasking the gradient updates at the edge, they are used to aggregate the gradient updates. The gradients are then used to update the parameters, and finally the parameters are uploaded to the cloud (cloud server).

[0097] The cloud server collects the parameters uploaded by each edge cluster, aggregates them, and updates them to the parameters for this training round when evaluating model performance. If performance does not meet the standards, it will issue parameter updates for the next round of training. However, this model has problems such as heterogeneous data and significant communication overhead, so the improvement ideas of this embodiment are as follows:

[0098] Considering the task M published by the cloud, different entities can choose different models to achieve the same function according to their hardware. Suppose a large model is required on the server to achieve higher accuracy, but the terminal device only needs high accuracy within a specific data range. In this case, some small models can be selected. Therefore, a small model can be selected as a local model on the terminal device, and then a large model can be selected as a global model on the terminal device to implement standard functions on the cloud and edge. To achieve this goal, this embodiment introduces knowledge distillation global model mutual learning, that is, improving the first and third steps to meet functional requirements and improve model performance. In order to reduce communication overhead, this embodiment reduces the dimensionality of the parameter mask to improve knowledge extraction and adds iteration masks in the model output, improving the second step to reduce traffic and protect data privacy.

[0099] The entire process is described in detail below.

[0100] 1. Parameter definition

[0101] The variable parameters and related explanations required in the model are shown in Table 1:

[0102] Table 1

[0103]

[0104] The relevant algorithms for mask addition are shown in Table 2:

[0105] Table 2

[0106] Serial number Algorithm Name Algorithm Introduction 1 KA.param() Generation algorithm of parameter pp 2 KA.gen() Public and private key generation algorithm 3 KA.agree() Symmetric key generation algorithm 4 AE.enc() Key encryption algorithm 5 AE.dec() Key decryption algorithm

[0107] 2. Program Process

[0108] Assume that the terminal device set E={e1,e2,…e i …e n}, terminal device e i With private dataset D i , divided into training data sets And the test dataset responsible for local model training It is worth noting that running on terminal devices is usually lightweight. In order to effectively train edge models, this embodiment adds an integrated distillation learning edge server with a cluster dataset D that performs global model training. p The cloud server coordinates the edge server cluster and aggregates the model across multiple clusters, which is consistent with the model definition of this embodiment. Its learning architecture is as follows Figure 2 shown.

[0109] Specifically, the main process of this embodiment is as follows:

[0110] First, the terminal device e i Get the assigned model M from the edge server i , and using private dataset D i Training is performed locally, where D i ={(x i1 ,y i1 ),(x i2 ,y i2 )…(x id ,y id )}, d is the dataset D i The length of the private data set D i Divide into training data sets and test dataset Among them, using the training data set Features of x i As Model M i Input, after forward propagation, the training result O is obtained i, by comparing with the data label y i Perform cross entropy calculation to obtain its own loss function, the calculation formula is as follows:

[0111]

[0112] Where q ik Indicates e i The probability of the kth category, q ik is calculated as follows:

[0113]

[0114] Where, O ik Represents the training result O i The kth component of ij Represents the training result O i The jth component of .

[0115] While calculating the loss function, the terminal device e i The training result O i Perform lightweight encryption to obtain E i The detailed encryption scheme will be introduced below. After encryption, the terminal device e i E i After sending it to the edge server, we will wait for the edge server to feedback the distillation temperature T and the outmean of the aggregated result. Using T and outmean, we can calculate the model M. i The distillation loss is as follows:

[0116] 1) Terminal device e i Using the distillation temperature T and outmean, calculate the model M i The distillation loss is calculated as follows:

[0117] 11) The probability distribution of the local model at the distillation temperature T is calculated using the following formula:

[0118]

[0119] Where q ik_T Indicates e i The probability of the kth category at the distillation temperature T;

[0120] 12) The probability distribution of the polymerization results at distillation temperature T is calculated using the following formula:

[0121]

[0122] 13) Calculate the distillation loss using the following formula:

[0123]

[0124] 14) represents the distillation loss of the i-th terminal device, which is calculated by the cross entropy of the probability of the aggregation result and the probability of the model at the distillation temperature T;

[0125] 2) Terminal equipment i Combine the self-loss and distillation loss as the loss function of this round Loss i , loss function Loss i The calculation formula is as follows:

[0126]

[0127] Where, α i Represents the loss adjustment factor, with a value range of 0-1, which is the balance between the self-loss and the distillation loss;

[0128] 3) According to the loss function Loss i Take the partial derivative of the model parameters to get the gradient G i :

[0129]

[0130] 4) Terminal equipment i According to the gradient G i Back propagation is used to update the parameters, ending this round of model training to obtain the updated model. The parameter update formula is as follows:

[0131] ω i =ω i -ηG i

[0132] Where, ω i It is e i The model parameters on η are the learning rate, which ranges from 0 to 0, and is generally 0.01. At this point, this round of training is completed, and the updated local model is obtained. The model accuracy is verified using the test set, and the edge server is waiting for scheduling.

[0133] As mentioned above, the terminal device e i The training result O i Perform lightweight encryption to obtain E i , and then sent to the edge server. Specifically, this embodiment uses an iterative masking scheme to train the results O i Encrypt and upload to the edge server as follows:

[0134] 1) The edge server records the numbers of the terminal devices participating in this training, which are recorded as set I. Generate two pairs of public and private keys based on the parameter pp: pp is a parameter related to the generation of public and private keys, which is generated by the secure prime number k through KA.param(k). For edge servers, the parameter pp is used to generate the public and private key pair.

[0135] 2) The edge server is responsible for each terminal device e in the set I. i , use the DH key exchange algorithm to obtain an encrypted symmetric key s u,i , the formula is as follows:

[0136]

[0137] By s u,i The symmetry of s u,i =s i,u , the key is used for encrypted transmission of the model result mask;

[0138] 3) Before the first round of training, each terminal device in set I randomly generates a random number seed seed i And use the pseudo random number generator PRG to generate a random vector b i , which is then expanded into vector b prai , with output The same size as the output mask, the formula is as follows:

[0139]

[0140] Where, Indicates the results obtained in the first round of training;

[0141] 4) Mask the result b prai and Add together to get And use s u,i As the key, and then use the encryption algorithm AE.enc() to encrypt, and then the encrypted training result O i Upload to the edge server, the encryption formula is as follows:

[0142]

[0143] Edge servers get e i,u Then use the symmetric key s u,i Decryption is performed using the decryption algorithm AE.dec(). The decryption formula is as follows:

[0144]

[0145] In the tth round of training, the calculation formula of the mask result is as follows:

[0146]

[0147] Where, Represents the terminal device e in round t i The results of the training, Denotes the masked result of the (t-1)th round. Since the edge server u has the masked results of each round, it can quickly remove the mask, obtain the result set, and perform subsequent distillation after aggregation.

[0148] The iterative masking and decryption scheme is summarized as follows:

[0149] Input: finite field F, parameter pp, output O of terminal device set I i , training round number epoch

[0150] The edge server u uses the parameter pp to generate a public-private key pair

[0151] e i Generate a public-private key pair using the parameter pp

[0152] For i in range I:

[0153]

[0154] End For

[0155] For i in range I:

[0156] Initialize the mask:

[0157]

[0158]

[0159] End For

[0160] For t in range epoch:

[0161] For i in range I:

[0162]

[0163]

[0164] Sent to edge server u

[0165] Edge server u decryption:

[0166]

[0167] Decrypted

[0168]

[0169] End For

[0170] End For.

[0171] Then, the edge server uses the following formula to calculate the average value outmean in one dimension:

[0172]

[0173] Where |I| is the number of terminal devices under the edge server.

[0174] Then, based on the calculated outmean, the edge server broadcasts the result to each terminal device to help the terminal device complete subsequent training; at the same time, the edge server also uses the outmean and distillation temperature T to calculate the KL divergence to guide the global model parameter training, such as Figure 3 shown.

[0175] according to Figure 3 , this embodiment specifically guides the global model parameter training process as follows:

[0176] 1) The edge server uses the local dataset D p Train a model and get an output O global , when the distillation temperature is 1, cross entropy is used to calculate its own loss. The calculation formula is as follows:

[0177]

[0178] Where Hard Loss represents the loss of the edge server itself, K represents the total number of categories, c k represents the probability of the kth category in the data label, q k represents the probability of the kth classification, which is calculated by the following formula:

[0179]

[0180] Where, O global_k Represents output O global The kth component of global_j Represents output O global The jth component of ;

[0181] 2) The edge server calculates the probability distribution at distillation temperature T = t using the following formula:

[0182]

[0183] 3) The edge server calculates the probability distribution of the kth category at distillation temperature T using the following formula:

[0184]

[0185] In the formula, outmean k Represents the kth component of the average value outmean, outmean j The jth component representing the mean outmean;

[0186] 4) The edge server uses the following formula to calculate the distillation loss Soft Loss :

[0187]

[0188] 5) The edge server combines its own loss and distillation loss to calculate the total loss function using the formula:

[0189] Loss global =(1-α)Hard Loss +αSoft Loss

[0190] Where α is the loss adjustment factor of the edge server;

[0191] 6) The edge server performs gradient updates on the training model based on the calculated total loss to obtain the updated global model parameters ω u .

[0192] Then, the edge server updates the global model parameters ω u Upload to the cloud server, and the cloud server uses average weight to aggregate and update the global model parameters. The update formula is as follows:

[0193]

[0194] Where |U| represents the number of edge clusters.

[0195] The distillation process of this embodiment is summarized as follows:

[0196] Input: terminal device set I, terminal local model M i , dataset D i , where D i =(x id ,y id ), batchsize, number of training rounds epoch, learning rate η, the value range is greater than 0.

[0197] For t in range epoch:

[0198] For i in range I:

[0199] D i Divide data into small batches according to the batch size

[0200] For(x id ,y id )in

[0201]

[0202]

[0203] End For

[0204] Send after adding mask To the edge server u, wait until the edge server feedback distillation temperature T and outmean t Client:

[0205] Calculate the probability distribution at distillation temperature T = 1:

[0206]

[0207] Calculate the probability distribution at distillation temperature T = T:

[0208]

[0209] Calculate outmean t Probability distribution at distillation temperature T = T:

[0210]

[0211] Calculate your own losses:

[0212]

[0213] Calculate distillation loss:

[0214]

[0215] Calculate the total loss function:

[0216]

[0217] Compute parameter gradients:

[0218]

[0219] use Backpropagation to update parameters:

[0220]

[0221] End for

[0222] Edge server u:

[0223] Receive from terminal device set I After decryption, calculate the average aggregation result outmean t :

[0224]

[0225] will be outmean t Send to terminal device

[0226] Using dataset D p Train the global model and get the output

[0227]

[0228] Calculate the probability distribution when T = 1 and T = T:

[0229]

[0230]

[0231] Calculate the own loss and distillation loss of the global model:

[0232]

[0233]

[0234] Calculate the loss function:

[0235]

[0236] Calculate the gradient and backpropagate to update the global model parameters:

[0237]

[0238]

[0239] End For.

[0240] Finally, the cloud server verifies whether the performance of the trained model meets the standard requirements based on the global model parameters obtained by cloud aggregation. If yes, the training ends; otherwise, the global model parameters are sent to each edge server to start the next round of training.

[0241] The scheme of this embodiment is theoretically analyzed.

[0242] 1. Performance indicators

[0243] Top-k accuracy: This is used to calculate the proportion of correct labels predicted among the top k results. In our experiments, we use top-k as the accuracy evaluation metric. The common top-1 is the classification accuracy, as shown in the following formula.

[0244]

[0245] Among them, True Positive (TP) means that the true class of the sample is positive, and the model recognition result is also positive; True Negative (TN) means that the true negative class is negative, and the model recognizes it as negative; False Negative (FN) means that the true class of the sample is positive, but the model recognizes it as negative; False Positive (FP) means that the true class of the sample is negative, but the model recognizes it as positive.

[0246]

[0247] Here, |.| represents the amount of data, |Data| also represents the communication overhead, and bandwidth represents the bandwidth. Communication latency and model training time are important metrics for measuring computing and communication overhead.

[0248] 2. Communication Overhead Analysis

[0249] The data from different devices in edge computing scenarios is mostly heterogeneous or incomplete, but this data can help edge servers quickly train models. Most current work uses federated learning to transmit data between terminal devices and edge servers. However, the traffic is huge, and generating the same dimensions as the parameters is also a considerable expense. This embodiment combines federated learning and knowledge distillation to reduce communication costs and protect data privacy to a certain extent. The specific communication overhead comparison is shown in Table 3:

[0250] Table 3

[0251] plan Communication overhead (data volume) Number of communications per round FedAvg <![CDATA[O(mn 4 )]]> (I+U) SMPC <![CDATA[O(mn 4 )]]> (5I+U) PPFLEC <![CDATA[O(mn 3 )]]> (I+5U) This embodiment scheme O(mn) (I+U)

[0252] Where m represents the number of samples in each training round, n represents the vector dimension, I represents the number of end devices, and U represents the number of edge servers. Since FedAvg and SMPC have a two-layer architecture, we expand it to a three-layer architecture, where the edge server to cloud server only performs model parameter aggregation.

[0253] Specifically, taking a convolutional neural network (CNN) as an example, the parameters of each layer are 4-dimensional tensors (m,n,p,q), and there are k layers in total. The mask matrix required to add a mask to the gradient or parameter is nested and contains m*n*p*q*k data. Assuming the tensor dimensions are the same, the amount of data passed is mn 4 , which will not only take up a lot of bandwidth, but also the generation process of the nested matrix will consume a lot of computing resources, especially the multiplication of high-dimensional tensors. From Table 3, we can see that the communication data of FedAvg and SMPC is mn 4 PPFLEC aggregates the datasets on the terminal devices and transmits them to the edge server. Since the image data is a three-dimensional tensor, the data volume is mn 3 In English teaching, only the results of training private data and vector masks need to be sent to the edge server, and the data volume is m*n. In the scheme, the high-dimensional gradients or parameters are reduced to one dimension, which reduces the communication volume.

[0254] Regarding the number of communication rounds, FedAvg only needs to upload and distribute between the end and the edge, and the edge and the cloud. The number is 2(I+U). However, SMPC has added dual masks. In one round of training, it requires five communications between the edges and one communication between the edge and the cloud, so the total number is 5I+U, which is not suitable for edge scenarios. PPFLEC extends the cloud to the edge, but it needs to add five masks in the edge cloud and one I+5U communication at the edge. This embodiment proposes an iterative mask that can achieve secure transmission of model results without multiple communications in one round of training. It only needs to communicate once during the key agreement. After training starts, it is the same as FedAvg.

[0255] Furthermore, edge servers can accelerate convergence and improve model accuracy by extracting models collected by terminal devices. They also provide positive feedback for local model training on terminal devices. For heterogeneous edge scenarios, edge servers offer a significant advantage in training intelligent tasks with the same objective. This makes them suitable for federated learning in horizontal edge scenarios.

[0256] 3. Privacy Analysis

[0257] This embodiment analyzes the privacy protection capability of the solution of this embodiment for terminal private data from two aspects: model architecture and iterative masking.

[0258] First, in this embodiment, the transmission between the edge and the end is not the model gradient, but the model training result. i As an example, take the model result O i Instead of model gradient O i, reduces traffic and effectively prevents gradient leakage attacks when the model structure is a black box. Secondly, we transmit model parameters between edge clouds. Federated learning training data is local. Attackers cannot obtain loss values ​​or calculate gradients to conduct gradient leakage attacks. However, the transmission of model results between edges also faces the risk of model theft and privacy leakage, especially in different models. The output results may contain input information, resulting in the leakage of sensitive information in the training set. An attacker can infer whether there are samples in the training set through attacks on members, but in this embodiment, an iteration mask is added to the output results, and the attacker cannot locate the sample through the mask. At the same time, the timestamp can ensure the freshness of the results and prevent the attacker from repeating the attack.

[0259] The model performance of this embodiment is verified through case experiments.

[0260] 1. Experimental Configuration

[0261] The experiment used 24 PCs as end devices, a high-performance server using an RTX3090 graphics card, a high-performance server using an RTX3080 graphics card as an edge server, and a cloud server. Each edge server was responsible for an edge cluster consisting of 12 end devices. The datasets were CIFAR10 and CIFAR100. Specifically, two edge servers were used. The dataset was divided into 24 parts; each end device received one part as a training and testing dataset. We used PyTorch as the framework and employed a combination of the ResNet residual sequence model, VGG sequence, and the ShuffleV1 model for federated learning. We primarily conducted experimental analysis on top-k accuracy, computational and communication overhead, and privacy protection within the model performance. The comparison options were FedAvg, SMPC, and PPFLEC.

[0262] 2. Accuracy Analysis

[0263] We use the CIFAR10 and CIFAR100 datasets as experimental data and use the ResNet20 model as the global model for training on the edge server. We use 1, 2, and 3 small models as local models for training on the terminal device (we distribute the small models evenly to the terminal devices). For the CIFRA10 dataset, ResNet18, VGG16, and VGG19 are used as local models to obtain Figure 4 、 Figure 5 and Figure 6 The accuracy and loss curves shown. Figure 4It shows that the loss value and accuracy of ResNet18 and ResNet20 during training show synchronous changes. Clearly, the loss value reaches the lowest at about 80 rounds, the training accuracy reaches the peak at about 80 rounds, and then reaches overfitting, showing volatility. On the test set, the two models of this embodiment converge at about 100 rounds. The test loss value of the global model Resnet20 is reduced to 0.28, and the test accuracy is 91.82%. After convergence, it is in a smooth state, indicating that the embodiment has good generalization ability. Figure 5 As shown in Figure 2, when we add VGG16 to the local model, the trend of the original loss value and accuracy changes. The number of epochs during training is also about 80, which is consistent with the Figure 4 . The difference is that the number of convergence rounds on the test set is about 95, and the convergence rate reaches 92.16% when the test accuracy converges. Compared with the training using only one local model, the number of convergence rounds is slightly reduced, the test accuracy is slightly improved, and the smoothness of the test set is maintained after the model converges. From this point of view, the present embodiment scheme accepts the number of local models well. The more local models, the higher the accuracy of ELFL and the faster the convergence speed. However, as the number of local models increases, the present embodiment scheme will be over-filled, resulting in performance degradation. We did a simple experiment to increase the local again using three models as local models, the first two remained unchanged, and obtained Figure 6 The experimental results are shown.

[0264] from Figure 6 We can see that the training with three models as the local model converged in 78 rounds, which is slightly less than the training with two models as the local model. The loss value at convergence on the test set remained at 0.25, and the test accuracy reached 92.11% at convergence, a slight decrease, but within the experimental error. Therefore, we can conclude that this embodiment can adapt to local models composed of multiple models and is suitable for edge scenarios with heterogeneous models and data.

[0265] We compared this embodiment with FedAvg, SMPC, and PPFLEC, and obtained a global model accuracy comparison chart, as shown in Figure 7 As shown in the figure, this embodiment uses three local models for training; the global model is ResNet20.

[0266] Figure 7It shows that the training efficiency of the scheme of this embodiment is much higher than that of the other three schemes. The accuracy of the first ten rounds can reach 75%, and the model converges in about 100 rounds, with an accuracy of 92.11% at convergence. In comparison, the baseline scheme FedAvg improves the accuracy faster in the first 50 rounds and the accuracy is 86.76% at convergence. At the same time, the training efficiency of SMPC and PPFLEC is slower than that of PPFLEC, and the impact of PPFLEC is significant. Finally, the accuracy of SMPC and PPFLEC can reach 81.88% and 82.99% respectively by adding and removing double masks. According to the accuracy of the global model and the number of convergence rounds, the scheme of this embodiment is much higher than the three comparison schemes, and the accuracy is 5.35% higher than the baseline scheme on a federal average. In addition, we compared the accuracy of the local models of FedAvg and the scheme of this embodiment, as shown in Table 4:

[0267] Table 4

[0268] Model This embodiment (%) FedAvg(%) Resnet18 94.51 83.64 Vgg16 93.35 91.62 Vgg19 91.72 84.5

[0269] As can be seen from Table 4, the local models trained by the solution in this embodiment are all higher in accuracy than FedAvg. Among them, Resnet18 improves by 10.87% over FedAvg, Vgg16 improves by 1.73% over FedAvg, and Vgg19 improves by 7.22% over FedAvg.

[0270] In order to verify the performance of this embodiment on other datasets, we use the CIFAR100 dataset for experiments. The local models are Resnet18, Resnet8, ShuffleV1 and Vgg16, and the global model is still Resnet20. Figure 8 、 Figure 9 、 Figure 10 and Figure 11 The loss values ​​and accuracy are shown.

[0271] from Figure 8 As can be seen, the solution in this embodiment converged after approximately 75 epochs on the training set, and then exhibited a waveform. The global ResNet20 model converged after approximately 110 epochs on the test set. The test convergence accuracy reached 68.34%, corresponding to a loss of 1.14. The local and global models showed similar trends and smooth stability after convergence on the test set. This embodiment has good versatility across various datasets.

[0272] Similarly, we use ResNet18 and VGG16 models as local models to train the global model ResNet20. Figure 9As shown in Figure 2, during the training process, the global model converged in about 80 rounds. During the test process, convergence was achieved at about 110 rounds. The accuracy reached 68.03%, which is 0.31% lower than when the local ResNet18 model was trained. This may be due to the poor effect of the CIFAR100 dataset on the VGG16 model. After convergence, the model accuracy curve is in a smooth state. In order to test the impact of model heterogeneity on the accuracy of the global model, we selected three different models as local models. We obtained Figure 10 and Figure 11 The experimental results.

[0273] from Figure 10 and Figure 11 It can be seen that when ResNet8, ResNet18, and VGG16 are used as local models, the convergence accuracy of the global model ResNet20 is 68.36% and the loss value is 1.15. When ResNet8, VGG8, and ShuffleV1 are used as local models, the accuracy of the global model is 67.49%, the loss value is 1.15, and the accuracy is reduced by 0.87%. The heterogeneity of the local model has a special impact on the solution of this embodiment, but the difference is not significant. As the number of distillation models integrated in the edge scenario increases, the global model trained by the solution of this embodiment will tend to be stable and have stronger generalization ability.

[0274] In addition, we compared the solution of this embodiment with FedAvg, SMPC and PPFLEC on the CIFAR100 dataset and obtained Figure 12 The global model accuracy comparison chart is shown. This embodiment uses three local models for training; the global model is ResNet20.

[0275] from Figure 12 As can be seen, on the CIFAR100 dataset, the accuracy of this embodiment's solution remains relatively high, exceeding the baseline solution FedAvg by 9.44% to 67.94%. In the case of insufficient data, the convergence accuracy of SMPC and PPFLEC is only 51.12% and 47.14%, respectively, 7.38% and 11.36% lower than that of FedAvg. This shows that this embodiment's solution can still maintain the performance of SMPC and PPFLEC on CIFAR100, while their performance on CIFAR10 is much worse. In summary, this embodiment's solution has good generalization capabilities when data is heterogeneous.

[0276] Similarly, we compared the accuracy of FedAvg and the local model in this embodiment. The results are shown in Table 5:

[0277] Table 5

[0278]

[0279] Table 5 shows that the local model obtained from the training accuracy of this embodiment is higher than that of FedAvg. In particular, the accuracy of the VGG8 model, which had poor training results with FedAvg, increased by 41.34% after training with this embodiment. This embodiment not only performs well in training global models, but also outperforms the FedAvg baseline in training local models. This embodiment can train global and local models through comprehensive distillation training, while traditional federated learning solutions such as FedAvg can only achieve this sequentially. From this perspective, this embodiment can also significantly shorten model training time.

[0280] 3. Computation and Communication Overhead Analysis

[0281] Regarding computational and communication overhead, we used the time required for each training round and the communication time required for each transmission round as metrics and compared this embodiment with FedAvg, SMPC, and PPFLEC. The datasets were CIFAR10 and CIFAR100. This embodiment used three local models (ResNet18, VGG16, and VGG19), while the global training model was ResNet20. The results are shown in Tables 6 and 7:

[0282] Table 6 Table 7

[0283]

[0284] As shown in Table 6, using the CIFAR10 dataset to train the ResNet20 model, this embodiment requires 120 epochs to achieve convergence. This number of epochs is fewer than the baseline FedAvg and significantly fewer than SMPC and PPFLEC. Multi-modal integrated distillation accelerates convergence. Compared to the dual masking and hashing functions used by SMPC and PPFLEC, the iterative masking used in this embodiment has less impact on model accuracy. The training time per epoch of this embodiment is 110.7 seconds, slightly higher than FedAvg. This is due to the iterative masking computation, but much lower than SMPC and PPFLEC, reflecting the lightweight nature of this embodiment. In terms of per-epoch communication time, this embodiment requires the shortest communication time because the data transmitted during communication is of low dimensionality—the product of the model result and the data batch dimensions. In contrast, the data transmitted by the other three embodiments is the dimensionality of the model parameters.

[0285] Similarly, Table 7 shows that as the dataset is replaced, CIFAR100 has more classification types. During the convergence process, the solution of this embodiment requires far fewer rounds than the other three solutions. In terms of training time per round, the solution of this embodiment is lower than FedAvg and PPFLEC, and far lower than SMPC. The solution of this embodiment also has the lowest communication time per round. In summary, the solution of this embodiment has low computational and communication overhead and a faster convergence rate.

[0286] In order to verify the training efficiency of different devices, we counted the total time for the two clusters to converge on different data sets, including training time and communication time, e.g. Figure 13 and Figure 14 shown.

[0287] from Figure 13 and Figure 14 It can be seen that the time cost of the scheme of this embodiment is lower than that of the other three schemes on different device clusters and training data sets. Specifically, the time cost of the scheme of this embodiment is slightly lower than that of FedAvg on the CIFAR10 dataset, and much lower than that of SMPC and PPFLEC. On the CIFAR100 dataset, it is higher than the other three schemes. For different devices, the time overhead of the scheme of this embodiment varies little, within 300 seconds, while the time overhead of FedAvg varies by more than 800 seconds, the time overhead of SMPC exceeds 1700 seconds, and the time overhead of PPFLEC exceeds 700 seconds. In short, the scheme of this embodiment is more suitable for devices and has more advantages in the case of heterogeneous devices in edge scenarios.

[0288] 4. Privacy Protection Analysis

[0289] We use the DLG attack to attack four schemes to test whether the original image can be recovered from the transferred data, thereby compromising the privacy of the original data. The DLG attack is one of the most effective attacks against traditional federated learning gradient transfer, which can recover the dataset by simulating the model's gradients. We randomly select images from the CIFAR100 dataset. The DLG attack randomly generates images. We obtain our selected images by simulating the gradients of each training round.

[0290] In the experiment, if Figure 15 , then the DLG attack is successful. At the same time, the loss value of the simulated gradient is 10000 at the beginning. The trend is 0.91 in the 30th round, 0.03 in the 60th round, 0.01 in the 90th round, and then 0. The image has been completely restored. If Figure 16, the DLG attack failed with a loss value of 83786.1. After 270 simulation iterations, the loss value was 83746.8, and the original image could not be restored. We tested 6400 images of the CIFAR10 dataset and obtained Figure 17 The attack results are shown.

[0291] exist Figure 17 In the figure, ASR represents the attack success rate. The yellow portion represents attacks between the terminal and the edge, and the green portion represents attacks between the edge and the cloud. The ASR for the DLG attack between the edge and the edge cloud is 0 because the iterative masking at the edge causes the attacker to intelligently obtain false gradients that cannot be recovered. Due to the parameters transmitted between the edge and the cloud, gradients cannot be calculated, and the image cannot be recovered. When attacking FedAvg, the success rate between the edge cloud and the edge is over 99.9%. It can be seen that federated learning without encryption or masking is prone to data leakage. When attacking SMPC, the attack success rate is 0 because double masks are added between the edges. However, we do not add masks between the edge and the cloud to recover the image. However, the image data obtained by the attacker is based on the aggregated gradients and is not necessarily the original data. In PPFLEC, the edge data is unprotected and directly aggregated to the edge server, which is prone to leakage. Due to the masking and hashing functions between the edge and the cloud, the data cannot be recovered. In summary, this embodiment is more suitable for privacy-preserving training of intelligent models in cloud scenarios.

[0292] The above embodiment is only one of the preferred implementation methods of the present invention and should not be used to limit the scope of protection of the present invention. Any changes or modifications that have no substantive meaning in the main design concept and spirit of the present invention, as long as the technical problems they solve are still consistent with the present invention, should be included in the scope of protection of the present invention.

Claims

1. A lightweight and trusted federated learning method for edge scenarios, characterized by: The following steps are involved: (1) The terminal device e in the third layer of the cloud-edge-end three-layer federated learning architecture i Obtain the assigned model M from the edge server in the second layer of the cloud-edge-end three-layer federated learning architecture i , and then use the private dataset to locally train the model M i Perform training and output training results O i ; (2) Using iterative masking scheme to train the results O i Encrypt and upload to the edge server; (3) The edge server performs training on the result O i Decrypt, and then for the training result O i , the edge server uses the following formula to calculate the average value outmean in one dimension: Where |I| is the number of terminal devices under the edge server; (4) Based on the calculated outmean, the edge server broadcasts the result to each terminal device to help the terminal device complete subsequent training; at the same time, the edge server also uses the outmean and distillation temperature T to calculate the KL divergence to guide the global model parameter training. The specific guidance of the global model parameter training process is as follows: (a) The edge server uses the local dataset D p Train a model and get an output O global , when the distillation temperature is 1, cross entropy is used to calculate its own loss. The calculation formula is as follows: Where Hard Loss represents the loss of the edge server itself, K represents the total number of categories, c k represents the probability of the kth category in the data label, q k represents the probability of the kth classification, which is calculated by the following formula: Where, O global_k Represents output O global The kth component of global_j Represents output O global The jth component of ; (b) The edge server calculates the probability distribution at distillation temperature T = t using the following formula: (c) The edge server calculates the probability distribution of the kth category at the distillation temperature T using the following formula: In the formula, outmean k Represents the kth component of the average value outmean, outmean j The jth component representing the mean outmean; (d) The edge server uses the following formula to calculate the distillation loss Soft Loss : (e) The edge server combines its own loss and distillation loss to calculate the total loss function using the formula: Loss global =(1-α)Hard Loss +αSoft Loss Where α is the loss adjustment factor of the edge server, ranging from 0 to 1; (f) The edge server performs gradient updates on the training model based on the calculated total loss to obtain the updated global model parameters ω u ; (5) The edge server will update the global model parameters ω u The data is uploaded to the cloud server in the first layer of the cloud-edge-end three-layer federated learning architecture. The cloud server uses average weights to aggregate and update the global model parameters. The update formula is as follows: Where |U| represents the number of edge clusters; (6) The cloud server verifies whether the performance of the training model meets the standard requirements based on the global model parameters obtained by cloud aggregation. If yes, the training ends; otherwise, the global model parameters are sent to each edge server to start the next round of training and execute step (7); (7) Terminal equipment e i Using the distillation temperature T and outmean, calculate the model M i distillation losses; (8) Terminal equipment e i Combine the self-loss and distillation loss as the loss function of this round Loss i , and then according to the loss function Loss i Take the partial derivative of the model parameters to get the gradient G i ; (9) Terminal equipment e i According to the gradient G i Use backpropagation to update the parameters, end this round of model training, obtain the updated model, and then return to step (2).

2. A lightweight trusted federated learning method in edge scenarios according to claim 1, characterized in that: The step (1) is specifically as follows: the terminal device e i Get the assigned model M from the edge server i , and using private dataset D i Training is performed locally, where D i ={(x i1 ,y i1 ),(x i2 ,y i2 )…(x id ,y id )}, d is the dataset D i The length of the private data set D i Divide into training data sets and test dataset Among them, using the training data set Features of x i As Model M i Input, after forward propagation, the training result O is obtained i , by comparing with the data label y i Perform cross entropy calculation to obtain its own loss function, the calculation formula is as follows: Where q ik Indicates e i The probability of the kth category, q ik is calculated as follows: Where, O ik Represents the training result O i The kth component of ij Represents the training result O i The jth component of .

3. A lightweight trusted federated learning method in edge scenarios according to claim 2, characterized in that: In step (7), the calculation model M i The distillation loss process is as follows: (g) Calculate the probability distribution of the local model at the distillation temperature T using the following formula: Where q ik_T Indicates e i The probability of the kth category at the distillation temperature T; (h) The probability distribution of the polymerization results at the distillation temperature T is calculated using the following formula: (i) Calculate the distillation loss using the following formula: (j) represents the distillation loss of the i-th terminal device, which is calculated by the cross entropy of the probability of the aggregation result and the probability of the model at the distillation temperature T.

4. A lightweight trusted federated learning method in edge scenarios according to claim 3, characterized in that: In step (8), the terminal device e is calculated using the following formula i The loss function of this round is Loss i : Where, α i Represents the loss adjustment factor, which ranges from 0 to 1 and is a balance between the self-loss and the distillation loss.

5. The lightweight trusted federated learning method in edge scenarios according to claim 4 is characterized in that: In step (9), the following formula is used to update the parameters: oh i =ω i -ηG i Where, ω i It is e i The model parameters on , η is the learning rate, and its value range is greater than 0.

6. A lightweight trusted federated learning method for edge scenarios according to any one of claims 1 to 5, characterized in that: The step (2) comprises the following steps: (2a) The edge server records the numbers of the terminal devices participating in this training, which are recorded as set I. Generate two pairs of public and private keys based on the parameter pp: KA.gen() represents the public-private key generation algorithm; for edge servers, the public-private key pair is generated using the parameter pp. (2b) The edge server is responsible for each terminal device e in set I. i , use the DH key exchange algorithm to obtain an encrypted symmetric key s u,i , the formula is as follows: By s u,i The symmetry of s u,i =s i,u , the key is used for encrypted transmission of the model result mask; (2c) Before the first round of training, each terminal device in set I randomly generates a random number seed seed i And use the pseudo random number generator PRG to generate a random vector b i , which is then expanded into vector b prai , with output The same size as the output mask, the formula is as follows: Where, Indicates the results obtained in the first round of training; (2d) Mask the result b prai and Add together to get And use s u,i As the key, and then use the encryption algorithm AE.enc() to encrypt, and then the encrypted training result O i Upload to the edge server, the encryption formula is as follows:

7. A lightweight trusted federated learning method in edge scenarios according to claim 6, characterized in that: In step (3), the decryption process of the edge server is: the edge server obtains e i,u Then use the symmetric key s u,i Decryption is performed using the decryption algorithm AE.dec(). The decryption formula is as follows: In the tth round of training, the calculation formula of the mask result is as follows: Where, Represents the terminal device e in round t i The results of the training, Represents the mask result of the (t-1)th round.

Images