A secure robust feature federation method

Abstract

The present application relates to a kind of safe robust feature joint method, belong to artificial intelligence field.The method includes: edge area division;System initialization: initialize global model, key generation and the key parameter of executing protocol;Device data equalization: the data distribution of equipment is collected, and the sampling mode of each class is generated for enhanced sampling or down sampling with reference to global data distribution, and the sample proportion of enhanced sampling and down sampling is set;Local model training: according to each class belongs to minority class set or majority class set, corresponding sampling strategy is adopted to execute random gradient descent algorithm, and gradient information is secret sharing;Edge security robust aggregation: design lightweight security aggregation protocol to support the implementation of robust aggregation algorithm based on gradient secret value;Cloud global model aggregation: receiving the local model aggregation result returned by each edge server, and global model aggregation is carried out using federated average algorithm.The present application improves the robustness and security of federated learning global model.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and relates to a safe and robust feature combination method. Background Technology

[0002] The rapid development and deep integration of the Internet of Things (IoT) and artificial intelligence (AI) have propelled the IoT into a new era of ubiquitous connectivity. Machine learning can fully uncover the inherent connections and effective value of massive amounts of heterogeneous data, greatly enhancing the intelligence of IoT systems. However, traditional machine learning methods require large-scale collection of raw data from terminal devices, causing IoT devices to lose control over their own data and facing serious privacy risks during data transmission and storage. Federated learning, as a distributed machine learning technology supporting feature aggregation, achieves model updates and iterations by aggregating parameters from multiple parties, jointly modeling while satisfying the privacy needs of participating parties, and fully unlocking the value of data.

[0003] Federated learning, as a feasible solution to the big data dilemma, technically breaks down data barriers and powerfully empowers numerous applications of the Internet of Things (IoT). However, federated learning systems are vulnerable to Byzantine attacks and inference attacks, leading to compromised system robustness and data privacy. First, intermediate parameters during IoT device training are easily tampered with by attackers. Due to the distributed training nature of federated learning and the inaccessibility of raw data, the parameter server struggles to assess the availability of model parameters uploaded by participants, causing global model divergence or even failure. For example, in a typical Byzantine attack, an attacker can send arbitrary intermediate parameters to the parameter server. Research shows that even a single Byzantine attacker can significantly reduce the performance of aggregated models, causing widely used aggregation rules to fail. Second, federated learning protects the privacy of raw data by transmitting intermediate parameters. However, existing research indicates that inference attacks can be launched using intermediate parameters to reconstruct the raw data of IoT devices and steal sensitive information.

[0004] Several studies have been conducted on defending against Byzantine attacks and privacy-preserving inference attacks. For example, Li S, Ngai E, and Voigt T proposed an automatically weighted geometric median algorithm in "Byzantine-robust Aggregation in Federated Learning Empowered Industrial IoT" [in IEEE Transactions on Industrial Informatics, vol. 19, no. 2, pp. 1165-1175, 2021]. This algorithm reweights the gradients based on Euclidean distance, automatically removing outlier gradients to achieve robustness, and designs a solution algorithm based on an alternating optimization strategy. Cao X and Lai L, in "Distributed Gradient Descent Algorithm Robust to an Arbitrary Number of Byzantine Attackers" [in IEEE Transactions on Signal Processing, vol. 67, no. 22, pp. 5850-5864, 2019], calculated noisy gradients using a small dataset to filter out malicious gradients, while gradients whose distances from the noisy gradients meet predefined conditions are accepted. Zhou H, Yang G, and Dai H, in “PFLF: Privacy-preserving Federated Learning Framework for Edge Computing” [in IEEE Transactions on Information Forensics and Security, vol. 17, pp. 1905-1918, 2022], calculated sensitivity based on the batch training data volume, combined with privacy budgeting to obtain the variance of noise added in differential privacy, and added noise during the device parameter upload and server parameter download stages to ensure the privacy of each device throughout the aggregation process. Lyu L, Yu J, and Nandakumar K, in “Towards Fair and Privacy-preserving Federated Deep Models” [in IEEE Transactions on Parallel and Distributed Systems, vol. 31, no. 11, pp. 2524-2541, 2020], designed a three-layer onion encryption scheme, using stream ciphers for homomorphic encryption to protect local model gradients.

[0005] While existing research has shown promising results in resisting Byzantine attacks and privacy-preserving inference attacks, the unique characteristics of IoT devices increase the difficulty of defending against these attacks. First, due to personalized data generation patterns in IoT scenarios, data tag distribution is typically highly heterogeneous; that is, IoT device data is not independently and identically distributed, which significantly weakens the effectiveness of most robust aggregation algorithms against Byzantine attacks. Second, IoT devices suffer from extremely limited computing resources, and the federated learning setup makes it difficult to delegate complex encryption tasks to a fully trusted third-party agent. Transmitting gradient plaintext information in an open network environment greatly increases the risk of privacy leakage, and offloading gradient information to a fully trusted third-party agent simultaneously can lead to severe network congestion and further increase the energy consumption of IoT devices. Finally, there is a conflict between parameter privacy and aggregation robustness in the implementation path. Therefore, there is an urgent need to design secure and robust feature joint methods. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a secure and robust feature union method that effectively improves the robustness and security of the federated learning global model under conditions of heterogeneous data and limited resources on the client side.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A secure and robust feature joint method is proposed. First, a data-balanced robust aggregation method is developed. The sampling method for each class is adjusted based on the global data distribution, and the gradients are standardized. Simultaneously, the weights of the gradients are adaptively adjusted based on the directional similarity of the uploaded gradients, effectively improving the performance of the robust aggregation algorithm in heterogeneous data environments. Second, a lightweight secure aggregation protocol is designed, employing arithmetic secret sharing and homomorphic encryption as underlying privacy protection techniques. The multiplication operation of arithmetic secret sharing is extended using multiplication triples, implementing the robust aggregation algorithm on the ciphertext. This method can simultaneously address robustness issues and privacy risks in heterogeneous data environments, achieving a good balance between model performance and computational overhead. The method specifically includes the following steps:

[0009] S1: Edge Area Division: Each edge server is responsible for managing an area based on its coverage capabilities, and manages the IoT devices within that area.

[0010] S2: System Initialization: Cloud server initializes global model parameters w t The data is then distributed to edge servers. After obtaining the global model parameters, the edge servers forward them to the IoT devices under their jurisdiction. The edge servers collect data used to generate the reference gradient g. rThe verification dataset; the edge server generates key pairs (pk, sk) for linear homomorphic encryption, and multiplication triples are generated between the edge server and the auxiliary server. , , <c>), and generate a private seed key s seed Each device generates a private seed key with the auxiliary server.

[0011] S3: Device Data Balancing: Cloud servers collect and distribute local data from participating devices. in Let represent the size of the data sample with label n, where N is the total number of labels. To expand the minority class samples and downsample the majority class samples based on the global data distribution, an enhancement threshold τ is set for the edge server, borrowing the idea of ​​z-score outlier detection algorithms. a and downsampling threshold τ d Calculate the z-score for each class in the data distribution of all devices, and obtain the minority class set Y by comparing it with a threshold. aug and the majority class set Y down The final ratio R of augmented and downsampled samples was obtained. ad and Y aug Y down R ad Send to all devices;

[0012] S4: Local model training: Obtaining global model parameters w t Afterwards, each IoT device queries each class in Y. aug Y down The algorithm assigns data to specific classes, and instead of randomly sampling samples when executing stochastic gradient descent, it sets the sampling method for each class to either augmented sampling or downsampling. This mitigates the challenges posed by differences in data distribution between devices to robust federated learning. Finally, the calculated gradients are normalized to eliminate the effects of Byzantine attacks and then secretly split and returned to the edge servers.

[0013] S5: Edge-Secure Robust Aggregation: Calculates the cosine similarity between each gradient and the reference gradient, and obtains a confidence score using the ReLU function to achieve robust aggregation. Utilizing arithmetic secret sharing and homomorphic encryption, device gradient information is secretly shared between the edge server and the auxiliary server, and robust aggregation is completed collaboratively using an efficient secure aggregation protocol.

[0014] S6: Cloud Global Model Aggregation: After receiving the local model aggregation results from each edge server, the cloud server uses the classic federated averaging algorithm to quickly aggregate the global model. If the global model metrics reach the training task's stopping criteria, local model training is stopped.

[0015] Furthermore, step S2 specifically includes:

[0016] S21: The edge server and the auxiliary server each generate a pair of random numbers (< / c> e , e )and( a , a ), c satisfies c = ( e + a )( e + a )＝ e e + a a + e a + a e ＝ <c> e + <c> a ,in< / c> < / c> e e and a a It can be computed locally on edge servers and auxiliary servers, simply by... e a and a e Without exposing a e,a b e,a In the case of [the event], it is passed to the other party.

[0017] S22: Edge servers use public key PK encryption. e 、 e Get Enc pk [ e ]、Enc pk [ e The encrypted data is then sent to the auxiliary server. Upon receiving the encrypted data, the auxiliary server calculates... Get Enc pk [ e a + e a To enhance privacy protection, the auxiliary server adds a random number r to the returned result, ultimately obtaining Enc. pk [ e a + e a +r], and return the calculation result to the edge server, while setting <c> a ＝< / c> a a -r.

[0018] S23: The edge server receives Enc pk [ e a + e a After adding r], decrypt it using the private key sk to obtain... e a + e a +r, simultaneously set <c> e ＝< / c> e e + e a + e a +r. This can easily verify c = <c> e + <c> a =ab.

[0019] S24: Each device and auxiliary server generate a private seed key based on the DH key exchange protocol, enabling key sharing between the device and auxiliary server in an insecure channel. First, the device and auxiliary server generate private random numbers m and n respectively. Based on the public factor G, they calculate mG and nG respectively and exchange them. Finally, both the device and auxiliary server obtain their private seed key by calculating mnG. Similarly, the private seed keys s of the edge server and the auxiliary server can be obtained. seed .

[0020] Furthermore, step S3 specifically includes the following steps:

[0021] S31: The cloud server will collect and aggregate all device data to obtain C←[C 1 C 2 C 3 ,...,C N ] Calculate the z-score for each class. n , represented as:

[0022]

[0023] Where μ represents the mean of C, and σ represents the standard deviation of C; if the z-score of the class is less than τ a or greater than τ d Then it is considered as a minority class set Y aug Or the majority class set Y down ;

[0024] S32: The cloud server will calculate the augmented sample ratio R a and downsampling sample ratio R d This is used to control the number of samples amplified and discarded, and is represented as:

[0025]

[0026]

[0027] Due to τ a Set to τ d The negative reciprocal of R; therefore, both the augmented sample proportion and the downsampled sample proportion can be expressed using R. ad This indicates that the cloud server will... aug Y down R ad Send to all devices.

[0028] Furthermore, step S4 specifically includes the following steps:

[0029] S41: The device receives the global model parameter w t Then, for n∈Y aug Enhanced sampling is required, as shown below:

[0030] (x,n)←(x,n)∪Aug((x,n),R ad [n]-1)

[0031] Where x represents the input data, and Aug(·) represents the augmentation sampling function, which performs operations such as flipping, rotating, cropping, deforming, and scaling on a certain type of sample to increase the number of samples; for n∈Y down Downsampling is required, as shown below:

[0032] (x,n)←(x,n)-Down((x,n),R ad [n])

[0033] Where Down(·) represents the downsampling function; if the sample is deleted, it will return the original sample; otherwise, it will return empty.

[0034] S42: After the sampling method is determined, the device runs the stochastic gradient descent algorithm on the local data to obtain g. k and the gradient g k Standardization is performed to balance the impact of each local gradient on the global model update, expressed as:

[0035]

[0036] S43: The device uses a pseudo-random signal generator to generate... Simultaneous calculation And send it to the edge server; where, This represents the secret shared value of the standardized device gradient split to the edge server, where k represents the device index.

[0037] Furthermore, step S5 specifically includes the following steps:

[0038] S51: The edge server runs the stochastic gradient descent algorithm on the validation dataset to obtain the reference gradient g. r And the reference gradient is standardized to obtain Simultaneously, it is encrypted using the public key pk. And send the encrypted result to the auxiliary server;

[0039] S52: Auxiliary server calculation and set Random sampling δ calculation get In addition, settings <cos k > a =δ; Finally, and <cos k > a The results are returned to the edge server; where T represents the transpose of the matrix, and K represents the total number of devices. <cos k > a This represents the secret shared value on the auxiliary server representing the cosine similarity between the device gradient and the reference gradient.

[0040] S53: Decrypted from the edge server and set in That is, it can be restored <cos k >= <cos k > a + <cos k > e Because the optimization objectives of Byzantine devices and honest devices are inconsistent, their optimization directions will deviate from the global convergence trend. Therefore, cosine similarity can be used to defend against Byzantine attacks, as gradients with low cosine similarity are more likely to be Byzantine gradients. Simultaneously, the ReLU(x) function is used to eliminate the negative impact of local gradients with negative cosine similarity on global model updates. Combining these two aspects, the confidence score TS is obtained. k , represented as:

[0041]

[0042] The ReLU(x) function is defined as returning x when x ≥ 0, and 0 otherwise; cos k This represents the cosine similarity between the device gradient and the reference gradient; finally, the edge server settings... <TS k > e =TS k -PRG(s seed ), and calculate Send to the auxiliary server;

[0043] S54: Auxiliary Server Setup <TS k > a =PRG(s seed ),calculate And return it to the edge server; both the edge server and the auxiliary server restore it locally. Then, the edge server and the auxiliary server each calculate the secret-shared value of the weighted gradient locally. and Finally, the auxiliary server will Send to the edge server;

[0044] S55: Edge server received After that, I received And calculate Finally, through TS k The weights of the standardized gradient are redistributed as the global gradient g. global , represented as:

[0045]

[0046] S56: The edge server updates the global model based on the aggregated gradient information, as shown below:

[0047] w t+1 =w t -ηg global

[0048] Where η represents the learning rate, w t+1 This represents the global model parameters for round t+1.

[0049] Furthermore, step S6 specifically includes the following steps:

[0050] S61: The cloud server receives the regional model parameters w returned by all edge servers. t+1 The global model is aggregated based on the traditional federated averaging algorithm, as shown below:

[0051]

[0052] in, It is the global model parameter in round t+1, n e is the total amount of data contained in an edge region, n is the total amount of data contained in all edge regions, and E represents the number of edge servers;

[0053] S62: The cloud server will... The broadcast is sent to all edge servers, which then broadcast it to all IoT devices under their jurisdiction for a new round of iterative training.

[0054] The beneficial effects of this invention are as follows: First, this invention proposes a robust aggregation method with balanced data, which approximates the global data distribution by using augmented sampling and downsampling, while simultaneously defending against Byzantine attacks by integrating normalization and cosine similarity. Furthermore, to ensure data privacy during robust aggregation, a lightweight and secure aggregation protocol is designed to achieve data privacy protection even with limited computing power on the device side, while ensuring the robustness of the model.

[0055] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0056] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0057] < / c> < / c> Figure 1 This is a block diagram of the secure and robust aggregation system under data heterogeneity according to the present invention;

[0058] Figure 2 A schematic diagram of the robust aggregation method for data balancing designed for this invention;

[0059] Figure 3 This is a flowchart of a lightweight secure aggregation protocol provided in an embodiment of the present invention. Detailed Implementation

[0060] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0061] Please see Figures 1-3 This invention provides an Internet of Things (IoT) communication system, comprising the following modules: Devices: Located in the perception layer of the IoT, these devices consist of a vast number of sensing devices that acquire environmental information through a sensor network. Each device has a limited range of movement and is assigned to a single edge server for management. The devices are responsible for data acquisition, model training, and secret gradient sharing.

[0062] Edge Servers: Located close to the device, edge servers possess strong communication, storage, and computing resources. They are responsible for adjusting sampling methods and receiving model updates uploaded by the device. They also collaborate with cloud servers to execute lightweight, secure aggregation protocols, supporting robust aggregation algorithms for local models in encrypted environments.

[0063] Cloud servers: Cloud servers are central devices with powerful communication, storage, and computing resources. They are responsible for receiving local model updates from all edge servers and then performing global model aggregation based on the amount of data in each region.

[0064] To address the aforementioned system, this invention provides a secure and robust feature combination method, specifically comprising the following steps:

[0065] Step 1: Edge Area Division: Each edge server is responsible for managing an area based on its coverage capabilities, and manages the IoT devices within that area.

[0066] Step 2: System Initialization: The cloud server initializes the global model parameters w. t The data is then distributed to edge servers. After obtaining the global model parameters, the edge servers forward them to the IoT devices under their jurisdiction. The edge servers collect data used to generate the reference gradient g. r The verification dataset. The edge server generates a key pair (pk, sk) for linear homomorphic encryption, and multiplication triples are generated between the edge server and the auxiliary server. , , <c>), and generate a private seed key s seed Each device generates a private seed key with the auxiliary server.

[0067] Step 2 specifically includes the following steps:

[0068] Step 2.1: The edge server and auxiliary server generate random numbers respectively.< / c> e , e )and( a , a ), c satisfies c = ( e + a )( e + a )＝ e e + a a + e a + a e ＝ <c> e + <c> a ,in< / c> < / c> e e and a a It can be computed locally on edge servers and auxiliary servers, simply by... e a and a e Without exposing a e,a b e,a In the case of [the event], it is passed to the other party.

[0069] Step 2.2: The edge server encrypts using a public key (pk). e 、 e Get Enc pk [ e ]、Enc pk [ e The encrypted data is then sent to the auxiliary server. Upon receiving the encrypted data, the auxiliary server calculates... Get Enc pk [ e a + e a To enhance privacy protection, the auxiliary server adds a random number r to the returned result, ultimately obtaining Enc. pk [ e a + e a +r], and return the calculation result to the edge server, while setting <c> a ＝< / c> a a -r.

[0070] Step 2.3: The edge server receives Enc pk [ e a + e a After adding r], decrypt it using the private key sk to obtain... e a + e a +r, simultaneously set <c> e ＝< / c> e e + e a + e a +r. This can easily verify c = <c> e + <c> a =ab.

[0071] Step 2.4: The private seed key generated by each device and the auxiliary server relies on the DH key exchange protocol to enable key sharing between the device and the auxiliary server in an insecure channel. First, the device and the auxiliary server generate private random numbers m and n respectively. Based on the public factor G, they calculate mG and nG respectively and exchange them. Finally, both the device and the auxiliary server obtain their private seed key by calculating mnG. Similarly, the private seed keys s of the edge server and the auxiliary server can be obtained. seed .

[0072] Step 3: Device Data Balancing: The cloud server collects and distributes local data from participating devices. in This represents the size of the data sample with label n. To augment the minority class samples and downsample the majority class samples based on the global data distribution, the edge server is configured with τ, borrowing the idea from z-score outlier detection algorithms. a and τ d Using the enhancement threshold and downsampling threshold as examples, calculate the z-score for each class in the data distribution of all devices, and obtain the minority class set Y by comparing it with the threshold. aug and the majority class set Y down The final ratio R of augmented and downsampled samples was obtained. ad and Y aug Y down R ad Send to all devices.

[0073] Step 3 specifically includes the following steps:

[0074] Step 3.1: The cloud server aggregates all collected device data to obtain C←[C 1 C 2 C 3 ,...,C N ] Calculate the z-score for each class. n , represented as:

[0075]

[0076] Where μ represents the mean of C, and σ represents the standard deviation of C. If the z-score of class is less than τ a or greater than τ d Then it is considered as a minority class set Y aug Or the majority class set Y down .

[0077] Step 3.2: The cloud server will calculate the augmented sample ratio R. a and downsampling sample ratio R d This is used to control the number of samples amplified and discarded, and is represented as:

[0078]

[0079]

[0080] Due to τ a Set to τ d The negative reciprocal of R. Therefore, both the augmented sample proportion and the downsampled sample proportion can be represented by R. ad This indicates that the cloud server will... aug Y down R ad Send to all devices.

[0081] Step 4: Local model training: Obtain global model parameters w t Afterwards, each IoT device queries each class in Y. aug Y down The algorithm assigns data to specific classes, and instead of randomly sampling samples when executing stochastic gradient descent, it sets the sampling method for each class to either augmented sampling or downsampling. This mitigates the challenges posed by differences in data distribution between devices to robust federated learning. Finally, the calculated gradients are normalized to eliminate the effects of Byzantine attacks and then secretly split and returned to the edge servers.

[0082] Step 4 specifically includes the following steps:

[0083] Step 4.1: The device receives the global model parameter w t Then, for n∈Y aug Enhanced sampling is required, as shown below:

[0084] (x,n)←(x,n)∪Aug((x,n),R ad [n]-1)

[0085] Where x represents the input data, and Aug(·) represents the augmentation sampling function, which performs operations such as flipping, rotating, cropping, deforming, and scaling on a certain type of sample to increase the number of samples. For n∈Y down Downsampling is required, as shown below:

[0086] (x,n)←(x,n)-Down((x,n),R ad [n])

[0087] Down(·) represents the downsampling function. If the sample is deleted, it will return the original sample; otherwise, it will return an empty value.

[0088] Step 4.2: After the sampling method is determined, the device runs the stochastic gradient descent algorithm on the local data to obtain g. k and the gradient g k Standardization is performed to balance the impact of each local gradient on the global model update, expressed as:

[0089]

[0090] Step 4.3: The device uses a pseudo-random signal generator to generate... Simultaneous calculation And then send it to the edge server.

[0091] Step 5: Edge-Secure Robust Aggregation: Calculate the cosine similarity between each gradient and the reference gradient, and obtain the confidence score using the ReLU function to achieve robust aggregation. Utilizing arithmetic secret sharing and homomorphic encryption, device gradient information is secretly shared between the edge server and the auxiliary server, and robust aggregation is completed collaboratively using an efficient secure aggregation protocol.

[0092] Step 5 specifically includes the following steps:

[0093] Step 5.1: The edge server runs the stochastic gradient descent algorithm on the validation dataset to obtain the reference gradient g. r And the reference gradient is standardized to obtain Simultaneously, it is encrypted using the public key pk. The encrypted result is then sent to the auxiliary server.

[0094] Step 5.2: Auxiliary server calculation and set Random sampling δ calculation get In addition, settings <cos k > a =δ. Finally, It is returned to the edge server along with the data.

[0095] Step 5.3: Decryption by the edge server and set in It can be restored <cos k >= <cos k > a + <cos k > e Because the optimization objectives of Byzantine devices and honest devices are inconsistent, their optimization directions will deviate from the global convergence trend. Therefore, cosine similarity can be used to defend against Byzantine attacks, as gradients with low cosine similarity are more likely to be Byzantine gradients. Simultaneously, the ReLU(x) function is used to eliminate the negative impact of local gradients with negative cosine similarity on global model updates. Combining these two aspects, the confidence score is obtained, expressed as:

[0096]

[0097] The ReLU(x) function is defined as returning x when x ≥ 0, and 0 otherwise. Finally, the edge server is configured... <TS k > e =TS k -PRG(s seed ), and calculate Send to the auxiliary server.

[0098] Step 5.4: Auxiliary Server Setup <TS k > a =PRG(s seed ),calculate This is then returned to the edge server. Both the edge server and the auxiliary server restore it locally. Then, the edge server and the auxiliary server each calculate the secret-shared value of the weighted gradient locally. and Finally, the auxiliary server will Send to the edge server.

[0099] Step 5.5: The edge server receives... After that, I received And calculate Finally, through TS k The weights of the standardized gradient are redistributed as the global gradient. This is represented as:

[0100]

[0101] Step 5.6: The edge server updates the global model based on the aggregated gradient information, as shown below:

[0102] w t+1 =w t -ηg global

[0103] Where η represents the learning rate.

[0104] Step 6: Cloud Global Model Aggregation: After receiving the local model aggregation results from each edge server, the cloud server uses the classic federated averaging algorithm to quickly aggregate the global model. If the global model metrics reach the training task's stopping criteria, local model training is stopped.

[0105] Step 6 specifically includes the following steps:

[0106] Step 6.1: The cloud server receives the regional model parameters w returned by all edge servers. t+1 The global model is aggregated based on the traditional federated averaging algorithm, as shown below:

[0107]

[0108] in, It is the global model parameter in round t+1, n e is the total amount of data contained in a single edge region, and n is the total amount of data contained in all edge regions.

[0109] Step 6.2: The cloud server will... The broadcast is sent to all edge servers, which then broadcast it to all IoT devices under their jurisdiction for a new round of iterative training.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.< / c> < / c>

Claims

1. A secure robust feature federation method, characterized in that, The method specifically comprises the following steps: S1: edge area division: each edge server divides the area responsible for management according to its range coverage capability, and manages the Internet of Things devices in the area; S2: System initialization: the cloud server initializes the global model parameters And is issued to the edge server, and the edge server obtains the global model parameters and forwards them to the Internet of Things devices under its jurisdiction; S3: Device data balancing: the cloud server collects the local data distribution of participating devices wherein denotes the data sample size of the label , N is the total number of labels; and the majority class samples are down-sampled based on the global data distribution; by referring to the outlier detection algorithm idea of score, the edge server sets the enhancement threshold and the down-sampling threshold , calculates the score of each class in all device data distributions, obtains the minority class set and the majority class set by comparing with the threshold, finally obtains the proportion of the enhanced samples and the down-sampled samples , and sends to all devices; S4: Local model training: obtain global model parameters After that, each IoT device queries the belonging of each class in the set, sets the sampling method of each class sample to be enhanced sampling or down-sampling; finally, the calculated gradient is standardized and returned to the edge server in secret. S5: edge security robust aggregation: the cosine similarity between each gradient and the reference gradient is calculated, and the confidence score is obtained through the ReLu function to realize robust aggregation; using arithmetic secret sharing and homomorphic encryption technology, the device gradient information is secretly shared between the edge server and the auxiliary server, and the robust aggregation is completed collaboratively using an efficient secure aggregation protocol; S6: cloud global model aggregation: after receiving the local model aggregation results returned by each edge server, the cloud server realizes the fast aggregation of the global model using the classic federated average algorithm; if the global model index reaches the training task stop standard, stop the local model training; In step S2, the edge server collects a verification dataset for generating a reference gradient ; the edge server generates a key pair for linear homomorphic encryption , a multiplication triple is generated between the edge server and the auxiliary server , and a private seed key is generated between each device and the auxiliary server ; Step S3 specifically comprises the following steps: S31: The cloud server distributes and aggregates all the collected device data to obtain , calculates the score of each class , and represents it as: ​ wherein, denotes the mean of , denotes the standard deviation of ; if the fraction of the class is less than or greater than , it is considered a minority class set or a majority class set ; S32: The cloud server calculates the augmented sample proportion and the down-sampled sample proportion to control the number of augmented and discarded samples, denoted as: Since the is set to the negative reciprocal of ; therefore both the upsampled sample ratio and the downsampled sample ratio can be expressed as ; the cloud server sends to all devices; Step S4 specifically comprises the following steps: S41: The device receives the global model parameters Afterwards, for , enhanced sampling needs to be performed, denoted as: wherein, represents the input data, represents an enhanced sampling function; for down-sampling is required, represented as: wherein, represents a down-sampling function that returns the original sample if removed, otherwise, returns nothing; S42: After the sampling method is determined, the device runs a random gradient descent algorithm on the local data to obtain and the gradient is standardized, denoted as: S43: The device generates simultaneously calculates and sends it to the edge server; wherein, denotes the secret sharing value of the normalized device gradient split to the edge server, k denotes the index of the device; Step S5 specifically comprises the following steps: S51: the edge server runs a random gradient descent algorithm on the verification data set to obtain a reference gradient , and standardizes the reference gradient to obtain , and encrypts the reference gradient using the public key to obtain , and sends the encrypted result to the auxiliary server; S52: auxiliary server computing and set ; random sampling , compute get ; in addition, set ; finally, return and to the edge server together; wherein, T denotes the transpose of the matrix, K denotes the total number of devices, denotes the secret sharing value of the cosine similarity of the device gradient and the reference gradient on the auxiliary server; S53: The edge server decrypts to obtain and sets wherein i.e., can restore ; obtains a confidence score , expressed as: wherein, The function is defined to return a value of when and a value of 0 otherwise; represents the cosine similarity between the device gradient and the reference gradient; finally, the edge server sets and computes and sends to the helper server; S54: the helper server sets , computes and returns it to the edge server; both the edge server and the helper server locally restore , Then the edge server and the helper server locally compute the secret share values of the weighted gradient and respectively; finally, the helper server sends to the edge server; S55: The edge server receives After that, get And calculate Finally, by Reassign weights to the normalized gradient as global gradient , denoted as: S56: the edge server updates the global model according to the aggregated gradient information, expressed as: where, denotes the learning rate, denotes the global model parameters of the round.

2. The feature federation method of claim 1, wherein, Step S6 specifically comprises the following steps: S61: The cloud server accepts the regional model parameters returned by all edge servers According to the traditional federated averaging algorithm, the global model is aggregated, which is represented as: wherein, is the global model parameter of the first wheel, is the total amount of data contained in an edge region, is the total amount of data contained in all edge regions, E denotes the number of edge servers; S62: The cloud server will broadcast broadcast to all edge servers, and then broadcast by the edge servers to all Internet of Things devices under their jurisdiction to conduct a new round of iterative training.

Images