A differentially private federated learning method for industrial Internet of Things

Abstract

The present invention belongs to the technical field of industrial Internet of Things, and discloses a differential privacy federated learning method for industrial Internet of Things. This method aims at the problem that differential privacy noise has a great impact on model performance, divides the model into a feature extractor and a classifier, and the classifier is retained locally and does not participate in aggregation, thereby reducing the degree to which the model is affected by noise, improving the performance of the model, and achieving a better trade-off between privacy and utility. Before adding noise, the feature extractor part participating in aggregation is sparsely populated, which alleviates the interference of noise and improves communication efficiency. The present invention also considers the problem that data collected by different devices in the industrial Internet of Things are not independent and identically distributed. The edge nodes cooperate with each other through federation, train feature extractors to extract more common low-dimensional features from the data, and maintain personalized classifier models locally for their respective tasks to prevent mutual interference between classifiers of edge nodes with different data distributions, while alleviating the problem of data not being independent and identically distributed.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial Internet of Things and relates to a differential privacy federated learning method for industrial Internet of Things. Background Art

[0002] With the development of artificial intelligence, machine learning models have been widely used in the Industrial Internet of Things (IIoT), especially in the fields of intelligent manufacturing and equipment fault detection. However, the training of machine learning models usually requires a large amount of industrial data, which is often highly sensitive, such as process parameters of production lines, equipment operation logs, and product quality inspection results. The leakage of these data may bring serious privacy risks. At the same time, due to the differences in the operating environment and workflow of different factories, production lines or equipment, these data usually present non-independent and identically distributed (non-iid) characteristics. In order to solve the problems of data privacy protection and data silos, Federated Learning (FL) as an emerging distributed machine learning paradigm has emerged in the IIoT scenario.

[0003] Through federated learning, each factory or production line can work together to train a global model without sharing the original data, thereby improving model performance while protecting data privacy. The core idea of ​​federated learning is data localization, that is, the data is retained on the edge nodes (such as edge servers or production equipment in the factory), and the global model is updated only by exchanging model parameters. Specifically, federated learning allows each edge node to train the data locally, upload the updated model parameters to the central server for aggregation, and build a global model. This approach not only reduces the need for data transmission and reduces the risk of data leakage, but also effectively addresses the problems of limited device resources and limited network bandwidth in industrial environments.

[0004] Although FL provides a good solution for data privacy protection in IIoT, it still faces the potential risk of privacy leakage. For example, attackers can determine whether certain data belongs to the training set through membership inference attacks, or try to restore training samples and labels through data reconstruction attacks. In the IIoT environment, this risk may be more serious because the data distribution between devices varies greatly, and privacy leakage may directly endanger the competitive advantage of enterprises. At the same time, the localized nature of federated learning amplifies the impact of non-iid problems on model training. Equipment from different factories or production lines may have significant differences in data distribution, resulting in the risk of performance degradation or even unstable training when the model is aggregated. Therefore, how to protect data privacy while solving the impact of data non-independent and identically distributed on model performance has become a key challenge for the widespread application of FL in IIoT.

[0005] In order to further improve the strength of privacy protection, researchers have begun to explore various defense methods, including but not limited to the introduction of data privacy protection technologies such as homomorphic encryption (HE), secure multi-party computation (SMC) and differential privacy (DP). Compared with HE and SMC algorithms, DP provides a more flexible and computationally efficient privacy protection method, especially for resource-constrained IIoT devices. In terms of computational efficiency, DP achieves privacy protection by adding noise to data or calculation results. This method usually has lower computational complexity and communication overhead. At the same time, DP allows a trade-off between privacy protection and data practicality. By adjusting the size of the noise, the strength of privacy protection and the accuracy of the data can be controlled, which has better flexibility. However, the noise in the DP mechanism may have a significant impact on the performance of the model, especially in the scenario where the data is not independent and identically distributed, which is common in IIoT.

[0006] In order to alleviate the performance degradation problem caused by DP, some studies have proposed the methods of Bounded Local Update Regularization (BLUR) and Local Update Sparsification (LUS). By adding noise-related regularization terms to the loss function of local training, the robustness of the model is improved, and the local model is sparsified to reduce the interference of noise. However, this regularization method has limited mitigation of noise interference and does not consider the problem of non-independent and identically distributed. In addition, some studies have proposed the sparse model perturbation (SMP) technology, which improves the privacy and accuracy trade-off of DP in federated learning by sparsifying the local model before perturbation, and provides differential privacy protection at the edge node level while maintaining high model accuracy. However, this method only uses the sparsification strategy and cannot fully improve the performance of the federated learning model under differential privacy. In addition, there are other studies on improving optimizers and training models, but these methods are still greatly affected by noise and often do not consider the impact of non-independent and identically distributed data. Summary of the invention

[0007] In response to the data privacy protection and data heterogeneity problems in the industrial Internet of Things, the present invention proposes a differential privacy federated learning method for the industrial Internet of Things. The differential privacy federated learning method for the industrial Internet of Things improves the model accuracy while providing sufficient differential privacy protection and can adapt to non-independent and identically distributed situations.

[0008] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a differential privacy federated learning method for industrial Internet of Things, wherein the federated learning system includes a central server and multiple edge nodes capable of collecting device data; the method divides the original complete global model of federated learning into two parts: a shared feature extractor and a classifier private to the edge node, and only adds noise to the shared feature extractor part, while the classifier is retained locally.

[0009] The differential privacy federated learning method for industrial Internet of Things specifically includes the following steps: Step 1. The central server initializes the global feature extractor and sends it to each edge node; each edge node initializes its own local classifier.

[0010] Step 2. After receiving the global feature extractor sent by the central server, each edge node updates its own local feature extractor.

[0011] Step 3. The edge node first fixes the local feature extractor and updates the local classifier with its own private data.

[0012] Step 4. The edge node fixes the local classifier and updates the local feature extractor with private data.

[0013] Step 5. The edge nodes use the top-k method to perform parameter sparsification on each convolutional layer of the updated local feature extractor part.

[0014] Step 6. The edge node prunes and adds noise to some parameters retained after the local feature extractor is sparsed, and sends them to the central server.

[0015] Step 7. The central server performs aggregation, updates the global feature extractor, and sends it to each edge node for a new round of training; repeat steps 2 to 7 until the preset communication round is reached.

[0016] Step 8. After the model training is completed, each edge node loads the local feature extractor sent by the central server and the local classifier of the edge node to form its own personalized model.

[0017] The present invention has the following advantages: As mentioned above, the present invention describes a differential privacy federated learning method for the industrial Internet of Things. This method addresses the problem that differential privacy noise has a significant impact on model performance, and proposes to divide the model into two parts: a feature extractor and a classifier, wherein the feature extractor is responsible for extracting common features of the data and participating in global aggregation to improve the generalization ability of the model; the classifier is retained locally and is only used for local tasks, thereby avoiding the degradation of classifier performance due to differential privacy noise interference. This division effectively reduces the impact of noise, significantly improves the performance of the model, and achieves a better balance between privacy protection and model utility. In addition, in order to further alleviate the interference of noise on model performance and adapt to the characteristics of limited device resources in IIoT, the present invention sparses the feature extractor part before adding noise, retaining only parameters that contribute more to model performance. This not only reduces the impact of noise, but also significantly reduces communication overhead and improves the efficiency of the system. In addition, in response to the common problem of non-independent and identically distributed data in the industrial Internet of Things, the present invention also designs a federal collaboration mechanism. Although there are significant differences in the distribution of data between different factories, production lines or equipment, by sharing feature extractors, each participant can learn a more general low-dimensional feature representation to extract key information from their respective industrial data; at the same time, the local classifier retains the personalized characteristics of each factory or equipment, avoiding the degradation of classifier performance or interference with each other due to differences in data distribution. In this way, not only is noise interference effectively reduced, but the negative impact of data non-independent and identically distributed on model performance is also alleviated, providing a reliable solution for privacy protection and efficient collaboration in IIoT scenarios. The present invention is particularly suitable for industrial Internet of Things scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flowchart of a differential privacy federated learning method for industrial Internet of Things in an embodiment of the present invention.

[0019] Figure 2 This is a model architecture diagram of the differential privacy federated learning method for industrial Internet of Things in an embodiment of the present invention.

[0020] Figure 3 A schematic diagram showing the comparison results of the model accuracy obtained by training the method of the present invention with the federated averaging method and the sparsification method on different data sets; Figure 3 (a) and (b) are the comparison results of the model accuracy obtained by training on the public image datasets Fashion-MNIST and Cifar10, respectively.

[0021] Figure 4 Schematic diagram of ablation experiment results of the model partitioning step and the sparsification step in the method of the present invention. DETAILED DESCRIPTION

[0022] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments: In the Industrial Internet of Things (IIoT) scenario, industrial data needs to be strictly protected for privacy, and the equipment in different factories or production lines is usually distributed in their own independent physical environments, generating non-independent and identically distributed data. Through the differential privacy federated learning method proposed in the present invention, industrial equipment can achieve performance improvement in their respective application scenarios while ensuring data privacy, especially for production lines or equipment with large differences in data distribution. This differential privacy federated learning method is still effective. Specifically, in the field of intelligent manufacturing, production equipment in different factories may generate significantly different operation logs, quality inspection data or sensor data due to differences in production processes, product types or environmental conditions. At the same time, these data usually involve the core business secrets or sensitive production parameters of the enterprise and must be strictly protected. In this case, traditional centralized machine learning methods are difficult to effectively handle the problem of non-independent and identically distributed data, and there is a high risk of privacy leakage. Through the personalized differential privacy federated learning method proposed in the present invention, production equipment can train personalized classifiers locally, and collaborate with other devices to learn general feature extractors. This method can not only better adapt to the respective data distribution, reduce data interference between different devices, but also improve the generalization ability of the overall model. In addition, the addition of differential privacy ensures the privacy of data during the collaboration process and avoids the leakage of sensitive information. It meets the high standards of the Industrial Internet of Things for data security and privacy protection and provides effective technical support for the development of industrial intelligence.

[0023] Specifically, the differential privacy federated learning method for industrial Internet of Things described in the present invention, to address the problem that differential privacy noise has a significant impact on model performance, proposes a partitioning model, divides the original complete global model into two parts, a feature extractor and a classifier, only adds noise to the shared feature extractor part, and introduces sparsification technology to further reduce the impact of differential privacy noise on the model; at the same time, the present invention introduces the technology of privatizing model classifiers to address the problem that the data of each edge node in the industrial Internet of Things are not independent and identically distributed. The edge nodes only train classifiers locally, thereby improving the personalization ability of the model.

[0024] like Figure 1 and Figure 2 As shown, the differential privacy federated learning method for industrial Internet of Things in this embodiment targets a federated learning system including a central server and multiple edge nodes that can collect device data.

[0025] To address the problem that differential privacy noise has a significant impact on the performance of the complete global model, this method proposes a partitioning model. The original complete global model of federated learning is divided into two parts: a shared feature extractor and a classifier private to the edge nodes. Only the shared feature extractor is noised, while the classifier is retained locally.

[0026] The differential privacy federated learning method for industrial Internet of Things includes the following steps: Step 1. The central server initializes the global feature extractor and sends it to each edge node; each edge node initializes its own local classifier.

[0027] Step 2. After each edge node receives the global feature extractor sent by the central server, it is used to update its own local feature extractor.

[0028] Step 3. The edge node first fixes the local feature extractor and updates the local classifier with its own private data.

[0029] The edge node uses its own private data to train and update the model classifier. The formula is as follows:

[0030] .

[0031] Among them, SGD is the stochastic gradient descent method. Represents edge node The updated classifier model in this round, Represents edge node The classifier model before this round of update, is the global feature extractor, is the cross entropy loss, For the current federation training round, is the classifier training round, the classifier is trained wheel, .

[0032] Step 4. After training a better classifier in step 3 above, the edge node fixes the local classifier and updates the local feature extractor with private data.

[0033] The training formula of the local feature extractor is as follows:

[0034] .

[0035] in, Represents edge node The local feature extractor after this round of update, Represents edge node The local feature extractor before this round of update, is the stochastic gradient descent method, Represents an edge node go through The local classifier after round of training, is the cross entropy loss, For the current federation training round, is the training round of local feature extractor, and the local feature extractor is trained in total wheel, .

[0036] During the model training process, the following optimization objectives are defined:

[0037] .

[0038] in, , is the cross entropy loss, It is a private local classifier for edge nodes. is the global feature extractor, is the total number of edge nodes, Represented by the local classifier and global feature extractor The complete model composed of represents a set of feasible classifiers, represents a set of feasible feature extractors, Represents an edge node The minimum cross entropy loss is Indicates all The minimum total cross entropy loss for each edge node.

[0039] Step 3 and step 4 work together to achieve the optimization goal. Step 3 optimizes the private local classifier parameters by minimizing the objective function, and step 4 optimizes the global feature extractor parameters by minimizing the objective function.

[0040] The combination of the two can jointly improve the classification performance and feature extraction capabilities of the model, thereby completing the overall training process of the model.

[0041] Step 5. The edge nodes use the top-k method to perform parameter sparsification on the updated local feature extractor.

[0042] To further reduce the impact of noise and communication overhead, the top-k method is used to sparse the parameters of each convolutional layer of the updated feature extractor part, setting the parameters that have little impact on the model to zero, as shown in the following formula:

[0043] .

[0044] in, Represents some parameters retained after sparsifying a convolutional layer of the local feature extractor; is the parameter vector of the convolutional layer After sorting in descending order by absolute value, The maximum value in the original parameter vector The corresponding subscript in; Select the front A larger parameter.

[0045] Step 6. The edge node trims and adds noise to the parameters retained after the local feature extractor is sparsed, and sends them to the central server. The process is as follows: First, the sparsely retained part is trimmed according to the trimming threshold. The formula is as follows:

[0046] .

[0047] in, is the update value of the local feature extractor, is the updated value after clipping, is the set clipping threshold.

[0048] Secondly, Gaussian noise is added to the cropped result according to the privacy budget. The formula is as follows:

[0049] .

[0050] in, is the updated value of the local feature extractor after cropping and adding noise, represents the added noise, is the noise multiplier, For the The set of edge nodes participating in a round.

[0051] Step 7. The central server performs aggregation, updates the global feature extractor, and sends it to each edge node for a new round of training. Repeat steps 2 to 7 above until the preset communication round is reached.

[0052] Step 8. After the model training is completed, each edge node loads the local feature extractor sent by the central server and the local classifier of the edge node to form its own personalized model.

[0053] The personalized model can be used to perform data processing tasks for the industrial Internet of Things. A typical application example of the personalized model is to use it as an image classification model for equipment fault detection tasks. The detected equipment fault detection image set is used as the private data of each edge node, and the type of equipment fault is used as a label. The model is trained according to the above steps 1 to 7 using the equipment fault detection image set. After the training is completed, a personalized image classification model is obtained. Each edge node can use its own trained personalized image classification model to perform specific equipment fault detection and classification tasks, so as to accurately predict its fault type based on the input equipment fault detection image.

[0054] The present invention realizes a more efficient differential privacy noise adding method by introducing personalized federated learning technology of model partitioning and model sparsification technology, reduces the degree to which the model is affected by differential privacy noise, and enhances the personalized performance of the model, which is particularly suitable for the situation where edge node data is not independent and identically distributed in practical applications. Specifically, the neural network model is divided into two parts: a feature extractor composed of a convolutional layer and a classifier composed of a fully connected layer. The feature extractor participates in model aggregation, and the classifier is only trained locally at the edge node, thereby obtaining a more personalized model. Secondly, the edge node first compresses the feature extractor to be uploaded, retains only the parameters with larger values, and then adds differential privacy noise to the reserved part according to the privacy protection requirements. The addition of differential privacy noise uses the commonly used Gaussian mechanism to achieve user-level differential privacy, and the noise size is determined by the privacy budget of the edge node. By partitioning and sparsifying the model, the impact of differential privacy noise on the model can be effectively alleviated, while providing sufficient privacy protection, the model performance is significantly improved, and it is more suitable for the situation where data is not independent and identically distributed, effectively solving the problem of privacy protection of edge node data in the field of the Internet of Things and heterogeneous data generated by different edge nodes.

[0055] like Figure 2 The training steps of the edge nodes are shown: .Fix the feature extractor and train the classifier; Fix the updated classifier and train the feature extractor; .Sparse feature extractor part; .Add differential privacy protection. After each round of local training, the uploaded part of the model is sparse and noise is added to achieve user-level differential privacy and prevent the global model from leaking sensitive data information of edge nodes. The edge node model is divided into two parts during local training, and the local training process is also divided into two steps accordingly, updating the classifier and feature extractor of the model respectively. This is because the feature extractor part composed of convolutional layers is mainly responsible for extracting features from the data. Although the sample labels are different, they still share a unified (low-dimensional) representation, so each edge node shares the feature extractor and obtains a better global feature extractor through collaborative learning of the federated architecture. The classifier is easily affected by non-independent and identically distributed data, which leads to mutual interference between different classifiers of each edge node and cannot be aggregated to obtain a better global classifier. However, if the classifier is separated from the model as a private classifier for each edge node, it can not only reduce the scope of noise, but also prevent mutual interference between heterogeneous data of edge nodes.

[0056] In order to intuitively demonstrate the effectiveness and superiority of the method of the present invention, the following experimental results are provided: Figure 3 and Figure 4 shown.

[0057] All experiments were implemented using pytorch and trained on two public image datasets: Fashion-MNIST and Cifar10. The Fashion-MNIST dataset contains 70,000 grayscale images of 28×28 pixels, divided into 10 categories, with 7,000 images in each category; the Cifar10 dataset contains 60,000 color images of 32×32 pixels, divided into 10 categories, with 6,000 images in each category. Both datasets use the Dirichlet method ( ) The data is divided into 50 edge nodes according to non-independent and identically distributed. During the training process, 10 edge nodes are randomly selected to participate in the training in each communication round. The local training is 5 rounds, the total communication rounds are 200 rounds, and the learning rate is set to 0.03. At the same time, in order to ensure the same differential privacy protection strength, the privacy budget of the method of the present invention and the comparison method is set to , the clipping threshold is 0.2.

[0058] Among them, there are two comparison methods, one is the classic differential privacy federated averaging algorithm DP-FedAvg, and the other is the differential privacy federated learning method FedSMP using sparsification technology. Among them, DP-FedAvg combines differential privacy and federated averaging algorithms. After the local training of the edge node is completed, differential privacy noise is added to the entire model, and then the model is uploaded to the server for aggregation. FedSMP improves the simple combination method. After the server aggregates the global model, the first k position information with larger values ​​in the model parameters is calculated, and broadcast to the edge nodes participating in the training together with the global model. After the local training is completed, the edge node first performs sparse processing on the local model, retaining only the parameters of these k positions, and then adds differential privacy noise to it and uploads it to the server for aggregation. Set up a comparison experiment to compare the model accuracy trained by the three methods on two non-independent and identically distributed data sets Fashion-MNIST and Cifar10.

[0059] Figure 3 The accuracy comparison of the model trained by the method of the present invention, the DP-FedAvg method and the FedSMP method is shown. The y-axis represents the accuracy, Ours represents the method of the present invention, DP-FedAvg represents the traditional differential privacy federated averaging method, and FedSMP represents the differential privacy federated learning method using sparsification technology. Figure 3 (a) and (b) are the comparison results of the model accuracy obtained by training on the public image datasets Fashion-MNIST and Cifar10. Figure 3 It can be seen that the method of the present invention has the highest accuracy after convergence, and its mitigation effect on differential privacy noise is better than the two baseline methods, which can provide a better trade-off between privacy and utility, and the performance gap among the three is more significant on the more complex Cifar10 dataset. At the same time, the method of the present invention can effectively deal with the situation that edge node data is not independent and identically distributed, which is common in the industrial Internet of Things. It has better personalized performance while providing differential privacy protection. In contrast, the traditional federated average algorithm has a significant performance drop when it needs to provide the same differential privacy protection on the non-independent and identically distributed Cifar10 dataset.

[0060] In addition, in order to demonstrate the effectiveness of the model partitioning module and the sparsification module in the method of the present invention, an ablation experiment was set up. On the dataset Cifar10 with a non-independent and identically distributed setting, the model was trained using the model partitioning method and the sparsification method respectively, and the accuracy was compared with the model trained using the complete algorithm of the method or the traditional differential privacy federated average algorithm.

[0061] Figure 4The effects of model partitioning and sparsification of the method of the present invention on mitigating differential privacy noise on the dataset Cifar10 in a non-independent and identically distributed setting are demonstrated, and the accuracy of the models obtained by training using a certain module alone, the complete algorithm of the method, and the traditional differential privacy federated averaging algorithm DP-FedAvg are compared. Ours represents the result of the complete algorithm of the method of the present invention, Ours_s represents the result when only the sparsification component is used, Ours_r represents the result when only the model partitioning technology is used, and Fedavg represents the result of the federated averaging method as a baseline plus differential privacy noise. Figure 4 As can be seen from the figure, the model performance is improved after adding model partitioning or sparsification technology, indicating that both components can reduce the interference of differential privacy noise. Moreover, the combination of model partitioning and sparsification technology can further improve the model performance and robustness.

[0062] Of course, the above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiments. It should be noted that all equivalent substitutions and obvious deformation forms made by any technician familiar with the field under the guidance of this specification fall within the essential scope of this specification and should be protected by the present invention.

Claims

1. A differential privacy federated learning method for industrial Internet of Things, wherein the federated learning system includes a central server and multiple edge nodes capable of collecting device data; characterized in that: The method divides the original complete global model of federated learning into two parts: a shared feature extractor and a private classifier of the edge node. Only the shared feature extractor is subjected to noise processing, and the classifier is retained locally. The differential privacy federated learning method for industrial Internet of Things specifically includes the following steps: Step 1. The central server initializes the global feature extractor and sends it to each edge node; each edge node initializes its own local classifier; Step 2. After receiving the global feature extractor sent by the central server, each edge node updates its own local feature extractor; Step 3. The edge node first fixes the local feature extractor and updates the local classifier with its own private data; Step 4. The edge node fixes the local classifier again and updates the local feature extractor with private data; Step 5. The edge nodes use the top-k method to perform parameter sparsification on each convolutional layer of the updated local feature extractor part; Step 6. The edge node crops and adds noise to some parameters retained after the local feature extractor is sparsed, and sends them to the central server; In step 6, the part retained by the local feature extractor after sparse processing is cropped and noise is added, and the process is as follows: First, the sparsely retained part is cropped according to the cropping threshold. The formula is as follows: Among them, Δφ is the update value of the local feature extractor, Δφ C is the updated value after clipping, and C is the set clipping threshold; Secondly, Gaussian noise is added to the cropped result according to the privacy budget. The formula is as follows: Among them, Δφ C,N is the updated value of the local feature extractor after cropping and adding noise, represents the added noise, σ is the noise multiplier, M t is the set of edge nodes participating in round t; Step 7. The central server performs aggregation, updates the global feature extractor, and sends it to each edge node for a new round of training; repeat steps 2 to 7 until the preset communication round is reached; Step 8. After the model training is completed, each edge node loads the local feature extractor sent by the central server and the local classifier of the edge node to form its own personalized model.

2. The differential privacy federated learning method for industrial Internet of Things according to claim 1 is characterized in that: In step 3, the training and updating formula of the local classifier is as follows: in, represents the classifier model of edge node i after this round of update, represents the classifier model of edge node i before this round of update, SGD is the stochastic gradient descent method, φ t is the global feature extractor, f i is the cross entropy loss, t is the current federated training round, s is the classifier training round, and the classifier is trained for τ h round, s=1,…,τ h .

3. The differential privacy federated learning method for industrial Internet of Things according to claim 2 is characterized in that: In step 4, the training formula of the local feature extractor is as follows: in, represents the local feature extractor of edge node i after this round of update, represents the local feature extractor of edge node i before this round of update. SGD is the stochastic gradient descent method. Indicates that edge node i passes through τ h The local classifier after round of training, f i is the cross entropy loss, t is the current federated training round, p is the local feature extractor training round, and the local feature extractor is trained for τ φ round, p=1,…,τ φ .

4. The differential privacy federated learning method for industrial Internet of Things according to claim 3 is characterized in that: In step 3 and step 4, during the model training process, the following optimization objectives are defined: Where i = 1, 2, ..., n, f i is the cross entropy loss, h i is the private local classifier of the edge node, φ is the global feature extractor, n is the total number of edge nodes, Represented by the local classifier h i The complete model consisting of a global feature extractor φ, represents a set of feasible classifiers, Φ represents a set of feasible feature extractors, represents the minimum cross entropy loss of edge node i, Represents the minimum total cross entropy loss of all n edge nodes; Step 3 and step 4 work together to achieve the optimization goal. Step 3 optimizes the private local classifier parameters by minimizing the objective function, and step 4 optimizes the global feature extractor parameters by minimizing the objective function. The combination of the two can jointly improve the classification performance and feature extraction capabilities of the model, thereby completing the overall training process of the model.

5. The differential privacy federated learning method for industrial Internet of Things according to claim 1 is characterized in that: In step 5, the formula for using the top-k method to perform parameter sparseness on each convolutional layer of the updated local feature extractor part is as follows: Among them, [top k (x)] j It represents the parameters retained after the sparsification of a convolutional layer of the local feature extractor. π(j) is the subscript corresponding to the jth maximum value in the original parameter vector x of the convolutional layer after the parameter vector x is sorted in descending order of absolute value. j≤k means that the first k larger parameters are selected.

Images