Internet of Things security method based on generative adversarial network
The GAN-based method generates classifier calibration features on the server side, addressing data heterogeneity in IoT federated learning, improving model accuracy and ensuring privacy.
Patent Information
- Application Number
- CN202411527620.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-01-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Federated learning in IoT systems faces challenges due to non-independent and identically distributed data, leading to poor model performance and convergence issues, particularly affecting classifier accuracy.
A method utilizing a generative adversarial network (GAN) to generate classifier calibration features without uploading original client data, training a lightweight GAN on the server side to improve model accuracy while ensuring privacy.
Enhances model accuracy by 4% compared to traditional federated learning methods while maintaining privacy protection in IoT systems.
Smart Images
Figure CN119254527A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of Internet of Things privacy security technology, and in particular relates to an Internet of Things security method based on a generative adversarial network. Background Art
[0002] Federated Learning (FL) is a privacy-preserving distributed machine learning method that aims to make full use of data distributed on multiple clients for model training while ensuring data privacy. However, there is an important challenge in federated learning, namely the non-independent and identically distributed (Non-IID) problem of data. Since the data of each client participating in federated learning usually have significant differences in real-world distribution, this data heterogeneity will lead to poor performance of the global model on some clients. Non-IID data will make the effect of model updates very different, thereby affecting the convergence speed and final accuracy of the global model. Moreover, this degradation phenomenon caused by data heterogeneity is particularly obvious in the classifier in the neural network model. Therefore, how to optimize the classifier degradation phenomenon caused by non-IID data while maintaining data privacy and improve the overall performance of the federated learning framework is an urgent problem to be solved. Summary of the invention
[0003] In order to solve the above technical problems, the present invention proposes an Internet of Things security method based on a generative adversarial network, which aims to train a lightweight GAN (generative adversarial network) to generate classifier calibration features without uploading the original data of the client, and retrain the classifier on the server side, thereby greatly improving the model accuracy while achieving Internet of Things privacy protection.
[0004] The present invention provides an Internet of Things security method based on a generative adversarial network, comprising:
[0005] Construct a federated learning model, wherein the federated learning model includes a server and a client, and train a generative adversarial network based on the server and the client to obtain a training result, wherein the training of the generative adversarial network based on the server and the client includes:
[0006] S1. Get the initial parameters of the server;
[0007] S2. Select the initial parameters to obtain client feature data;
[0008] S3. Train the client local model using the client feature data to obtain training results;
[0009] S4, performing federated aggregation on the initial parameters and the training results to obtain synthetic features;
[0010] S5. Training a server-side classifier based on the synthetic features to obtain current global model parameters;
[0011] S6. Select the current global model parameters and repeat steps S2-S5 until the global model parameters meet the preset conditions, then terminate the iteration.
[0012] Optionally, the server-side initial parameters include:
[0013] For each category of data, a lightweight model is used to initialize the GAN model parameters to obtain the first parameters;
[0014] Initialize the global model parameters on the server side and obtain the second parameter;
[0015] Based on the first parameter and the second parameter, the server-side initial parameter is obtained.
[0016] Optionally, selecting the initial parameters to obtain client feature data includes:
[0017] The data types in the client data set are counted, and the initial parameters are selected in combination with the threshold to obtain the client feature data.
[0018] Optionally, selecting the initial parameter in combination with a threshold includes:
[0019] If the amount of data of a certain category in the initial parameters is greater than the threshold, then the data is added to the data set in the client;
[0020] Otherwise, no data is added.
[0021] Optionally, the method for obtaining the threshold is:
[0022]
[0023] Among them, θ is the threshold, w 0 is the server-side initialization global model parameter, F u is a feature extractor with parameter u, d is the feature vector dimension, C v is a classifier with parameters v, is the synthetic feature of the corresponding category generated by the generator
[0024] Optionally, the client feature data is used to train the client local model. The method for obtaining the training result is:
[0025]
[0026] Among them, g jis the feature dataset obtained by the local feature extractor for category j in this client; X j is all the original data of category j in this client, is the local feature extractor, w k t+1 is the local model parameter of client k in round t+1 obtained by training with client k, u k t+1 is the local feature extractor parameter, θ is the threshold, and d i It is the data of category j assigned to client k.
[0027] Optionally, the method for obtaining the synthetic feature is:
[0028]
[0029] Among them, z is a set of noises with a number of num_samples that obey the normal distribution. is the synthetic feature of the corresponding category, k sum is the number of clients, w k t+1 are the local model parameters of client k in the t+1th round obtained by training with client k.
[0030] Optionally, training a server-side classifier based on the synthetic features to obtain current global model parameters includes:
[0031] Training a server-side classifier based on the synthetic features to obtain trained classifier parameters;
[0032] The classifier parameters and the initial parameters of the global model are weighted and aggregated to obtain the current global model parameters.
[0033] Optionally, the method of weighted aggregation of the classifier parameters and the global model initial parameters is:
[0034] v server ′=w 1 v server +w 2 v
[0035] Among them, w 1 is the weight coefficient of the classifier after retraining; w 2 is the weight coefficient of the original classifier after federation aggregation, v server is the global model parameter on the server side, v server ′ is the updated global model parameter, and v is the parameter vector.
[0036] Compared with the prior art, the present invention has the following advantages and technical effects:
[0037] The present invention aims to train a lightweight GAN (generative adversarial network) to generate classifier calibration features without uploading the original data of the client, and retrain the classifier on the server side, thereby greatly improving the model accuracy while achieving privacy protection in the Internet of Things. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0039] Figure 1 is a flow chart of an Internet of Things security method based on a generative adversarial network according to an embodiment of the present invention;
[0040] Figure 2 It is a schematic diagram of the framework of the Internet of Things privacy security method based on the generative adversarial network according to an embodiment of the present invention;
[0041] Figure 3 This is a schematic diagram of the comparison results between the present invention and the most classic federated learning algorithm FedAvg model. DETAILED DESCRIPTION
[0042] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0043] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0044] Due to privacy issues, the client's original data cannot be directly uploaded to calibrate the classifier under the framework of federated learning, and the Generative Adversarial Network (GAN) is a neural network architecture composed of a generator and a discriminator. The generator attempts to deceive the discriminator by generating data from random noise, while the discriminator attempts to distinguish between the generated data and the real data. Through this adversarial training, GAN (Generative Adversarial Network) is able to generate highly realistic data samples. Using GAN (Generative Adversarial Network), the features required for calibrating the classifier can be generated by the trained generator without uploading local data or local features. In view of the shortcomings of the prior art, the present invention proposes a security method for the Internet of Things based on a generative adversarial network. Figure 1 As shown, specifically including:
[0045] S1. Get the initial parameters of the server;
[0046] S2. Select initial parameters and obtain client feature data;
[0047] S3. Train the client local model using the client feature data to obtain training results;
[0048] S4, perform federated aggregation on the initial parameters and training results to obtain synthetic features;
[0049] S5. Train the server-side classifier based on the synthetic features to obtain the current global model parameters;
[0050] S6. Select the current global model parameters and repeat steps S2-S5 until the global model parameters meet the preset conditions, then terminate the iteration.
[0051] Specifically, in step 1, a lightweight WGAN-GP (generative adversarial network) model is used to initialize the GAN (generative adversarial network) model parameters.
[0052] Step 2: Federated learning initialization.
[0053] Step 3: Save the class list select_list_all of trainable GAN (Generative Adversarial Network) according to the threshold θ for client selection.
[0054] Step 4: Local model training.
[0055] Step 5: Use the local model parameter w of client k in the t+1th round obtained by training with client k k t+1 , get the local feature extractor parameters u k t+1 , the data d of category j assigned to client k is i Input to the feature extractor to obtain the parameters (G j , D j ) feature dataset.
[0056] Step 6: For the feature dataset g j Random sampling is performed, and the number of samples is determined by the threshold θ. After sampling, a new feature data set g is obtained j ′. The new feature dataset is used as training (G j , D j ) input, train GAN (generate adversarial network) to get the updated (G j ′,D j ′), and returns the updated model parameters to the server to complete the client-side GAN (generative adversarial network) training.
[0057] Step 7: After the model is aggregated, the server receives the GAN model parameters (G 0 ′,D 0 ′), (G 1 ′,D 1 ′)…(G 9 ′,D 9 ′) and then use each generator to generate a certain number of synthetic features of the corresponding category Update again.
[0058] Step 8: Train the model in the IoT privacy and security framework based on the generative adversarial network optimization, and continuously iterate to complete the training. The model in the IoT privacy and security framework based on the generative adversarial network optimization is as follows: Figure 2 shown.
[0059] Furthermore, obtaining the initial parameters of the server includes:
[0060] For each category of data, a lightweight model is used to initialize the global model parameters to obtain the first parameters;
[0061] Specifically, when initializing the GAN (Generative Adversarial Network) model parameters, in order to reduce the training difficulty of each client, only one category of data is trained for each client. Therefore, a lightweight WGAN-GP (Generative Adversarial Network) model is initialized for each category of data during initialization, corresponding to (G 0 , D 0 ), (G 1 , D 1 )…(G 9 , D 9 ).
[0062] Initialize the global model parameters on the server side and obtain the second parameter;
[0063] Specifically, federated learning initialization: Initialize the global model parameters w on the server side 0 , and distribute it to each client participating in the training, and the global model parameter w 0 , divided into feature extractor F according to the neural network model structure u and classifier C v , where F u is a feature extractor with parameters u, which maps each input sample to a d-dimensional feature vector, where C v is a classifier with parameter v, is a fully connected layer whose output represents the logarithm of the class confidence score.
[0064] Based on the first parameter and the second parameter, the server-side initial parameters are obtained.
[0065] Furthermore, the initial parameters are selected to obtain the client characteristic data including:
[0066] Count the data types in the client data set, and select the initial parameters based on the threshold to obtain the client feature data.
[0067] Furthermore, selecting the initial parameters in combination with the threshold includes:
[0068] If the number of data in a certain category in the initial parameters is greater than the threshold, it is added to the dataset in the client;
[0069] Otherwise, no data is added.
[0070] Specifically, in each iteration, the server randomly selects a portion of clients S t Participate in training. The client counts the types of data contained in its own data set, and saves the class list select_list_all of trainable GAN (Generative Adversarial Network) according to the threshold θ. When the number of data in a certain category in the local data set is greater than the threshold θ, it is added to the list, and the client selection is finally completed.
[0071] Furthermore, the method for obtaining the threshold is:
[0072]
[0073] Among them, θ is the threshold, w 0 is the server-side initialization global model parameter, F u is a feature extractor with parameter u, d is the feature vector dimension, C v is a classifier with parameters v, is the synthetic feature of the corresponding category generated by the generator
[0074] Furthermore, the client feature data is used to train the client local model, and the method for obtaining the training result is:
[0075]
[0076] Among them, g j is the feature dataset obtained by the local feature extractor for category j in this client; X j is all the original data of category j in this client, is the local feature extractor, w k t+1 is the local model parameter of client k in round t+1 obtained by training with client k, u k t+1is the local feature extractor parameter, θ is the threshold, and d i It is the data of category j assigned to client k.
[0077] Specifically, the local client receives the global model parameter w of this round broadcasted by the server t Afterwards, the client uses local data to perform gradient descent on the model parameters.
[0078] After obtaining the feature dataset g of category j of client k j Finally, in order to make all categories of GAN (generative adversarial network) have similar convergence speed, in order to generate more uniform generated features, before training GAN (generative adversarial network), g j Random sampling is performed, and the number of samples is determined by the threshold θ. After sampling, a new feature data set g is obtained j ′. The new feature dataset is used as training (G j , D j ) input, train GAN to get the updated (G j ′,D j ′), and returns the updated model parameters to the server.
[0079] Furthermore, the server receives the GAN (Generative Adversarial Network) model parameters (G 0 ′,D 0 ′), (G 1 ′,D 1 ′)…(G 9 ′,D 9 ′) and then use each generator to generate a certain number of synthetic features of the corresponding category The method to obtain synthetic features is:
[0080]
[0081] Among them, z is a set of noises with a number of num_samples that obey the normal distribution. is the synthetic feature of the corresponding category, k sum is the number of clients, w k t+1 are the local model parameters of client k in the t+1th round obtained by training with client k.
[0082] Specifically, all generators are used to generate a synthetic feature dataset F with a number of num_samples synthetic features and evenly distributed categories. fake , use this dataset to retrain the global model w aggregated by the server t+1 Classifier in The parameter v inserver , the formula is as follows:
[0083]
[0084] in represents the vector after parameter update; v represents the parameter vector of the current iteration; η represents the learning rate, which controls the step size of each update; Indicates the value of the loss function under the current parameter v, represents the input feature of the i-th sample at the t-th iteration, y i represents the true label of the i-th sample, z is a set of noises with a number of num_samples that follow a normal distribution, k sum is the number of clients, w k t+1 are the local model parameters of client k in the t+1th round obtained by training with client k.
[0085] Furthermore, the server-side classifier is trained based on the synthetic features to obtain the current global model parameters including:
[0086] Train the server-side classifier based on the synthetic features and obtain the trained classifier parameters;
[0087] The classifier parameters and the initial parameters of the global model are weighted and aggregated to obtain the current global model parameters. Further, the method of weighted aggregation of the classifier parameters and the initial parameters of the global model is:
[0088] v server ′=w 1 v server +w 2 v
[0089] Among them, w 1 is the weight coefficient of the classifier after retraining; w 2 is the weight coefficient of the original classifier after federation aggregation, v server is the global model parameter on the server side, v server ′ is the updated global model parameter.
[0090] In order to comprehensively evaluate the superior performance of the innovative method proposed in the present invention in practical applications, this embodiment selects a special experimental data set CIFAR-10, and compares the accuracy of the proposed method with the traditional federated learning algorithm. The experimental environment was built on the Ubuntu 18.04.3LTS platform, using Intel CORE I713700KFCPU and NVIDIAGeForce RTX 1080TI GPU (32GB RAM). In terms of the selection of programming languages and frameworks, the deep learning model in this article is built using Python language and Pytorch framework. In a specific implementation plan, the CIFAR-10 data set is divided into a training set and a test set, where the training set has 50,000 images and the test set has 10,000 images.
[0091] In the specific implementation method, the method of the present invention is experimentally compared with the most classic federated learning algorithm FedAvg model, such as Figure 3 As shown, not only the accuracy rate is improved by 4%, but also the privacy protection of IoT is achieved.
[0092] In summary, compared with the classic federated learning method, the method proposed in this embodiment can not only achieve privacy and security protection of the Internet of Things, but also has higher accuracy.
[0093] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A security method for the Internet of Things based on a generative adversarial network, characterized in that: include: Construct a federated learning model, wherein the federated learning model includes a server and a client, and train a generative adversarial network based on the server and the client to obtain a training result, wherein the training of the generative adversarial network based on the server and the client includes: S1. Get the initial parameters of the server; S2. Select the initial parameters to obtain client feature data; S3. Train the client local model using the client feature data to obtain training results; S4, performing federated aggregation on the initial parameters and the training results to obtain synthetic features; S5. Training a server-side classifier based on the synthetic features to obtain current global model parameters; S6. Select the current global model parameters and repeat steps S2-S5 until the global model parameters meet the preset conditions, then terminate the iteration.
2. According to claim 1, a method for Internet of Things security based on generative adversarial networks is characterized in that: Obtaining the initial parameters on the server side includes: For each category of data, a lightweight model is used to initialize the GAN model parameters to obtain the first parameters; Initialize the global model parameters on the server side and obtain the second parameter; Based on the first parameter and the second parameter, the server-side initial parameter is obtained.
3. The IoT security method based on generative adversarial network according to claim 1 is characterized in that: Selecting the initial parameters to obtain client feature data includes: The data types in the client data set are counted, and the initial parameters are selected in combination with the threshold to obtain the client feature data.
4. The IoT security method based on generative adversarial network according to claim 3 is characterized in that: Selecting the initial parameters in combination with the threshold includes: If the amount of data of a certain category in the initial parameters is greater than the threshold, then the data is added to the data set in the client; Otherwise, no data is added.
5. The IoT security method based on generative adversarial network according to claim 3 is characterized in that: The method for obtaining the threshold is: Among them, θ is the threshold, w0 is the global model parameter initialized on the server side, and F u is a feature extractor with parameter u, d is the feature vector dimension, C v is a classifier with parameters v, is the synthetic feature of the corresponding category generated by the generator 6. The IoT security method based on generative adversarial network according to claim 1 is characterized in that: The client feature data is used to train the client local model. The method to obtain the training results is as follows: Among them, g j is the feature dataset obtained by the local feature extractor for category j in this client; X j is all the original data of category j in this client, is the local feature extractor, w k t+1 is the local model parameter of client k in round t+1 obtained by training with client k, u k t+1 is the local feature extractor parameter, θ is the threshold, and d i It is the data of category j assigned to client k.
7. The IoT security method based on generative adversarial network according to claim 1 is characterized in that: The method for obtaining the synthetic feature is: Among them, z is a set of noises with a number of num_samples that obey the normal distribution. is the synthetic feature of the corresponding category, k sum is the number of clients, w k t+1 are the local model parameters of client k in the t+1th round obtained by training with client k.
8. The IoT security method based on generative adversarial network according to claim 1 is characterized in that: Training the server-side classifier based on the synthetic features to obtain the current global model parameters includes: Training a server-side classifier based on the synthetic features to obtain trained classifier parameters; The classifier parameters and the initial parameters of the global model are weighted and aggregated to obtain the current global model parameters.
9. The IoT security method based on generative adversarial network according to claim 8, characterized in that: The method for weighted aggregation of the classifier parameters and the initial parameters of the global model is: in server ′=w1v server +w2v Among them, w1 is the weight coefficient of the retrained classifier; w2 is the weight coefficient of the original classifier after federation aggregation, v server is the global model parameter on the server side, v server ′ is the updated global model parameter, and v is the parameter vector.