Efficient IoT federated learning method and system based on adaptive differential privacy

Abstract

The present invention proposes an efficient Internet of Things federated learning method and system based on adaptive differential privacy. The efficient Internet of Things federated learning method based on adaptive differential privacy includes: extracting the local model of the client and monitoring the iteration process corresponding to the local model in real time; collecting the parameter information of the model during the iteration process corresponding to the local model, and constructing a mechanism for quantifying the importance of model parameters through the parameter information; adding noise perturbations to the parameters of the model iteration according to the mechanism for quantifying the importance of model parameters. The system includes modules corresponding to the steps of the method.

Description

Technical Field

[0001] The present invention proposes an efficient Internet of Things federated learning method and system based on adaptive differential privacy, belonging to the technical field of federated learning. Background Art

[0002] With the development of IoT sensors and communication technology, IoT devices will generate a large amount of data. In order to fully utilize the value of data while protecting user data privacy, the current common method is to use federated learning plus some common encryption methods to learn and train models. The general federated learning method still has the risk of privacy leakage, and attackers can infer user data information through the gradient of the model. Federated learning plus general encryption algorithms usually increase communication pressure. At the same time, if the encryption level is not well controlled, it will lose the meaning of encryption if it is too low, and it will be difficult for training to converge if it is too high. Summary of the invention

[0003] The present invention provides an efficient IoT federated learning method and system based on adaptive differential privacy to solve the problem that the encryption degree of the prior art cannot be well controlled. The technical solutions adopted are as follows:

[0004] An efficient Internet of Things federated learning method based on adaptive differential privacy, the efficient Internet of Things federated learning method based on adaptive differential privacy comprising:

[0005] Extracting the local model of the client and monitoring the iteration process corresponding to the local model in real time;

[0006] Collecting parameter information of the model during the iteration process corresponding to the local model, and constructing a mechanism for quantifying the importance of model parameters through the parameter information;

[0007] Noise perturbations are added to the parameters of the model iterations according to a mechanism that quantifies the importance of the model parameters.

[0008] Further, extracting the local model of the client and monitoring the iteration process corresponding to the local model in real time include:

[0009] Extract the client's local model and the preset number of iteration cycles for local model training;

[0010] Setting the number of iteration buffers according to a preset number of iteration cycles of the local model training;

[0011] Monitor in real time whether the local model has started training iterations;

[0012] When the training iteration is started, the iteration buffer number is retrieved;

[0013] The iteration status of the local model is monitored in real time. When the number of iterations experienced by the local model after the iteration is started reaches the iteration buffer number, the parameter information of the iteration process of the local model after the iteration number reaches the iteration buffer number is monitored and collected in real time.

[0014] Furthermore, the number of iteration buffering times is obtained by the following formula:

[0015] N = int[(1+ε)×N0 / 10]

[0016] Among them, N represents the number of iteration buffers; ε represents the privacy parameter used to measure the scale of random noise in local differential privacy; N0 represents the total number of iterations preset to complete model training.

[0017] Furthermore, in the iteration process corresponding to the local model, parameter information of the model is collected, including:

[0018] Collecting the gradient update size of the model iteration during the iteration process corresponding to the local model;

[0019] Collecting the weight parameter size of the model iteration during the iteration process corresponding to the local model;

[0020] Collecting the relationship between the local update and the global update trend of the model iteration during the iteration process corresponding to the local model;

[0021] A mechanism for quantifying the importance of model parameters is constructed through the relationship between the gradient update size of the model iteration, the weight parameter size, and the local update and global update trends.

[0022] Furthermore, the mechanism for quantifying the importance of model parameters includes noise injection, local differential privacy, and global differential privacy; the mechanism for quantifying the importance of model parameters is as follows:

[0023] The noise injection is:

[0024] For each client, when calculating the gradient, a noise adoption number is added to each gradient to prevent malicious attackers from inferring sensitive information from a single gradient;

[0025] Local differential privacy is:

[0026] Setting a privacy parameter, wherein the privacy parameter is used to control the privacy level of differential privacy;

[0027] After each iteration, random noise is set by the privacy parameter;

[0028] Adding the random noise to the next iteration training of the local model;

[0029] Global Differential Privacy:

[0030] Transmitting the noisy model parameters to a central server;

[0031] The random noise corresponding to the global difference is obtained by using the differential privacy aggregation algorithm, and the random noise corresponding to the global difference is added to the next iterative training of the local model.

[0032] An efficient Internet of Things federated learning system based on adaptive differential privacy, the efficient Internet of Things federated learning system based on adaptive differential privacy comprising:

[0033] A local model extraction module is used to extract the local model of the client and monitor the iteration process corresponding to the local model in real time;

[0034] A parameter information collection module, used to collect parameter information of the model during the iteration process corresponding to the local model, and to construct a mechanism for quantifying the importance of model parameters through the parameter information;

[0035] The perturbation information adding module is used to add noise perturbations to the parameters of the model iteration according to the mechanism of quantifying the importance of the model parameters.

[0036] Furthermore, the local model extraction module includes:

[0037] An iteration cycle number extraction module is used to extract the client's local model and the preset iteration cycle number of the local model training;

[0038] An iteration buffer number setting module, used to set the iteration buffer number according to a preset number of iteration cycles of the local model training;

[0039] Real-time monitoring module, used to monitor in real time whether the local model has started training iterations;

[0040] A number calling module, used for calling the iteration buffer number when the training iteration is started;

[0041] The iteration status monitoring module is used to monitor the iteration status of the local model in real time. When the number of iterations experienced by the local model after the iteration is started reaches the iteration buffer number, the parameter information of the iteration process of the local model after the iteration number reaches the iteration buffer number is monitored and collected in real time.

[0042] Furthermore, the number of iteration buffering times is obtained by the following formula:

[0043] N = int[(1+ε)×N0 / 10]

[0044] Among them, N represents the number of iteration buffers; ε represents the privacy parameter used to measure the scale of random noise in local differential privacy; N0 represents the total number of iterations preset to complete model training.

[0045] Furthermore, the parameter information collection module includes:

[0046] A gradient update size information acquisition module, used to collect the gradient update size of the model iteration during the iteration process corresponding to the local model;

[0047] A weight parameter size information acquisition module, used to collect the weight parameter size of the model iteration during the iteration process corresponding to the local model;

[0048] An update relationship acquisition module, used for collecting the relationship between the local update and the global update trend of the model iteration in the iteration process corresponding to the local model;

[0049] The mechanism building module is used to build a mechanism to quantify the importance of model parameters through the relationship between the gradient update size of the model iteration, the weight parameter size, and the local update and global update trends.

[0050] Furthermore, the mechanism for quantifying the importance of model parameters includes noise injection, local differential privacy, and global differential privacy; the mechanism for quantifying the importance of model parameters is as follows:

[0051] The noise injection is:

[0052] For each client, when calculating the gradient, a noise adoption number is added to each gradient to prevent malicious attackers from inferring sensitive information from a single gradient;

[0053] Local differential privacy is:

[0054] Setting a privacy parameter, wherein the privacy parameter is used to control the privacy level of differential privacy;

[0055] After each iteration, random noise is set by the privacy parameter;

[0056] Adding the random noise to the next iteration training of the local model;

[0057] Global Differential Privacy:

[0058] Transmitting the noisy model parameters to a central server;

[0059] The random noise corresponding to the global difference is obtained by using the differential privacy aggregation algorithm, and the random noise corresponding to the global difference is added to the next iterative training of the local model.

[0060] Beneficial effects of the present invention:

[0061] In the iterative process of local model training, the efficient IoT federated learning method and system client based on adaptive differential privacy proposed in the present invention first designs a mechanism to quantify the importance of model parameters by comprehensively analyzing the relationship between the gradient update size, the weight parameter size, and the local update and global update trends; then, noise perturbations are added to the model parameters in a differentiated manner according to the importance of each model parameter. Under the premise of protecting privacy, the availability of the training model is improved, making IoT federated learning training easier. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is a flow chart of the method of the present invention;

[0063] Figure 2 The system block diagram of the system of the present invention. DETAILED DESCRIPTION

[0064] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0065] The embodiment of the present invention proposes an efficient IoT federated learning method based on adaptive differential privacy, such as Figure 1 As shown, the efficient IoT federated learning method based on adaptive differential privacy includes:

[0066] S1. Extracting a local model of the client and monitoring the iteration process corresponding to the local model in real time;

[0067] S2. Collecting parameter information of the model during the iteration process corresponding to the local model, and constructing a mechanism for quantifying the importance of model parameters through the parameter information;

[0068] S3. Add noise perturbations to the parameters of the model iterations according to a mechanism that quantifies the importance of model parameters.

[0069] The working principle of the above technical solution is as follows: Extract local model and monitor iteration process (S1): First, extract the local model from the client device and monitor the iteration process of the local model in real time. In federated learning, each client device will independently perform model training iterations, and the S1 stage is used to extract these local models.

[0070] Collect model parameter information and build importance mechanism (S2): During the iteration process corresponding to the local model, collect the model parameter information. This parameter information may include weights, gradients, etc. Then, build a mechanism through this parameter information to quantify the importance of model parameters. This can help determine which parameters have the greatest impact on the performance of the global model.

[0071] Adding differential privacy noise (S3): According to the mechanism of model parameter importance, differential privacy noise perturbations are added to the parameters of the model iteration. This noise is to protect the privacy of the model parameters so that sensitive information of individual clients will not be leaked during the global model aggregation process. The size and distribution of the noise can be adjusted according to the mechanism of parameter importance to more effectively protect privacy.

[0072] The effects of the above technical solution are: Differential privacy protection: By adding differential privacy noise to the model parameters, the technical solution of this embodiment helps to protect the privacy of the model parameters and ensure that the sensitive information of the client is not leaked. This is very important for IoT applications that process sensitive data.

[0073] Adaptability: By building an importance mechanism, we can adaptively determine which model parameters require more differential privacy protection. This allows different noise levels to be distributed among different parameters to minimize adverse effects on model performance.

[0074] Efficiency: The technical solution of this embodiment can efficiently add differential privacy noise during the model training process without significantly affecting the training speed and model performance.

[0075] In general, the goal of the technical solution of this embodiment is to combine adaptivity and differential privacy to protect the privacy of model parameters while improving the efficiency and performance of federated learning, which is particularly suitable for situations where sensitive data needs to be processed in IoT applications.

[0076] An embodiment of the present invention extracts a local model of a client and monitors the iteration process corresponding to the local model in real time, including:

[0077] S101, extracting the local model of the client and the preset number of iteration cycles of the local model training;

[0078] S102, setting the number of iteration buffers according to a preset number of iteration cycles of the local model training;

[0079] S103, monitoring in real time whether the local model is started for training iteration;

[0080] S104, when the training iteration is started, calling the iteration buffer number;

[0081] S105. Real-time monitoring of the iteration status of the local model. When the number of iterations experienced by the local model after the iteration is started reaches the iteration buffer number, real-time monitoring and collection of parameter information of the iteration process of the local model after the iteration number reaches the iteration buffer number is performed.

[0082] The number of iteration buffer times is obtained by the following formula:

[0083] N = int[(1+ε)×N0 / 10]

[0084] Among them, N represents the number of iteration buffers; ε represents the privacy parameter used to measure the scale of random noise in local differential privacy; N0 represents the total number of iterations preset to complete model training.

[0085] The working principle of the above technical solution is: Get local model and set iteration number (S101): First, extract the local model from the client device and obtain the preset iteration cycle number. These iteration cycle numbers are usually used to control the iteration number of model training to achieve a specific training goal.

[0086] Setting the number of iteration buffers (S102): Based on a preset number of iteration cycles, setting the number of iteration buffers. The number of iteration buffers is a threshold indicating when to monitor and collect parameter information during the local model training process.

[0087] Real-time monitoring of training iteration start (S103): Real-time monitoring of whether the local model has started the training iteration process. This usually involves detecting the training state of the local model to determine whether the training has started.

[0088] Retrieving the number of iteration buffers (S104): When it is detected that a training iteration has been started, the number of iteration buffers is retrieved, which is a flag used to determine when to perform parameter information monitoring and collection.

[0089] Real-time monitoring and collection of parameter information (S105): Once the number of training iterations of the local model reaches the iteration buffer number, the iteration status of the local model is monitored in real time and parameter information is collected, which may include parameter information such as model weights and gradients.

[0090] The effects of the above technical solution are: Real-time monitoring: By real-time monitoring of the training iteration process of the local model, parameter information can be collected at an appropriate time. This helps to capture the status and performance of the local model during the training process.

[0091] Dynamic control: By setting the number of iteration buffers, it is possible to dynamically determine when to monitor and collect parameter information to avoid monitoring every iteration, thereby reducing overhead.

[0092] Data collection: Real-time collection of parameter information can be used for further analysis and decision-making, such as evaluating model performance and monitoring training progress.

[0093] In general, the technical solution of this embodiment is intended to allow real-time monitoring of the training process of the local model and the collection of parameter information to support better model management and performance monitoring. This is beneficial for applications such as federated learning.

[0094] In one embodiment of the present invention, collecting parameter information of the model during the iteration process corresponding to the local model includes:

[0095] S201, collecting the gradient update size of the model iteration during the iteration process corresponding to the local model;

[0096] S202, collecting the weight parameter size of the model iteration during the iteration process corresponding to the local model;

[0097] S203, collecting the relationship between the local update and the global update trend of the model iteration during the iteration process corresponding to the local model;

[0098] S204. Construct a mechanism to quantify the importance of model parameters through the relationship between the gradient update size of the model iteration, the weight parameter size, and the local update and global update trends.

[0099] The working principle of the above technical solution is: Collecting the gradient update size (S201): In the iteration process of the local model, the size of the gradient update of the model iteration is collected. The gradient update size reflects the change of the model parameters in each iteration and is one of the important indicators of model training.

[0100] Collecting weight parameter size (S202): Similarly, in the iteration process, the size of the weight parameter of the model iteration is collected. The weight parameter size represents the absolute value of different parameters in the model and is also one of the indicators for evaluating the importance of the parameter.

[0101] Collecting the relationship between local updates and global updates (S203): During the iteration process, the relationship between the local updates and global updates of the model iteration is collected. This may include analyzing the relationship between the performance improvement of the local model and the trend of the global model to determine which parameters are most important for the performance improvement of the overall model.

[0102] Constructing a mechanism to quantify the importance of model parameters (S204): Based on the above parameter information, construct a mechanism to quantify the importance of model parameters. This mechanism can use various algorithms and techniques to analyze the impact of parameters to determine which parameters have a greater impact on the performance and convergence speed of the model.

[0103] The effects of the above technical solutions are: Parameter importance assessment: By quantifying the importance of model parameters, it can help determine which parameters are most important for improving the performance of the overall model. This helps to protect and optimize the parameters more targeted.

[0104] Adaptability: The technical solution of this embodiment constructs a mechanism for parameter importance based on the data of actual model training, so it is adaptive and can be adjusted according to the conditions of different models and data sets.

[0105] Optimizing model management: Quantifying the importance of model parameters can be used to optimize model management strategies, including the addition of differential privacy noise and model aggregation methods, to improve the performance of federated learning.

[0106] In general, the goal of the technical solution of this embodiment is to help better understand the behavior of the model during the iteration process by quantifying the importance of model parameters, thereby achieving better model protection, optimization and performance improvement.

[0107] In one embodiment of the present invention, the mechanism for quantifying the importance of model parameters includes noise injection, local differential privacy, and global differential privacy; the mechanism for quantifying the importance of model parameters is as follows:

[0108] The noise injection is:

[0109] For each client, when calculating the gradient, a noise adoption number is added to each gradient to prevent malicious attackers from inferring sensitive information from a single gradient;

[0110] Local differential privacy is:

[0111] Setting a privacy parameter, wherein the privacy parameter is used to control the privacy level of differential privacy;

[0112] After each iteration, random noise is set by the privacy parameter;

[0113] Adding the random noise to the next iteration training of the local model;

[0114] Global Differential Privacy:

[0115] Transmitting the noisy model parameters to a central server;

[0116] The random noise corresponding to the global difference is obtained by using the differential privacy aggregation algorithm, and the random noise corresponding to the global difference is added to the next iterative training of the local model.

[0117] The working principle of the above technical solution is: Noise injection: The mechanism of noise injection is to process the gradient on the client device to increase the protection of differential privacy. The specific working principle is as follows:

[0118] For each client, when calculating the gradient, a noise adoption number is added to each gradient. This noise adoption number is random and obscures the exact value of the gradient to some extent.

[0119] The purpose of noise injection is to prevent malicious attackers from inferring sensitive information from a single gradient and improve the privacy of model training.

[0120] Local differential privacy: The local differential privacy mechanism is used to increase the protection of differential privacy on the client device and control the privacy level of differential privacy. The working principle is as follows:

[0121] Set the privacy parameter, which is used to control the privacy level of differential privacy. Smaller parameter values ​​indicate stricter privacy protection.

[0122] After each iteration, random noise is generated by the privacy parameter.

[0123] The generated random noise is added to the next iteration training of the local model to blur the gradient information.

[0124] This mechanism can adjust the privacy level according to the privacy parameters of differential privacy to balance the trade-off between privacy and model performance.

[0125] Global differential privacy: The global differential privacy mechanism is used to increase the protection of differential privacy when model parameters are transmitted to the central server.

[0126] The noisy model parameters are transmitted to a central server.

[0127] The central server uses a differential privacy aggregation algorithm to obtain random noise corresponding to the global difference.

[0128] The central server returns the random noise corresponding to the global difference to the client device.

[0129] The client device adds the random noise corresponding to the global difference to the next iteration of training of the local model.

[0130] This mechanism helps to provide differential privacy protection during model parameter transmission.

[0131] The effects of the above technical solutions are: Increased differential privacy: Through these mechanisms, the protection of differential privacy is increased, which can effectively reduce the risk of information leakage of gradients and model parameters, thereby protecting the privacy of user data.

[0132] Adjustable privacy level: The setting of privacy parameters can adjust the privacy level of differential privacy to meet different privacy requirements, from stricter to more relaxed privacy requirements.

[0133] Improved model security: These mechanisms improve the security of the model, making it more resistant to malicious attacks and privacy violations.

[0134] In general, the goal of these differential privacy mechanisms is to provide effective privacy protection in federated learning while maintaining an acceptable level of model performance. This is very beneficial for applications that process sensitive data.

[0135] The embodiment of the present invention proposes an efficient IoT federated learning system based on adaptive differential privacy, such as Figure 2 As shown, the efficient IoT federated learning system based on adaptive differential privacy includes:

[0136] A local model extraction module is used to extract the local model of the client and monitor the iteration process corresponding to the local model in real time;

[0137] A parameter information collection module, used to collect parameter information of the model during the iteration process corresponding to the local model, and to construct a mechanism for quantifying the importance of model parameters through the parameter information;

[0138] The perturbation information adding module is used to add noise perturbations to the parameters of the model iteration according to the mechanism of quantifying the importance of the model parameters.

[0139] The working principle of the above technical solution is as follows: First, extract the local model from the client device and monitor the iteration process of the local model in real time. In federated learning, each client device will independently perform model training iterations, and the S1 stage is used to extract these local models.

[0140] During the iteration process corresponding to the local model, the parameter information of the model is collected. This parameter information may include weights, gradients, etc. Then, a mechanism is built through this parameter information to quantify the importance of model parameters. This can help determine which parameters have the greatest impact on the performance of the global model.

[0141] According to the mechanism of model parameter importance, differential privacy noise perturbation is added to the parameters of model iteration. This noise is to protect the privacy of model parameters so that sensitive information of individual clients will not be leaked during the global model aggregation process. The size and distribution of the noise can be adjusted according to the mechanism of parameter importance to more effectively protect privacy.

[0142] The effect of the above technical solution is: by adding differential privacy noise to the model parameters, the technical solution of this embodiment helps to protect the privacy of the model parameters and ensure that the sensitive information of the client is not leaked. This is very important for IoT applications that process sensitive data. By constructing an importance mechanism, it is possible to adaptively determine which model parameters require more differential privacy protection. This allows different noise levels to be allocated between different parameters to minimize the adverse effects on model performance. The technical solution of this embodiment can efficiently add differential privacy noise during model training without significantly affecting the training speed and model performance. The goal of the technical solution of this embodiment is to combine adaptivity and differential privacy to protect the privacy of model parameters while improving the efficiency and performance of federated learning, which is particularly suitable for situations where sensitive data needs to be processed in IoT applications.

[0143] In one embodiment of the present invention, the local model extraction module includes:

[0144] An iteration cycle number extraction module is used to extract the client's local model and the preset iteration cycle number of the local model training;

[0145] An iteration buffer number setting module, used to set the iteration buffer number according to a preset number of iteration cycles of the local model training;

[0146] Real-time monitoring module, used to monitor in real time whether the local model has started training iterations;

[0147] A number calling module, used for calling the iteration buffer number when the training iteration is started;

[0148] The iteration status monitoring module is used to monitor the iteration status of the local model in real time. When the number of iterations experienced by the local model after the iteration is started reaches the iteration buffer number, the parameter information of the iteration process of the local model after the iteration number reaches the iteration buffer number is monitored and collected in real time.

[0149] The number of iteration buffer times is obtained by the following formula:

[0150] N = int[(1+ε)×N0 / 10]

[0151] Among them, N represents the number of iteration buffers; ε represents the privacy parameter used to measure the scale of random noise in local differential privacy; N0 represents the total number of iterations preset to complete model training.

[0152] The working principle of the above technical solution is as follows: first, extract the local model from the client device and obtain the preset number of iteration cycles. These iteration cycle numbers are usually used to control the number of iterations of model training to achieve specific training goals. Based on the preset number of iteration cycles, set the iteration buffer number. The iteration buffer number is a threshold that indicates when to monitor and collect parameter information during the local model training process. Monitor in real time whether the local model has started the training iteration process. This usually involves detecting the training status of the local model to determine whether training has started. When it is detected that the training iteration has started, call the iteration buffer number, which is a sign to determine when to monitor and collect parameter information. Once the number of training iterations of the local model reaches the iteration buffer number, start monitoring the iteration of the local model in real time and collect parameter information. This can include parameter information such as model weights and gradients.

[0153] The effect of the above technical solution is: by monitoring the training iteration process of the local model in real time, parameter information can be collected at an appropriate time. This helps to capture the state and performance of the local model during the training process. By setting the number of iteration buffers, it is possible to dynamically determine when to monitor and collect parameter information to avoid monitoring each iteration, thereby reducing overhead. Real-time collection of parameter information can be used for further analysis and decision-making, such as evaluating the performance of the model, monitoring the progress of training, etc. The technical solution of this embodiment is intended to allow real-time monitoring of the training process of the local model and collection of parameter information to support better model management and performance monitoring. This is beneficial for applications such as federated learning.

[0154] In one embodiment of the present invention, the parameter information collection module includes:

[0155] A gradient update size information acquisition module, used to collect the gradient update size of the model iteration during the iteration process corresponding to the local model;

[0156] A weight parameter size information acquisition module, used to collect the weight parameter size of the model iteration during the iteration process corresponding to the local model;

[0157] An update relationship acquisition module, used for collecting the relationship between the local update and the global update trend of the model iteration in the iteration process corresponding to the local model;

[0158] The mechanism building module is used to build a mechanism to quantify the importance of model parameters through the relationship between the gradient update size of the model iteration, the weight parameter size, and the local update and global update trends.

[0159] The working principle of the above technical solution is as follows: during the iteration process of the local model, the size of the gradient update of the model iteration is collected. The size of the gradient update reflects the change of the model parameters in each iteration and is one of the important indicators of model training. Similarly, during the iteration process, the size of the weight parameter of the model iteration is collected. The weight parameter size represents the absolute value of different parameters in the model and is also one of the indicators for evaluating the importance of the parameters. During the iteration process, the relationship between the local update and the global update of the model iteration is collected. This can include analyzing the trend relationship between the performance improvement of the local model and the global model to determine which parameters are most important for the performance improvement of the overall model. Based on the above parameter information, a mechanism is constructed to quantify the importance of model parameters. This mechanism can use various algorithms and techniques to analyze the impact of parameters to determine which parameters have a greater impact on the performance and convergence speed of the model.

[0160] The effect of the above technical solution is: by quantifying the importance of model parameters, it can help determine which parameters are most important for improving the performance of the overall model. This helps to protect and optimize the parameters more targeted. The technical solution of this embodiment constructs a mechanism for parameter importance based on the data of actual model training, so it is adaptive and can be adjusted according to the conditions of different models and data sets. The importance of quantified model parameters can be used to optimize model management strategies, including the addition of differential privacy noise and model aggregation methods to improve the performance of federated learning.

[0161] In general, the goal of the technical solution of this embodiment is to help better understand the behavior of the model during the iteration process by quantifying the importance of model parameters, thereby achieving better model protection, optimization and performance improvement.

[0162] In one embodiment of the present invention, the mechanism for quantifying the importance of model parameters includes noise injection, local differential privacy, and global differential privacy; the mechanism for quantifying the importance of model parameters is as follows:

[0163] The noise injection is:

[0164] For each client, when calculating the gradient, a noise adoption number is added to each gradient to prevent malicious attackers from inferring sensitive information from a single gradient;

[0165] Local differential privacy is:

[0166] Setting a privacy parameter, wherein the privacy parameter is used to control the privacy level of differential privacy;

[0167] After each iteration, random noise is set by the privacy parameter;

[0168] Adding the random noise to the next iteration training of the local model;

[0169] Global Differential Privacy:

[0170] Transmitting the noisy model parameters to a central server;

[0171] The random noise corresponding to the global difference is obtained by using the differential privacy aggregation algorithm, and the random noise corresponding to the global difference is added to the next iterative training of the local model.

[0172] The working principle of the above technical solution is: the mechanism of noise injection is to process the gradient on the client device to increase the protection of differential privacy. The specific working principle is as follows:

[0173] For each client, when calculating the gradient, a noise adoption number is added to each gradient. This noise adoption number is random and obscures the exact value of the gradient to some extent.

[0174] The purpose of noise injection is to prevent malicious attackers from inferring sensitive information from a single gradient and improve the privacy of model training.

[0175] The local differential privacy mechanism is used to increase the protection of differential privacy on the client device and control the privacy level of differential privacy. The working principle is as follows:

[0176] Set the privacy parameter, which is used to control the privacy level of differential privacy. Smaller parameter values ​​indicate stricter privacy protection.

[0177] After each iteration, random noise is generated by the privacy parameter.

[0178] The generated random noise is added to the next iteration training of the local model to blur the gradient information.

[0179] This mechanism can adjust the privacy level according to the privacy parameters of differential privacy to balance the trade-off between privacy and model performance.

[0180] The global differential privacy mechanism is used to increase the protection of differential privacy when model parameters are transmitted to the central server.

[0181] The noisy model parameters are transmitted to a central server.

[0182] The central server uses a differential privacy aggregation algorithm to obtain random noise corresponding to the global difference.

[0183] The central server returns the random noise corresponding to the global difference to the client device.

[0184] The client device adds the random noise corresponding to the global difference to the next iteration of training of the local model.

[0185] This mechanism helps to provide differential privacy protection during model parameter transmission.

[0186] The effect of the above technical solution is: through these mechanisms, the protection of differential privacy is increased, which can effectively reduce the risk of information leakage of gradients and model parameters, thereby protecting the privacy of user data. The setting of privacy parameters can adjust the privacy level of differential privacy to meet different privacy requirements, from stricter to more relaxed privacy requirements. These mechanisms improve the security of the model and make it more resistant to malicious attacks and privacy violations.

[0187] In general, the goal of these differential privacy mechanisms is to provide effective privacy protection in federated learning while maintaining an acceptable level of model performance. This is very beneficial for applications that process sensitive data.

[0188] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An efficient IoT federated learning method based on adaptive differential privacy, characterized by: The efficient IoT federated learning method based on adaptive differential privacy includes: Extracting the local model of the client and monitoring the iteration process corresponding to the local model in real time; Collecting parameter information of the model during the iteration process corresponding to the local model, and constructing a mechanism for quantifying the importance of model parameters through the parameter information; Add noise perturbations to the parameters of the model iterations according to a mechanism that quantifies the importance of the model parameters; The process of extracting the local model of the client and monitoring the iteration process corresponding to the local model in real time includes: Extract the client's local model and the preset number of iteration cycles for local model training; Setting the number of iteration buffers according to a preset number of iteration cycles of the local model training; Monitor in real time whether the local model has started training iterations; When the training iteration is started, the iteration buffer number is retrieved; Monitor the iteration status of the local model in real time. When the number of iterations experienced by the local model after the iteration is started reaches the iteration buffer number, perform real-time monitoring and collection of parameter information of the iteration process of the local model after the number of iterations reaches the iteration buffer number. The number of iteration buffer times is obtained by the following formula: N = int[(1 + ε) × N0 / 10] Among them, N represents the number of iteration buffers; ε represents the privacy parameter used to measure the scale of random noise in local differential privacy; N0 represents the total number of iterations preset to complete model training.

2. The efficient IoT federated learning method based on adaptive differential privacy according to claim 1 is characterized in that: Collecting model parameter information during the iteration process corresponding to the local model includes: Collecting the gradient update size of the model iteration during the iteration process corresponding to the local model; Collecting the weight parameter size of the model iteration during the iteration process corresponding to the local model; Collecting the relationship between the local update and the global update trend of the model iteration during the iteration process corresponding to the local model; A mechanism for quantifying the importance of model parameters is constructed through the relationship between the gradient update size of the model iteration, the weight parameter size, and the local update and global update trends.

3. An efficient IoT federated learning system based on adaptive differential privacy, characterized by: The efficient IoT federated learning system based on adaptive differential privacy includes: A local model extraction module is used to extract the local model of the client and monitor the iteration process corresponding to the local model in real time; A parameter information collection module, used to collect parameter information of the model during the iteration process corresponding to the local model, and to construct a mechanism for quantifying the importance of model parameters through the parameter information; A disturbance information adding module, which is used to add noise disturbances to the parameters of the model iteration according to a mechanism to quantify the importance of the model parameters; Wherein, the local model extraction module includes: An iteration cycle number extraction module is used to extract the client's local model and the preset iteration cycle number of the local model training; An iteration buffer number setting module, used to set the iteration buffer number according to a preset number of iteration cycles of the local model training; Real-time monitoring module, used to monitor in real time whether the local model has started training iterations; A number calling module, used for calling the iteration buffer number when the training iteration is started; The iteration status monitoring module is used to monitor the iteration status of the local model in real time. When the number of iterations experienced by the local model after the iteration is started reaches the iteration buffer number, the parameter information of the iteration process of the local model after the iteration number reaches the iteration buffer number is monitored and collected in real time. The number of iteration buffer times is obtained by the following formula: N = int[(1 + ε) × N0 / 10] Among them, N represents the number of iteration buffers; ε represents the privacy parameter used to measure the scale of random noise in local differential privacy; N0 represents the total number of iterations preset to complete model training.

4. The efficient IoT federated learning system based on adaptive differential privacy according to claim 3 is characterized in that: The parameter information collection module includes: A gradient update size information acquisition module, used to collect the gradient update size of the model iteration during the iteration process corresponding to the local model; A weight parameter size information acquisition module, used to collect the weight parameter size of the model iteration during the iteration process corresponding to the local model; An update relationship acquisition module, used for collecting the relationship between the local update and the global update trend of the model iteration in the iteration process corresponding to the local model; The mechanism building module is used to build a mechanism to quantify the importance of model parameters through the relationship between the gradient update size of the model iteration, the weight parameter size, and the local update and global update trends.

Images