An industrial internet of things fault detection method based on federal incremental learning
Patent Information
- Application Number
- CN202311506506.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-13
- Publication Date
- 2026-11-03
- Estimated Expiration
- 2043-11-13
AI Technical Summary
In the Industrial Internet of Things (IIoT), a major challenge is how to quickly extract and learn the changing characteristics of fault data, effectively achieve federated aggregation, and prevent the malicious collection of information from participating parties.
A federated incremental learning approach is adopted. The local fault detection model is trained by calculating the gradient of the data class to learn the weights and adding noise. The aggregate weights of the local fault detection models of the participants are calculated. Finally, the local fault detection models of different participants are fully aggregated to obtain the global fault detection model. The differential privacy mechanism of the local model is implemented by combining Opacus.
It achieves high-precision convergence and personalized training of local models, alleviates the degradation problem caused by model aggregation, improves fault detection capabilities, and accelerates the effective convergence of global models.
Smart Images
Figure CN117579468B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of industrial Internet of Things (IoT) fault detection, specifically relating to an industrial IoT fault detection method based on federated incremental learning. Background Technology
[0002] In the Industrial Internet of Things (IIoT) field, federated learning has seen a significant increase in practical application demand in areas such as real-time monitoring and intelligent decision-making in industrial production due to its outstanding performance in data privacy protection and improved model generality. Fault detection is one of the important research topics in the IIoT field. At the same time, the amount, characteristics, and categories of fault data that need to be detected are all growing rapidly, and the correlations between faults are also generating increasingly diverse new characteristics. How to quickly extract and learn the changing fault data features, and effectively achieve federated aggregation while preventing the malicious collection of information from participating parties as much as possible, has become a major challenge in fault detection in the IIoT. Summary of the Invention
[0003] The purpose of this invention is to provide an industrial IoT fault detection method based on federated incremental learning. Focusing on the fault detection task in the industrial IoT, this invention emphasizes the convergence problem of federated training algorithms on local datasets and the efficient aggregation problem on a global scale. Based on the idea of synergistic local learning and global aggregation, it provides technical support for improving the security and performance of artificial intelligence models in the industrial IoT.
[0004] To achieve the above objectives, the technical solution of the present invention is: an industrial Internet of Things (IoT) fault detection method based on federated incremental learning, comprising the following steps:
[0005] Step 1: Calculate the gradient learning weights for the data classes and add noise to train the local fault detection model;
[0006] Step 2: Calculate the aggregate weights of the local fault detection model for participating parties;
[0007] Step 3: Combine the local fault detection models of different participants to obtain the global fault detection model.
[0008] In one embodiment of the present invention, step 1 is implemented as follows: the numerical representations of the fault detection accuracy of the local fault detection model and the fault detection accuracy of the existing global fault detection model are respectively extracted into ordered lists. and Based on the characteristics of the fault detection task, an ordered list is used. and Knowledge distillation is performed to calculate the gradient learning weights for fault data classes. Based on the obtained gradient learning weights for the data classes, the fault detection model is used to perform different gradient propagation learning on the data classes on the training set, and Gaussian noise is added during the learning process.
[0009] In one embodiment of the present invention, the gradient learning weights for fault data classes are calculated as follows:
[0010] Step 1: Calculate the accuracy of distillation data.
[0011] Data distillation is performed on the classification accuracy of data sample classes using a global fault detection model and a local fault detection model; the calculation formula is shown in formula (1):
[0012]
[0013] in, λ1 and λ2 represent the class accuracy of the distillation data, and are the distillation weights of the global and local learning characteristics in the class-accuracy distillation.
[0014] Step 2: Calculate the gradient learning weights
[0015] The gradient learning weights of the data classes during the local training of the federated learning model for the local fault detection model are calculated using the accuracy of the distilled data classes. The calculation formulas are shown in formulas (2) and (3):
[0016]
[0017]
[0018] in, Δ represents the data class precision factor and the interval adjustment offset, respectively. The precision factor is obtained by nonlinear mapping of the distillation class precision and its value range is (0, 1). Represents the number of data types; ω i ε i These represent the gradient learning weights corresponding to the i-th data class and the global learning round number when the i-th data class is added to the training, respectively.
[0019] Step 3: Calculate the gradient after adding noise and learn propagation.
[0020] Opacus is used to generate random Gaussian noise that matches the learning gradient scale, and gradient clipping and Gaussian noise addition are performed to calculate the gradient of the noisy data class. The obtained noisy data class gradient is then combined with the gradient learning weights and substituted into the learning propagation process of the local fault detection model.
[0021] In one embodiment of the present invention, step 2 is implemented as follows: based on the basic information of the participants in this round of federated training, including the data increment of the participants in this round and the total number of rounds of federated aggregation, the aggregation weight of the local fault detection model of each participant is calculated; the specific implementation is as follows:
[0022] Step 1: Calculate the learning depth of participant data
[0023] The participant data learning depth is calculated using the participant's data increment, the number of training participations, and the representation parameters that are continuously iterated throughout the training process; the calculation formula is shown in formula (4):
[0024]
[0025] in, These represent the number of newly added data samples in this round and the total number of cumulative data samples in the previous round, respectively. This represents the depth of data learning that is continuously iterated during the training process;
[0026] Step 2: Calculate the aggregate weights of the local fault detection model.
[0027] Based on the learning depth of the participants' data, data value range mapping and normalization are performed, and the aggregation weight of the local fault detection model is calculated; the calculation formulas are shown in formulas (5) and (6):
[0028]
[0029]
[0030] Where N represents the number of participants in this round of training; represents the learning depth of the i-th participant's data after completing the data range mapping and the learning depth of the i-th participant's data after completing the normalization, respectively, and the learning depth of the normalized data is used as the aggregation weight of the local fault detection model.
[0031] In one embodiment of the present invention, step 3 is implemented as follows: The local fault detection models of the participants in this round of federated training are uniformly collected to the central server. The model parameters are then weighted and calculated by combining the aggregated weights of the local fault detection models of the participants to obtain the global fault detection model parameters. The weighted calculation is as follows:
[0032] Step 1: Weighted calculation of global fault model parameters
[0033] Using the aggregated weights calculated from the local fault detection models of each participant, the parameters of the local fault detection models are weighted and summed respectively, and the mean of the calculated model parameters is used as the global fault detection model parameters; the calculation formula is shown in formula (7):
[0034]
[0035] Where, m i , Let represent the local fault detection model parameters and the global fault detection model parameters of the i-th participant, respectively.
[0036] Compared with existing technologies, this invention has the following advantages: The fault detection model proposed in this invention completes fault data preprocessing, local training of each participant, and global model aggregation, and implements a differential privacy mechanism for the local model based on Opacus. The effectiveness of the proposed model is confirmed through comparison with various existing federated incremental learning models; it accelerates the effective and high-precision convergence of the local model in incremental learning, realizes personalized training of the local model, effectively alleviates the model degradation problem caused by model aggregation through global aggregation, and accelerates the effective convergence of the global model, ensuring efficient convergence of the local model while improving the model's fault detection capability. Attached Figure Description
[0037] Figure 1 This forms the overall framework of the method of the present invention. Detailed Implementation
[0038] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.
[0039] This invention provides an industrial IoT fault detection method based on federated incremental learning. The overall framework of the method is shown in the attached figure. Figure 1 As shown, the process includes: performing knowledge distillation on the accuracy of different data classes calculated by the global and local models to extract the model's knowledge learning features; performing nonlinear mapping on the distilled class accuracy; and calculating the gradient learning weights for different data classes based on the current local data training level. Next, the data learning depth is abstracted from the training data of different participants to describe the proportion of incremental data relative to the already learned data for that participant. Then, the aggregation weights in this round of global aggregation are calculated based on the model's training participation, and aggregation training is completed on the central server using the aggregation weights of each participant. Finally, a global fault detection model is used to detect faults in industrial IoT tasks and determine whether a fault has occurred.
[0040] The purpose of this invention is to provide an industrial IoT fault detection method based on federated incremental learning. Focusing on the fault detection task in the industrial IoT, this invention emphasizes the convergence problem of federated training algorithms on local datasets and the effective aggregation problem on a global scale. Based on the idea of synergistic local learning and global aggregation, it provides technical support for improving the security and performance of artificial intelligence models in the industrial IoT. The specific steps are as follows:
[0041] Step 1: Calculate the gradient learning weights for the data classes and add noise to train the local fault detection model;
[0042] Step 2: Calculate the aggregate weights of the local fault detection model for participating parties;
[0043] Step 3: Combine the local fault detection models of different participants to obtain the global fault detection model.
[0044] The method of the present invention is specifically implemented as follows.
[0045] Step 1: Extract the numerical representations of the fault detection accuracy of the local fault detection model and the fault detection accuracy of the existing global fault detection model into ordered lists respectively. and Based on the characteristics of the fault detection task, an ordered list of detection accuracy is used. and Knowledge distillation is performed to calculate the gradient learning weights for fault data classes. Based on the obtained gradient learning weights for the data classes, the fault detection model is used to perform different gradient propagation learning on the data classes on the training set, and Gaussian noise is added during the learning process.
[0046] Step 1: Calculate the accuracy of distillation data.
[0047] Data distillation is performed to assess the classification accuracy of data sample classes using a global fault detection model and a local fault detection model. The calculation formula is shown in formula (1):
[0048]
[0049] in, λ1 and λ2 represent the class accuracy of the distillation data, and are the distillation weights of the global and local learning characteristics in the class-accuracy distillation.
[0050] Step 2: Calculate the gradient learning weights
[0051] The gradient learning weights of the data classes during the local training of the federated learning model for the local fault detection model are calculated using the accuracy of the distilled data classes. The calculation formulas are shown in formulas (2) and (3):
[0052]
[0053]
[0054] in, Δ represents the data class precision factor and the interval adjustment offset, respectively. The precision factor is obtained by nonlinear mapping of the distillation class precision and its value range is (0, 1). Represents the number of data types; ω i ε i These represent the gradient learning weights corresponding to the i-th data class and the global learning round number when the i-th data class is added to the training, respectively. The algorithm for calculating the gradient learning weights of the data classes is shown in Algorithm 1.
[0055]
[0056] Step 3: Calculate the gradient after adding noise and learn propagation.
[0057] Opacus is used to generate random Gaussian noise that matches the learning gradient scale, and gradient clipping and Gaussian noise addition are performed to calculate the gradient of the noisy data class. The obtained noisy data class gradient is then combined with the gradient learning weights and substituted into the learning propagation process of the local fault detection model.
[0058] Step 2: Based on the basic information of the participants in this round of federated training, including the data increment of each participant in this round and the total number of rounds of federated aggregation, calculate the aggregation weight of the local fault detection model of each participant.
[0059] Step 1: Calculate the learning depth of participant data
[0060] The participant data learning depth is calculated using the participant's data increment, the number of training participations, and the representation parameters iterated throughout the training process. The calculation formula is shown in formula (4):
[0061]
[0062] in, These represent the number of newly added data samples in this round and the total number of cumulative data samples in the previous round, respectively. This represents the depth of data learning that is continuously iterated during the training process.
[0063] Step 2: Calculate the aggregate weights of the local fault detection model.
[0064] Based on the learning depth of the participants' data, data value range mapping and normalization are performed to calculate the aggregation weight of the local fault detection model. The calculation formulas are shown in formulas (5) and (6):
[0065]
[0066]
[0067] Where N represents the number of participants in this round of training; Let represent the learning depth of the i-th participant's data after data range mapping and the learning depth of the i-th participant's data after normalization, respectively. The learning depth of the normalized data will be used as the aggregation weight of the local fault detection model. The specific algorithm is shown in Algorithm 2.
[0068]
[0069] Step 3: Collect all local models from the participants in this round of federated training to the central server. Combine the aggregated weights of the local fault models obtained from weight requirement 3, and perform weighted calculations on the model parameters to obtain the global fault detection model parameters. The specific calculation method is as follows:
[0070] Step 1: Weighted calculation of global fault model parameters
[0071] Using the aggregated weights calculated from the local models of each participant, the local model parameters are weighted and summed respectively. The mean of the calculated model parameters is then used as the global fault detection model parameter. The calculation formula is shown in formula (7):
[0072]
[0073] Where, m i , Let represent the local fault detection model parameters and the global fault detection model parameters of the i-th participant, respectively.
[0074] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.
Claims
1. An industrial IoT fault detection method based on federated incremental learning, characterized in that, Includes the following steps: Step 1: Calculate the gradient learning weights for the data classes and add noise to train the local fault detection model; Step 2: Calculate the aggregate weights of the local fault detection model for participating parties; Step 3: Weighted aggregation of local fault detection models from different participants to obtain a global fault detection model; Step 1 is implemented by abstracting the numerical representations of the fault detection accuracy of the local fault detection model and the fault detection accuracy of the existing global fault detection model into ordered lists respectively. and Based on the characteristics of the fault detection task, an ordered list is used. and Perform knowledge distillation and calculate the gradient learning weights for fault data classes; Based on the obtained data class gradient learning weights, the local fault detection model learns different gradients for the data classes on the training set, and Gaussian noise is added during the learning process. Step 2 is implemented as follows: Based on the basic information of the participants in this round of federated training, including the data increment of each participant in this round and the total number of rounds of federated aggregation, the aggregation weight of the local fault detection model of each participant is calculated; the specific implementation is as follows: Step 1: Calculate the learning depth of participant data The learning depth of the participant data is calculated using the data increment of the participants, the number of training participations, and the representation parameters that are continuously iterated throughout the training process. The calculation formula is shown in formula (4): （4） in, These represent the number of newly added data samples in this round and the total number of cumulative data samples in the previous round, respectively. This represents the depth of data learning that is continuously iterated during the training process; Step 2: Calculate the aggregate weights of the local fault detection model. Based on the learning depth of the participants' data, data value range mapping and normalization are performed, and the aggregation weight of the local fault detection model is calculated; the calculation formulas are shown in formulas (5) and (6): （5） （6） Where N represents the number of participants in this round of training; represents the learning depth of the i-th participant's data after completing the data range mapping and the learning depth of the i-th participant's data after completing the normalization, respectively, and the learning depth of the normalized data is used as the aggregation weight of the local fault detection model.
2. The industrial IoT fault detection method based on federated incremental learning according to claim 1, characterized in that, The gradient learning weights for fault data classes are calculated as follows: Step 1: Calculate the accuracy of distillation data. Data distillation was performed to assess the classification accuracy of data sample classes using a global fault detection model and a local fault detection model. The calculation formula is shown in formula (1): （1） in, Indicates the accuracy of distillation data. It is the distillation weight of global and local learning characteristics in precision distillation; Step 2: Calculate the gradient learning weights The gradient learning weights of the data classes during the local training of the local fault detection model in federated learning are calculated using the accuracy of the distilled data classes. The calculation formulas are shown in formulas (2) and (3): （2） （3） in, These represent the data class precision factor and the interval adjustment offset, respectively. The precision factor is obtained by nonlinear mapping of the distillation class precision and its value range is (0, 1). Represents the number of data types; These represent the gradient learning weights corresponding to the i-th data class and the global learning round number when the i-th data class is added to the training, respectively. Step 3: Calculate the gradient after adding noise and learn propagation. Opacus is used to generate random Gaussian noise that matches the learning gradient scale, and gradient clipping and Gaussian noise addition are performed to calculate the gradient of the data class after noise addition. The obtained gradient of the data class after noise addition is combined with the gradient learning weight and substituted into the learning propagation process of the local fault detection model.
3. The industrial IoT fault detection method based on federated incremental learning according to claim 1, characterized in that, Step 3 is implemented as follows: Local fault detection models from all participants in this round of federated training are collected and sent to the central server. The weights of the local fault detection models are then aggregated, and the model parameters are weighted to calculate the global fault detection model parameters. The weighted calculation is as follows: Step 1: Weighted calculation of global fault model parameters Using the aggregate weights calculated for the local fault detection models of each participant, the parameters of the local fault detection models are weighted and summed respectively, and the mean of the calculated model parameters is used as the global fault detection model parameters. The calculation formula is shown in formula (7): （7） in, Let represent the local fault detection model parameters and the global fault detection model parameters of the i-th participant, respectively, and N represent the number of participants in this round of training.