Method, system and equipment for improving robustness of industrial internet of things monitor and medium
By employing an adversarial training model using temporal convolutional neural networks and projective gradient descent, the robustness of industrial IoT monitors is improved, the problem of adversarial attack misdirection is solved, and the resistance to small-scale attacks is enhanced.
Patent Information
- Application Number
- CN202411006595.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-26
AI Technical Summary
Existing machine learning-based industrial IoT monitors are susceptible to micro-adversarial attacks, which can lead to a failure to detect anomalies in a timely manner and cause losses to industrial systems.
A temporal convolutional neural network is used for feature extraction, and an adversarial training model based on projective gradient descent is used to train the industrial IoT monitor to improve its robustness.
It improves the monitor's resistance to small-scale attacks, effectively solves the problem of misleading adversarial attacks, and enhances the monitor's robustness.
Smart Images

Figure CN119026667A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of Internet of Things, and specifically relates to a method, device, equipment and medium for improving the robustness of an industrial Internet of Things monitor. Background Art
[0002] The Internet of Things can support specific applications, enabling automation and intelligent behavior based on sensor signals attached to items such as vehicles, machinery, and household appliances. In the Industrial Internet of Things, the design of machine learning-based monitors can effectively predict the working conditions of complex industrial systems, thereby solving multiple difficult problems on industrial platforms.
[0003] However, existing machine learning-based industrial IoT monitors are easily misled by tiny adversarial attacks. If the model cannot make accurate judgments on tiny signals, the interfered monitor may not be able to make timely judgments on the anomalies of the IoT system, which will lead to wrong predictions and cause significant losses to the industrial system. Therefore, it is necessary to improve the resistance of existing machine learning-based industrial IoT monitors to tiny disturbances and improve the robustness of industrial IoT monitors. Summary of the invention
[0004] The purpose of the present invention is to provide a method for improving the robustness of an industrial Internet of Things monitor, which can effectively solve the problem that the existing monitor is easily misled by adversarial attacks and improve the robustness of the monitor against small attacks;
[0005] The second object of the present invention is to provide an industrial Internet of Things monitor robustness improvement system for implementing the above method;
[0006] The third object of the present invention is to provide a terminal device, which can implement a method for improving the robustness of an industrial Internet of Things monitor when executing its own program;
[0007] The fourth object of the present invention is to provide a computer-readable storage medium for storing a corresponding computer program for a method for improving the robustness of an industrial Internet of Things monitor.
[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows:
[0009] A method for improving the robustness of an industrial Internet of Things monitor comprises the following steps:
[0010] S1, collect time series data from industrial IoT monitors;
[0011] S2. Use a temporal convolutional neural network to extract features from the collected time series data and convert it into a vector that can be used for abnormality identification;
[0012] S3, establishing an adversarial training model based on projected gradient descent, and using the feature vector extracted in step S2 to train the adversarial training model;
[0013] S4. Apply the trained adversarial training model to the industrial Internet of Things monitor to improve the robustness of the industrial Internet of Things monitor.
[0014] As a limitation, the step of training the adversarial training model in step S3 includes:
[0015] The worst-case perturbation is obtained by maximizing the perturbation of the loss function through the internal process;
[0016] The model is updated to minimize the loss function of the model in the worst case.
[0017] As a further limitation, when the attack value of the worst-case perturbation on the model exceeds 1 in any dimension, the attack value will remain at 1.
[0018] As a second limitation, the process formula for training the adversarial training model in step S3 is:
[0019]
[0020] Where x is the training data, y is the system status, and η is the signal generated by the projected gradient descent algorithm. represents the loss function of model training, θ is the set of model parameters, For the data set, is the expected error.
[0021] As a further limitation, the signal generated by the projected gradient descent algorithm is:
[0022]
[0023] In the formula, is the gradient function under the training data x, f is the maximum value of the loss function, ε is the ∞-norm constraint, and sign(·) is the sign function.
[0024] As a further limitation, the worst-case disturbance is obtained by:
[0025]
[0026] In the formula, is the search area around the training data x, Clip ε (·) is used to limit the maximum value to ε.
[0027] A robustness improvement system for an industrial Internet of Things monitor, used to implement the above-mentioned robustness improvement method for an industrial Internet of Things monitor, the system comprising: a data acquisition module, a feature extraction module, a model construction module, and a model application module;
[0028] The data acquisition module collects the time series data from the industrial Internet of Things monitor and inputs the collected time series data into the feature extraction module;
[0029] The feature extraction module uses a temporal convolutional neural network to extract features from the time series data collected by the data acquisition module and inputs the obtained vectors into the model construction module;
[0030] The model building module builds an adversarial training model based on projected gradient descent, uses the vectors obtained by the feature extraction module to train the model, and then inputs the trained model into the model application module;
[0031] The model application module applies the adversarial training model trained by the model construction module to the industrial Internet of Things monitor.
[0032] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for improving the robustness of an industrial Internet of Things monitor when executing the computer program.
[0033] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement the above-mentioned method for improving the robustness of an industrial Internet of Things monitor.
[0034] Due to the adoption of the above technical solution, the present invention has the following technical advances compared with the prior art:
[0035] (1) The present invention uses a time convolutional neural network to extract features from the time series data of the industrial Internet of Things monitor, which can effectively extract hidden time information from the original time series data;
[0036] (2) The present invention improves the robustness of the industrial Internet of Things monitor against small attacks through an adversarial training method based on projected gradient descent, and effectively solves the problem that the existing industrial Internet of Things monitor is easily misled by adversarial attacks.
[0037] The present invention belongs to the technical field of Internet of Things, can effectively solve the problem that the existing monitor is easily misled by adversarial attacks, and improves the robustness of the monitor against small attacks. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.
[0039] In the attached picture:
[0040] Figure 1 Processing flow chart of Example 1 of the present invention;
[0041] Figure 2 This is a structural block diagram of Example 2 of the present invention. DETAILED DESCRIPTION
[0042] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0043] Example 1 A method for improving the robustness of an industrial Internet of Things monitor
[0044] It should be noted that the evaluation data set used to verify the robustness improvement effect of the industrial Internet of Things monitor in the present invention is the hydraulic system sensor measurement data obtained from the UC Irvine machine learning library, which is used as a general benchmark for evaluating the performance of the latest industrial Internet of Things monitor. The data set is constructed by recording the values of 17 sensors in 2205 cycles, including 6 pressure sensors, 2 flow sensors, 5 temperature sensors, 1 power sensor, 1 cooling efficiency sensor, 1 cooling power sensor, 1 efficiency factor sensor, 1 motor power sensor and 1 vibration sensor. Since the sampling frequency of different sensors varies from 1Hz to 100Hz, the total size of the sensor values is also different. In addition, the data set contains 4 fault states, which represent cooling power fault (Cooler), valve fault (Valve), pump fault (Pump) and accumulator fault (Accumulator).
[0045] like Figure 1 As shown, this embodiment includes the following steps performed in sequence:
[0046] S100, collects time series data from 17 industrial IoT monitors in the hydraulic system.
[0047] Before using the time convolutional neural network for feature extraction, it is necessary to first collect the time series data of the industrial Internet of Things monitor. The data set of the hydraulic system used in this embodiment contains 17 monitors, and the time series data of all of them need to be collected.
[0048] S200, use a time convolutional neural network to extract features from all collected time series data and convert them into vectors that can be used for abnormality identification.
[0049] Temporal Convolutional Neural Network (TCN) is a deep learning model for processing sequence data. It is particularly suitable for tasks such as time series prediction and language modeling. It can process sequence data by using causal convolution and dilated convolution while maintaining the advantages of convolutional neural networks. The TCN structure includes four core components: causal convolution, dilated convolution, residual connection and fully connected layer:
[0050] Causal convolution ensures that the output of the current time step depends only on the input of the current time step and before, and does not depend on future inputs. This can be achieved by aligning the center of the convolution kernel to the current time step. Given an input sequence a = [a1, a2, …, a t ] and convolution kernel w=[w1,w2,…,w k ], the output u(t) of the causal convolution can be expressed as: Where u(t) is the output at time step t, a(t) is the input sequence, w(k) is the convolution kernel, and K is the size of the convolution kernel.
[0051] The dilated convolution expands the receptive field by inserting holes, or gaps, between the elements in the convolution kernel to capture longer dependencies. The parameter of the dilated convolution is the dilation factor d. The output of the dilated convolution can be expressed as: Where d is the dilation rate, which represents the spacing between the elements of the convolution kernel.
[0052] Residual connections help train deep networks by directly adding the input to the output, thereby alleviating the vanishing gradient problem. The formula for residual connections can be expressed as: O(t) = σ(F(a) + a(t)) where F(a) represents the output after passing through the convolutional layer, a(t) is the input sequence, and σ(·) is the activation function.
[0053] The last layer of TCN is usually a fully connected layer, which is used to map the output of the convolutional layer to the required output dimension. The fully connected layer can be expressed as: y = W h + b, where W is the weight matrix, h is the output from the convolutional layer, and b is the bias vector.
[0054] Combining the above factors, the output of a typical TCN layer can be expressed as:
[0055]
[0056] A multi-layer TCN is composed of multiple stacked TCN layers, each of which performs causal dilated convolution and contains residual connections. The input of each layer is the output of the previous layer. By stacking layer by layer, TCN can capture dependencies over a longer time range. TCN usually consists of multiple stacked causal convolution layers, dilated convolution layers, and residual blocks. With this structure, TCN can efficiently capture long-term dependencies and maintain the stability and trainability of the model.
[0057] S300, establishing an adversarial training model based on projected gradient descent, and using feature extracted vectors to train the adversarial training model.
[0058] The training steps include:
[0059] The worst-case perturbation is obtained by maximizing the perturbation of the loss function through the internal process;
[0060] The model is updated to minimize the loss function of the model in the worst case.
[0061] During the model update process, when the attack value of any dimension exceeds 1, the attack value will remain at 1.
[0062] The above process formula based on training the adversarial training model is:
[0063]
[0064] Where x is the training data, y is the system status, and η is the signal generated by the projected gradient descent algorithm. represents the loss function of model training, θ is the set of model parameters, For the data set, is the expected error.
[0065]
[0066] In the formula, is the gradient function under the training data x, f is the maximum value of the loss function, ε is the ∞-norm constraint, and sign(·) is the sign function.
[0067] The worst-case disturbance is obtained as follows:
[0068]
[0069] In the formula, is the search area around the training data x, Clip ε (·) is used to limit the maximum value to ε.
[0070] The perturbation attack of projected gradient descent is considered to be the most effective attack in a bounded region with a given ∞-norm around x.
[0071] The overall adversarial training process is as follows:
[0072] Build TCN network;
[0073] Training speed ilr, drop rate ds, decay rate dr.
[0074] Repeat epoch+1
[0075] Set learning rate lr=min(ilr,ilr*dr epoch / ds )
[0076] Repeat batch+1
[0077] Get training data x and system status y
[0078] x'=x
[0079] Repeat inner procedure n+1
[0080]
[0081] x'=x +step
[0082] attack=x'-x
[0083] attack=Clip(attack,-ε,ε)
[0084] x'=Clip(x',0,1)
[0085] calculate
[0086] Get updated gradients
[0087]
[0088] End
[0089] S400, apply the trained adversarial training model to the industrial Internet of Things monitor to improve the robustness of the industrial Internet of Things monitor.
[0090] As shown in Table 1, the performance of the present invention in comparison with other benchmark methods in cooling power failure (Cooler), valve failure (Valve), pump failure (Pump) and accumulator failure (Accumulator) is shown. It can be seen from the table that the application of the adversarial training model trained in this embodiment can improve the robustness of the industrial Internet of Things monitor.
[0091] Table 1 Results comparison
[0092]
[0093]
[0094] Example 2: A robustness improvement system for industrial Internet of Things monitors
[0095] like Figure 2 As shown, this embodiment is used to implement embodiment 1, and this embodiment includes: a data acquisition module, a feature extraction module, a model construction module, and a model application module.
[0096] The data acquisition module collects the time series data from the industrial Internet of Things monitor and inputs the collected time series data into the feature extraction module.
[0097] The feature extraction module uses a temporal convolutional neural network to extract features from the time series data collected by the data acquisition module and inputs the obtained vectors into the model construction module.
[0098] The model building module builds an adversarial training model based on projected gradient descent, uses the vectors obtained by the feature extraction module to train the model, and then inputs the trained model into the model application module.
[0099] The model application module applies the adversarial training model trained by the model construction module to the industrial Internet of Things monitor.
[0100] Embodiment 3 A terminal device
[0101] This embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it is used to implement a method for improving the robustness of an industrial Internet of Things monitor in Embodiment 1.
[0102] Embodiment 4 A computer readable storage medium
[0103] The computer-readable storage medium in this embodiment stores a computer program, and when the computer program is executed by the processor, it is used to implement a method for improving the robustness of an industrial Internet of Things monitor in Example 1.
[0104] Among them, the computer-readable storage medium can be a computer storage medium or a communication medium. The communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The computer storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. For example, a computer-readable storage medium is coupled to a processor so that the processor can read information from the computer-readable storage medium and write information to the computer-readable storage medium. Of course, the computer-readable storage medium can also be a component of the processor. The processor and the computer-readable storage medium can be located in an application-specific integrated circuit (ASIC). In addition, the ASIC can be located in a user device. Of course, the processor and the computer-readable storage medium can also exist in a communication device as discrete components. Specifically, the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk, etc. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.
Claims
1. A method for improving the robustness of an industrial Internet of Things monitor, characterized in that: The following steps are involved: S1, collect time series data from industrial IoT monitors; S2. Use a temporal convolutional neural network to extract features from the collected time series data and convert it into a vector that can be used for abnormality identification; S3, establishing an adversarial training model based on projected gradient descent, and using the feature vector extracted in step S2 to train the adversarial training model; S4. Apply the trained adversarial training model to the industrial Internet of Things monitor to improve the robustness of the industrial Internet of Things monitor.
2. The method for improving the robustness of an industrial Internet of Things monitor according to claim 1, characterized in that: The step of training the adversarial training model in step S3 includes: The worst-case perturbation is obtained by maximizing the perturbation of the loss function through the internal process; The model is updated to minimize the loss function of the model in the worst case.
3. The method for improving the robustness of an industrial Internet of Things monitor according to claim 2, characterized in that: When the attack value of the worst-case perturbation on the model exceeds 1 in any dimension, the attack value will remain at 1.
4. The method for improving the robustness of an industrial Internet of Things monitor according to claim 1, characterized in that: The process formula for training the adversarial training model in step S3 is: Where x is the training data, y is the system status, and η is the signal generated by the projected gradient descent algorithm. represents the loss function of model training, θ is the set of model parameters, For the data set, is the expected error.
5. The method for improving the robustness of an industrial Internet of Things monitor according to claim 4 is characterized in that: The signal generated by the projected gradient descent algorithm is: In the formula, is the gradient function under the training data x, f is the maximum value of the loss function, ε is the ∞-norm constraint, and sign(·) is the sign function.
6. The method for improving the robustness of an industrial Internet of Things monitor according to claim 5, characterized in that: The worst-case disturbance is obtained by: In the formula, is the search area around the training data x, Clip ε (·) is used to limit the maximum value to ε.
7. A system for improving the robustness of an industrial Internet of Things monitor, used to implement a method for improving the robustness of an industrial Internet of Things monitor as described in any one of claims 1 to 6, characterized in that: The system includes: a data acquisition module, a feature extraction module, a model construction module, and a model application module; The data acquisition module collects the time series data from the industrial Internet of Things monitor and inputs the collected time series data into the feature extraction module; The feature extraction module uses a temporal convolutional neural network to extract features from the time series data collected by the data acquisition module and inputs the obtained vectors into the model construction module; The model building module builds an adversarial training model based on projected gradient descent, uses the vectors obtained by the feature extraction module to train the model, and then inputs the trained model into the model application module; The model application module applies the adversarial training model trained by the model construction module to the industrial Internet of Things monitor.
8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for improving the robustness of an industrial Internet of Things monitor according to any one of claims 1 to 6 is implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement the method for improving the robustness of an industrial Internet of Things monitor as described in any one of claims 1 to 6.
