Industrial control time series data anomaly detection method

Abstract

The invention relates to the technical field of industrial control security and the technical field of computers, and discloses an industrial control time series data anomaly detection method, which comprises the following steps: effectively integrating a novel auto-encoder, a generative adversarial network and a self-supervised comparative learning network, and modeling long-term dependence by using an improved long-short-term memory network, extracting short-term features by combining causal convolution and expansion convolution of a time sequence convolutional network, and accurately positioning key features in a time sequence by adopting a self-attention mechanism in the last layer. A dynamic double-optimization strategy of a generative adversarial network and a self-supervised comparative learning network is introduced to optimize parameters of the novel auto-encoder, various types of pollution data can be effectively identified, the generative adversarial network can further learn time sequence features and generate diversified abnormal sequences, the detection precision is remarkably improved, and the method is suitable for large-scale popularization and application. Therefore, the accuracy of anomaly detection is improved, and the adaptability to a complex industrial control environment is enhanced.

Description

Technical Field

[0001] The present application relates to the fields of industrial control safety technology and computer technology, and in particular to a method for detecting anomalies in industrial control time series data. Background Art

[0002] With the widespread application of Industrial Control System (ICS) technology, the network traffic time series data, sensor status and other key operating parameter time series data generated in the industrial control environment have become an important basis for monitoring and diagnosis. These data not only reflect the operating status of the equipment, but also carry a lot of valuable information about the production process, environmental conditions and operating behavior, providing solid data support for real-time monitoring, fault prediction and system optimization.

[0003] However, due to the complexity and diversity of the industrial environment, these time series data are often affected by various pollution factors such as sensor noise, periodic fluctuations, short-term data loss, fluctuations caused by maintenance operations, etc. These influences not only increase the difficulty of data analysis, but also put forward higher requirements for accuracy and reliability. On this basis, it also affects the detection accuracy of abnormal data. Summary of the invention

[0004] In view of this, the present application provides an industrial control time series data anomaly detection method, which achieves the purpose of effectively distinguishing various data pollution and real anomalies by dynamically adjusting the parameter values ​​involved in the time series feature learning part, and effectively solves the problem of decreased accuracy and weak adaptability faced by most time series detection methods due to multi-type data pollution.

[0005] According to one aspect of the present application, a method for detecting anomalies in industrial control time series data is provided, which is applied to an enhanced neural network framework, wherein the enhanced neural network framework includes a novel autoencoder and an anomaly detection module, wherein the novel autoencoder includes a novel encoder and a novel decoder with a reciprocal structure, wherein the novel encoder includes a long short-term memory network, a temporal convolutional network, and a self-attention mechanism module; the method includes: The industrial control time series data to be detected is obtained and input into the enhanced neural network framework, so as to implement data anomaly detection by executing the following steps through the enhanced neural network framework: By using each gate in the long short-term memory network with a gating mechanism, based on the dynamic adjustment factor of the activation value, cross-validate the optimal activation value corresponding to each gate of the industrial control time series data to be tested under various data pollution conditions, and adjust each gate through the optimal activation value to output the long-term dependency feature; After dynamically adjusting the contribution of the residual part of the temporal convolutional network according to the long-term dependency feature based on the residual connection scaling factor using the temporal convolutional network, the short-term feature is output; The self-attention mechanism module uses a self-attention mechanism to locate key features in the short-term features and output them, wherein the output key features are characterized by reconstructed time series and low-dimensional feature representation; The novel decoder reversely decodes the key features to obtain the industrial control timing data to be compared; The anomaly detection module obtains an anomaly detection result by comparing the industrial control timing data to be detected with the industrial control timing data to be compared.

[0006] Through the above technical solution, the present application provides an industrial control time series data anomaly detection method, which effectively integrates the new autoencoder, generative adversarial network and self-supervised contrastive learning network to form an efficient and robust time series anomaly detection model, and through the dynamic contrast loss mechanism, effectively solves the problems of weak adaptability, single application scenario and reduced accuracy in the face of multi-type data pollution in time series anomaly detection in industrial environments.

[0007] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of an enhanced neural network framework for an industrial control time series data anomaly detection method provided by an embodiment of the present application is shown; Figure 2 A schematic diagram of a novel encoder in a novel autoencoder provided in an embodiment of the present application is shown; Figure 3 A schematic diagram of a temporal convolutional network provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0009] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0010] In this embodiment, a method for detecting anomalies in industrial control time series data is provided, which is applied to an enhanced neural network framework. The enhanced neural network framework includes a novel autoencoder and an anomaly detection module. The novel autoencoder is as follows: Figure 1As shown, it includes a novel encoder 101 and a novel decoder 102 having a reciprocal structure; the novel encoder 101 is as shown in Figure 2 The diagram includes a long short-term memory network 1011, a temporal convolutional network 1012, and a self-attention mechanism module 1013; the method includes: The industrial control time series data to be detected is obtained and input into the enhanced neural network framework, so as to implement data anomaly detection by executing the following steps through the enhanced neural network framework: Step 101, using each gate in the long short-term memory network with a gating mechanism, based on the dynamic adjustment factor of the activation value, cross-validate the optimal activation value corresponding to each gate of the industrial control time series data to be detected under various data pollution conditions, and adjust each gate through the optimal activation value to output the long-term dependency feature.

[0011] Step 102, using each gate in the long short-term memory network with a gating mechanism added, based on the dynamic adjustment factor of the activation value, cross-validate the optimal activation value corresponding to each gate of the industrial control time series data to be detected under various data pollution conditions, and adjust each gate through the optimal activation value to output the long-term dependency feature.

[0012] Step 103, using the temporal convolutional network based on the residual connection scaling factor, dynamically adjusting the contribution of the residual part of the temporal convolutional network according to the long-term dependency feature, and outputting the short-term feature.

[0013] Step 104: the self-attention mechanism module uses the self-attention mechanism to locate key features in the short-term features and outputs them, wherein the output key features are represented by reconstructed time series and low-dimensional feature representation.

[0014] Step 105: the novel decoder reversely decodes the key features to obtain the industrial control timing data to be compared.

[0015] Step 106: the anomaly detection module obtains an anomaly detection result by comparing the industrial control timing data to be detected with the industrial control timing data to be compared.

[0016] In the above embodiment of the present application, the industrial control timing data to be detected is obtained and input into the enhanced neural network framework, so as to perform the following steps through the enhanced neural network framework to realize data anomaly detection: using each gate in the long short-term memory network with a gating mechanism added, based on the dynamic adjustment factor of the activation value, cross-validate the optimal activation value corresponding to each gate of the industrial control timing data to be detected under various data pollution conditions, and output the long-term dependency feature after adjusting each gate by the optimal activation value; using the time series convolution network based on the residual connection scaling factor, dynamically adjust the contribution of the residual part of the time series convolution network according to the long-term dependency feature, and output the short-term feature, that is, receive the long-term dependency feature output by the long short-term memory network through the time series convolution network, combine causal convolution and dilated convolution, and construct the short-term feature extraction part of the new encoder to ensure that the short-term feature relationship is captured while avoiding future information leakage. The self-attention mechanism module uses the self-attention mechanism to locate and output key features in the short-term features. By introducing the self-attention mechanism, the contaminated features are dynamically identified and their weights are adjusted, thereby reducing the influence of noise and focusing on the key time segments that truly have abnormal features.

[0017] In particular, the output of the self-attention mechanism module can be , is the feature dimension after attention weight adjustment. Finally, the new encoder combines the features of short-term, long-term dependency and self-attention mechanism, and outputs the reconstructed time series and its low-dimensional feature representation as Then, the new decoder reversely decodes the key features to obtain the industrial control timing data to be compared; the anomaly detection module obtains the anomaly detection result by comparing the industrial control timing data to be detected with the industrial control timing data to be compared.

[0018] Industrial Control Time Series Data refers to a series of data points recorded over time in industrial automation and control systems. These data are usually used to monitor, analyze and optimize industrial processes. Specifically, industrial control time series data includes: 1. Sensor data, including: Temperature sensors, which record temperature changes of equipment or environment. Pressure sensors, which monitor pressure changes in pipes, containers or systems. Flow sensors, which measure the flow of liquids or gases. Humidity sensors, which record humidity levels in the air.

[0019] 2. Equipment status data, including: Motor operating status, such as speed, current, voltage, etc. Pump operating status, including flow, pressure, power, etc. Machine vibration data, used to monitor the mechanical health of the equipment.

[0020] 3. Process control data, including: controlling valve opening, recording valve position or opening changes. Liquid level control, such as liquid level changes in storage tanks or containers. Parameters in chemical reaction processes, such as temperature, pressure, reactant concentration, etc.

[0021] 4. Energy management data, including: Power consumption, recording power usage in different time periods. Gas or steam consumption, monitoring the flow and consumption rate of gas or steam.

[0022] 5. Environmental monitoring data, including: air quality monitoring, such as PM2.5, PM10, carbon dioxide concentration, etc. Water quality monitoring, including pH value, dissolved oxygen, turbidity, etc.

[0023] 6. Production data, including: production line output rate, record the number of products produced per unit time. Product quality data, such as size, weight, defect rate, etc.

[0024] 7. Security system data, including: Fire alarm system, recording the time, location and status of fire alarms. Security access control system, monitoring the switch status of access control and personnel entry and exit records.

[0025] Industrial control timing data can be collected through data acquisition systems (such as SCADA systems, PLCs, sensor networks, etc.) and stored in databases for subsequent analysis. By analyzing these industrial control timing data, engineers and operators can understand the operating status of industrial processes, identify potential problems, optimize production efficiency, and ensure the safety and reliability of the system.

[0026] The new encoder 101 can be constructed by combining long-term dependency modeling, short-term feature extraction and self-attention mechanism. The new encoder 101 can effectively distinguish various data pollution and real anomalies, and enhance the accuracy and robustness of abnormal sequence detection. Figure 2 shown.

[0027] Figure 2 The leftmost side represents the input industrial control timing data to be detected x0...x n, the time step (i.e., moment) t ranges from 0 to n, and the input industrial control time series data to be detected can be represented by (B, T, F), where B represents the batch size (BatchSize), T represents the time step (Time Steps), and F represents the feature dimension (Feature Dimension). Next, the long short-term memory network (LSTM) 1011 is used to process sequence data (industrial control time series data to be detected), has a memory and forgetting mechanism, can effectively capture the long-term dependency in the time series, and output long-term dependency features, followed by the temporal convolutional network (Temporal Convolutional Network, TCN) 1012, which is used to improve the deep learning performance of the network. It is a deep learning model specially designed for processing time series data and can effectively capture the features in the time series. It is suitable for time series prediction and classification, especially for time series prediction and sequence classification tasks. Through its unique convolution operation, it can accurately predict future data points or classification sequences, and introduce skip connections to solve the gradient disappearance problem in deep networks, enhance the network's expression ability and training stability. In particular, the temporal convolutional network 1012 also includes three residual submodules, namely the first residual submodule 10121, the second residual submodule 10122 and the third residual submodule 10123, which have similar functions, namely, enhancing the expressive power of the network. More specifically, TCN captures local features in time series through convolution operations, which are crucial for understanding the dynamic behavior of time series. By stacking multiple layers of convolution, TCN can capture more complex time dependencies, thereby improving the expressive power and prediction accuracy of the model and outputting short-term features. Next is the self-attention mechanism module 1013, which is used to focus on key time segments that truly have abnormal features. Finally, through the fully connected layer, the key features x0...x are output. n , the key features of the output can be represented by (B, T, 32). In particular, the self-attention mechanism is a special form of the attention mechanism, which is mainly used to process the relationship between elements in sequence data or collection data. In the self-attention mechanism, each element calculates the correlation with all other elements and assigns weights according to this correlation.

[0028] The gating mechanism of the long short-term memory network (LSTM) is its core design, which is mainly used to control the retention, update and output of information. Specifically, LSTM can include the following three key gating mechanisms: 1. Forget Gate, which is responsible for deciding which historical information to discard from the cell state. The sigmoid function generates a value between 0 and 1, where 0 means complete forgetfulness and 1 means complete retention.

[0029] 2. Input Gate (or update gate) is used to filter the new information in the current input that needs to be stored in the cell state. It consists of two operations: determining which information to update through the sigmoid function, and generating candidate memory content through the tanh function.

[0030] 3. Output Gate: controls how much information in the cell state needs to be exposed to the hidden state at the current moment, thereby affecting the final output.

[0031] In particular, the forget gate and the input gate work together to update the cell state and form a dynamic control of long-term memory. This gating mechanism enables LSTM to effectively solve the gradient vanishing problem of traditional RNN (Recurrent Neural Network) and capture long-term dependencies through cell states.

[0032] To this end, the long short-term memory network is improved and a dynamic adjustment factor of the activation value is added so that when the long short-term memory network uses the gating mechanism, it can cross-validate the optimal activation value corresponding to each gate in the gating mechanism for the industrial control timing data to be tested under various data pollution conditions. After the long short-term memory network adjusts each gate through the optimal activation value, it outputs long-term dependent features, which can enable the long short-term memory network to better learn features.

[0033] In particular, when detecting whether there are anomalies in industrial control time series data, data pollution situations may include the following: 1. Sensor noise: During the measurement process, the sensor may be subject to various random interferences, such as electromagnetic interference, temperature fluctuations, etc., which may cause deviations or fluctuations in the measured data. This noise can appear as random jitter or high-frequency fluctuations on the data points, affecting the accuracy and reliability of the data.

[0034] 2. Cyclic fluctuations: There are often cyclical factors in the industrial production process, such as periodic vibration of equipment operation, periodic start and stop of production lines, etc. These cyclical fluctuations may mask the real abnormal signals, making abnormality detection more difficult.

[0035] 3. Short-term data loss: Due to communication failure, sensor failure or data transmission problems, some data may be lost in a short period of time. Data loss may cause the time series data to be broken or discontinuous, affecting the integrity of the data and the accuracy of the analysis results.

[0036] 4. Fluctuations caused by maintenance operations. In industrial control systems, equipment maintenance and overhaul are inevitable. Maintenance operations may cause temporary changes in equipment status, thereby introducing abnormal fluctuations or mutations in time series data.

[0037] Optionally, in step 101, each gate in the long short-term memory network with a gating mechanism added thereto is used to cross-validate the optimal activation value corresponding to each gate of the industrial control time series data to be detected under various data pollution conditions based on the dynamic adjustment factor of the activation value, and before outputting the long-term dependency feature after adjusting each gate by the optimal activation value, the method further includes: Step 107 , determining the data scenario to which the industrial control time series data to be detected belongs, wherein the data scenario includes a scenario with high data real-time demand, a scenario with high data pollution, and a scenario with insufficient data.

[0038] Step 108, constructing a total loss function of the self-supervised contrastive learning network and the generative adversarial network, and optimizing the parameters of the new autoencoder by dynamically adjusting the loss dynamic adjustment factors corresponding to the self-supervised contrastive learning network and the generative adversarial network in the constructed total loss function, wherein the total loss function is: , is the total loss function, is the contrastive loss of the self-supervised contrastive learning network, is the adversarial loss of the generative adversarial network, is the loss dynamic adjustment factor of the self-supervised contrastive learning network, Dynamic adjustment factor for the loss of the generative adversarial network.

[0039] Step 109, when the data scenario to which the industrial control time series data to be detected belongs is a scenario with high real-time data requirements, the loss dynamic adjustment factor of the generative adversarial network is set to a minimum preset threshold close to 0; when the data scenario to which the industrial control time series data to be detected belongs is a scenario with high data pollution, the loss dynamic adjustment factor of the self-supervised contrastive learning network is set to be smaller than the loss dynamic adjustment factor of the generative adversarial network; and when the data scenario to which the industrial control time series data to be detected belongs is a scenario with insufficient data, the loss dynamic adjustment factor of the generative adversarial network is set to 0.

[0040] In the above embodiments of the present application, in order to enhance the learning and discrimination capabilities of multi-type data pollution and abnormal patterns, an optimization mechanism with dynamic adaptability is designed for different data environments and detection requirements (data high real-time demand scenarios or data high pollution scenarios or data insufficient scenarios). By dynamically adjusting the weight parameters of contrast loss and adversarial loss, combined with generative adversarial networks and self-supervised contrastive learning networks, time series anomaly detection can be achieved in real and complex industrial control scenarios. By combining a new autoencoder, a self-supervised contrastive learning network and a generative adversarial network to perform time series anomaly detection, it can well adapt to the problems of multi-type data pollution and anomaly identification in industrial control environments, and effectively integrate the new autoencoder, generative adversarial network and self-supervised contrastive learning network to form an efficient and robust time series anomaly detection model.

[0041] Autoencoders originate from the fields of artificial intelligence and machine learning, and in particular, they play an important role in the study of neural networks. An autoencoder is a neural network used for unsupervised learning, whose main goal is to learn a compressed representation of input data (i.e., encoding) and to be able to reconstruct the original data from this compressed representation (i.e., decoding). An autoencoder consists of two parts: an encoder and a decoder. The encoder converts the input data into a representation in a latent space (or encoding), while the decoder attempts to convert this representation back into the original data space to reconstruct the input data as accurately as possible. Through training, autoencoders are able to learn an effective representation of the data, which can be used for various tasks such as data dimensionality reduction, feature extraction, denoising, and anomaly detection.

[0042] In the above embodiment of the present application, the "new autoencoder" is reconstructed by combining the long short-term memory network, the time series convolutional network and the self-attention mechanism. At the same time, the activation value dynamic adjustment factor and the residual connection scaling factor are added for real-time dynamic adjustment, which enables the constructed new autoencoder to better learn the characteristics of industrial control time series data, thereby improving the subsequent anomaly detection accuracy of industrial control time series data.

[0043] Self-supervised contrastive learning network, self-supervised learning is a machine learning method that uses the intrinsic information of the data itself as a supervisory signal to train the model without the need for manually annotated labels. Contrastive learning is a technique in self-supervised learning that learns the representation of data by comparing similar and dissimilar samples. In contrastive learning, the model is trained to distinguish between positive samples (similar) and negative samples (dissimilar). Self-supervised contrastive learning network combines the ideas of self-supervised learning and contrastive learning, using unlabeled data to learn effective representations of data by comparing positive and negative samples.

[0044] In Generative Adversarial Networks (GANs), generative models are a type of machine learning model that can generate new data similar to training data. Adversarial training is a training method that involves two models (generators and discriminators) that compete and evolve together. Specifically, the generator attempts to generate realistic fake data, while the discriminator attempts to distinguish between real data and fake data. Generative Adversarial Networks (GANs) are neural network architectures composed of generators and discriminators that use adversarial training to generate new data that is similar to the distribution of real data.

[0045] In particular, Generative Adversarial Networks (GANs) consist of two main parts: a generator and a discriminator. The generator tries to generate realistic data samples, while the discriminator tries to distinguish between real data and generated data. Through this adversarial training, the generator can gradually learn to generate more and more realistic data. Self-Supervised Contrastive Learning Networks, that is, networks that use self-supervised learning and contrastive learning techniques. Self-supervised learning is a learning method that does not require manually labeled data. It uses the intrinsic information of the data itself to generate supervision signals. Contrastive learning is a self-supervised learning method that learns the representation of data by comparing similar and dissimilar samples. In such a network, some form of contrastive loss function is usually used to optimize the model so that it can better distinguish between similar and dissimilar samples.

[0046] In the above embodiment of the present application, specifically, in the design of the total loss function, a loss dynamic adjustment factor is constructed. and , used to adjust the contrast loss respectively and combat loss The overall (total) loss function is defined as follows: .

[0047] in, and The value of is adjusted dynamically according to the input data characteristics and scene requirements. Used to optimize feature representation capabilities and enhance the model's ability to distinguish complex abnormal patterns. Generative adversarial networks are used to improve the model's robustness to data contamination and its ability to capture real anomalies.

[0048] Furthermore, for scenarios with high data real-time requirements, that is, when the detection results need to be output quickly, we can reduce the reliance on generative adversarial training (which takes longer but has higher accuracy), and only retain the self-supervised contrastive learning network part, that is, set , allowing the enhanced neural network framework to fully focus on , thereby reducing the computational complexity and meeting high real-time detection requirements.

[0049] For scenarios with high data pollution, that is, when there is a lot of polluted data and the detection accuracy needs to be further improved without paying attention to the time-consuming issue, you can set Improve The optimization weights of are strengthened to enhance the generative adversarial training, thereby enhancing the robustness to data pollution.

[0050] For scenarios with insufficient data, we can fully adopt the self-supervised contrastive learning network, that is, we completely rely on contrastive loss for optimization, corresponding to Set to 0.

[0051] Regarding the generative adversarial training network, the generative adversarial training network includes a generator and a discriminator, in which the generator shares a structure with the new encoder. This design not only simplifies the structure of the generator, but also enables the generator to better capture long-term and short-term dependencies in the time series through shared feature extraction capabilities, and generate abnormal samples that are more in line with the temporal law. At the same time, a pure convolutional neural network is selected as the discriminator. The discriminator uses a multi-layer convolutional neural network to perform hierarchical temporal dependency detection to detect the characteristics of short-term dependencies, and uses the loss function of the Wasserstein generative adversarial network (WGAN) for optimization. Different from the structure of the generator using a long short-term memory network combined with a temporal convolutional neural network, the discriminator uses a multi-layer convolutional neural network to focus more on extracting short-term temporal dependencies, avoiding the generator and the discriminator from sharing too much structural information. The generative adversarial network part learns the time series characteristics of various data pollution and abnormal patterns, generates different types of abnormal samples through the generator, reduces overfitting of a certain type of features, and improves the robustness of anomaly detection; at the same time, it can learn the time series change rules caused by different pollution or faults from adversarial training, and has stronger anomaly classification capabilities in the face of severe pollution. In particular, the parameter range of the generator's long short-term memory network unit can be [32,256], and the expansion rate of the discriminator can be set to [1,2,4]. The learning effect at different levels is determined by hyperparameter search.

[0052] Regarding the self-supervised contrastive learning network, the contrastive loss function minimizes the Euclidean distance between abnormal sample features, while maximizing the distance between normal and abnormal sample features, guiding the new encoder to better learn the feature differences of different abnormal patterns and quickly improve the new encoder's ability to learn abnormalities. In addition to using contrastive learning alone, contrastive learning can also be combined with a generative adversarial network to use the abnormal sample features generated by the generator to compare with the normal data features.

[0053] To this end, the weights determined by dynamic search are and The setting range can be [0.1,1.0]. If the real-time requirement is high, set =0; if the data is heavily polluted, set To enhance comparative learning ability.

[0054] Optionally, the gate includes multiple types, including an input gate, a forget gate, and an output gate. Step 102 uses each gate in the long short-term memory network with a gating mechanism added, and based on the activation value dynamic adjustment factor, cross-validates the optimal activation value corresponding to each gate of the industrial control time series data to be detected under various data pollution conditions, and adjusts each gate through the optimal activation value to output the long-term dependency feature, specifically including: Step 1021, for any of the various gates in the gating mechanism, based on the activation value dynamic adjustment factor and the optimal activation value determination formula, cross-validation is performed to obtain the optimal activation value corresponding to the gate under various data pollution conditions for the industrial control time series data to be detected, wherein the optimal activation value determination formula is: , , is the optimal activation value of the gate at time t, is the weight matrix, The industrial control timing data to be tested input at the current moment and the hidden state at the previous moment t-1 The splicing, is the bias vector, is the optimal activation value of the gate adjusted based on the activation value dynamic adjustment factor, The Sigmoid activation function is used to make the optimal activation value between 0 and 1.

[0055] Step 1022, for any of the various gates, the output of the gate is controlled by the gating calculation formula corresponding to the gate and the optimal activation value of the gate, until the outputs of the various gates are respectively controlled and the long-term dependent features are output, wherein the gating calculation formulas corresponding to the different gates are respectively: , , , , , , Respectively through the Sigmoid activation function The calculated input gate, forget gate and output gate, To pass The candidate cell states calculated by the function, , , and are the weight matrices of the input gate, forget gate, output gate, and candidate cell state, respectively. , , and They are the bias items of input gate, forget gate, output gate and candidate cell state respectively. Input industrial control timing data at the current moment and the hidden state at the previous moment t-1 The splicing, , and are the optimal activation values ​​of the input gate, forget gate and output gate at time t, respectively. The Sigmoid activation function is used to make the optimal activation value between 0 and 1.

[0056] Step 1023, using the latest information stored in the memory unit and the industrial control timing data to be detected input at the current moment, output the long-term dependency features through the output gate, wherein the output long-term dependency features include the information at the current moment and the long-term dependency features at the previous moment.

[0057] In the above embodiment of the present application, a long short-term memory network with a gating mechanism is used to construct a long-term dependency modeling part in a new encoder, that is, a long short-term memory network 1011. This part introduces a dynamic adjustment factor of the activation value (the value range can be set to [0.1, 1.0]), and cross-validates the optimal activation value of the industrial control time series data to be detected under various data pollution conditions, so that the model (long short-term memory network) can select the appropriate input gate, output gate and forget gate activation values ​​according to different types of data pollution, effectively avoiding excessive dependence on polluted data and reducing unnecessary noise interference. At the same time, the dynamic adjustment factor of the activation value will automatically adjust the activation value of each gate according to the current input feature at each time step (moment) t. For example, when the temperature jumps from 25°C to 40°C, the dynamic adjustment factor of the activation value is automatically adjusted, thereby reducing the impact of extreme values ​​on the model (long short-term memory network).

[0058] In the gating calculation formulas corresponding to different types of gates, Represents the input gate, which determines how much of the input information at the current moment will be integrated into the cell state. It is calculated by the sigmoid function (σ), which outputs a value between 0 and 1 to smoothly control the inflow of information. Represents the forget gate, which determines how much information in the cell state of the previous moment will be forgotten. Similarly, this is also calculated by the sigmoid function, outputting a value between 0 and 1 to control the degree of information forgetting. Represents the output gate, which determines how much information in the current cell state will be output to the hidden state. This is also calculated by the sigmoid function, which outputs a value between 0 and 1 to control the output of information. is the candidate cell state, which is calculated by the tanh function, which outputs a value between -1 and 1. The candidate cell state contains the comprehensive information of the current input information and the hidden state at the previous moment, and is the basis for updating the cell state. , , and are weight matrices, which correspond to the calculation of the input gate, forget gate, output gate, and candidate cell state, respectively. , , and is a bias term used to adjust the calculated base value. , and As the optimal activation values ​​of the input gate, forget gate and output gate respectively, the opening degree of the input gate, forget gate and output gate can be further fine-tuned.

[0059] By applying the above embodiments of the present application, the long short-term memory network (LSTM) 1011 can process industrial control time series data more flexibly, especially in the face of data pollution, by constructing a dynamic adjustment factor for the activation value. Specifically, the dynamic adjustment factor of the activation value automatically adjusts the optimal activation value of each gate (such as the input gate, the forget gate, and the output gate) according to the current input feature at each time step (moment) t. This dynamic adjustment helps the LSTM network to better adapt to the data characteristics at different times, especially when the data is polluted or noisy. At the same time, through the cross-validation method, the LSTM network can optimize the activation value corresponding to each gate under pollution conditions to obtain the optimal activation value. This process aims to find the optimal activation value combination for each gate to maintain the best performance under different data pollution levels (situations). The core of the LSTM network is the ability to capture and maintain long-term dependency features. This is due to its internal memory unit and gating mechanism. The memory unit is responsible for storing and updating information, while the gating mechanism (input gate, forget gate, output gate) controls the inflow, outflow and forgetting of information. By dynamically adjusting the optimal activation values ​​of these gates, the LSTM network can more accurately control the flow of information, thereby better capturing and maintaining long-term dependency features. When processing industrial control time series data, the LSTM network gradually accumulates and updates information through its memory cells and gating mechanisms. When long-term dependency features need to be output, the LSTM network uses its internal state (that is, the information in the memory cell) and the current input features to generate output through the output gate. At the same time, this output not only contains the information at the current moment, but also integrates the long-term dependency features of the previous moment.

[0060] In summary, by constructing a dynamic adjustment factor for activation values ​​and cross-validating the best activation value at each moment based on the dynamic adjustment factor for activation values, the LSTM network can process industrial control time series data more flexibly and accurately output its long-term dependency features. This mechanism enables the LSTM network to maintain high performance and stability in the face of data pollution and noise.

[0061] Optionally, the temporal convolutional network includes a first convolutional layer, a second convolutional layer and a third convolutional layer, and the expansion rates corresponding to the first convolutional layer to the third convolutional layer increase layer by layer. Step 103 uses the temporal convolutional network based on the residual connection scaling factor to dynamically adjust the contribution of the residual part of the temporal convolutional network according to the long-term dependency feature, and then outputs the short-term feature, which specifically includes: Step 1031, sequentially connect the first convolution layer, the second convolution layer and the third convolution layer through the dynamic residual, use the residual connection scaling factor, dynamically adjust the contribution of the residual part of the temporal convolution network according to the long-term dependency feature, and output the short-term feature, wherein the output short-term feature is expressed by the following formula: , After the temporal convolutional network The convolution operation and the contribution of the residual part are dynamically adjusted to produce the short-term features of the output. is the long-term dependence characteristic at time t, is the weight of the convolution kernel of the temporal convolutional network, To determine the expansion rate of the sampling interval during wide-base operation, is the residual connection scaling factor.

[0062] In the above embodiments of the present application, the time series convolutional network (Convolutional Neural Networks, CNNs) is a deep learning model that is particularly suitable for processing data with a grid structure. Each layer of the time series convolutional network is composed of a residual sub-module, and uses a layer-by-layer increasing expansion rate to enhance the receptive field and adapt to complex time series patterns. In order to optimize the gradient flow and improve the performance of the model, dynamic residual connections are used between layers. The contribution of the residual part is dynamically adjusted through the residual connection scaling factor to identify abnormal short-term changes and effectively filter out the influence of noise, focusing on short-term features with actual abnormal patterns. In addition, by reasonably combining Layer Norm (layer normalization) and Batch Norm (batch normalization), as well as streamlining Dropout (random inactivation layer), the training stability and expression ability of the time series convolutional network can be significantly improved. For example, the time series convolutional network Figure 3 As shown in the figure, the random dropout layer is Dropout, the rectified linear unit variant is ReLU*, ReLU is Rectified Linear Unit, and is a rectified linear unit. Batch normalization is Batch Norm. Dilated Causal Conv is Dilated Causal Conv, and layer normalization is Layer Norm. Finally, the low-dimensional feature dimension of the output of the temporal convolutional network 1012 can be expressed as , is the extracted short-term feature dimension.

[0063] Residual connection is a special design of temporal convolutional network, first proposed in deep residual network (ResNet). Residual connection helps alleviate the problem of gradient vanishing or gradient exploding during deep network training by adding shortcuts (or called jump connections) between certain layers of the network, allowing the original input information to be passed directly to the subsequent layers. The residual submodule usually consists of two or more convolutional layers, and the input is directly added to the output of these convolutional layers through shortcuts (or other forms of merging, such as splicing), and then this result is used as the input of the next residual submodule. This design makes it easier for the network to learn identity mapping, that is, if the network believes that some layers do not need to change the input, it can directly learn to make the output of these layers close to the input.

[0064] Batch Normalization (Batch Norm) is a normalization technique used to improve the speed and stability of neural network training. It normalizes the features of each small batch of data so that the mean of each feature is 0 and the variance is 1, and then scales and offsets it through learnable parameters to restore the expressiveness of the data.

[0065] About Layer Normalization (Layer Norm), Layer Norm is another normalization technique that does not normalize on the batch dimension, but on the feature dimension. This means that it is independent of the batch size and is therefore more suitable for scenarios such as small batches or recurrent neural networks (RNNs).

[0066] By applying the above embodiments of the present application, that is, by combining Layer Norm and Batch Norm reasonably, the performance of the model (temporal convolutional network) can be further improved. For example, Batch Norm is used in some layers, while Layer Norm is used in other layers (especially those that benefit from processing independent of batch size). This combination can be adjusted according to specific tasks and model architecture to achieve the best effect.

[0067] About the random dropout layer (Dropout), Dropout is a regularization technique used to prevent neural networks from overfitting. By randomly dropping some neurons in the network during training (i.e. setting their outputs to 0), Dropout can force the network to learn more robust feature representations.

[0068] In the above implementation of the present application, Dropout is streamlined. Simplifying the Dropout layer means using the Dropout layer with caution in the design to avoid excessive use that may lead to model learning difficulties or performance degradation. The streamlined Dropout layer can be done by selecting an appropriate Dropout rate (i.e., the proportion of neurons discarded) and determining where to use Dropout in the network. In particular, it is also possible to use Dropout only in certain parts of the network (such as the fully connected layer) and not in other parts such as the convolutional layer.

[0069] When outputting short-term features through the temporal convolutional network, that is, in the formula middle, Represents the output feature. This is the output feature result (short-term feature) after the convolution operation of the temporal convolutional network and the contribution of the residual part are dynamically adjusted. It is the long-term dependency feature at time t, which is the original feature data processed by the temporal convolutional network. Represents the weights of the convolution kernel. These weights are used in the convolution operation to extract specific patterns in the input features. d represents the dilation rate, which determines the sampling interval during the convolution operation and can affect the scale and range of feature extraction. β is the residual connection scaling factor, which is used to introduce a dynamic residual part. It determines the weight of the residual connection, making the residual connection more flexible, which helps to improve the training stability and expressiveness of the model. In particular, the dilation rate range of the temporal convolutional network can be {1,2,4}, and the dynamic adjustment range of the residual connection scaling factor can be set to [0.1,1.0]. Random search is used for parameter adjustment, and its optimal value is found through experiments. For example, when processing power consumption data, the voltage may rise when the device starts, which may cause the temperature reading to briefly soar to 50°C. The temporal convolutional network filters out such transient phenomena by adjusting the importance of dynamic weighting of short-term features, ensuring that the model focuses on truly abnormal patterns.

[0070] Optionally, in step 104, the self-attention mechanism module uses the self-attention mechanism to locate key features in the short-term features and outputs them, specifically including: Step 1041, using a self-attention mechanism to dynamically identify the contamination features in the short-term features, and adjust the attention weights of the identified contamination features to obtain and output key features.

[0071] In the above embodiment of the present application, the self-attention mechanism module 1013 can represent short-term features in the form of vectors or tensors, that is, the features can be time series data. Ensure that all features are in the same dimensional space for subsequent processing. Then generate query, key, and value vectors for each feature, for example, which can be achieved by linear transformation (such as a fully connected layer). Calculate the dot product of the query vector and all key vectors, and then apply the softmax function to obtain the attention score. These scores represent the importance of each feature to the current query feature. By analyzing the attention score, features with abnormally high or low scores can be identified, which may be contamination features or noise. Next, a strategy is designed to dynamically adjust the attention weights of the identified contamination features. For example, lower weights can be assigned to these features, or the weights can be adjusted based on the comparison of their scores with a preset threshold. Apply the adjusted weights, the weight range setting can be [0.0, 1.0], and recalculate the attention score of each feature to ensure that key features receive more attention. The value vector is weighted and summed using the adjusted attention weights to obtain a weighted representation of each feature. According to the weighted representation, those features with high weights and significant contributions to the task are selected as key features. The selected key features are integrated into a compact representation, such as a vector, matrix or tensor, depending on the task requirements. To this end, the integrated key features are output to subsequent processing layers or models for further analysis, prediction or decision-making. In particular, the self-attention mechanism is usually implemented as a layer or module of a neural network, which can be combined with other layers (such as convolutional layers, fully connected layers), and the model can be trained through optimization algorithms such as backpropagation and gradient descent to minimize the loss function and improve performance. At the same time, regularization techniques (such as dropout, L2 regularization) can be applied to prevent overfitting and improve the generalization ability of the model.

[0072] Optionally, the abnormality detection module in step 106 obtains an abnormality detection result by comparing the industrial control timing data to be detected with the industrial control timing data to be compared, which specifically includes: Step 1061, the abnormality detection module compares the industrial control timing data to be detected with the industrial control timing data to be compared. When the comparison result of the industrial control timing data to be detected and the industrial control timing data to be compared shows that there is a difference exceeding a preset difference threshold, abnormal industrial control timing data exists.

[0073] In the above embodiment of the present application, the input and output are compared, and the difference between the input data and the decoded output data (such as MSE) can be calculated, and a threshold (preset difference threshold) can be set to determine whether the difference is significant. If the difference exceeds the threshold, it is determined to be abnormal; otherwise, it is normal. The threshold can be set through historical data.

[0074] Optionally, before obtaining the industrial control time series data to be detected and inputting it into the enhanced neural network framework to perform the following steps to realize data anomaly detection through the enhanced neural network framework, the method further includes: Step 110, acquiring original industrial control time series data in real time, performing data preprocessing on the original industrial control time series data, and obtaining the industrial control time series data to be detected, wherein the data preprocessing includes at least one of data outlier removal and data normalization.

[0075] In the above embodiments of the present application, in order to ensure the quality and consistency of the input data, the KNN interpolation method or the linear interpolation method can be used to process missing data, the interquartile range method or the Z-score normalization method can be used to eliminate extreme outliers, and the time series data can be normalized by linear normalization. The final processed data can be expressed as (B, T, F), where B is the batch size, T is the time step, and F is the number of features.

[0076] Specifically, for example, actual time series data in industrial control systems, including sensor data, equipment network load flow data, etc., are selected, and missing values ​​are filled using KNN interpolation or linear interpolation. For example, in industrial control systems, sensors occasionally lose power briefly, resulting in missing data. KNN interpolation is used to infer missing values ​​based on the spatial similarity of adjacent points to ensure the continuity and integrity of the time series.

[0077] Regarding the removal of data outliers, the interquartile range method can be introduced to remove extreme outliers. On each time series feature dimension, the upper and lower quartiles Q1 and Q3 are defined, and the abnormal threshold is defined as Q1−1.5×IQR, Q3+1.5×IQR, and data outside this range is removed. For example, in temperature sensor data, if some readings are outside the normal range (such as above 40°C), they are considered abnormal and removed to avoid false alarms. Among them, IQR is the abbreviation of "Interquartile Range", which means "interquartile range". IQR is a statistic that describes the distribution of data and represents the data range of the middle 50% of the data set. Specifically, IQR is the difference between the upper quartile (Q3) and the lower quartile (Q1). IQR can be used to measure the degree of dispersion of data. A larger IQR indicates that the data distribution is more dispersed, and a smaller IQR indicates that the data distribution is more concentrated.

[0078] Regarding data normalization, time series data can be converted to between 0 and 1 through linear normalization to ensure the dimensional consistency of data of different dimensions, as shown in the following formula: , It is the time series data of the industrial control to be tested after data normalization processing. is the original industrial control timing data, and are the minimum and maximum values ​​in the original industrial control timing data respectively. Therefore, the time series dimension of the industrial control timing data to be detected after normalization is (B, T, F).

[0079] By applying the technical solution of this embodiment and dynamically adjusting the parameter values ​​involved in the time series feature learning part, the purpose of effectively distinguishing various types of data pollution and real anomalies can be achieved, and the problem of decreased accuracy and weak adaptability of most time series detection methods faced with multi-type data pollution can be effectively solved.

[0080] Through the description of the above implementation methods, technicians in this field can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, and can also be implemented by hardware by dynamically adjusting the parameter values ​​involved in the time series feature learning part to achieve the purpose of effectively distinguishing various data pollution and real anomalies, effectively solving the problems of decreased accuracy and weak adaptability of most time series detection methods in the face of multi-type data pollution.

[0081] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0082] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be made by technicians in this field should fall within the scope of protection of this application.

Claims

1. A method for detecting anomalies in industrial control time series data, characterized in that: Applied to an enhanced neural network framework, the enhanced neural network framework includes a novel autoencoder and an anomaly detection module, the novel autoencoder includes a novel encoder and a novel decoder with a reciprocal structure, the novel encoder includes a long short-term memory network, a temporal convolutional network and a self-attention mechanism module; the method includes: The industrial control time series data to be detected is obtained and input into the enhanced neural network framework, so as to implement data anomaly detection by executing the following steps through the enhanced neural network framework: By using each gate in the long short-term memory network with a gating mechanism, based on the dynamic adjustment factor of the activation value, cross-validate the optimal activation value corresponding to each gate of the industrial control time series data to be tested under various data pollution conditions, and adjust each gate through the optimal activation value to output the long-term dependency feature; After dynamically adjusting the contribution of the residual part of the temporal convolutional network according to the long-term dependency feature based on the residual connection scaling factor using the temporal convolutional network, the short-term feature is output; The self-attention mechanism module uses a self-attention mechanism to locate key features in the short-term features and output them, wherein the output key features are characterized by reconstructed time series and low-dimensional feature representation; The novel decoder reversely decodes the key features to obtain the industrial control timing data to be compared; The anomaly detection module obtains an anomaly detection result by comparing the industrial control timing data to be detected with the industrial control timing data to be compared.

2. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: The enhanced neural network framework also includes a self-supervised contrastive learning network and a generative adversarial network; the gates in the long short-term memory network with the added gating mechanism are cross-validated based on the activation value dynamic adjustment factor for each gate under various data pollution conditions for the industrial control time series data to be detected, and before the long-term dependent features are output after adjusting each gate by the optimal activation value, the method also includes: Determine the data scenario to which the industrial control time series data to be detected belongs, and use the self-supervised contrastive learning network and the generative adversarial network to jointly optimize the parameters of the new autoencoder for the determined data scenario, wherein the data scenario includes a scenario with high real-time data demand, a scenario with high data pollution, and a scenario with insufficient data.

3. The method for detecting anomalies in industrial control time series data according to claim 2, characterized in that: The method of using the self-supervised contrastive learning network and the generative adversarial network to jointly optimize the parameters of the new autoencoder for a certain data scenario includes: The total loss function of the self-supervised contrastive learning network and the generative adversarial network is constructed, and the parameters of the new autoencoder are optimized by dynamically adjusting the loss dynamic adjustment factors corresponding to the self-supervised contrastive learning network and the generative adversarial network in the constructed total loss function, wherein the total loss function is: ， is the total loss function, is the contrastive loss of the self-supervised contrastive learning network, is the adversarial loss of the generative adversarial network, is the loss dynamic adjustment factor of the self-supervised contrastive learning network, Dynamic adjustment factor for the loss of the generative adversarial network; When the data scenario to which the industrial control time series data to be detected belongs is a scenario with high real-time data requirements, the loss dynamic adjustment factor of the generative adversarial network is set to the minimum preset threshold close to 0; when the data scenario to which the industrial control time series data to be detected belongs is a scenario with high data pollution, the loss dynamic adjustment factor of the self-supervised contrastive learning network is set to be smaller than the loss dynamic adjustment factor of the generative adversarial network; and when the data scenario to which the industrial control time series data to be detected belongs is a scenario with insufficient data, the loss dynamic adjustment factor of the generative adversarial network is set to 0.

4. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: There are multiple gates. The dynamic adjustment factor based on the activation value cross-validates the optimal activation value corresponding to each gate under various data pollution conditions of the industrial control time series data to be detected, including: For any of the various gates in the gating mechanism, based on the dynamic adjustment factor of the activation value and the optimal activation value determination formula, cross-validation is performed to obtain the optimal activation value corresponding to the gate under various data pollution conditions for the industrial control timing data to be detected, wherein the optimal activation value determination formula is: ， ， is the optimal activation value of the gate at time t, is the weight matrix, The industrial control timing data to be tested input at the current moment and the hidden state at the previous moment t-1 The splicing, is the bias vector, is the optimal activation value of the gate adjusted based on the activation value dynamic adjustment factor, The Sigmoid activation function is used to make the optimal activation value between 0 and 1.

5. The method for detecting anomalies in industrial control time series data according to claim 4, characterized in that: The gates include an input gate, a forget gate, and an output gate. The output long-term dependency features after adjusting each gate by the optimal activation value include: For any of the various gates, the output of the gate is controlled by the gating calculation formula corresponding to the gate and the optimal activation value of the gate until the output of the various gates is controlled respectively and the long-term dependent characteristics are output, wherein the gating calculation formulas corresponding to different gates are respectively: ， ， ， ， , , Respectively through the Sigmoid activation function The calculated input gate, forget gate and output gate, To pass The candidate cell states calculated by the function, , , and are the weight matrices of the input gate, forget gate, output gate, and candidate cell state, respectively. , , and They are the bias items of input gate, forget gate, output gate and candidate cell state respectively. Input industrial control timing data at the current moment and the hidden state at the previous moment t-1 The splicing, , and are the optimal activation values ​​of the input gate, forget gate and output gate at time t, respectively. The Sigmoid activation function is used to make the optimal activation value between 0 and 1.

6. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: The long short-term memory network includes a memory unit, the memory unit is used to store and update information, and the output long-term dependency features include: The latest information stored in the memory unit and the industrial control timing data to be detected input at the current moment are used to output the long-term dependency features through the output gate, wherein the output long-term dependency features include the information at the current moment and the long-term dependency features at the previous moment.

7. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: The temporal convolutional network includes a first convolutional layer, a second convolutional layer and a third convolutional layer, and the expansion rates corresponding to the first convolutional layer to the third convolutional layer increase layer by layer. The temporal convolutional network is used to dynamically adjust the contribution of the residual part of the temporal convolutional network according to the long-term dependency feature based on the residual connection scaling factor, and then output short-term features, including: The first convolution layer, the second convolution layer and the third convolution layer are sequentially connected through the dynamic residual, and the residual connection scaling factor is used to dynamically adjust the contribution of the residual part of the temporal convolution network according to the long-term dependency feature, and then the short-term feature is output, wherein the output short-term feature is expressed by the following formula: ， After the temporal convolutional network The convolution operation and the contribution of the residual part are dynamically adjusted to produce the short-term features of the output. is the long-term dependence characteristic at time t, is the weight of the convolution kernel of the temporal convolutional network, To determine the expansion rate of the sampling interval during wide-base operation, is the residual connection scaling factor.

8. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: The self-attention mechanism module uses the self-attention mechanism to locate key features in the short-term features and outputs them, including: The self-attention mechanism module adopts the self-attention mechanism to dynamically identify the contamination features in the short-term features, and adjusts the attention weights of the identified contamination features to obtain and output key features.

9. The method for detecting anomalies in industrial control time series data according to claim 1, characterized in that: The abnormality detection module obtains an abnormality detection result by comparing the industrial control timing data to be detected with the industrial control timing data to be compared, including: The abnormality detection module compares the industrial control timing data to be detected with the industrial control timing data to be compared. When the comparison result of the industrial control timing data to be detected and the industrial control timing data to be compared shows that there is a difference exceeding a preset difference threshold, abnormal industrial control timing data exists.

10. The method for detecting anomalies in industrial control time series data according to any one of claims 1 to 9, characterized in that: Before obtaining the to-be-detected industrial control time series data and inputting it into the enhanced neural network framework to perform the following steps to realize data anomaly detection through the enhanced neural network framework, the method further includes: The original industrial control time series data is acquired in real time, and the industrial control time series data to be detected is obtained after data preprocessing is performed on the original industrial control time series data, wherein the data preprocessing includes at least one of data outlier removal and data normalization.

Images