An Anomaly Detection Method for Industrial Control Systems Based on Memory Convolutional Neural Networks
By employing a lightweight memory convolutional neural network architecture and multi-feature fusion, the problem of insufficient temporal feature capture in industrial control systems is solved, enabling efficient anomaly detection and real-time response, while reducing deployment costs and hardware upgrade pressure.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- EZHOU POWER SUPPLY COMPANY STATE GRID HUBEI ELECTRIC POWER
- Filing Date
- 2026-01-21
- Publication Date
- 2026-06-02
AI Technical Summary
Existing convolutional neural networks are difficult to effectively capture the features of time-series data in industrial control systems, resulting in insufficient anomaly detection accuracy. Furthermore, recurrent neural networks and long short-term memory networks have high computational complexity, making them difficult to deploy on industrial control computers and achieve real-time detection.
Employing a lightweight memory convolutional neural network architecture, combined with adaptive gating networks and temporal memory units, an anomaly detection model is constructed through multi-feature collaborative fusion and dynamic anti-interference strategies. This model adapts to the limited resources of industrial control computers and achieves millisecond-level inference.
It improves the sensitivity of identifying subtle anomalies and covert attacks, reduces the false positive rate, simplifies model deployment costs, adapts to the operating rules of different industrial scenarios, and ensures the stability of industrial control systems and their real-time monitoring requirements.
Smart Images
Figure CN122131738A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial control system safety technology, and more specifically, to an anomaly detection method for industrial control systems based on a memory convolutional neural network. Background Technology
[0002] Industrial control systems, as the core infrastructure of intelligent manufacturing, are widely used in key fields such as energy, chemical industry, and intelligent manufacturing. Their stable operation is directly related to production safety, economic benefits, and public safety. Anomaly detection is a core technology for ensuring the safety of industrial control systems. It requires real-time analysis of time-series data and network traffic generated by sensors and actuators to accurately identify abnormal states that deviate from normal operating modes, thereby avoiding serious consequences such as equipment downtime and safety accidents.
[0003] In recent years, the development of deep learning technology has provided new possibilities for anomaly detection. Convolutional Neural Networks (CNNs), as a commonly used deep learning model, have achieved remarkable results in fields such as image recognition and speech recognition. However, CNNs have limited capabilities in processing time-series data and cannot effectively capture temporal features in time-series data. This makes it difficult to identify subtle and hidden anomalies that depend on time-series patterns, resulting in insufficient detection accuracy, high false positive and false negative rates, and an inability to meet the reliability requirements of anomaly identification in industrial control systems. In contrast, while Recurrent Neural Networks (RNNs) and Long Short-Term Memory Networks (LSTMs) have the ability to capture temporal features, they have high computational complexity, a large number of parameters, and high hardware computing power requirements. However, industrial control computers in industrial settings often have limited CPU performance and insufficient memory resources, making model deployment difficult and inference latency exceeding the real-time detection threshold of industrial control systems, failing to achieve millisecond-level anomaly response and affecting the timeliness of fault handling. Therefore, there is an urgent need to provide an anomaly detection method for industrial control systems based on memory convolutional neural networks to solve the above problems. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention provides an anomaly detection method for industrial control systems based on a memory-based convolutional neural network. It employs a memory-based convolutional neural network architecture, integrating lightweight modules to simplify the structure, adapting to the limited resources of industrial control computers, and achieving millisecond-level inference. Through multi-feature collaborative fusion, it improves the sensitivity of identifying subtle anomalies and covert attacks. Combined with dynamic anti-interference and adaptive learning rate strategies, it reduces false positives and parameter tuning costs. The solution is easy to integrate, requires no large-scale modification, balances innovation and practicality, effectively ensures the stability of industrial control systems, and reduces security risks.
[0005] To achieve the above objectives, the technical solution of the present invention is as follows:
[0006] An anomaly detection method for industrial control systems based on memory convolutional neural networks includes the following steps:
[0007] S1. During the training data acquisition phase, network traffic is collected in real time from the industrial control system, and corresponding deep industrial control protocol parsing is performed. A set of sensor and actuator data reflecting the operating status of the entire industrial control system is obtained at all times, and an industrial control anomaly detection training dataset is constructed based on the periodically collected sensor and actuator data;
[0008] S2. Establish an anomaly detection model based on hierarchical adaptive temporal feature fusion. Train the anomaly detection model using the industrial control anomaly detection training dataset until the model training loss reaches a set threshold or the maximum number of training iterations is reached, thus completing the anomaly detection model training. The anomaly detection model includes a main feature extraction network and a lightweight embedding network. Adaptive Gating Networks Sequential memory unit , as well as convolutional layers, pooling layers, and fully connected layers containing historical fully connected layers;
[0009] S3. In the industrial control system anomaly detection phase, network traffic is collected from the industrial control system in real time, and corresponding deep industrial control protocol parsing is performed to obtain sensor and actuator data reflecting the current operating status of the entire industrial control system. The data is input into the trained anomaly detection model, and the anomaly detection result is output. The anomaly detection result is set as follows: the model output of 1 indicates that the industrial control system is operating normally, and the output of 0 indicates that the system is operating abnormally.
[0010] As a preferred embodiment of the present invention, the construction of the industrial control anomaly detection training dataset based on periodically collected sensor and actuator data in S1 is specifically as follows:
[0011] The sensor and actuator data are set based on periodic acquisition. ,in This represents the total number of sensors and actuators in an industrial control system.
[0012] Set the collection period ,according to Decision on whether the industrial control system is malfunctioning Data tags Value: Normal In case of abnormality ; in each Real-time collection , obtain training samples ;
[0013] The network traffic of the industrial control system is periodically collected and analyzed until the number of training samples with abnormal data labels reaches a set threshold, thus obtaining the industrial control system anomaly detection training dataset. ,in For training dataset The total number of samples, .
[0014] As a preferred embodiment of the present invention, step S2 specifically comprises:
[0015] S21: From the training dataset Training samples are selected sequentially. Training samples Includes data vector Dimension Extract high-level abstract features, low-level alignment features, and temporal memory features;
[0016] S22: Perform a weighted summation of the three features to obtain an adaptive temporal fusion sample with the same dimension as the original input. ;
[0017] S23: Will The input convolutional layer performs secondary feature extraction, updates the temporal memory, and adjusts the model parameters through the loss function and backpropagation algorithm.
[0018] S24: Repeat steps S21 to S23 until the model training loss reaches the set threshold or the maximum number of training iterations is reached, and the model training is completed.
[0019] As a preferred embodiment of the present invention, S21 specifically comprises:
[0020] S211: Transfer data vector Inputting the front-end feature extraction layer of the main feature extraction network yields high-level abstract features. , dimension ;
[0021] S212: Transfer data vector Independent Input Lightweight Embedded Network To obtain low-level alignment features , dimension , used to achieve feature space alignment;
[0022] S213: Acquiring Temporal Memory Features :like It is the training dataset. The first sample, then Let it be a zero vector (dimension) If it is not the first sample, then from the sequential memory unit. Retrieve the temporal memory retained from the previous moment. (dimension) ).
[0023] As a preferred embodiment of the present invention, S22 specifically comprises:
[0024] S221: Will , and Channel concatenation is performed to obtain the initial fusion vector. , dimension ;Will Input Adaptive Gating Network This yields three sets of original weights at different dimensions. Each group of weight dimensions is ;
[0025] S222: Adjusting the original weights Perform Softmax normalization to obtain the attention weight vector. ,satisfy ;
[0026] S223: Through element-wise multiplication ( The fused sample is obtained by weighted summation. The formula is:
[0027]
[0028] The fusion sample The dimension is .
[0029] As a preferred embodiment of the present invention, S223 obtains the fused sample. After the fused samples are input into the convolutional layer for secondary feature extraction, a dynamic noise suppression and temporal consistency verification step is also included, specifically:
[0030] Dynamic noise suppression filters random noise in sensor / actuator data based on time-series statistical characteristics:
[0031] Calculate the temporal variance of the effective features at the three most recent moments in the fused sample and the historical memory set. The formula is:
[0032]
[0033] in, The number of historical fusion samples used in the calculation ( ), The average of the historical fused samples and the current fused samples;
[0034] Set noise threshold Threshold filtering is performed on the feature values of each dimension in the fused sample: if the feature value of a certain dimension is equal to... The absolute value of the difference in corresponding dimensions Then use Replace the feature value with the corresponding dimension value to obtain the denoised fused sample;
[0035] Timing consistency verification ensures that feature changes conform to the physical laws of the industrial control system.
[0036] Calculate the cosine similarity between the denoised fused sample and the denoised fused sample from the previous time step. The formula is:
[0037]
[0038] in," " is the vector dot product, Let L2 be the norm of the vector;
[0039] like , The preset consistency threshold is obtained based on the time-series correlation statistics of normal industrial control data. If the current sample is deemed to have an abnormal mutation risk, the denoised fused sample is then subjected to weighted smoothing with the denoised fused sample from the previous time step, using the following formula:
[0040] ;
[0041] like Then directly order ;
[0042] The fused samples, after noise suppression and temporal consistency verification, will then be input into the convolutional layer for secondary feature extraction.
[0043] As a preferred embodiment of the present invention, S23 will... The secondary feature extraction process for input convolutional layers specifically includes:
[0044] S231: Set the dimension of the input matrix of the convolutional layer to be... ,in , It is a rounding function;
[0045] fusion samples Perform data reorganization, sequentially... The data is filled into the input matrix; if the number of data bits is insufficient, zeros are filled into the input matrix.
[0046] The padded input matrix is fed into a convolutional layer for feature extraction, compressed by a pooling layer, and then fed into a historical fully connected layer for feature integration.
[0047] As a preferred embodiment of the present invention, S23 further includes the following:
[0048] S232: Extract new temporal memories from the output of the historical fully connected layer. The As the next training sample Sequential memory ,Right now of This is for use in feature fusion at the next moment;
[0049] S233: The loss for a single training sample is calculated using the binary cross-entropy loss function, with the following formula:
[0050]
[0051] In the formula, For the first Data labels for each training sample To predict the probability of the positive class for the model, ;
[0052] S234: Adjust the model parameters using the backpropagation algorithm, the formula is:
[0053]
[0054] In the formula, For the model parameters to be adjusted, These are the model parameters before adjustment. This is the learning rate.
[0055] As a preferred embodiment of the present invention, step S234 further includes an adaptive learning rate adjustment step based on the training loss trend, specifically:
[0056] Set the sliding window size Record the training loss for each batch within the window and calculate the rate of change of loss within the window. :
[0057]
[0058] In the formula, For the first Training loss per batch, Reflecting the changing trend of losses ( This indicates a decrease in losses. (Indicates that losses have stagnated or increased).
[0059] Set learning rate adjustment threshold and , For the threshold of rapid loss decline To adjust the learning rate adaptively to the loss stagnation threshold :
[0060] like Maintain the current learning rate Keep it unchanged to avoid reducing it too early, which would slow down convergence.
[0061] like Adjust the learning rate to Gradually decrease the learning rate to achieve stable convergence;
[0062] like Adjust the learning rate to To avoid getting trapped in local optima;
[0063] The adjusted learning rate is substituted into the backpropagation algorithm formula for updating the model parameters in the current batch, and the above adjustment steps are repeated for the next batch.
[0064] This invention also provides an anomaly detection system for industrial control systems based on a memory convolutional neural network, comprising:
[0065] The data acquisition module is used to collect network traffic from the industrial control system in real time during the training data acquisition phase, and to perform corresponding deep industrial control protocol parsing. A set of sensor and actuator data reflecting the operating status of the entire industrial control system is obtained at all times, and an industrial control anomaly detection training dataset is constructed based on the periodically collected sensor and actuator data;
[0066] The model training module is used to build an anomaly detection model based on a memory convolutional neural network. The anomaly detection model is trained according to the industrial control anomaly detection training dataset until the training loss of the anomaly detection model reaches a set threshold or the maximum number of training iterations is reached, thus completing the anomaly detection model training.
[0067] The anomaly detection module is used to collect network traffic from the industrial control system in real time during the anomaly detection phase of the industrial control system, and perform corresponding deep industrial control protocol parsing to obtain sensor and actuator data that reflects the current operating status of the entire industrial control system. The sensor and actuator data are then input into the anomaly detection model, and the anomaly detection results are output.
[0068] The beneficial technical effects of this invention are:
[0069] By employing a memory-based convolutional neural network architecture, integrating lightweight embedding networks, adaptive gating networks, and temporal memory units, the model structure is simplified while ensuring the integrity of feature extraction. This avoids the problems of large parameter count and high computational requirements of traditional deep learning models. The model does not require high-performance hardware support and can be directly adapted to the limited CPU performance and memory resources of industrial control computers. Inference latency is controlled at the millisecond level, which not only meets the core requirements of real-time monitoring in industrial control systems but also reduces deployment costs and hardware upgrade pressure.
[0070] By using an adaptive gating network to achieve the synergistic fusion of high-level abstract features, low-level alignment features, and historical temporal memory features, the shortcomings of single feature extraction are solved, allowing the model to fully capture the long-distance dependencies of industrial control data and significantly improve the sensitivity of identifying subtle anomalies and covert attacks.
[0071] By adaptively adjusting the noise threshold using the temporal variance of historical fusion samples, the model can accurately handle complex and ever-changing interference signals in industrial control environments, maintaining stable feature extraction performance even in low signal-to-noise ratio environments. Temporal consistency verification utilizes cosine similarity judgment and weighted smoothing to avoid misjudgments caused by abrupt signal changes, enabling the model to adapt to the operational patterns of different industrial scenarios and demonstrating good versatility in energy, chemical, and intelligent manufacturing fields. The adaptive learning rate adjustment strategy based on the training loss trend calculates the rate of change of loss in real time through a sliding window and dynamically adjusts parameters, solving the problems of slow convergence, oscillation, or getting trapped in local optima caused by a fixed learning rate. This not only accelerates model convergence speed and improves training accuracy and stability but also reduces manual parameter tuning costs, helping the model quickly adapt to the personalized training needs of different industrial control systems.
[0072] By integrating key technologies such as lightweight architecture, multi-feature fusion, dynamic anti-interference, and adaptive training, a complete solution with logical coherence and ease of operation is formed. All technical steps are based on basic statistical calculations and a simple network structure, without complex hardware dependencies or cumbersome operating procedures. It can be directly integrated into existing industrial control and monitoring systems without large-scale modifications to existing equipment. It balances technological innovation with engineering practicality, effectively ensuring the stable operation of industrial control systems and reducing the risk of safety accidents and economic losses. Attached Figure Description
[0073] Figure 1 This is the overall flowchart of the present invention.
[0074] Figure 2 This is an overall structural diagram of the anomaly detection model in this invention.
[0075] Figure 3 This is a one-dimensional convolution parameter setting table in an embodiment of the present invention.
[0076] Figure 4 This is the parameter setting table for the fully connected layer in this embodiment of the invention.
[0077] Figure 5 This is a comparison table of the embodiments of the present invention and traditional convolutional neural networks. Detailed Implementation
[0078] In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0079] Combination Figure 1-5 The present invention provides the following embodiments:
[0080] An anomaly detection method for industrial control systems based on memory convolutional neural networks includes the following steps:
[0081] S1. During the training data acquisition phase, network traffic is collected in real time from the industrial control system, and corresponding deep industrial control protocol parsing is performed. A set of sensor and actuator data reflecting the operating status of the entire industrial control system is obtained at all times, and an industrial control anomaly detection training dataset is constructed based on the periodically collected sensor and actuator data;
[0082] S2. Establish an anomaly detection model based on hierarchical adaptive temporal feature fusion. Train the anomaly detection model using the industrial control anomaly detection training dataset until the model training loss reaches a set threshold or the maximum number of training iterations is reached, thus completing the anomaly detection model training. The anomaly detection model includes a main feature extraction network and a lightweight embedding network. Adaptive Gating Networks Sequential memory unit , as well as convolutional layers, pooling layers, and fully connected layers containing historical fully connected layers;
[0083] S3. In the industrial control system anomaly detection phase, network traffic is collected from the industrial control system in real time, and corresponding deep industrial control protocol parsing is performed to obtain sensor and actuator data reflecting the current operating status of the entire industrial control system. The data is input into the trained anomaly detection model, and the anomaly detection result is output. The anomaly detection result is set as follows: the model output of 1 indicates that the industrial control system is operating normally, and the output of 0 indicates that the system is operating abnormally.
[0084] Furthermore, the industrial control anomaly detection training dataset constructed in S1 based on periodically collected sensor and actuator data is as follows:
[0085] The sensor and actuator data are set based on periodic acquisition. ,in This represents the total number of sensors and actuators in an industrial control system.
[0086] Set the collection period ,according to Decision on whether the industrial control system is malfunctioning Data tags Value: Normal In case of abnormality ; in each Real-time collection , obtain training samples ;
[0087] The network traffic of the industrial control system is periodically collected and analyzed until the number of training samples with abnormal data labels reaches a set threshold, thus obtaining the industrial control system anomaly detection training dataset. ,in For training dataset The total number of samples, .
[0088] Industrial control data has strong temporal correlation. Periodic collection can retain the temporal dimension characteristics of the data, which can meet the model's need to capture temporal patterns. Labeling according to system status clarifies the boundaries of sample categories. Setting a threshold for the number of abnormal samples avoids class imbalance in the dataset, ensuring that the model can fully learn abnormal patterns and improve generalization ability.
[0089] Furthermore, step S2 is as follows:
[0090] S21: From the training dataset Training samples are selected sequentially. Training samples Includes data vector Dimension Extract high-level abstract features, low-level alignment features, and temporal memory features;
[0091] S22: Perform a weighted summation of the three features to obtain an adaptive temporal fusion sample with the same dimension as the original input. ;
[0092] S23: Will The input convolutional layer performs secondary feature extraction, updates the temporal memory, and adjusts the model parameters through the loss function and backpropagation algorithm.
[0093] S24: Repeat steps S21 to S23 until the model training loss reaches the set threshold or the maximum number of training iterations is reached, and the model training is completed.
[0094] The step-by-step approach of feature extraction, fusion, training, and iteration aligns with the core logic of deep learning model training. Multi-dimensional features are extracted first and then fused to avoid missing information from a single feature. Parameters are iteratively optimized to allow the model to gradually learn normal and abnormal patterns in the data until it converges to the optimal state, ensuring detection accuracy.
[0095] Furthermore, S21 specifically refers to:
[0096] S211: Transfer data vector Inputting the front-end feature extraction layer of the main feature extraction network yields high-level abstract features. , dimension ;
[0097] S212: Transfer data vector Independent Input Lightweight Embedded Network To obtain low-level alignment features , dimension , used to achieve feature space alignment;
[0098] S213: Acquiring Temporal Memory Features :like It is the training dataset. The first sample, then Let it be the zero vector, with dimension If it is not the first sample, then start from the sequential memory unit. Retrieve the temporal memory retained from the previous moment. Dimension .
[0099] High-level abstract features focus on global data patterns, low-level alignment features retain local details, and temporal memory features relate to historical states. These three complement each other to form a complete feature system. Independent extraction of each feature avoids information interference, lightweight embedded networks adapt to industrial control computing power limitations, and temporal memory initialization rules ensure the stability of the training start.
[0100] Furthermore, S22 specifically refers to:
[0101] S221: Will , and Channel concatenation is performed to obtain the initial fusion vector. , dimension ;Will Input Adaptive Gating Network This yields three sets of original weights at different dimensions. Each group of weight dimensions is ;
[0102] S222: Adjusting the original weights Perform Softmax normalization to obtain the attention weight vector. ,satisfy ;
[0103] S223: Through element-wise multiplication ( The fused sample is obtained by weighted summation. The formula is:
[0104]
[0105] The fusion sample The dimension is .
[0106] Channel splicing integrates the original information of the three features, providing a complete data foundation for weight allocation; the adaptive gating network dynamically generates weights based on data characteristics, achieving accurate fusion with a high proportion of useful features and a low proportion of redundant features; Softmax normalization ensures that the sum of weights is 1, guaranteeing the rationality of the fusion operation; the combination of element-level multiplication and weighted summation preserves feature dimensional information and achieves multi-feature collaborative enhancement.
[0107] Furthermore, S223 obtained the fused sample. After the fused samples are input into the convolutional layer for secondary feature extraction, a dynamic noise suppression and temporal consistency verification step is also included, specifically:
[0108] Dynamic noise suppression filters random noise in sensor / actuator data based on time-series statistical characteristics:
[0109] Calculate the temporal variance of the effective features at the three most recent moments in the fused sample and the historical memory set. The formula is:
[0110]
[0111] in, The number of historical fusion samples used in the calculation ( ), The average of the historical fused samples and the current fused samples;
[0112] Set noise threshold Threshold filtering is performed on the feature values of each dimension in the fused sample: if the feature value of a certain dimension is equal to... The absolute value of the difference in corresponding dimensions Then use Replace the feature value with the corresponding dimension value to obtain the denoised fused sample;
[0113] Timing consistency verification ensures that feature changes conform to the physical laws of the industrial control system.
[0114] Calculate the cosine similarity between the denoised fused sample and the denoised fused sample from the previous time step. The formula is:
[0115]
[0116] in," " is the vector dot product, Let L2 be the norm of the vector;
[0117] like , The preset consistency threshold is obtained based on the time-series correlation statistics of normal industrial control data. If the current sample is deemed to have an abnormal mutation risk, the denoised fused sample is then subjected to weighted smoothing with the denoised fused sample from the previous time step, using the following formula:
[0118] ;
[0119] like Then directly order ;
[0120] The fused samples, after noise suppression and temporal consistency verification, will then be input into the convolutional layer for secondary feature extraction.
[0121] Random noise in industrial control environments follows time-series statistical laws. By dynamically setting thresholds based on historical sample variances, interference signals can be accurately filtered out, avoiding the problem of poor adaptability of fixed thresholds. The operating status of industrial control systems is mainly characterized by gradual changes. Cosine similarity can quantify the amplitude of feature changes, and consistency verification ensures that feature changes conform to physical laws. Weighted smoothing can correct abnormal and sudden signals, reduce false alarms, and improve the model's anti-interference ability.
[0122] Furthermore, in S23, The secondary feature extraction process for input convolutional layers specifically includes:
[0123] S231: Set the dimension of the input matrix of the convolutional layer to be... ,in , corresponding weighted sum The number of channels, It is a rounding function;
[0124] fusion samples Perform data reorganization, sequentially... The data is filled into the input matrix; if the number of data bits is insufficient, zeros are filled into the input matrix.
[0125] The padded input matrix is fed into a convolutional layer for feature extraction, compressed by a pooling layer, and then fed into a historical fully connected layer for feature integration.
[0126] Convolutional layers have fixed requirements for input dimensions, reorganizing one-dimensional fused samples into a two-dimensional matrix to adapt to the local feature extraction logic of convolution operations; rounding up and zero padding ensure data integrity and avoid information loss due to dimension mismatch; pooling layers compress redundant features and reduce computational load; historical fully connected layers integrate temporal and spatial features to further improve feature representation capabilities.
[0127] Furthermore, S23 also includes the following:
[0128] S232: Extract new temporal memories from the output of the historical fully connected layer. Sequential memory Dimension The As the next training sample Sequential memory ,Right now of This is for use in feature fusion at the next moment;
[0129] S233: The loss for a single training sample is calculated using the binary cross-entropy loss function, with the following formula:
[0130]
[0131] In the formula, For the first Data labels for each training sample To predict the probability of the positive class for the model, ;
[0132] S234: Adjust the model parameters using the backpropagation algorithm, the formula is:
[0133]
[0134] In the formula, For the model parameters to be adjusted, These are the model parameters before adjustment. This is the learning rate.
[0135] The time-series memory iterative update enables the continuous transmission and accumulation of historical information, adapting to the long-distance dependency characteristics of industrial control data; the binary cross-entropy loss function can accurately quantify the prediction error of binary classification tasks, meeting the classification requirements of "normal / abnormal" in anomaly detection; the backpropagation algorithm adjusts parameters through gradient descent, allowing the model to gradually approach the optimal solution and improve prediction accuracy.
[0136] Furthermore, step S234 also includes an adaptive learning rate adjustment step based on the training loss trend, specifically:
[0137] Set the sliding window size Record the training loss for each batch within the window and calculate the rate of change of loss within the window. :
[0138]
[0139] In the formula, For the first Training loss per batch, Reflecting the changing trend of losses ( This indicates a decrease in losses. (Indicates that losses have stagnated or increased).
[0140] Set learning rate adjustment threshold and , For the threshold of rapid loss decline To adjust the learning rate adaptively to the loss stagnation threshold :
[0141] like Maintain the current learning rate Keep it unchanged to avoid reducing it too early, which would slow down convergence.
[0142] like Adjust the learning rate to Gradually decrease the learning rate to achieve stable convergence;
[0143] like Adjust the learning rate to To avoid getting trapped in local optima;
[0144] The adjusted learning rate is substituted into the backpropagation algorithm formula for updating the model parameters in the current batch, and the above adjustment steps are repeated for the next batch.
[0145] A sliding window monitors training dynamics in real time, and the rate of change of loss intuitively reflects the model's convergence status. The learning rate is dynamically adjusted according to the convergence stage to solve the problem of slow convergence in the early stage and easy oscillation in the later stage when the learning rate is fixed. By adjusting the threshold division strategy, the model avoids getting stuck in local optima while ensuring the convergence speed, thus improving training efficiency and stability.
[0146] This invention also provides an anomaly detection system for industrial control systems based on a memory convolutional neural network, comprising:
[0147] The data acquisition module is used to collect network traffic from the industrial control system in real time during the training data acquisition phase, and to perform corresponding deep industrial control protocol parsing. A set of sensor and actuator data reflecting the operating status of the entire industrial control system is obtained at all times, and an industrial control anomaly detection training dataset is constructed based on the periodically collected sensor and actuator data;
[0148] The model training module is used to build an anomaly detection model based on a memory convolutional neural network. The anomaly detection model is trained according to the industrial control anomaly detection training dataset until the training loss of the anomaly detection model reaches a set threshold or the maximum number of training iterations is reached, thus completing the anomaly detection model training.
[0149] The anomaly detection module is used to collect network traffic from the industrial control system in real time during the anomaly detection phase of the industrial control system, and perform corresponding deep industrial control protocol parsing to obtain sensor and actuator data that reflects the current operating status of the entire industrial control system. The sensor and actuator data are then input into the anomaly detection model, and the anomaly detection results are output.
[0150] To verify the effectiveness and superiority of the method provided in this embodiment, some specific examples are provided below:
[0151] An anomaly detection model was established, consisting of four one-dimensional convolutional layers, one two-dimensional max-pooling layer, and four fully connected layers. This model incorporates a lightweight embedding network and an adaptive gating network within the feature fusion network. Specifically, the parameters of the one-dimensional convolutional layers are set as follows: Figure 3 As shown, the specific parameter settings for the 2D max pooling layer are as follows: the pooling window size is set to 2, the stride is 2, and there is no padding by default. The parameter settings for the fully connected layer are as follows. Figure 4 As shown.
[0152] The method described in this embodiment improves the accuracy and precision of anomaly detection compared to a traditional convolutional neural network with the same parameters. The traditional convolutional neural network refers to the one described in this invention that removes the historical fully connected layers. The comparison results are as follows: Figure 5 As shown.
[0153] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. An anomaly detection method for industrial control systems based on a memory convolutional neural network, characterized in that, Includes the following steps: S1. During the training data acquisition phase, network traffic is collected in real time from the industrial control system, and corresponding deep industrial control protocol parsing is performed. A set of sensor and actuator data reflecting the operating status of the entire industrial control system is obtained at all times, and an industrial control anomaly detection training dataset is constructed based on the periodically collected sensor and actuator data; S2. Establish an anomaly detection model based on hierarchical adaptive temporal feature fusion. Train the anomaly detection model using the industrial control anomaly detection training dataset until the model training loss reaches a set threshold or the maximum number of training iterations is reached, thus completing the anomaly detection model training. The anomaly detection model includes a main feature extraction network and a lightweight embedding network. Adaptive Gating Networks Sequential memory unit , as well as convolutional layers, pooling layers, and fully connected layers containing historical fully connected layers; S3. In the industrial control system anomaly detection phase, network traffic is collected from the industrial control system in real time, and corresponding deep industrial control protocol parsing is performed to obtain sensor and actuator data reflecting the current operating status of the entire industrial control system. The data is input into the trained anomaly detection model, and the anomaly detection result is output. The anomaly detection result is set as follows: the model output of 1 indicates that the industrial control system is operating normally, and the output of 0 indicates that the system is operating abnormally.
2. The anomaly detection method for industrial control systems based on a memory convolutional neural network according to claim 1, characterized in that, The industrial control anomaly detection training dataset constructed in S1 based on periodically acquired sensor and actuator data is as follows: The sensor and actuator data are set based on periodic acquisition. ,in This represents the total number of sensors and actuators in an industrial control system. Set the collection period ,according to Decision on whether the industrial control system is malfunctioning Data tags Value: Normal In case of abnormality ; in each Real-time collection , obtain training samples ; The network traffic of the industrial control system is periodically collected and analyzed until the number of training samples with abnormal data labels reaches a set threshold, thus obtaining the industrial control system anomaly detection training dataset. ,in For training dataset The total number of samples, .
3. The anomaly detection method for industrial control systems based on memory convolutional neural networks according to claim 1, characterized in that, The specific steps in S2 are as follows: S21: From the training dataset Training samples are selected sequentially. Training samples Includes data vector Dimension Extract high-level abstract features, low-level alignment features, and temporal memory features; S22: Perform a weighted summation of the three features to obtain an adaptive temporal fusion sample with the same dimension as the original input. ; S23: Will The input convolutional layer performs secondary feature extraction, updates the temporal memory, and adjusts the model parameters through the loss function and backpropagation algorithm. S24: Repeat steps S21 to S23 until the model training loss reaches the set threshold or the maximum number of training iterations is reached, and the model training is completed.
4. The anomaly detection method for industrial control systems based on memory convolutional neural networks according to claim 3, characterized in that, S21 specifically refers to: S211: Transfer data vector Inputting the front-end feature extraction layer of the main feature extraction network yields high-level abstract features. , dimension ; S212: Transfer data vector Independent Input Lightweight Embedded Networks To obtain low-level alignment features , dimension , used to achieve feature space alignment; S213: Acquiring Temporal Memory Features :like It is the training dataset. The first sample, then Let it be a zero vector (dimension) If it is not the first sample, then from the sequential memory unit. Retrieve the temporal memory retained from the previous moment. (dimension) ).
5. The anomaly detection method for industrial control systems based on a memory convolutional neural network according to claim 3, characterized in that, S22 specifically refers to: S221: Will , and Channel concatenation is performed to obtain the initial fusion vector. , dimension ;Will Input Adaptive Gating Network This yields three sets of original weights at different dimensions. Each group of weight dimensions is ; S222: Adjusting the original weights Perform Softmax normalization to obtain the attention weight vector. ,satisfy ; S223: Through element-wise multiplication ( The fused sample is obtained by weighted summation. The formula is: The fusion sample The dimension is .
6. The anomaly detection method for industrial control systems based on memory convolutional neural networks according to claim 1, characterized in that, S223 obtained the fusion sample After the fused samples are input into the convolutional layer for secondary feature extraction, a dynamic noise suppression and temporal consistency verification step is also included, specifically: Dynamic noise suppression filters random noise in sensor / actuator data based on time-series statistical characteristics: Calculate the temporal variance of the effective features at the three most recent moments in the fused sample and the historical memory set. The formula is: in, The number of historical fusion samples used in the calculation ( ), The average of the historical fused samples and the current fused samples; Set noise threshold Threshold filtering is performed on the feature values of each dimension in the fused sample: if the feature value of a certain dimension is equal to... The absolute value of the difference in corresponding dimensions Then use Replace the feature value with the corresponding dimension value to obtain the denoised fused sample; Timing consistency verification ensures that feature changes conform to the physical laws of the industrial control system. Calculate the cosine similarity between the denoised fused sample and the denoised fused sample from the previous time step. The formula is: in," " is the vector dot product, Let L2 be the norm of the vector; like , The preset consistency threshold is obtained based on the time-series correlation statistics of normal industrial control data. If the current sample is deemed to have an abnormal mutation risk, the denoised fused sample is then subjected to weighted smoothing with the denoised fused sample from the previous time step, using the following formula: ； like Then directly order ; The fused samples, after noise suppression and temporal consistency verification, will then be input into the convolutional layer for secondary feature extraction.
7. The anomaly detection method for industrial control systems based on a memory convolutional neural network according to claim 1, characterized in that, Lieutenant General S23 The secondary feature extraction process for input convolutional layers specifically includes: S231: Set the dimension of the input matrix of the convolutional layer to be... ,in , It is a rounding function; fusion samples Perform data reorganization, sequentially... The data is filled into the input matrix; if the number of data bits is insufficient, zeros are filled into the input matrix. The padded input matrix is fed into a convolutional layer for feature extraction, compressed by a pooling layer, and then fed into a historical fully connected layer for feature integration.
8. The anomaly detection method for industrial control systems based on a memory convolutional neural network according to claim 7, characterized in that, S23 also includes the following: S232: Extract new temporal memories from the output of the historical fully connected layer. The As the next training sample Sequential memory This is for use in feature fusion at the next moment; S233: The loss for a single training sample is calculated using the binary cross-entropy loss function, with the following formula: In the formula, For the first Data labels for each training sample Predict the probability of the positive class for the model; S234: Adjust the model parameters using the backpropagation algorithm, the formula is: In the formula, For the model parameters to be adjusted, These are the model parameters before adjustment. This is the learning rate.
9. The anomaly detection method for industrial control systems based on a memory convolutional neural network according to claim 7, characterized in that, Step S234 also includes an adaptive learning rate adjustment step based on the training loss trend, specifically: Set the sliding window size Record the training loss for each batch within the window and calculate the rate of change of loss within the window. : In the formula, For the first Training loss per batch, Reflecting the changing trend of losses ( This indicates a decrease in losses. (Indicates that losses have stagnated or increased). Set learning rate adjustment threshold and , For the threshold of rapid loss decline To adjust the learning rate adaptively to the loss stagnation threshold : like Maintain the current learning rate Keep it unchanged to avoid reducing it too early, which would slow down convergence. like Adjust the learning rate to Gradually decrease the learning rate to achieve stable convergence; like Adjust the learning rate to To avoid getting trapped in local optima; The adjusted learning rate is substituted into the backpropagation algorithm formula for updating the model parameters in the current batch, and the above adjustment steps are repeated for the next batch.
10. An anomaly detection system for an industrial control system based on a memory convolutional neural network, characterized in that, It includes a data acquisition module, a model training module, and an anomaly detection module; The data acquisition module is used to collect network traffic from the industrial control system in real time during the training data acquisition phase, and to perform corresponding deep industrial control protocol parsing. A set of sensor and actuator data reflecting the operating status of the entire industrial control system is obtained at all times, and an industrial control anomaly detection training dataset is constructed based on the periodically collected sensor and actuator data; The model training module is used to build an anomaly detection model based on a memory convolutional neural network. The anomaly detection model is trained according to the industrial control anomaly detection training dataset until the training loss of the anomaly detection model reaches a set threshold or the maximum number of training iterations is reached, thus completing the anomaly detection model training. The anomaly detection module is used to collect network traffic from the industrial control system in real time during the anomaly detection phase of the industrial control system, and perform corresponding deep industrial control protocol parsing to obtain sensor and actuator data that reflects the current operating status of the entire industrial control system. The sensor and actuator data are then input into the anomaly detection model, and the anomaly detection results are output.