Industrial control anomaly detection and classification method based on TransCNN and data equalization strategy

Abstract

The invention belongs to the technical field of industrial network traffic anomaly detection, and discloses an industrial control anomaly detection and classification method based on a TransCNN and a data balance strategy. Carrying out anomaly detection through an improved transfomer model, and dividing the data into abnormal data and normal data; based on a data equalization strategy, high-quality generated data and normal data after undersampling are generated and form a sample equalization data set together with real data, and abnormal data classification is performed through a Light GBM algorithm. According to the method, the representation of each layer of the deep learning model is integrated through hierarchical feature fusion, so that the loss of low-layer information is avoided. A divergence regularization strategy is introduced, the independence of an attention head is enhanced, a one-dimensional convolution layer is adopted to accurately process spatial features, and the space-time complexity is effectively captured. In order to solve the problem of data imbalance, the invention provides a data balancing strategy, through undersampling normal data and oversampling abnormal data, the classification precision is improved, confusion is reduced, and the overall classification performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial control traffic anomaly detection, and in particular to an industrial control anomaly detection and classification method based on TransCNN and a data balancing strategy. Background Art

[0002] To address industrial control system security issues, researchers use anomaly detection technology to protect systems. Anomaly detection is a proactive protection technique that monitors and identifies anomalous behavior within and outside the system in real time. By employing security configuration strategies, anomaly detection captures the current system status, detects malicious traffic, and gathers attack information, thereby ensuring the availability and integrity of the entire industrial control system.

[0003] Early research on anomaly detection in industrial control systems primarily employed rule-based or threshold-based methods. These methods can be categorized as state estimation-based, least squares-based, hidden Markov model-based, and Kalman filter-based. These methods assume that all system states are measurable. With the rapid development of artificial intelligence (AI), anomaly detection in industrial control systems is increasingly leveraging machine learning and deep learning methods to improve detection accuracy and efficiency.

[0004] The paper "Tang C, Xu L, Yang B, et al. GRU-based interpretable multivariate time series anomaly detection in industrial control system[J].Computers & Security, 2023, 127: 103094." proposes an interpretable multivariate time series anomaly detection method (GRN) based on gated recurrent units and graph neural networks. This method aims to improve the performance and interpretability of anomaly detection in industrial control systems. On a real-world water treatment dataset, the GRN demonstrates superior performance compared to nine other state-of-the-art algorithms in both detection precision and recall. The proposed GRN model demonstrates superior performance in both detection precision and recall. However, it cannot efficiently parallelize computations when processing time series data.

[0005] The paper "Wang T, Zhang L, Wang X. Fault detection for motor drive control system of industrial robots using CNN-LSTM-based observers[J]. CES Transactions on Electrical Machines and Systems, 2023." proposes an anomaly detection technology for industrial robot drive control systems. This technology leverages the CNN's ability to extract spatiotemporal features and the LSTM's strengths in processing long-term dependencies in time series to improve the accuracy and efficiency of anomaly detection. However, due to the complexity and training cost of the proposed model, this method is difficult to apply to practical industrial control scenarios and also lacks parallel computing capabilities.

[0006] The paper "Ma M, Fu L, Zhai Z, et al. Transformer-based Kalman Filter with EM algorithm for time series prediction and anomaly detection of complex systems[J]. Measurement, 2024: 114378." proposes a novel model structure that combines the Transformer with the Kalman filter and employs an expectation-maximization algorithm to estimate the hidden parameters of the Kalman filter, thus avoiding the heavy reliance on expert experience in traditional methods. However, this model lacks special treatment for the multi-head self-attention mechanism, resulting in a problem in which each attention head tends to capture similar features, thereby reducing the overall representational power of the model.

[0007] In most real-world industrial control scenarios, data imbalance is often significant. This extreme imbalance can cause classification algorithms to misclassify abnormal data as normal, severely impacting the accuracy of minority class recognition. To address this challenge, researchers have employed some traditional methods to address data imbalance.

[0008] The paper "Talukder MA, Hasan KF, Islam MM, et al. A dependable hybrid machine learning model for network intrusion detection[J]. Journal of Information Security and Applications, 2023, 72: 103405." proposed a SMOTE algorithm to balance abnormal samples in network intrusion datasets, and proposed a hybrid model that combines multiple machine learning techniques to solve the limitations of traditional single models in complex attack scenarios.

[0009] The paper "Li X, Yue C, Liu X, et al. ACWGAN-GP for milling tool breakage monitoring with imbalanced data[J]. Robotics and Computer-Integrated Manufacturing, 2024, 85: 102624." proposes a tool breakage monitoring method based on ACWGAN-GP specifically for data imbalance. This method enhances training stability by introducing Wasserstein distance and gradient penalty terms, effectively generating fault samples from a variety of categories.

[0010] The paper "Talukder MA, Hasan KF, Islam MM, et al. A dependable hybrid machine learning model for network intrusion detection[J]. Journal of Information Security and Applications, 2023, 72: 103405." proposed the SMOTE oversampling technology, which mainly learns based on the local neighborhood of the sample points and fails to fully consider the overall distribution characteristics of the minority class. Therefore, the data generated by this method is difficult to effectively capture and fit the true distribution of the minority class, which in turn affects the authenticity of the generated samples and the generalization ability of the model.

[0011] The paper "Li X, Yue C, Liu X, et al. ACWGAN-GP for milling tool breakage monitoring with imbalanced data[J]. Robotics and Computer-IntegratedManufacturing, 2024, 85: 102624." proposed a method that successfully improved the stability of the adversarial training process, but did not adopt a targeted undersampling strategy for the majority class, and therefore failed to fully address the imbalance problem of the dataset.

[0012] Based on the above research, anomaly detection algorithms and data balancing strategies face three major challenges: 1) They fail to effectively address the issues of insufficient spatiotemporal information integration and inter-layer information bias in industrial control anomaly detection; 2) they fail to effectively address the problem of unbalanced industrial control data leading to suboptimal anomaly classification; and 3) they fail to effectively address the quality stability of data generated by generative adversarial networks. Summary of the Invention

[0013] This paper studies anomaly detection and data imbalance processing methods, proposing an industrial control anomaly detection and classification method based on TransCNN and a data balancing strategy. This method integrates the representations of each layer of the deep learning model through hierarchical feature fusion to avoid loss of low-level information. A divergence regularization strategy is introduced to enhance the independence of attention heads, and a one-dimensional convolutional layer is used to accurately process spatial features and effectively capture spatiotemporal complexity. To address the data imbalance problem, this paper proposes a data balancing strategy that improves classification accuracy and reduces confusion by undersampling normal data and oversampling abnormal data, thereby enhancing overall classification performance.

[0014] The technical solution of the present invention is as follows: an industrial control anomaly detection and classification method based on TransCNN and data balancing strategy, which performs anomaly detection through an improved transfomer model and divides the data into abnormal data and normal data; based on the data balancing strategy, high-quality generated data and under-sampled normal data are generated, and together with the real data form a sample balanced data set, and abnormal data classification is performed through the LightGBM algorithm.

[0015] Furthermore, the data balancing strategy is specifically as follows: normal data is undersampled using the nearest neighbor cleaning method to obtain undersampled normal data; abnormal data is oversampled using the improved ACGAN algorithm to generate high-quality generated data.

[0016] Furthermore, the nearest neighbor cleaning method is as follows: calculating the distance between the majority class samples of normal data and other samples, identifying and deleting the majority class samples in the category overlapping area, and obtaining under-sampled normal data.

[0017] Furthermore, the improved ACGAN algorithm is specifically as follows:

[0018] Prepare a training set, including real samples and their labels; randomly initialize the discriminator parameters , generator parameters ;

[0019] Perform a set number of iterative trainings; randomly extract real samples and their corresponding labels, …, , the label is …, ; Random collection latent vector …, , generate fake samples through the generator = …, ;

[0020] The fake samples and real samples are input into the discriminator together, and the discriminator determines and Authenticity and labeling, authenticity judgment , , category label judgment ;

[0021] Calculate the discriminative loss , classification loss And the distance loss function , generate mixed samples; get the loss function of the improved ACGAN algorithm ;

[0022]

[0023]

[0024]

[0025]

[0026]

[0027] in is the loss function of the original generator, is a weight function that adjusts information loss;

[0028] Update parameters through loss function to optimize generator and discriminator;

[0029] Evaluate the similarity between generated data and real data, and filter out data with low similarity; the evaluation indicators are;

[0030]

[0031]

[0032]

[0033] is a measure of the similarity between the probability distributions of generated data and true data, Indicates two dimensional vector and The actual distance between , Respectively represent True samples of generated samples;

[0034] Retention satisfaction and Generated data; , The thresholds corresponding to the two indicators respectively;

[0035] Improve the discriminator part of the ACGAN algorithm to output a comprehensive classification probability vector ,in Indicates that the generated sample belongs to The probability of the categories, is the total number of categories; I(p) is the probability of a certain category; when Retain generated data; is the set threshold, and finally a high-quality generated dataset is generated.

[0036] Furthermore, the anomaly detection process is as follows:

[0037] Split the input time series into different time window forms of data, and get ; Use the sliding window mechanism to create overlapping windows, and the length of each window is set to , and the step size between windows is 1, the length is sequence Input into the improved transformer model to obtain the corresponding feature representation layer;

[0038] The feature representation layer obtains the predicted time series through the 1D CNN decoder ;

[0039] Calculate the prediction error for each timestamp to get the anomaly score , set the threshold s. When the anomaly score is greater than the threshold, the data at this time point in the sequence is considered abnormal data;

[0040] .

[0041] Furthermore, the improved transformer model includes N layers of transformer encoder layers, and each layer of transformer encoder layer obtains a corresponding feature representation. ,

[0042]

[0043]

[0044] The feature representations of each Transformer encoder layer are stacked in the original order to construct a feature representation layer that integrates the feature information of each layer; the formula is as follows:

[0045]

[0046] Furthermore, the Transformer encoder layer includes a multi-head self-attention layer and a fully connected network layer;

[0047] The multi-head self-attention layer includes multiple self-attention mechanism modules, each of which independently processes input information and generates output results. The output data of all self-attention mechanism modules are spliced ​​and integrated through a fully connected network layer to form the final output;

[0048] The calculation formula for obtaining the attention score matrix through the self-attention mechanism is as follows;

[0049]

[0050] Represents the dimension of the key vector sequence;

[0051] The query matrix , key matrix , value matrix The formula for the attention score matrix input to the respective attention mechanism modules is as follows;

[0052]

[0053] 、 、 and Represents the parameter matrix of the fully connected layer; the key vector dimension satisfy ,in represents the unified input dimension of the Transformer model, represents the number of attention mechanism modules, represents the self-attention head;

[0054] To alleviate the problem of different self-attention heads learning similar features, a divergence regularization method is adopted to change the feature representation of the input data in the multi-head attention, so that different self-attention heads can capture the diverse features of the data; subspace divergence regularization is performed to maximize the cosine distance between the subspaces input to different self-attention heads;

[0055] Subspace divergence The calculation process is shown in the formula;

[0056]

[0057] Perform weight divergence regularization to maximize the different self-attention heads The difference between them is used to disperse the positions of attention of different self-attention heads; weight divergence The calculation process is as follows;

[0058] .

[0059] Furthermore, the 1D CNN generates the same output dimension as the Transformer layer The same feature map, learning to generate follows the encoder output dimension of , this process is shown in the formula;

[0060]

[0061] Through the fully connected layer, Mapping becomes equal to the dimension of the input sequence -dimensional vector , The specific calculation process is shown in the formula;

[0062] .

[0063] Beneficial effects of the present invention: The present invention proposes an industrial control anomaly detection and classification method based on TransCNN and data balancing strategy:

[0064] (1) A TransCNN-based industrial control anomaly detection algorithm is designed. By improving the encoder part of the TransCNN model, an improved transfomer model is obtained. The output of the encoder layer is stacked to achieve information integration from the bottom to the top layer, strengthen the integration of spatiotemporal information, and introduce a divergence regularization strategy to improve the independence of the attention mechanism, effectively capture complex features, and achieve accurate anomaly detection.

[0065] (2) An imbalanced data anomaly classification algorithm based on data balancing strategy is proposed. The nearest neighbor cleaning method is used to undersample normal data, and the improved auxiliary classification adversarial network is combined to oversample abnormal data to reduce the interference of the majority class on the classification effect, improve the classification accuracy, and enhance the discrimination of the overlapping areas of the dataset.

[0066] (3) Carefully screen the generated samples to ensure that the generated data is similar to the real data in statistical characteristics, and give priority to retaining high-quality samples, thereby improving the stability and reliability of the generated data. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is the overall flow chart of the anomaly detection part;

[0068] Figure 2 Generate a flow chart for data based on data balancing strategy;

[0069] Figure 3 This is the overall framework diagram of the anomaly detection part;

[0070] Figure 4 The accuracy graph of different classification algorithms. DETAILED DESCRIPTION

[0071] The specific implementation of the present invention is further described in detail below with reference to the accompanying drawings and examples.

[0072] This paper studies anomaly detection and data imbalance processing methods, proposing an industrial control anomaly detection and classification method based on TransCNN and a data balancing strategy. This method integrates the representations of each layer of the deep learning model through hierarchical feature fusion to avoid loss of low-level information. A divergence regularization strategy is introduced to enhance the independence of attention heads, and a one-dimensional convolutional layer is used to accurately process spatial features and effectively capture spatiotemporal complexity. To address the data imbalance issue, a data balancing strategy is proposed. By undersampling normal data and oversampling abnormal data, classification accuracy is improved, confusion is reduced, and overall classification performance is enhanced.

[0073] This paper proposes an industrial control anomaly detection and classification method based on TransCNN and data balancing strategy. Figure 1 and Figure 2 As shown, the industrial control anomaly detection and classification method based on TransCNN and data balancing strategy of the present invention includes the following steps:

[0074] Step 1: Preprocess the data input to the anomaly detection module and convert the given time series data into a form suitable for continuous time series prediction. This is to split the input time series into different windows, and then process the windowed data to obtain The specific steps are:

[0075] Step 1.1. Remove these warm-up data from the dataset.

[0076] Step 1.2: Normalize all features. The processing formula is as follows:

[0077]

[0078] in, is the current value, is the minimum value of the eigenvalue, is the maximum value of the eigenvalue, is the normalized value.

[0079] Step 1.3: Position-encode the data and split it into different windows.

[0080] Step 2: Set the length to sequence Input to the encoder part of the improved TransCNN model to obtain the corresponding feature representation layer. Step 2.1, use the query-key-value (QKV) model to process the input sequence.

[0081] Adopting a multi-head self-attention mechanism Pattern for input sequence For processing, the query matrix, key matrix and value matrix are 、 、 , the output sequence is The calculation formula for obtaining the attention score matrix through the self-attention mechanism is as follows

[0082]

[0083] Represents the dimensionality of the key vector sequence.

[0084] Step 2.2, calculate the attention score matrix

[0085] Will The input is fed into multiple self-attention mechanism modules. Each self-attention mechanism module processes the input information independently and generates output results. The output data of all self-attention mechanism modules are concatenated and integrated through a fully connected network layer to form the final output. The attention score matrix of each attention mechanism module is shown below

[0086]

[0087] 、 、 and Represents the parameter matrix of the fully connected layer. Key vector dimension satisfy ,in represents the unified input dimension of the Transformer model, represents the number of attention mechanism modules, represents an attention head.

[0088] Step 2.3: Use a masked multi-head self-attention mechanism to ensure that the information of future timestamps is not used when processing the current timestamp data.

[0089] Step 2.4: To alleviate the problem of self-attention heads learning similar features, adopt divergence regularization. In subspace divergence regularization, maximize the cosine distance between the subspaces of inputs to different self-attention heads.

[0090] Subspace divergence The calculation process is shown in the formula.

[0091]

[0092] Perform weight regularization to maximize the different self-attention heads The difference between them is used to disperse the locations that different heads focus on. The calculation process is shown as follows.

[0093]

[0094] The above is after one encoder layer. After the time window sequence enters the encoder, it needs to pass through N layers of Transformer encoder layers. Each time it passes through an encoder layer, a corresponding feature representation will be obtained. , the specific process is shown in the formula.

[0095]

[0096]

[0097] Step 2.5: Stack the feature representations of each layer in the original order to construct the final output representation that integrates the feature information of each layer. The following is the public statement:

[0098]

[0099] Step 3: Input the hierarchical fusion feature representation into the 1D CNN decoder to obtain the predicted time series .

[0100] Step 3.1: 1D CNN can generate the same output dimension as the Transformer layer The same feature map, learning to generate follows the encoder output dimension of , this process is shown in the formula.

[0101]

[0102] Step 3.2, through the fully connected layer, Mapping becomes equal to the dimension of the input sequence -dimensional vector , The specific calculation process is shown in the formula.

[0103]

[0104] Step 4: Calculate the anomaly score. Calculate the prediction error for each timestamp to get the anomaly score , the formula is as follows, and the anomaly score is used to determine whether it is an anomaly.

[0105]

[0106] Step 5: Input the obtained abnormal data and normal data into the data balancing module to obtain a relatively balanced data set, and use the nearest neighbor cleaning method to undersample the normal data.

[0107] Step 6: Oversample the data using the oversampling technique of the improved ACGAN algorithm.

[0108] Step 6.1: Prepare the training set, including real samples and their labels. And randomly initialize the discriminator parameters. , generator parameters .

[0109] Step 6.2: Perform a set number of iterative trainings. Randomly extract real samples and their corresponding labels. …, , their labels are …, Random collection latent vector …, . Generate fake samples through the generator = …,

[0110] Step 6.3: Input the fake sample and the real sample into the discriminator, and the discriminator determines and True and false and label, true and false judgment , , category label judgment .

[0111] Step 6.4, calculate the discriminant loss , classification loss , distance loss function , generate mixed samples, and obtain the loss function of the improved ACGAN algorithm The formula is as follows:

[0112]

[0113]

[0114]

[0115]

[0116] in It is the loss function of the original generator, is a weight function that adjusts information loss.

[0117] Step 6.5: Update the parameters of the generator and discriminator through the loss function, and finally generate the expanded training set.

[0118] Step 7: Use the LightGBM classification algorithm to classify the balanced data into abnormalities.

[0119] Step 7.1: Based on the balanced data set, collect and process industrial control system data to ensure data quality.

[0120] Step 7.2: Use the histogram algorithm to discretize the eigenvalues ​​and construct a histogram to determine the optimal splitting point.

[0121] Step 7.3: Apply the gradient-based single-side sampling (GOSS) technique to retain high-gradient samples and randomly sample low-gradient samples to improve training efficiency.

[0122] Step 7.4: Use the exclusive feature bundling (EFB) technique to retain high-gradient samples and randomly sample low-gradient samples to reduce feature dimensions and improve model performance.

[0123] Step 7.4: Implement the leaf-based growth strategy and select the leaf node with the highest gain for splitting to improve the model training speed and efficiency.

[0124] Step 7.5: Train the LightGBM model, use the balanced data for anomaly classification, and optimize the model parameters to improve accuracy.

[0125] This example primarily utilizes the SWaT dataset and the BATADAL dataset. The SWaT dataset is one of the most commonly used public datasets in the field of industrial control system security. This dataset collects actual operational data from a modern water treatment system, documenting normal operation, abnormal conditions, and attack events. Data is collected at a rate of once per second, covering 11 consecutive days of data from 51 sensors. During this period, 496,800 samples were collected in the first seven days, reflecting normal system operation; while 449,919 samples were collected in the last four days, documenting attacks against the system. The dataset records 36 different attacks, resulting in approximately 12% of anomalous data samples. The BATADAL dataset, collected from the industrial control environment of a water supply system, documents both normal operation and external attacks. The dataset can be divided into two main parts: the first part includes twelve months of normal operation data, collected hourly, totaling 8,762 data samples; the second part records five months of operational data, including attack events, totaling 4,178 data points. In the second half of the test, the system encountered seven attacks of varying durations, which can be categorized into four types based on their mechanisms. These security incidents resulted in 11.8% of the data being labeled as abnormal samples, reflecting the system's response to different attack scenarios.

[0126] Step 1: Preprocess the dataset

[0127] Non-stationary data processing. Choose to delete the first 100,000 data in the data set. Choose to delete the first 100,000 data in the data set;

[0128] Using data normalization technology, the range of all eigenvalues ​​is uniformly adjusted to [0,1]. The specific calculation formula is as follows:

[0129]

[0130] in, is the current value, is the minimum value of the eigenvalue, is the maximum value of the eigenvalue, is the normalized value.

[0131] Step 2: Introduce evaluation metrics. Use precision to measure the model's ability to accurately predict system anomalies, recall to assess the model's ability to identify all anomalies, and F1-Score to evaluate the combined performance of the two, thereby measuring the model's overall anomaly detection performance. The specific calculation formulas for precision, recall, and F1-Score are shown below.

[0132]

[0133]

[0134]

[0135]

[0136] Step 3: Initialize the TransCNN model and verify the anomaly detection effect on the SWaT dataset and BATADAL dataset. The TransCNN model architecture is shown in the figure below. Figure 3 shown.

[0137] Determine the sliding window size and anomaly score threshold. Set the sliding window size to 10, the anomaly score to 0.8, and the sliding window step size to 1.

[0138] Perform model training. Input the processed time series into the Transformer encoder to obtain feature representation. The formula is:

[0139]

[0140]

[0141] The stacked representation features are input into the 1D CNN decoder and the prediction sequence is generated through the fully connected layer

[0142] Iterate and continuously optimize the parameters in the TransCNN model.

[0143] The generated prediction sequence is input into the anomaly judgment module, and the anomaly score is calculated for judgment. The formula is as follows;

[0144]

[0145] Step 5: Verify the anomaly classification effect on the SWaT dataset and BATADAL dataset.

[0146] The original data is divided into training set and test set in a ratio of 7:3.

[0147] Calculate the imbalance rate in the dataset using the following formula and optimize the imbalance rate in the dataset.

[0148]

[0149] in, represents the number of majority classes, represents the number of minority classes, The larger the value, the greater the data imbalance rate.

[0150] In the SWAT dataset, we used undersampling to reduce the number of normal data to 40,000. Oversampling was used for the other four types of abnormal data to enhance the diversity and representativeness of the dataset. Specifically, the number of abnormal data for the SSSP type was increased to 40,000, while the number of data for the MSMP, MSSP, and SSMP types was increased to 20,000, respectively; as shown in Table 1.

[0151] Table 1 SWaT data distribution

[0152]

[0153] In the BATADAL dataset, a data balancing model is used to undersample the normal data, retaining only 3,000 normal data items. Then, the other four types of abnormal data items are oversampled to expand the data to 1,000 items, as shown in Table 2.

[0154] Table 2 BATADAL data distribution

[0155]

[0156] The abnormal data is oversampled using the nearest neighbor cleaning method.

[0157] The ACGAN model is used to perform undersampling on normal data.

[0158] Normal data enters the filter for processing. Generating data requires satisfying three conditions: KL divergence and Euclidean distance and , the formula is as follows:

[0159]

[0160]

[0161] in , Respectively represent True samples of generated samples, , They correspond to the thresholds of the two indicators respectively.

[0162]

[0163] Where p is the discriminator of ACGAN outputs a comprehensive classification probability vector , w3 is the corresponding threshold.

[0164] Experimental settings: The batch size is set to 64, and the total number of training rounds is 2000. The Adam optimizer is used with a learning rate of 0.0005. In addition, the classification threshold of the discriminator is set to 0.8.

[0165] Use the LightGBM model for iterative training and output the results of abnormal classification.

[0166] The LightGBM algorithm is compared with other commonly used classification algorithms, including decision tree, logistic regression, naive Bayes and random forest. In these experiments, the imbalance rate is set to 3. The experimental results are shown in Figure 2. Figure 4 The experimental results clearly show that the LightGBM algorithm performs best in both datasets, demonstrating its superiority in handling imbalanced data anomaly classification tasks. Therefore, this study selected LightGBM as the anomaly classification algorithm after data balancing to further improve classification accuracy.

[0167] To verify the effectiveness of the proposed method, a comparative experimental analysis was conducted using commonly used multivariate time series anomaly detection models, including CNN-LSTM, GDN, DAGMM, MAD-GAN, and Omni Anomaly, by demonstrating the superiority of the TransCNN-based anomaly detection model. The experiments primarily compared the Precision, Recall, and F1 scores of these models on the SWaT and BATADAL datasets. The experimental results are shown in Table 3. The experimental results show that the TransCNN-based anomaly detection model achieved the best F1 scores on both the SWaT and BATADAL datasets, particularly achieving the highest F1 score and the best recall rate on the SWaT dataset. These results demonstrate TransCNN's outstanding ability to process complex data, particularly in enhancing the perception and recognition of abnormal states. By effectively fusing spatiotemporal information with inter-layer information, TransCNN is able to comprehensively capture the global features and long-term dependencies of the data, providing an efficient strategy for the security protection of industrial control systems.

[0168] Table 3 Comparison of anomaly detection experimental results

[0169]

Claims

1. A method for industrial control anomaly detection and classification based on TransCNN and data balancing strategy, characterized by: Anomaly detection is performed through the improved transfomer model, which is divided into abnormal data and normal data. Based on the data balancing strategy, high-quality generated data and under-sampled normal data are generated to form a sample balanced data set together with the real data, and abnormal data is classified using the LightGBM algorithm.

2. The industrial control anomaly detection and classification method based on TransCNN and data balancing strategy according to claim 1 is characterized in that: The data balancing strategy is specifically as follows: normal data is undersampled using the nearest neighbor cleaning method to obtain undersampled normal data; abnormal data is oversampled using the improved ACGAN algorithm to generate high-quality generated data.

3. The industrial control anomaly detection and classification method based on TransCNN and data balancing strategy according to claim 2 is characterized in that: The nearest neighbor cleaning method is as follows: calculating the distance between the majority class samples of normal data and other samples, identifying and deleting the majority class samples in the class overlapping area, and obtaining under-sampled normal data.

4. The industrial control anomaly detection and classification method based on TransCNN and data balancing strategy according to claim 2 is characterized in that: The improved ACGAN algorithm is specifically as follows: Prepare a training set, including real samples and their labels; randomly initialize the discriminator parameters , generator parameters ; Perform a set number of iterative trainings; Randomly extract real samples and their corresponding labels, …, , the label is …, ; Random collection latent vector …, , generate fake samples through the generator = …, ; The fake samples and real samples are input into the discriminator together, and the discriminator determines and Authenticity and labeling, authenticity judgment , , category label judgment ; Calculate the discriminative loss , classification loss And the distance loss function , generate mixed samples; Get the loss function of the improved ACGAN algorithm ; ， ， ， ， ,in is the loss function of the original generator, is a weight function that adjusts information loss; Update parameters through loss function to optimize generator and discriminator; Evaluate the similarity between generated data and real data, and filter out data with low similarity; the evaluation indicators are; ， ， , is a measure of the similarity between the probability distributions of generated data and true data, Indicates two dimensional vector and The actual distance between , Respectively represent True samples of generated samples; Retention satisfaction and Generated data; , The thresholds corresponding to the two indicators respectively; Improve the discriminator part of the ACGAN algorithm to output a comprehensive classification probability vector ,in Indicates that the generated sample belongs to The probability of the categories, is the total number of categories; I(p) is the probability of a certain category; when Retain generated data; is the set threshold, and finally a high-quality generated dataset is generated.

5. The industrial control anomaly detection and classification method based on TransCNN and data balancing strategy according to claim 1 is characterized in that: The anomaly detection process is as follows: Split the input time series into different time window forms of data, and get ; Use the sliding window mechanism to create overlapping windows, and the length of each window is set to , and the step size between windows is 1, the length is sequence Input into the improved transformer model to obtain the corresponding feature representation layer; The feature representation layer obtains the predicted time series through the 1D CNN decoder ; Calculate the prediction error for each timestamp to get the anomaly score , set the threshold s. When the anomaly score is greater than the threshold, the data at this time point in the sequence is considered abnormal data; 。 6. The industrial control anomaly detection and classification method based on TransCNN and data balancing strategy according to claim 5 is characterized in that: The improved transformer model includes N layers of Transformer encoder layers. Each time a layer of Transformer encoder layer is passed, a corresponding feature representation is obtained. , ， , the feature representations of each Transformer encoder layer are stacked in the original order to construct a feature representation layer that integrates the feature information of each layer; the formula is as follows: where stack represents the stacking operation in the original order; 。 7. The industrial control anomaly detection and classification method based on TransCNN and data balancing strategy according to claim 6 is characterized in that: The Transformer encoder layer includes a multi-head self-attention layer and a fully connected network layer; The multi-head self-attention layer includes multiple self-attention mechanism modules, each of which independently processes input information and generates output results. The output data of all self-attention mechanism modules are spliced ​​and integrated through a fully connected network layer to form the final output; The calculation formula for obtaining the attention score matrix through the self-attention mechanism is as follows; ， Represents the dimension of the key vector sequence; The query matrix , key matrix , value matrix The formula for the attention score matrix input to each attention mechanism module is as follows; , 、 、 and Represents the parameter matrix of the fully connected layer; the key vector dimension satisfy ,in represents the unified input dimension of the Transformer model, represents the number of attention mechanism modules, represents the self-attention head; To alleviate the problem of different self-attention heads learning similar features, a divergence regularization method is adopted to change the feature representation of the input data in the multi-head attention, so that different self-attention heads can capture the diverse features of the data; subspace divergence regularization is performed to maximize the cosine distance between the subspaces input to different self-attention heads; Subspace divergence The calculation process is shown in the formula; , perform weight divergence regularization to maximize the different self-attention heads The difference between them is used to disperse the positions of attention of different self-attention heads; weight divergence The calculation process is as follows; 。 8. The industrial control anomaly detection and classification method based on TransCNN and data balancing strategy according to claim 5 is characterized in that: The 1D CNN generates the output dimension of the Transformer layer The same feature map, learning to generate follows the encoder output dimension of , this process is shown in the formula; , through the fully connected layer, Mapping becomes equal to the dimension of the input sequence -dimensional vector , The specific calculation process is shown in the formula; 。

Images