Multi-level industrial control system anomaly detection method based on dual models

Abstract

The invention provides a multi-level industrial control system anomaly detection method based on dual models. According to the method, normal training set data of an industrial field public data set is used to parallelly and independently train a reconstruction model and a prediction model, so that the model learns a potential feature mode, the trained model can output a reconstruction error and a prediction error to a verification set, then an attention mechanism is introduced into a fusion layer, and a fusion model is obtained. And the reconstruction error and the prediction error are dynamically weighted, so that a comprehensive anomaly score is calculated, and whether an anomaly occurs at the current moment is judged by judging the relationship between the score and a threshold value.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial control monitoring, and in particular to a multi-level industrial control system anomaly detection method based on a dual model. Background Art

[0002] Industrial control systems (ICS) are being upgraded towards high levels of automation and intelligence in modern manufacturing, greatly improving production efficiency and flexibility. However, due to the complexity of the production environment and the driving force of large-scale data, ICS face a wider range of network threats, more failure points, and diverse anomalies. This makes anomaly detection a key challenge in ensuring production safety and efficiency.

[0003] Industrial control systems (ICS), the core hub of industrial automation, encompass a wide range of applications, from factory automation control equipment to critical infrastructure, including power generation, water supply, transportation, and manufacturing. ICS utilize sensors, controllers, and actuators to collaboratively manage complex industrial processes, enabling efficient production and precise control. While deeper integration with IT and internet networks has brought greater production efficiency and flexibility, it has also exposed ICS to numerous risks and challenges, including a wider range of cyber threats, more potential failure points, and a greater variety of abnormal situations.

[0004] To address these challenges, monitoring technology has become a critical component of industrial control system (IC) security. Monitoring technology in ICS encompasses not only network traffic analysis of IT systems but also comprehensive monitoring of physical devices, industrial protocols, production processes, and system behavior. Within ICS, production process monitoring is a key component in ensuring the safety and efficiency of production processes. Production process monitoring focuses on the physical state of industrial equipment. By collecting and analyzing equipment operating parameters in real time, it can promptly detect potential anomalies and prevent production accidents. This not only helps improve product quality and reduce production costs, but also extends equipment life and reduces maintenance costs. Therefore, production process monitoring holds significant practical value in modern industrial production. In the context of the data-driven Industrial Internet, industrial equipment is equipped with numerous sensors, generating vast amounts of real-time data. This provides a rich foundation for data-driven anomaly detection methods. With the rapid development of big data and artificial intelligence technologies, data-driven methods have gained widespread application in anomaly detection. Deep learning, in particular, has become a mainstream approach in anomaly detection research due to its powerful feature extraction and pattern recognition capabilities. Deep learning models can automatically learn deep feature representations from large amounts of complex data, effectively improving the accuracy and robustness of anomaly detection. Furthermore, deep learning methods excel at processing nonlinear, multidimensional, and dynamically changing data, significantly advancing the development of industrial anomaly detection technology.

[0005] Data-driven anomaly detection under the deep learning technology mainly includes two methods: reconstruction-based and prediction-based anomaly detection. Both methods are important in industrial applications and have unique theoretical basis and applicable scenarios.

[0006] Reconstruction-based anomaly detection methods usually learn the pattern of normal data and try to reconstruct the original data from the input data. When the reconstruction error of the data is large, it indicates that the data point may be abnormal. This method is widely used in autoencoders and principal component analysis (PCA) techniques. The limitations of this method are as follows: (1) The model assumes that the normal data has a stable structure, such as periodicity, linear relationship, or specific spatial features. When the real data distribution is complex and has multi-modal characteristics or strong nonlinearity, the capacity and expression ability of the reconstruction model may not be sufficient, causing the reconstruction error to be unstable. (2) Some reconstruction models have difficulties in training and optimization on high-dimensional data, especially in training, it is difficult to ensure that the model only fits the normal data, and in some cases, the model may also be able to reconstruct some abnormal data, thereby reducing the reliability of the method. (3) The reconstruction process and error indicators often lack clear causal or mechanism explanations, and can only be judged from the error value, which lacks interpretability.

[0007] Prediction-based methods usually use time series prediction models or feedforward networks based on process modeling to predict the next time step or a future window. When the predicted value and the true value are significantly different, it is determined to be abnormal. The advantage of this method is that it is suitable for time series-based process monitoring in industrial control systems and can capture dynamic patterns in data. The limitations of this method are as follows: (1) It has high requirements for the scene, and must ensure that the data has time series correlation and predictability. If the input sequence can predict the future state well, the anomaly judgment is not reliable; (2) Similar to the reconstruction method, the prediction model may not converge to a predictor suitable for all states in a highly complex, non-stationary, nonlinear, and multi-modal industrial scene; (3) For complex control processes, the cause of the prediction bias is difficult to trace, and it still lacks interpretability.

[0008] In addition, some studies combine reconstruction models with prediction models, which often simply embed them in the network structure, such as embedding the Long Short-Term Memory (LSTM) as part of the autoencoder into the encoder or decoder structure of the autoencoder. Simply stacking the model increases the complexity, which can lead to overfitting and make it difficult to deploy in industrial environments. There are too many adjustable parameters, and the simple stacking method does not fully utilize the complementary advantages of the two methods, and it is difficult to achieve significantly better results than using each method alone. SUMMARY

[0009] The present invention aims to overcome the aforementioned shortcomings of the prior art by providing a multi-level, dual-model-based method for detecting anomalies in industrial control systems. This approach addresses the instability and inaccuracy of traditional anomaly detection solutions when dealing with multimodal data. This method, centered on an integrated framework, independently trains the reconstruction and prediction models and incorporates an attention mechanism at the fusion layer to fully leverage the detection advantages of both. This approach not only flexibly addresses complex anomaly patterns but also improves detection performance and model interpretability across multiple dimensions.

[0010] To achieve the above object, the present invention adopts the following technical solutions:

[0011] A multi-level industrial control system anomaly detection method based on a dual model includes the following steps:

[0012] S1: Collect the industrial dataset X, clean and standardize X, and obtain the preprocessed dataset X norm , divided into training set, validation set and test set;

[0013] S2: At the same time, use the training set data to conduct parallel training of the reconstruction model and the prediction model to obtain the total loss function of the corresponding model;

[0014] Substitute the validation set data into the total loss function of the trained reconstruction model to obtain the reconstruction error e r ; Use the validation set data to substitute into the total loss function of the trained prediction model to obtain the prediction error e p ;

[0015] S3: Use the attention mechanism to fuse the reconstruction model and the prediction model:

[0016] S31: The reconstruction error e obtained in step S2 is r and the prediction error e p As input features, the feature vector f = [e r ,e p ] T ;

[0017] S32: Generate information Q representing the current focus, K representing the stored information features, and V representing the actual information associated with the key through a feedforward neural network;

[0018] S33: Calculate the fused attention weight α by the following formula and further transform it to generate the comprehensive anomaly score s i :

[0019]

[0020] Where, α rrepresents the attention weight of the reconstruction model, α p represents the attention weight of the prediction model; Q r , K r Corresponding to the reconstruction error e r ;Q p , K p Corresponding to the prediction error e p ;

[0021] s i =σ(α r ·e r +α p ·e p )

[0022] Where σ is the Sigmoid activation function, s i is anomaly score. If the error value is greater than the defined threshold value Υ, the window data point is considered abnormal. The abnormal window data point is represented by F i =1{s i >Υ}.

[0023] In industrial control systems, the multimodality and complexity of data make traditional single-model anomaly detection methods ineffective in addressing diverse abnormal behaviors. Traditional methods are typically based on single models, such as reconstruction models or prediction models. The former primarily focuses on deviations from the overall data structure, while the latter focuses on time series trends. However, single models exhibit instability when faced with anomalous data of varying types and patterns, prone to misjudgments or omissions. Furthermore, data in industrial environments often contain noise and temporal characteristics, further complicating anomaly detection. To this end, this paper proposes a novel anomaly detection framework that employs dual-model training: a reconstruction model (VAE) and a prediction model (GRU). These models learn the underlying distribution and temporal dynamics of normal data, respectively, forming the first layer of detection. Secondly, an attention mechanism is introduced in the second layer to integrate the results of the dual-model training in the first layer. Anomaly scores are generated by dynamically assigning weights to the reconstruction and prediction errors. Analysis of the attention weights also provides rich interpretable information. This research provides an efficient, reliable, and interpretable anomaly detection solution for intelligent monitoring in industrial control systems, effectively promoting the development of intelligent manufacturing and the Industrial Internet.

[0024] The present invention uses the normal training set data of the public data set in the industrial field to independently train the reconstruction model and the prediction model in parallel, so that they can learn the potential feature patterns. The trained model can output the reconstruction error and prediction error for the verification set. Then, the attention mechanism is introduced in the fusion layer to dynamically weight the reconstruction error and prediction error to calculate the comprehensive anomaly score. By judging the relationship with the threshold, it is determined whether an anomaly occurs at the current moment. The key features of the adaptive selection of the anomaly point can be obtained by the attention mechanism, the source of the anomaly can be located, and the anomaly formation mechanism can be explained.

[0025] The complete workflow of the solution of the present invention can be described in the following three stages:

[0026] (1) Data preprocessing: Multidimensional time series data is collected through industrial sensors, and data standardization, sliding window segmentation, and processing of missing values ​​and outliers are completed. A data pipeline is constructed to meet the needs of different models. For the reconstruction model (the variational autoencoder (VAE) used in this invention), the multidimensional data needs to be flattened into a one-dimensional vector. For the prediction model (the gated recurrent unit (GRU) used in this invention), the model requires three-dimensional input data to maintain the order and dependency of the time series.

[0027] (2) Dual-model parallel training: The VAE and GRU models are initialized separately. In the VAE model, the input data is compressed into the latent space through the encoder, and then reconstructed through the decoder. The reconstruction error and KL divergence are calculated as part of the total loss function to update the model parameters. In the GRU model, the data is passed through the GRU layer to generate a prediction for the next time step. The prediction error is calculated as the loss function and the model parameters are updated. Both models independently optimize their respective objectives at the same time, fully utilizing computing resources and preserving model parameters.

[0028] (3) Application of the attention mechanism in the fusion layer: The reconstruction errors (e r ) and prediction error (e p ) as input features, forming a feature vector f = [e r ,e p ] T , the feature vector is input into the fusion layer based on the attention mechanism. The attention mechanism automatically adjusts the contribution ratio of reconstruction error and prediction error in the final anomaly score by learning weight parameters, thereby achieving dynamic fusion and giving more representative errors higher weights to enhance the ability to capture key anomaly features.

[0029] The following is a detailed explanation of the scheme of the present invention:

[0030] (1) Data preprocessing

[0031] Original industrial data set X={X1,X2,…X N}, N is the feature dimension, and the normalization process described in step S1 is:

[0032]

[0033] Among them, X is the original data, μ is the mean, σ is the standard deviation, X norm is the preprocessed data;

[0034] X norm Divide into multiple window segments, represented as:

[0035] W i ={X norm,t ,X norm,t+1 ,…,X norm,t+W-1}

[0036] t=1,1+S,1+2S,…

[0037] Among them, W i represents the i-th sliding window, X norm,t is the N-dimensional feature vector of the t-th time step after data preprocessing, and S is the step size.

[0038] (2) About model training

[0039] S2: At the same time, use the training set data to conduct parallel training of the reconstruction model and the prediction model to obtain the total loss function of the corresponding model;

[0040] S21: The training of the reconstruction model includes:

[0041] The original data used to train the reconstruction model is represented as:

[0042]

[0043] in, is the flattened input vector, W is the window size, and N is the number of features;

[0044] (1) Original data The mean μ and logarithmic variance logσ of the latent variable are calculated by the encoder 2 :

[0045]

[0046] Among them, θ Encoder Represents the parameters of the encoder;

[0047] (2) Using the latent space, the latent variable z implied by the original data is expressed as a function of mean and variance:

[0048] z=μ+σ⊙∈,∈~Ν(0,Ι)

[0049] Where z is the latent variable, μ is the mean vector of the latent variable, and σ is the standard deviation vector of the latent variable, which is given by σ=exp(0.5·logσ 2 ) is calculated, ∈ is the random noise vector of standard normal distribution, ⊙ is the element-by-element multiplication;

[0050] (3) The decoder is responsible for reconstructing the latent variable z into the original data space and obtaining the reconstructed data:

[0051]

[0052] Among them, θ Decoder Represents the parameters of the decoder;

[0053] (4) Constructing the total loss function of the reconstruction model

[0054]

[0055] in, Represents the difference between the reconstructed data and the original data, defined as

[0056] is the KL divergence, which represents the potential distribution N(μ,σ) generated by the encoder 2 ) and the difference between the prior distribution N(0,Ι), the calculation formula is J is the dimension of the latent space;

[0057] λ is the weight coefficient for balancing reconstruction error and KL divergence;

[0058] S22: Prediction model training

[0059] (1) The GRU layer consists of multiple GRU units. Each GRU unit contains an update gate and a reset gate. Through the recursive connection of time steps, the temporal dynamic characteristics of the data are captured:

[0060] Input sequence Through the GRU layer, the following data is generated:

[0061] z t =σ(W z ·[h t-1 ,x t ]+b z )

[0062] rt =σ(W r ·[h t-1 ,x t ]+b r )

[0063]

[0064] Among them, z t is the update gate, r t is the reset gate, h t is the current hidden state, x t is the current input, ⊙ represents element-wise multiplication, σ is the Sigmoid activation function, tanh is the hyperbolic tangent activation function, W r is the weight matrix of the reset gate, W z is the weight matrix of the update gate, b r is the bias vector of the reset gate, b z is the bias vector of the update gate, W is the weight matrix of the candidate hidden state, and b is the bias vector of the candidate hidden state;

[0065] (2) The fully connected layer connects the hidden state of the last time step of the GRU layer to the output layer, using the hidden state h of the last time step W , generates predictions for the data points at the next time step through the fully connected layer

[0066]

[0067] Among them, FC represents the fully connected operation, is the predicted value;

[0068] (3) Obtain the total loss function of the prediction model

[0069]

[0070] Where B is the batch size;

[0071] Substitute the validation set data into the total loss function of the trained reconstruction model to obtain the reconstruction error e r ; Use the validation set data to substitute into the total loss function of the trained prediction model to obtain the prediction error e p .

[0072] (3) Fusion of the two models

[0073] S3: Use the attention mechanism to fuse the reconstruction model and the prediction model:

[0074] S31: The reconstruction error e obtained in step S2 is rand the prediction error e p As input features, the feature vector f = [e r ,e p ] T ;

[0075] S32: Generate information Q representing the current focus, K representing the stored information features, and V representing the actual information associated with the key through a feedforward neural network:

[0076] Q=W q f i +b q

[0077] K=W k f i +b k

[0078] V=W v f i +b v

[0079] Among them, W q , W k , W v is the weight matrix, b q ,b k ,b v is the bias vector;

[0080] S33: Calculate the fused attention weight α by the following formula and further transform it to generate the comprehensive anomaly score s i :

[0081]

[0082] Where, α r represents the attention weight of the reconstruction model, α p represents the attention weight of the prediction model; Q r , K r Corresponding to the reconstruction error e r ;Q p , K p Corresponding to the prediction error e p ;

[0083] s i =σ(α r ·e r +α p ·e p )

[0084] Where σ is the Sigmoid activation function, s iis anomaly score. If the error value is greater than the defined threshold value Υ, the window data point is considered abnormal. The abnormal window data point is represented by F i =1{s i >Υ};

[0085] S34: Using binary cross entropy loss function Optimize the fused model:

[0086]

[0087] Among them, y i is the real data of the test set, i=1 indicates abnormality, and i=0 indicates normality.

[0088] The solution of the present invention contributes to the interpretability of anomaly detection. First, the α in the attention weight matrix r and α p Represent the reconstruction error e r and the prediction error e p The weights in the comprehensive score, the values ​​of these two weights reflect the contribution of each error source to the final anomaly score, which can be visualized by α r and α p This allows for an intuitive understanding of the relative importance of structural and dynamic anomalies within a data window. The absolute value of the weight reflects the severity of the anomaly. Based on the attention weights, this solution, combined with specific business scenarios, can generate detailed anomaly explanation reports, including the anomaly type, severity, and possible source. This not only enhances model transparency but also provides strong support for subsequent troubleshooting and decision-making.

[0089] Compared with the prior art, the present invention has the following beneficial effects:

[0090] This paper proposes a multi-level anomaly detection framework with a dual mechanism: by training the reconstruction model and the prediction model separately, the reconstruction model focuses on analyzing the deviation of the overall data structure from the normal pattern, while the prediction model focuses on the evolution trend of the time series and future state deviations. Finally, an attention mechanism is introduced at the fusion layer to fully integrate the reconstruction error and prediction error, calculate a comprehensive anomaly score, and achieve efficient and accurate anomaly detection. This multi-level architecture can adaptively cope with the complex characteristics of multimodal data, significantly improving the accuracy and robustness of detection.

[0091] Significantly Improved Detection Performance and Explainability: Compared to single detection models, this solution not only significantly improves anomaly identification accuracy but also provides explanations of anomaly sources and mechanisms. Through the model's multi-level feature extraction and decision fusion, this approach can quickly locate anomalies in industrial systems, providing a scientific basis for anomaly analysis and subsequent intervention, enhancing the practicality and intelligence of anomaly detection in industrial systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 Schematic diagram of the process of the multi-level industrial control system anomaly detection method based on the dual model according to the first embodiment of the present invention;

[0093] Figure 2 The structural composition of the reconstruction model in the multi-level industrial control system anomaly detection method based on dual models in Example 1 of the present invention;

[0094] Figure 3 This is the structural composition of the prediction model in the multi-level industrial control system anomaly detection method based on a dual model in Example 1 of the present invention. DETAILED DESCRIPTION

[0095] For better explanation of the purpose, technical scheme and advantage of the present invention, the present invention will be further described below in conjunction with specific embodiment and accompanying drawing, but embodiment does not limit the present invention in any form.Unless otherwise stated, the reagent, method and equipment adopted in the present invention are conventional reagents, methods and equipment in the art.Unless otherwise stated, reagents and materials used in the present invention are commercially available.

[0096] Example 1

[0097] This embodiment provides a multi-level industrial control system anomaly detection method based on a dual model. The flowchart is shown in Figure 1 As shown, the following steps are included:

[0098] S1: Collect the industrial dataset X, clean and standardize X, and obtain the preprocessed dataset X norm , divided into training set, validation set and test set;

[0099] Before the dual models are trained in parallel, the collected raw data needs to be preprocessed and classified because it contains normal and abnormal industrial process data.

[0100] The multidimensional time series data set in the original industrial control system is defined as X = {X1, X2, ... X N}, where N is the feature dimension. A series of preprocessing steps are performed to ensure data quality and adaptability. After basic data cleaning, the Z-score normalization method is used to eliminate the dimensional differences of different sensor data and ensure that all features are on the same scale:

[0101]

[0102] Among them, X is the original data, μ is the mean, σ is the standard deviation, X norm is the preprocessed data.

[0103] In order to divide the continuous time series data into windows of fixed length and maintain time dependency and local features, the window size W and step size S are set to divide the data into multiple window segments.

[0104] W i ={X norm,t ,X norm,t+1 ,…,X norm,t+W-1},t=1,1+S,1+2S,…

[0105] Among them, W i represents the i-th sliding window, X norm,t is the N-dimensional feature vector of the t-th time step after data preprocessing.

[0106] In order to meet the different data input format requirements of the variational autoencoder model and the gated recurrent unit prediction model, a data pipeline is constructed to perform the next step of data processing respectively.

[0107] VAE model requirements: Flatten multidimensional data into a one-dimensional vector to accommodate the fully connected layer input of VER.

[0108]

[0109] in, is the flattened input vector, W is the window size, and N is the number of features.

[0110] GRU model requirements: Maintain three-dimensional input data to preserve the order and dependencies of the time series.

[0111]

[0112] in, To maintain the three-dimensional structure of the input data, B is the batch size, W is the window size, and N is the number of features.

[0113] S2. Dual model training

[0114] (1) Training of reconstruction model VAE

[0115] A variational autoencoder is a form of deep generative model. It is a generative network structure based on variational Bayesian inference. In this solution, the VAE model includes the following three key components:

[0116] 1.1 Encoder

[0117] In order to map the input high-dimensional data to the latent space and extract the structural features of the data, the encoder is designed to consist of a multi-layer fully connected network. The mean μ and logarithmic variance logσ of the latent variables are calculated by the encoder. 2 .

[0118]

[0119] Among them, θ Encoder Represents the parameters of the encoder.

[0120] 1.2 Latent Space

[0121] The latent space is used to represent the implicit features of the input data. To achieve differentiability, the reparameterization technique is used to express the latent variable z as a function of the mean and variance:

[0122] z=μ+σ⊙∈,∈~Ν(0,Ι)

[0123] Where z is the latent variable, μ is the mean vector of the latent variable, and σ is the standard deviation vector of the latent variable, which is given by σ=exp(0.5·logσ 2 ) is calculated, ∈ is the random noise vector of standard normal distribution, and ⊙ is the element-by-element multiplication.

[0124] 1.3 Decoder

[0125] Symmetrical to the encoder, it consists of a multi-layer fully connected network, responsible for reconstructing the latent variable z into the original data space to obtain the reconstructed data

[0126]

[0127] Among them, θ Decoder Decoder parameters.

[0128] Variational Autoencoder VAE learns the potential distribution of data by maximizing the lower bound of data (Evidence Lower Bound, ELBO). Its goal is to make the reconstructed data as close as possible to the original input while making the latent variables obey the standard normal distribution. The error between the reconstructed data and the original input data is measured by express:

[0129]

[0130] The KL divergence is used to measure the potential distribution N(μ,σ) generated by the encoder 2 ) and the prior distribution N(0,Ι) to regularize the latent space, so that the model can generate a smoother and more continuous representation in the latent space:

[0131]

[0132] where J is the dimension of the latent space.

[0133] Combined reconstruction error And KL divergence, forming the total loss function of VAE:

[0134]

[0135] Among them, λ is the weight coefficient for balancing the reconstruction error and KL divergence.

[0136] (2) Training of the prediction model GRU

[0137] The gated recurrent unit model focuses on capturing the dynamic characteristics of time series data and is an efficient recurrent neural network variant. In this paper, the GRU model includes two key components: the GRU layer and the fully connected layer:

[0138] 2.1GRU Layer

[0139] It consists of multiple GRU units, each of which contains an update gate and a reset gate to control the flow and memory of information. It captures the temporal dynamic characteristics of the data through recursive connections of time steps:

[0140] Input sequence Through the GRU layer, the following data is generated:

[0141] z t =σ(W z ·[h t-1 ,x t ]+b z )

[0142] r t =σ(W r ·[h t-1 ,x t ]+b r )

[0143]

[0144] Among them, z t is the update gate, r t is the reset gate, h t is the current hidden state, xt is the current input, ⊙ represents element-wise multiplication, σ is the Sigmoid activation function, tanh is the hyperbolic tangent activation function, W r is the weight matrix of the reset gate, W z is the weight matrix of the update gate, b r is the bias vector of the reset gate, b z is the bias vector of the update gate, W is the weight matrix of the candidate hidden state, and b is the bias vector of the candidate hidden state;

[0145] 2.2 Fully Connected Layer

[0146] The fully connected layer connects the hidden state of the last time step of the GRU layer to the output layer, using the hidden state h of the last time step W , generates predictions for the data points at the next time step through the fully connected layer

[0147]

[0148] Among them, FC represents the fully connected operation, is the predicted value;

[0149] 2.3 Obtaining the total loss function of the prediction model

[0150]

[0151] Where B is the batch size.

[0152] S3: Use the attention mechanism to fuse the reconstruction model and the prediction model:

[0153] By training on normal data from the tool data training set, the dual mechanism model learns the structure and dynamic characteristics of normal data, enabling effective reconstruction and prediction. The VAE learns the latent distribution of normal data, while the GRU learns the temporal dynamic characteristics of normal data, completing the first level of the dual mechanism model. The next level focuses on the organic integration of the dual mechanism model, leveraging its advantages to improve detection accuracy and overall performance. This is the key to our invention.

[0154] The present invention takes the normal and abnormal data windows from the validation set and calculates them through the trained VAE model and GRU model respectively to obtain the reconstruction error e r and the prediction error e p . r and e p Combined into a feature vector f i =[e r ,e p] T , a feedforward neural network is introduced to generate Query (Q) representing the information that currently needs attention, Key (K) representing the stored information features, and Value (V) representing the actual information associated with the key.

[0155] Q=W q f i +b q

[0156] K=W k f i +b k

[0157] V=W v f i +b v

[0158] Among them, W q , W k , W v is the weight matrix, b q ,b k ,b v is the bias vector.

[0159] The fused attention weight α is calculated by the following formula and further transformed to generate the comprehensive anomaly score s i :

[0160]

[0161] Where, α r represents the attention weight of the reconstruction model, α p represents the attention weight of the prediction model; Q r , K r Corresponding to the reconstruction error e r ;Q p , K p Corresponding to the prediction error e p ;

[0162] The calculated weights are used to perform weighted summation of the reconstruction error and the prediction error, and the Sigmoid function is used to convert the error into a comprehensive anomaly score:

[0163] s i =σ(α r ·e r +α p ·e p )

[0164] Where σ is the Sigmoid activation function, s i is anomaly score. If the error value is greater than the defined threshold value Υ, the window data point is considered abnormal. The abnormal window data point is represented by Fi = 1 {s i > Y}.

[0165] The fusion layer is optimized using a binary cross-entropy loss function:

[0166]

[0167] where y i is the true data of the test set, i = 1 represents an anomaly, and i = 0 represents normal.

[0168] The binary cross-entropy loss function is suitable for binary classification tasks and can measure the difference between the predicted probability distribution and the true label.

[0169] By training the model so that the loss function converges, the model can label the data points according to the anomaly score, thereby achieving anomaly detection in the production process of the industrial field.

[0170] Finally, it should be explained that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit the protection scope of the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the essence and scope of the technical solutions of the present application.

Claims

1. A multi-level industrial control system anomaly detection method based on a dual model, characterized in that: The steps include: Anomaly detection method for multi-level industrial control systems based on dual model S1: Collect the industrial dataset X, clean and standardize X, and obtain the preprocessed dataset X norm , divided into training set, validation set and test set; S2: At the same time, use the training set data to conduct parallel training of the reconstruction model and the prediction model to obtain the total loss function of the corresponding model; Substitute the validation set data into the total loss function of the trained reconstruction model to obtain the reconstruction error e r ; Use the validation set data to substitute into the total loss function of the trained prediction model to obtain the prediction error e p ; S3: Use the attention mechanism to fuse the reconstruction model and the prediction model: S31: The reconstruction error e obtained in step S2 is r and the prediction error e p As input features, the feature vector f = [e r ,e p ] T ; S32: Generate information Q representing the current focus, K representing the stored information features, and V representing the actual information associated with the key through a feedforward neural network; S33: Calculate the fused attention weight α by the following formula and further transform it to generate the comprehensive anomaly score s i : Where, α r represents the attention weight of the reconstruction model, α p represents the attention weight of the prediction model; Q r , K r Corresponding to the reconstruction error e r ;Q p , K p Corresponding to the prediction error e p ; s i =σ(α r ·e r +a p ·e p ) Where σ is the Sigmoid activation function, s i is anomaly score. If the error value is greater than the defined threshold value Υ, the window data point is considered abnormal. The abnormal window data point is represented by F i =1{s i >Υ}.

2. The method for detecting anomalies in a multi-level industrial control system based on a dual model according to claim 1, characterized in that: Original industrial data set X={X1,X2,…X N }, N is the feature dimension, and the normalization process described in step S1 is: Among them, X is the original data, μ is the mean, σ is the standard deviation, X norm is the preprocessed data.

3. The multi-level industrial control system anomaly detection method based on dual model according to claim 1 is characterized in that: The X norm Divide into multiple window segments, represented as: W i ={X norm,t ,X norm,t+1 ,…,X norm,t+W-1 } t=1,1+S,1+2S,… Among them, W i represents the i-th sliding window, X norm,t is the N-dimensional feature vector of the t-th time step after data preprocessing, and S is the step size.

4. The method for detecting anomalies in a multi-level industrial control system based on a dual model according to claim 1, characterized in that: The reconstruction model is a VAE model, and the pre-prediction model is a GRU model.

5. The method for detecting anomalies in a multi-level industrial control system based on a dual model according to any one of claims 1 to 4, characterized in that: The original data used to train the reconstruction model is represented as: in, is the flattened input vector, W is the window size, and N is the number of features.

6. The method for detecting anomalies in a multi-level industrial control system based on a dual model according to any one of claims 1 or 5, characterized in that: The VAE model includes an encoder, a latent space, and a decoder. The training process of the reconstruction model described in step S2 is: (1) Original data The mean μ and logarithmic variance logσ of the latent variable are calculated by the encoder 2 : Among them, θ Encoder Represents the parameters of the encoder; (2) Using the latent space, the latent variable z implied by the original data is expressed as a function of mean and variance: z=μ+σ⊙∈,∈~Ν(0,Ι) Where z is the latent variable, μ is the mean vector of the latent variable, and σ is the standard deviation vector of the latent variable, which is given by σ=exp(0.5·logσ 2 ) is calculated, ∈ is the random noise vector of standard normal distribution, ⊙ is the element-by-element multiplication; (3) The decoder is responsible for reconstructing the latent variable z into the original data space and obtaining the reconstructed data: Among them, θ Decoder Represents the parameters of the decoder; (4) Constructing the total loss function of the reconstruction model in, Represents the difference between the reconstructed data and the original data, defined as is the KL divergence, which represents the potential distribution N(μ,σ) generated by the encoder 2 ) and the difference between the prior distribution N(0,Ι), the calculation formula is J is the dimension of the latent space; λ is the weight coefficient that balances the reconstruction error and KL divergence.

7. The method for detecting anomalies in a multi-level industrial control system based on a dual model according to any one of claims 1 to 4, characterized in that: The data used to train the prediction model is represented as: in, To maintain the three-dimensional structure of the input data, B is the batch size, W is the window size, and N is the number of features.

8. The method for detecting anomalies in a multi-level industrial control system based on a dual model according to any one of claims 1 or 7, characterized in that: The prediction model GRU model includes a GRU layer and a fully connected layer. The training process of the prediction model described in step S2 is: (1) The GRU layer consists of multiple GRU units. Each GRU unit contains an update gate and a reset gate. Through the recursive connection of time steps, the temporal dynamic characteristics of the data are captured: Input sequence Through the GRU layer, the following data is generated: z t =σ(W z ·[h t-1 ,x t ]+b z ) r t =σ(W r ·[h t-1 ,x t ]+b r ) Among them, z t is the update gate, r t is the reset gate, h t is the current hidden state, x t is the current input, ⊙ represents element-wise multiplication, σ is the Sigmoid activation function, tanh is the hyperbolic tangent activation function, W r is the weight matrix of the reset gate, W z is the weight matrix of the update gate, b r is the bias vector of the reset gate, b z is the bias vector of the update gate, W is the weight matrix of the candidate hidden state, and b is the bias vector of the candidate hidden state; (2) The fully connected layer connects the hidden state of the last time step of the GRU layer to the output layer, using the hidden state h of the last time step W , generates predictions for the data points at the next time step through the fully connected layer Among them, FC represents the fully connected operation, is the predicted value; (3) Obtain the total loss function of the prediction model Where B is the batch size.

9. The method for detecting anomalies in a multi-level industrial control system based on a dual model according to claim 1, wherein: The Q, K, and V calculation process is as follows: Q=W q f i +b q K=W k f i +b k V=W v f i +b v Among them, W q , W k , W v is the weight matrix, b q ,b k ,b v is the bias vector.

10. The method for detecting anomalies in a multi-level industrial control system based on a dual model according to claim 1, characterized in that: After the reconstruction model and the prediction model are fused, the fusion layer is optimized using the test set data to obtain the binary cross entropy loss function. Among them, y i is the real data of the test set, i=1 indicates abnormality, and i=0 indicates normality.

Images