Industrial control abnormal flow detection method and system based on coupling variational auto-encoder

Abstract

The invention discloses an industrial control abnormal flow detection method and system based on a coupling variational auto-encoder, and relates to the field of industrial control system network security, the method comprises the steps of data acquisition and preprocessing, model construction, establishment of a model comprising two coupling variational auto-encoder (C-VAE) tuples, and establishment of a model comprising two C-VAE tuples; defining a target function based on an entropy regularization optimal transmission theory, converting an optimal transmission problem into a variational auto-encoder form, and combining an evidence lower bound with the optimal transmission theory; model training; and anomaly detection: applying the trained model to real-time network traffic data, calculating a reconstruction error, and if the error exceeds a preset threshold, marking as abnormal traffic. The method improves the accuracy and robustness of abnormal flow detection of the industrial control system, and is suitable for real-time monitoring and protection.

Description

Technical Field

[0001] This invention relates to the field of network security technology for industrial control systems, and in particular to a method and system for detecting abnormal flow in industrial control systems based on a coupled variational autoencoder. Background Technology

[0002] As industrial control systems (ICS) become increasingly integrated with information networks, their increased openness brings serious security challenges. A cyberattack on an ICS can lead to significant economic losses or even security incidents. Therefore, anomaly detection of ICS network traffic is a crucial step in ensuring its security.

[0003] Deep learning models have become a research hotspot in the field of intrusion detection due to their powerful automatic feature learning capabilities. Variational autoencoders (VAEs), as a powerful deep generative model, are applied to unsupervised anomaly detection. Their basic idea is: the model is trained on normal data to learn its latent distribution; for new input data, if its reconstruction error is high, it indicates that it deviates from the normal distribution and is judged as an anomaly.

[0004] However, the closest existing technologies (such as CN116933195A and CN116628612A) still have the following drawbacks, even though they utilize VAEs or introduce attention mechanisms for improvement: The prior distribution is too simplistic: Traditional VAEs typically assume that the prior distribution p(z) is an isotropic multivariate Gaussian distribution. This simplistic assumption can lead to "over-regularization," forcing the encoder to produce an overly smooth latent representation that fails to capture the complex and sparse characteristics of industrial control traffic.

[0005] The prior hole problem: There is a severe mismatch between the simple prior and the aggregated posterior distribution q_ϕ(z) generated by the encoder. This leads to low-quality samples generated by random sampling from the prior and distorts the evaluation of the model evidence lower bound (ELBO), ultimately reducing the accuracy of anomaly detection.

[0006] Variational posterior distribution family of representation is limited: It is usually assumed that the variational posterior is Gaussian, which limits its ability to approximate complex real posterior distributions and makes it perform poorly on high-dimensional, redundant industrial control flow data. Summary of the Invention

[0007] To address the technical problems in detecting abnormal flow in industrial control systems in existing technologies, this invention provides a method and system for detecting abnormal flow in industrial control systems based on a coupled variational autoencoder.

[0008] This invention is achieved through the following technical solution: An industrial control system abnormal flow detection method based on a coupled variational autoencoder includes: Data acquisition and preprocessing: Acquiring network traffic data from the industrial control system and performing preprocessing; Model building: Create a model containing two C-VAE tuples; For model training, normal traffic data and abnormal traffic data are respectively put into two tuples of the model, and the dual C-VAE tuple model is trained. The model parameters are adjusted until the model can accurately reconstruct normal traffic data. Anomaly detection involves applying the trained dual C-VAE tuple model to real-time network traffic data and calculating the reconstruction error. If the error exceeds a preset threshold, the traffic is marked as abnormal.

[0009] Furthermore, the preprocessing includes directly using the traffic data bytes as input data and padding the traffic data.

[0010] Furthermore, the model construction defines entropy-regularized optimal transmission, and its objective function is: ,in It is a regularization parameter. It is data distribution. It is a prior distribution. Indicates the distribution of the given data and prior distribution Between these, consider the optimal transmission cost of the entropy regularization term; This indicates a desire to find a joint distribution. This minimizes the expression; : indicates in joint distribution Below, negative log-likelihood Expected value; It is the conditional probability distribution of data point x given the latent variable z, parameterized as ; Describe the joint distribution With prior distribution and data distribution The KL divergence between the products.

[0011] Furthermore, the model construction also includes transforming the OT problem into the form of a variational autoencoder, and combining the evidence lower bound of the variational autoencoder with optimal transport theory.

[0012] Furthermore, the optimization objective of the C-VAE is: in It is the weight of the penalty item; : Indicates the distribution of data and approximate posterior distribution Below, negative log-likelihood Expected value; It is the conditional probability distribution of data point x given the latent variable z, parameterized as ; Indicates the distribution of data Below, approximate posterior distribution With prior distribution The expected value of the KL divergence between them; It is another regularization parameter that controls the weight of the KL divergence between the prior distribution and the aggregated posterior distribution; Represents the aggregate posterior distribution With prior distribution The KL divergence between them.

[0013] Furthermore, the model training includes fixing the decoder, optimizing the encoder, and using dual formulas to solve the optimal transmission problem.

[0014] Furthermore, the dual formula is: ;in This indicates that, given regularization parameters In the case of distribution To distribution The optimal transmission distance; u and v are dual variables.

[0015] Furthermore, the model training also includes updating the model parameters using stochastic gradient descent.

[0016] This invention also provides an industrial control abnormal flow detection system based on a coupled variational autoencoder, which, based on the aforementioned method for detecting industrial control abnormal flow using a coupled variational autoencoder, includes: The data acquisition and preprocessing module is used to acquire network traffic data from the industrial control system and perform preprocessing. The model building module is used to build a model containing two C-VAE tuples. The model training module is used to put normal traffic data and abnormal traffic data into two tuples of the model respectively, train the dual C-VAE tuple model, and adjust the model parameters until the model can accurately reconstruct normal traffic data. The anomaly detection module applies the trained dual C-VAE tuple model to real-time network traffic data and calculates the reconstruction error. If the error exceeds a preset threshold, it is marked as abnormal traffic.

[0017] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing program instructions for an industrial control abnormal flow detection method based on a coupled variational autoencoder. The program instructions for the industrial control abnormal flow detection method based on a coupled variational autoencoder can be executed by one or more processors to implement the steps of the industrial control abnormal flow detection method based on a coupled variational autoencoder as described above.

[0018] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The Entropy Regularized Optimal Transport (EOT) theory is combined with the VAE framework for use in the field of abnormal flow detection in industrial control. By using the mathematical tools of EOT to force the coupling between the data distribution and the prior distribution, the "prior hole problem" is fundamentally solved.

[0019] (2) Model structure innovation: The model design adopts a dual C-VAE tuple, one of which focuses on learning the characteristics of normal traffic, and the other on abnormal traffic. By comparing the differences in the potential characteristics learned by the two, more accurate anomaly identification and sample generation can be achieved.

[0020] (3) An alternating optimization algorithm based on the original EOT formula and the dual formula was proposed, which was used to train the decoder and encoder respectively, thus achieving more flexible and stable model training.

[0021] (4) The preprocessing method directly uses traffic bytes as features and adopts a fixed-dimensional zero-padding / truncation preprocessing method, which is particularly suitable for industrial control network traffic with short data packet lengths and different formats. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic flowchart of an industrial control abnormal flow detection method using a coupled variational autoencoder according to an embodiment of this application; Figure 2 This is a schematic diagram of a dual C-VAE tuple model according to an embodiment of this application. Detailed Implementation

[0023] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0024] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0025] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The illustrations only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the shape, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0026] See Figure 1 An industrial control abnormal flow detection method based on a coupled variational autoencoder includes the following steps: S1: Data Acquisition and Preprocessing S11: Data Acquisition; specifically includes collecting network traffic data of industrial control systems, including traffic under normal operating conditions and traffic under abnormal conditions; S12: Data Preprocessing The traffic data bytes are directly used as input data, and the traffic data is padded. Let the traffic data be X = ( , , ..., ,… ) , Let be the i-th byte of X, representing the i-th feature value of the data, and m represent the total number of features. If m is less than MIS (Maximum Investigation Size), the data packet is padded with zeros to ensure the total number of features is equal to MIS. If the number of features exceeds the MIS value, features exceeding the MIS value are deleted. Industrial traffic data packets are shorter than conventional traffic. Based on data packet analysis, the experiment sets the MIS value to 500 dimensions, and the data processing is represented as follows: in This represents the traffic data after data padding.

[0027] S2: Model Building Establish a model containing two C-VAE tuples, such as Figure 2 As shown, the two models have the same construction, and are specifically defined as follows: Define the data space X, which is the space where the observed variable x is located, and the latent space Z, which is the space where the latent variable z is located. Z is a low-dimensional representation space used to encode data.

[0028] Define probability distribution and prior distribution. (It can be any distribution, not limited to Gaussian distribution; continuous distributions such as...) Or discrete distribution such as Approximate posterior distribution The variational posterior distribution parameterized by the encoder can be a complex distribution (not limited to Gaussian).

[0029] Define decoder It is the decoder distribution parameterized by the decoder, responsible for mapping from the latent space Z back to the data space X.

[0030] Constructing the cost function In C-VAE, the cost function is defined as the negative log-likelihood, i.e. .

[0031] The objective function of Entropy Regularized Optimal Transport (EOT) is defined as follows: ,in It is a regularization parameter that controls the weight of the entropy term. It is data distribution. It is a prior distribution. Indicates the distribution of the given data and prior distribution Between these, consider the optimal transmission cost of the entropy regularization term. This indicates a desire to find a joint distribution. This minimizes the expression. : indicates in joint distribution Below, negative log-likelihood The expected value. Here It is the conditional probability distribution of data point x given the latent variable z, parameterized as . Describe the joint distribution With prior distribution and data distribution The KL divergence between products. This term encourages... near .

[0032] The OT problem is transformed into the form of a variational autoencoder (VAE), and the evidence lower bound (ELBO) of the VAE is combined with optimal transport (OT) theory. The optimization objective of the C-VAE is then: in It is the weight of the penalty item. : Indicates the distribution of data and approximate posterior distribution Below, negative log-likelihood The expected value. Here It is the conditional probability distribution of data point x given the latent variable z, parameterized as . Indicates the distribution of data Below, approximate posterior distribution With prior distribution The expected value of the KL divergence between them. It is another regularization parameter that controls the weight of the KL divergence between the prior distribution and the aggregated posterior distribution. Represents the aggregate posterior distribution With prior distribution The KL divergence between them. Here By integrating over all x Obtained.

[0033] S3: Model Training Fixed decoder Optimize encoder The optimal transport problem is solved using the dual formula, where the dual formula is: .in This indicates that, given regularization parameters In the case of distribution To distribution The optimal transmission (OT) distance is determined. u and v are dual variables, which are continuous functions defined on spaces X and Y. These functions are optimized in the dual problem to find the optimal transmission schedule.

[0034] and Let u and v represent the distributions of the dual variables u and v respectively. and The expected value. It is the entropy regularization term, which penalizes the interaction between the dual variables u and v and the cost function. The differences between them. Regularization parameters. The strength of regularization was controlled. and These represent two probability distributions for optimal transmission. In C-VAE, Typically, it's a data distribution. ,and It is the prior distribution of the latent variables. By optimizing the dual variables u and v, the optimal transmission plan can be found, which corresponds to finding the optimal coding distribution in the latent space. .

[0035] Based on the encoder output obtained in step S2, optimize the decoder. Minimize the EOT objective function.

[0036] Repeat the iterations until convergence or the desired number of iterations is reached.

[0037] Update model parameters using stochastic gradient descent (SGD) or other optimization algorithms. To minimize the EOT objective function.

[0038] Normal traffic data and abnormal traffic data are respectively put into two tuples in the model, and the dual C-VAE tuple model is trained. The model parameters are adjusted until the model can accurately reconstruct normal traffic data.

[0039] S4: Anomaly Detection The trained dual C-VAE tuple model was applied to real-time network traffic data.

[0040] Calculate the reconstruction error; if the error exceeds a preset threshold, it is marked as abnormal traffic.

[0041] In this implementation, the following is achieved: (1) The Entropy Regularized Optimal Transport (EOT) theory is combined with the VAE framework for use in the field of abnormal flow detection in industrial control. By using the mathematical tools of EOT to force the coupling between the data distribution and the prior distribution, the "prior hole problem" is fundamentally solved.

[0042] (2) Model structure innovation: The model design adopts a dual C-VAE tuple, one of which focuses on learning the characteristics of normal traffic, and the other on abnormal traffic. By comparing the differences in the potential characteristics learned by the two, more accurate anomaly identification and sample generation can be achieved.

[0043] (3) An alternating optimization algorithm based on the original EOT formula and the dual formula was proposed, which was used to train the decoder and encoder respectively, thus achieving more flexible and stable model training.

[0044] (4) The preprocessing method directly uses traffic bytes as features and adopts a fixed-dimensional zero-padding / truncation preprocessing method, which is particularly suitable for industrial control network traffic with short data packet lengths and different formats.

[0045] This invention also proposes an industrial control abnormal flow detection system based on a coupled variational autoencoder, which, based on the above-described method for detecting industrial control abnormal flow using a coupled variational autoencoder, includes: The data acquisition and preprocessing module is used to acquire network traffic data from the industrial control system and perform preprocessing. The model building module is used to build a model containing two C-VAE tuples. The model training module is used to put normal traffic data and abnormal traffic data into two tuples of the model respectively, train the dual C-VAE tuple model, and adjust the model parameters until the model can accurately reconstruct normal traffic data. The anomaly detection module applies the trained dual C-VAE tuple model to real-time network traffic data and calculates the reconstruction error. If the error exceeds a preset threshold, it is marked as abnormal traffic.

[0046] Furthermore, this embodiment of the invention also proposes a computer-readable storage medium storing program instructions for an industrial control system abnormal flow detection method based on a coupled variational autoencoder. The program instructions for the industrial control system abnormal flow detection method based on a coupled variational autoencoder can be executed by one or more processors to implement the steps of the industrial control system abnormal flow detection method based on a coupled variational autoencoder as described above.

[0047] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for detecting abnormal industrial control traffic based on coupled variational autoencoder, characterized in that, The method comprises the following steps: Data acquisition and preprocessing: collecting network traffic data of an industrial control system and preprocessing the data; Model construction: constructing a model comprising two C-VAE tuples; Model training: putting normal traffic data and abnormal traffic data into two tuples of the model respectively, training the double C-VAE tuple model, adjusting the model parameters until the model can accurately reconstruct the normal traffic data; Abnormality detection: applying the trained double C-VAE tuple model to real-time network traffic data and calculating the reconstruction error, and if the error exceeds a preset threshold, marking the traffic as abnormal.

2. The method of claim 1, wherein the method is based on a coupled variational autoencoder. The preprocessing comprises directly taking network traffic data bytes as input data and padding the network traffic data.

3. The method of claim 1, wherein the method is based on a coupled variational autoencoder. The model construction defines entropy regularization optimal transport, and the objective function is: where is a regularization parameter, is a data distribution, is a prior distribution, denotes the KL divergence between the given data distribution and prior distribution ; denotes the existence of a joint distribution such that the expression is minimized; denotes the expected value of the negative log-likelihood under the joint distribution ; is the conditional probability distribution of data points x given latent variables z, parameterized as ; denotes the KL divergence between the joint distribution and the product of the prior distribution and the data distribution .

4. The method of claim 3, wherein the method is based on a coupled variational autoencoder. The model construction further comprises converting the OT problem into the form of a variational autoencoder and combining the evidence lower bound of the variational autoencoder with the optimal transport theory.

5. The method of claim 4, wherein the method is based on a coupled variational autoencoder. The optimization objective of the C-VAE tuple is: where is a weight of the penalty term; denotes the expectation of the negative log-likelihood under the data distribution and the approximate posterior distribution ; is the conditional probability distribution of the data point x given the latent variable z, parameterized as ; denotes the expectation of the KL divergence between the approximate posterior distribution and the prior distribution under the data distribution ; is another regularization parameter, controlling the weight of the KL divergence between the prior distribution and the aggregated posterior distribution; denotes the KL divergence between the aggregated posterior distribution and the prior distribution .

6. The method of claim 1, wherein the method is based on a coupled variational autoencoder. The model training comprises fixing the decoder, optimizing the encoder, and solving the transport problem based on the dual formula.

7. The method of claim 6, wherein the method is based on a coupled variational autoencoder. The dual formula is: ; where represents the optimal transport distance from distribution to distribution given a regularization parameter ; u and v are dual variables; and represent the expected values of dual variables u and v over distributions and respectively. is the entropy regularization term, which penalizes the difference between dual variables u and v and the cost function . The regularization parameter controls the strength of regularization. and represent the two probability distributions to be optimally transported. In C-VAE, is typically the data distribution , while is the prior distribution of latent variables . By optimizing dual variables u and v, the optimal transport plan can be found, which corresponds to finding the best encoding distribution in the latent space.

8. The method of claim 7, wherein the method is based on a coupled variational autoencoder. The model training further comprises updating the model parameters using stochastic gradient descent.

9. An industrial control abnormal traffic detection system based on coupled variational autoencoders, based on the industrial control abnormal traffic detection method based on coupled variational autoencoders according to any one of claims 1 to 8, comprising: a data acquisition and preprocessing module for collecting network traffic data of an industrial control system and preprocessing the data; a model construction module for constructing a model comprising two C-VAE tuples; a model training module for putting normal traffic data and abnormal traffic data into two tuples of the model respectively, training the double C-VAE tuple model, adjusting the model parameters until the model can accurately reconstruct the normal traffic data; an abnormality detection module for applying the trained double C-VAE tuple model to real-time network traffic data and calculating the reconstruction error, and if the error exceeds a preset threshold, marking the traffic as abnormal.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores program instructions of the industrial control abnormal traffic detection method based on coupled variational autoencoders, and the program instructions of the industrial control abnormal traffic detection method based on coupled variational autoencoders can be executed by one or more processors to implement the steps of the industrial control abnormal traffic detection method based on coupled variational autoencoders according to any one of claims 1 to 8.

Images