A network traffic threat detection and early warning method based on decoupled representation learning

Abstract

The application relates to the technical field of network security and discloses a network traffic threat detection and early warning method based on decoupling representation learning, which comprises the following steps: S1, detecting and identifying network traffic data in a network traffic management device to construct a data set containing a threat traffic data set and a normal traffic data set; S2, obtaining potential representation of the threat traffic data set according to a pre-trained Glow generation model; S3, constructing a learning strategy based on traffic changes in a potential space of the Glow generation model according to the potential representation to obtain a threat interpretable direction of a candidate traversal; S4, designing a comparison module through contrastive learning and extracting decoupling representation according to the threat interpretable direction; S5, constructing a threat detection model based on the decoupling representation and setting an early warning mechanism through a result of the threat detection model. The two can be obviously distinguished through decoupling, so that the efficiency of the detection process is improved. The method can well cope with detection and analysis of massive network traffic data.

Description

Technical Field

[0001] This application relates to the technical field of network security, and in particular to a method for network traffic threat detection and early warning based on decoupled representation learning. Background Technology

[0002] The rapid development of the internet age has brought great convenience to modern people, enabling them to exchange data online and realize the transmission and interaction of various messages. However, with the advancement of technology, more and more users are bringing massive amounts of network data, making it difficult for network management equipment to cope.

[0003] Network traffic management devices, while receiving network user data, also need to detect whether there are network threat intrusion attempts. Faced with massive amounts of data, traditional methods are inefficient at identifying the threat portion of network traffic, potentially leading to significant time losses. Threat data and ordinary traffic data may not differ significantly in some external manifestations, and the two may become entangled during the identification process. How to separate threat data from ordinary data is a problem worth considering. Summary of the Invention

[0004] To improve the accuracy of distinguishing between threat data and ordinary data, this application provides a network traffic threat detection and early warning method based on decoupled representation learning, employing the following technical solution: A network traffic threat detection and early warning method based on decoupled representation learning includes: S1. Detect and identify network traffic data in network traffic management devices, and construct a dataset containing threat traffic datasets and normal traffic datasets; S2. Obtain the potential representation of the threat traffic data set based on the pre-trained Glow generative model; S3. In the latent space of the Glow generation model, construct a learning strategy based on traffic changes according to the latent representation to obtain the threat interpretable direction of the candidate traversal; S4. Design a comparison module through comparative learning, and extract decoupling representations based on the interpretable direction of the threat. S5. Construct a threat detection model based on the decoupled representation, and set an early warning mechanism based on the results of the threat detection model.

[0005] Optionally, step S1 includes: S1-1. Preprocess the network traffic data; S1-2. Divide the network traffic data into threat traffic datasets and normal traffic datasets.

[0006] Optionally, step S2 includes: S2-1. Construct the Glow generation model and training set, and pre-train the Glow generation model based on the training set; S2-2. Input the threat traffic dataset into the Glow generation model to obtain the potential representation of the threat traffic data.

[0007] Optionally, the process of constructing the Glow generative model includes: The Glow generation model is defined as {G: Z->I}, where Z represents the potential space and I represents the network traffic data space; The parameters of the Glow generative model are constrained by the following two equations:

[0008]

[0009] Where z represents the latent variable and x represents the training data. Let z represent a solvable probability density function, derived from a simple spherical multivariate Gaussian distribution. Sampled, It is reversible; The optimization objective function used in training is:

[0010] in, Indicates obtaining The optimal value under the given conditions Represents the direction index, where , Where b is the offset size, b is the number of directions detected, and b is the classification term. Using the cross-entropy function, For the prediction of the direction index, the regression term Using mean absolute error For the prediction of offsets, the method seeks to obtain the most interpretable column number b in N by minimizing the optimization objective function, making the corresponding network traffic data transformations more distinguishable and simplifying the classification in the interpretive direction.

[0011] Optionally, step S3 includes: S3-1, Constructing a learner for network traffic understanding ; S3-2, Through the learner The threat interpretable directions of the threat traffic dataset trained by the Glow generative model are discovered, and these threat interpretable directions are mapped from the potential space to the network traffic data space. ; The Glow generative model is an untrainable component at this stage, and its parameters do not change during the learning process.

[0012] Optionally, the learner Includes navigation module N and reconstruction module R; The learner The optimization objective function is:

[0013] in, Indicates obtaining The optimal value under the given conditions Represents the direction index, where , Where b is the offset size, b is the number of directions detected, and b is the classification term. Using the cross-entropy function, For the prediction of the direction index, the regression term Using mean absolute error For the prediction of offsets, the method seeks to obtain the most interpretable column number b in N by minimizing the optimization objective function, making the corresponding network traffic data transformations more distinguishable and simplifying the classification in the interpretive direction.

[0014] Optionally, step S4 includes: S4-1. Map the interpretable directions of candidate threats to the traffic space to generate traffic pairs. Where z represents a latent variable, Represents a unit vector , Let G(z) represent the value in the unit vector, and let G(z) represent the first flow data generated from the latent variable z of the Glow generative model. Indicates from shift encoding The second set of traffic data generated; S4-2, Transfer the flow rate to... Encoding as decoupled samples , , ; in, denoted by , where E represents the change space and E represents the decoupled encoder.

[0015] Optionally, step S5 includes: S5-1. Construct a threat detection model based on the extracted decoupled representation; S5-2. Set up an early warning mechanism based on threat detection results.

[0016] In summary, this application includes at least one of the following beneficial technical effects: Learning interpretable visual image representations to reveal factors contributing to image variation is a hot research topic in computer vision, and decoupling representation learning is currently a popular approach in this field. Many traditional decoupling methods fail to achieve satisfactory results, as at least two physical properties remain entangled when changing a potential variation factor in image generation. Decoupling representation learning can generate an interpretable space within the model's latent space and utilize contrastive modules to construct a variation space to extract decoupling representations, thus achieving a decoupling representation space for visual images. Applying decoupling representation learning to network traffic management problems allows threat data and ordinary data in network traffic data to be viewed as two entangled categories. Decoupling these categories enables their clear distinction, improving the efficiency of the detection process. Utilizing decoupling representation learning to mine latent representations of network traffic data, moving from network traffic identification to understanding, is similar to the process in image research and can effectively address the detection and analysis of massive amounts of network traffic data. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the network traffic threat detection and early warning method of this invention. Detailed Implementation

[0018] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0019] In the description of this specification, the references to "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples" refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0020] This application discloses a network traffic threat detection and early warning method based on decoupled representation learning, referring to... Figure 1 A network traffic threat detection and early warning method based on decoupled representation learning, comprising: S1. Detect and identify network traffic data in network traffic management devices, and construct a dataset containing threat traffic datasets and normal traffic datasets; S2. Obtain the potential representation of the threat traffic data set based on the pre-trained Glow generative model; S3. In the latent space of the Glow generation model, construct a learning strategy based on traffic changes according to the latent representation to obtain the threat interpretable direction of the candidate traversal; S4. Design a comparison module through comparative learning, and extract decoupling representations based on the interpretable direction of the threat. S5. Construct a threat detection model based on the decoupled representation, and set an early warning mechanism based on the results of the threat detection model.

[0021] Optionally, step S1 includes: S1-1. In order to obtain the difference between threat traffic data and ordinary traffic data in the representation space, the network traffic data is preprocessed, including data cleaning and standardization, to obtain clean and standardized threat traffic datasets and ordinary traffic datasets for subsequent analysis. S1-2. Divide the network traffic data into threat traffic datasets and normal traffic datasets. The presence of intrusion threats may be differentiated by potential factors within the data. These differences in factors will quickly identify the existence of intrusion threats. Some intrusion threats may not only threaten the network through direct data transmission; for example, DDoS attacks can consume network services, and if the maximum number of users the management device can handle is exceeded, normal users will be unable to use the network. Therefore, this step requires employing multiple traditional detection methods and using diverse data to detect current threat behaviors, thereby improving the accuracy of the dataset.

[0022] Optionally, step S2 includes: S2-1. Construct the Glow generation model and training set, and pre-train the Glow generation model based on the training set. The Glow generation model consists of a series of reversible transformations, which can achieve accurate log-likelihood optimization of the data distribution, laying the foundation for decoupled representation learning. S2-2. Input the threat traffic dataset into the Glow generation model to map the latent space to the traffic data space, so that the latent encoding in the latent space can map the data distribution of the target traffic to obtain the latent representation of the threat traffic data.

[0023] The process of constructing the Glow generative model includes: The Glow generation model is defined as {G: Z->I}, where Z represents the potential space and I represents the network traffic data space; The parameters of the Glow generative model are constrained by the following two equations:

[0024]

[0025] Where z represents the latent variable and x represents the training data. Let z represent a solvable probability density function, derived from a simple spherical multivariate Gaussian distribution. Sampled, The mapping function between x and z is invertible; therefore, given a data point x, we can... Latent variable inference is performed. Furthermore, a series of transformations are used to learn the generation and inference process of network traffic data. Through training with reversible transformations, the distribution of the target network traffic data is gradually obtained to represent its effective characterization. For example... ; The optimization objective function used in training is:

[0026] in, For the first Intermediate variables in the second training iteration To optimize the objective function, To Take the logarithm. This represents the total number of transformations used. This represents the change in the objective function from x to z. Indicates from arrive The amount of change in the objective function during the process of change.

[0027] The Glow generative model is used to train the target traffic dataset, mapping the latent space to the traffic data space. This allows the latent encoding in the latent space to depict the data distribution of the target traffic, preparing for further extraction and decoupling representation.

[0028] Optionally, step S3 includes: S3-1, Constructing a learner for network traffic understanding Its purpose is to discover the interpretable directions of threats in traffic data trained by the Glow generative model, and to map the obtained directions from the potential space to the network traffic data space. S3-2, Through the learner The threat interpretable directions of the threat traffic dataset trained by the Glow generative model are discovered, enabling the mapping of these threat interpretable directions from the potential space to the network traffic data space. ; The Glow generative model is an untrainable component at this stage, and its parameters do not change during the learning process.

[0029] Optionally, the learner It includes a navigation module N and a reconstruction module R, where N represents the navigation module and is represented by a matrix. constitute, Let be the dimension of the latent space of the Glow model, and b be the number of discovered directions. R represents the reconstruction module, which yields... The first part can be generated by the latent variable z, and the second part can be generated by the displacement variable. Generate. The latter is the corresponding form of the former. Move the b-th column in the direction size; The learner The optimization objective function is:

[0030] in, Indicates obtaining The optimal value under the given conditions Represents the direction index, where , Where b is the offset size, b is the number of directions detected, and b is the classification term. Using the cross-entropy function, For the prediction of the direction index, the regression term Using mean absolute error For the prediction of offsets, the method seeks to obtain the most interpretable column number b in N by minimizing the optimization objective function, making the corresponding network traffic data transformations more distinguishable and simplifying the classification in the interpretive direction.

[0031] Optionally, step S4 uses contrastive learning to decouple the previously discovered interpretable threat directions for representation learning, including: S4-1, The learning device The threats discovered can be interpreted in a direction for decoupling representation learning. A contrast module for the decoupling encoder is designed. Constructing a space of change This maps the interpretable directions of candidate threats to the traffic space, generating traffic pairs. To decouple samples in order to extract decoupling representations; where z represents a latent variable, Represents a unit vector , Let G(z) represent the value in the unit vector, and let G(z) represent the first flow data generated from the latent variable z of the Glow generative model. Indicates from shift encoding The second set of traffic data generated; S4-2, Transfer the flow rate to... Encoding as decoupled samples , , ; in, denoted by , where E represents the change space and E represents the decoupled encoder. The variations between different traffic data are modeled as differences in the corresponding encoded representations of the decoupled encoder, and moving along the discovered direction causes significant data changes, enabling the decoupled representation to achieve a response with separated dimensions. Therefore, in The process pulls decoupled samples traversing the same direction together and pushes decoupled samples traversing different directions apart, improving the understanding of generative traffic data. This enables decoupled representations to achieve a response that separates dimensions, extracting decoupled representations based on conjugate samples in a variable space, thereby improving the performance of generative visual image understanding.

[0032] To improve the results of decoupled representation learning, a reconstruction loss based on the BCE loss function is used. Paired observations exist. , For traffic data, q represents the query key. From the positive key set or negative key set ,Right now Configuration item Ci is ; Based on the BCE loss function applied to a sample with N positive samples and M negative samples, the loss is set... The form can be transformed into

[0033] in, This represents the Sigmoid function. represents the hyperparameter, and S represents the number of samples.

[0034] According to the above formula, let By comparing the positive and negative aspects, As a similarity score, if and If it comes from a directly opposite direction, meaning it has the same interpretable threat, then... The score should be as large as possible ( Conversely, this meaningful representation can be used to discover the interpretable directions of threats arising from changes in different types of traffic data, enabling the extraction of decoupled representations.

[0035] Optionally, step S5 includes: S5-1. Construct a threat detection model based on the extracted decoupled representation; S5-2. Set up an early warning mechanism based on threat detection results.

[0036] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A network traffic threat detection and early warning method based on decoupled representation learning, characterized in that, include: S1. Detect and identify network traffic data in network traffic management devices, and construct a dataset containing threat traffic datasets and normal traffic datasets; S2. Obtain the potential representation of the threat traffic data set based on the pre-trained Glow generative model; S3. In the latent space of the Glow generation model, construct a learning strategy based on traffic changes according to the latent representation to obtain the threat interpretable direction of the candidate traversal; S4. Design a comparison module through comparative learning, and extract decoupling representations based on the interpretable direction of the threat. S5. Construct a threat detection model based on the decoupled representation, and set an early warning mechanism based on the results of the threat detection model.

2. The network traffic threat detection and early warning method based on decoupled representation learning according to claim 1, characterized in that, Step S1 includes: S1-1. Preprocess the network traffic data; S1-2. Divide the network traffic data into threat traffic datasets and normal traffic datasets.

3. The network traffic threat detection and early warning method based on decoupled representation learning according to claim 1, characterized in that, Step S2 includes: S2-1. Construct the Glow generation model and training set, and pre-train the Glow generation model based on the training set; S2-2. Input the threat traffic dataset into the Glow generation model to obtain the potential representation of the threat traffic data.

4. The network traffic threat detection and early warning method based on decoupled representation learning according to claim 2, characterized in that, The process of constructing the Glow generative model includes: The Glow generation model is defined as {G: Z->I}, where Z represents the potential space and I represents the network traffic data space; The parameters of the Glow generative model are constrained by the following two equations: ； ； Where z represents the latent variable and x represents the training data. Let z represent a solvable probability density function, derived from a simple spherical multivariate Gaussian distribution. Sampled, It is reversible; The optimization objective function used in training is: ； in,, For the first Intermediate variables in the second training iteration To optimize the objective function, To Take the logarithm. This represents the total number of transformations used. This represents the change in the objective function from x to z. Indicates from arrive The amount of change in the objective function during the process of change.

5. The network traffic threat detection and early warning method based on decoupled representation learning according to claim 1, characterized in that, Step S3 includes: S3-1, Constructing a learner for network traffic understanding ; S3-2, Through the learner The threat interpretable directions of the threat traffic dataset trained by the Glow generative model are discovered, and these threat interpretable directions are mapped from the potential space to the network traffic data space. ; The Glow generative model is an untrainable component at this stage, and its parameters do not change during the learning process.

6. The network traffic threat detection and early warning method based on decoupled representation learning according to claim 4, characterized in that, The learner Includes navigation module N and reconstruction module R; The learner The optimization objective function is: ； in, Indicates obtaining The optimal value under the given conditions Represents the direction index, where , Where b is the offset size, b is the number of directions detected, and b is the classification term. Using the cross-entropy function, For the prediction of the direction index, the regression term Using mean absolute error For the prediction of offsets, the method seeks to obtain the most interpretable column number b in N by minimizing the optimization objective function, making the corresponding network traffic data transformations more distinguishable and simplifying the classification in the interpretive direction.

7. The network traffic threat detection and early warning method based on decoupled representation learning according to claim 5, characterized in that, Step S4 includes: S4-1. Map the interpretable directions of candidate threats to the traffic space to generate traffic pairs. Where z represents a latent variable, Represents a unit vector , Let G(z) represent the value in the unit vector, and let G(z) represent the first flow data generated from the latent variable z of the Glow generative model. Indicates from shift encoding The second set of traffic data generated; S4-2, Transfer the flow rate pair Encoding as decoupled samples , , ; in, denoted by , where E represents the change space and E represents the decoupled encoder.

8. The network traffic threat detection and early warning method based on decoupled representation learning according to claim 1, characterized in that, Step S5 includes: S5-1. Construct a threat detection model based on the extracted decoupled representation; S5-2. Set up an early warning mechanism based on threat detection results.

Images