A data flow anomaly detection method and device based on multi-view

Abstract

The present invention discloses a data traffic anomaly detection method and device based on multiple views. The method comprises: importing the data traffic of a network port into a detection device through traffic mirroring; preprocessing the data traffic to form a session queue; screening the session queue to obtain target traffic, marking each target traffic and generating a corresponding label, and integrating the target traffic to obtain a target traffic set; performing multi-view feature extraction based on the target traffic set to obtain multi-view features; extracting a preliminary representation of the multi-view features; screening the preliminary representation to obtain a representation matrix; splicing the representation matrices of different views to obtain a global representation matrix; inputting the global representation matrix into a multi-task learning model to output anomaly detection results corresponding to multiple tasks. The method can well mine the deep-level features of data communication network traffic and realize high-precision detection of network threats existing in the data communication network.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a data traffic anomaly detection method and device based on multiple views. Background Art

[0002] With the further stimulation of policies such as the "New Infrastructure" initiative, IoT technology is becoming increasingly integrated with power industry production. Grid companies have variously proposed leveraging miniaturized and networked sensor technologies to promote the development of the Power IoT in power distribution and energy demand. However, while the benefits of the Power IoT technology come with increased security risks, cyber threats targeting the Power IoT are increasing. These threats are highly heterogeneous in their presentation and their camouflage capabilities are constantly evolving, making high-precision threat detection increasingly difficult. To strengthen the security of the Power IoT, it is necessary to analyze network traffic within the power grid and ensure the security of the Power IoT cyberspace.

[0003] As a critical infrastructure for ensuring public welfare and modern society, the power system is a high-value target for cyberattacks. Although my country's power system has established a deep security protection system based on perimeter security, relying on a physically isolated dispatching data network, the rapid development of cyberattack technology has gradually broken through this protection system. While existing statistical analysis and machine learning methods can detect some cyberattacks to a certain extent, they struggle to fully exploit the deep characteristics of cyberattack traffic. For example, patent CN112738014B discloses a method and system for detecting industrial control traffic anomalies based on a convolutional temporal network. The method includes taking industrial control protocol traffic as input, splitting it by read and write functions, and merging, normalizing, and grouping the split packets into unit time windows for learning a prediction model. Taking traffic data as input to form a dataset, a neural network model with an encoder-decoder architecture and a ConvLSTM layer is used to obtain a traffic data prediction model that can predict the next window data using the current window data. The resulting prediction model is used to predict the traffic packets to be detected, determining the distance between the predicted data and the actual data. A normalized score is calculated for the intra-group gap information, resulting in a distribution of windows and scores. The score distribution of the read and write models is fused in a weighted manner, and the distribution information is used to detect abnormal data traffic.

[0004] Although current detection methods based on neural network learning can detect threat traffic, considering the high heterogeneity between different network threat traffic, developing corresponding models for each threat will undoubtedly require a lot of cost, and when multiple detection models are deployed, their complexity, resource requirements and maintenance costs will increase sharply. Summary of the Invention

[0005] The present invention provides a data traffic anomaly detection method and device based on multiple views, which can perform comprehensive anomaly detection on data traffic and reduce costs.

[0006] A data traffic anomaly detection method based on multiple views, comprising:

[0007] Importing data traffic from network ports into the detection device through traffic mirroring;

[0008] Preprocessing the data traffic to form a session queue;

[0009] Screening the session queue to obtain target flows, marking each target flow and generating a corresponding label, and integrating the target flows to obtain a target flow set;

[0010] Performing multi-view feature extraction based on the target traffic set to obtain multi-view features;

[0011] extracting a preliminary representation of the multi-view features;

[0012] screening the preliminary representations to obtain a representation matrix;

[0013] The obtained representation matrices are spliced ​​to obtain a global representation matrix;

[0014] The global representation matrix is ​​input into a multi-task learning model, and anomaly detection results corresponding to multiple tasks are output.

[0015] Furthermore, the data traffic is pre-processed to form a session queue, including:

[0016] Deduplication processing is performed on identical data packets in the data traffic;

[0017] Sort the deduplicated data packets in ascending order of SEQ value;

[0018] The sorted data packets are diverted according to the same source IP, destination IP, source port, destination port and application layer protocol, and the diverted data packets are stored in queues in units of sessions to form session queues.

[0019] Further, the session queue is filtered based on the filtering rules in cpdump to obtain target traffic;

[0020] The target traffic set is stored in a hash table container, with the target traffic label as the key and the target traffic data packet as the value.

[0021] Furthermore, multi-view feature extraction is performed based on the target traffic set to obtain multi-view features, including:

[0022] Extracting features of fragment length, packet length distribution, and flag information of target traffic in the target traffic set to obtain packet-level view features;

[0023] Extracting features of inter-packet delay, transmission rate, and packet number distribution for target traffic in the target traffic set to obtain session-level view features;

[0024] Extracting features of protocol type, protocol field, and protocol version from target traffic in the target traffic set to obtain protocol-level view features;

[0025] The target traffic in the target traffic set is subjected to feature extraction of IP address, port number and device bandwidth to obtain host-level view features.

[0026] Furthermore, extracting a preliminary representation of the multi-view features includes:

[0027] performing convolution processing, normalization processing, and linear rectification on the packet-level view features, session-level view features, and host-level view features, respectively, to obtain a preliminary representation of the packet-level view, a preliminary representation of the session-level view, and a preliminary representation of the host-level view;

[0028] The protocol-level view features are One-Hot encoded, and feature extraction is performed based on a multi-layer perceptron. After normalization, a preliminary representation of the protocol-level view is obtained.

[0029] Furthermore, the packet-level view preliminary representation, the session-level view preliminary representation, and the host-level view preliminary representation are screened based on an attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix;

[0030] The preliminary representation of the protocol-level view is screened based on the multi-head attention mechanism to obtain a protocol-level representation matrix.

[0031] Furthermore, the preliminary representation of the protocol-level view is screened based on the multi-head attention mechanism to obtain a protocol-level representation matrix, including:

[0032] Performing a linear transformation on the preliminary representation of the protocol-level view to generate a query vector, a key vector, and a value vector;

[0033] Transforming the query vector, key vector, and value vector to generate a weight matrix;

[0034] Performing residual connection and normalization processing on the weight matrix and the preliminary representation of the protocol-level view to obtain the protocol-level representation matrix;

[0035] The weight matrix is ​​obtained according to the following formula:

[0036]

[0037] Among them, A(K,Q,V) represents the weight matrix, Q represents the query vector, K represents the key vector, and d k is the hidden representation dimension, and V represents the value vector.

[0038] Furthermore, the packet-level view preliminary representation, the session-level view preliminary representation, and the host-level view preliminary representation are screened based on the attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix, including:

[0039] Performing convolution processing, normalization processing, and introducing nonlinear features on the preliminary representation of the packet-level view, the preliminary representation of the session-level view, and the preliminary representation of the host-level view, respectively, to obtain a packet-level mask matrix, a session-level mask matrix, and a host-level mask matrix;

[0040] Performing residual connection and normalization processing on the packet-level mask matrix and the packet-level preliminary view representation to obtain a packet-level representation matrix;

[0041] Performing residual connection and normalization processing on the session-level mask matrix and the session-level view preliminary representation to obtain a session-level representation matrix;

[0042] The host-level mask matrix and the host-level preliminary view representation are residually connected and normalized to obtain a host-level representation matrix.

[0043] Furthermore, the multi-task learning model includes an input module, a residual module and an output module. The input module mines and downsamples the global representation matrix to obtain a preliminary latent variable representation; the residual module performs deep feature mining on the preliminary latent variable representation to obtain a common latent variable representation; the output module outputs anomaly detection results corresponding to multiple tasks according to the number of tasks set in the configuration file.

[0044] A data traffic anomaly detection device based on multiple views, comprising:

[0045] Import module, used to import data traffic of network ports through traffic mirroring;

[0046] A preprocessing module, configured to preprocess the data traffic to form a session queue;

[0047] an integration module, configured to screen the session queue to obtain target flows, mark each target flow and generate a corresponding label, and integrate the target flows to obtain a target flow set;

[0048] A view feature extraction module, configured to extract multi-view features based on the target traffic set to obtain multi-view features;

[0049] A preliminary extraction module, configured to extract a preliminary representation of the multi-view features;

[0050] A screening module, configured to screen the preliminary representations to obtain a representation matrix;

[0051] A splicing module is used to splice the obtained representation matrices to obtain a global representation matrix;

[0052] The detection module is used to input the global representation matrix into a multi-task learning model and output anomaly detection results corresponding to multiple tasks.

[0053] Furthermore, the preprocessing module preprocesses the data traffic to form a session queue, including:

[0054] Deduplication processing is performed on identical data packets in the data traffic;

[0055] Sort the deduplicated data packets in ascending order of SEQ value;

[0056] The sorted data packets are diverted according to the same source IP, destination IP, source port, destination port and application layer protocol, and the diverted data packets are stored in queues in units of sessions to form session queues.

[0057] Furthermore, the integration module filters the session queue based on the filtering rules in cpdump to obtain target traffic;

[0058] The target traffic set is stored in a hash table container, with the target traffic label as the key and the target traffic data packet as the value.

[0059] Furthermore, the view feature extraction module performs multi-view feature extraction based on the target traffic set to obtain multi-view features, including:

[0060] Extracting features of fragment length, packet length distribution, and flag information of target traffic in the target traffic set to obtain packet-level view features;

[0061] Extracting features of inter-packet delay, transmission rate, and packet number distribution for target traffic in the target traffic set to obtain session-level view features;

[0062] Extracting features of protocol type, protocol field, and protocol version from target traffic in the target traffic set to obtain protocol-level view features;

[0063] The target traffic in the target traffic set is subjected to feature extraction of IP address, port number and device bandwidth to obtain host-level view features.

[0064] Furthermore, the preliminary extraction module extracts a preliminary representation of the multi-view features, including:

[0065] performing convolution processing, normalization processing, and linear rectification on the packet-level view features, session-level view features, and host-level view features, respectively, to obtain a preliminary representation of the packet-level view, a preliminary representation of the session-level view, and a preliminary representation of the host-level view;

[0066] The protocol-level view features are One-Hot encoded, and feature extraction is performed based on a multi-layer perceptron. After normalization, a preliminary representation of the protocol-level view is obtained.

[0067] Furthermore, the screening module screens the packet-level view preliminary representation, the session-level view preliminary representation, and the host-level view preliminary representation based on an attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix;

[0068] The screening module screens the preliminary representation of the protocol-level view based on a multi-head attention mechanism to obtain a protocol-level representation matrix.

[0069] Furthermore, the screening module screens the preliminary representation of the protocol-level view based on a multi-head attention mechanism to obtain a protocol-level representation matrix, including:

[0070] Performing a linear transformation on the preliminary representation of the protocol-level view to generate a query vector, a key vector, and a value vector;

[0071] Transforming the query vector, key vector, and value vector to generate a weight matrix;

[0072] Performing residual connection and normalization processing on the weight matrix and the preliminary representation of the protocol-level view to obtain the protocol-level representation matrix;

[0073] The weight matrix is ​​obtained according to the following formula:

[0074]

[0075] Among them, A(K,Q,V) represents the weight matrix, Q represents the query vector, K represents the key vector, and d k is the hidden representation dimension, and V represents the value vector.

[0076] Furthermore, the screening module screens the preliminary representation of the packet-level view, the preliminary representation of the session-level view, and the preliminary representation of the host-level view based on the attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix, including:

[0077] Performing convolution processing, normalization processing, and introducing nonlinear features on the preliminary representation of the packet-level view, the preliminary representation of the session-level view, and the preliminary representation of the host-level view, respectively, to obtain a packet-level mask matrix, a session-level mask matrix, and a host-level mask matrix;

[0078] Performing residual connection and normalization processing on the packet-level mask matrix and the packet-level preliminary view representation to obtain a packet-level representation matrix;

[0079] Performing residual connection and normalization processing on the session-level mask matrix and the session-level view preliminary representation to obtain a session-level representation matrix;

[0080] The host-level mask matrix and the host-level preliminary view representation are residually connected and normalized to obtain a host-level representation matrix.

[0081] Furthermore, the multi-task learning model includes an input module, a residual module and an output module. The input module mines and downsamples the global representation matrix to obtain a preliminary latent variable representation; the residual module performs deep feature mining on the preliminary latent variable representation to obtain a common latent variable representation; the output module outputs anomaly detection results corresponding to multiple tasks according to the number of tasks set in the configuration file.

[0082] The multi-view data flow anomaly detection method and device provided by the present invention have at least the following beneficial effects:

[0083] (1) By integrating the multi-dimensional features of different views of data communication network traffic, a more representative and comprehensive representation is formed. At the same time, combined with the multi-task learning framework model, it can well mine the deep-level characteristics of data communication network traffic, flexibly adapt to different network environments and network threat types, and achieve high-precision detection of network threats existing in data communication networks. This is of great significance for achieving efficient management of data communication network traffic and maintaining data communication network security, and is particularly suitable for power data communication networks;

[0084] (2) By integrating the multi-dimensional features of different views of data communication network traffic, a more representative and comprehensive representation is formed, which can capture the characteristics of network traffic from different angles. After the fusion of multiple view features, the limitations of a single view may be compensated, and the multiple aspects of traffic characteristics can be comprehensively considered to further reduce the false alarm rate of detection and enhance the sensitivity to unknown network threats. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 The present invention provides a flowchart of an embodiment of a method for detecting data traffic anomalies based on multiple views.

[0086] Figure 2 Schematic diagram of an embodiment of the multi-view data traffic anomaly detection method provided by the present invention

[0087] Figure 3 The present invention provides a flowchart of an embodiment of preprocessing data traffic in the multi-view based data traffic anomaly detection method.

[0088] Figure 4 The present invention provides a flowchart of an embodiment of multi-view feature extraction in the multi-view based data traffic anomaly detection method.

[0089] Figure 5 The present invention provides a flowchart of an embodiment of extracting preliminary characterization of multi-view features in the multi-view based data traffic anomaly detection method.

[0090] Figure 6 The present invention provides a flowchart of an embodiment of screening preliminary representations of protocol-level views in the multi-view-based data traffic anomaly detection method.

[0091] Figure 7 The present invention provides a flowchart of an embodiment of screening the preliminary representation of the packet-level view, the preliminary representation of the session-level view, and the preliminary representation of the host-level view in the multi-view-based data traffic anomaly detection method provided by the present invention.

[0092] Figure 8 This is a flow chart of an embodiment of the multi-view based data traffic anomaly detection device provided by the present invention. DETAILED DESCRIPTION

[0093] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0094] refer to Figure 1 and Figure 2 In some embodiments, a method for detecting data traffic anomalies based on multiple views is provided, comprising:

[0095] S1, importing the data traffic of the network port into the detection device through traffic mirroring;

[0096] S2. Preprocess the data traffic to form a session queue;

[0097] S3. Screening the session queue to obtain target flows, marking each target flow and generating a corresponding label, and integrating the target flows to obtain a target flow set;

[0098] S4. Perform multi-view feature extraction based on the target traffic set to obtain multi-view features;

[0099] S5. extracting a preliminary representation of the multi-view features;

[0100] S6. Screening the preliminary representation to obtain a representation matrix;

[0101] S7, concatenating the obtained representation matrices to obtain a global representation matrix;

[0102] S8. Input the global representation matrix into a multi-task learning model, and output anomaly detection results corresponding to multiple tasks.

[0103] Specifically, in step S1, the data traffic can be the traffic in the power data communication network, which is copied to a separate detection server through traffic mirroring, and the traffic is stored in a cache pool for subsequent processing; the traffic cache pool uses Tpacketv3 cache to manage the reception and processing of data packets.

[0104] Further, refer to Figure 3 In step S2, the data traffic is pre-processed to form a session queue, including:

[0105] S21, performing deduplication processing on identical data packets in the data traffic;

[0106] S22, sorting the deduplicated data packets in ascending order of SEQ values;

[0107] S23. Divert the sorted data packets according to the same source IP, destination IP, source port, destination port, and application layer protocol, and store the diverted data packets in a queue in units of sessions to form a session queue.

[0108] Specifically, in step S23, the splitting is to divide the data packets with the same source IP, destination IP, source port, destination port and application layer protocol into one flow.

[0109] Furthermore, in step S3, the session queue is filtered based on the filtering rules in cpdump to obtain the target traffic;

[0110] The target traffic set is stored in a hash table container, with the target traffic label as the key and the target traffic data packet as the value.

[0111] Further, refer to Figure 4 In step S4, multi-view feature extraction is performed based on the target traffic set to obtain multi-view features, including:

[0112] S41, extracting features of fragment length, packet length distribution, and flag information of target traffic in the target traffic set to obtain packet-level view features;

[0113] S42, extracting features of inter-packet delay, transmission rate, and packet number distribution for the target traffic in the target traffic set to obtain session-level view features;

[0114] S43, extracting features of protocol type, protocol field, and protocol version from the target traffic in the target traffic set to obtain protocol-level view features;

[0115] S44: Extract features of the IP address, port number, and device bandwidth of the target traffic in the target traffic set to obtain host-level view features.

[0116] Specifically, the detailed information of multi-view features is shown in Table 1:

[0117] Table 1

[0118]

[0119]

[0120] In some embodiments, after step S4, the process further includes normalizing the packet-level view features, the session-level view features, and the host-level view features, specifically:

[0121] For each eigenvector x that needs to be normalized, the maximum and minimum normalization method is used to normalize it. The normalized vector x normalized for:

[0122]

[0123] where x min represents the minimum value of the eigenvector, x max Indicates the maximum value of the eigenvector.

[0124] No normalization is performed on the protocol-level view features.

[0125] Further, refer to Figure 5 In step S5, a preliminary representation of the multi-view features is extracted, including:

[0126] S51, performing convolution processing, normalization processing, and linear rectification on the packet-level view features, session-level view features, and host-level view features, respectively, to obtain a preliminary packet-level view representation, a preliminary session-level view representation, and a preliminary host-level view representation;

[0127] S52: One-hot encoding is performed on the protocol-level view features, and feature extraction is performed based on a multi-layer perceptron. A preliminary representation of the protocol-level view is obtained after normalization.

[0128] Specifically, refer to Figure 2 In step S51, a first preliminary representation extraction module is used to extract preliminary representations of the package-level view features, session-level view features, and host-level view features. The first preliminary representation extraction module includes a convolution layer, a BN layer, and a ReLU layer. The convolution layer performs convolution processing on the input package-level view features, session-level view features, and host-level view features. The BN layer normalizes the output of the convolution layer. The ReLU layer performs linear rectification on the output of the BN layer to obtain preliminary representations of the package-level view, session-level view, and host-level view.

[0129] Furthermore, in step S52, a second preliminary representation extraction module is used to extract preliminary representations of the protocol-level view features, wherein the second preliminary representation extraction module includes a One-Hot encoding module, a multi-layer perceptron, and a ReLU layer. The One-Hot encoding module performs One-Hot encoding on the protocol-level view features, the multi-layer perceptron performs feature extraction on the output of the One-Hot encoding module, and the ReLU layer normalizes the output of the multi-layer perceptron to obtain a preliminary representation of the protocol-level view.

[0130] Furthermore, in step S6, the packet-level view preliminary representation, the session-level view preliminary representation, and the host-level view preliminary representation are screened based on an attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix;

[0131] The preliminary representation of the protocol-level view is screened based on the multi-head attention mechanism to obtain a protocol-level representation matrix.

[0132] Further, refer to Figure 6 In step S6, the protocol-level view preliminary representation is screened based on the multi-head attention mechanism to obtain a protocol-level representation matrix, including:

[0133] S61. Perform a linear transformation on the preliminary representation of the protocol-level view to generate a query vector, a key vector, and a value vector;

[0134] S62: transform the query vector, key vector, and value vector to generate a weight matrix;

[0135] S63: Perform residual connection and normalization processing on the weight matrix and the preliminary representation of the protocol-level view to obtain the protocol-level representation matrix.

[0136] Specifically, in step S61, the input protocol-level view preliminary representation is linearly transformed using the following formula to generate a query vector, a key vector, and a value vector:

[0137] Q=XW Q ; (2)

[0138] K=XW K ; (3)

[0139] V=XW V ; (4)

[0140] Where X represents the initial representation of the input protocol level view, W Q 、W K and W V They are the initialization parameter matrices during the training process, Q represents the query vector, K represents the key vector, and V represents the value vector.

[0141] Furthermore, in step S62, the weight matrix is ​​obtained according to the following formula:

[0142]

[0143] Among them, A(K,Q,V) represents the weight matrix, Q represents the query vector, K represents the key vector, and d k is the hidden representation dimension, and V represents the value vector.

[0144] Specifically, refer to Figure 2 , the preliminary representation of the protocol-level view is screened based on the multi-head attention module, wherein the multi-head attention module includes three Linear layers, a multi-head attention layer and an ADD&Norm layer. The three Linear layers respectively perform linear changes on the preliminary representation of the protocol-level view to generate a query vector, a key vector and a value vector. The multi-head attention layer transforms the query vector, the key vector and the value vector to generate a weight matrix. The ADD&Norm layer performs residual connection and normalization processing on the weight matrix and the preliminary representation of the protocol-level view to obtain a protocol-level representation matrix containing important view information.

[0145] Further, refer to Figure 7In step S6, the packet-level view preliminary representation, the session-level view preliminary representation, and the host-level view preliminary representation are screened based on the attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix, including:

[0146] S6A, performing convolution processing, normalization processing, and introducing nonlinear features on the preliminary representation of the packet-level view, the preliminary representation of the session-level view, and the preliminary representation of the host-level view, respectively, to obtain a packet-level mask matrix, a session-level mask matrix, and a host-level mask matrix;

[0147] S6B, performing residual connection and normalization processing on the packet-level mask matrix and the packet-level preliminary view representation to obtain a packet-level representation matrix;

[0148] S6C, performing residual connection and normalization processing on the session-level mask matrix and the session-level preliminary view representation to obtain a session-level representation matrix;

[0149] S6D: Perform residual connection and normalization processing on the host-level mask matrix and the host-level preliminary view representation to obtain a host-level representation matrix.

[0150] Specifically, refer to Figure 2 , based on the attention mask module, the packet-level view preliminary representation, the session-level view preliminary representation and the host-level view preliminary representation are screened, wherein the attention mask module includes two convolutional layers, two BN layers, a ReLU layer, a Sigmoid layer and an ADD&Norm layer, wherein the first convolutional layer is used to perform convolution processing on the input packet-level view preliminary representation, the session-level view preliminary representation and the host-level view preliminary representation, the first BN layer normalizes the output result of the first convolutional layer, the ReLU layer introduces nonlinear features to the output result of the first BN layer, and the second The convolution layer performs convolution processing on the output result of the ReLU layer, the second BN layer normalizes the output result of the second convolution layer, and the Sigmoid layer introduces nonlinear features to the output result of the second BN layer, outputting the packet-level mask matrix, session-level mask matrix, and host-level mask matrix. The ADD&Norm layer performs residual connection and normalization on the packet-level view preliminary representation, session-level view preliminary representation, and host-level view preliminary representation with the packet-level mask matrix, session-level mask matrix, and host-level mask matrix respectively to obtain the packet-level representation matrix, session-level representation matrix, and host-level representation matrix.

[0151] Furthermore, in step S7, the representation matrices of different views are spliced ​​to obtain a global representation matrix. Specifically, the packet-level representation matrix, the session-level representation matrix, and the host-level representation matrix are spliced ​​along the feature dimension direction (vertical direction).

[0152] Considering the large number of devices in the power network, the types of threats and even the network traffic generated by network attacks against various types of devices are highly heterogeneous. It is obviously difficult to apply the method of detecting only one type of network threat from a single view in the current power Internet of Things network environment. Based on this, the above embodiment forms a more representative and comprehensive representation by integrating the multi-dimensional features of different views of data communication network traffic. It can capture the characteristics of network traffic from different angles. After the features of multiple views are integrated, they can make up for the limitations that may exist in a single view. In addition, it can comprehensively consider the characteristics of multiple aspects of traffic to further reduce the false alarm rate of detection and enhance the sensitivity to unknown network threats.

[0153] Furthermore, in step S8, the multi-task learning model includes an input module, a residual module and an output module. The input module mines and downsamples the global representation matrix to obtain a preliminary latent variable representation; the residual module performs deep feature mining on the preliminary latent variable representation to obtain a common latent variable representation; the output module outputs anomaly detection results corresponding to multiple tasks according to the number of tasks set in the configuration file.

[0154] Specifically, refer to Figure 2 The input module includes two BN layers, a one-dimensional convolution layer and a ReLU layer. The first BN layer normalizes the input global representation matrix, the one-dimensional convolution layer convolutions the output of the first BN layer, the ReLU layer introduces nonlinear features to the output of the one-dimensional convolution layer, and the second BN layer normalizes the output of the ReLU layer to obtain a preliminary latent variable representation.

[0155] Furthermore, the residual module includes seven one-dimensional convolutional layers and six BN+ReLU layers. The preliminary latent variable representation output by the input module is convolved by the first one-dimensional convolutional layer, normalized by the first BN+ReLU layer and nonlinear features are introduced, and then input to the second one-dimensional convolutional layer for convolution operation. The output result is superimposed with the preliminary latent variable representation and input to the second BN+ReLU layer for normalization and nonlinear features are introduced. The output result of the second BN+ReLU layer is input to the third one-dimensional convolutional layer and the fourth one-dimensional convolutional layer for convolution respectively. The output result of the third one-dimensional convolutional layer is normalized by the third BN+ReLU layer and nonlinear features are introduced, and its output result is input to the fourth one-dimensional convolutional layer. Five one-dimensional convolutional layers are used for processing. The output result of the fifth one-dimensional convolutional layer is superimposed on the output result of the fourth convolutional layer and input into the fourth BN+ReLU layer for normalization and introduction of nonlinear features. The output result of the fourth BN+ReLU layer is input into the sixth one-dimensional convolutional layer for convolution processing, and then input into the fifth BN+ReLU layer for normalization and introduction of nonlinear features. The output result of the fifth BN+ReLU layer is input into the seventh one-dimensional convolutional layer for convolution processing. The output result of the seventh one-dimensional convolutional layer and the output result of the fourth BN+ReLU layer are superimposed and input into the sixth BN+ReLU layer for normalization and introduction of nonlinear features. Finally, the output obtains the public latent variable representation.

[0156] Furthermore, the output module includes n task output modules, each task output module corresponds to the output result of a task, and each task output module includes a one-dimensional convolution layer, a ReLU layer, a Flatten layer, and a fully connected layer. According to the number of tasks set in the configuration file, the anomaly detection results corresponding to multiple tasks are represented based on the output common latent variables.

[0157] By integrating the multi-dimensional features of different views of data communication network traffic, a more representative and comprehensive representation is formed. At the same time, combined with the multi-task learning framework model, it can well mine the deep-level characteristics of data communication network traffic, flexibly adapt to different network environments and network threat types, and achieve high-precision detection of network threats existing in data communication networks. It is of great significance to achieve efficient management of data communication network traffic and maintain data communication network security, and is particularly suitable for power data communication networks.

[0158] Further, refer to Figure 8 In some embodiments, a multi-view-based data traffic anomaly detection device is provided, comprising:

[0159] Import module 201, used for importing data traffic of network ports through traffic mirroring;

[0160] A pre-processing module 202 is used to pre-process the data traffic to form a session queue;

[0161] An integration module 203 is configured to screen the session queue to obtain target flows, mark each target flow and generate a corresponding tag, and integrate the target flows to obtain a target flow set;

[0162] A view feature extraction module 204 is configured to extract multi-view features based on the target traffic set to obtain multi-view features;

[0163] A preliminary extraction module 205 is used to extract a preliminary representation of the multi-view features;

[0164] A screening module 206 is used to screen the preliminary representations to obtain a representation matrix;

[0165] A splicing module 207 is used to splice the obtained representation matrices to obtain a global representation matrix;

[0166] The detection module 208 is configured to input the global representation matrix into a multi-task learning model and output anomaly detection results corresponding to multiple tasks.

[0167] Furthermore, the pre-processing module 202 pre-processes the data traffic to form a session queue, including:

[0168] Deduplication processing is performed on identical data packets in the data traffic;

[0169] Sort the deduplicated data packets in ascending order of SEQ value;

[0170] The sorted data packets are diverted according to the same source IP, destination IP, source port, destination port and application layer protocol, and the diverted data packets are stored in queues in units of sessions to form session queues.

[0171] Furthermore, the integration module 203 filters the session queue based on the filtering rules in cpdump to obtain the target traffic;

[0172] The target traffic set is stored in a hash table container, with the target traffic label as the key and the target traffic data packet as the value.

[0173] Furthermore, the view feature extraction module 204 performs multi-view feature extraction based on the target traffic set to obtain multi-view features, including:

[0174] Extracting features of fragment length, packet length distribution, and flag information of target traffic in the target traffic set to obtain packet-level view features;

[0175] Extracting features of inter-packet delay, transmission rate, and packet number distribution for target traffic in the target traffic set to obtain session-level view features;

[0176] Extracting features of protocol type, protocol field, and protocol version from target traffic in the target traffic set to obtain protocol-level view features;

[0177] The target traffic in the target traffic set is subjected to feature extraction of IP address, port number and device bandwidth to obtain host-level view features.

[0178] Furthermore, the preliminary extraction module 205 extracts preliminary representations of the multi-view features, including:

[0179] performing convolution processing, normalization processing, and linear rectification on the packet-level view features, session-level view features, and host-level view features, respectively, to obtain a preliminary representation of the packet-level view, a preliminary representation of the session-level view, and a preliminary representation of the host-level view;

[0180] The protocol-level view features are One-Hot encoded, and feature extraction is performed based on a multi-layer perceptron. After normalization, a preliminary representation of the protocol-level view is obtained.

[0181] Furthermore, the screening module 206 screens the packet-level view preliminary representation, the session-level view preliminary representation, and the host-level view preliminary representation based on the attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix;

[0182] The screening module screens the preliminary representation of the protocol-level view based on a multi-head attention mechanism to obtain a protocol-level representation matrix.

[0183] Furthermore, the screening module 206 screens the preliminary representation of the protocol-level view based on the multi-head attention mechanism to obtain a protocol-level representation matrix, including:

[0184] Performing a linear transformation on the preliminary representation of the protocol-level view to generate a query vector, a key vector, and a value vector;

[0185] Transforming the query vector, key vector, and value vector to generate a weight matrix;

[0186] Performing residual connection and normalization processing on the weight matrix and the preliminary representation of the protocol-level view to obtain the protocol-level representation matrix;

[0187] The weight matrix is ​​obtained according to the following formula:

[0188]

[0189] Among them, A(K,Q,V) represents the weight matrix, Q represents the query vector, K represents the key vector, and d k is the hidden representation dimension, and V represents the value vector.

[0190] Furthermore, the screening module 206 screens the preliminary representations of the packet-level view, the preliminary representations of the session-level view, and the preliminary representations of the host-level view based on the attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix, including:

[0191] Performing convolution processing, normalization processing, and introducing nonlinear features on the preliminary representation of the packet-level view, the preliminary representation of the session-level view, and the preliminary representation of the host-level view, respectively, to obtain a packet-level mask matrix, a session-level mask matrix, and a host-level mask matrix;

[0192] Performing residual connection and normalization processing on the packet-level mask matrix and the packet-level preliminary view representation to obtain a packet-level representation matrix;

[0193] Performing residual connection and normalization processing on the session-level mask matrix and the session-level view preliminary representation to obtain a session-level representation matrix;

[0194] The host-level mask matrix and the host-level preliminary view representation are residually connected and normalized to obtain a host-level representation matrix.

[0195] Furthermore, the multi-task learning model includes an input module, a residual module and an output module. The input module mines and downsamples the global representation matrix to obtain a preliminary latent variable representation; the residual module performs deep feature mining on the preliminary latent variable representation to obtain a common latent variable representation; the output module outputs anomaly detection results corresponding to multiple tasks according to the number of tasks set in the configuration file.

[0196] The multi-view data traffic anomaly detection method and device provided in the above embodiment have at least the following beneficial effects:

[0197] (1) By integrating the multi-dimensional features of different views of data communication network traffic, a more representative and comprehensive representation is formed. At the same time, combined with the multi-task learning framework model, it can well mine the deep-level characteristics of data communication network traffic, flexibly adapt to different network environments and network threat types, and achieve high-precision detection of network threats existing in data communication networks. This is of great significance for achieving efficient management of data communication network traffic and maintaining data communication network security, and is particularly suitable for power data communication networks;

[0198] (2) By integrating the multi-dimensional features of different views of data communication network traffic, a more representative and comprehensive representation is formed, which can capture the characteristics of network traffic from different angles. After the fusion of multiple view features, the limitations of a single view may be compensated, and the multiple aspects of traffic characteristics can be comprehensively considered to further reduce the false alarm rate of detection and enhance the sensitivity to unknown network threats.

[0199] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the invention. Thus, the present invention is intended to include such changes and modifications as fall within the scope of the claims and their equivalents.

Claims

1. A data traffic anomaly detection method based on multiple views, characterized in that: include: Importing data traffic from network ports into the detection device through traffic mirroring; Preprocessing the data traffic to form a session queue; Screening the session queue to obtain target flows, marking each target flow and generating a corresponding label, and integrating the target flows to obtain a target flow set; Performing multi-view feature extraction based on the target traffic set to obtain multi-view features; extracting a preliminary representation of the multi-view features; screening the preliminary representations to obtain a representation matrix; The obtained representation matrices are spliced ​​to obtain a global representation matrix; Inputting the global representation matrix into a multi-task learning model and outputting anomaly detection results corresponding to multiple tasks; Performing multi-view feature extraction based on the target traffic set to obtain multi-view features includes: Extracting features of fragment length, packet length distribution, and flag information of target traffic in the target traffic set to obtain packet-level view features; Extracting features of inter-packet delay, transmission rate, and packet number distribution for target traffic in the target traffic set to obtain session-level view features; Extracting features of protocol type, protocol field, and protocol version from target traffic in the target traffic set to obtain protocol-level view features; Extracting features of the IP address, port number, and device bandwidth of the target traffic in the target traffic set to obtain host-level view features; Extracting a preliminary representation of the multi-view features, including: performing convolution processing, normalization processing, and linear rectification on the packet-level view features, session-level view features, and host-level view features, respectively, to obtain a preliminary representation of the packet-level view, a preliminary representation of the session-level view, and a preliminary representation of the host-level view; One-hot encoding is performed on the protocol-level view features, and feature extraction is performed based on a multi-layer perceptron. A preliminary representation of the protocol-level view is obtained after normalization. The packet-level view preliminary representation, the session-level view preliminary representation, and the host-level view preliminary representation are screened based on an attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix; The preliminary representation of the protocol-level view is screened based on the multi-head attention mechanism to obtain a protocol-level representation matrix.

2. The method according to claim 1, characterized in that Preprocessing the data traffic to form a session queue includes: Deduplication processing is performed on identical data packets in the data traffic; Sort the deduplicated data packets in ascending order of SEQ value; The sorted data packets are diverted according to the same source IP, destination IP, source port, destination port and application layer protocol, and the diverted data packets are stored in queues in units of sessions to form session queues.

3. The method according to claim 1, characterized in that The target traffic set is stored in a hash table container, with the target traffic label as the key and the target traffic data packet as the value.

4. The method according to claim 1, wherein The preliminary representation of the protocol-level view is screened based on the multi-head attention mechanism to obtain a protocol-level representation matrix, including: Performing a linear transformation on the preliminary representation of the protocol-level view to generate a query vector, a key vector, and a value vector; Transforming the query vector, key vector, and value vector to generate a weight matrix; The weight matrix is ​​residually connected and normalized with the preliminary representation of the protocol-level view to obtain the protocol-level representation matrix.

5. The method according to claim 1, wherein The packet-level view preliminary representation, the session-level view preliminary representation, and the host-level view preliminary representation are screened based on the attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix, including: Performing convolution processing, normalization processing, and introducing nonlinear features on the preliminary representation of the packet-level view, the preliminary representation of the session-level view, and the preliminary representation of the host-level view, respectively, to obtain a packet-level mask matrix, a session-level mask matrix, and a host-level mask matrix; Performing residual connection and normalization processing on the packet-level mask matrix and the packet-level preliminary view representation to obtain a packet-level representation matrix; Performing residual connection and normalization processing on the session-level mask matrix and the session-level view preliminary representation to obtain a session-level representation matrix; The host-level mask matrix and the host-level preliminary view representation are residually connected and normalized to obtain a host-level representation matrix.

6. The method according to claim 1, characterized in that The multi-task learning model includes an input module, a residual module and an output module. The input module mines and downsamples the global representation matrix to obtain a preliminary latent variable representation; the residual module performs deep feature mining on the preliminary latent variable representation to obtain a common latent variable representation; the output module outputs anomaly detection results corresponding to multiple tasks according to the number of tasks set in the configuration file.

7. A data traffic anomaly detection device based on multiple views, characterized in that: include: Import module, used to import data traffic of network ports through traffic mirroring; A preprocessing module, configured to preprocess the data traffic to form a session queue; an integration module, configured to screen the session queue to obtain target flows, mark each target flow and generate a corresponding label, and integrate the target flows to obtain a target flow set; A view feature extraction module, configured to extract multi-view features based on the target traffic set to obtain multi-view features; A preliminary extraction module, configured to extract a preliminary representation of the multi-view features; A screening module, configured to screen the preliminary representations to obtain a representation matrix; A splicing module is used to splice the obtained representation matrices to obtain a global representation matrix; a detection module, configured to input the global representation matrix into a multi-task learning model and output anomaly detection results corresponding to multiple tasks; The view feature extraction module performs multi-view feature extraction based on the target traffic set to obtain multi-view features, including: Extracting features of fragment length, packet length distribution, and flag information of target traffic in the target traffic set to obtain packet-level view features; Extracting features of inter-packet delay, transmission rate, and packet number distribution for target traffic in the target traffic set to obtain session-level view features; Extracting features of protocol type, protocol field, and protocol version from target traffic in the target traffic set to obtain protocol-level view features; Extracting features of the IP address, port number, and device bandwidth of the target traffic in the target traffic set to obtain host-level view features; The preliminary extraction module extracts a preliminary representation of the multi-view features, including: performing convolution processing, normalization processing, and linear rectification on the packet-level view features, session-level view features, and host-level view features, respectively, to obtain a preliminary representation of the packet-level view, a preliminary representation of the session-level view, and a preliminary representation of the host-level view; One-hot encoding is performed on the protocol-level view features, and feature extraction is performed based on a multi-layer perceptron. A preliminary representation of the protocol-level view is obtained after normalization. The screening module screens the packet-level view preliminary representation, the session-level view preliminary representation, and the host-level view preliminary representation based on an attention mask mechanism to obtain a packet-level representation matrix, a session-level representation matrix, and a host-level representation matrix; The screening module screens the preliminary representation of the protocol-level view based on a multi-head attention mechanism to obtain a protocol-level representation matrix.

Images