A method for identifying abnormal network traffic

Abstract

The application relates to an abnormal network flow identification method, and belongs to the technical field of anomaly identification. The method solves the problem that the prior art cannot accurately identify abnormal network flow. The method comprises the following steps: extracting a network flow sequence to be detected, and pre-processing the extracted network flow sequence; inputting the pre-processed network flow sequence to be detected into a trained abnormal flow identification model to obtain global features of the network flow sequence to be detected; the abnormal flow identification model is used for dynamically modeling input data in time and space to extract global features; calculating an abnormal value based on the global features of the network flow sequence to be detected and the global features of normal network flow samples; and judging whether the network flow sequence to be detected contains abnormal flow based on the abnormal value. The accuracy of abnormal network flow identification is improved.

Description

Technical Field

[0001] This invention relates to the field of anomaly identification technology, and in particular to a method for identifying abnormal network traffic. Background Technology

[0002] Efficiently and accurately identifying abnormal traffic in a network is a key component of a network security protection system, and is of paramount importance for timely detection of intrusions, prevention of attack spread, and ensuring the healthy operation of the network.

[0003] Existing rule / signature-based anomaly traffic identification methods rely on predefined rule bases or attack signatures (such as Snort, Suricata, etc.). When the content, protocol characteristics, or behavioral patterns of network traffic packets match malicious signatures in the rule base, they are judged as anomaly. These methods can only detect known attack patterns and are ineffective against new, unknown (zero-day) attacks or variant attacks. They are ill-suited to complex and ever-changing network environments and attack methods. Strict rules may lead to false positives (misclassifying normal traffic as anomaly), while outdated rules may result in false negatives (failure to detect new attacks).

[0004] Statistical threshold-based methods set thresholds by using baseline values ​​or distribution models of basic network traffic metrics (such as traffic rate, number of connections, and packet size distribution) under normal conditions. When real-time traffic metrics significantly deviate from the baseline or exceed the threshold, an anomaly is identified. These methods rely on simple statistical indicators and cannot effectively characterize complex attack behavior patterns and contextual relationships. The choice of threshold has a significant impact on detection effectiveness; too low a threshold leads to high false positives, while too high a threshold leads to high false negatives. Furthermore, static thresholds are difficult to adapt to dynamic changes in network traffic.

[0005] Traditional machine learning methods utilize classification (such as SVM and decision trees) or clustering algorithms to identify abnormal traffic based on manually extracted features (such as flow duration, packet count, and port distribution entropy). Model performance is highly dependent on the quality and effectiveness of manually designed features, requiring deep domain knowledge, and the feature design process is tedious and time-consuming. Manually designed features often only capture local or shallow characteristics of traffic, making it difficult to fully exploit high-dimensional, non-linear, and long-dependency traffic patterns.

[0006] With the development of artificial intelligence such as deep learning and reinforcement learning, abnormal traffic identification has also been achieved by training neural network models. However, existing network traffic anomaly identification is based on static computer networks, which is not suitable for dynamically changing computer networks, and therefore cannot accurately identify abnormal network traffic. Summary of the Invention

[0007] Based on the above analysis, the embodiments of the present invention aim to provide an abnormal network traffic identification method to solve the problem that existing models cannot accurately identify abnormal network traffic.

[0008] On one hand, embodiments of the present invention provide a method for identifying abnormal network traffic, comprising the following steps:

[0009] Extract the network traffic sequence to be detected, and preprocess the extracted network traffic sequence;

[0010] The preprocessed network traffic sequence to be detected is input into the trained abnormal traffic identification model to obtain the global features of the network traffic sequence to be detected; the abnormal traffic identification model is used to perform dynamic spatiotemporal modeling on the input data to extract global features;

[0011] Anomalies are calculated based on the global features of the network traffic sequence to be detected and the global features of normal network traffic samples. Based on the anomalies, it is determined whether there is abnormal traffic in the network traffic sequence to be detected.

[0012] Based on the above method, outliers are calculated in the following way:

[0013] Clustering of normal network traffic samples based on global features;

[0014] The cluster type closest to the network traffic sequence to be detected is determined based on the distance from the global features of the network traffic sequence to each cluster center;

[0015] Outliers are calculated based on the global characteristics of the network traffic sequence to be detected and the distribution distance between similar cluster types.

[0016] Based on the further improvement of the above method, outliers are calculated using the following formula based on the distribution distance between the global features of the network traffic sequence to be detected and the closest cluster type:

[0017] ;

[0018] in, This represents the center point of the closest cluster type. Represents the global features of the network traffic sequence to be detected. Robust Mahalanobis distance to the center of the nearest cluster type, The scaling factor representing the absolute deviation of the median. This indicates the adjustment parameter.

[0019] Based on a further improvement of the above method, the scaling factor for the absolute deviation of the median is calculated using the following formula:

[0020] ;

[0021] in, This indicates taking the median. This represents the median of the robust Mahalanobis distance from the global features of all samples in the closest cluster type to the center point. This represents the global features of samples in the closest cluster type.

[0022] Based on the further improvement of the above method, the robust Mahalanobis distance is calculated using the following formula:

[0023] ;

[0024] ;

[0025] ;

[0026] in, This represents the covariance matrix of the global features of samples in the closest cluster type. Represents the weighted covariance matrix. This represents the global features of samples in the closest cluster type. This indicates the number of samples in the closest cluster type. This represents the 0.95 quantile of the chi-square distribution of samples in the closest cluster type. express The weights, where the superscript T indicates transpose. This indicates the closest cluster type.

[0027] Based on the above method, a trained abnormal traffic identification model is obtained in the following way:

[0028] Normal network traffic is extracted from network traffic logs, and a sample set is constructed based on the timestamps of the network traffic; the sample includes time-series network traffic data.

[0029] Construct a dynamic graph neural network model; the dynamic graph neural network model includes a graph attention network, a recurrent neural network, and a graph convolutional network;

[0030] The graph attention network is used to extract spatial features of the input data; the recurrent neural network is used to extract temporal features based on the spatial features to obtain spatiotemporal features of the input data; and the graph convolutional network is used to globally fuse the spatiotemporal features to obtain global features.

[0031] The dynamic graph neural network model is trained based on the constructed sample set to obtain a trained abnormal traffic identification model.

[0032] Based on further improvements to the above method,

[0033] For each sample, a graph attention network corresponding to that time step is constructed based on the network traffic data at each time step; the graph attention network is used to extract the spatial features of each monitored end at the corresponding time step; thus obtaining the spatial feature sequence of each monitored end.

[0034] The recurrent neural network is used to extract the spatiotemporal features of each monitored end based on the spatial feature sequence of each monitored end.

[0035] The graph convolutional network performs global fusion of the spatiotemporal features of each monitored end to obtain global features.

[0036] Based on the further improvement of the above method, the nodes of the graph attention network corresponding to this time step correspond one-to-one with the monitored end.

[0037] Construct a graph attention network for each time step based on the network traffic data at that time step, including:

[0038] For each time step, iterate through each network traffic data point in that time step and construct edges between the nodes corresponding to the monitored end related to each network traffic data point.

[0039] If a node has no edges with itself or any other nodes, it is an invalid node; otherwise, it is a valid node. This yields the graph attention network corresponding to the given time step.

[0040] The graph attention network corresponding to the time step is used to extract the spatial features of each valid node at the time step; the spatial features of each invalid node at the time step are zero vectors.

[0041] Based on the further improvement of the above method, and using the constructed training sample set, the training loss is calculated using the following formula to train the dynamic graph neural network model:

[0042] ;

[0043] in, Indicates distance loss. This indicates a loss of consistency.

[0044] Based on the further improvement of the above method, the distance loss is calculated using the following formula:

[0045] ;

[0046] Where N represents the number of samples in the current training batch. This represents the global feature of the i-th sample in the current training batch. This represents the center of the hypersphere containing the sample. Denotes the 2-norm of a matrix. Let F represent the F-norm of the matrix, and L represent the number of layers in the dynamic graph neural network model. The first part represents the dynamic graph neural network model. Parameters of the layer network This represents the weighting parameter.

[0047] Compared with existing technologies, this application extracts the network sequence to be detected into a pre-trained abnormal traffic identification model, performs dynamic spatiotemporal modeling on the network traffic sequence to be detected to extract global features, and then calculates the outlier value of the network traffic sequence to be detected based on the global features of the network traffic sequence to be detected and the global features of normal network traffic samples, thereby determining whether the network traffic sequence to be detected is abnormal. By learning the dynamic change characteristics of computer networks through spatiotemporal modeling, the accuracy of identification is improved.

[0048] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0049] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0050] Figure 1 This is a flowchart of an abnormal network traffic identification method according to an embodiment of the present invention. Detailed Implementation

[0051] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0052] A specific embodiment of the present invention discloses a method for identifying abnormal network traffic, such as... Figure 1 As shown, it includes the following steps:

[0053] S1. Extract the network traffic sequence to be detected and preprocess the extracted network traffic sequence;

[0054] S2. Input the preprocessed network traffic sequence to be detected into the trained abnormal traffic identification model to obtain the global features of the network traffic sequence to be detected; the abnormal traffic identification model is used to perform dynamic spatiotemporal modeling on the input data to extract global features;

[0055] S3. Calculate outliers based on the global features of the network traffic sequence to be detected and the global features of normal network traffic samples, and determine whether there is abnormal traffic in the network traffic sequence to be detected based on the outliers.

[0056] Compared with existing technologies, the abnormal network traffic identification method provided in this embodiment extracts the network sequence to be detected into a trained abnormal traffic identification model, performs dynamic spatiotemporal modeling on the network traffic sequence to be detected to extract global features, and then calculates the abnormal value of the network traffic sequence to be detected based on the global features of the network traffic sequence to be detected and the global features of normal network traffic samples, thereby determining whether the network traffic sequence to be detected is abnormal. By learning the dynamic change characteristics of computer networks through spatiotemporal modeling, the accuracy of identification is improved.

[0057] In practice, since it is difficult to identify complex network attacks from a single traffic flow, it is necessary to extract a network traffic sequence over a period of time for detection.

[0058] During implementation, preprocessing of the extracted network traffic sequence includes deduplication of network traffic data at each time step.

[0059] Each network traffic item includes the network traffic timestamp, the identity information of the initiator (e.g., username and domain name), the identity information of the receiver (e.g., username and domain name), the type of network traffic (e.g., login, logout, device connection), and the packet size.

[0060] Since the collected data contains many duplicate events, in order to improve data processing efficiency without losing key semantics, duplicate deletion is performed to improve recognition efficiency.

[0061] In practice, for example, every second is considered a time step. For each time step of the network traffic sequence to be detected, duplicate network traffic in that time step is deleted. Duplicate network traffic is network traffic that is identical in all information except for the timestamp. In practice, only the network traffic with the earliest timestamp is retained, and duplicate traffic after that timestamp is deleted.

[0062] The preprocessed network traffic sequence to be detected is encoded and then input into a trained abnormal traffic identification model to obtain the global features of the network traffic sequence. In implementation, existing encoding methods can be used to encode the network traffic data.

[0063] The trained abnormal traffic identification model is obtained using the following method:

[0064] Normal network traffic is extracted from network traffic logs, and a sample set is constructed based on the timestamps of the network traffic; the sample includes time-series network traffic data.

[0065] Construct a dynamic graph neural network model; the dynamic graph neural network model includes a graph attention network, a recurrent neural network, and a graph convolutional network;

[0066] The graph attention network is used to extract spatial features of the input data; the recurrent neural network is used to extract temporal features based on the spatial features to obtain spatiotemporal features of the input data; and the graph convolutional network is used to globally fuse the spatiotemporal features to obtain global features.

[0067] The dynamic graph neural network model is trained based on the constructed sample set to obtain a trained abnormal traffic identification model.

[0068] During implementation, network traffic log data records all network traffic occurring at the monitored end, extracts normal network traffic from it, and constructs a sample set. Each sample in the sample set includes network traffic time-series data.

[0069] During implementation, network traffic is divided into batches according to timestamp order; for each batch, duplicate network traffic at each time step in that batch is deleted; and the network traffic of each batch after deduplication is used as a sample.

[0070] In practice, for example, network traffic per minute can be divided into batches.

[0071] Since the collected network traffic contains many duplicate events, in order to improve data processing efficiency without losing key semantics, duplicate network traffic at each time step is removed in accordance with the aforementioned method for deduplication. Each batch of network traffic after deduplication is encoded as a sample.

[0072] To accurately capture the dynamic characteristics of network traffic, a dynamic graph neural network model is constructed, which includes graph attention networks, recurrent neural networks, and graph convolutional networks.

[0073] In practice, computer networks undergo structural and attribute changes; computer nodes and edges (network connections, network traffic) appear, change, or disappear over time, thus exhibiting both spatial and temporal dynamism. Therefore, this invention uses a graph attention network to extract spatial features from the input data; a recurrent neural network to extract temporal features to obtain the spatiotemporal features of the input data; and then a graph convolutional network to globally fuse the spatiotemporal features to obtain global features, thereby enabling dynamic spatiotemporal modeling of the input data.

[0074] Specifically, for each sample, a graph attention network corresponding to that time step is constructed based on the network traffic data at each time step; the graph attention network is used to extract the spatial features of each monitored end at the corresponding time step; thus obtaining the spatial feature sequence of each monitored end.

[0075] Recurrent neural networks are used to extract temporal features from the spatial feature sequence of each monitored end to obtain the spatiotemporal features of each monitored end.

[0076] Graph convolutional networks globally fuse the spatiotemporal features of each monitored endpoint to obtain global features.

[0077] During implementation, since the monitored network changes continuously at each time step, a graph attention network is constructed for each time step to extract the spatial features of each monitored end at that time step, thereby obtaining the spatial feature sequence of each monitored end.

[0078] During implementation, each node of the graph attention network at each time step corresponds one-to-one with the monitored end.

[0079] Construct a graph attention network for each time step based on the network traffic data at that time step, including:

[0080] For each time step, iterate through each traffic data point in that time step and construct edges between the nodes corresponding to the monitored end related to each network traffic data point.

[0081] If a node has no edges with itself or any other nodes, it is an invalid node; otherwise, it is a valid node. This yields the graph attention network corresponding to the given time step.

[0082] The graph attention network corresponding to the time step is used to extract the spatial features of each valid node at the time step; the spatial features of each invalid node at the time step are zero vectors.

[0083] In implementation, the nodes of a graph attention network are all network endpoints in the monitoring network; that is, there is a one-to-one correspondence between the nodes of the graph attention network and the monitored endpoints. The edges between nodes are determined based on the network traffic.

[0084] For a given time step, each network traffic flow in that time step is traversed sequentially, and edges are constructed between the nodes corresponding to the monitored ends associated with each network traffic flow, thereby constructing the graph attention network corresponding to that time step.

[0085] It should be noted that the constructed edges are directed edges, meaning that each edge points from the node corresponding to the originating node of the network traffic to the node corresponding to the receiving node. There may be multiple edges between two nodes. An edge can also point to itself.

[0086] Since a node may not send network traffic to any other node within a given time step (i.e., it has no edges), it is an invalid node at that time step, and its spatial feature at that time step can be set to a zero vector. Nodes connected by edges are valid nodes at that time step.

[0087] In practice, the graph attention network corresponding to each time step is a stacked multi-layer network.

[0088] In implementation, it is important to note that each edge in the graph network has a corresponding embedding vector. The embedding vector is obtained by encoding information such as the type of network traffic and packet size corresponding to each edge.

[0089] The input data for each node in the first layer of the graph attention network is an encoded vector of the attribute information of the network endpoint corresponding to the node at the current time step. Attribute information includes, for example, the identity of the network endpoint, available CPU size, bandwidth, and storage. The input data for each node in other layers is the output data of the corresponding node in the previous layer.

[0090] During implementation, a graph attention network corresponding to the time step is used to extract the spatial features of each valid node at that time step.

[0091] Specifically, for each valid node, each graph attention layer extracts its feature vector using the following formula:

[0092] ;

[0093] in, The first character of the graph attention network is shown. The feature vector of the i-th valid node in the layer. Let H represent the set of H-hop neighbors originating from the i-th node in a graph attention network. Indicates the attention coefficient. The first character of the graph attention network is shown. The network parameters of the layer, The first character of the graph attention network is shown. The feature vector of the j-th valid node in the layer, Let represent the embedding vector of the k-th path from the i-th valid node to the j-th valid node. This represents the activation function. This represents the number of paths from the i-th valid node to the j-th valid node.

[0094] The feature vector of each valid node output by the final graph attention layer is the spatial feature of that valid node at that time step.

[0095] In implementation, the set of H-hop neighbor nodes originating from the i-th node is the set of all nodes reachable via H-hops from the i-th valid node, following the edges in the graph, including the i-th node itself and the neighbor nodes of the aggregation node. High-order relationships between nodes in the graph are captured through stacked multi-layer graph attention. Since each neighbor node and the edges between nodes have different effects on the i-th node, graph attention is used to assign different weights, i.e., attention coefficients, to each neighbor node.

[0096] For each path from the i-th valid node to the j-th valid node, the embedding vectors corresponding to the edges that make up the path are fused to obtain the embedding vector corresponding to the path. In implementation, a weighted method can be used for fusion, where edges closer to the i-th valid node have higher weights, and edges farther from the i-th valid node have lower weights.

[0097] Specifically, the attention coefficient is calculated using the following formula:

[0098] ;

[0099] in, Indicates the first The feature vector of the s-th valid node in the layer. Indicates the first The feature vector of the j-th valid node in the layer, The first character of the graph attention network is shown. Attention vectors of layers The transpose of is used to calculate the attention coefficient. || represents the activation function, and || represents concatenation. This represents the number of paths from the i-th valid node to the s-th valid node. This represents the embedding vector of the k-th path from the i-th valid node to the s-th valid node.

[0100] After calculating the spatial features of each node at each time step using a graph attention network, the spatial feature sequence of each monitored end is obtained according to the time step order. If a node is an invalid node at a certain time step, the spatial feature of the monitored end corresponding to that node at that time step is set to a zero vector.

[0101] During implementation, long-term dependencies are learned through recurrent neural networks, and the temporal features of each monitored end are extracted.

[0102] In practice, a Long Short-Term Memory (LSTM) network can be used to extract temporal features and obtain the spatiotemporal features of each monitored end.

[0103] In practice, after obtaining the spatiotemporal features of each monitored end through a recurrent neural network, the spatiotemporal features of the batch are fused through a graph convolutional network to obtain global features.

[0104] Specifically, each node in the graph convolutional network corresponds one-to-one with the monitored end; the edges between the nodes of the graph convolutional network are constructed in the following way:

[0105] For each pair of nodes in a graph convolutional network, calculate the frequency of visits between that pair of nodes;

[0106] If the access frequency is greater than a preset threshold, an edge is constructed between the node pairs, and the weight of the edge is the access frequency.

[0107] In practice, for a given node pair, its access frequency is the ratio of the amount of network traffic between that node pair within that batch to the total amount of network traffic in that batch. If the access frequency is less than or equal to a preset threshold, it indicates that the network traffic frequency between the node pairs is very low, and its impact on the nodes can be ignored. Therefore, to improve computational efficiency, edges are only constructed between node pairs with access frequencies greater than the preset threshold, and weights are assigned to these edges.

[0108] The input to a graph convolutional network is the spatiotemporal features of each node. Graph convolutional networks also employ a multi-layer stacked structure. The feature extraction process for graph convolutional networks is a current technique.

[0109] In practice, the features of each node output by the graph convolutional network are concatenated to obtain the global features corresponding to the sample.

[0110] When training a dynamic graph neural network model based on a constructed training sample set, the parameters of the dynamic graph neural network model are updated by calculating the training loss.

[0111] Specifically, based on the constructed training sample set, the training loss is calculated using the following formula to train the dynamic graph neural network model:

[0112] ;

[0113] in, express, This indicates a loss of consistency.

[0114] Since the training set consists of normal samples, the loss is calculated by minimizing the sample distance and consistency error.

[0115] Specifically, the distance loss is calculated using the following formula:

[0116] ;

[0117] Where N represents the number of samples in the current training batch. This represents the global feature of the i-th sample in the current training batch. This represents the center of the hypersphere containing the sample. Denotes the 2-norm of a matrix. Let F represent the F-norm of the matrix, and L represent the number of layers in the dynamic graph neural network model. The first part represents the dynamic graph neural network model. Parameters of the layer network This represents the weighting parameter.

[0118] The first term in the distance loss is used to aggregate normal samples into a hypersphere, that is, to compress normal samples as much as possible into a special space. Within a region that is as compact as possible. The second term represents network weight regularization to prevent overfitting.

[0119] During implementation, the center This can be achieved by extracting some normal samples and mapping them to a pre-trained network. Given the spatial dimension, calculate the mean of the vector after mapping the normal samples to obtain the center. .

[0120] If a node's neighbor information can be well used to predict its own features, it means that the node's features are "consistent" with those of its neighbors. Normal nodes exhibit consistency, while abnormal nodes may be difficult to reconstruct due to their features being incompatible with their neighbors' patterns, thus lacking consistency. Consistency loss encourages learned node embeddings to preserve the structure and feature patterns of their local neighborhoods.

[0121] Specifically, the consistency loss is calculated using the following formula:

[0122] ;

[0123] Where N represents the number of samples in the current training batch. This represents the number of nodes in a graph convolutional network. Let represent the set of neighboring nodes of the j-th node of the i-th sample in a graph convolutional network. This represents the number of neighboring nodes of the j-th node of the i-th sample in the graph convolutional network. This represents the feature of the k-th node output by the graph convolutional network. This represents the feature of the j-th node output by the graph convolutional network. Represents the refactorer. This represents the 2-norm of a matrix.

[0124] During implementation, the refactorer can be an MLP or a linear layer.

[0125] The parameters of the dynamic graph neural network model are updated by backpropagating the gradient through loss calculation. When the model converges, the training ends, and a trained network attack detection model is obtained.

[0126] Network traffic detection can be performed based on a pre-trained network attack detection model. During implementation, the pre-processed network traffic sequence to be detected is input into the pre-trained abnormal traffic identification model to obtain the global features of the network traffic sequence.

[0127] Anomalies are calculated based on the global features of the network traffic sequence to be detected and the global features of normal network traffic samples. Based on the anomalies, it is determined whether there is abnormal traffic in the network traffic sequence to be detected.

[0128] Specifically, outliers are calculated using the following method:

[0129] S31. Cluster the normal network traffic samples based on the global features of the normal network traffic samples;

[0130] S32. Determine the closest cluster type of the network traffic sequence to be detected based on the distance from the global features of the network traffic sequence to each cluster center;

[0131] S33. Calculate outliers based on the global features of the network traffic sequence to be detected and the distribution distance between similar cluster types.

[0132] In implementation, existing clustering methods can be used to train a clustering model on the samples in the sample set based on the global features of the samples, resulting in multiple cluster types. The centroid of each cluster can be the median vector of the global features of the samples of that type (a vector composed of the medians in each feature dimension). The median is less affected by outliers and is therefore more robust.

[0133] The global features of the network traffic sequence to be detected are input into the trained clustering model to calculate the probability of each cluster type input into the model, and the type with the highest probability is selected as the closest cluster type.

[0134] In practice, the distance between the global features of the network traffic sequence to be detected and each cluster center can be calculated, and the cluster type containing the cluster center with the smallest distance can be selected as the closest cluster type.

[0135] Because features are correlated, to improve the robustness of anomaly calculation, outliers are calculated based on the distribution distance between the global features of the network traffic sequence to be detected and the closest cluster type. The specific formula is:

[0136] ;

[0137] in, This represents the center point of the closest cluster type. Represents the global features of the network traffic sequence to be detected. Robust Mahalanobis distance to the center of the nearest cluster type, The scaling factor representing the absolute deviation of the median. This indicates an adjustment parameter, which is a very small positive number to prevent the denominator from being zero.

[0138] The numerator of the above formula represents the standardized distance from the network traffic sequence to be detected to the nearest neighbor cluster, and the denominator represents the inherent dispersion scale of the closest cluster type. If the ratio is approximately 1, it is located at the typical boundary of that nearest neighbor cluster type; if it exceeds 1, it is likely an anomaly. In implementation, a threshold can be set empirically; outliers exceeding the threshold are judged as abnormal network traffic.

[0139] Abnormal traffic can be accurately identified by measuring the ratio of the degree to which network traffic being detected is closest to the cluster center to the degree of dispersion of the cluster type itself.

[0140] Specifically, the scaling factor for the absolute deviation of the median is calculated using the following formula:

[0141] ;

[0142] in, This indicates taking the median. This represents the median of the robust Mahalanobis distance from the global features of all samples in the closest cluster type to the center point. This represents the global features of samples in the closest cluster type. This represents the global feature of each sample in the closest cluster type, calculated by comparing each sample with... Mahalanobis distance and The deviation is then taken as the median to obtain the scaling factor.

[0143] The constant 1.4826 makes the MAD consistent with the standard deviation of the normal distribution.

[0144] Specifically, the robust Mahalanobis distance is calculated using the following formula:

[0145] ;

[0146] ;

[0147] ;

[0148] in, This represents the covariance matrix of the global features of samples in the closest cluster type. Represents the weighted covariance matrix. This represents the global features of samples in the closest cluster type. This indicates the number of samples in the closest cluster type. This represents the 0.95 quantile of the chi-square distribution of samples in the closest cluster type. express The weights, where the superscript T indicates transpose. This indicates the closest cluster type.

[0149] It should be noted that the degrees of freedom of the chi-square distribution are the lengths of the global features.

[0150] In practice, when calculating the weights, points near the center point retain all weights, while the squared Mahalanobis distance to the center is greater than 1. Points far from the center may be outliers and their weights should be reduced. Points far from the center are given smaller weights, thus reducing their impact on covariance moment estimation and improving the accuracy of anomaly identification.

[0151] By employing median centering and weighted covariance to resist outlier contamination, the accuracy of anomaly identification is improved.

[0152] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0153] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An abnormal network traffic identification method, characterized by, The method comprises the following steps: extracting a network traffic sequence to be detected, and preprocessing the extracted network traffic sequence; the preprocessing of the extracted network traffic sequence comprises: removing duplicates for network traffic data of each time step; inputting the preprocessed network traffic sequence to be detected into a trained abnormal traffic identification model to obtain global features of the network traffic sequence to be detected; the abnormal traffic identification model is used for dynamic space-time modeling of input data to extract global features; calculating an abnormal value based on the global features of the network traffic sequence to be detected and global features of normal network traffic samples, and judging whether the network traffic sequence to be detected has abnormal traffic based on the abnormal value; the abnormal traffic identification model comprises a graph attention network, a recurrent neural network and a graph convolution network; the graph attention network is used for extracting spatial features of input data; the recurrent neural network is used for extracting time features based on the spatial features to obtain space-time features of the input data; and the graph convolution network is used for globally fusing the space-time features to obtain global features; for each sample, a graph attention network corresponding to each time step is constructed according to network traffic data of each time step; the graph attention network is used for extracting spatial features of each monitored end at the corresponding time step; and spatial feature sequences of each monitored end are obtained; the recurrent neural network is used for extracting time features based on the spatial feature sequences of each monitored end to obtain space-time features of each monitored end; the graph convolution network globally fuses the space-time features of each monitored end to obtain global features; nodes of the graph attention network corresponding to the time step correspond to the monitored ends one by one; the graph attention network corresponding to each time step is constructed according to network traffic data of each time step, comprising: for each time step, each network traffic data of the time step is sequentially traversed, and edges are constructed between nodes corresponding to monitored ends related to each network traffic data; if a node has no edges between itself and other nodes, the node is an invalid node, otherwise, the node is a valid node; and the graph attention network corresponding to the time step is obtained; the graph attention network corresponding to the time step is used for extracting spatial features of each valid node of the time step; and spatial features of each invalid node of the time step are zero vectors.

2. The method of claim 1, wherein, The abnormal value is calculated in the following manner: the normal network traffic samples are clustered based on the global features of the normal network traffic samples; the global features of the network traffic sequence to be detected are used to determine a closest cluster type of the network traffic sequence to be detected according to distances to cluster centers; the abnormal value is calculated based on a distribution distance between the global features of the network traffic sequence to be detected and the closest cluster type.

3. The method of claim 2, wherein, The abnormal value is calculated based on a distribution distance between the global features of the network traffic sequence to be detected and the closest cluster type in the following formula: ； wherein, denotes the center point of the closest cluster type, denotes a global feature of the network traffic sequence to be detected denotes the robust Mahalanobis distance to the center point of the closest cluster type, denotes a scaling factor for the median absolute deviation, denotes a tuning parameter.

4. The method of claim 3, wherein, The scaling factor of the median absolute deviation is calculated in the following formula: ； wherein, denotes the median, denotes the median of the robust Mahalanobis distances of all samples in the closest cluster type to the center point, denotes the global features of the samples in the closest cluster type.

5. The method of claim 3, wherein, The robust Mahalanobis distance is calculated in the following formula: ； ； ； wherein denotes the covariance matrix of the global features of the samples in the closest cluster type, denotes the weighted covariance matrix, denotes the global features of the samples in the closest cluster type, denotes the number of samples in the closest cluster type, denotes the 0.95 quantile of the chi-square distribution of the samples in the closest cluster type, denotes the weight of the, upper index T denotes the transpose, denotes the closest cluster type.

6. The method of claim 1, wherein, The trained abnormal traffic identification model is obtained in the following manner: normal network traffic is extracted based on network traffic logs, and a sample set is constructed based on time stamps of the network traffic; the sample comprises network traffic time series data; construct a dynamic graph neural network model; the dynamic graph neural network model comprises a graph attention network, a recurrent neural network and a graph convolution network; train the dynamic graph neural network model based on the constructed sample set to obtain a trained abnormal traffic identification model.

7. The method of claim 6, wherein, Based on the constructed training sample set, the dynamic graph neural network model is trained by using the following formula to calculate the training loss: ； wherein, denotes a distance loss, denotes a consistency loss.

8. The method of claim 7, wherein, The distance loss is calculated by using the following formula: ； wherein N represents the number of samples of the current training batch, represents the global feature of the i-th sample of the current training batch, represents the center of the hypersphere where the sample is located, represents the 2-norm of a matrix, represents the F-norm of a matrix, and L represents the number of layers of the dynamic graph neural network model, represents the i-th layer of the dynamic graph neural network model, represents the parameters of the i-th layer network, represents the weight parameters.

Images