Network traffic anomaly detection model training method and network traffic anomaly detection method

Abstract

The invention provides a network traffic anomaly detection model training method and a network traffic anomaly detection method, and relates to the field of artificial intelligence. Comprising the following steps: extracting historical network flow characteristic information, and calculating a mutual information matrix between network nodes; time sequence data is extracted, and a trend significance score is generated through Mann-Kendall trend test; fusing the mutual information matrix and the trend significance score to construct a dynamic adjacency matrix; and executing a graph convolution operation based on a dynamic adjacency matrix to update traffic features, calculating a loss value through a loss function according to the updated network traffic feature information, and updating weight parameters of the network traffic anomaly detection model according to the loss value. According to the scheme, the noise interference of the manually preset adjacent matrix is eliminated through the mutual information matrix, the trend significance score is utilized to capture the time sequence abnormity such as traffic abrupt change, and the dynamic adjacent matrix adaptively balances the spatial-temporal feature weight, so that the graph convolutional network accurately identifies the complex attack scene, and the model detection precision and the scene generalization ability are significantly improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for training a network traffic anomaly detection model and a method for detecting network traffic anomalies. Background Technology

[0002] With the rapid development of network technology, the number of internet users has exploded, and the complexity and volume of network traffic are constantly increasing. However, network technology is a double-edged sword; while driving rapid social development, it has also brought a series of cybersecurity problems. Therefore, abnormal network traffic early warning and tracing are important issues for cybersecurity centers. As a carrier of information transmission and interaction, network traffic contains a large amount of information related to network behavior. By analyzing network traffic data and promptly detecting various abnormal network behaviors to achieve network situational awareness, appropriate measures can be taken to defend against cyberattacks and maintain cyberspace security.

[0003] Currently, traffic anomaly detection methods based on graph neural networks have become one of the mainstream technologies. Traditional Graph Convolutional Networks (GCNs) model network nodes (such as IP addresses and devices) as vertices in a graph and construct a static adjacency matrix based on the connection relationships between nodes (such as physical topology and communication frequency) to represent the network spatial structure. In anomaly detection tasks, GCNs capture spatial dependencies in the network by aggregating the traffic characteristics of neighboring nodes (such as packet size, protocol type, and connection frequency) and combine this with a classifier (such as a fully connected layer) to identify anomalous nodes. For example, by constructing an adjacency matrix using predefined IP subnet relationships or historical communication records, GCNs can perform feature propagation and pattern mining on coordinated attack nodes in distributed denial-of-service (DDoS) attacks, thereby achieving anomaly traffic classification. These methods demonstrate high detection efficiency in complex network scenarios due to their effective utilization of network topology information.

[0004] However, the performance of traditional GCNs heavily relies on predefined static adjacency matrices. In real-world network environments, the relationships between nodes often change dynamically with traffic (e.g., abnormal communication caused by sudden attacks, temporary connections triggered by load balancing), and static adjacency matrices cannot reflect these dynamic relationships in real time. For example, in Advanced Persistent Threat (APT) scenarios, attackers penetrate networks through low-frequency, dispersed inter-node collaboration, and their communication patterns evolve gradually over time; static graph structures struggle to capture these temporal dependencies. Furthermore, manually defining adjacency matrices relies on prior knowledge (e.g., fixed subnetting), which can easily introduce noisy connections or miss key relationships when the network size increases or the topology changes frequently, leading to increased false detection rates and decreased generalization ability. This deficiency makes traditional GCNs ill-suited for dynamic network environments, becoming a key bottleneck restricting the improvement of anomaly detection accuracy. Summary of the Invention

[0005] This application provides a network traffic anomaly detection model training method and a network traffic anomaly detection method to solve the technical problem that traditional GCN relies on a predefined static adjacency matrix, which cannot capture dynamically changing node relationships, is difficult to adapt to dynamic network environments, and limits the accuracy of network traffic anomaly detection.

[0006] Firstly, this application provides a network traffic anomaly detection model training method, including:

[0007] Based on historical network traffic data, network traffic characteristic information of each network node is extracted, wherein the historical network traffic data is generated by communication between the network nodes.

[0008] Calculate the mutual information of the network traffic feature information of each pair of network nodes to obtain the mutual information matrix of the network nodes. The elements in the mutual information matrix are the mutual information to indicate the degree of correlation of the network traffic features between the network nodes.

[0009] Based on the historical network traffic data, network traffic time series data of each network node is extracted. The network traffic time series data is used to indicate the network traffic change characteristics of each network node within a preset time window.

[0010] The network traffic time series data is subjected to Mann-Kendall trend test to obtain the network traffic trend significance score between each pair of network nodes. The network traffic trend significance score indicates the significance of the time-series rise and fall trend of network traffic between each pair of network nodes.

[0011] A dynamic adjacency matrix is ​​generated based on the trend significance score and the mutual information matrix;

[0012] Perform a dynamic graph convolution operation on the network traffic feature information and the dynamic adjacency matrix to obtain the updated network traffic feature information;

[0013] Based on the updated network traffic feature information, a loss value is calculated using a loss function, and the weight parameters of the network traffic anomaly detection model are updated based on the loss value.

[0014] Secondly, this application provides a method for detecting network traffic anomalies. The method is based on a network traffic anomaly detection model, which is trained using the training method described in the first aspect. The method includes:

[0015] Obtain network traffic data for each network node, wherein the network traffic data is generated by communication between the network nodes;

[0016] The network traffic data is input into the network traffic anomaly detection model, which extracts network traffic feature information from the network traffic data and calculates anomaly scores for the network traffic data based on the network traffic feature information.

[0017] Thirdly, this application provides a network traffic anomaly detection model training device, the device comprising:

[0018] The first extraction module is used to extract network traffic characteristic information of each network node based on historical network traffic data, wherein the historical network traffic data is generated by communication between the network nodes.

[0019] The mutual information calculation module is used to calculate the mutual information of the network traffic feature information of each pair of network nodes, and obtain the mutual information matrix of the network nodes. The elements in the mutual information matrix are the mutual information to indicate the degree of correlation of the network traffic features between the network nodes.

[0020] The second extraction module is used to extract network traffic time series data of each network node based on the historical network traffic data. The network traffic time series data is used to indicate the network traffic change characteristics of each network node within a preset time window.

[0021] The trend testing module is used to perform Mann-Kendall trend testing on the network traffic time series data to obtain the network traffic trend significance score between each pair of network nodes. The network traffic trend significance score indicates the significance of the time-series rise and fall trend of network traffic between each pair of network nodes.

[0022] A generation module is used to generate a dynamic adjacency matrix based on the trend significance score and the mutual information matrix;

[0023] The convolution module is used to perform dynamic graph convolution operations on the network traffic feature information and the dynamic adjacency matrix to obtain updated network traffic feature information.

[0024] The weight update module is used to calculate the loss value through a loss function based on the updated network traffic feature information, and update the weight parameters of the network traffic anomaly detection model based on the loss value.

[0025] Fourthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0026] The memory stores computer-executed instructions;

[0027] The processor executes computer execution instructions stored in the memory to implement the method as described in the first aspect, or to implement the method as described in the second aspect.

[0028] Fifthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect or the method described in the second aspect.

[0029] Sixthly, this application provides a computer program product comprising a computer program that, when executed by a processor, implements the method described in the first aspect or the method described in the second aspect.

[0030] The technical effects of the network traffic anomaly detection model training method and the network traffic anomaly detection method provided in this application are as follows:

[0031] First, a mutual information matrix is ​​calculated based on network traffic feature information. Since mutual information can quantify the nonlinear feature correlation between nodes, it can dynamically represent real-time spatial relationships, avoiding noise interference and correlation omissions caused by manually preset static adjacency matrices. Second, a trend significance score is generated by analyzing network traffic time series data through MK trend test. Since this score accurately depicts the temporal rise and fall trend of node traffic (such as a sudden surge in traffic caused by a sudden attack), it can reveal the dynamic evolution law of node behavior from the time dimension. Furthermore, a dynamic adjacency matrix is ​​generated by combining the mutual information matrix and the trend significance score. Since this matrix integrates the spatial correlation between nodes and the temporal trend significance, graph convolution operations can accurately capture spatiotemporal collaborative anomalies across nodes in complex attack scenarios (such as low-frequency APT attacks). Finally, the prediction error of the updated network traffic feature information is calculated through the loss function and the model weight parameters are updated in reverse. Since the model optimizes the weights of mutual information calculation, trend analysis, and dynamic graph convolution end-to-end, it can adaptively learn abnormal patterns in dynamic network environments without manual intervention, significantly improving the model's detection accuracy and scenario generalization ability. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0033] Figure 1 This is a schematic diagram of the network traffic anomaly detection model training method provided in the embodiments of this application;

[0034] Figure 2 This is a schematic diagram of the method for generating a dynamic adjacency matrix provided in an embodiment of this application;

[0035] Figure 3 This is a schematic flowchart of the network traffic anomaly detection method provided in the embodiments of this application;

[0036] Figure 4 This is a schematic diagram of the network traffic anomaly detection model training device provided in an embodiment of this application;

[0037] Figure 5 This is a schematic diagram of the electronic device structure provided in an embodiment of this application.

[0038] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0039] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0040] It should be noted that the network traffic anomaly detection model training method and network traffic anomaly detection method provided in this application can be used in the field of artificial intelligence technology, or in any field other than artificial intelligence technology. The application fields of the network traffic anomaly detection model training method and network traffic anomaly detection method provided in this application are not limited.

[0041] Currently, traffic anomaly detection methods based on graph neural networks have become one of the mainstream technologies. Traditional Graph Convolutional Networks (GCNs) model network nodes (such as IP addresses and devices) as vertices in a graph and construct a static adjacency matrix based on the connection relationships between nodes (such as physical topology and communication frequency) to represent the network spatial structure. In anomaly detection tasks, GCNs capture spatial dependencies in the network by aggregating the traffic characteristics of neighboring nodes (such as packet size, protocol type, and connection frequency) and combine this with a classifier (such as a fully connected layer) to identify anomalous nodes. For example, by constructing an adjacency matrix using predefined IP subnet relationships or historical communication records, GCNs can perform feature propagation and pattern mining on coordinated attack nodes in distributed denial-of-service (DDoS) attacks, thereby achieving anomaly traffic classification. These methods demonstrate high detection efficiency in complex network scenarios due to their effective utilization of network topology information.

[0042] However, the performance of traditional GCNs heavily relies on predefined static adjacency matrices. In real-world network environments, the relationships between nodes often change dynamically with traffic (e.g., abnormal communication caused by sudden attacks, temporary connections triggered by load balancing), and static adjacency matrices cannot reflect these dynamic relationships in real time. For example, in Advanced Persistent Threat (APT) scenarios, attackers penetrate networks through low-frequency, dispersed inter-node collaboration, and their communication patterns evolve gradually over time; static graph structures struggle to capture these temporal dependencies. Furthermore, manually defining adjacency matrices relies on prior knowledge (e.g., fixed subnetting), which can easily introduce noisy connections or miss key relationships when the network size increases or the topology changes frequently, leading to increased false detection rates and decreased generalization ability. This deficiency makes traditional GCNs ill-suited for dynamic network environments, becoming a key bottleneck restricting the improvement of anomaly detection accuracy.

[0043] Since mutual information measures the nonlinear dependency between two variables, in dynamic graph modeling, MI can quantify the real-time correlation strength of traffic characteristics between nodes, and its advantages include:

[0044] Unsupervised learning: It does not require prior knowledge and directly mines potential correlations from data (such as hidden C&C communication paths in APT attacks).

[0045] Nonlinear capture: Breaking through the linear assumption of traditional correlation coefficients (such as Pearson), it adapts to complex attack patterns (such as asymmetric dependencies in encrypted traffic).

[0046] Dynamic updates: The MI matrix is ​​calculated in real time based on the traffic data within the sliding window, reflecting the instantaneous state of node associations.

[0047] The Mann-Kendall (MK) trend test is a nonparametric statistical method used to detect monotonic trends (such as continuous growth / decline) in time series. Its core indicator is:

[0048] Z-score: Quantifies the strength of a trend (positive values ​​indicate an upward trend, negative values ​​indicate a downward trend);

[0049] p-value: assesses the significance of the trend (reject the null hypothesis of no trend when p < threshold).

[0050] Therefore, the specific problems that MK testing can solve are:

[0051] Temporal dependency modeling: capturing the gradual evolution of attack traffic over time (such as the gradually increasing communication frequency in APT attacks).

[0052] Suppress random fluctuations: Filter short-term noise (such as normal traffic jitter) by using the p-value to focus on persistent abnormal trends.

[0053] Therefore, combining mutual information matrix and MK detection can be considered for network traffic anomaly detection.

[0054] To address the aforementioned issues, the technical concept of this application is as follows: First, network traffic feature information of each network node is extracted based on network traffic data. Then, mutual information is calculated based on the network traffic feature information to obtain the mutual information matrix of the network nodes. Network traffic time series data of the network nodes is obtained from the network traffic data. This time series data indicates the network traffic change characteristics of each network node within a preset time window. A MK trend test is performed on the network traffic time series data of the network nodes to obtain a network traffic trend significance score for each network node. This trend significance score indicates the significance of the temporal rise and fall trend of network traffic for each network node. Based on the trend significance score and the mutual information matrix, a dynamic adjacency matrix is ​​obtained. A dynamic graph convolution operation is then performed on the network traffic feature information and the dynamic adjacency matrix to obtain updated network traffic feature information. Finally, a loss value is calculated using a damage function based on the updated network traffic feature information, and the weight parameters of the model are updated based on the loss value.

[0055] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0056] Figure 1 This is a schematic diagram of the network traffic anomaly detection model training method provided in the embodiments of this application. The execution subject of this method can be a network traffic analysis server or a cloud computing platform, such as... Figure 1 As shown, the method includes:

[0057] S101. Based on historical network traffic data, extract the network traffic characteristic information of each network node. The historical network traffic data is generated by communication between each network node.

[0058] It should be noted that, in this application, a network node refers to an entity with independent identification and behavioral capabilities in network communication, including but not limited to:

[0059] Physical devices: servers, routers, switches, firewalls, IoT terminals (such as cameras and sensors);

[0060] Logical entities such as:

[0061] 1. IP address: source IP, destination IP (e.g., the public IP of the web server, the private IP of the internal database).

[0062] 2. Virtual nodes: Virtual machines (VMs), containers (Docker / Kubernetes Pods), and microservice instances in a cloud environment;

[0063] User terminals: personal computers, mobile devices (phones / tablets), smart home appliances (such as smart TVs);

[0064] Network services: Domain Name System (DNS) server, load balancer, API gateway.

[0065] Specifically, after obtaining historical network traffic data, the data needs to be preprocessed, including the following steps:

[0066] 1. Feature Classification: Based on data type, historical network traffic data is divided into numerical (such as traffic bytes, response time) and categorical (such as protocol type, port number) types, and the numerical type is further subdivided into continuous (bandwidth utilization) and discrete (TCP connection count) types.

[0067] 2. Feature Encoding: Since categorical features cannot be directly input into the anomaly detection model, one-hot encoding can be used to convert protocol types (such as HTTP / HTTPS) into binary vectors, or embedding can be used to map high cardinality features (such as IP addresses) into low-dimensional dense vectors.

[0068] 3. Normalization and standardization: To eliminate differences in units, continuous numerical features are normalized using Min-Max (e.g., scaling the bandwidth value to the [0,1] interval) or standardized using Z-Score (e.g., adjusting the mean of the delay to 0 and the variance to 1), thereby ensuring that different features contribute equally to model training.

[0069] S102. Calculate the mutual information of network traffic characteristics of each pair of network nodes to obtain the mutual information matrix of network nodes. The elements in the mutual information matrix are mutual information to indicate the degree of correlation of network traffic characteristics between network nodes.

[0070] Specifically, this step quantifies the nonlinear correlation of network traffic characteristics between nodes using mutual information (MI) to replace manually defining the adjacency matrix. Since directly calculating the MI of high-dimensional data is highly complex, a binning method can be used for approximate estimation. Specific implementation methods may include:

[0071] 1. Divide continuous features into equal-width or equal-frequency intervals (for example, the number of bytes in traffic can be divided into 10 intervals).

[0072] II. Statistical Network Node Pairs Calculate the joint probability based on the co-occurrence frequency of network feature information:

[0073]

[0074] MI is calculated using the following formula:

[0075] S103. Based on historical network traffic data, extract the network traffic time series data of each network node. The network traffic time series data is used to indicate the network traffic change characteristics of each network node within a preset time window.

[0076] The approach in this step is to segment historical network traffic data into time-series data to capture the gradual changes in attack behavior. For example, network traffic time-series data can be statistically aggregated for each network node within a time window to generate multi-dimensional time series. Basic statistics may include:

[0077] Traffic volume: total number of bytes, number of packets, and average packet size within a time window.

[0078] Latency metrics: average response time, standard deviation of latency, maximum / minimum RTT.

[0079] Protocol distribution: Percentage of HTTP / HTTPS / TCP / UDP requests.

[0080] It can also collect high-level features, such as:

[0081] Surge detection: The ratio of current traffic within the current time window to the historical average;

[0082] Entropy calculation: Information entropy of protocol type or target port (detecting scanning behavior).

[0083] Since time series data needs to reflect the dynamic evolution of network node behavior, it is necessary to choose an appropriate time granularity and statistical measures. Specifically, the time window can be a fixed window (e.g., divided into 15-minute intervals, counting the number of TCP SYN packets for each node within each window) or an event-driven window (e.g., divided into windows every 1000 requests). Next, Z-score normalization can be performed on the time series data of each network node.

[0084]

[0085] in , These are the mean and standard deviation of the current time series data.

[0086] S104. Perform the Mann-Kendall trend test on the network traffic time series data to obtain the network traffic trend significance score between each pair of network nodes. The network traffic trend significance score indicates the degree of significance of the time-series rise and fall trend of network traffic between each pair of network nodes.

[0087] The technical approach of this step is to identify significant trends in traffic timing through nonparametric tests to distinguish between normal fluctuations and potential attacks. Since the MK test does not require assumptions about data distribution, it is suitable for complex network environments. Specific implementation methods may include:

[0088] For network node pairs Network traffic time series data Calculate the standardized Z-value:

[0089]

[0090] in,

[0091]

[0092] It is a symbolic function.

[0093] Next, calculate based on the Z value. The value (e.g., by looking up a standard normal distribution table) defines the significance score of network traffic trends as: , The smaller the value, the higher the score.

[0094] It should be noted that Z > 1.96 indicates a significant upward trend, while Z < -1.96 indicates a significant downward trend.

[0095] S105. Generate a dynamic adjacency matrix based on the trend significance score and mutual information matrix;

[0096] The technical approach in this step is to construct a dynamic adjacency matrix by linearly weighting and fusing MI (Minimum Interchangeability) and trend scores to balance spatial correlation and temporal trend. Since the contribution of spatiotemporal features varies across different scenarios, adjustable weights need to be designed. A specific implementation could be as follows:

[0097] First, the weights can be dynamically adjusted based on network topology stability (e.g., reducing the MI weight when the topology changes frequently):

[0098]

[0099] in, Preset weights.

[0100] S106. Perform dynamic graph convolution operation on the network traffic feature information and the dynamic adjacency matrix to obtain the updated network traffic feature information.

[0101] The purpose of this step is to enhance the detectability of anomalous signals by aggregating node features through dynamic graph convolution. Since the dynamic adjacency matrix reflects real-time correlation strength, an efficient convolution operator needs to be designed. Specifically,

[0102]

[0103] in, This is the traffic characteristic matrix of network nodes. For learnable parameters, This is the LeakyReLU activation function.

[0104] Furthermore, it is possible to... Perform sparsity optimization to reduce computational cost.

[0105] S107. Based on the updated network traffic characteristic information, calculate the loss value using the loss function, and update the weight parameters of the network traffic anomaly detection model based on the loss value.

[0106] The purpose of this step is to optimize model parameters through supervised learning to distinguish between normal and abnormal traffic patterns. Specifically, normal and abnormal nodes can be labeled in the training data, i.e., historical network traffic data. Then, the contrastive loss can be calculated to increase the feature distance between normal and abnormal nodes.

[0107]

[0108] in, The interval hyperparameter controls the minimum feature distance between normal and abnormal nodes; , This is to update the network traffic characteristics information. The calculation logic is as follows: if normal nodes... with abnormal nodes distance If the distance is increased, a loss is incurred, forcing the model to adjust the weight parameters to increase the distance; otherwise, the loss is 0 and the model does not participate in gradient updates.

[0109] The network traffic anomaly detection model training method provided in this embodiment has the following technical effects:

[0110] First, a mutual information matrix is ​​calculated based on network traffic feature information. Since mutual information can quantify the nonlinear feature correlation between nodes, it can dynamically represent real-time spatial relationships, avoiding noise interference and correlation omissions caused by manually preset static adjacency matrices. Second, a trend significance score is generated by analyzing network traffic time series data through MK trend test. Since this score accurately depicts the temporal rise and fall trend of node traffic (such as a sudden surge in traffic caused by a sudden attack), it can reveal the dynamic evolution law of node behavior from the time dimension. Furthermore, a dynamic adjacency matrix is ​​generated by combining the mutual information matrix and the trend significance score. Since this matrix integrates the spatial correlation between nodes and the temporal trend significance, graph convolution operations can accurately capture spatiotemporal collaborative anomalies across nodes in complex attack scenarios (such as low-frequency APT attacks). Finally, the prediction error of the updated network traffic feature information is calculated through the loss function and the model weight parameters are updated in reverse. Since the model optimizes the weights of mutual information calculation, trend analysis, and dynamic graph convolution end-to-end, it can adaptively learn abnormal patterns in dynamic network environments without manual intervention, significantly improving the model's detection accuracy and scenario generalization ability.

[0111] Figure 2 This is a schematic diagram of the method for generating a dynamic adjacency matrix provided in an embodiment of this application, such as... Figure 2 As shown, optionally, generating a dynamic adjacency matrix based on the trend significance score and the mutual information matrix includes:

[0112] S201. Determine the gating weight of each network node based on the trend significance score between each pair of network nodes.

[0113] The purpose of this step is to dynamically adjust the gating weights based on the significance of traffic trends between network nodes, thereby addressing the problem that static adjacency matrices cannot adapt to spatiotemporal changes. Since the trend significance score (MK) quantifies the temporal correlation strength of traffic changes between nodes, the technical approach of this step is to convert the MK score into gating weights through a nonlinear mapping, thus achieving a dynamic balance between spatial correlation (mutual information) and temporal trend. Specifically, the trend significance score can be input into the Sigmoid function to generate the gating weights, i.e.:

[0114]

[0115] in, , These are scientific parameters that can be optimized using training data. The Sigmoid function compresses the MK score to the (0,1) interval; for example, an initial value can be set. =1, Setting the gating weights to 0 makes them positively correlated with the MK score. This step allows the model to adaptively allocate weights based on the significance of traffic trends; for example, when the MK score is high (e.g., a sudden attack causing a clear time-series trend), the gating weights... By approaching 0, the contribution of spatial correlation (MI) is reduced. The technical effect is to make the adjacency matrix pay more attention to anomalous signals in the time dimension, thereby improving the sensitivity to the detection of sudden attacks.

[0116] S202. Based on the gating weights, the trend significance score and mutual information matrix of each pair of network nodes are weighted and fused to generate a dynamic adjacency matrix. The higher the trend significance score, the lower the proportion of mutual information in the dynamic adjacency matrix, and the higher the proportion of trend significance score in the dynamic adjacency matrix.

[0117] The purpose of this step is to construct a dynamically balanced adjacency matrix by fusing mutual information (MI) and trend significance score (MK) through gating weights. This addresses the problem that traditional static graph structures cannot simultaneously capture spatial correlations and temporal evolution. The technical approach involves designing a linear weighting formula that makes the weights of temporal trend and spatial correlation complementary.

[0118]

[0119] Specifically, for example, gating weights can be used. The result is multiplied element-wise with the mutual information matrix and then added to the complementary weights of the trend significance score matrix. For example, the fusion result can be further normalized (e.g., using Softmax) so that the sum of the edge weights of the adjacency matrix is ​​1. When the trend significance score is high (e.g....), the result is further enhanced. >0.8), When the adjacency matrix weight approaches 1, the MK score dominates the weighting, thus enhancing the detection capability against progressive attacks (such as low-frequency APT penetration). Conversely, when the trend significance score is low, the MI weighting increases, and the model relies on spatial correlations between network nodes to identify anomalies (such as abnormal communication paths in DDoS attacks). The technical effect of this step is to achieve adaptive fusion of spatiotemporal features through a dynamic weighting mechanism, which preserves the stability of spatial correlations while introducing the flexibility of temporal trends, thereby reducing false alarm rates and improving detection coverage in complex network environments.

[0120] Furthermore, the network traffic anomaly detection model may include a variational autoencoder based on a probabilistic graphical model, which extracts network traffic feature information of each network node based on historical network traffic data, including:

[0121] By using a variational autoencoder, historical network traffic data is encoded to generate latent variables for each network node as network traffic feature information.

[0122] Specifically, the historical network traffic data is first preprocessed, and the specific preprocessing steps can be found in the above embodiment. Next, a probabilistic graphical model using a variational autoencoder (VAE) is employed to perform nonlinear dimensionality reduction on the preprocessed historical network traffic data, addressing the problem that traditional linear methods struggle to capture complex traffic patterns. Since the core of VAE is learning latent variables through an encoder... The probability distribution is determined and constrained to approximate a standard Gaussian prior. The technical approach is to map the input data to the mean and variance of latent variables, and then sample the latent variables through reparameterization. A specific implementation could be a three-layer fully connected neural network encoder, with the preprocessed feature vector as input. Output the mean of the latent variable distribution. and variance Then through (in Generate latent variables; for example, the decoder can use a symmetric three-layer fully connected network to generate latent variables. Reconstructed This approach optimizes the weight parameters of the anomaly detection model by minimizing the reconstruction error and KL divergence. The technical effect is to extract low-dimensional dense traffic feature representations through latent variables, filter noise and retain key behavioral patterns, and provide highly discriminative feature inputs for subsequent anomaly detection.

[0123] Optionally, the mutual information of network traffic characteristics of each pair of network nodes is calculated to obtain the mutual information matrix of the network nodes, including:

[0124] Calculate the mutual information of the latent variables for each pair of network nodes to obtain the mutual information matrix.

[0125] The purpose of this step is to quantify node correlations through mutual information between latent variables, thereby addressing the subjectivity and static nature of manually defined adjacency matrices. Since the latent variables (generated by VAEs) have already undergone nonlinear dimensionality reduction and noise reduction of the original traffic data, the technical approach is to leverage their low-dimensional density to efficiently calculate the nonlinear dependencies between nodes. A specific implementation method could be to use KL divergence to approximate the mutual information, i.e., for network nodes... With network nodes Latent variables , The formula for calculating mutual information is:

[0126]

[0127] For example, the joint distribution can be fitted using kernel density estimation (KDE). Marginal distribution , Alternatively, a neural network estimator (such as the MINE algorithm) can be used to directly optimize the lower bound of mutual information, thereby avoiding high-dimensional integral calculations. The technical effect is to extract the essential correlation of latent variables, reduce noise interference in the original data, and improve the detection sensitivity of low-frequency covert attacks (such as APT node collaboration).

[0128] Optionally, based on the variational autoencoder in the anomaly detection model, a loss value can be calculated using a loss function according to the updated network traffic feature information, and the weight parameters of the network traffic anomaly detection model can be updated according to the loss value, including:

[0129] 1. Based on network traffic characteristics, calculate the reconstruction loss and KL divergence loss using the reconstruction loss function and the KL divergence loss function;

[0130] The purpose of this step is to address the difficulty of balancing data reconstruction and distribution matching by using a single loss function, through joint optimization of the reconstruction error and latent variable distribution constraints. Since variational autoencoders (VAEs) need to simultaneously ensure high-fidelity reconstruction of the input data and that the latent variables follow a standard Gaussian prior, the technical approach is to separately calculate the reconstruction loss (measuring the difference between the input and reconstructed data) and the KL divergence loss (measuring the deviation of the latent variable distribution from the prior distribution). A specific implementation could be, for example, defining the reconstruction loss as the mean squared error (MSE). That is, for the ... The raw traffic feature vector of each network node (such as standardized packet size, protocol type encoding, etc.). and the decoder reconstructs the output of the first Feature vectors of network nodes The reconstruction loss is calculated using the following formula:

[0131]

[0132] This refers to the number of training samples per batch (e.g., inputting features from 256 nodes per batch).

[0133] Meanwhile, the KL divergence loss is defined as the distribution of latent variables. Compared with standard Gaussian distribution The difference, the KL divergence loss calculation formula is:

[0134]

[0135] in For the dimension of latent variables, and The mean and variance of the encoder output. For the first The latent variable of the nth network node The mean of the dimension is calculated by the encoder network; For the first The latent variable of the nth network node The variance of the dimension is ensured to be non-negative by exponential operation after the encoder output.

[0136] 2. Calculate the weighted sum of the reconstruction loss and the KL divergence loss, use the weighted sum as the loss value, and update the weight parameters of the network traffic anomaly detection model based on the loss value.

[0137] In this step, the weighted sum of the reconstruction loss and the KL divergence loss is:

[0138]

[0139] in This is the balance coefficient.

[0140] Furthermore, this step optimizes the model parameters using the backpropagation algorithm to address the inefficiency of traditional manual parameter tuning. Since the encoder and decoder parameters of a VAE need to be updated synchronously to minimize the total loss, the approach is to use gradient descent to calculate the partial derivatives of the loss with respect to the parameters and iteratively update them. A specific implementation could be achieved using the Adam optimizer, whose parameter update formula is as follows:

[0141]

[0142] in These are the model weight parameters (including encoder weights and decoder weights). Let the learning rate be (e.g., set) =0.001), for example, a dynamic learning rate decay strategy can be set (such as after every 10 rounds of training). 0.9 This allows for stable convergence in the later stages of training. The technical effect is to achieve efficient training of the model through automated gradient optimization, while avoiding getting trapped in local optima and ensuring a balance between the distribution characteristics of the latent variable space and the reconstruction capability.

[0143] Figure 3 This is a schematic flowchart of a network traffic anomaly detection method provided in an embodiment of this application. The method is implemented based on the network traffic anomaly detection model trained in the aforementioned embodiments. Figure 3 As shown, the method includes:

[0144] S301. Obtain network traffic data for each network node. The network traffic data is generated by communication between each network node.

[0145] The purpose of this step is to collect real-time communication data from distributed network nodes to solve the problems of detection delay and information loss caused by the dispersed data sources and heterogeneous formats in traditional methods. Since network traffic data is dynamically generated by inter-node communication (such as HTTP request and response, TCP connection handshake), the technical approach is to capture the full data packets using network probes or traffic mirroring technology. Specific implementation methods include deploying optical splitters or NetFlow probes on core network switches to collect metadata such as source / destination IP, port number, protocol type, packet size, and timestamps (e.g., extracting the SNI field from the TLS handshake phase for encrypted traffic). Simultaneously, the data format is standardized to time series or key-value pairs (e.g., JSON format: {"src_ip":"192.168.1.1", "dst_ip":"10.0.0.1", "bytes":1500, "timestamp":"2023-10-01T12:34:56Z"}), and transmitted in real-time to the analysis server via a message queue (such as Kafka). The technical effect is to ensure the integrity and real-time nature of the raw traffic data through a unified data collection and transmission mechanism, providing reliable input for subsequent feature extraction and anomaly scoring.

[0146] S302. Input the network traffic data into the network traffic anomaly detection model. The network detection model extracts the network traffic feature information from the network traffic data and calculates the anomaly score of the network traffic data based on the network traffic feature information.

[0147] This step automatically extracts deep features and quantifies the degree of anomaly through a pre-trained network traffic anomaly detection model, addressing the problems of low efficiency and high false positive rate associated with manual feature engineering. For example, the model can be based on a variational autoencoder (VAE) architecture to encode the raw traffic data into low-dimensional latent variables, and calculate the anomaly score through reconstruction error and distribution offset. Specifically, the network traffic data from step S301 can be input into the encoder network (such as a three-layer fully connected layer) to generate latent variables. Then the decoder based on The network traffic data is reconstructed, and the anomaly score is calculated as a weighted sum of the reconstruction error and the latent variable KL divergence. For example, in real-time detection scenarios, a dynamic threshold can be set (such as 3 times the standard deviation of the historical normal score mean). When the weighted sum > the threshold, an alarm is triggered. The technical effect is to significantly improve detection efficiency and accuracy through end-to-end automated feature learning and anomaly quantification, while also supporting generalized detection of unknown attack patterns.

[0148] Furthermore, after an alarm is triggered, the alert information can include the type of abnormal traffic, its source, and its impact, enabling network administrators to handle the abnormal traffic promptly. The alert information can be returned to network administrators for timely intervention. Feedback can be provided via email, SMS, or other methods.

[0149] Figure 4 This is a schematic diagram of the network traffic anomaly detection model training device provided in the embodiments of this application, as shown below. Figure 4 As shown, the device 40 includes:

[0150] The first extraction module 401 is used to extract network traffic characteristic information of each network node based on historical network traffic data, which is generated by communication between each network node.

[0151] The mutual information calculation module 402 is used to calculate the mutual information of network traffic feature information of each pair of network nodes, and obtain the mutual information matrix of network nodes. The elements in the mutual information matrix are mutual information to indicate the degree of correlation of network traffic features between network nodes.

[0152] The second extraction module 403 is used to extract network traffic time series data of each network node based on historical network traffic data. The network traffic time series data is used to indicate the network traffic change characteristics of each network node within a preset time window.

[0153] The trend test module 404 is used to perform Mann-Kendall trend test on network traffic time series data to obtain the network traffic trend significance score between each pair of network nodes. The network traffic trend significance score indicates the significance of the time series rise and fall trend of network traffic between each pair of network nodes.

[0154] The generation module 405 is used to generate a dynamic adjacency matrix based on the trend significance score and the mutual information matrix;

[0155] Convolution module 406 is used to perform dynamic graph convolution operations on network traffic feature information and dynamic adjacency matrix to obtain updated network traffic feature information.

[0156] The weight update module 407 is used to calculate the loss value through the loss function based on the updated network traffic feature information, and update the weight parameters of the network traffic anomaly detection model based on the loss value.

[0157] Figure 5 This is a schematic diagram of the electronic device structure provided in the embodiments of this application, such as... Figure 5As shown, the device 50 includes at least one processor 501 and a memory 502. Optionally, the device 50 also includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0158] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0159] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0160] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0161] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0162] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0163] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0164] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0165] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0166] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0167] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0168] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0169] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0170] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for training a network traffic anomaly detection model, characterized in that, The method includes: Based on historical network traffic data, network traffic characteristic information of each network node is extracted, wherein the historical network traffic data is generated by communication between the network nodes. Calculate the mutual information of the network traffic feature information of each pair of network nodes to obtain the mutual information matrix of the network nodes. The elements in the mutual information matrix are the mutual information to indicate the degree of correlation of the network traffic features between the network nodes. Based on the historical network traffic data, network traffic time series data of each network node is extracted. The network traffic time series data is used to indicate the network traffic change characteristics of each network node within a preset time window. The network traffic time series data is subjected to Mann-Kendall trend test to obtain the network traffic trend significance score between each pair of network nodes. The network traffic trend significance score indicates the significance of the time-series rise and fall trend of network traffic between each pair of network nodes. A dynamic adjacency matrix is ​​generated based on the trend significance score and the mutual information matrix; Perform a dynamic graph convolution operation on the network traffic feature information and the dynamic adjacency matrix to obtain the updated network traffic feature information; Based on the updated network traffic feature information, a loss value is calculated using a loss function, and the weight parameters of the network traffic anomaly detection model are updated based on the loss value.

2. The method according to claim 1, characterized in that, The step of generating a dynamic adjacency matrix based on the trend significance score and the mutual information matrix includes: Based on the trend significance score between each pair of network nodes, the gating weight of each network node is determined; Based on the gating weights, the trend significance score and the mutual information matrix of each pair of network nodes are weighted and fused to generate the dynamic adjacency matrix. The higher the trend significance score, the lower the proportion of mutual information in the dynamic adjacency matrix, and the higher the proportion of trend significance score in the dynamic adjacency matrix.

3. The method according to claim 1, characterized in that, The network traffic anomaly detection model includes a variational autoencoder based on a probabilistic graphical model. The step of extracting network traffic feature information for each network node based on historical network traffic data includes: The variational autoencoder encodes the historical network traffic data to generate latent variables for each network node as network traffic feature information.

4. The method according to claim 3, characterized in that, The step of calculating the mutual information of the network traffic characteristic information of each pair of network nodes to obtain the mutual information matrix of the network nodes includes: The mutual information of the latent variables of each pair of network nodes is calculated to obtain the mutual information matrix.

5. The method according to claim 2, characterized in that, The step of calculating a loss value using a loss function based on the updated network traffic feature information, and updating the weight parameters of the network traffic anomaly detection model based on the loss value, includes: Based on the network traffic characteristic information, the reconstruction loss and KL divergence loss are calculated using the reconstruction loss function and the KL divergence loss function; Calculate the weighted sum of the reconstruction loss and the KL divergence loss, use the weighted sum as the loss value, and update the weight parameters of the network traffic anomaly detection model based on the loss value.

6. A method for detecting network traffic anomalies, characterized in that, The network traffic anomaly detection method is implemented based on a network traffic anomaly detection model, which is trained using the training method described in any one of claims 1-5. The network traffic anomaly detection method includes: Obtain network traffic data for each network node, wherein the network traffic data is generated by communication between the network nodes; The network traffic data is input into the network traffic anomaly detection model, which extracts network traffic feature information from the network traffic data and calculates anomaly scores for the network traffic data based on the network traffic feature information.

7. A network traffic anomaly detection model training device, characterized in that, The device includes: The first extraction module is used to extract network traffic characteristic information of each network node based on historical network traffic data, wherein the historical network traffic data is generated by communication between the network nodes. The mutual information calculation module is used to calculate the mutual information of the network traffic feature information of each pair of network nodes, and obtain the mutual information matrix of the network nodes. The elements in the mutual information matrix are the mutual information to indicate the degree of correlation of the network traffic features between the network nodes. The second extraction module is used to extract network traffic time series data of each network node based on the historical network traffic data. The network traffic time series data is used to indicate the network traffic change characteristics of each network node within a preset time window. The trend testing module is used to perform Mann-Kendall trend testing on the network traffic time series data to obtain the network traffic trend significance score between each pair of network nodes. The network traffic trend significance score indicates the significance of the time-series rise and fall trend of network traffic between each pair of network nodes. A generation module is used to generate a dynamic adjacency matrix based on the trend significance score and the mutual information matrix; The convolution module is used to perform dynamic graph convolution operations on the network traffic feature information and the dynamic adjacency matrix to obtain updated network traffic feature information. The weight update module is used to calculate the loss value through a loss function based on the updated network traffic feature information, and update the weight parameters of the network traffic anomaly detection model based on the loss value.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as claimed in any one of claims 1 to 5, or to implement the method as claimed in claim 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 5, or to implement the method as described in claim 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 5, or implements the method as described in claim 6.

Images