A network abnormal traffic identification model evaluation method, device, equipment and medium

Abstract

The application discloses a network abnormal traffic identification model evaluation method and device, equipment and medium, and relates to the technical field of network flow monitoring. The method comprises the following steps: for a network abnormal traffic identification model trained by a target graph neural network, different reference neural networks are used to distinguish different sources of node representation direction concentration in the target graph neural network, the original input baseline, the architecture baseline, the optimization effect and the graph condition effect are explained and decomposed, the network abnormal traffic identification model optimization adjustment has data basis, and therefore, the debugging cost of the network abnormal traffic identification model is reduced, and the optimization efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of network traffic monitoring technology, and in particular to a method, apparatus, device and medium for evaluating a network abnormal traffic identification model. Background Technology

[0002] With the rapid expansion of digital businesses, network traffic is surging, and the concealment of abnormal traffic behaviors such as distributed attacks, covert malicious access, and lateral movement within internal networks is constantly improving. Traditional detection algorithms based on single-flow characteristics are struggling to capture coordinated attacks between devices. Graph Neural Networks (GNNs) can abstract network terminals, IPs, and service servers as graph nodes, construct adjacency graphs based on communication connections, and mine traffic correlation patterns through neighborhood message passing. This has become the mainstream technical approach for identifying abnormal network traffic.

[0003] In the GNN traffic model evaluation system, the industry uses the Mean Resultant Length (R) of directional statistics to measure the clustering of node embeddings, that is, the concentration of node representation direction; the closer R is to 1, the clearer the distinction between normal and abnormal traffic node representations, and the better the model detection performance.

[0004] In real-world optimization scenarios, if the model's R-value is too low and there are more false alarms or missed detections of abnormal traffic, developers cannot determine whether the defect originates from traffic feature preprocessing, network structure design, training parameters, or graph topology construction. They can only iterate and debug each module in full, resulting in extremely high debugging and trial-and-error costs. In particular, network traffic often consists of heterogeneous graphs (normal devices often communicate with malicious IPs), and existing methods cannot quantify the positive or negative contribution of the graph structure to the model, further exacerbating the difficulty of model optimization and severely restricting the efficiency of GNN deployment in traffic security detection scenarios. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, device, and medium for evaluating network abnormal traffic identification models, which can reduce model adjustment costs and improve model optimization efficiency.

[0006] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for evaluating a network anomaly traffic identification model, including: Based on the input graph data to be processed in the network anomaly traffic identification model, a node feature matrix is ​​determined; each row in the node feature matrix represents the original feature vector of a node; the network anomaly traffic identification model is obtained by training the target graph neural network using a training set; The class-weighted average synthesis length corresponding to the output node features of the network abnormal traffic identification model is calculated based on the node feature matrix and used as the original input baseline component. The architecture baseline increment of the network anomaly traffic identification model is calculated based on the class-weighted average synthesis length corresponding to the output node features of the first reference neural network and the original input baseline components; the first reference neural network is the target graph neural network that has not been trained. The optimization effect increment of the network abnormal traffic identification model is calculated based on the class-weighted average synthesis length corresponding to the output node features of the second reference neural network, the original input baseline components, and the architecture baseline increment; the second reference neural network is obtained by training the target graph neural network with the training set after removing the graph structure and graph aggregation operations. The graph conditional effect increment of the network abnormal traffic identification model is calculated based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network and the second reference neural network; the network of the third reference neural network is obtained by training the graph-aware backbone network using the training set, and the graph-aware backbone network includes graph aggregation operations. The residual term is calculated based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network, the original input baseline component, the architecture baseline increment, the optimization effect increment, and the graph conditional effect increment; the residual term is used to explain whether there are nonlinear conflicts in the network architecture, training process, and input graph structure of the target graph neural network.

[0007] Optionally, the class-weighted average synthesis length corresponding to the output node features of the network anomaly traffic identification model is calculated based on the node feature matrix and used as the original input baseline component, specifically including the following formula: ; ; in, Represents a node The first unit vector, Represents a node The original feature vector, where N is the number of nodes. This represents the original input baseline component.

[0008] Optionally, based on the class-weighted average synthesis length corresponding to the features of the output nodes of the first reference neural network and the original input baseline components, the architectural baseline increment of the network anomaly traffic identification model is calculated, specifically including: The original node features are input into the first reference neural network, and the output features of the penultimate layer are extracted. The baseline increment of the architecture is then calculated according to the following formula: ; ; ; in, For the nodes in the output features of the penultimate layer of the first reference neural network Features For nodes The second unit vector, The class-weighted average synthesis length corresponding to the features of the output nodes of the first reference neural network. Indicates the original input baseline component. This represents the baseline increment of the architecture.

[0009] Optionally, the optimization effect increment of the network anomaly traffic identification model is calculated based on the class-weighted average synthesis length corresponding to the features of the output nodes of the second reference neural network, the original input baseline components, and the architecture baseline increment. Specifically, this includes: The original node features are input into the second reference neural network, and the output features of the penultimate layer are extracted. The optimization effect increment is calculated according to the following formula: ; ; ; ; in, For the nodes in the output features of the penultimate layer of the second reference neural network Features For nodes The third unit vector, This represents the set of nodes of type c, where c is the node type index. This represents the intra-class average vector of the unit eigenvectors of the c-th class nodes. Represents a node Category tags, Let represent the element-wise summation of the third unit vector of all nodes in class c. This represents the class-weighted average synthesis length corresponding to the features of the output nodes of the second reference neural network. Indicates the original input baseline component. This represents the baseline increment of the architecture. This represents the increment of the optimization effect.

[0010] Optionally, the graph conditional effect increment of the network anomaly traffic identification model is calculated based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network and the second reference neural network, specifically including: The original node features are input into the third reference neural network, and the output features of the penultimate layer are extracted. The graph conditional effect increment is then calculated according to the following formula: ; ; ; in, For the nodes in the output features of the penultimate layer of the third reference neural network Features For nodes The fourth unit vector, The class-weighted average synthesis length corresponding to the output node features of the third reference neural network, where K represents the number of node categories. The class-weighted average synthesis length corresponding to the features of the output nodes of the second reference neural network. The increment of the conditional effect in the graph is given.

[0011] Optionally, the residual term is calculated based on the class-weighted average synthesis length corresponding to the features of the output nodes of the third reference neural network, the original input baseline component, the architecture baseline increment, the optimization effect increment, and the graph conditional effect increment, specifically including the following formula: ; in, Represents the residual term. This represents the class-weighted average synthesis length corresponding to the features of the output node of the third reference neural network. This represents the original input baseline component. This represents the baseline increment of the architecture. This represents the increment of the optimization effect. This represents the increment of the conditional effect in the graph.

[0012] Optionally, the second reference neural network is a multilayer perceptron.

[0013] Secondly, this application provides a network abnormal traffic identification model evaluation device, which applies the aforementioned network abnormal traffic identification model evaluation method, and the network abnormal traffic identification model evaluation device includes: The graph data preprocessing module is used to determine the node feature matrix based on the input graph data to be processed by the network anomaly traffic identification model; each row in the node feature matrix represents the original feature vector of a node; the network anomaly traffic identification model is obtained by training the target graph neural network using a training set; The original input baseline component determination module is used to calculate the class-weighted average synthesis length corresponding to the output node features of the network abnormal traffic identification model based on the node feature matrix, and use it as the original input baseline component. The architecture baseline increment determination module is used to calculate the architecture baseline increment of the network abnormal traffic identification model based on the class-weighted average synthesis length corresponding to the output node features of the first reference neural network and the original input baseline components; the first reference neural network is the target graph neural network that has not been trained; The optimization effect increment determination module is used to calculate the optimization effect increment of the network abnormal traffic identification model based on the class-weighted average synthesis length corresponding to the output node features of the second reference neural network, the original input baseline components, and the architecture baseline increment; the second reference neural network is obtained by training the target graph neural network with the training set after removing the graph structure and graph aggregation operations. The graph conditional effect increment determination module is used to calculate the graph conditional effect increment of the network abnormal traffic identification model based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network and the second reference neural network; the network of the third reference neural network is obtained by training the graph-aware backbone network using the training set, and the graph-aware backbone network includes graph aggregation operations. The residual term determination module is used to calculate residual terms based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network, the original input baseline component, the architecture baseline increment, the optimization effect increment, and the graph conditional effect increment; the residual terms are used to indicate whether there are nonlinear conflicts in the network architecture, training process, and input graph structure of the target graph neural network.

[0014] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the network abnormal traffic identification model evaluation method described in any one of the above.

[0015] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the network anomaly traffic identification model evaluation method described in any one of the above.

[0016] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a method, apparatus, device, and medium for evaluating a network anomaly traffic identification model. For a network anomaly traffic identification model trained by a target graph neural network, different reference neural networks are used to distinguish the different sources of the concentration of node representation directions in the target graph neural network. This enables interpretable decomposition of the original input baseline, architecture baseline, optimization effect, and graph conditional effect, avoiding the inability to distinguish whether the concentration comes from the network architecture itself, the optimization effect of the training process, or the conditional effect brought about by the graph structure, based on a single average synthesis length. This provides a data basis for optimizing and adjusting the network anomaly traffic identification model, thereby reducing the debugging cost of the network anomaly traffic identification model and improving the tuning efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a network abnormal traffic identification model evaluation method provided in one embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the functional modules of a network abnormal traffic identification model evaluation device provided in an embodiment of this application.

[0020] Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] Graph neural networks are applied to the field of network traffic anomaly detection. Graph recognition systems assume that node features have directional concentration on a unit hypersphere, meaning that nodes of the same category are clustered around several category centers. This concentration is typically measured using the mean resultant length (R) in directional statistics, where R=1 indicates that all unit vectors are identical, and R=0 indicates a uniform distribution on the sphere.

[0024] Existing GNN evaluation methods can only obtain the overall R index of the model, and do not have the ability to decompose components. They cannot distinguish that the directional concentration comes from four independent components: ① the discriminative power of the original traffic features themselves; ② the inherent nonlinear mapping effect of the GNN network architecture; ③ the feature optimization gain brought by supervised training; ④ the additional aggregation gain brought by the message passing of the graph structure.

[0025] Related technologies employ message-passing mechanisms such as Graph Convolutional Networks (GCNs), which increase the R-value with increasing layer count on homogeneous graphs (where similar nodes tend to connect). However, existing methods only report a single R-value, failing to distinguish whether this concentration originates from the network architecture itself, optimization effects during training, or conditional effects imposed by the graph structure. This makes it difficult for researchers to accurately determine the true contribution of message passing to the representation geometry, and also fails to explain why GNN performance degrades on heterogeneous graphs (where dissimilar nodes tend to connect), or why directional concentration is even lower than that of ordinary multilayer perceptrons (MLPs) without graph structures. Consequently, there is a degree of blindness in the optimization process of network traffic anomaly identification models.

[0026] In one exemplary embodiment, this application provides a method for evaluating a network anomaly traffic identification model, such as... Figure 1 As shown, the method includes steps 101-106.

[0027] Step 101: Determine the node feature matrix based on the input graph data to be processed for the network anomaly traffic identification model; each row in the node feature matrix represents the original feature vector of a node; the network anomaly traffic identification model is obtained by training the target graph neural network using a training set.

[0028] Step 102: Calculate the class-weighted average synthesis length corresponding to the output node features of the network abnormal traffic identification model based on the node feature matrix, and use it as the original input baseline component.

[0029] The smaller the original input baseline component, the worse the network anomaly traffic identification model's ability to distinguish input network traffic features.

[0030] If the original input baseline component value is too low, specifically less than the first set value, it indicates that the original network traffic feature discrimination is insufficient. Therefore, the original traffic features of the input network abnormal traffic identification model are denoised, normalized, and redundant features are removed to expand the effective traffic feature dimension and improve the category clustering of the original traffic feature data.

[0031] The range of the original input baseline component values ​​is: Exceeding The range is meaningless. The range of the first set value is 0.3 to 0.6. Preferably, the value of the first set value is 0.4, 0.5 or 0.6.

[0032] The first set value is determined as follows: 1. Dataset quantile method: Statistically analyze all graph samples C0, and take the 20%-40% quantile as the threshold; 2. Model performance mapping method: Find the C0 corresponding to a significant decrease in detection AUC as the threshold; 3. Directly select a fixed threshold of 0.4 / 0.5 / 0.6 for rapid deployment.

[0033] Step 103: Calculate the architecture baseline increment of the network abnormal traffic identification model based on the class-weighted average synthesis length corresponding to the output node features of the first reference neural network and the original input baseline components; the first reference neural network is the target graph neural network that has not been trained.

[0034] If the baseline increment of the architecture is abnormal, specifically if it exceeds the set range, it indicates that the graph neural network architecture design is unreasonable. This means that the number of network layers, the dimension of hidden layers, or the activation function of the target graph neural network need to be adjusted to eliminate the interference of invalid directional concentration caused by the network architecture itself.

[0035] If the baseline increment of the architecture exceeds the upper limit of the set range, then reduce the number of network layers, lower the hidden layer dimension, or replace the activation function. For example, reduce the number of network layers from more than 4 layers to 2-3 layers, reduce the hidden layer dimension from 256 to 128 or 64, and replace the ReLU activation function with the LeakyReLU activation function or the GELU activation function, instead of using the Tanh activation function.

[0036] If the baseline increment of the architecture is lower than the lower limit of the set range, then the number of network layers is increased, the hidden layer dimension is increased, or the activation function is replaced. For example, the number of network layers is increased from 1 to 2 or 3, the hidden layer dimension is increased from 64 to 128, and the LeakyReLU activation function or GELU activation function is replaced with the ReLU activation function or ELU activation function.

[0037] The priority for adjusting the number of network layers, hidden layer dimensions and activation functions from high to low is: activation function, hidden layer dimension, number of network layers.

[0038] Step 104: Synthesize the length based on the class-weighted average corresponding to the node features output by the second reference neural network, the original input baseline component and the architecture baseline increment, and calculate the optimization effect increment of the network abnormal traffic identification model; the second reference neural network is obtained by removing the graph structure and graph aggregation operations from the target graph neural network and then training with the training set.

[0039] Specifically, the second reference neural network is a multi-layer perceptron with all graph structures and graph aggregation operations removed from the target graph neural network. The network depth, hidden layer dimensions and activation function of the perceptron are consistent with those of the target graph neural network, and supervised training is completed using exactly the same training set and training parameters.

[0040] If the value of the optimization effect increment is relatively low, specifically it is less than the second set value, it indicates that the training parameters of the target graph neural network need to be adjusted.

[0041] The theoretical value range of the optimization effect increment is [-1,1], and the second set value T2 is a commonly used reasonable threshold range in engineering: 0.2~0.4. Preferably, the value of the second set value is 0.2, 0.3 or 0.4.

[0042] Judgment rule: when C2<T2, the category aggregation gain brought by training supervision optimization is insufficient, and training parameters need to be adjusted.

[0043] The three methods for determining the second set value are as follows: 1. Dataset quantile method: Calculate C2 for all test graph samples in batches, and take the 20%~35% quantile as T2; the smaller the quantile, the looser the threshold; 2. Model performance mapping method: Fix features, graph structure and network architecture, only change training hyperparameters, and find the C2 value corresponding to the obvious decline of anomaly detection AUC and recall rate as T2; 3. Directly use the fixed threshold of 0.2 / 0.3 / 0.4 for rapid implementation without a large number of statistical experiments.

[0044] Step 105: Calculate the graph conditional effect increment of the network abnormal traffic identification model based on the class-weighted average synthesized length corresponding to the node features output by the third reference neural network and the second reference neural network; the third reference neural network is obtained by training the graph-aware backbone network using the training set, and the graph-aware backbone network includes graph aggregation operations.

[0045] If the value of the graph conditional effect increment is low, specifically when it is less than the third set value, it indicates that the graph structure and the message passing module fail to function. Therefore, the construction rule of the network communication topology graph is optimized, the normalization mode of the adjacency matrix is adjusted, the graph aggregation strategy is improved, and the characterization ability of the graph structure for traffic features is enhanced.

[0046] The graph conditional effect increment C3 represents the additional aggregation gain brought by graph structure message passing, with a value range of [-1,1]. The standard range for engineering implementation is 0.15~0.35, that is, the value range of the third set value T3 is 0.15~0.35. Preferably, the values of the third set value T3 are 0.15, 0.25 and 0.35.

[0047] 1. Recommended threshold for general conventional traffic scenarios: T3=0.25.

[0048] 2. Loose threshold for homomorphic graphs (a large number of interconnections between homogeneous IPs, such as the same service cluster in the intranet): T3=0.15.

[0049] 3. Strict threshold for heterogeneous graphs (normal hosts and malicious IPs are mixed and connected, APT lateral penetration): T3=0.35 Judgment rule: If C3<T3, the graph topology and graph aggregation have no positive gain, and the graph construction, adjacency matrix and aggregation function need to be optimized.

[0050] Step 106: Calculate a residual term according to the class-weighted average combined length corresponding to the node features output by the third reference neural network, the original input baseline component, the architecture baseline increment, the optimization effect increment and the graph conditional effect increment; the residual term is used to indicate whether there is a non-linear conflict among the network architecture, training process and input graph structure of the target graph neural network.

[0051] If the value of the residual term is too large, specifically when it is greater than the fourth set value, it indicates that there is a strong non-linear conflict among the network architecture, the training process and the graph structure. The network architecture, training configuration and graph topology structure are adjusted in a coordinated manner to reduce coupling interference between multiple modules and improve the overall stability of the model.

[0052] The residual term Δ represents the non-linear interactive conflict among the architecture, training and graph structure. The value range of the residual term is approximately [-0.5, 0.5], and the applicable range for engineering is 0.1~0.25, that is, the value range of the fourth set value T4 is 0.1~0.25. Preferably, the value of the fourth set value T4 is 0.1, 0.15 or 0.25.

[0053] 1. Preferred threshold for general conventional traffic scenarios: T4=0.15.

[0054] 2. Simple intranet homomorphic traffic (few conflicts, loose judgment): T4=0.1.

[0055] 3. Encrypted traffic, APT heterogeneous graphs (modules are prone to coupling and conflict, strict judgment): T4=0.25.

[0056] Judgment rule: △>T4 indicates severe nonlinear conflicts among multiple modules, requiring simultaneous adjustments to the architecture, training, and graph topology.

[0057] The three ways to determine the value of the fourth setpoint T4 are as follows: 1. Dataset quantile method: Calculate the residuals Δ of all flow map samples in batches, sort them in ascending order, and take the 70%~80% quantile as T4; the higher the quantile, the stricter the judgment. With fixed features, we traverse the network layers, training hyperparameters, adjacency normalization, and aggregation function combinations, observe the AUC fluctuations of the model and the oscillation inflection points of the training loss, and take the residual term △ corresponding to the significant increase in oscillation as T4.

[0058] 2. Model stability calibration method (engineering implementation); 3. Directly select fixed thresholds of 0.1 / 0.15 / 0.25 to adapt to most network abnormal traffic GNN evaluation scenarios.

[0059] This application outputs the original input baseline component, architecture baseline increment, optimization effect increment, graph condition effect increment, and residual term, and uses the above five items as the optimization basis for the network abnormal traffic identification model.

[0060] This application enables the decomposition of directional concentration into multiple interpretable components and the quantitative evaluation of the contribution of each component. Specifically, it distinguishes the different sources of directional concentration represented by nodes in a graph neural network by using different reference neural networks. This allows for the interpretable decomposition of the original input baseline, architecture baseline, optimization effects, and graph conditional effects. It quantitatively distinguishes the contribution of different factors to the directional concentration represented by nodes. Based on the decomposition results, it diagnoses and optimizes the graph neural network model used for traffic identification, providing data support for the optimization and adjustment of the network anomaly traffic identification model. This improves the diagnostic capability of the network anomaly traffic identification model, thereby reducing the debugging cost and improving the optimization efficiency. Ultimately, it enhances the accuracy and robustness of identifying network anomaly traffic and network attack behaviors.

[0061] This application decomposes the overall average synthesis length R into the sum of four core components and a residual term. Specifically, it defines: .

[0062] The meanings and measurement methods of each component and residual are as follows: C0: The original input baseline component, obtained by measuring the original input features.

[0063] C1: Architecture baseline increment, representing the contribution of a non-linear deep network (even if its weights are randomly initialized and have not been trained) itself to the directional concentration. This component depends only on the network architecture and depth.

[0064] C2: Optimization effect increment, representing the impact of cross-entropy training on directional concentration. This component depends on the supervision signal but not on the graph structure, and can be observed by training a regular multilayer perceptron on the same training set as the node features.

[0065] C3: Graph conditional effect increment, representing the additional concentration contribution from message passing on top of C1 and C2. This component is strongly correlated with graph structural characteristics (especially iso-matching / dissimilar-matching).

[0066] Δ: Residual term, used to absorb the nonlinear interactions between the three intervention axes (architecture, optimization, graph).

[0067] In an exemplary embodiment, the target graph neural network is specifically a graph convolutional network, a graph attention network (GAT), or a graph sampling and aggregation network (GraphSAGE).

[0068] In an exemplary embodiment, the input graph data to be processed by the network anomaly traffic identification model is G=(V,E), where V is the set of nodes ( (where N is the total number of nodes in the graph) and E is the set of edges.

[0069] The adjacency matrix A is obtained based on the edge set E, i.e., the adjacency matrix. Adjacency matrix Used for graph aggregation operations in subsequent steps (and for GCN message passing in steps 103 and 105).

[0070] Based on the node set V, obtain the node feature matrix X. Each row of this matrix corresponds to the original feature vector of a node, and the feature dimension is d (i.e., each node...). Corresponding to a d-dimensional feature vector The node feature matrix is ​​the input data for all neural network models, including ordinary multilayer perceptrons and graph-aware graph convolutional networks.

[0071] A node label vector Y is obtained based on the node set, where each element represents the category to which the corresponding node belongs (a total of K categories). This label vector is used to calculate the cross-entropy loss in supervised learning (i.e., for the calculation of cross-entropy loss in steps 104 and 105), driving the model training process.

[0072] The node set includes communication devices such as network hosts, IP terminals, and servers. The edge set represents the connection relationship between the communication devices. The adjacency matrix generated from the edge set is used for graph aggregation and message passing.

[0073] Network traffic data is used as the original feature of nodes, including multi-dimensional traffic features such as packet size, communication duration, packet frequency, protocol type, number of bytes, and number of connections. Each element in the node feature matrix X represents the original feature of the node. The node label vector Y includes node classification labels, which are used to distinguish normal traffic nodes from abnormal attack traffic nodes, and are used for supervised training of the model.

[0074] The network abnormal traffic identification model is used to determine whether a network device or IP is a normal traffic node or an abnormal traffic node, thus completing the security identification of abnormal network traffic.

[0075] In an exemplary embodiment, the class-weighted average synthesis length corresponding to the output node features of the network anomaly traffic identification model is calculated based on the node feature matrix and used as the original input baseline component, specifically including the following formula: ; ; Let C0 = R0.

[0076] in, Represents a node The first unit vector, Represents a node The original feature vector, where N is the number of nodes. This represents the class-weighted average synthesis length corresponding to the node features output by the network anomaly traffic identification model. This represents the original input baseline component.

[0077] In an exemplary embodiment, the first reference neural network specifically has the same depth L, the same hidden layer dimension, and the same activation function as the target graph neural network, but its weights are randomly initialized.

[0078] Step 103 specifically includes: inputting the original node features (a node feature matrix formed by concatenating all original node features) into the first reference neural network, extracting the output features of the penultimate layer (node ​​hidden layer feature vectors), and calculating the architecture baseline increment according to the following formula: ; ; ; in, For the nodes in the output features of the penultimate layer of the first reference neural network Features For nodes The second unit vector, The class-weighted average synthesis length corresponding to the features of the output nodes of the first reference neural network. Indicates the original input baseline component. This represents the baseline increment of the architecture.

[0079] In one exemplary embodiment, a network with the same architecture that ignores the graph structure is constructed, namely a multilayer perceptron, which is also a second reference neural network, with the same number of layers as the target graph neural network, without using adjacency matrix aggregation.

[0080] The multilayer perceptron is trained using the same node feature matrix and node label vector Y (node ​​feature matrix X as input and node label vector Y as supervision signal) with cross-entropy loss (training configuration consistent with the target graph neural network).

[0081] Training configuration includes optimizer, learning rate, loss function, number of training epochs, and batch size. In one example, the training configuration is as follows: Optimizer: Adam; Learning rate: 0.001; Loss function: Multi-class cross-entropy loss; Number of training epochs: 100 epochs; Batch: Full graph training (the entire traffic graph is input, without batch sampling); Regular expression: weight decay = 1e-4, dropout rate = 0.2; Early stopping strategy: Stop training if the validation set AUC does not improve for 10 consecutive rounds; Hardware / Precision: Single GPU, float32 training.

[0082] Step 104 specifically includes: inputting the original node features into the second reference neural network, extracting the output features of the penultimate layer, and calculating the optimization effect increment according to the following formula: ; ; ; ; in, For the nodes in the output features of the penultimate layer of the second reference neural network Features (nodes) (the original latent feature vector). For nodes The third unit vector, Specifically, nodes The unit vector obtained by L2 normalization of latent features, where c is the node class index. K represents the total number of categories in the traffic samples. For example, c=1 represents normal traffic, and c=2 represents abnormal traffic. This represents the set of nodes of type c. This represents the total number of nodes contained in this category (the cardinality of the set). This represents the intra-class average vector of the unit eigenvectors of the c-th class nodes. Represents a node The traffic graph uses category labels, where nodes represent IP addresses, hosts, and servers. Labels distinguish between normal traffic and abnormal attack traffic. The total number of categories is K. Let represent the element-wise summation of the third unit vector of all nodes in class c. This represents the class-weighted average synthesis length (mean of class center 2-norm) corresponding to the features of the output node of the second reference neural network. Indicates the original input baseline component. This represents the baseline increment of the architecture. This represents the increment of the optimization effect.

[0083] In an exemplary embodiment, a graph-aware backbone network (such as a GCN) is constructed. The architecture of the graph-aware backbone network is exactly the same as that of the second reference neural network (using the same graph, node feature matrix X, and node label vector Y, and training the graph-aware backbone network with the same training configuration, but the graph-aware backbone network includes graph aggregation operations). Its aggregation operation uses a symmetric normalized adjacency matrix. Each layer is updated to ).

[0084] in, Depth matrix (diagonal matrix) Indicates the network number The node feature matrix output by the layer (the first layer) (Latent feature matrix of nodes after layer aggregation) Dimensions , Indicates the first The node latent feature matrix after layer aggregation, where N is the total number of nodes in the flow graph. For the first Hidden layer feature dimensions Indicates the first The layer can learn the weight parameter matrix. These represent non-linear activation functions, such as ReLU, ELU, and Sigmoid. The same activation function (controlling a single variable) is used for both the first and second reference MLP networks.

[0085] Step 105 specifically includes: inputting the original node features into the third reference neural network, extracting the output features of the penultimate layer, and calculating the graph conditional effect increment according to the following formula; ; ; ; in, For the nodes in the output features of the penultimate layer of the third reference neural network Features For nodes The fourth unit vector, The class-weighted average synthesis length corresponding to the output node features of the third reference neural network, where K represents the number of node categories. The class-weighted average synthesis length corresponding to the features of the output nodes of the second reference neural network. The increment of the conditional effect in the graph is given.

[0086] In an exemplary embodiment, since R is a nonlinear functional, the four components are not strictly orthogonal and have nonlinear interaction terms. Therefore, the difference between the global R (or intra-class Rw) of the target graph neural network and the sum of its components is defined as the residual. Rw is the class-weighted average synthesis length, which is calculated as follows: first, the nodes are grouped by traffic labels, and the class center of the normalized vector of each class is calculated. Then, the 2-norm of all class center vectors is averaged to measure the model's ability to distinguish between normal and abnormal traffic nodes.

[0087] Step 106 specifically includes the following formula: ; in, Represents the residual term. This represents the class-weighted average synthesis length corresponding to the features of the output node of the third reference neural network. This represents the original input baseline component. This represents the baseline increment of the architecture. This represents the increment of the optimization effect. This represents the increment of the conditional effect in the graph.

[0088] The final decomposition result output by this application is represented as: {C0, C1, C2, C3, Δ}.

[0089] This application has the following technical advantages: 1. Traceable Diagnostic Capability: By decomposing the directional concentration into multiple independent components—the original input baseline, the architecture baseline increment, the optimization effect increment, the graph conditional effect increment, and the residual term—the root cause of poor clustering in graph neural network representations can be accurately located. This distinguishes whether the problem originates from the original traffic characteristics, network architecture, training process, or graph topology, overcoming the limitation of related technologies in being unable to trace the source. This effect stems from the technical steps of this application's method, which involves multi-component decomposition of the average synthesis length and quantitative calculation of each module.

[0090] 2. Highly targeted model tuning: Based on the quantization results of each component, targeted tuning can be achieved, avoiding the problems of blindly adjusting network parameters and structure in traditional methods. This significantly reduces the debugging cost of network anomaly traffic identification models and improves tuning efficiency. This effect stems from the one-to-one correspondence between the decomposition results and each module of the model, as well as the accompanying scenario-specific adjustment strategies.

[0091] 3. Improved accuracy in identifying abnormal traffic: The graph neural network diagnosed and optimized by the method described in this application exhibits stronger clustering of node feature categories, significantly improving the distinction between normal and abnormal traffic. Ultimately, this enhances the accuracy of identifying abnormal network traffic and network attacks, as well as the system's robustness. This improvement stems from targeted optimizations to the original features, network architecture, training strategy, and graph aggregation module.

[0092] Based on the same inventive concept, this application also provides a method for evaluating a network abnormal traffic identification model, used to implement the aforementioned network abnormal traffic identification model evaluation device. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more embodiments of the network abnormal traffic identification model evaluation device provided below can be found in the limitations of the network abnormal traffic identification model evaluation method described above, and will not be repeated here.

[0093] In one exemplary embodiment, such as Figure 2 As shown, a network anomaly traffic identification model evaluation device is provided. The network anomaly traffic identification model evaluation device applies the aforementioned network anomaly traffic identification model evaluation method. The network anomaly traffic identification model evaluation device includes: The graph data preprocessing module is used to determine the node feature matrix based on the input graph data to be processed by the network anomaly traffic identification model; each row in the node feature matrix represents the original feature vector of a node; the network anomaly traffic identification model is obtained by training the target graph neural network using a training set; The original input baseline component determination module is used to calculate the class-weighted average synthesis length corresponding to the output node features of the network abnormal traffic identification model based on the node feature matrix, and use it as the original input baseline component. The architecture baseline increment determination module is used to calculate the architecture baseline increment of the network abnormal traffic identification model based on the class-weighted average synthesis length corresponding to the output node features of the first reference neural network and the original input baseline components; the first reference neural network is the target graph neural network that has not been trained; The optimization effect increment determination module is used to calculate the optimization effect increment of the network abnormal traffic identification model based on the class-weighted average synthesis length corresponding to the output node features of the second reference neural network, the original input baseline components, and the architecture baseline increment; the second reference neural network is obtained by training the target graph neural network with the training set after removing the graph structure and graph aggregation operations. The graph conditional effect increment determination module is used to calculate the graph conditional effect increment of the network abnormal traffic identification model based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network and the second reference neural network; the network of the third reference neural network is obtained by training the graph-aware backbone network using the training set, and the graph-aware backbone network includes graph aggregation operations. The residual term determination module is used to calculate residual terms based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network, the original input baseline component, the architecture baseline increment, the optimization effect increment, and the graph conditional effect increment; the residual terms are used to indicate whether there are nonlinear conflicts in the network architecture, training process, and input graph structure of the target graph neural network.

[0094] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 3 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a network anomaly traffic identification model evaluation method.

[0095] The computer device specifically refers to a computer, network security detection equipment, or traffic analysis server.

[0096] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0097] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0098] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0099] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0100] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0101] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0102] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0103] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for evaluating a network anomaly traffic identification model, characterized in that, include: Based on the input graph data to be processed in the network anomaly traffic identification model, determine the node feature matrix; Each row in the node feature matrix represents the original feature vector of a node; The network abnormal traffic identification model is obtained by training the target graph neural network using a training set; The class-weighted average synthesis length corresponding to the output node features of the network abnormal traffic identification model is calculated based on the node feature matrix and used as the original input baseline component. The architecture baseline increment of the network abnormal traffic identification model is calculated based on the class-weighted average synthesis length corresponding to the output node features of the first reference neural network and the original input baseline components. The first reference neural network is the target graph neural network that has not been trained; The optimization effect increment of the network anomaly traffic identification model is calculated based on the class-weighted average synthesis length corresponding to the output node features of the second reference neural network, the original input baseline components, and the architecture baseline increment. The second reference neural network is obtained by training the target graph neural network with the training set after removing the graph structure and graph aggregation operations. The graph conditional effect increment of the network anomaly traffic identification model is calculated based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network and the second reference neural network. The third reference neural network is obtained by training the graph-aware backbone network using the training set, and the graph-aware backbone network includes graph aggregation operations. The residual term is calculated based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network, the original input baseline component, the architecture baseline increment, the optimization effect increment, and the graph conditional effect increment; the residual term is used to explain whether there are nonlinear conflicts in the network architecture, training process, and input graph structure of the target graph neural network.

2. The network abnormal traffic identification model evaluation method according to claim 1, characterized in that, The class-weighted average synthesis length corresponding to the output node features of the network anomaly traffic identification model is calculated based on the node feature matrix and used as the original input baseline component. Specifically, the following formula is included: ； ； in, Represents a node The first unit vector, Represents a node The original feature vector, where N is the number of nodes. This represents the original input baseline component.

3. The network abnormal traffic identification model evaluation method according to claim 1, characterized in that, Based on the class-weighted average synthesis length corresponding to the output node features of the first reference neural network and the original input baseline components, the architecture baseline increment of the network abnormal traffic identification model is calculated, specifically including: The original node features are input into the first reference neural network, and the output features of the penultimate layer are extracted. The baseline increment of the architecture is then calculated according to the following formula: ； ； ； in, For the nodes in the output features of the penultimate layer of the first reference neural network Features For nodes The second unit vector, The class-weighted average synthesis length corresponding to the features of the output nodes of the first reference neural network. Represents the original input baseline component. This represents the baseline increment of the architecture.

4. The network abnormal traffic identification model evaluation method according to claim 1, characterized in that, Based on the class-weighted average synthesis length corresponding to the output node features of the second reference neural network, the original input baseline components, and the architecture baseline increment, the optimization effect increment of the network anomaly traffic identification model is calculated, specifically including: The original node features are input into the second reference neural network, and the output features of the penultimate layer are extracted. The optimization effect increment is calculated according to the following formula: ； ； ； ； in, For the nodes in the output features of the penultimate layer of the second reference neural network Features For nodes The third unit vector, This represents the set of nodes of type c, where c is the node type index. This represents the intra-class average vector of the unit eigenvectors of the c-th class nodes. Represents a node Category tags, Let represent the element-wise summation of the third unit vector of all nodes in class c. This represents the class-weighted average synthesis length corresponding to the features of the output nodes of the second reference neural network. Indicates the original input baseline component. This represents the baseline increment of the architecture. This represents the increment of the optimization effect.

5. The network abnormal traffic identification model evaluation method according to claim 1, characterized in that, Based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network and the second reference neural network, the graph conditional effect increment of the network anomaly traffic identification model is calculated, specifically including: The original node features are input into the third reference neural network, and the output features of the penultimate layer are extracted. The graph conditional effect increment is then calculated according to the following formula: ； ； ； in, For the nodes in the output features of the penultimate layer of the third reference neural network Features For nodes The fourth unit vector, The class-weighted average synthesis length corresponding to the output node features of the third reference neural network, where K represents the number of node categories. The class-weighted average synthesis length corresponding to the features of the output nodes of the second reference neural network. The increment of the conditional effect in the graph is given.

6. The network abnormal traffic identification model evaluation method according to claim 1, characterized in that, The residual term is calculated based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network, the original input baseline component, the architecture baseline increment, the optimization effect increment, and the graph conditional effect increment, specifically including the following formula: ； in, Represents the residual term. This represents the class-weighted average synthesis length corresponding to the features of the output node of the third reference neural network. This represents the original input baseline component. This represents the baseline increment of the architecture. This represents the increment of the optimization effect. This represents the increment of the conditional effect in the graph.

7. The network abnormal traffic identification model evaluation method according to claim 1, characterized in that, The second reference neural network is a multilayer perceptron.

8. A network abnormal traffic identification model evaluation device, characterized in that, The network abnormal traffic identification model evaluation device applies the network abnormal traffic identification model evaluation method according to any one of claims 1-7, and the network abnormal traffic identification model evaluation device comprises: The graph data preprocessing module is used to determine the node feature matrix based on the input graph data to be processed by the network anomaly traffic identification model; each row in the node feature matrix represents the original feature vector of a node; the network anomaly traffic identification model is obtained by training the target graph neural network using a training set; The original input baseline component determination module is used to calculate the class-weighted average synthesis length corresponding to the output node features of the network abnormal traffic identification model based on the node feature matrix, and use it as the original input baseline component. The architecture baseline increment determination module is used to calculate the architecture baseline increment of the network abnormal traffic identification model based on the class-weighted average synthesis length corresponding to the output node features of the first reference neural network and the original input baseline components; the first reference neural network is the target graph neural network that has not been trained; The optimization effect increment determination module is used to calculate the optimization effect increment of the network abnormal traffic identification model based on the class-weighted average synthesis length corresponding to the output node features of the second reference neural network, the original input baseline components, and the architecture baseline increment; the second reference neural network is obtained by training the target graph neural network with the training set after removing the graph structure and graph aggregation operations. The graph conditional effect increment determination module is used to calculate the graph conditional effect increment of the network abnormal traffic identification model based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network and the second reference neural network; the network of the third reference neural network is obtained by training the graph-aware backbone network using the training set, and the graph-aware backbone network includes graph aggregation operations. The residual term determination module is used to calculate residual terms based on the class-weighted average synthesis length corresponding to the output node features of the third reference neural network, the original input baseline component, the architecture baseline increment, the optimization effect increment, and the graph conditional effect increment; the residual terms are used to indicate whether there are nonlinear conflicts in the network architecture, training process, and input graph structure of the target graph neural network.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the network anomaly traffic identification model evaluation method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the network abnormal traffic identification model evaluation method as described in any one of claims 1-7.

Images