A network traffic data identification method, device, medium and product for generalized graph distribution outlier detection

Abstract

The application discloses a network traffic data identification method and device for out-of-distribution detection of a general graph, a medium and a product, and relates to the technical field of network data identification. The method comprises the following steps: converting network traffic data into graph structure data and inputting the graph structure data into a pre-trained graph neural network; the graph neural network aggregates node neighborhood information through a graph convolution layer contained in the graph neural network, and outputs feature representation of a node or a graph and a classification logic value corresponding to a preset category; based on the classification logic value, an out-of-distribution score is calculated for each graph structure data through a preset energy function; the out-of-distribution score is compared with a preset determination threshold; if the out-of-distribution score is higher than the preset determination threshold, it is determined that the corresponding network traffic data is out-of-distribution traffic; wherein the preset determination threshold is obtained according to an out-of-distribution score distribution of known in-distribution traffic samples. The application can realize reliable and automatic identification of unknown threat traffic.

Description

Technical Field

[0001] This application relates to the field of network data identification technology, and in particular to a method, device, medium and product for identifying network traffic data using generalized graph distribution detection. Background Technology

[0002] In real-world network environments, traffic data is dynamic, complex, and highly correlated. Traditional methods for detecting network traffic anomalies or performing security analysis are typically based on predefined rules, feature libraries of known attack patterns, or the assumption that traffic data follows an independent and identically distributed statistical learning model. However, as network attack methods evolve, the traffic patterns generated by new attacks, unknown malware, or advanced persistent threats (APTs) often differ significantly from the "normal" or "known anomalous" traffic distributions observed during model training; these are known as out-of-distribution (OOD) data.

[0003] When deploying machine learning-based detection models, encountering such out-of-distribution traffic can lead to overconfident misclassifications due to the lack of prior training with the corresponding patterns. This can result in misclassifying unknown attacks as normal traffic, leading to serious security misses. While existing technologies include graph-based anomaly detection (GAD), novelty detection (GND), and open set recognition (OSR) to identify unknown or anomalous patterns, these methods are often tailored to specific problems and suffer from gaps in their definition, evaluation, and implementation frameworks. They lack a unified paradigm for handling generalized graph out-of-distribution detection, making it difficult to reliably and automatically identify unknown threat traffic without requiring a large number of known anomalous samples. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, medium, and product for identifying network traffic data outside of generalized graph distribution detection, which can achieve reliable and automated identification of unknown threat traffic.

[0005] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for identifying network traffic data outside of generalized graph distribution detection, including: Acquire network traffic data containing communication entities and interaction relationships; The network traffic data is converted into a graph structure representation to obtain graph structure data; wherein communication entities are mapped as nodes, and the communication behavior between communication entities is mapped as edges between nodes. The graph structure data is input into a pre-trained graph neural network; the graph neural network aggregates node neighborhood information through its graph convolutional layers and outputs feature representations of nodes or graphs and classification logic values ​​corresponding to preset categories. Based on the classification logic value, a scalar out-of-distribution score is calculated for each graph structure data using a preset energy function; the out-of-distribution score is used to quantify the degree to which the graph structure data deviates from the data distribution within the training distribution; The out-of-distribution score is compared with a preset judgment threshold; if the out-of-distribution score is higher than the preset judgment threshold, the corresponding network traffic data is determined to be out-of-distribution traffic; wherein, the preset judgment threshold is preset based on the out-of-distribution score distribution of known in-distribution traffic samples.

[0006] Optionally, after comparing the out-of-distribution score with a preset determination threshold; and determining that the corresponding network traffic data is out-of-distribution traffic if the out-of-distribution score is higher than the preset determination threshold, the method further includes: Perform preset security response operations on network traffic data that is identified as out-of-distribution traffic.

[0007] Optionally, the preset security response actions include: triggering a security alarm, blocking network connection, limiting communication rate, or logging security information.

[0008] Optionally, inputting the graph structure data into a pre-trained graph neural network specifically includes: The graph structure data is subjected to feature standardization processing to obtain normalized graph data; the feature standardization processing includes at least one of the following operations: normalization of node features and edge features, missing value imputation, and outlier truncation; The normalized graph data is input into a pre-trained graph neural network.

[0009] Optionally, the network traffic data includes at least one of the following: source IP address, destination IP address, source port number, destination port number, communication protocol type, packet size, number of packets, timestamp, or session duration.

[0010] Optionally, the preset energy function is: ; Where fy(x) is the Logit output of the model for class y.

[0011] Optionally, the graph structure data is constructed based on network traffic data collected within a preset time window; The preset judgment threshold is set at the position where the out-of-distribution score distribution of the known distribution flow sample and the out-of-distribution score distribution of the expected out-of-distribution flow sample have the least overlap.

[0012] In a second aspect, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the network traffic data identification method for generalized graph distribution detection as described above.

[0013] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the network traffic data identification method for generalized graph distribution detection as described above.

[0014] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the network traffic data identification method for generalized graph distribution external detection as described above.

[0015] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a method, device, medium, and product for identifying network traffic data using generalized graph out-of-distribution detection. The method includes: acquiring network traffic data containing communication entities and interaction relationships; converting the network traffic data into a graph structure representation to obtain graph structure data; wherein communication entities are mapped as nodes, and communication behaviors between communication entities are mapped as edges between nodes; inputting the graph structure data into a pre-trained graph neural network; the graph neural network aggregates node neighborhood information through its graph convolutional layers, outputting feature representations of nodes or the graph and classification logic values ​​corresponding to preset categories; based on the classification logic values, calculating a scalar out-of-distribution score for each graph structure data using a preset energy function; the out-of-distribution score quantifies the degree to which the graph structure data deviates from the data distribution within the training distribution; comparing the out-of-distribution score with a preset judgment threshold; if the out-of-distribution score is higher than the preset judgment threshold, then determining the corresponding network traffic data as out-of-distribution traffic; wherein the preset judgment threshold is preset based on the out-of-distribution score distribution of known in-distribution traffic samples. This application, by converting network traffic data into graph structure data, can naturally capture the complex relationships between network hosts, services, and sessions, providing rich structural context information for traffic analysis. Furthermore, by leveraging the graph convolutional layers in graph neural networks to aggregate node neighborhood information, deep-level graph structure feature representations can be learned. Based on this, the out-of-distribution score calculated through the energy function can sensitively quantify the degree of deviation between the current traffic graph and the normal distribution learned during training. Therefore, even when facing network attacks or anomalous behaviors from entirely new distributions not encountered during training, this application can achieve effective detection by comparing the score with a threshold, significantly improving the ability to discover unknown threats and enabling reliable and automated identification of unknown threat traffic. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is an application environment diagram of a network traffic data identification method for generalized graph distribution external detection in one embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating a method for identifying network traffic data from the detection of generalized graph distributions, provided as an embodiment of this application.

[0019] Figure 3 This is a schematic diagram of an off-distribution detection paradigm provided in an embodiment of this application.

[0020] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] With its powerful modeling capabilities, graph-structured data, as a core tool for analyzing complex systems, has been widely applied in many key fields such as social networks, recommender systems, citation networks, fraud detection, network intrusion prevention, biological systems, and financial transactions. However, real-world graph data often exhibits dynamic evolution and heterogeneous diversity, leading to instability and diversity in its data distribution. This makes the independent and identically distributed (i.i.d.) assumption relied upon by traditional models difficult to hold in reality. When deployed models encounter out-of-deformation (OOD) samples, their performance may degrade significantly or even catastrophically, triggering serious security and trust crises. Therefore, the OOD problem has become one of the core bottlenecks restricting the large-scale practical application of graph machine learning models.

[0023] To address these challenges, researchers have taken different approaches, leading to a series of research directions such as Graph Anomaly Detection (GAD), Graph Novelty Detection (GND), Graph Open-Set Recognition (GOSR), and Graph Out-of-Distribution Detection (GOD). Although these fields are highly related in their core objective—identifying and processing "unknown" data—they have long been developing in a fragmented manner, with significant gaps in problem definition, evaluation processes, and even academic discourse. This fragmented development has caused unnecessary confusion for researchers and practitioners, and has hindered the cross-integration and effective transfer of technologies.

[0024] This application is applied to network traffic analysis and network security protection scenarios. By constructing network traffic data into graph structure data and using graph neural networks for out-of-distribution detection, it can identify unknown network behaviors, abnormal traffic, or unseen communication patterns.

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] The network traffic data identification method for generalized graph distribution out-of-distribution detection provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server.

[0027] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0028] In one exemplary embodiment, such as Figure 2 As shown, a method for identifying network traffic data outside of generalized graph distribution detection is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1Taking server 104 as an example, the explanation includes the following steps S1 to S5. Wherein: S1. Obtain network traffic data containing communication entities and interaction relationships.

[0029] In this embodiment, the network traffic acquisition module collects network traffic data in the target network environment, specifically including: Deploy traffic acquisition components at network nodes, including but not limited to switches, routers, servers, or mirror ports; Network communication data can be collected in real time or periodically through traffic mirroring, data capture, or traffic export. The collected network traffic data includes at least one or more of the following fields: Source IP address, destination IP address; source port number, destination port number; communication protocol type; packet size, number of packets; timestamp or session duration.

[0030] The collected network traffic data is stored in a local cache or transmitted to a backend analysis server.

[0031] S2. The network traffic data is converted into a graph structure representation to obtain graph structure data; wherein communication entities are mapped as nodes, and communication behaviors between communication entities are mapped as edges between nodes. The graph structure data is constructed based on network traffic data collected within a preset time window.

[0032] In this embodiment, the graph construction module can construct graph structure data based on the network traffic data, specifically including: According to preset mapping rules, communication entities in network traffic are mapped to nodes in the graph. These nodes include, but are not limited to: host nodes; IP address nodes; network session nodes; and service or port nodes.

[0033] Based on the communication behavior between nodes, edge relationships are established between nodes. The edge represents the network communication behavior between the nodes, and the edge can be a directed edge or an undirected edge.

[0034] Associate attribute features with nodes and edges respectively, where: Node characteristics include at least communication frequency, number of connections, port distribution, or statistical characteristics; Edge characteristics include at least the number of data packets, the number of bytes transmitted, and the duration of communication; Generate the corresponding graph structure data within the preset time window.

[0035] S3. Input the graph structure data into a pre-trained graph neural network; the graph neural network aggregates node neighborhood information through its graph convolutional layers and outputs the feature representation of the node or graph and the classification logic value corresponding to the preset category.

[0036] In this embodiment, the graph structure data is first preprocessed: the network traffic graph can be preprocessed by the feature extraction module, specifically including: Normalize or standardize the node and edge features.

[0037] Fill in, correct, or truncate missing or abnormal feature values.

[0038] The processed node feature matrix, edge feature matrix, and graph structure relationships are encoded into a unified data representation format.

[0039] The processed graph structure data is used as input to the subsequent graph neural network model.

[0040] The graph neural network inference module performs feature learning on the graph structure data. The graph neural network model includes at least one graph convolutional layer for aggregating neighborhood feature information of nodes; High-dimensional feature representations of nodes, subgraphs, or the entire graph in a network traffic graph are obtained through multi-layer graph feature transformation. The linear mapping layer outputs the corresponding classification prediction result or intermediate representation vector.

[0041] The training process of a graph neural network is as follows: During the model training phase, the graph neural network uses graph structure data constructed from distributed network traffic samples as training input. Specifically: Input data: The training input is standardized graph structure data, specifically including: a node feature matrix, used to describe the statistical attributes and behavioral characteristics of the communicating entities; an adjacency matrix or edge relationship information, used to describe the interaction relationships between the communicating entities; and edge features (if they exist), used to describe the communication behavioral attributes.

[0042] Output data: After computation through multi-layer graph convolution or graph attention, the graph neural network outputs: node-level or graph-level feature representations; classification logits corresponding to the preset categories and classification probability distributions.

[0043] Tag information: The training phase uses only distributed network traffic samples and is supervised based on their known service type, security category, or traffic type. Labels are used in a category-based format to calculate the cross-entropy loss function, thereby optimizing the model parameters.

[0044] Through the above training process, the graph neural network can learn the structural features and category discrimination boundaries of network traffic within the distribution, providing a reliable feature representation basis for subsequent out-of-distribution detection.

[0045] S4. Based on the classification logic value, calculate a scalar out-of-distribution score for each graph structure data using a preset energy function; the out-of-distribution score is used to quantify the degree to which the graph structure data deviates from the data distribution within the training distribution.

[0046] The preset energy function is: ; Where fy(x) is the Logit output of the model for class y.

[0047] In this embodiment, the out-of-distribution scoring calculation module can perform out-of-distribution scoring on the graph structure data, specifically including: Based on the intermediate feature representation or classification Logits output by the graph neural network model, an out-of-distribution score is calculated for each graph structure data. The out-of-distribution score measures the similarity between the current network traffic sample and the in-distribution network traffic samples from the training phase. The out-of-distribution score is a scalar value; a higher score indicates that the sample is more likely to be out-of-distribution network traffic.

[0048] In this application, the out-of-distribution score is calculated directly using a preset energy function based on the classification logits output by the graph neural network. Specifically: The graph neural network first learns features from the input graph structure data and outputs Logits values ​​for each category. Then, based on these Logits values, a scalar out-of-distribution score is calculated using an energy function. The energy function maps multi-category Logits to a single confidence metric. This out-of-distribution score can be calculated directly without additional training modules and is used to measure the degree to which the current sample deviates from the training distribution. Finally, by comparing this score with a preset threshold, out-of-distribution sample determination is achieved.

[0049] S5. Compare the out-of-distribution score with a preset judgment threshold; if the out-of-distribution score is higher than the preset judgment threshold, then determine that the corresponding network traffic data is out-of-distribution traffic; wherein, the preset judgment threshold is preset based on the out-of-distribution score distribution of known in-distribution traffic samples. The preset judgment threshold is set at the position where the out-of-distribution score distribution of known in-distribution traffic samples and the out-of-distribution score distribution of expected out-of-distribution traffic samples have the least overlap.

[0050] This embodiment allows the decision module to determine network traffic samples based on the out-of-distribution score, specifically including: After the model training is completed, the out-of-distribution score distribution is statistically analyzed based on the in-distribution network traffic samples. Set a preset threshold for out-of-distribution judgment based on statistical results; When the out-of-distribution score of the network traffic sample to be detected is higher than the preset judgment threshold, the network traffic data is determined to be out-of-distribution traffic.

[0051] Perform at least one processing operation on network traffic data determined to be out of distribution, including: Triggering an anomaly or security alert; Block or restrict network traffic; Mark the network traffic as unknown and log it.

[0052] Compared with the prior art, this embodiment has at least the following technical effects: By modeling network traffic data as graph structure data, the complex relationships between network communication entities can be effectively depicted; By introducing graph neural networks for feature learning, the ability to model complex network traffic patterns has been improved. By using out-of-distribution scoring and threshold determination mechanisms, we can effectively identify unknown attack traffic or unseen communication patterns. This improves the robustness and security of the network traffic analysis system in real network environments without relying on pre-labeled anomaly samples.

[0053] As an optional implementation, the generalized out-of-distribution (OOD) detection method in this embodiment may include the following steps: The OOD detection problem refers to the ability of a machine learning model, when deployed in a real-world environment, to identify whether the input data comes from a distribution not seen during training (i.e., OOD data). Specifically, the OOD detection problem can be formalized as a binary decision problem. Suppose the training data comes from an in-distribution (ID) data distribution. During testing, the model will not only encounter challenges from... The samples will also encounter samples from another unknown distribution. (and ≠ OOD samples.

[0054] When the model encounters an input x∈X during testing, it needs to determine whether this input comes from a known distribution. (i.e., ID data) still comes from an unknown distribution. (i.e., OOD data). OOD detection can be formalized as a binary classification problem, with the goal of determining at test time whether input x comes from Pin or... The goal of OOD detection is to maximize the detection rate of OOD data through an optimization problem, while keeping the classification error rate and OOD error rate of ID data at a low level. Mathematically, this problem can be represented by the following optimization problem: in: (x) is the OOD detection function, used to determine whether the input x is OOD; (x) is a multi-class classification function used to classify ID data; α and τ are set thresholds that control the classification error rate and OOD error rate, respectively. This represents the set of parameters for a graph neural network model, including the weight parameters and bias parameters for each layer. This represents the calculation of the expected value of a sample x under an out-of-distribution data distribution Pout, where P... out This represents the data distribution that is unknown or not involved in the training. Indicates the data distribution P within the distribution. in The expected value of sample x is calculated below, where P in This represents the known data distribution during the training phase; P represents the joint distribution of sample x and its corresponding label y. xy Next, the expected value of the sample and label pair is calculated.

[0055] The key to out-of-distribution detection lies in establishing a reliable confidence evaluation mechanism for the model. For example... Figure 3 The general research paradigm shown demonstrates that for any neural network and task, a corresponding scoring function can be designed to quantify the model's confidence level for each input sample. In this GNN architecture, the specific implementation process of this mechanism is as follows: Feature extraction and classification prediction: The input graph data is processed by a series of GNN Blocks (which may include graph convolution, activation functions and regularization components), and finally outputs Logits through a linear layer, and then passes through a softmax function to obtain the standard classification probability distribution P(y).

[0056] OOD Score Calculation: To perform OOD detection, the model calculates an OOD output score in parallel. The illustration uses the energy function E(x) as a typical example of the scoring function, defined as follows: , where fy(x) is the model's Logit output for class y. This function maps Logits to a scalar score; the lower the energy value, the more similar the sample is to the training distribution (ID data), and the higher the confidence level.

[0057] Score Distribution and Threshold Determination: After model training, the calculated scores of an ID validation set are collected, and their score density distribution is plotted. The ideal distribution is shown by the curve in the figure: ID data (solid line) scores are generally concentrated in the low-energy (high-confidence) region, while OOD data (dashed line) scores are distributed in the high-energy (low-confidence) region. By setting a threshold δ in the region with the least overlap between the two, effective determination can be achieved: samples with scores below δ are classified as ID data and fed into the classifier f(x); samples with scores above δ are classified as OOD data and are thus intercepted or specially processed to suppress error propagation.

[0058] Based on the differences in training data composition and detection targets, generalized graph OOD detection can be divided into three interrelated sub-tasks: Graph Anomaly Detection (GAD): This method aims to discover samples that significantly deviate from normal patterns, encompassing both sensory and semantic anomalies. It can employ statistical models, reconstruction error methods, and ensemble learning methods.

[0059] Graph Novelty Detection (GND) and Open Set Recognition (OSR): These methods aim to identify novel patterns not seen in the training set, including single-class and multi-class novelty, and commonly use density estimation, cluster analysis, and generative models. They mainly solve open set classification problems, classifying among known categories and rejecting unknown categories, and commonly use confidence thresholding and GAN adversarial training.

[0060] Classic OOD detection: This method extends traditional OOD detection methods to the field of graph data. It is based on distribution assumptions and commonly uses output probability calibration and Bayesian uncertainty modeling.

[0061] The core value of this unified paradigm lies in breaking down traditional classification barriers and providing a common technical foundation and methodological framework for different sub-tasks.

[0062] In simple terms, the generalized graph out-of-distribution detection method in this embodiment mainly includes the following steps: Step S1: Graph Data Construction and Preprocessing Input: Raw network traffic data.

[0063] Execution method: The network traffic data is converted into graph structure data and then standardized, specifically including: The communication entities are mapped to nodes; communication behaviors are mapped to edge relationships between nodes; node feature matrices and adjacency matrices are constructed; node features are normalized, missing values ​​are filled, and outliers are handled.

[0064] Output: Standardized graph structure data (node ​​feature matrix, adjacency matrix, and structural relationship information).

[0065] Step S2: Feature extraction and classification prediction using graph neural networks: Input: Standardized graph structure data.

[0066] Implementation method: A graph neural network model is used to learn features from graph-structured data, including: Node neighborhood information is aggregated through graph convolutional layers or graph attention layers; non-linear mapping is performed through activation functions and regularization layers; and classification logits are obtained through linear output layers.

[0067] Output: High-dimensional feature representation of the graph data and the corresponding category classification Logits results.

[0068] Step S3: Calculation of out-of-distribution scores: Input: The category Logits output from step S2.

[0069] Execution method: The out-of-distribution score of each sample is calculated based on the energy function. Specifically, the multi-class output is converted into a single score index by exponentially summing the Logits of all categories and taking the logarithm mapping, which is used to measure the degree to which the sample deviates from the training distribution.

[0070] Output: The out-of-distribution score for each sample.

[0071] Step S4: Determine the threshold: Input: The set of out-of-distribution scores for in-distribution validation data.

[0072] Implementation method: Statistically analyze the sample score distribution within the distribution, and determine the out-of-distribution judgment threshold through density analysis or quantile statistics.

[0073] Output: Threshold for out-of-distribution detection.

[0074] Step S5: Out-of-distribution detection and judgment: Input: Out-of-distribution score and judgment threshold of the sample to be tested.

[0075] Execution method: Compare the sample score with a threshold: When the score does not exceed the threshold, it is determined to be an in-distribution sample and the classification result is output; When the score exceeds the threshold, it is determined to be an out-of-distribution sample.

[0076] Output: Sample distribution determination results (in-distribution or out-of-distribution) and in-distribution sample classification results.

[0077] This solution achieves effective detection of unknown network traffic and new attack behaviors through a unified process of graph structure modeling, graph neural network feature learning, energy function scoring mechanism, and threshold determination.

[0078] Specifically, the method for detecting out-of-distribution (OOD) generalized graphs is as follows: I. Overall Execution Flow of the Method (Divided into Steps): The core of this generalized graph out-of-distribution detection method is to achieve accurate identification of out-of-distribution samples in graph data through a closed-loop process of "data preprocessing → feature extraction and classification prediction → OOD score calculation → threshold determination → test set detection and judgment". The specific steps are as follows: |Step Number|Step Name|Input|Output|Core Algorithm / Method|.

[0079] |1|Data Preprocessing|Original graph data (including training set ID data, validation set ID data, and test set data to be detected)|Standardized graph data (including node feature matrix, adjacency matrix, and label information)|Graph data cleaning and standardization processing algorithms|.

[0080] |2|Feature Extraction and Classification Prediction|Standardized Graph Data (input into the corresponding datasets during training / validation / testing phases)|Classification Logits Output, ID Data Classification Probability Distribution P(y)|Graph Neural Network (GNN) Model (including graph convolutional layers, activation layers, regularization layers, and linear output layers), softmax function|.

[0081] |3|OOD score calculation|Classification Logits output from step 2|OOD score (energy value E(x)) for each sample|Energy function|.

[0082] |4|Threshold Determination|OOD score set of validation set ID data|Optimal decision threshold δ|Density distribution analysis, statistical threshold selection method|.

[0083] |5|Test Set Detection and Judgment|Test set sample OOD score, optimal threshold δ|Sample judgment result (ID / OOD), ID sample classification result|Threshold comparison judgment rule, classification function f_θ(x)|.

[0084] II. Detailed instructions for each step: Step 1: Data Preprocessing 1.1 Algorithm / Method Name: Graph data cleaning and standardization algorithms: 1.2 Specific execution process: 1. Unified Data Format: The input raw graph data (including training set ID data, validation set ID data, and test set data to be detected) is uniformly converted into the standard format of "node feature matrix + adjacency matrix + labels (training / validation set only)". Wherein: The node feature matrix X∈R^(n×d) is: n is the number of nodes in the graph, and d is the feature dimension of each node, ensuring that the feature dimensions of all samples are consistent. The adjacency matrix A∈R^(n×n) uses a binary adjacency matrix to represent the connection relationship between nodes. A_ij=1 indicates that there is an edge connection between node i and node j, and A_ij=0 indicates that there is no connection. Label y (training / validation set only): uses one-hot encoding format to label the category of ID data.

[0085] 2. Outlier handling: For missing values ​​in the node feature matrix, the strategy of "filling in the mean of features of nodes of the same type" is adopted; for isolated nodes (nodes without any edge connections), if they are training / validation set ID data, they are directly removed; if they are test set data to be detected, they are retained and marked as "isolated node samples".

[0086] 3. Feature Standardization: Perform Z-score standardization on the node feature matrix X, using the following formula: X' _ij =(X _ij μ _j ) / σ _j ; Where, μ _j Let σ be the mean of the j-th feature. _j X represents the standard deviation of the j-th feature, ensuring that the numerical ranges of each feature dimension are consistent and avoiding the influence of feature scale during model training. _ij This represents the value of the i-th node in the j-th feature dimension of the node feature matrix X, where: i represents the node number in the graph; j represents the j-th attribute dimension of the node feature; and X is the node feature matrix, used to describe the attribute information of each node.

[0087] 1.3 Input / Output Description: Input: Raw graph data (which may be graph structure data in different formats, such as adjacency lists, graph database exported files, etc., including training set ID data, validation set ID data, and test set data to be detected); Output: Standardized graph data (node ​​feature matrix X', adjacency matrix A, label y (training / validation set only)), all output data meet the input requirements of the GNN model.

[0088] Step 2: Feature Extraction and Classification Prediction 2.1 Algorithm / Method Name: Graph Neural Network (GNN) feature extraction and classification model (including graph convolutional layer, activation layer, regularization layer, and linear output layer), and softmax classification function.

[0089] 2.2 Specific Implementation Process 1. Model Structure Definition: The GNN model consists of 3 core modules, and the structure and function of each module are as follows: Graph convolutional layers: These layers employ graph convolution operations (such as GCN, GAT, or GraphSAGE) to effectively extract graph structure information by aggregating the features of individual nodes with those of their neighboring nodes. Taking GCN as an example, the convolution operation formula is: H^(l+1)=σ(ÃH^(l)W^(l)); Where H^(l) is the feature output of the l-th layer, W^(l) is the trainable weight matrix of the l-th layer, σ is the activation function, and à is the normalized adjacency matrix (Ã=Û^(-1 / 2)(I+A)Û^(-1 / 2), where Û is the degree matrix). Û represents the degree matrix of the graph, which is a diagonal matrix whose diagonal elements represent the degree value of the corresponding node, i.e., the number of edges connecting the node to other nodes. Û^(-1 / 2) represents the negative first power of the degree matrix Û, used to normalize the adjacency matrix to reduce the impact of degree differences between different nodes on the feature propagation process. I represents the identity matrix, used to introduce self-connections in the adjacency matrix, allowing nodes to retain their own features during information aggregation. A represents the adjacency matrix of the graph, used to describe the connection relationships between nodes. When there are edge connections between nodes, the corresponding element is 1; otherwise, it is 0.

[0090] Activation layer: The ReLU function (σ(x)=max(0,x)) is used to introduce nonlinear transformation and enhance the model's ability to fit complex features.

[0091] Regularization layer: A Dropout layer (dropoutrate=0.5) is used to randomly discard some neuron outputs to avoid model overfitting.

[0092] Linear output layer: A fully connected layer is used to map the feature output of the last GNN layer to a dimension equal to the number of ID data categories, resulting in classification Logits (dimension c, where c is the total number of ID data categories).

[0093] 2. Model Training (Training Set ID Data Only): Loss function: The cross-entropy loss function is used, and the optimization objective is to minimize the classification error of the ID data. The formula is: L_cls=-Σ(y_i·log(P(y_i))); Where y_i is the true label of the i-th sample, and P(y_i) is the classification probability output by the model; Optimizer: The Adam optimizer is used, with a learning rate set to 1e. -3 The iteration count is 200 rounds, and the validation set performance is evaluated every 50 rounds. The model parameters with the highest classification accuracy on the validation set are retained.

[0094] 3. Classification prediction (performed during training / validation / testing phases): The standardized graph data (node ​​feature matrix X', adjacency matrix A) output from step 1 is input into the trained GNN model. After processing through graph convolutional layers, activation layers, and regularization layers, the classification Logits are obtained through a linear output layer. Applying the softmax function to the Logits transforms the data to obtain the classification probability distribution P(y) of the ID data, as shown in the formula: P(y=k)=exp(Logits_k) / Σ(exp(Logits_m))(m∈[1,c]); Wherein, Logits_k is the Logits output of the k-th class, and P(y=k) represents the probability that the sample belongs to the k-th class; Logits_m represents the classification logic output value of the graph neural network model for the input sample in the m-th class (the original prediction score without softma normalization), where: m represents the class index, the Logits vector is output by the last linear mapping layer of the model; Logits_m is used to characterize the relative confidence level of the sample belonging to the m-th class, and can be further converted into the probability value of the corresponding class through the softmax function.

[0095] 2.3 Input / Output Description: Input: The standardized graph data output from step 1 (node ​​feature matrix X', adjacency matrix A); Output: Classification Logits (dimension c), classification probability distribution P(y) (dimension c, satisfying ΣP(y=k)=1).

[0096] Step 3: OOD score calculation: 3.1 Algorithm / Method Name: Energy Function Scoring Algorithm: 3.2 Specific Execution Process: 1. Based on the classification Logits output in step 2, the OOD score E(x) for each sample is calculated using the energy function, as follows: E(x)=-log(Σ(exp(Logits_y)))(y∈[1,c]); Where Logits_y is the Logits output of class y, and Σ(exp(Logits_y)) is the exponential sum of Logits for all classes; 2. Scoring logic explanation: The lower the energy value E(x), the more likely the sample's Logits are concentrated in a few classes (i.e., the higher the model's confidence in class assignment of the sample), and the more likely it is to belong to ID data; the higher the energy value E(x), the more likely the sample's Logits are dispersed across multiple classes (i.e., the lower the model's confidence in class assignment of the sample), and the more likely it is to belong to OOD data.

[0097] 3.3 Input / Output Description: Input: The category Logits (with dimension c) output from step 2; Output: The OOD score E(x) of the sample (scalar value, unitless, ranging from [0,+∞)).

[0098] Step 4: Threshold Determination: 4.1 Algorithm / Method Name: ID data scoring density distribution analysis and statistical threshold selection method.

[0099] 4.2 Specific Implementation Process: 1. Collect the OOD scores E(x) of all ID data in the validation set, and construct the score set S={E_1,E_2,...,E_m} (m is the number of ID data samples in the validation set); 2. Plot the density distribution of the rating set S (such as the kernel density estimation curve) and observe the distribution characteristics of the ID data ratings (usually a unimodal distribution, concentrated in the low-energy region). 3. Set threshold selection target: maximize the detection rate of OOD data while ensuring that the false positive rate of ID data (the proportion of ID data that is identified as OOD data) is ≤ α (α is a preset value, the default α=0.05); 4. Determine the optimal threshold δ based on the density distribution map: In the density distribution map, find the smallest δ value that satisfies the condition that "the proportion of samples with scores ≤ δ to the total number of ID data in the validation set is ≥ (1-α)". This δ is the optimal decision threshold. For example, if α = 0.05, then δ is the 95th percentile of the ID data scores in the validation set.

[0100] 4.3 Input / Output Description: Input: The set of OOD scores S for the validation set ID data; Output: Optimal decision threshold δ (scalar value, consistent with the range of E(x)).

[0101] Step 5: Test set detection and judgment: 5.1 Algorithm / Method Name: Threshold comparison judgment rules, classification function f_θ(x); 5.2 Specific Implementation Process: 1. For each sample to be tested in the test set, perform steps 1-3 to obtain the OOD score E_test(x) for that sample; 2. Perform threshold comparison: If E_test(x)≤δ: determine that the sample is ID data, input the classification probability distribution P(y) output in step 2 into the classification function f_θ(x), and output the category to which the sample belongs (i.e. the category with the highest probability in P(y)); If E_test(x) > δ: determine that the sample is OOD data, output the "OOD sample" label, and do not classify the category; 3. Results Output: For all test set samples, compile the judgment results (ID / OOD) and the classification results of ID samples to form the final detection report.

[0102] 5.3 Input / Output Description Input: OOD score E_test(x) of the sample to be tested in the test set, and the optimal threshold δ determined in step 4; Output: Final judgment results (ID / OOD labeling) of test set samples, and category classification results of ID samples.

[0103] III. Supplementary Explanation 1. The core innovation of this method lies in the "unified paradigm": through the above 5 steps, sub-tasks such as graph anomaly detection (GAD), graph novelty detection (GND), open set recognition (OSR), and classic OOD detection are integrated into the same process. Only the value of the threshold α needs to be adjusted according to the specific sub-task (for example, α can be set to 0.01 for the GAD task to strictly control the false positive rate of ID data; α can be set to 0.05 for the OSR task to balance the false positive rate and the OOD detection rate). 2. The algorithms / methods of each step can be replaced according to the actual scenario: For example, the GNN model in step 2 can be replaced with different types of graph neural networks such as GAT and GraphSAGE; the energy function in step 3 can be replaced with other OOD scoring functions such as confidence score and entropy value. After the replacement, only the threshold determination process in step 4 needs to be re-executed. 3. All steps are executed by computer equipment and implemented through software programs: they can be divided into a data preprocessing module, a GNN feature extraction and classification module, an OOD score calculation module, a threshold determination module, and a detection and judgment module. Standardized data transmission is achieved between the modules through data interfaces.

[0104] Compared with the prior art, this embodiment has at least the following technical effects: (a) Improve the ability to identify unknown network traffic and new types of attack behaviors: This embodiment constructs a network traffic graph from network traffic data, modeling network communication entities and their relationships in a graph structure, thus avoiding the approach of relying solely on single traffic features for judgment. Furthermore, by utilizing graph neural networks to learn feature representations of the network traffic graph, it is possible to fully extract structural relationship information between network communication entities.

[0105] Therefore, even when faced with unknown network traffic patterns or new attack behaviors that did not appear during the training phase, it is still possible to effectively identify them through the out-of-distribution scoring mechanism, thereby significantly improving the detection capability of out-of-distribution network traffic.

[0106] (ii) Improving detection robustness without the need for pre-labeling of anomalous samples: Existing technologies typically rely on training with a large number of known anomalous samples, making it difficult to cover constantly evolving forms of network attacks. This embodiment calculates out-of-distribution scores based on the output of a graph neural network and sets a judgment threshold based on in-distribution network traffic samples, thereby enabling the detection of unknown traffic without relying on explicitly labeled anomalous traffic samples.

[0107] This technical solution effectively reduces the reliance on prior information about anomalous samples and improves the robustness of the system in real and complex network environments.

[0108] (iii) Improve the overall accuracy of network traffic analysis and reduce false positives and false negatives: This embodiment reduces the impact of noise and outliers on the model inference results by preprocessing the node and edge features of the network traffic graph; at the same time, it distinguishes between in-distribution traffic and out-of-distribution traffic through an out-of-distribution scoring and threshold determination mechanism.

[0109] The aforementioned technical methods avoid forcibly sending out-of-distribution traffic into the classification model for misclassification, thereby effectively reducing misclassification and missed classification and improving the overall accuracy of network traffic analysis.

[0110] (iv) Enhance the system's applicability and scalability in large-scale network environments: This embodiment uses a modular approach to complete network traffic collection and graph construction, and performs unified processing on the graph structure data, enabling the system to adapt to network environments of different sizes and topologies.

[0111] Since the method is based on a unified graph modeling and distributed decision process, it can continue to be applied when the network size increases or the network structure changes, only the graph construction rules or model parameters need to be adjusted, thereby improving the scalability and engineering deployability of the system.

[0112] (v) Enhance the real-time performance and reliability of network security protection: By performing operations such as alarming, blocking, or logging on network traffic samples determined to be outside the distribution range, this embodiment can promptly identify and respond to abnormal or unknown traffic during network operation, preventing the further spread of abnormal traffic.

[0113] The effectiveness of this technology stems directly from the combination of out-of-distribution scoring mechanisms and threshold determination strategies, enabling network security protection systems to achieve higher real-time performance and reliability.

[0114] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 4 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for identifying network traffic data using generalized graph distribution detection.

[0115] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0116] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0117] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0118] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0119] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0120] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0121] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0122] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for identifying network traffic data from generalized graph distribution external detection, characterized in that, include: Acquire network traffic data containing communication entities and interaction relationships; The network traffic data is converted into a graph structure representation to obtain graph structure data; In this context, communication entities are mapped as nodes, and the communication behavior between communication entities is mapped as edges between nodes; The graph structure data is input into a pre-trained graph neural network; the graph neural network aggregates node neighborhood information through its graph convolutional layers and outputs feature representations of nodes or graphs and classification logic values ​​corresponding to preset categories. Based on the classification logic value, a scalar out-of-distribution score is calculated for each graph structure data using a preset energy function; the out-of-distribution score is used to quantify the degree to which the graph structure data deviates from the data distribution within the training distribution; The out-of-distribution score is compared with a preset judgment threshold; if the out-of-distribution score is higher than the preset judgment threshold, the corresponding network traffic data is determined to be out-of-distribution traffic; wherein, the preset judgment threshold is preset based on the out-of-distribution score distribution of known in-distribution traffic samples.

2. The network traffic data identification method for generalized graph distribution external detection according to claim 1, characterized in that, The out-of-distribution score is compared with a preset judgment threshold; If the out-of-distribution score is higher than the preset judgment threshold, then after determining that the corresponding network traffic data is out-of-distribution traffic, the process further includes: Perform preset security response operations on network traffic data that is identified as out-of-distribution traffic.

3. The network traffic data identification method for generalized graph distribution external detection according to claim 2, characterized in that, Preset security response actions include: triggering a security alarm, blocking network connections, limiting communication rates, or logging security information.

4. The network traffic data identification method for generalized graph distribution external detection according to claim 1, characterized in that, The graph structure data is input into a pre-trained graph neural network, specifically including: The graph structure data is subjected to feature standardization processing to obtain normalized graph data; the feature standardization processing includes at least one of the following operations: normalization of node features and edge features, missing value imputation, and outlier truncation; The normalized graph data is input into a pre-trained graph neural network.

5. The network traffic data identification method for generalized graph distribution external detection according to claim 1, characterized in that, The network traffic data includes at least one of the following: source IP address, destination IP address, source port number, destination port number, communication protocol type, data packet size, number of data packets, timestamp, or session duration.

6. The network traffic data identification method for generalized graph distribution external detection according to claim 1, characterized in that, The preset energy function is: ； Where fy(x) is the Logit output of the model for class y.

7. The network traffic data identification method for generalized graph distribution external detection according to claim 1, characterized in that, The graph structure data is constructed based on network traffic data collected within a preset time window; The preset judgment threshold is set at the position where the out-of-distribution score distribution of the known distribution flow sample and the out-of-distribution score distribution of the expected out-of-distribution flow sample have the least overlap.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the network traffic data identification method for generalized graph distribution out-of-detection as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the network traffic data identification method for generalized graph distribution external detection as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the network traffic data identification method for generalized graph distribution external detection as described in any one of claims 1-7.

Images