Encrypted network abnormal flow detection method, system, device and medium

Abstract

The invention relates to the technical field of network security, in particular to an encrypted network abnormal traffic detection method, system and device and a medium. According to the invention, by combining edge calculation and federated learning, the efficiency and precision of encrypted network traffic anomaly detection are improved. The edge device independently trains a local model and uploads the local model to the central server, and the central server aggregates model parameters through federal learning to generate a global detection model. The method can be closer to a data source for real-time analysis, and delay and bandwidth consumption caused by data transmission are reduced. Meanwhile, by means of local training and model aggregation, concentrated storage and transmission of sensitive data are avoided, and data privacy is effectively guaranteed. The global model is optimized for multiple rounds, abnormal behaviors in encrypted traffic can be accurately identified, the network security protection capability is improved, and an efficient and accurate network security solution with privacy protection is provided.

Description

Technical Field

[0001] The present disclosure relates to the field of network security technology, and in particular to a method, system, device and medium for detecting abnormal traffic in an encrypted network. Background Art

[0002] With the rapid development of network technology, the widespread application of encryption, and the popularization of high-bandwidth, low-latency communication technologies such as 5G and WiFi 6, the proportion of encrypted network traffic in communications continues to increase. At the same time, IoT devices have also entered a period of rapid development. Various edge devices, such as industrial control equipment, medical sensors, and smart homes, are gradually connected to the internet, generating large amounts of encrypted traffic. As the nation's largest communications infrastructure operator, China Tower leverages its extensive network of base stations and towers to provide communication and computing support for these devices. In the 5G era, the synergy between base stations and edge devices becomes particularly important.

[0003] However, with the widespread use of these devices, network security risks are becoming increasingly prominent. For example, medical devices and smart cars may become targets of attacks, facing risks such as data leakage and remote control. Therefore, how to detect anomalies in encrypted network traffic in real time in edge computing environments and ensure data security has become a critical issue that needs to be addressed.

[0004] While encryption technology plays a vital role in protecting privacy and data security, it also provides a hiding place for abnormal traffic and malicious behavior, posing significant challenges to network traffic monitoring and anomaly detection. Traditional traffic analysis methods, which primarily rely on low-level statistical features of header information and payload, struggle with encryption scenarios, particularly when dealing with short flows or complex traffic behavior, and fail to fully exploit potential correlations between headers and payloads. Furthermore, traditional malicious traffic detection systems are typically deployed at upstream network gateways to centrally monitor and issue alerts. This approach is ineffective against scenarios such as small LAN attacks, near-source attacks, and intranet worm propagation. It struggles to capture the internal behavior and state of edge devices and suffers from insufficient real-time performance, hindering rapid response to attacks. Furthermore, existing neural network-based detection models are complex and require high computational and memory resources. The limited processing power of most IoT edge devices makes it difficult to support these complex detection algorithms. These shortcomings highlight the need to design lightweight and efficient encrypted traffic detection methods to protect edge devices.

[0005] Current anomaly detection methods for encrypted network traffic primarily focus on identifying anomalous behavior by analyzing external characteristics of encrypted traffic, such as traffic volume, packet exchange frequency, and traffic statistics. With the widespread use of encrypted traffic, the effectiveness of these traditional methods has been limited, primarily because they fail to fully account for the underlying characteristics and complex behavioral patterns of encrypted traffic.

[0006] In existing technologies, detection methods for encrypted network traffic mainly include the following categories:

[0007] Traffic statistics-based detection methods: These methods typically rely on external traffic characteristics, such as packet size, transmission rate, and protocol type, to analyze these characteristics and determine whether there are abnormal traffic patterns. These methods are relatively simple to implement, but lack the ability to deeply analyze the content of encrypted traffic, resulting in low anomaly detection accuracy in practical applications. This makes them prone to false positives and false negatives, especially in the face of complex attacks.

[0008] Machine learning-based detection methods: As encrypted traffic becomes increasingly complex, machine learning methods are widely used for traffic anomaly detection. By extracting usable features from encrypted traffic, machine learning models can be trained to identify anomalous patterns in the traffic. Common algorithms include support vector machines (SVMs), decision trees, and random forests. While these methods can improve detection accuracy to some extent, they still face the following challenges: First, their feature extraction capabilities are limited, preventing them from deeply exploring the complex behavior within encrypted traffic; second, when processing large amounts of encrypted traffic, the computational overhead is high, making it difficult to meet the requirements of real-time detection.

[0009] Deep learning-based detection methods: Deep learning methods, particularly convolutional neural networks (CNNs), recurrent neural networks (RNNs), and graph neural networks (GNNs), are increasingly being used for anomaly detection in encrypted traffic. These methods can extract richer features, especially for complex traffic behaviors and potential attack patterns. However, these methods typically require high computing resources and most employ centralized data processing, failing to fully utilize distributed computing resources. This limits their effectiveness in large-scale network environments.

[0010] Existing encrypted network traffic anomaly detection technologies have the following shortcomings:

[0011] 1. Low processing efficiency: Traditional encrypted traffic detection methods cannot fully utilize the distributed computing capabilities of edge computing, resulting in the need for a large amount of central server computing resources during traffic analysis and model training, which affects detection efficiency.

[0012] 2. Low detection accuracy: Existing detection methods based on traffic statistics and traditional machine learning methods often fail to effectively extract key features from encrypted traffic, resulting in low accuracy in detecting abnormal traffic. This is especially true for large-scale and complex traffic data, where limitations in feature extraction and model training make it difficult to guarantee detection accuracy.

[0013] 3. Distributed computing power is underutilized: Existing technologies mostly rely on centralized computing resources and fail to effectively utilize the distributed computing capabilities of edge computing devices. While edge devices can offload data processing and model training tasks, existing technologies don't fully exploit this advantage, leading to computing bottlenecks and inefficiencies.

[0014] 4. Insufficient computing resources: Although existing deep learning and machine learning-based methods have improved detection accuracy, the computing resources required for their training and inference processes are huge, making it difficult to achieve efficient real-time detection in large-scale edge device environments.

[0015] In summary, existing methods for detecting anomalies in encrypted network traffic suffer from shortcomings in detection accuracy, computational efficiency, and distributed computing capabilities. Existing methods often rely on simple traffic statistics, such as the number and length of traffic packets, failing to deeply explore the complex behavioral patterns within encrypted traffic, resulting in low anomaly detection accuracy. Furthermore, existing methods often require extensive centralized computing resources, failing to meet the real-time detection requirements of large-scale network environments. Furthermore, existing methods fail to effectively leverage the distributed computing capabilities of edge computing devices, resulting in computational bottlenecks and low efficiency. Summary of the Invention

[0016] In response to the above problems, the present disclosure provides a method, system, device and medium for detecting abnormal traffic in an encrypted network. Through multi-scale feature fusion and edge computing distributed processing, it aims to improve the accuracy and efficiency of encrypted traffic anomaly detection, fully utilize the computing power of edge devices, and thus solve the defects in the existing technology.

[0017] In a first aspect, a method for detecting abnormal traffic in an encrypted network is provided, the method comprising:

[0018] Each edge device in the edge computing environment independently collects encrypted network traffic data, which includes headers and payloads. Each edge device cleans the collected traffic data locally to obtain preprocessed data for feature extraction and model training.

[0019] Each edge device calculates the potential correlation between header and payload bytes locally based on pre-processed data through point-by-point mutual information (PMI), constructs a byte-level traffic graph, and obtains byte-level features.

[0020] Each edge device locally integrates the IP address information in the preprocessed data to construct a global interaction graph between IP hosts. Each node in the graph represents an IP host, and the edges between nodes represent the interaction relationship between hosts. The interaction graph is then feature extracted using a graph neural network (GNN) to generate global graph embedding features.

[0021] Each edge device locally combines the time dimension of the pre-processed data, applies a time series encoder to the pre-processed data containing time series information, extracts time dependencies, and generates a time series feature vector;

[0022] Each edge device locally performs multi-scale feature fusion on byte-level features, global graph embedding features, and time series features. This fusion is done by dynamically weighting each feature based on its importance, generating a high-dimensional, unified fusion feature. This fusion feature serves as the input for the local encrypted traffic classification model, also known as the local model.

[0023] Each edge device independently trains a local encrypted traffic classification model based on fused features. Each edge device uses local traffic data and its labels to train the model and calculate local model parameters. The training process is completely performed locally. After training is complete, each edge device uploads the local model parameters to the central server.

[0024] The central server receives local model parameters uploaded by each edge device through the federated learning framework and uses an aggregation algorithm to perform a weighted average of the local model parameters based on the contribution of each edge device to obtain the global model parameters. The contribution includes: the importance of the device in the network and the amount of local data of each edge device;

[0025] The central server sends the aggregated global model parameters to each edge device, and each edge device uses the global model parameters for the next round of local training. After each round of training, each edge device updates its local model parameters and uploads the updated local model parameters to the central server. Through repeated local training and model aggregation, the final global model is obtained.

[0026] The final global model is deployed on the central server for real-time detection of abnormal traffic in encrypted networks.

[0027] Furthermore, each edge device in the edge computing environment independently collects traffic data from the encrypted network. The traffic data includes headers and payloads. Each edge device cleans the collected traffic data locally to obtain preprocessed data for feature extraction and model training, including:

[0028] Each edge device only collects and processes data locally and does not transmit it to the central server.

[0029] Furthermore, each edge device in the edge computing environment independently collects traffic data from the encrypted network. The traffic data includes headers and payloads. Each edge device cleans the collected traffic data locally to obtain preprocessed data for feature extraction and model training, including:

[0030] Each edge device in the edge computing environment, including routers, cameras, and IoT gateways, collects normal traffic data and abnormal traffic data in simulated attack scenarios and stores them as pcap files; simulated attack scenarios include distributed denial of service attacks and data injection;

[0031] Perform feature extraction and data preprocessing on the pcap file, including extracting header information and payload byte sequence, cleaning the data and splitting it by time window, and finally generating a csv file containing labels.

[0032] Furthermore, each edge device cleans the collected traffic data locally, including:

[0033] Each edge device deduplicates the traffic data containing time series and removes duplicate traffic records;

[0034] Clean the traffic data, remove empty packets, and correct abnormal data through interpolation or averaging;

[0035] The traffic data is divided into fixed time windows to generate continuous time segments.

[0036] Furthermore, each edge device calculates the potential correlation between header and payload bytes locally based on preprocessed data using point-by-point mutual information (PMI), constructs a byte-level traffic graph, and obtains byte-level features, including:

[0037] Each edge device parses the header and payload byte sequences in the traffic data packet, treating each byte as a node to form an initial byte set;

[0038] Calculate the potential correlation between bytes and use the point-by-point mutual information (PMI) method to quantify the correlation strength of byte co-occurrence. The PMI calculation formula is:

[0039]

[0040] Where x and y represent two bytes or byte pairs in a data packet. PMI(x,y) represents the strength of the association between bytes x and y, that is, the joint probability of the byte pair x and y appearing simultaneously. P(x) and P(y) represent the marginal probabilities of byte x and byte y appearing independently, respectively. A larger PMI value indicates a higher probability that the two bytes appear simultaneously in the same data packet, indicating a stronger semantic or structural association.

[0041] Based on the PMI calculation results, the association strength of all nodes and edges is ranked, and several edges with higher PMI values ​​are retained to form a byte-level traffic graph that reflects the local association relationship in the traffic packet and obtain byte-level features.

[0042] Furthermore, each edge device locally integrates the IP address information in the preprocessed data to construct a global interaction graph between IP hosts. Each node in the graph represents an IP host, and the edges between nodes represent the interaction relationship between hosts. The interaction graph is then feature extracted using a graph neural network (GNN) to generate global graph embedding features, including:

[0043] Each edge device builds an interaction graph between IP hosts based on the IP addresses in the collected traffic data, where each node represents an IP host and the edges between nodes represent the interaction information between hosts;

[0044] The edges of the graph are established through the interaction relationship between the source IP and the target IP. The edge weights are represented by the interaction strength, forming a complete global interaction graph structure. The interaction strength includes: byte flow and packet exchange frequency.

[0045] In the graph neural network GNN, a multi-layer message passing mechanism is adopted, and each layer updates the node features through the neighborhood aggregation strategy. The specific formula is:

[0046]

[0047] in, represents the feature representation of node v at the k+1th layer, v is the target node, u is the neighbor node of node v, N(v) represents the set of neighbor nodes of node v; AGG(·) is the neighborhood feature aggregation function, which is used to integrate the feature information of node v and its neighbor nodes; W is the learnable weight matrix; σ(·) is the nonlinear activation function; Represents the feature representation of neighbor node u at the kth layer;

[0048] Through a multi-layer message passing mechanism, the local and global interaction characteristics of nodes in the global interaction graph are captured, and global graph embedding features are generated to characterize the global interaction patterns and structural information of traffic.

[0049] Furthermore, each edge device locally combines the time dimension of the preprocessed data, applies a time series encoder to the preprocessed data containing time series information, extracts time dependencies, and generates a time series feature vector, including:

[0050] Each edge device constructs a time series input, divides the traffic data into continuous time segments according to the time window, and arranges them in chronological order to form time series data;

[0051] Input the time series data into the time series encoder, which captures the dynamic changes and temporal dependencies of traffic characteristics in the time series. The time series encoder includes LSTM or Transformer.

[0052] The feature vector output by the encoder is extracted and used as a time series feature vector to characterize the temporal dynamic characteristics of the traffic.

[0053] Furthermore, each edge device locally performs multi-scale feature fusion on byte-level features, global graph embedding features, and time series features. This fusion is done by dynamically weighting each feature based on its importance, generating a high-dimensional unified fusion feature. This fusion feature serves as the input for the local encrypted traffic classification model, which is also known as the local model. This model includes:

[0054] Each edge device normalizes byte-level features, global graph embedding features, and time series features to ensure consistent dimensions across different features.

[0055] Based on the importance of byte-level features, global graph embedding features, and time series features in encrypted traffic detection, the fusion weight of each feature is calculated, and adaptive weight values ​​are assigned to different features.

[0056] The weighted byte-level features, global graph embedding features, and time series features are concatenated to generate high-dimensional unified fusion features to capture multi-scale information of traffic.

[0057] Furthermore, based on the importance of byte-level features, global graph embedding features, and time series features in encrypted traffic detection, the fusion weight of each feature is calculated, and weight values ​​are assigned to different features based on their adaptability, including:

[0058] In a dynamic weighting mechanism, the importance of byte-level features, global graph embedding features, and temporal features in encrypted traffic detection is evaluated. By calculating the impact of each feature on the loss function and using gradient information to evaluate the contribution of the feature, the degree of influence of the feature on the classification result is determined.

[0059] A dynamic weighting mechanism assigns a weight to each feature based on its importance. The weights are adjusted in real time during training based on model feedback. Feature importance is measured by gradient size; features with larger gradient changes receive higher weights, ensuring that features with greater impact on the classification task dominate the fusion process.

[0060] After determining the weight of each feature, the byte-level feature, global graph embedding feature, and temporal feature are multiplied by their corresponding weights and then weighted fused to generate the fused feature F final ; The specific weighted fusion calculation formula is:

[0061] F final=w1·F byte +w2·F graph +w3·F time

[0062] Among them, F byte 、F graph 、F time Represent byte-level features, global graph embedding features and temporal features respectively, w1, w2, w3 are their dynamic weights; the generated fusion feature F final Used as input to the local encrypted traffic classification model.

[0063] Furthermore, each edge device independently trains a local encrypted traffic classification model based on the fused features. Each edge device uses local traffic data and its labels to train the model and calculate local model parameters. The training process is performed entirely locally. After training is complete, each edge device uploads the local model parameters to the central server, including:

[0064] The generated fusion features are input into a local encrypted traffic classification model, and model parameters are adjusted through local training and optimization to minimize the classification loss function. The local encrypted traffic classification model includes: a deep neural network or other classification algorithm; local training and optimization of model parameters include: using Stochastic Gradient Descent, SGD or Adam optimizer; classification loss function includes: cross entropy loss;

[0065] After training is completed, each edge device uploads the updated local model parameters to the central server; only the local model parameters are uploaded, and no local data is transmitted; the local model parameters include weights and biases.

[0066] Furthermore, the central server sends the aggregated global model parameters to each edge device, and each edge device uses the global model parameters for the next round of local training. After each round of training, each edge device updates its local model parameters and uploads the updated local model parameters to the central server. Through repeated local training and model aggregation, a global model is obtained, including:

[0067] The central server sends the aggregated global model parameters to each edge device. Specifically, the central server uses the federated learning framework to aggregate the model parameters collected from each device by weighted average, generating global model parameters, and then passes these global model parameters to each edge device. Each edge device receives and loads these global model parameters as the initial state for the next round of local training; the local model parameters include weights and biases.

[0068] Edge devices use global model parameters for local training. In each round of training, edge devices independently train the encrypted traffic classification model using the received global model parameters and local fusion features. During local training, the device optimizes the model parameters based on local data to improve the model's performance in local tasks.

[0069] The edge device updates the local model parameters and uploads them to the central server. After each round of training, each edge device uploads the updated model parameters, including the weights and biases adjusted after training, to the central server; the central server then aggregates the local model parameters from each device to generate a new global model.

[0070] After multiple rounds of local training and global model aggregation, the final global model is deployed on the central server for real-time processing of new encrypted traffic data. During the testing phase, the new traffic data is extracted and fused before being input into the global model for inference. The global model outputs the traffic classification results based on the input traffic features and compares them with the true labels to evaluate the model's performance.

[0071] In a second aspect, a system for detecting abnormal traffic in an encrypted network includes:

[0072] A data acquisition unit, a first feature generation unit, a second feature generation unit, a third feature generation unit, a feature fusion unit, an edge training unit, a center weighting unit, an iteration unit, and a detection unit;

[0073] A data collection unit is used to independently collect encrypted network traffic data from each edge device in the edge computing environment. The traffic data includes headers and payloads. Each edge device cleans the collected traffic data locally to obtain pre-processed data for feature extraction and model training.

[0074] The first feature generation unit is used for each edge device to calculate the potential correlation between the header and payload bytes through point-by-point mutual information (PMI) based on pre-processed data, construct a byte-level traffic map, and obtain byte-level features;

[0075] The second feature generation unit is used by each edge device to locally integrate the IP address information in the preprocessed data to build a global interaction graph between IP hosts. Each node in the graph represents an IP host, and the edges between nodes represent the interaction relationship between hosts. The interaction graph is then extracted using a graph neural network (GNN) to generate global graph embedding features.

[0076] The third feature generation unit is configured for each edge device to locally combine the time dimension of the preprocessed data, apply a time series encoder to the preprocessed data containing time series information, extract time dependencies, and generate a time series feature vector;

[0077] The feature fusion unit is used by each edge device to locally perform multi-scale feature fusion on byte-level features, global graph embedding features, and time series features. This unit uses a dynamic weighting mechanism to calculate a weighted value based on the importance of each feature, generating a high-dimensional unified fusion feature. This fusion feature serves as the input for the local encrypted traffic classification model, also known as the local model.

[0078] The edge training unit is used for each edge device to independently train a local encrypted traffic classification model based on fused features. Each edge device uses local traffic data and its labels to train the model and calculate local model parameters. The training process is completely performed locally. After training is completed, each edge device uploads the local model parameters to the central server.

[0079] The central weighting unit is used by the central server to receive local model parameters uploaded by each edge device through the federated learning framework, and adopts an aggregation algorithm to perform a weighted average of the local model parameters based on the contribution of each edge device to obtain the global model parameters. The contribution includes: the importance of the device in the network and the amount of local data of each edge device;

[0080] The iteration unit is used by the central server to send the aggregated global model parameters to each edge device. Each edge device then uses the global model parameters for the next round of local training. After each round of training, each edge device updates its local model parameters and uploads the updated local model parameters to the central server. Through repeated local training and model aggregation, the final global model is obtained.

[0081] The detection unit is used to deploy the final global model on the central server for real-time detection of abnormal traffic in the encrypted network.

[0082] Furthermore, the data acquisition unit is specifically used to:

[0083] Each edge device only collects and processes data locally and does not transmit it to the central server.

[0084] Furthermore, the data acquisition unit is further specifically used for:

[0085] Each edge device in the edge computing environment, including routers, cameras, and IoT gateways, collects normal traffic data and abnormal traffic data in simulated attack scenarios and stores them as pcap files; simulated attack scenarios include distributed denial of service attacks and data injection;

[0086] Perform feature extraction and data preprocessing on the pcap file, including extracting header information and payload byte sequence, cleaning the data and splitting it by time window, and finally generating a csv file containing labels.

[0087] Furthermore, the data acquisition unit is further specifically used for:

[0088] Each edge device deduplicates the traffic data containing time series and removes duplicate traffic records;

[0089] Clean the traffic data, remove empty packets, and correct abnormal data through interpolation or averaging;

[0090] The traffic data is divided into fixed time windows to generate continuous time segments.

[0091] Furthermore, the first feature generating unit is specifically configured to:

[0092] Each edge device parses the header and payload byte sequences in the traffic data packet, treating each byte as a node to form an initial byte set;

[0093] Calculate the potential correlation between bytes and use the point-by-point mutual information (PMI) method to quantify the correlation strength of byte co-occurrence. The PMI calculation formula is:

[0094]

[0095] Where x and y represent two bytes or byte pairs in the data packet, PMI(x,y) represents the strength of association between byte x and byte y, that is, the joint probability that byte pair x and y appear at the same time, and P(x) and P(y) represent the marginal probabilities that byte x and byte y appear independently, respectively.

[0096] Based on the PMI calculation results, the association strength of all nodes and edges is ranked, and several edges with higher PMI values ​​are retained to form a byte-level traffic graph that reflects the local association relationship in the traffic packet and obtain byte-level features.

[0097] Furthermore, the second feature generating unit is specifically configured to:

[0098] Each edge device builds an interaction graph between IP hosts based on the IP addresses in the collected traffic data, where each node represents an IP host and the edges between nodes represent the interaction information between hosts;

[0099] The edges of the graph are established through the interaction relationship between the source IP and the target IP. The edge weights are represented by the interaction strength, forming a complete global interaction graph structure. The interaction strength includes: byte flow and packet exchange frequency.

[0100] In the graph neural network GNN, a multi-layer message passing mechanism is adopted, and each layer updates the node features through the neighborhood aggregation strategy. The specific formula is:

[0101]

[0102] in, represents the feature representation of node v at the k+1th layer, v is the target node, u is the neighbor node of node v, N(v) represents the set of neighbor nodes of node v; AGG(·) is the neighborhood feature aggregation function, which is used to integrate the feature information of node v and its neighbor nodes; W is the learnable weight matrix; σ(·) is the nonlinear activation function; Represents the feature representation of neighbor node u at the kth layer;

[0103] Through a multi-layer message passing mechanism, the local and global interaction characteristics of nodes in the global interaction graph are captured, and global graph embedding features are generated to characterize the global interaction patterns and structural information of traffic.

[0104] Furthermore, the third feature generating unit is specifically configured to:

[0105] Each edge device constructs a time series input, divides the traffic data into continuous time segments according to the time window, and arranges them in chronological order to form time series data;

[0106] Input the time series data into the time series encoder, which captures the dynamic changes and temporal dependencies of traffic characteristics in the time series. The time series encoder includes LSTM or Transformer.

[0107] The feature vector output by the encoder is extracted and used as a time series feature vector to characterize the temporal dynamic characteristics of the traffic.

[0108] Furthermore, the feature fusion unit is specifically used to:

[0109] Each edge device normalizes byte-level features, global graph embedding features, and time series features to ensure consistent dimensions across different features.

[0110] Based on the importance of byte-level features, global graph embedding features, and time series features in encrypted traffic detection, the fusion weight of each feature is calculated, and adaptive weight values ​​are assigned to different features.

[0111] The weighted byte-level features, global graph embedding features, and time series features are concatenated to generate high-dimensional unified fusion features to capture multi-scale information of traffic.

[0112] Furthermore, based on the importance of byte-level features, global graph embedding features, and time series features in encrypted traffic detection, the fusion weight of each feature is calculated, and weight values ​​are assigned to different features based on their adaptability, including:

[0113] In a dynamic weighting mechanism, the importance of byte-level features, global graph embedding features, and temporal features in encrypted traffic detection is evaluated. By calculating the impact of each feature on the loss function and using gradient information to evaluate the contribution of the feature, the degree of influence of the feature on the classification result is determined.

[0114] A dynamic weighting mechanism assigns a weight to each feature based on its importance. The weights are adjusted in real time during training based on model feedback. Feature importance is measured by gradient size; features with larger gradient changes receive higher weights, ensuring that features with greater impact on the classification task dominate the fusion process.

[0115] After determining the weight of each feature, the byte-level feature, global graph embedding feature, and temporal feature are multiplied by their corresponding weights and then weighted fused to generate the fused feature F final ; The specific weighted fusion calculation formula is:

[0116] F final =w1·F byte +w2·F graph +w3·F time

[0117] Among them, F byte 、F graph 、F time Represent byte-level features, global graph embedding features and temporal features respectively, w1, w2, w3 are their dynamic weights; the generated fusion feature F final Used as input to the local encrypted traffic classification model.

[0118] Furthermore, the edge training unit is specifically used to:

[0119] The generated fusion features are input into a local encrypted traffic classification model, and model parameters are adjusted through local training and optimization to minimize the classification loss function. The local encrypted traffic classification model includes: a deep neural network or other classification algorithm; local training and optimization of model parameters include: using Stochastic Gradient Descent, SGD or Adam optimizer; classification loss function includes: cross entropy loss;

[0120] After training is completed, each edge device uploads the updated local model parameters to the central server; only the local model parameters are uploaded, and no local data is transmitted; the local model parameters include weights and biases.

[0121] Furthermore, the iteration unit is specifically used to:

[0122] The central server sends the aggregated global model parameters to each edge device. Specifically, the central server uses the federated learning framework to aggregate the model parameters collected from each device by weighted average, generating global model parameters, and then passes these global model parameters to each edge device. Each edge device receives and loads these global model parameters as the initial state for the next round of local training; the local model parameters include weights and biases.

[0123] Edge devices use global model parameters for local training. In each round of training, edge devices independently train the encrypted traffic classification model using the received global model parameters and local fusion features. During local training, the device optimizes the model parameters based on local data to improve the model's performance in local tasks.

[0124] Edge devices update their local model parameters and upload them to the central server. After each round of training, each edge device uploads the updated model parameters, including the trained weights and biases, to the central server. The central server then aggregates the local model parameters from each device to generate a new global model. After multiple rounds of local training and global model aggregation, the final global model is generated.

[0125] Furthermore, the detection unit is specifically used to:

[0126] The final global model is deployed on a central server to process new encrypted traffic data in real time. During the testing phase, new traffic data is extracted and fused before being input into the final global model for inference. The final global model outputs traffic classification results based on the input traffic features and compares them with the true labels to evaluate the model's performance.

[0127] According to a third aspect, an electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0128] a memory storing a computer program;

[0129] The processor is used to implement the above-mentioned method for detecting abnormal traffic in an encrypted network when executing the computer program stored in the memory.

[0130] In a fourth aspect, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned method for detecting abnormal traffic in an encrypted network.

[0131] The present disclosure has at least the following beneficial effects:

[0132] By constructing a byte-level traffic graph and adopting an improved graph neural network (GNN), this paper can effectively capture the potential correlation between headers and payloads in encrypted network traffic, fully utilize the local and global characteristics of the traffic, and improve the accuracy of encrypted traffic anomaly detection.

[0133] This paper adopts a multi-scale feature fusion method to organically combine byte-level features, global graph embedding features and time dynamic features, significantly improves the expressiveness of features through a dynamic weighting mechanism, and effectively adapts to complex traffic scenarios.

[0134] The present disclosure improves the efficiency and accuracy of encrypted network traffic anomaly detection by combining edge computing and federated learning methods. The edge device independently trains the local model and uploads it to the central server. The central server aggregates the model parameters through federated learning to generate a global detection model. This method can perform real-time analysis closer to the data source, reducing the delay and bandwidth consumption caused by data transmission. At the same time, through local training and model aggregation, the centralized storage and transmission of sensitive data is avoided, effectively protecting data privacy. After multiple rounds of optimization, the global model can accurately identify abnormal behavior in encrypted traffic, improve network security protection capabilities, and provide an efficient, accurate and privacy-protected network security solution.

[0135] Other features and advantages of the present disclosure will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present disclosure. The purpose and other advantages of the present disclosure can be achieved and obtained through the structures indicated in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0136] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0137] Figure 1 This is a flow chart of the detection method according to an embodiment of the present disclosure;

[0138] Figure 2 This is a schematic diagram of the detection system architecture according to an embodiment of the present disclosure;

[0139] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0140] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0141] like Figure 1 A method for detecting abnormal traffic in an encrypted network is shown, the method comprising:

[0142] In S101, each edge device in the edge computing environment independently collects traffic data from the encrypted network. The traffic data includes a header and a payload. Each edge device cleans the collected traffic data locally to obtain preprocessed data for feature extraction and model training.

[0143] S102: Each edge device calculates the potential correlation between header and payload bytes based on the pre-processed data locally using point-by-point mutual information (PMI), constructs a byte-level traffic graph, and obtains byte-level features.

[0144] In step S103, each edge device locally integrates the IP address information in the preprocessed data to construct a global interaction graph between the IP hosts. Each node in the graph represents an IP host, and the edges between the nodes represent the interaction relationship between the hosts. The interaction graph is then feature extracted using a graph neural network (GNN) to generate global graph embedding features.

[0145] S104, each edge device locally combines the time dimension of the pre-processed data, applies a time series encoder to the pre-processed data containing time series information, extracts time dependencies, and generates a time series feature vector;

[0146] At S105, each edge device locally performs multi-scale feature fusion on byte-level features, global graph embedding features, and time series features. This fusion is performed using a dynamic weighting mechanism to calculate a weighted value based on the importance of each feature. This generates a high-dimensional unified fusion feature, which serves as the input to the local encrypted traffic classification model, also known as the local model.

[0147] In step S106, each edge device independently trains a local encrypted traffic classification model based on the fused features. Each edge device uses local traffic data and its labels to train the model and calculate local model parameters. The training process is performed entirely locally. After the training is completed, each edge device uploads the local model parameters to the central server.

[0148] S107: The central server receives the local model parameters uploaded by each edge device through the federated learning framework and uses an aggregation algorithm to perform a weighted average of the local model parameters based on the contribution of each edge device to obtain the global model parameters. The contribution includes the importance of the device in the network and the amount of local data of each edge device.

[0149] S108: The central server sends the aggregated global model parameters to each edge device. Each edge device uses the global model parameters for the next round of local training. After each round of training, each edge device updates its local model parameters and uploads the updated local model parameters to the central server. Through repeated local training and model aggregation, the final global model is obtained.

[0150] S109, deploying the final global model on the central server for real-time detection of abnormal traffic in the encrypted network.

[0151] When implementing it specifically, Figure 1 As shown, the introduction is as follows:

[0152] The present disclosure can make full use of the distributed computing capabilities of edge computing devices, and by distributing data processing and model training tasks to multiple edge devices, it can improve processing efficiency and reduce the computing burden of the central server. By combining byte-level features, global interaction features, and time series features, the accuracy of encrypted traffic anomaly detection is improved. The federated learning framework enables each edge device to independently train the traffic classification model locally, and upload the model parameters to the central server for aggregation to generate a global model. In this way, the computing bottleneck in the traditional centralized method is avoided, and the detection efficiency of the overall system is improved.

[0153] In step S101, each device in the edge computing environment (such as routers, cameras, IoT gateways, etc.) independently collects encrypted network traffic data, including headers and payloads. Each device extracts key fields (such as IP address, port, protocol type, byte sequence, etc.) and sorts the data to generate time-series traffic data. During this step, each device collects and processes data locally, avoiding the transmission of sensitive data to a central server.

[0154] Each edge device independently preprocesses the collected traffic data. This includes operations such as data deduplication, cleaning, and time windowing to remove noise and ensure data quality. Each device performs preprocessing locally to ensure high-quality traffic data, which is subsequently used for feature extraction and model training.

[0155] In step S102, each edge device uses point-wise mutual information (PMI) based on its locally collected traffic data to calculate the potential correlation between header and payload bytes and construct a byte-level traffic map. This process is performed independently on each edge device to extract local byte-level features that provide foundational features for subsequent traffic classification models.

[0156] At step S103, each edge device integrates the IP address information from the collected traffic data to construct an interaction graph between IP hosts. Each node in the graph represents an IP host, and the edges between nodes represent the interaction relationships between hosts. A graph neural network (GNN) is then used to extract features from the interaction graph and generate global graph embedding features. Each device independently extracts graph features locally and stores the global interaction structure information associated with the local data.

[0157] In step S104, each edge device applies a time series encoder (such as an LSTM or Transformer) to the collected traffic data, which contains time series information, to extract temporal dependencies and generate a time series feature vector. This process is performed independently on each edge device, capturing the dynamic characteristics of local traffic.

[0158] At step S105, each edge device performs multi-scale feature fusion on byte-level features, global graph embedding features, and time series features. A dynamic weighting mechanism calculates weights based on the importance of each feature, generating a high-dimensional unified feature representation. This serves as input to the local encrypted traffic classification model, also known as the local model. Each device independently performs feature fusion and generates the final fused features, providing data support for model training.

[0159] In step S106, each edge device independently trains an encrypted traffic classification model based on the generated fusion features. Each device uses local traffic data and its labels to train the model and calculate local model parameters. The training process is entirely local, avoiding the transmission of sensitive data. After training is complete, each device uploads the model parameters to the central server.

[0160] In step S107, the central server receives the local model parameters uploaded by each edge device through the federated learning framework and uses an aggregation algorithm (such as FedAvg) to perform a weighted average of these parameters. The aggregation process weights the parameters based on the device's contribution, which includes the device's importance in the network and the amount of local data each device has.

[0161] In S108, the central server distributes the aggregated global model parameters to each edge device, which then uses them for the next round of local training. After each round of training, each edge device updates its local model parameters and uploads the updated parameters to the central server. Through repeated local training and model aggregation, the federated learning framework continuously optimizes the global model, thereby improving the accuracy and efficiency of encrypted traffic anomaly detection.

[0162] By collecting traffic data and extracting multi-scale features, a global detection model is generated using distributed collaboration and federated learning across edge devices. This multi-scale feature fusion method, combined with a federated distributed collaboration algorithm, improves detection efficiency and accuracy, enabling efficient classification and anomaly detection of encrypted traffic, providing a precise solution for network traffic security on edge devices.

[0163] S101 processes the raw network traffic data of edge devices into sample data that meets the requirements of deep learning. This helps improve model training and optimize detection performance. Data deduplication, cleaning, and time segmentation help improve data quality and preserve time series features. This process includes the following steps:

[0164] For each device in the edge computing environment (such as routers, cameras, IoT gateways, etc.), collect normal traffic data and abnormal traffic data in simulated attack scenarios (such as distributed denial of service (DDoS) attacks, data injection, etc.), and store them as pcap files;

[0165] Perform feature extraction and data preprocessing on the pcap file, including extracting header information and payload byte sequence, calculating the potential correlation between bytes to generate a byte-level relationship matrix, cleaning the data and splitting it by time window, and finally generating a CSV file containing labels.

[0166] Each edge device deduplicates the traffic data containing time series, removes duplicate traffic records, and ensures the uniqueness of the data;

[0167] Clean traffic data, remove invalid data packets (such as empty data packets) and correct outliers to ensure data quality;

[0168] The traffic data is divided into fixed time windows to generate continuous time segments, providing structured input data for subsequent feature extraction.

[0169] S102 includes the following steps:

[0170] Each edge device parses the header and payload byte sequences in the traffic data packet, treating each byte as a node to form an initial byte set;

[0171] Calculate the potential correlation between bytes and use the point-wise mutual information (PMI) method to quantify the correlation strength of byte co-occurrence. The PMI calculation formula is:

[0172]

[0173] Where x and y represent two bytes or byte pairs in the data packet. P(x,y) is the joint probability of the byte pair x and y occurring simultaneously, and P(x) and P(y) are the marginal probabilities of the byte x and byte y occurring independently, respectively.

[0174] A byte-level traffic graph is constructed based on the PMI (point-wise mutual information) calculation results.

[0175] Through the local features of byte-level traffic graphs, the implicit patterns of header and payload bytes in traffic packets are captured, revealing the association between key bytes in traffic packets.

[0176] S102, by parsing traffic data packets and constructing byte-level traffic graphs, helps to quantify the correlation between bytes and capture the implicit patterns and correlation features of key bytes in traffic packets.

[0177] Among them, a byte-level traffic graph is constructed based on the results of PMI (point-wise mutual information) calculation. By constructing a byte-level traffic graph, the ability to represent the local characteristics and association patterns of key bytes is improved. Specifically, the following are included:

[0178] The bytes in the header and payload are treated as nodes in the traffic graph, and the initial state of each node is defined as the independent characteristics of the bytes;

[0179] Based on the byte association relationship calculated by the PMI value, edges are established between nodes, and the PMI value is used as the weight of the edge to quantify the association strength between nodes;

[0180] The associations of all nodes and edges are screened, and edges with higher PMI values ​​are retained to form a byte-level traffic graph that reflects the local association relationships in the traffic packet, which is used to represent the local characteristics between key bytes in the traffic packet.

[0181] S103 includes the following steps:

[0182] Each edge device builds an interaction graph between IP hosts based on global interaction information such as IP addresses in the collected traffic data. Each node represents an IP host, and the edges between nodes represent the interaction information between hosts.

[0183] The edges of the graph are established through the interaction relationship between the source IP and the target IP. The edge weight is represented by the interaction intensity (such as byte flow and packet exchange frequency), forming a complete global interaction graph structure.

[0184] In the graph neural network (GNN), a multi-layer message passing mechanism is adopted, and each layer updates the node features through the neighborhood aggregation strategy. The specific formula is:

[0185]

[0186] Where AGG is the aggregation function, N(v) is the neighborhood of node v, σ is the activation function, and W is the weight matrix;

[0187] Through a multi-layer message passing mechanism, the local and global interaction characteristics of nodes in the global interaction graph are captured, and global graph embedding features are generated to characterize the global interaction pattern and structural information of the traffic.

[0188] S103, by constructing a global interaction graph and applying graph neural networks, helps capture the global interaction patterns and structural information in traffic data, further improves the ability to express the interaction characteristics and global behavior patterns between nodes, and provides a more comprehensive feature representation for subsequent model training.

[0189] S104 includes the following steps:

[0190] Each edge device constructs a time series input, divides the traffic data into continuous time segments according to the time window, and arranges them in chronological order to form time series data;

[0191] Input the time series data into a time series encoder (such as LSTM or Transformer) to capture the dynamic changes and temporal dependencies of traffic features in the time series through the encoder;

[0192] The feature vector output by the encoder is extracted and used as a time series feature vector to characterize the temporal dynamic characteristics of the traffic.

[0193] S104, by constructing time series input and utilizing a time series encoder, effectively captures the dynamic changes and time dependencies of traffic characteristics, and improves the model's ability to model the temporal dynamic characteristics of traffic.

[0194] S105 includes the following steps:

[0195] Each edge device standardizes byte-level features, global graph embedding features, and time series features to ensure dimensional consistency between different features.

[0196] Through a dynamic weighting mechanism, which automatically calculates the fusion weight of each feature based on the importance of byte-level features, global graph embedding features, and time series features in encrypted traffic detection, and assigns adaptive weight values ​​to different features;

[0197] The weighted byte-level features, global graph embedding features, and temporal features are concatenated to generate a high-dimensional unified feature representation to capture multi-scale information of traffic.

[0198] S105, through feature normalization, dynamic weighting and feature splicing, fully considers multi-scale information such as byte level, global interaction and timing, generates a unified multi-scale feature vector, and comprehensively characterizes the multi-level characteristics of traffic.

[0199] Among them, through the dynamic weighting mechanism, this mechanism refers to automatically calculating the fusion weight of each feature based on the importance of byte-level features, global graph embedding features, and time series features in encrypted traffic detection, and assigning adaptive weight values ​​to different features. This allows the model to strengthen the influence of key features during the feature fusion process and optimize the effect of encrypted traffic anomaly detection. Specifically, it includes:

[0200] In the dynamic weighting mechanism, the importance of byte-level features, global graph embedding features, and temporal features in encrypted traffic detection is first evaluated. By calculating the impact of each feature on the loss function and using gradient information to evaluate the feature contribution, we can determine which features have a greater impact on the classification results and which have a smaller impact.

[0201] A dynamic weighting mechanism assigns a weight to each feature based on its importance. This weight is adjusted in real time during training based on model feedback. Feature importance is measured by the magnitude of the gradient; features with larger gradient changes receive higher weights, ensuring that features with greater impact on the classification task dominate the fusion process.

[0202] After determining the weight of each feature, the byte-level features, global graph embedding features, and temporal features are multiplied by their corresponding weights and then weighted fused to generate the final feature representation. The specific weighted fusion calculation formula is:

[0203] F final =w1·F byte +w2·F graph +w3·F time

[0204] Among them, F byte 、F graph 、F time Represent byte-level features, global graph embedding features and temporal features respectively, w1, w2, w3 are their dynamic weights. The final fusion feature F generated finalUsed as input to the encrypted traffic classification model for subsequent local training.

[0205] S106 includes the following steps:

[0206] On each device, fusion feature F is generated through feature extraction and multi-scale feature fusion operations. final , which is used as the input of the encrypted traffic classification model for subsequent local training.

[0207] The generated fusion features are input into the encrypted traffic classification model (such as a deep neural network or other classification algorithms), and the model parameters are adjusted through local training and optimization (such as using Stochastic Gradient Descent, SGD or Adam optimizer) to minimize the classification loss function (such as cross entropy loss), thereby improving the classification accuracy of the model on local data.

[0208] After training is complete, each device uploads the updated model parameters (such as weights and biases) to the central server. To ensure data privacy, only the model parameters are uploaded, and no local data is transmitted.

[0209] S107 includes the following steps:

[0210] The central server distributes the aggregated global model parameters to each edge device. Specifically, the central server uses the federated learning framework to aggregate the model parameters (such as weights and biases) collected from each device, generating global model parameters. These global model parameters are then distributed to each edge device. Each edge device receives and loads these global model parameters, using them as the initial state for the next round of local training.

[0211] Edge devices use global model parameters for local training. In each round of training, edge devices independently train the encrypted traffic classification model using the received global model parameters and local traffic data (such as fused features). During local training, the device optimizes model parameters (such as weights and biases) based on local data to improve the model's performance in the local task.

[0212] Edge devices update their local model parameters and upload them to the central server. After each round of training, each edge device uploads the updated model parameters (including trained and adjusted weights and biases) to the central server. The central server then aggregates the local model parameters from each device to generate a new global model.

[0213] After multiple rounds of local training and global model aggregation, the resulting global model is deployed on a central server to process new encrypted traffic data in real time. During the testing phase, new traffic data undergoes feature extraction and fusion processing before being fed into the global model for inference. Based on the input traffic features, the global model outputs a traffic classification result (such as "malicious" or "normal") and compares it with the ground-truth label to evaluate the model's performance.

[0214] S107, through multiple rounds of local training and global model aggregation, edge devices independently train models and upload updated parameters to generate a global model. This process combines distributed computing and federated learning technologies, protecting data privacy while improving collaboration efficiency. Decentralized training and parameter aggregation not only reduces data transmission latency and bandwidth consumption, but also optimizes the global model's detection capabilities, ultimately achieving efficient identification of malicious encrypted traffic.

[0215] This paper adopts a multi-scale feature fusion method, which combines byte-level features, global interaction features and time series features to perform multi-scale feature fusion, effectively improving the accuracy of encrypted traffic anomaly detection and fully capturing the complex behavior patterns in encrypted traffic.

[0216] The present disclosure is based on distributed traffic processing of edge computing. The present disclosure fully utilizes the distributed computing capabilities of edge computing devices to distribute traffic data processing and model training tasks to multiple edge devices, thereby reducing computing resource requirements, improving processing efficiency, and reducing the computing pressure of the central server.

[0217] This paper adopts a federated learning framework, which enables each edge device to independently train the traffic classification model locally and upload the model parameters to the central server for aggregation, thereby avoiding the bottleneck of centralized computing and improving the scalability and real-time performance of the detection system.

[0218] This paper adopts a dynamic weighted feature fusion mechanism, which performs weighted fusion of different features (byte-level features, global graph embedding features, and time series features) through a dynamic weighting mechanism to ensure that the fused features are more representative and adaptable, thereby improving the detection accuracy of the model.

[0219] The present invention adopts byte-level traffic map construction and feature extraction, constructs a byte-level traffic map through point-by-point mutual information (PMI) calculation, extracts local features of traffic at the byte level, captures the potential correlation between header and payload bytes in traffic packets, and enhances the ability to capture traffic details.

[0220] The present invention performs global interaction feature extraction based on graph neural network (GNN): adopts graph neural network (GNN) to process the global interaction graph, generates global graph embedding features through message passing mechanism, which can effectively characterize the global interaction pattern of traffic and improve the global traffic analysis capability.

[0221] The present disclosure adopts a time series encoder for application in traffic behavior analysis, uses a time series encoder (such as LSTM or Transformer) to model the time dependency of traffic, extracts time series features, and enhances the ability to capture dynamic changes in traffic.

[0222] This paper combines efficient local training with model aggregation. Edge devices train traffic classification models locally and upload model parameters to the central server. Combined with the aggregation method of federated learning, it ensures efficient generation of global models in a distributed environment.

[0223] Through these technological innovations, the shortcomings of existing encrypted traffic detection methods in terms of accuracy, efficiency, computing resources and distributed computing power utilization are solved, providing an efficient, accurate and scalable encrypted traffic anomaly detection solution.

[0224] like Figure 2 As shown, an encrypted network abnormal traffic detection system includes:

[0225] Data collection unit 201, first feature generation unit 202, second feature generation unit 203, third feature generation unit 204, feature fusion unit 205, edge training unit 206, center weighting unit 207, iteration unit 208 and detection unit 209;

[0226] The data collection unit 201 is configured to independently collect traffic data of the encrypted network from each edge device in the edge computing environment. The traffic data includes a header and a payload. Each edge device cleans the collected traffic data locally to obtain pre-processed data for feature extraction and model training.

[0227] The first feature generation unit 202 is configured for each edge device to calculate the potential correlation between header and payload bytes based on pre-processed data locally by point-by-point mutual information (PMI), construct a byte-level traffic graph, and obtain byte-level features;

[0228] The second feature generation unit 203 is configured to locally integrate the IP address information in the pre-processed data on each edge device to construct a global interaction graph between the IP hosts. Each node in the graph represents an IP host, and the edges between the nodes represent the interaction relationship between the hosts. The interaction graph is then subjected to feature extraction using a graph neural network (GNN) to generate a global graph embedding feature.

[0229] The third feature generation unit 204 is configured for each edge device to locally combine the time dimension of the pre-processed data, apply a time series encoder to the pre-processed data containing time series information, extract the time dependency, and generate a time series feature vector;

[0230] The feature fusion unit 205 is configured to locally fuse byte-level features, global graph embedding features, and time series features on each edge device. This unit calculates a weighted value based on the importance of each feature through a dynamic weighting mechanism to generate a high-dimensional unified fusion feature, which serves as input to the local encrypted traffic classification model, also known as the local model.

[0231] The edge training unit 206 is configured to independently train a local encrypted traffic classification model based on the fusion features on each edge device. Each edge device uses local traffic data and its labels to train the model and calculate local model parameters. The training process is performed entirely locally. After training is completed, each edge device uploads the local model parameters to the central server.

[0232] The central weighting unit 207 is used for the central server to receive the local model parameters uploaded by each edge device through the federated learning framework, and use an aggregation algorithm to perform a weighted average of the local model parameters based on the contribution of each edge device to obtain the global model parameters. The contribution includes: the importance of the device in the network and the amount of local data of each edge device;

[0233] Iteration unit 208 is used for the central server to send the aggregated global model parameters to each edge device. Each edge device uses the global model parameters to perform the next round of local training. After each round of training, each edge device updates its local model parameters and uploads the updated local model parameters to the central server. Through repeated local training and model aggregation, the final global model is obtained.

[0234] The detection unit 209 is used to deploy the final global model on the central server for real-time detection of abnormal traffic in the encrypted network.

[0235] This method collects and extracts encrypted network traffic data in real time and generates unified feature data using multi-scale feature fusion. On edge devices, a federated learning framework is used to independently train local models, and the model parameters are uploaded to a central server for aggregation to generate a global detection model. Finally, training is performed using an optimized machine learning algorithm, and test data is classified and evaluated. This method effectively improves the efficiency and accuracy of encrypted network traffic anomaly detection, providing an efficient and accurate solution for network security.

[0236] In specific implementation, the present invention discloses an encrypted network abnormal traffic detection system and an encrypted network abnormal traffic detection method implementation process that corresponds one to one, which will not be described in detail here.

[0237] like Figure 3 As shown, the present disclosure provides an electronic device, including a processor 301, a communication interface 302, a memory 303 and a communication bus 304, wherein the processor 301, the communication interface 302 and the memory 303 communicate with each other through the communication bus 304;

[0238] Memory 303, storing computer programs;

[0239] The processor 301 is configured to implement the above method when executing the computer program stored in the memory 303 .

[0240] The present disclosure provides a computer-readable storage medium storing a computer program, which implements the above method when executed by a processor.

[0241] The computer-readable storage medium may be included in the device / apparatus described in the above embodiments, or may exist independently without being incorporated into the device / apparatus. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present disclosure.

[0242] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as, but not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0243] Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for detecting abnormal traffic in an encrypted network, characterized in that: The method comprises: Each edge device in the edge computing environment independently collects encrypted network traffic data, which includes headers and payloads. Each edge device cleans the collected traffic data locally to obtain preprocessed data for feature extraction and model training. Each edge device calculates the potential correlation between header and payload bytes locally based on pre-processed data through point-by-point mutual information (PMI), constructs a byte-level traffic graph, and obtains byte-level features. Each edge device locally integrates the IP address information in the preprocessed data to construct a global interaction graph between IP hosts. Each node in the graph represents an IP host, and the edges between nodes represent the interaction relationship between hosts. The interaction graph is then feature extracted using a graph neural network (GNN) to generate global graph embedding features. Each edge device locally combines the time dimension of the pre-processed data, applies a time series encoder to the pre-processed data containing time series information, extracts time dependencies, and generates a time series feature vector; Each edge device locally performs multi-scale feature fusion on byte-level features, global graph embedding features, and time series features. This fusion is done by dynamically weighting each feature based on its importance, generating a high-dimensional, unified fusion feature. This fusion feature serves as the input for the local encrypted traffic classification model, also known as the local model. Each edge device independently trains a local encrypted traffic classification model based on fused features. Each edge device uses local traffic data and its labels to train the model and calculate local model parameters. The training process is completely performed locally. After training is complete, each edge device uploads the local model parameters to the central server. The central server receives local model parameters uploaded by each edge device through the federated learning framework and uses an aggregation algorithm to perform a weighted average of the local model parameters based on the contribution of each edge device to obtain the global model parameters. The contribution includes: the importance of the device in the network and the amount of local data of each edge device; The central server sends the aggregated global model parameters to each edge device, and each edge device uses the global model parameters for the next round of local training. After each round of training, each edge device updates its local model parameters and uploads the updated local model parameters to the central server. Through repeated local training and model aggregation, the final global model is obtained. The final global model is deployed on the central server for real-time detection of abnormal traffic in encrypted networks.

2. The method for detecting abnormal traffic in an encrypted network according to claim 1, wherein: In an edge computing environment, each edge device independently collects encrypted network traffic data, which includes headers and payloads. Each edge device cleans the collected traffic data locally to obtain preprocessed data for feature extraction and model training, including: Each edge device only collects and processes data locally and does not transmit it to the central server.

3. The method for detecting abnormal traffic in an encrypted network according to claim 1, wherein: Each edge device in the edge computing environment independently collects traffic data of the encrypted network, which includes headers and payloads; Each edge device cleans the collected traffic data locally to obtain preprocessed data for feature extraction and model training, including: Each edge device in the edge computing environment, including routers, cameras, and IoT gateways, collects normal traffic data and abnormal traffic data in simulated attack scenarios and stores them as pcap files; simulated attack scenarios include distributed denial of service attacks and data injection; Perform feature extraction and data preprocessing on the pcap file, including extracting header information and payload byte sequence, cleaning the data and splitting it by time window, and finally generating a csv file containing labels.

4. The method for detecting abnormal traffic in an encrypted network according to claim 1, wherein: Each edge device cleans the collected traffic data locally, including: Each edge device deduplicates the traffic data containing time series and removes duplicate traffic records; Clean the traffic data, remove empty packets, and correct abnormal data through interpolation or averaging; The traffic data is divided into fixed time windows to generate continuous time segments.

5. The method for detecting abnormal traffic in an encrypted network according to claim 1, wherein: Each edge device calculates the potential correlation between header and payload bytes using point-by-point mutual information (PMI) based on pre-processed data, constructs a byte-level traffic graph, and obtains byte-level features, including: Each edge device parses the header and payload byte sequences in the traffic data packet, treating each byte as a node to form an initial byte set; Calculate the potential correlation between bytes and use the point-by-point mutual information (PMI) method to quantify the correlation strength of byte co-occurrence. The PMI calculation formula is: Where x and y represent two bytes or byte pairs in the data packet, PMI(x,y) represents the strength of association between byte x and byte y, that is, the joint probability that byte pair x and y appear at the same time, and P(x) and P(y) represent the marginal probabilities that byte x and byte y appear independently, respectively. Based on the PMI calculation results, the association strength of all nodes and edges is ranked, and several edges with higher PMI values ​​are retained to form a byte-level traffic graph that reflects the local association relationship in the traffic packet and obtain byte-level features.

6. The method for detecting abnormal traffic in an encrypted network according to claim 1, wherein: Each edge device locally integrates the IP address information in the pre-processed data to build a global interaction graph between IP hosts. Each node in the graph represents an IP host, and the edges between nodes represent the interaction relationship between hosts. The interaction graph is extracted through the graph neural network (GNN) to generate global graph embedding features, including: Each edge device builds an interaction graph between IP hosts based on the IP addresses in the collected traffic data, where each node represents an IP host and the edges between nodes represent the interaction information between hosts; The edges of the graph are established through the interaction relationship between the source IP and the target IP. The edge weights are represented by the interaction strength, forming a complete global interaction graph structure. The interaction strength includes: byte flow and packet exchange frequency. In the graph neural network GNN, a multi-layer message passing mechanism is adopted, and each layer updates the node features through the neighborhood aggregation strategy. The specific formula is: in, represents the feature representation of node v at the k+1th layer, v is the target node, u is the neighbor node of node v, N(v) represents the set of neighbor nodes of node v; AGG(·) is the neighborhood feature aggregation function, which is used to integrate the feature information of node v and its neighbor nodes; W is the learnable weight matrix; σ(·) is the nonlinear activation function; Represents the feature representation of neighbor node u at the kth layer; Through a multi-layer message passing mechanism, the local and global interaction characteristics of nodes in the global interaction graph are captured, and global graph embedding features are generated to characterize the global interaction patterns and structural information of traffic.

7. The method for detecting abnormal traffic in an encrypted network according to claim 1, wherein: Each edge device locally combines the time dimension of the preprocessed data, applies a time series encoder to the preprocessed data containing time series information, extracts time dependencies, and generates a time series feature vector, including: Each edge device constructs a time series input, divides the traffic data into continuous time segments according to the time window, and arranges them in chronological order to form time series data; Input the time series data into the time series encoder, which captures the dynamic changes and temporal dependencies of traffic characteristics in the time series. The time series encoder includes LSTM or Transformer. The feature vector output by the encoder is extracted and used as a time series feature vector to characterize the temporal dynamic characteristics of the traffic.

8. The method for detecting abnormal traffic in an encrypted network according to claim 1, wherein: Each edge device locally performs multi-scale feature fusion on byte-level features, global graph embedding features, and time series features. This fusion is done by dynamically weighting each feature based on its importance, generating a high-dimensional unified fusion feature. This fusion feature serves as the input for the local encrypted traffic classification model, which is also known as the local model. This model includes: Each edge device normalizes byte-level features, global graph embedding features, and time series features to ensure consistent dimensions across different features. Based on the importance of byte-level features, global graph embedding features, and time series features in encrypted traffic detection, the fusion weight of each feature is calculated, and adaptive weight values ​​are assigned to different features. The weighted byte-level features, global graph embedding features, and time series features are concatenated to generate high-dimensional unified fusion features to capture multi-scale information of traffic.

9. The method for detecting abnormal traffic in an encrypted network according to claim 8, wherein: Based on the importance of byte-level features, global graph embedding features, and time series features in encrypted traffic detection, the fusion weight of each feature is calculated, and weight values ​​are assigned to different features based on their adaptability, including: In a dynamic weighting mechanism, the importance of byte-level features, global graph embedding features, and temporal features in encrypted traffic detection is evaluated. By calculating the impact of each feature on the loss function and using gradient information to evaluate the contribution of the feature, the degree of influence of the feature on the classification result is determined. A dynamic weighting mechanism assigns a weight to each feature based on its importance. The weights are adjusted in real time during training based on model feedback. Feature importance is measured by gradient size; features with larger gradient changes receive higher weights, ensuring that features with greater impact on the classification task dominate the fusion process. After determining the weight of each feature, the byte-level feature, global graph embedding feature, and temporal feature are multiplied by their corresponding weights and then weighted fused to generate the fused feature F final ; The specific weighted fusion calculation formula is: F final =w1·F byte +w2·F graph +w3·F time Among them, F byte 、F graph 、F time Represent byte-level features, global graph embedding features and temporal features respectively, w1, w2, w3 are their dynamic weights; the generated fusion feature F final Used as input to the local encrypted traffic classification model.

10. The method for detecting abnormal traffic in an encrypted network according to claim 1, wherein: Each edge device independently trains a local encrypted traffic classification model based on fused features. Each edge device uses local traffic data and its labels to train the model and calculate local model parameters. The training process is completely performed locally; After training is complete, each edge device uploads the local model parameters to the central server, including: The generated fusion features are input into a local encrypted traffic classification model, and the model parameters are adjusted through local training and optimization to minimize the classification loss function. The local encrypted traffic classification model includes: a deep neural network or other classification algorithm; the local training and optimization of model parameters include: using Stochastic Gradient Descent, SGD or Adam optimizer; the classification loss function includes: cross entropy loss; After training is completed, each edge device uploads the updated local model parameters to the central server; only the local model parameters are uploaded, and no local data is transmitted; the local model parameters include weights and biases.

11. The method for detecting abnormal traffic in an encrypted network according to claim 1, wherein: The central server sends the aggregated global model parameters to each edge device, and each edge device uses the global model parameters for the next round of local training. After each round of training, each edge device updates its local model parameters and uploads the updated local model parameters to the central server. Through repeated local training and model aggregation, a global model is obtained, including: The central server sends the aggregated global model parameters to each edge device. Specifically, the central server uses the federated learning framework to perform a weighted average aggregation of the model parameters collected from each device to generate global model parameters, which are then passed to each edge device. Each edge device receives and loads these global model parameters as the initial state for the next round of local training. Local model parameters include weights and biases. Edge devices use global model parameters for local training. In each round of training, edge devices use the received global model parameters and combine them with local fusion features to independently train the encrypted traffic classification model. During local training, the device optimizes the model parameters based on local data to improve the model's performance in local tasks. The edge device updates the local model parameters and uploads them to the central server. After each round of training, each edge device uploads the updated model parameters, including the trained weights and biases, to the central server. The central server then aggregates the local model parameters from each device to generate a new global model. After multiple rounds of local training and global model aggregation, the final global model is deployed on the central server for real-time processing of new encrypted traffic data. During the testing phase, the new traffic data is extracted and fused before being input into the global model for inference. The global model outputs the traffic classification results based on the input traffic features and compares them with the true labels to evaluate the model's performance.

12. An encrypted network abnormal traffic detection system, characterized in that: include: A data acquisition unit, a first feature generation unit, a second feature generation unit, a third feature generation unit, a feature fusion unit, an edge training unit, a center weighting unit, an iteration unit, and a detection unit; A data collection unit is used to independently collect encrypted network traffic data from each edge device in the edge computing environment. The traffic data includes headers and payloads. Each edge device cleans the collected traffic data locally to obtain pre-processed data for feature extraction and model training. The first feature generation unit is used for each edge device to calculate the potential correlation between the header and payload bytes through point-by-point mutual information (PMI) based on pre-processed data, construct a byte-level traffic map, and obtain byte-level features; The second feature generation unit is used by each edge device to locally integrate the IP address information in the preprocessed data to build a global interaction graph between IP hosts. Each node in the graph represents an IP host, and the edges between nodes represent the interaction relationship between hosts. The interaction graph is then extracted using a graph neural network (GNN) to generate global graph embedding features. The third feature generation unit is configured for each edge device to locally combine the time dimension of the preprocessed data, apply a time series encoder to the preprocessed data containing time series information, extract time dependencies, and generate a time series feature vector; The feature fusion unit is used by each edge device to locally perform multi-scale feature fusion on byte-level features, global graph embedding features, and time series features. This unit uses a dynamic weighting mechanism to calculate a weighted value based on the importance of each feature, generating a high-dimensional unified fusion feature. This fusion feature serves as the input for the local encrypted traffic classification model, also known as the local model. The edge training unit is used for each edge device to independently train the local encrypted traffic classification model based on the fusion features. Each edge device uses local traffic data and its labels to train the model and calculate the local model parameters. The training process is performed entirely locally; after training is completed, each edge device uploads the local model parameters to the central server; The central weighting unit is used by the central server to receive local model parameters uploaded by each edge device through the federated learning framework, and adopts an aggregation algorithm to perform a weighted average of the local model parameters based on the contribution of each edge device to obtain the global model parameters. The contribution includes: the importance of the device in the network and the amount of local data of each edge device; The iteration unit is used by the central server to send the aggregated global model parameters to each edge device. Each edge device then uses the global model parameters for the next round of local training. After each round of training, each edge device updates its local model parameters and uploads the updated local model parameters to the central server. Through repeated local training and model aggregation, the final global model is obtained. The detection unit is used to deploy the final global model on the central server for real-time detection of abnormal traffic in the encrypted network.

13. The encryption network abnormal traffic detection system according to claim 12, characterized in that: The data acquisition unit is specifically used for: Each edge device only collects and processes data locally and does not transmit it to the central server.

14. The encryption network abnormal traffic detection system according to claim 12, characterized in that: The data acquisition unit is further specifically used for: Each edge device in the edge computing environment, including routers, cameras, and IoT gateways, collects normal traffic data and abnormal traffic data in simulated attack scenarios and stores them as pcap files; simulated attack scenarios include distributed denial of service attacks and data injection; Perform feature extraction and data preprocessing on the pcap file, including extracting header information and payload byte sequence, cleaning the data and splitting it by time window, and finally generating a csv file containing labels.

15. The encryption network abnormal traffic detection system according to claim 12, characterized in that: The data acquisition unit is further specifically used for: Each edge device deduplicates the traffic data containing time series and removes duplicate traffic records; Clean the traffic data, remove empty packets, and correct abnormal data through interpolation or averaging; The traffic data is divided into fixed time windows to generate continuous time segments.

16. The encryption network abnormal traffic detection system according to claim 12, characterized in that: The first feature generation unit is specifically used to: Each edge device parses the header and payload byte sequences in the traffic data packet, treating each byte as a node to form an initial byte set; Calculate the potential correlation between bytes and use the point-by-point mutual information (PMI) method to quantify the correlation strength of byte co-occurrence. The PMI calculation formula is: Where x and y represent two bytes or byte pairs in the data packet, PMI(x,y) represents the strength of association between byte x and byte y, that is, the joint probability that byte pair x and y appear at the same time, and P(x) and P(y) represent the marginal probabilities that byte x and byte y appear independently, respectively. Based on the PMI calculation results, the association strength of all nodes and edges is ranked, and several edges with higher PMI values ​​are retained to form a byte-level traffic graph that reflects the local association relationship in the traffic packet and obtain byte-level features.

17. The system for detecting abnormal traffic in an encrypted network according to claim 12, wherein: The second feature generation unit is specifically used to: Each edge device builds an interaction graph between IP hosts based on the IP addresses in the collected traffic data, where each node represents an IP host and the edges between nodes represent the interaction information between hosts; The edges of the graph are established through the interaction relationship between the source IP and the target IP. The edge weights are represented by the interaction strength, forming a complete global interaction graph structure. The interaction strength includes: byte flow and packet exchange frequency. In the graph neural network GNN, a multi-layer message passing mechanism is adopted, and each layer updates the node features through the neighborhood aggregation strategy. The specific formula is: in, represents the feature representation of node v at the k+1th layer, v is the target node, u is the neighbor node of node v, N(v) represents the set of neighbor nodes of node v; AGG(·) is the neighborhood feature aggregation function, which is used to integrate the feature information of node v and its neighbor nodes; W is the learnable weight matrix; σ(·) is the nonlinear activation function; Represents the feature representation of neighbor node u at the kth layer; Through a multi-layer message passing mechanism, the local and global interaction characteristics of nodes in the global interaction graph are captured, and global graph embedding features are generated to characterize the global interaction patterns and structural information of traffic.

18. The encryption network abnormal traffic detection system according to claim 12, characterized in that: The third feature generation unit is specifically used to: Each edge device constructs a time series input, divides the traffic data into continuous time segments according to the time window, and arranges them in chronological order to form time series data; Input the time series data into the time series encoder, and use the encoder to capture the dynamic changes and time dependencies of traffic characteristics in the time series; Temporal encoders, including LSTM or Transformer; The feature vector output by the encoder is extracted and used as a time series feature vector to characterize the temporal dynamic characteristics of the traffic.

19. The encryption network abnormal traffic detection system according to claim 12, characterized in that: Feature fusion unit, specifically used for: Each edge device normalizes byte-level features, global graph embedding features, and time series features to ensure consistent dimensions across different features. Based on the importance of byte-level features, global graph embedding features, and time series features in encrypted traffic detection, the fusion weight of each feature is calculated, and adaptive weight values ​​are assigned to different features. The weighted byte-level features, global graph embedding features, and time series features are concatenated to generate high-dimensional unified fusion features to capture multi-scale information of traffic.

20. The encryption network abnormal traffic detection system according to claim 19, characterized in that: Based on the importance of byte-level features, global graph embedding features, and time series features in encrypted traffic detection, the fusion weight of each feature is calculated, and weight values ​​are assigned to different features based on their adaptability, including: In a dynamic weighting mechanism, the importance of byte-level features, global graph embedding features, and temporal features in encrypted traffic detection is evaluated. By calculating the impact of each feature on the loss function and using gradient information to evaluate the contribution of the feature, the degree of influence of the feature on the classification result is determined. A dynamic weighting mechanism assigns a weight to each feature based on its importance. The weights are adjusted in real time during training based on model feedback. Feature importance is measured by gradient size; features with larger gradient changes receive higher weights, ensuring that features with greater impact on the classification task dominate the fusion process. After determining the weight of each feature, the byte-level feature, global graph embedding feature, and temporal feature are multiplied by their corresponding weights and then weighted fused to generate the fused feature F final ; The specific weighted fusion calculation formula is: F final =w1·F byte +w2·F graph +w3·F time Among them, F byte 、F graph 、F time Represent byte-level features, global graph embedding features and temporal features respectively, w1, w2, w3 are their dynamic weights; the generated fusion feature F final Used as input to the local encrypted traffic classification model.

21. The encryption network abnormal traffic detection system according to claim 12, characterized in that: Edge training unit, specifically used for: The generated fusion features are input into a local encrypted traffic classification model, and the model parameters are adjusted through local training and optimization to minimize the classification loss function. The local encrypted traffic classification model includes: a deep neural network or other classification algorithm; the local training and optimization of model parameters include: using Stochastic Gradient Descent, SGD or Adam optimizer; the classification loss function includes: cross entropy loss; After training is completed, each edge device uploads the updated local model parameters to the central server; only the local model parameters are uploaded, and no local data is transmitted; the local model parameters include weights and biases.

22. The encryption network abnormal traffic detection system according to claim 12, characterized in that: Iteration unit, specifically used for: The central server sends the aggregated global model parameters to each edge device. Specifically, the central server uses the federated learning framework to perform a weighted average aggregation of the model parameters collected from each device to generate global model parameters, which are then passed to each edge device. Each edge device receives and loads these global model parameters as the initial state for the next round of local training. Local model parameters include weights and biases. Edge devices use global model parameters for local training. In each round of training, edge devices use the received global model parameters and combine them with local fusion features to independently train the encrypted traffic classification model. During local training, the device optimizes the model parameters based on local data to improve the model's performance in local tasks. The edge device updates the local model parameters and uploads them to the central server. After each round of training, each edge device uploads the updated model parameters, including the weights and biases adjusted after training, to the central server. The central server then aggregates the local model parameters from each device to generate a new global model. After multiple rounds of local training and global model aggregation, the final global model is generated.

23. The encryption network abnormal traffic detection system according to claim 12, characterized in that: The detection unit is specifically used to: The final global model is deployed on the central server to process new encrypted traffic data in real time. During the testing phase, new traffic data is processed through feature extraction and fusion, and then input into the final global model for inference. The final global model outputs the traffic classification results based on the input traffic features and compares them with the true labels to evaluate the performance of the model.

24. An electronic device, characterized in that: The processor, the communication interface, the memory and the communication bus are connected to each other via the communication bus. a memory storing a computer program; The processor is configured to implement the method for detecting abnormal traffic in an encrypted network according to any one of claims 1 to 11 when executing a computer program stored in a memory.

25. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting abnormal traffic in an encrypted network according to any one of claims 1 to 11 is implemented.

Images