Custom encrypted traffic threat detection method and system of self-attention encoder based on load statistical characteristics

Abstract

The invention discloses a self-defined encrypted traffic threat detection method and system for a self-attention encoder based on load statistical characteristics, and the method and system are applied to a threat detection device, and the method comprises the steps: obtaining network traffic data; extracting load statistical characteristics and session process characteristics of the network flow data; constructing an encrypted traffic session representation vector based on the load statistical feature and the session process feature; and performing threat detection on the encrypted traffic session representation vector by using a pre-trained self-attention encoder. According to the method, the accuracy and efficiency of user-defined encrypted traffic threat detection are improved, and the generalization ability of the model in a few-sample scene is ensured through comparative learning pre-training.

Description

Technical Field

[0001] This invention relates to the field of network security and encrypted traffic analysis technology, and in particular to custom encrypted traffic threat detection technology. Background Technology

[0002] In today's network environment, encrypted traffic is widely used in various online services and communications to ensure the confidentiality and integrity of user data. With the widespread adoption of standard encryption protocols such as TLS / SSL, more and more cyber attackers are using encrypted channels to conceal their malicious activities, rendering traditional threat identification methods based on plaintext detection increasingly ineffective. Of particular note is that, to evade detection by common security products, attackers are increasingly inclined to use custom encryption protocols or niche encryption tools to construct communication links, carrying out malicious activities such as data leakage, remote control, and web attacks.

[0003] Conventional encrypted traffic detection methods mainly fall into two categories: one is detection methods based on man-in-the-middle decryption technology, which perform content auditing after decrypting the traffic through a proxy. This method has problems such as high performance overhead, risk of user privacy leakage, and only support for a limited number of known protocols. The other is detection methods based on feature engineering and machine learning, which rely on manually extracting statistical features of the stream or using deep learning models to process the raw byte stream. However, when faced with unknown protocols, few sample scenarios, or diverse encryption behaviors, the generalization ability is limited, the false positive rate is high, and it is often difficult to effectively handle custom encrypted traffic with strong payload randomness and complex structure.

[0004] Existing methods generally suffer from the following bottlenecks when processing encrypted traffic: directly truncating or zero-padding the ciphertext payload destroys the statistical structure and introduces noise; relying on a large number of labeled samples and expert features makes it difficult to adapt to rapidly evolving attack methods; and the identification effect is not ideal for protocol-independent, zero-sample, or few-sample custom encrypted threats. Therefore, there is an urgent need for a high-precision threat detection scheme that can effectively extract multi-dimensional statistical and structural features of encrypted sessions without relying on decryption and is applicable to small-sample scenarios. Summary of the Invention

[0005] This application provides a custom encrypted traffic threat detection method based on a self-attention encoder with load statistical features, and proposes a corresponding detection system to solve the technical problems in existing encrypted traffic detection technologies, such as weak ability to identify unknown protocols, reliance on a large amount of labeled data and manual feature design, high false alarm rate, and poor performance in processing custom encrypted traffic.

[0006] The technical solution of this invention: a custom encrypted traffic threat detection method based on a self-attention encoder with load statistical features, comprising the following steps: ① Obtain network traffic data; ② Extract the load statistics and session process features of the network traffic data; ③ Construct an encrypted traffic session representation vector based on the aforementioned load statistical characteristics and session process characteristics; ④ Use a pre-trained self-attention encoder to identify the encrypted traffic session representation vector and output the threat detection result.

[0007] Furthermore, obtaining network traffic data in ① includes: capturing raw traffic data packets from network devices or traffic monitoring systems; performing session segmentation and reassembly on the raw traffic data packets to generate bidirectional session data.

[0008] Furthermore, the load statistical characteristics and session process characteristics described in ② include: For each raw traffic data packet, extract its network layer and transport layer protocol field features; For the payload portion of each raw traffic data packet, calculate multi-dimensional statistical indicators, including at least one of entropy, smoothing entropy, frequency statistics, run statistics, and autocorrelation statistics; The protocol field features and payload statistical features are concatenated into a packet-level feature vector; Organize all packet-level feature vectors in the session in chronological order to form a session matrix.

[0009] Furthermore, the encrypted traffic session representation vector described in ③ includes mapping packet-level feature vectors to encoder input dimensions using a linear layer; Add position and orientation codes according to the encoder input dimensions; After adding positional and orientation encodings, the data is then input into a self-attention encoder to extract the session-level representation vector.

[0010] Furthermore, ④ uses a pre-trained self-attention encoder for recognition, including: Encoder pre-training was performed using unlabeled encrypted traffic data through contrastive learning. Fine-tuning of the pre-trained encoder using labeled encrypted traffic data; The encrypted traffic session representation vector is input into the classifier, and the output is the judgment result of whether it is malicious traffic.

[0011] A custom encrypted traffic threat detection system based on a self-attention encoder using load statistics features includes: The traffic acquisition unit is used to acquire network traffic data, including raw traffic data packets captured from network devices or traffic monitoring systems, and to perform session segmentation and reassembly of the raw traffic data packets to generate bidirectional session data. The feature extraction unit is used to extract the load statistics features and session process features of the network traffic data. For each raw traffic data packet, it extracts the protocol field features of the network layer and transport layer; for the load part of each raw traffic data packet, it calculates multi-dimensional statistical indicators; it concatenates the protocol field features and load statistics features into a packet-level feature vector; and it organizes all packet-level feature vectors in the session in chronological order to form a session matrix. The representation building unit constructs encrypted traffic session representation vectors using payload statistical features and session process features. A linear layer maps the packet-level feature vectors to the encoder input dimension. Positional and directional encodings are added to the encoder input dimension. After adding positional and directional encodings, the vectors are input into the self-attention encoder to extract the session-level representation vectors. The threat identification unit is used to identify and judge the encrypted traffic representation vector using a pre-trained self-attention encoder. Specifically, it uses unlabeled encrypted traffic data to pre-train the encoder through contrastive learning; it uses labeled encrypted traffic data to fine-tune the pre-trained encoder; it inputs the encrypted traffic session representation vector into the classifier and outputs the judgment result of whether it is malicious traffic.

[0012] Preferably, the multidimensional statistical indicators include at least one of entropy, smoothing entropy, frequency statistics, runs statistics, and autocorrelation statistics.

[0013] An electronic device includes: at least one processor, and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the method described above.

[0014] A storage medium having a computer program stored thereon that, when executed by a processor, implements the method described above.

[0015] The beneficial effects of this invention are as follows: by extracting multi-dimensional statistical features from encrypted traffic payloads and combining them with a self-attention encoder for session-level representation, it can effectively identify threat traffic in custom encryption protocols without decryption; by adopting a contrastive learning pre-training and fine-tuning mechanism, it significantly reduces the dependence on labeled data and improves the model's detection capability and generalization performance in scenarios with few or no samples; the overall solution has high detection accuracy and low false alarm rate, and is suitable for deployment in actual network security environments. Attached Figure Description

[0016] Figure 1 This is an overall flowchart of the embodiments in this application; Figure 2 This application describes the composition and extraction features of network traffic data packets in the embodiments of this application; Figure 3This is a schematic diagram illustrating the process of extracting data packet protocol features and payload statistical features in an embodiment of this application, which details how to extract protocol features and payload statistical features from raw network traffic; Figure 4 This is an architectural block diagram of the custom encrypted traffic threat detection system in this application embodiment, which describes the functions and interactions of each module of the system. Detailed Implementation

[0017] like Figure 1 The diagram shown is a flowchart of the encrypted traffic threat detection method in the application embodiment, including: S1. Obtain network traffic data; S2. Analysis and processing based on self-attention encoder and load statistical characteristics; S3. Output the inspection results.

[0018] like Figure 2 The diagram illustrates the structure and extraction features of network traffic data packets in this embodiment. Each network traffic data packet employs a layered structure, including: a MAC header, an IP header, a transport layer header (TCP or UDP), and a payload. The MAC header contains link layer information such as the source and destination MAC addresses; the IP header contains network layer information such as the source and destination IP addresses, protocol type, and Time-to-Live (TTL); the transport layer header contains transport layer information such as the source port, destination port, sequence number, acknowledgment number, and flags; and the payload contains the actual transmitted application layer data. For encrypted traffic, the payload is encrypted and exhibits randomized characteristics.

[0019] like Figure 2 As shown, the specific features extracted from network traffic data packets include load statistics features and session process features: 1. Time characteristics: packet capture timestamp, time interval between the packet and the previous packet; 2. Directional characteristics: Data packet transmission direction (client to server or server to client). 3. IP Protocol Characteristics: Fields extracted from the IP header, including protocol version, header length, type of service, total length, identifier, fragmentation flag, time to live, header checksum, source IP address, and destination IP address; 4. Transmission protocol characteristics: Fields extracted from the TCP / UDP header, including source port, destination port, sequence number, acknowledgment number, data offset, reserved bits, flags (URG, ACK, PSH, RST, SYN, FIN), window size, checksum, urgent pointer, etc. 5. Multi-dimensional statistical characteristics of the load: 19 types of statistical indicators, including entropy, smoothing entropy, frequency statistics, run statistics, autocorrelation statistics, etc., calculated for the load component.

[0020] Figure 3 The diagram shows the processing flow for load statistical feature extraction, which mainly includes the following steps: S301: Obtaining the original data packet The system captures raw network traffic packets in real time using traffic probes or mirror ports deployed on network nodes (such as core switches and gateway servers). The captured packets are typically in PCAP format and contain a complete link-layer frame structure. Each packet carries a precise timestamp (accurate to microseconds) for subsequent packet interval calculation. The system segments the captured packets into sessions based on a five-tuple (source IP address, destination IP address, source port, destination port, transport layer protocol), reassembling packets belonging to the same communication session into a bidirectional data stream to provide context for subsequent feature extraction.

[0021] S302: Feature Extraction For each data packet in the session, perform refined feature extraction: 1. Parse session process characteristics: Parse the MAC header, IP header, and TCP / UDP header of data packets layer by layer at the network layer and transport layer, and extract predefined 25-dimensional protocol field characteristics, including but not limited to: IP packet TTL (Time to Live), total length, identifier; TCP connection flags (SYN, ACK, FIN, RST, PSH, URG), window size, sequence number, acknowledgment number, etc. 2. Calculate the statistical characteristics of the payload: For the raw byte sequence of the transport layer payload, calculate 19-dimensional statistical indicators. These indicators are used to quantify the randomness and distribution characteristics of the payload, and mainly include: byte entropy, smoothing entropy, single-bit frequency statistics, intra-block frequency statistics, total run count statistics, run distribution statistics, autocorrelation statistics, etc.

[0022] S303: Adaptive MinMax Standardization The 44-dimensional feature vector (25-dimensional protocol field features + 19-dimensional statistical indicators) extracted in the previous step is normalized to construct an encrypted traffic session representation vector, thereby eliminating the influence of differences in feature dimensions and numerical ranges on the model. The processed protocol field features and payload statistical features are concatenated to form a packet-level feature vector; all packet-level feature vectors in the session are organized in chronological order to form a session matrix. The package-level feature vectors are mapped to the encoder input dimension. This application uses an adaptive MinMax normalization method. - For features whose numerical ranges are relatively concentrated, their 0th and 95th percentiles in the dataset are used as the minimum and maximum values, respectively, for scaling.

[0023] - For a very small number of features with extremely large numerical ranges, perform a logarithmic transformation first, followed by the MinMax scaling described above. This method effectively avoids the interference of outliers on the standardization process, making the feature data more suitable for neural network processing.

[0024] S304: Self-attention encoder processing The standardized feature sequence is input into a multi-layer self-attention encoder based on the Transformer architecture to identify the encrypted traffic session representation vector and output the threat detection result. The encoder consists of 6 identical layers stacked together, each layer containing a multi-head self-attention mechanism (12 heads) and a feedforward neural network (with a hidden layer dimension of 384).

[0025] - Position and orientation encoding: Before inputting the encoder, learnable position and orientation encodings are added to the feature vector of each packet, enabling the model to perceive the order and direction of packet transmission in the session.

[0026] - Self-attention computation: Packets with added positional and directional encodings are input into the self-attention encoder to extract session-level representation vectors. The encoder calculates the relevance weights between each packet and all other packets in the session through a self-attention mechanism, thereby capturing long-distance dependencies and global contextual information, and can automatically focus on key packets and their features that are highly relevant to malicious behavior.

[0027] The encoder outputs a new, context-rich feature representation for each bag.

[0028] S305: Session Load Statistical Characteristics The feature vector corresponding to the first special marker ([CLS]) position in the entire sequence output by the encoder is taken as the global representation of the entire session. This fixed-length vector (e.g., 768-dimensional) aggregates the interaction information and statistical characteristics of all packets in the entire session, forming an abstract and efficient "fingerprint" or "feature representation" of the encrypted traffic session.

[0029] S306: Neural Networks Perform Feature Merging and Transformation The global representation of the session obtained in the previous step is input into a multilayer perceptron (MLP) neural network. This neural network does not directly perform decision classification, but rather acts as a feature merging and nonlinear transformer.

[0030] Figure 4 The diagram shows the internal processing flow of the overall detection system, which mainly includes the following steps: S401: PCAP file input The system receives network traffic packet capture files (PCAP format) from network devices or traffic monitoring systems. These files contain the raw binary data of all network traffic captured within a specific time period, with each packet fully preserving its MAC header, IP header, transport layer header, and payload. The system supports batch processing of multiple PCAP files and can handle large traffic files generated in high-speed network environments.

[0031] S402: Extract Session Use network packet processing libraries such as Scapy to parse PCAP files and reassemble sessions. 1. Parse the headers of each data packet at all levels and extract the five-tuple information (source IP address, destination IP address, source port, destination port, transport layer protocol). 2. Divide data packets into different sessions based on the 5-tuple information; 3. Sort the data packets within each session according to their timestamps to ensure the correct timing of the packet sequence; 4. Generate a list of sessions, each containing a complete sequence of data packets and its metadata.

[0032] S403: Session Process Feature Extraction For each session, stream-level statistical features are extracted, mainly including: 1. Forward / backward packet count, total bytes, average packet length, and packet length variance; 2. Statistical characteristics of packet arrival time intervals (minimum, maximum, mean, variance); 3. TCP flag statistics (statistics on the number of SYN, ACK, FIN, RST, etc.); 4. Session duration, packets per second, bytes per second; 5. Packet size distribution characteristics (proportion of small, medium, and large packages); These features together constitute the session process feature vector, which is used to describe the overall behavioral characteristics of the session.

[0033] S404: Load Statistical Feature Extraction Refined statistical feature extraction is performed on the payload portion of each data packet in the session: 1. Calculate the entropy and smoothing entropy of the payload bytes to measure the degree of randomness of the payload; 2. Perform frequency statistics, including single-bit frequency and intra-block frequency statistics; 3. Calculate the statistical characteristics of the tours, including the total number of tours and the tour distribution; 4. Extracted from advanced statistical indicators such as correlation statistics and matrix rank statistics; 5. Generate a 19-dimensional load statistical feature vector for each package; 6. Organize the statistical characteristics of all packages into a feature matrix in chronological order.

[0034] S405: Feature Merging Fusing session process features and load statistics features: 1. Normalize the two types of feature vectors separately to eliminate the difference in dimensions; 2. Use feature concatenation to merge session-level features and packet-level features; 3. Employ an attention mechanism to perform weighted fusion of different types of features; 4. Generate the final encrypted traffic session representation vector, which includes both the overall behavioral characteristics of the session and the detailed statistical characteristics of each packet.

[0035] S406: Loss Function Calculation Calculate the loss function based on the model training objective: 1. For supervised learning tasks, the cross-entropy loss function is used to calculate the difference between the predicted results and the true labels; 2. For the contrastive learning pre-training task, the InfoNCE loss function is used to shorten the distance between positive sample pairs and widen the distance between negative sample pairs; 3. Combine regularization terms to prevent overfitting and improve the model's generalization ability; 4. The loss function calculation results are used for backpropagation and model parameter optimization.

Claims

1. A custom encrypted traffic threat detection method based on a self-attention encoder with load statistical features, characterized by: The process includes the following steps: ① Obtain network traffic data; ② Extract the load statistics and session process features of the network traffic data; ③ Construct an encrypted traffic session representation vector based on the aforementioned load statistical characteristics and session process characteristics; ④ Use a pre-trained self-attention encoder to identify the encrypted traffic session representation vector and output the threat detection result.

2. The custom encrypted traffic threat detection method based on a self-attention encoder with load statistical features according to claim 1, characterized in that: ① The acquisition of network traffic data includes: capturing raw traffic data packets from network devices or traffic monitoring systems; performing session segmentation and reassembly on the raw traffic data packets to generate bidirectional data streams.

3. The custom encrypted traffic threat detection method based on a self-attention encoder with load statistical features according to claim 2, characterized in that: The load statistics and session process characteristics mentioned in section ② include: For each raw traffic data packet, extract its network layer and transport layer protocol field features; For the payload portion of each raw traffic data packet, calculate at least one of the following multi-dimensional statistical indicators: byte entropy, smoothing entropy, single-bit frequency statistics, intra-block frequency statistics, total run count statistics, run distribution statistics, and autocorrelation statistics. The protocol field features and payload statistical features are concatenated into a packet-level feature vector; Organize all packet-level feature vectors in the session in chronological order to form a session matrix.

4. The custom encrypted traffic threat detection method based on a self-attention encoder with load statistical features according to claim 3, characterized in that: The encrypted traffic session representation vector described in ③ includes mapping packet-level feature vectors to encoder input dimensions using a linear layer; Add position and orientation codes according to the encoder input dimensions; After adding positional and orientation encodings, the data is then input into a self-attention encoder to extract the session-level representation vector.

5. The custom encrypted traffic threat detection method based on a self-attention encoder with load statistical features according to claim 3, characterized in that: ④ Recognition using a pre-trained self-attention encoder includes: Encoder pre-training was performed using unlabeled encrypted traffic data through contrastive learning. Fine-tuning of the pre-trained encoder using labeled encrypted traffic data; The encrypted traffic session representation vector is input into the classifier, and the output is the judgment result of whether it is malicious traffic.

6. A custom encrypted traffic threat detection system based on a self-attention encoder with load statistical characteristics. Its features include: a traffic acquisition unit, used to acquire network traffic data, including raw traffic data packets captured from network devices or traffic monitoring systems, and to perform session segmentation and reassembly on the raw traffic data packets to generate bidirectional session data; The feature extraction unit is used to extract the load statistics features and session process features of the network traffic data, and to extract the protocol field features of the network layer and transport layer for each raw traffic data packet. For the payload portion of each raw traffic data packet, calculate multi-dimensional statistical indicators; concatenate protocol field features with payload statistical features to form a packet-level feature vector; organize all packet-level feature vectors in the session in chronological order to form a session matrix; The representation building unit constructs encrypted traffic session representation vectors using payload statistical features and session process features. A linear layer maps the packet-level feature vectors to the encoder input dimension. Positional and directional encodings are added to the encoder input dimension. After adding positional and directional encodings, the vectors are input into the self-attention encoder to extract the session-level representation vectors. The threat identification unit is used to identify and judge the encrypted traffic representation vector using a pre-trained self-attention encoder. Specifically, it uses unlabeled encrypted traffic data to pre-train the encoder through contrastive learning; it uses labeled encrypted traffic data to fine-tune the pre-trained encoder; it inputs the encrypted traffic session representation vector into the classifier and outputs the judgment result of whether it is malicious traffic.

7. The custom encrypted traffic threat detection system based on a self-attention encoder with load statistical characteristics according to claim 6, characterized in that: The multi-dimensional statistical indicators include at least one of the following: byte entropy, smoothing entropy, single-bit frequency statistics, intra-block frequency statistics, total run count statistics, run distribution statistics, and autocorrelation statistics.

8. An electronic device comprising: At least one processor, and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the processor to enable the processor to perform the method of any one of claims 1-5.

9. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the method described in any one of claims 1-5.

Images