Traffic detection method and device, computer device and storage medium

Abstract

The application discloses a kind of flow detection method, device, computer equipment and storage medium, method includes: in target network acquisition network flow data;Based on the visible metadata of encrypted traffic and transmission statistics, statistical processing is carried out in pre-set time window, and the window characteristic vector corresponding to each time window is obtained;In each pre-set time window, the communication entity in encrypted traffic is taken as node, the window characteristic vector is taken as the node feature of corresponding node, and the interaction between communication entities in time window is taken as edge, and the space-time interaction graph is obtained by construction;Based on the preset neural differential model, the space-time interaction graph is processed by continuous time modeling, and the detection result of encrypted traffic is obtained;Based on the detection result of encrypted traffic, the flow detection result of target network is determined.Encrypted traffic is detected by space-time interaction graph and neural differential modeling, while ensuring privacy, to improve detection accuracy and robustness.

Description

Flow detection methods, devices, computer equipment and storage media Technical Field

[0001] This invention relates to the field of network security, and in particular to a traffic detection method, apparatus, computer equipment, and storage medium. Background Technology

[0002] In the field of network security, most existing traffic detection methods rely on traditional deep packet inspection (DPI), port identification, or signature-based detection techniques. These methods typically require parsing the complete content of network packets to identify protocol types or detect abnormal behavior. However, with the increasing prevalence of encrypted network communications, more and more traffic is being transmitted using encryption protocols such as TLS, QUIC, and SSH, making it impossible for traditional methods to directly obtain plaintext data characteristics, resulting in a significant decrease in detection effectiveness. Furthermore, some traffic analysis methods based on statistical features or machine learning often rely on discrete-time step sequence modeling (such as LSTM), which struggles to accurately capture the complex temporal dependencies between communication entities when faced with highly dynamic and non-stationary encrypted traffic, leading to insufficient stability of detection results and a high false positive rate.

[0003] It is evident that existing traffic detection methods are insufficient in terms of accuracy and robustness when dealing with encrypted traffic. There is an urgent need to propose a traffic detection method that can improve the accuracy and reliability of network traffic detection while ensuring the privacy characteristics of encrypted traffic. Summary of the Invention

[0004] Therefore, it is necessary to provide a flow detection method, device, computer equipment, and storage medium to address the aforementioned technical problems and solve the issues of low detection accuracy and robustness of traditional flow detection methods.

[0005] A flow detection method, the method comprising:

[0006] Collect network traffic data in the target network, including encrypted traffic;

[0007] Based on the visible metadata and transmission statistics of the encrypted traffic, statistical processing is performed within a preset time window to obtain the window feature vector corresponding to each time window.

[0008] Within each preset time window, the communication entities in the encrypted traffic are used as nodes, the window feature vector is used as the node feature of the corresponding node, and the interaction relationship between the communication entities within the time window is used as the edge to construct a spatiotemporal interaction graph.

[0009] The spatiotemporal interaction graph is modeled continuously based on a preset neural differential model to obtain the detection result of the encrypted traffic.

[0010] Based on the detection results of the encrypted traffic, the traffic detection results of the target network are determined.

[0011] Optionally, the network traffic data further includes unencrypted traffic, and determining the traffic detection result of the target network based on the detection result of the encrypted traffic includes:

[0012] Statistical features at different granularities are extracted from the unencrypted traffic to obtain multi-scale statistical features. In the multi-scale statistical features, the granularity of each scale is different from that of other scales.

[0013] Based on the aforementioned multi-scale statistical features, feature fusion processing is performed to obtain a fused feature vector;

[0014] The fused feature vector is provided to a preset traffic classification model so that the traffic classification model outputs the traffic classification result corresponding to the fused feature vector;

[0015] The fused feature vector is provided to a preset anomaly detection model so that the anomaly detection model outputs the anomaly detection result corresponding to the fused feature vector;

[0016] The detection result of the unencrypted traffic is determined based on the traffic classification result and the anomaly detection result;

[0017] Based on the detection results of the unencrypted traffic and the detection results of the encrypted traffic, the traffic detection results of the target network are determined.

[0018] Optionally, the step of extracting statistical features at different granularities from the unencrypted traffic to obtain multi-scale statistical features includes:

[0019] The unencrypted traffic is then subjected to protocol decoding to obtain the decoded traffic.

[0020] Statistical features at different granularities are extracted from the decoded traffic to obtain multi-scale statistical features.

[0021] Optionally, the feature fusion processing based on the multi-scale statistical features to obtain the fused feature vector includes:

[0022] Dimensionality reduction is performed based on the multi-scale statistical features to obtain a low-dimensional feature vector.

[0023] Generate the corresponding binary hash code based on the low-dimensional feature vector;

[0024] The binary hash code is concatenated with the multi-scale statistical features to form the fused feature vector.

[0025] Optionally, determining the traffic detection result of the target network based on the detection results of the unencrypted traffic and the detection results of the encrypted traffic includes:

[0026] The detection results of the unencrypted traffic and the detection results of the encrypted traffic are collected and integrated to form a comprehensive result record;

[0027] The results of the synthesis are formatted to obtain structured result data in a preset format;

[0028] The structured result data is used as the traffic detection result, which includes at least one of report output and database storage.

[0029] Optionally, the visible metadata and transmission statistics of the encrypted traffic are statistically processed within a preset time window to obtain a window feature vector corresponding to each time window, including:

[0030] Within the time window, visible metadata and transmission statistics of the encrypted traffic are extracted as window features;

[0031] The window features are normalized to obtain the window feature vector.

[0032] Optionally, the continuous-time modeling of the spatiotemporal interaction graph based on a preset neural differential model to obtain the detection result of the encrypted traffic includes:

[0033] The spatiotemporal interaction graph is subjected to feature fusion processing based on a spatiotemporal attention mechanism to obtain fused interaction features.

[0034] The fused interactive features are provided to a preset neural differential model for continuous-time modeling to obtain the detection results of the encrypted traffic.

[0035] A flow detection device, the device comprising:

[0036] The first acquisition module is used to acquire network traffic data in the target network, the network traffic data including encrypted traffic;

[0037] The first statistics module is used to perform statistical processing within a preset time window based on the visible metadata and transmission statistics of the encrypted traffic, and obtain the window feature vector corresponding to each time window.

[0038] The first construction module is used to construct a spatiotemporal interaction graph within each preset time window, taking the communication entities in the encrypted traffic as nodes, the window feature vector as the node features of the corresponding nodes, and the interaction relationship between the communication entities within the time window as edges.

[0039] The first modeling module is used to perform continuous-time modeling processing on the spatiotemporal interaction graph based on a preset neural differential model to obtain the detection result of the encrypted traffic;

[0040] The first determining module is used to determine the traffic detection result of the target network based on the detection result of the encrypted traffic.

[0041] A computer device includes a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-described flow detection method when executing the computer-readable instructions.

[0042] A readable storage medium having computer-readable instructions stored thereon, which, when executed by a processor, implement the flow detection method.

[0043] The aforementioned traffic detection method, apparatus, computer equipment, and storage medium collect network traffic data in a target network, including encrypted traffic. Based on the visible metadata and transmission statistics of the encrypted traffic, statistical processing is performed within a preset time window to obtain a window feature vector corresponding to each time window. Within each preset time window, communication entities in the encrypted traffic are used as nodes, the window feature vector is used as the node feature of the corresponding node, and the interaction relationships between communication entities within the time window are used as edges to construct a spatiotemporal interaction graph. A continuous-time modeling process is performed on the spatiotemporal interaction graph based on a preset neural differential model to obtain the detection result of the encrypted traffic. Based on the detection result of the encrypted traffic, the traffic detection result of the target network is determined. By extracting the visible metadata and transmission statistics of the encrypted traffic within a time window and constructing a spatiotemporal interaction graph, and then combining this with continuous-time modeling processing using a neural differential model, accurate detection of encrypted traffic can be achieved without decrypting the traffic content. This not only protects user privacy but also better captures complex temporal and interaction characteristics, significantly improving the accuracy and robustness of the detection. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 is a flowchart illustrating a flow detection method according to an embodiment of the present invention;

[0046] Figure 2 is a schematic diagram of the flow detection device in one embodiment of the present invention;

[0047] Figure 3 is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] In one embodiment, as shown in FIG1, a flow detection method is provided, comprising the following steps:

[0050] 101. Collect network traffic data in the target network.

[0051] In this embodiment of the invention, the above-mentioned data collection can be achieved through a traffic collection module. The target network can be an enterprise internal LAN, a data center network, a carrier backbone network, or an IoT terminal access network, etc., which can cover a variety of practical application scenarios.

[0052] The aforementioned network traffic data can be complete network communication data packets from the data link layer (L2 layer) to the application layer (L7 layer), including packet header information, metadata information, and visible transmission statistics. To ensure comprehensiveness and real-time performance, the traffic acquisition module can be implemented based on the libpcap library. In a Linux system environment, the network interface to be captured (such as "eth0") is specified through the pcap_open_live() function, which returns a handle for subsequent operations.

[0053] During the data collection process, filtering rules can be set using the `pcap_compile()` and `pcap_setfilter()` functions. For example, only IP packets can be captured (the filter expression is "ip"), thereby reducing unnecessary data processing. Subsequently, the `pcap_loop()` or `pcap_dispatch()` functions are used for loop capture. Whenever a packet arrives at the network interface, a pre-defined callback function is called to receive the packet, and then performs initial processing and caching.

[0054] To handle the massive traffic in high-speed network environments, the traffic acquisition module can adopt a multi-threaded architecture: one thread is dedicated to capturing data packets and writing them into a shared buffer; another thread reads data packets from the buffer and passes them to subsequent processing modules. By properly configuring the buffer size, data packet loss due to buffer overflow can be effectively reduced, thereby ensuring the integrity and reliability of traffic acquisition.

[0055] The aforementioned network traffic data includes encrypted traffic. This encrypted traffic can be data packets transmitted using encryption protocols such as TLS, QUIC, and SSH. The payload of these packets is ciphertext and cannot be directly parsed, but it can still be analyzed and detected through visible metadata and transmission statistics (such as packet length, arrival time interval, source and destination port numbers, etc.).

[0056] 102. Based on the visible metadata and transmission statistics of encrypted traffic, perform statistical processing within a preset time window to obtain the window feature vector corresponding to each time window.

[0057] In this embodiment of the invention, the aforementioned visible metadata may be protocol message fields that are still observable during the transmission of encrypted traffic, such as source IP address, destination IP address, source port number, destination port number, transport protocol type, TLS version information, or certificate fingerprint, etc.

[0058] The aforementioned transmission statistics can be based on data packet level statistics, such as the number of data packets within a time window, data packet length distribution (e.g., mean, variance, maximum, minimum), arrival time interval distribution of adjacent data packets, and uplink / downlink traffic ratio, etc.

[0059] The aforementioned preset time window can be a time interval based on a fixed duration or a sliding duration, such as a time interval of 500ms, 1s, or 5s, used to ensure statistical stability while taking into account real-time performance. The specific time window can be adjusted according to actual needs and traffic characteristics.

[0060] Specifically, by organizing and statistically analyzing the visible metadata and transmission statistics of encrypted traffic within each preset time window, a window feature vector representing the communication behavior characteristics within the aforementioned time window can be obtained, providing input for subsequent interaction modeling and detection analysis.

[0061] 103. Within each preset time window, the communication entities in the encrypted traffic are used as nodes, the window feature vector is used as the node feature of the corresponding node, and the interaction relationship between the communication entities within the time window is used as the edge to construct a spatiotemporal interaction graph.

[0062] In this embodiment of the invention, the spatiotemporal interaction graph can be constructed within each time window according to the following steps:

[0063] Node definition: Treating network entities as graph nodes Network entities can include at least one of IP address, port number, and certificate subject / fingerprint. For each node, the obtained window feature vector of the entity within the current time window is assigned to the node as a node feature, used to characterize the statistical and behavioral features of the entity within that time window.

[0064] Edge definition and interactive judgment: For any two nodes and Check whether there is any communication interaction (e.g., TCP connection, DNS query-response, etc.) between the two within the time window [t-Δt, t]. For nodes within this time window and The set of data packets between, where t represents a point in time. Represented as an empty set: If If interaction is detected, an edge is established. And calculate edge weights To reflect the intensity of interaction:

[0065]

[0066] in, Indicates the number of bytes in data packet p; if If the edge weight is zero, then the edge is not created, or the edge weight is considered to be 0.

[0067] Dynamic adjacency matrix construction: Finally, based on the edge set With interaction strength Constructing a time-dependent adjacency matrix Initialize an N×N zero matrix (where N is the number of nodes in the node set V), then iterate through the set. For each edge Assignment:

[0068]

[0069] Pairs of nodes that have not interacted are kept at 0. Time-dependent adjacency matrix. Defined as:

[0070]

[0071] Here, the rows and columns of the matrix correspond to the nodes in the node set V, and the element values ​​represent the interaction strength between nodes within that time window.

[0072] The sequence and dependencies are as follows: First, nodes are defined to clarify the basic elements of the graph; then, edges are defined and their interaction strengths are calculated; finally, a dynamic adjacency matrix is ​​constructed based on the edge set and its strength. Edge definitions depend on the results of node definitions, and the construction of the adjacency matrix depends on the edge set and interaction strength information. Therefore, within each time window, a graph containing "node set V - edge set" is obtained. -Dynamic Adjacency Matrix - A spatiotemporal interaction graph of "node features (window feature vectors)" can be updated as the time window rolls to characterize the spatiotemporal evolution of encrypted traffic protocol behavior.

[0073] In summary, the construction of the spatiotemporal interaction graph, through a hierarchical definition process of "node-edge-adjacency matrix," comprehensively maps the behavioral characteristics and interaction strength of communication entities in encrypted traffic across the time dimension, achieving a dynamic representation of the state evolution of encrypted traffic protocols. This approach not only provides fine-grained characterization of interaction patterns between communication entities but also continuously updates as the time window rolls, effectively capturing potential malicious communication characteristics. It provides structured and temporally ordered data support for subsequent continuous-time modeling based on neural differential models, significantly improving the detection capability of complex attack behaviors in encrypted traffic.

[0074] 104. Based on the preset neural differential model, the spatiotemporal interaction graph is processed by continuous time modeling to obtain the detection results of encrypted traffic.

[0075] In this embodiment of the invention, the neural differential model is used to map the node features and edge interaction features in the spatiotemporal interaction graph into the dynamic evolution process of the protocol state.

[0076] Specifically, the node features and interaction relationships obtained from the spatiotemporal interaction graph can be directly input into the neural differential model. This model models the temporal characteristics of encrypted traffic through continuous-time differential equations, capturing the dynamic patterns of communication patterns over time. By analyzing the protocol states output by the model, potential abnormal behaviors or malicious communications can be identified, thereby generating detection results for encrypted traffic.

[0077] This processing method avoids the shortcomings of traditional discrete modeling in capturing sudden or long-distance dependencies, enables fine-grained characterization of encrypted traffic behavior, and provides a highly timely and accurate basis for subsequent traffic security analysis and attack detection.

[0078] Alternatively, the spatiotemporal interaction graph can be fused based on the spatiotemporal attention mechanism to obtain fused interaction features. These fused interaction features can then be provided to a preset neural differential model for continuous-time modeling to obtain the detection results of encrypted traffic.

[0079] 105. Based on the detection results of encrypted traffic, determine the traffic detection results of the target network.

[0080] In this embodiment of the invention, when all traffic in the target network is encrypted, the detection result of the encrypted traffic can be directly used as the traffic detection result of the target network, thereby achieving rapid identification.

[0081] When both encrypted and unencrypted traffic exist within the target network's traffic data, the detection results of both encrypted and unencrypted traffic can be combined, collected, integrated, and analyzed to form a comprehensive target network traffic detection result. This comprehensive detection result can reflect both potential malicious communication within encrypted traffic and abnormal behavioral characteristics within unencrypted traffic, thereby comprehensively improving the ability to perceive the network security status of the target network.

[0082] In this embodiment of the invention, network traffic data, including encrypted traffic, is collected in the target network. Based on the visible metadata and transmission statistics of the encrypted traffic, statistical processing is performed within a preset time window to obtain a window feature vector corresponding to each time window. Within each preset time window, communication entities in the encrypted traffic are used as nodes, the window feature vector is used as the node feature of the corresponding node, and the interaction relationship between communication entities within the time window is used as edges to construct a spatiotemporal interaction graph. A continuous-time modeling process is performed on the spatiotemporal interaction graph based on a preset neural differential model to obtain the detection result of the encrypted traffic. Based on the detection result of the encrypted traffic, the traffic detection result of the target network is determined. By extracting the visible metadata and transmission statistics of the encrypted traffic within a time window and constructing a spatiotemporal interaction graph, and then combining it with continuous-time modeling processing using a neural differential model, accurate detection of encrypted traffic can be achieved without decrypting the traffic content. This not only protects user privacy but also better captures complex temporal and interaction features, significantly improving the accuracy and robustness of detection.

[0083] It is understood that in the specific implementation of this application, data related to network traffic, encrypted traffic, and unencrypted traffic are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required. Furthermore, the collection, use, and processing of related data, as well as the construction and use of neural differential models, must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0084] Optionally, the network traffic data also includes unencrypted traffic. In the step of determining the traffic detection result of the target network based on the detection result of encrypted traffic, statistical features at different granularities can be extracted from the unencrypted traffic to obtain multi-scale statistical features. In the multi-scale statistical features, the granularity of each scale is different from that of other scales. Feature fusion processing is performed based on the multi-scale statistical features to obtain a fused feature vector. The fused feature vector is provided to a preset traffic classification model so that the traffic classification model outputs the traffic classification result corresponding to the fused feature vector. The fused feature vector is provided to a preset anomaly detection model so that the anomaly detection model outputs the anomaly detection result corresponding to the fused feature vector. The detection result of unencrypted traffic is determined based on the traffic classification result and the anomaly detection result. The traffic detection result of the target network is determined based on the detection results of unencrypted traffic and the detection results of encrypted traffic.

[0085] In this embodiment of the invention, network traffic data may include unencrypted traffic in addition to encrypted traffic. For unencrypted traffic, statistical features at different granularities can be further extracted to obtain multi-scale statistical features. Multi-scale statistical features can characterize the content characteristics of traffic at three different levels: character level, word level, and sub-word level. For example, character-level statistics may include character distribution and character entropy to measure the randomness of traffic; word-level statistics can be calculated using position-weighted TF-IDF to consider the positional information of words in traffic; and sub-word-level statistics can obtain sub-word units and their co-occurrence relationships through byte pair encoding (BPE), thereby comprehensively reflecting the multi-dimensional feature distribution of traffic.

[0086] After obtaining multi-scale statistical features, semantic hashing encoding and nonlinear dimensionality reduction techniques can be further combined to perform feature fusion, forming a fused feature vector. This fused feature vector can maintain the semantic similarity between features while reducing storage and computational complexity, thereby improving the discriminative power of traffic representation and enhancing analysis efficiency.

[0087] In the classification and detection stage, the fused feature vector can be input into different deep learning models: on the one hand, it is provided to a pre-defined convolutional neural network classification model, which uses convolution and pooling operations to extract traffic feature patterns, and finally outputs the traffic classification results through the softmax function, such as distinguishing normal traffic from various known attack traffic; on the other hand, the fused feature vector is provided to an autoencoder-based anomaly detection model, which determines whether the traffic is abnormal by minimizing the reconstruction error. If the reconstruction error exceeds the threshold, it is determined to be abnormal traffic.

[0088] Finally, the traffic classification results output by the classification model are fused with the anomaly detection results output by the anomaly detection model to obtain the detection results for unencrypted traffic. Then, the detection results for unencrypted traffic are combined with those for encrypted traffic to obtain the overall traffic detection result for the target network. In this way, not only can potential threats from encrypted traffic be detected while ensuring privacy, but also fine-grained feature extraction and deep learning models can be used to comprehensively analyze unencrypted traffic, thereby achieving high-precision detection of all traffic on the target network.

[0089] Specifically, the statistical extraction of the above-mentioned multi-scale statistical features can be performed by referring to the following steps:

[0090] Character-level statistics: Performing character-level statistical analysis on the text content after decoding unencrypted traffic. This includes calculating the frequency distribution of different character types in the text (e.g., the proportion of uppercase letters, numbers, and special symbols) and calculating character entropy. Used to measure the randomness of the overall distribution:

[0091]

[0092] Here, C is the character set, and p(c) is the probability of character c appearing. Character entropy can reflect the regularity and complexity of traffic content.

[0093] Word-level statistics: Based on traditional TF-IDF, a position-weighted mechanism is introduced to obtain position-weighted TF-IDF (PW-TF-IDF), which takes into account the positional information of words in the traffic content (e.g., words at the beginning or end of the data packet have higher weights). The calculation formula is as follows:

[0094]

[0095] in, The normalized position of word w in text D (value range: 0~1). This is the location weighting coefficient (empirical value taken as 0.5). By incorporating location information, it is possible to more accurately distinguish between protocol content and potential attack payloads.

[0096] Sub-word statistics: Byte-pair encoding (BPE) is used to break down traffic text into sub-word units and statistically analyze the frequency distribution and co-occurrence relationships of these sub-words. For example, "malware" is broken down into "mal" and "ware," and their co-occurrence frequency is counted. Sub-word statistics can capture potential variant features in attack payloads, improving the robustness of detection against obfuscation or variant attacks.

[0097] The traffic classification results corresponding to the fused feature vectors output by the above traffic classification model can be found in the following steps:

[0098] Feature vector input: The fused feature vectors are arranged into a feature matrix X of size m×n, where m is the feature dimension and n is the number of samples. This matrix serves as the input to a convolutional neural network (CNN).

[0099] Convolutional layer processing: utilizing multiple convolutional kernels Perform a convolution operation on the input matrix X to obtain the feature map.

[0100]

[0101] in, This represents the convolution operation. For bias terms For the activation function, ReLU is typically used:

[0102]

[0103] Convolution operations can extract local patterns from traffic features, while weight sharing mechanisms improve computational efficiency.

[0104] Pooling and Feature Extraction: Pooling operations are applied to the feature maps output by the convolutional layers to downsample the features, reduce redundancy, and improve the robustness of the model. After multiple convolutions and pooling, the resulting high-order features are flattened into a one-dimensional vector.

[0105] Fully connected layer and classification output: The flattened feature vector is input into the fully connected layer to obtain the output vector z, and the probability distribution of each category is calculated using the softmax function.

[0106]

[0107] in, Let be the j-th output element, and K be the total number of traffic categories. The final output probability distribution represents the confidence level of each sample under different traffic categories.

[0108] The above anomaly detection model outputs the anomaly detection result corresponding to the fused feature vector. Please refer to the following steps for details:

[0109] Encoding stage: The fused feature vector x is input into the encoder part of the autoencoder. The encoder consists of a multi-layer neural network used to learn a low-dimensional latent representation of the traffic features.

[0110]

[0111] Here, z represents the latent representation, θ are the encoder parameters, and f is the nonlinear mapping function. This process is equivalent to compressing the input features and extracting their core feature patterns.

[0112] Decoding stage: The encoded latent representation z is input to the decoder, which attempts to reconstruct a result from z that is close to the original input.

[0113]

[0114] in, This represents the reconstructed data, where ϕ is the decoder parameter and g is the decoding function. Normal traffic patterns can usually be reconstructed well, while abnormal traffic reconstruction is less effective.

[0115] Reconstruction error calculation: The difference between the input and the reconstruction result is calculated using the mean square error function.

[0116]

[0117] Where n is the number of samples. and These are the i-th elements of the input and the reconstructed elements, respectively.

[0118] Anomaly detection: When the reconstruction error L is lower than the preset threshold, the traffic is considered normal;

[0119] When the reconstruction error L is higher than the preset threshold, the traffic is judged to be abnormal.

[0120] Optionally, in the step of extracting statistical features at different granularities from unencrypted traffic to obtain multi-scale statistical features, protocol decoding can also be performed on the unencrypted traffic to obtain decoded traffic; and statistical features at different granularities can be extracted from the decoded traffic to obtain multi-scale statistical features.

[0121] In this embodiment of the invention, protocol decoding can be performed on unencrypted traffic, parsing the original data packets into structured information according to preset protocol rules to obtain decoded traffic data. Decoding allows for the acquisition of richer and more explicit protocol layer field information, such as IP header fields, TCP / UDP port information, HTTP request / response messages, and DNS query / response messages. These decoded protocol elements provide a more accurate data source for subsequent feature statistics.

[0122] Based on the decoded traffic data, statistical features can be further extracted from different granular dimensions to obtain multi-scale statistical features.

[0123] Specifically, the above protocol decoding process can be referred to in the following steps:

[0124] Protocol identification: First, the protocol type of the collected data packets is identified.

[0125] The Ethernet layer can determine the upper-layer protocol through the EtherType field, for example, 0x0800 indicates the IPv4 protocol; in the IP packet, the protocol type field in the IP header can be used to further distinguish protocol types such as TCP, UDP, and ICMP.

[0126] This step allows you to quickly identify the protocol family to which the data packet belongs, thus initiating the corresponding parsing process.

[0127] TCP protocol decoding: Parsing the TCP header, including fields such as source port, destination port, sequence number, acknowledgment number, window size, and checksum.

[0128] Among them, the port number can identify the specific application, the sequence number and acknowledgment number ensure the reliability and orderliness of the transmission, and the window size reflects the current flow control status of the transmission.

[0129] UDP protocol decoding: Extract the source port, destination port, message length, and checksum field from the UDP header.

[0130] Since UDP is a connectionless protocol, the decoding process is mainly used to identify the application endpoints of the communication and the integrity of the data transmission.

[0131] ICMP protocol decoding: parsing the type, code, and checksum fields of ICMP messages.

[0132] For example, a type value of 8 indicates a Ping request, and a type value of 0 indicates a Ping response. Decoding can help diagnose network connectivity and the cause of errors.

[0133] Application layer protocol decoding: After identifying the upper layer protocol as an application layer protocol (such as HTTP, FTP, etc.), the application layer payload is further parsed.

[0134] For example, the HTTP protocol can extract the request method (GET, POST, etc.), request URL, request header fields, response status code, and response body content.

[0135] This step allows the decoding module to delve into the specific content of application-layer interaction behavior, thereby providing more refined feature information.

[0136] Optionally, in the step of performing feature fusion processing based on multi-scale statistical features to obtain a fused feature vector, dimensionality reduction processing can also be performed based on multi-scale statistical features to obtain a low-dimensional feature vector; a corresponding binary hash code can be generated based on the low-dimensional feature vector; and the binary hash code can be concatenated with the multi-scale statistical features to form a fused feature vector.

[0137] In this embodiment of the invention, multi-scale statistical features obtained at the character level, word level, and sub-word level can be concatenated to form a high-dimensional feature vector. , where d is the feature dimension, representing the number of features contained in vector x, and x is the concatenated multi-scale statistical feature vector.

[0138] To reduce the storage and computational overhead of high-dimensional features while preserving their semantic similarity, they can be mapped to a low-dimensional space using random projection to obtain low-dimensional feature vectors. The calculation formula is as follows: Where z is the reduced-dimensional traffic feature vector, obtained through the above formula, which yields a low-dimensional vector with dimension k. The bias term is used to break the symmetry and improve the model's expressive power. x is the high-dimensional original feature, such as length or port number, and d is the original feature dimension, which needs to be solved by dimensionality reduction. It is a random projection matrix used for dimensionality reduction, that is, to map a d-dimensional high-dimensional feature x to a k-dimensional low-dimensional space.

[0139] Subsequently, the low-dimensional feature vector z is converted into a binary hash code using a sign function. The details are as follows:

[0140]

[0141] in, Let be the i-th hash bit, and k be the hash code length, consistent with the dimension of the low-dimensional vector z. The resulting binary hash code h has the advantages of compact representation and high retrieval efficiency, and can be used to achieve efficient feature indexing and similarity retrieval, thereby reducing storage and computation costs while still preserving the semantic correlation between features.

[0142] Finally, the generated binary hash code h is concatenated with the original multi-scale statistical feature vector x to form the final fused feature vector f.

[0143] Specifically, the similarity between samples can be measured by calculating the similarity of their binary hash codes: Let the hash codes corresponding to the two samples be... Their similarity can be calculated using the following formula:

[0144]

[0145] in, Let f represent the Hamming distance between hash codes, and k be the number of hash bits. To balance retrieval efficiency and discriminative ability, the hash codes can be concatenated with the original multi-scale statistical features to form a fused feature vector f: .

[0146] The fused feature vector f retains the fine-grained representation of the original statistical features and has the ability to quickly calculate similarity in the hash space, which can be directly used for subsequent traffic classification and anomaly detection.

[0147] In summary, by concatenating, reducing the dimensionality of, and semantically hashing multi-scale statistical features, and then concatenating the resulting binary hash code with the original features to form a fused feature vector, storage and computational overhead are significantly reduced while maintaining the discriminativeness of feature representation and the efficiency of similarity calculation. This provides high-quality input features for subsequent traffic classification and anomaly detection.

[0148] Optionally, in the step of determining the traffic detection results of the target network based on the detection results of unencrypted traffic and the detection results of encrypted traffic, the detection results of unencrypted traffic and the detection results of encrypted traffic can be collected and integrated to form a comprehensive result record; the comprehensive result record can be formatted to obtain structured result data in a preset format; the structured result data can be used as the traffic detection result, and the traffic detection result includes at least one of report output and database storage.

[0149] In this embodiment of the invention, after collecting and integrating the detection results of unencrypted traffic and encrypted traffic to form a comprehensive result record, the detection results can also be uniformly organized, formatted and output to provide structured data support for subsequent security analysis and tracing.

[0150] Specifically, the analysis results, such as traffic classification, anomaly detection, and encryption protocol identification, can be collected first. Information such as the classification type (e.g., normal traffic or specific attack type), anomaly status and confidence level, and the identified encryption protocol type for each traffic sample can be integrated according to the traffic sample dimension to form a complete comprehensive result record. To ensure efficient and reliable data transmission, a message queue mechanism can be used, where the analysis process pushes the results to the message queue, and then reads and integrates them uniformly.

[0151] Based on this, the comprehensive results can be formatted and converted into a preset standardized structured format (such as JSON, XML, etc.). The structured results can include fields such as the five-tuple information of the data packets (source IP, destination IP, source port, destination port, protocol type), traffic classification results, anomaly detection results (whether it is anomaly, anomaly type, confidence level), and encryption protocol identification results, making it easier for subsequent parsing, storage, and retrieval.

[0152] To enhance readability and intuitiveness, the formatted structured data can be displayed visually, such as in the form of bar charts, line charts, pie charts, etc., showing the proportion of different types of traffic, the trend of abnormal traffic volume, and the distribution of encryption protocol usage, so that users can intuitively grasp the overall distribution and abnormal patterns of the detection results.

[0153] Ultimately, the structured results data can be output as a report file for manual viewing, or written into a database for subsequent retrieval and correlation analysis, or pushed to other security systems (such as intrusion detection systems, security information and event management systems, etc.) to provide the structured results data as traffic detection results of the target network to users or other systems.

[0154] Optionally, in the step of performing statistical processing on the visible metadata and transmission statistics of encrypted traffic within a preset time window to obtain the window feature vector corresponding to each time window, the visible metadata and transmission statistics of encrypted traffic can also be extracted as window features within the time window; the window features can then be normalized to obtain the window feature vector.

[0155] In this embodiment of the invention, before performing statistical processing on the visible metadata and transmission statistics of encrypted traffic, the continuous encrypted traffic can be divided into preset time windows. The size of each time window can be flexibly set according to actual needs and traffic characteristics, for example, it can be set to 1 second.

[0156] Within each time window, visible metadata (such as source / destination IP address, port number, TLS certificate fingerprint, etc.) and transmission statistics (such as the number of packets, mean, variance, maximum, and minimum packet sizes, mean and variance of packet arrival time intervals, etc.) can be extracted from the data packets within that time period as window features to characterize the overall communication behavior within that time window. For example, when the packet size sequence within the time window is... At that time, the average data packet size can be calculated:

[0157]

[0158] Where n is the number of data packets within the window. It is the size of the nth data packet.

[0159] Considering that the value ranges of different features may vary significantly, to avoid the impact of differences in units on model training, the extracted window features can be normalized, scaling them to a uniform numerical range. Specifically, for any feature value x, it can be normalized based on its minimum value in the training data. and maximum value Normalization is performed to obtain the normalized value:

[0160]

[0161] Through the above processing, a window feature vector with uniform dimensions can be obtained, which can be used as the node feature input model of the corresponding node in the spatiotemporal interaction graph, thereby improving the comparability between features and the stability of model convergence.

[0162] Optionally, in the step of performing continuous-time modeling processing on the spatiotemporal interaction graph based on a preset neural differential model to obtain the detection result of encrypted traffic, the spatiotemporal interaction graph can also be fused based on a spatiotemporal attention mechanism to obtain fused interaction features; the fused interaction features are then provided to the preset neural differential model for continuous-time modeling processing to obtain the detection result of encrypted traffic.

[0163] In this embodiment of the invention, before performing continuous time modeling on the spatiotemporal interaction graph, the interaction information in it can be fused using a spatiotemporal attention mechanism to highlight key dynamic interaction patterns, thereby improving the accuracy and robustness of subsequent modeling.

[0164] Specifically, attention weights can be calculated from both spatial and temporal dimensions, as detailed in the following steps:

[0165] Spatial attention: for any pair of nodes in the graph Based on their node characteristics at the current time t and And the strength of the interaction between the two (i.e., the interaction relationship). Construct the comprehensive feature vector:

[0166]

[0167] Then through a learnable weight matrix Calculate the unnormalized attention score using a non-linear activation function:

[0168]

[0169] Then normalization is performed to obtain the nodes. and its neighboring nodes Spatial attention weights between:

[0170]

[0171] in, For nodes The set of neighboring nodes, Reflects the nodes For nodes In terms of spatial dimension importance, k is a node. The set of neighboring nodes The index of any neighboring node in the array is used for normalization.

[0172] Temporal attention: for a single node Consider its different time steps The characteristics and interactions are analyzed and placed at time steps. Features With the corresponding dynamic adjacency matrix row vector spliced ​​together as:

[0173]

[0174] Using a learnable weight matrix Calculate the unnormalized temporal attention score:

[0175]

[0176] After normalization, the temporal attention weights are obtained:

[0177]

[0178] in, Indicates time step For nodes The influence weights are used to characterize the temporal evolution of its interaction patterns.

[0179] Spatiotemporal attention fusion: incorporating spatial attention weights And time attention weight Together they affect the original interaction strength The interaction strength after fusion is obtained:

[0180]

[0181] The intensity of the fused interaction (i.e., the fused interaction features) Taking into account both the importance of interactions between nodes and their temporal evolution, this approach more accurately reflects the key interaction patterns in the diagram.

[0182] After obtaining the fused interaction features, the interaction features obtained through the spatiotemporal attention mechanism can be used as input to provide a preset neural differential model to continuously model the evolution of the protocol state of encrypted traffic over time, thereby generating the detection results of encrypted traffic.

[0183] Specifically, for each node (representing a network entity, such as a client or server), the feature vector of that node at time t can be... The strength of the fusion and interaction between the node and its neighboring nodes is also integrated to form a node. Protocol state vector at time t The specific algorithm is as follows:

[0184]

[0185] in, Used to characterize the node's own statistical and protocol-specific features (such as mean packet length, time interval variance, TLS version, SNI field, etc.). To reflect the interaction strength after incorporating spatiotemporal attention, reflecting the node Its neighboring nodes The importance of interaction between them For nodes Let j be the set of neighboring nodes of node i, where j represents the neighboring node adjacent to node i.

[0186] The global protocol state s(t) of the entire spatiotemporal interaction graph at time t is obtained by concatenating the protocol state vectors of all nodes. Then, a continuous-time model of the protocol state is performed using neural differential equations to describe the dynamic evolution of the protocol state over time.

[0187]

[0188] in, θ is a function implemented by a neural network used to learn the changing patterns of the protocol state, where θ is its trainable parameter and t is the time variable.

[0189] Known initial protocol state Under the premise of this, the protocol state vector at any time point is obtained by iteratively solving using numerical integration methods, specifically as follows:

[0190]

[0191] Here, Δt is the time step, used to control the solution accuracy and computational cost. Through this continuous-time modeling process, a smooth evolution trajectory of the protocol state over time can be obtained, and potential malicious communication behaviors can be identified based on the abnormal patterns that appear during the evolution process, thereby outputting accurate encrypted traffic detection results.

[0192] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0193] In one embodiment, a flow detection device is provided, which corresponds one-to-one with the flow detection method in the above embodiments. As shown in FIG2, the flow detection device includes a first acquisition module 201, a first statistics module 202, a first construction module 203, a first modeling module 204, and a first determination module 205. Detailed descriptions of each functional module are as follows:

[0194] The first acquisition module 201 is used to acquire network traffic data in the target network, the network traffic data including encrypted traffic;

[0195] The first statistics module 202 is used to perform statistical processing within a preset time window based on the visible metadata and transmission statistics information of the encrypted traffic, and obtain the window feature vector corresponding to each time window.

[0196] The first construction module 203 is used to construct a spatiotemporal interaction graph within each preset time window, using communication entities in encrypted traffic as nodes, the window feature vector as the node feature of the corresponding node, and the interaction relationship between communication entities within the time window as edges.

[0197] The first modeling module 204 is used to perform continuous-time modeling processing on the spatiotemporal interaction graph based on a preset neural differential model to obtain the detection result of the encrypted traffic;

[0198] The first determining module 205 is used to determine the traffic detection result of the target network based on the detection result of the encrypted traffic.

[0199] Optionally, the network traffic data further includes unencrypted traffic, and the first determining module 205 is further configured to:

[0200] Statistical features at different granularities are extracted from the unencrypted traffic to obtain multi-scale statistical features. In the multi-scale statistical features, the granularity of each scale is different from that of other scales.

[0201] Based on the aforementioned multi-scale statistical features, feature fusion processing is performed to obtain a fused feature vector;

[0202] The fused feature vector is provided to a preset traffic classification model so that the traffic classification model outputs the traffic classification result corresponding to the fused feature vector;

[0203] The fused feature vector is provided to a preset anomaly detection model so that the anomaly detection model outputs the anomaly detection result corresponding to the fused feature vector;

[0204] The detection result of the unencrypted traffic is determined based on the traffic classification result and the anomaly detection result;

[0205] Based on the detection results of the unencrypted traffic and the detection results of the encrypted traffic, the traffic detection results of the target network are determined.

[0206] Optionally, the first determining module 205 is further configured to:

[0207] The unencrypted traffic is then subjected to protocol decoding to obtain the decoded traffic.

[0208] Statistical features at different granularities are extracted from the decoded traffic to obtain multi-scale statistical features.

[0209] Optionally, the first determining module 205 is further configured to:

[0210] Dimensionality reduction is performed based on the multi-scale statistical features to obtain a low-dimensional feature vector.

[0211] Generate the corresponding binary hash code based on the low-dimensional feature vector;

[0212] The binary hash code is concatenated with the multi-scale statistical features to form the fused feature vector.

[0213] Optionally, the first determining module 205 is further configured to:

[0214] The detection results of the unencrypted traffic and the detection results of the encrypted traffic are collected and integrated to form a comprehensive result record;

[0215] The results of the synthesis are formatted to obtain structured result data in a preset format;

[0216] The structured result data is used as the traffic detection result, which includes at least one of report output and database storage.

[0217] Optionally, the first statistics module 202 is further configured to:

[0218] Within the time window, visible metadata and transmission statistics of the encrypted traffic are extracted as window features;

[0219] The window features are normalized to obtain the window feature vector.

[0220] Optionally, the first modeling module 204 is further configured to:

[0221] The spatiotemporal interaction graph is subjected to feature fusion processing based on a spatiotemporal attention mechanism to obtain fused interaction features.

[0222] The fused interactive features are provided to a preset neural differential model for continuous-time modeling to obtain the detection results of the encrypted traffic.

[0223] Specific limitations regarding the flow detection device can be found in the limitations of the flow detection method described above, and will not be repeated here. Each module in the aforementioned flow detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0224] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram is shown in Figure 3. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a readable storage medium. The readable storage medium stores computer-readable instructions. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer-readable instructions are executed by the processor, a traffic detection method is implemented. The readable storage medium provided in this embodiment includes both non-volatile readable storage media and volatile readable storage media.

[0225] In this application embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it implements the steps of the traffic detection method described above.

[0226] In one embodiment of the application, a readable storage medium is provided, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, they implement the steps of the traffic detection method described above.

[0227] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0228] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0229] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A flow rate detection method, characterized in that, The method includes: collecting network traffic data in a target network, the network traffic data including encrypted traffic; performing statistical processing on the visible metadata and transmission statistics of the encrypted traffic within a preset time window to obtain a window feature vector corresponding to each time window; within each preset time window, using communication entities in the encrypted traffic as nodes, using the window feature vector as the node feature of the corresponding node, and using the interaction relationship between communication entities within the time window as edges to construct a spatiotemporal interaction graph; performing continuous-time modeling processing on the spatiotemporal interaction graph based on a preset neural differential model to obtain the detection result of the encrypted traffic; determining the traffic detection result of the target network based on the detection result of the encrypted traffic; the network traffic data also includes unencrypted traffic, and the determination of the traffic detection result of the target network based on the detection result of the encrypted traffic... The traffic detection results of the target network include: extracting statistical features at different granularities from the unencrypted traffic to obtain multi-scale statistical features, wherein the granularity of each scale in the multi-scale statistical features is different from that of other scales; performing feature fusion processing based on the multi-scale statistical features to obtain a fused feature vector; providing the fused feature vector to a preset traffic classification model so that the traffic classification model outputs the traffic classification result corresponding to the fused feature vector; providing the fused feature vector to a preset anomaly detection model so that the anomaly detection model outputs the anomaly detection result corresponding to the fused feature vector; determining the detection result of the unencrypted traffic based on the traffic classification result and the anomaly detection result; and determining the traffic detection result of the target network based on the detection result of the unencrypted traffic and the detection result of the encrypted traffic.

2. The flow detection method as described in claim 1, characterized in that, The step of extracting statistical features at different granularities from the unencrypted traffic to obtain multi-scale statistical features includes: performing protocol decoding processing on the unencrypted traffic to obtain decoded traffic; and extracting statistical features at different granularities from the decoded traffic to obtain multi-scale statistical features.

3. The flow detection method as described in claim 1, characterized in that, The step of performing feature fusion processing based on the multi-scale statistical features to obtain a fused feature vector includes: performing dimensionality reduction processing based on the multi-scale statistical features to obtain a low-dimensional feature vector; generating a corresponding binary hash code based on the low-dimensional feature vector; and concatenating the binary hash code with the multi-scale statistical features to form the fused feature vector.

4. The flow detection method as described in claim 1, characterized in that, The step of determining the traffic detection result of the target network based on the detection results of the unencrypted traffic and the detection results of the encrypted traffic includes: collecting and integrating the detection results of the unencrypted traffic and the detection results of the encrypted traffic to form a comprehensive result record; formatting the comprehensive result record to obtain structured result data in a preset format; and using the structured result data as the traffic detection result, wherein the traffic detection result includes at least one of report output and database storage.

5. The flow detection method according to any one of claims 1 to 4, characterized in that, The step of performing statistical processing on the visible metadata and transmission statistics of the encrypted traffic within a preset time window to obtain a window feature vector corresponding to each time window includes: extracting the visible metadata and transmission statistics of the encrypted traffic as window features within the time window; and normalizing the window features to obtain the window feature vector.

6. The flow detection method according to any one of claims 1 to 4, characterized in that, The step of performing continuous-time modeling processing on the spatiotemporal interaction graph based on a preset neural differential model to obtain the detection result of the encrypted traffic includes: performing feature fusion processing on the spatiotemporal interaction graph based on a spatiotemporal attention mechanism to obtain fused interaction features; and providing the fused interaction features to the preset neural differential model for continuous-time modeling processing to obtain the detection result of the encrypted traffic.

7. A flow detection device, characterized in that, The device includes: a first acquisition module for acquiring network traffic data in a target network, the network traffic data including encrypted traffic; a first statistics module for performing statistical processing within a preset time window based on the visible metadata and transmission statistics of the encrypted traffic to obtain a window feature vector corresponding to each time window; a first construction module for constructing a spatiotemporal interaction graph within each preset time window, using communication entities in the encrypted traffic as nodes, the window feature vector as the node features of the corresponding nodes, and the interaction relationships between communication entities within the time window as edges; a first modeling module for performing continuous-time modeling processing on the spatiotemporal interaction graph based on a preset neural differential model to obtain the detection result of the encrypted traffic; and a first determination module for determining the traffic detection result of the target network based on the detection result of the encrypted traffic. The network traffic data also includes unencrypted traffic. The first determining module is further configured to: extract statistical features at different granularities from the unencrypted traffic to obtain multi-scale statistical features, wherein the granularity of each scale in the multi-scale statistical features is different from the granularity of other scales; perform feature fusion processing based on the multi-scale statistical features to obtain a fused feature vector; provide the fused feature vector to a preset traffic classification model so that the traffic classification model outputs the traffic classification result corresponding to the fused feature vector; provide the fused feature vector to a preset anomaly detection model so that the anomaly detection model outputs the anomaly detection result corresponding to the fused feature vector; determine the detection result of the unencrypted traffic based on the traffic classification result and the anomaly detection result; and determine the traffic detection result of the target network based on the detection result of the unencrypted traffic and the detection result of the encrypted traffic.

8. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and running on the processor, characterized in that, When the processor executes the computer-readable instructions, it implements the traffic detection method as described in any one of claims 1 to 6.

9. A readable storage medium having computer-readable instructions stored thereon, characterized in that, When the computer-readable instructions are executed by a processor, they implement the traffic detection method as described in any one of claims 1 to 6.

Images