Traffic detection method and device, computer equipment and storage medium

Abstract

The invention discloses a traffic detection method and device, computer equipment and a storage medium. The method comprises the following steps: collecting network traffic data in a target network; based on the visible metadata of the encrypted traffic and the transmission statistical information, performing statistical processing in a preset time window to obtain a window feature vector corresponding to each time window; in each preset time window, taking the communication entities in the encrypted traffic as nodes, taking the window feature vectors as node features of the corresponding nodes, and taking interaction relationships between the communication entities in the time window as edges to construct a space-time interaction diagram; performing continuous time modeling processing on the space-time interaction diagram based on a preset neural differential model to obtain a detection result of the encrypted traffic; and determining a traffic detection result of the target network based on the detection result of the encrypted traffic. Encrypted traffic is detected through a space-time interaction graph and neural differential modeling, and the detection accuracy and robustness are improved while privacy is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of network security, and in particular to a traffic detection method and device, a computer device and a storage medium. BACKGROUND

[0002] In the field of network security, existing traffic detection methods mostly rely on traditional deep packet inspection (DPI), port identification or signature-based detection techniques. Such methods usually require parsing the complete content of network packets to identify protocol types or discover abnormal behavior. However, with the increasing proportion of network communication encryption, more and more traffic is transmitted using encrypted protocols such as TLS, QUIC, SSH, etc., making it impossible for traditional methods to directly obtain plaintext data features, and the detection effect is significantly reduced. In addition, some traffic analysis methods based on statistical features or machine learning often rely on discrete time step sequence modeling (such as LSTM), which is difficult to accurately capture the complex timing dependency between communication entities when facing high dynamic and non-stationary encrypted traffic, resulting in insufficient stability of the detection results and high false alarm rate.

[0003] Therefore, the existing traffic detection methods have deficiencies in detection accuracy and robustness when facing encrypted traffic, and there is an urgent need to propose a traffic detection method that can improve the accuracy and reliability of network traffic detection while ensuring the privacy of encrypted traffic. SUMMARY

[0004] Therefore, it is necessary to provide a traffic detection method, device, computer device and storage medium to solve the problem of low detection accuracy and robustness of traditional traffic detection methods.

[0005] A traffic detection method, the method comprising: collecting network traffic data in a target network, the network traffic data including encrypted traffic; based on the visible metadata and transmission statistics of the encrypted traffic, performing statistical processing within a preset time window to obtain a window feature vector corresponding to each time window; in each preset time window, taking a communication entity in the encrypted traffic as a node, taking the window feature vector as a node feature of the corresponding node, and taking the interaction relationship between the communication entities within the time window as an edge, to construct a space-time interaction graph; based on a preset neural differential model, performing continuous time modeling processing on the space-time interaction graph to obtain a detection result of the encrypted traffic; based on the detection result of the encrypted traffic, determining a traffic detection result of the target network.

[0006] Optionally, the network traffic data further comprises non-encrypted traffic, and the determining of the traffic detection result of the target network based on the detection result of the encrypted traffic comprises: extracting statistical features of different granularities from the non-encrypted traffic to obtain multi-scale statistical features, wherein the granularity of each scale is different from that of other scales; performing feature fusion processing based on the multi-scale statistical features to obtain a fusion feature vector; providing the fusion feature vector to a preset traffic classification model to enable the traffic classification model to output a traffic classification result corresponding to the fusion feature vector; providing the fusion feature vector to a preset anomaly detection model to enable the anomaly detection model to output an anomaly detection result corresponding to the fusion feature vector; determining the detection result of the non-encrypted traffic based on the traffic classification result and the anomaly detection result; determining the traffic detection result of the target network based on the detection result of the non-encrypted traffic and the detection result of the encrypted traffic.

[0007] Optionally, the extracting of statistical features of different granularities from the non-encrypted traffic to obtain multi-scale statistical features comprises: performing protocol decoding processing based on the non-encrypted traffic to obtain decoded traffic; extracting statistical features of different granularities from the decoded traffic to obtain multi-scale statistical features.

[0008] Optionally, the performing of feature fusion processing based on the multi-scale statistical features to obtain a fusion feature vector comprises: performing dimensionality reduction processing based on the multi-scale statistical features to obtain a low-dimensional feature vector; generating a corresponding binary hash code based on the low-dimensional feature vector; concatenating the binary hash code and the multi-scale statistical features to form the fusion feature vector.

[0009] Optionally, the determining of the traffic detection result of the target network based on the detection result of the non-encrypted traffic and the detection result of the encrypted traffic comprises: collecting and integrating the detection result of the non-encrypted traffic and the detection result of the encrypted traffic to form a comprehensive result record; performing formatting processing on the comprehensive result record to obtain structured result data in a preset format; using the structured result data as the traffic detection result, wherein the traffic detection result comprises at least one of report output and database storage.

[0010] Optionally, the visible metadata and transmission statistics of the encrypted traffic are statistically processed in a preset time window to obtain a window feature vector corresponding to each time window, including: extracting the visible metadata and transmission statistics of the encrypted traffic in the time window as window features; normalizing the window features to obtain the window feature vector.

[0011] Optionally, the spatio-temporal interaction graph is continuously time modeled based on a preset neural differential model to obtain the detection result of the encrypted traffic, including: performing feature fusion processing on the spatio-temporal interaction graph based on a spatio-temporal attention mechanism to obtain fused interaction features; providing the fused interaction features to a preset neural differential model for continuous time modeling processing to obtain the detection result of the encrypted traffic.

[0012] A traffic detection device, the device comprising: a first acquisition module configured to acquire network traffic data in a target network, the network traffic data including encrypted traffic; a first statistical module configured to statistically process the visible metadata and transmission statistics of the encrypted traffic in a preset time window to obtain a window feature vector corresponding to each time window; a first construction module configured to, in each preset time window, take a communication entity in the encrypted traffic as a node, take the window feature vector as a node feature of the corresponding node, and take an interaction relationship between the communication entities in the time window as an edge, to construct a spatio-temporal interaction graph; a first modeling module configured to continuously time model the spatio-temporal interaction graph based on a preset neural differential model to obtain a detection result of the encrypted traffic; a first determination module configured to determine a traffic detection result of the target network based on the detection result of the encrypted traffic.

[0013] A computer device comprising a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, the processor implementing the above traffic detection method when executing the computer readable instructions.

[0014] A readable storage medium having computer readable instructions stored thereon, the computer readable instructions being executed by a processor to implement the traffic detection method.

[0015] The traffic detection method, device, computer device and storage medium collect network traffic data in a target network, the network traffic data including encrypted traffic; based on visible metadata and transmission statistical information of the encrypted traffic, statistical processing is performed in a preset time window to obtain a window feature vector corresponding to each time window; in each preset time window, a communication entity in the encrypted traffic is taken as a node, the window feature vector is taken as a node feature of the corresponding node, and an interaction relationship between the communication entities in the time window is taken as an edge to construct a space-time interaction graph; based on a preset neural differential model, continuous time modeling processing is performed on the space-time interaction graph to obtain a detection result of the encrypted traffic; and based on the detection result of the encrypted traffic, a traffic detection result of the target network is determined. By extracting the visible metadata and transmission statistical information of the encrypted traffic in the time window and constructing the space-time interaction graph, and then combining the neural differential model to perform continuous time modeling processing, the encrypted traffic can be accurately detected without decrypting the traffic content, so that the complex timing and interaction features can be better captured while the user privacy is protected, and the detection accuracy and robustness are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 is a flowchart of a traffic detection method in an embodiment of the present application; Figure 2 is a structural schematic diagram of a traffic detection device in an embodiment of the present application; Figure 3 is a schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0019] In an embodiment, as shown in Figure 1 , a traffic detection method is provided, including the following steps: 101. Collecting network traffic data in a target network.

[0020] In the embodiments of the present application, the collection described above can be realized by a traffic collection module. The target network described above can be an enterprise internal LAN, a data center network, a carrier backbone network, or an Internet of Things terminal access network, etc., which can cover various actual application scenarios.

[0021] The network traffic data described above can be complete network communication data packets from the data link layer (L2 layer) to the application layer (L7 layer), including the header information, metadata information, and visible transmission statistical information of the data packets, etc. In order to ensure the comprehensiveness and real-time performance of the collection, the traffic collection module described above can be realized based on the libpcap library, and in the Linux system environment, the pcap_open_live() function is used to specify the network interface (such as “eth0”) to be captured, and a handle is returned for subsequent operations.

[0022] During the collection process, the filter rules can be set by the pcap_compile() and pcap_setfilter() functions, such as capturing only IP packets (filter expression “ip”), so as to reduce unnecessary data processing amount. Subsequently, the pcap_loop() or pcap_dispatch() function is used for loop capture, and whenever a data packet arrives at the network interface, the preset callback function will be called and receive the data packet, and then the data packet is preliminarily sorted and cached.

[0023] In order to cope with the massive traffic in the high-speed network environment, the traffic collection module can adopt a multi-thread architecture: one thread is responsible for data packet capture, and the captured data packets are written into a shared buffer; another thread reads the data packets from the buffer and transmits them to the subsequent processing module. By reasonably configuring the buffer size, the loss of data packets due to buffer overflow can be effectively reduced, so as to ensure the integrity and reliability of the traffic collection.

[0024] The network traffic data described above includes encrypted traffic, and the encrypted traffic can be data packets transmitted by using TLS, QUIC, SSH, etc. encryption protocol, and the payload part is ciphertext and cannot be directly parsed, but still can be analyzed and detected by the visible metadata and transmission statistical information (such as data packet length, arrival time interval, source and destination port number, etc.).

[0025] 102, based on the visible metadata and transmission statistical information of the encrypted traffic, statistical processing is performed within a preset time window to obtain a window feature vector corresponding to each time window.

[0026] In the embodiments of the present application, the visible metadata can be protocol message fields of the encrypted traffic that can still be observed during transmission, such as source IP address, destination IP address, source port number, destination port number, transmission protocol type, TLS version information, or certificate fingerprint, etc. The transmission statistical information can be a statistical quantity based on the packet level, such as the number of packets in a time window, the packet length distribution (such as mean, variance, maximum, minimum), the arrival time interval distribution of adjacent packets, and the uplink / downlink traffic ratio, etc. The preset time window can be a time interval based on a fixed time length or a sliding time length, such as a time period of 500ms, 1s, or 5s, to ensure statistical stability while considering real-time performance, which can be adjusted according to actual needs and traffic characteristics.

[0027] Specifically, by organizing and counting the visible metadata and transmission statistical information of the encrypted traffic in each preset time window, a window feature vector representing the communication behavior characteristics in the above time window can be obtained, which provides input for subsequent interaction modeling and detection analysis.

[0028] 103、In each preset time window, the communication entities in the encrypted traffic are taken as nodes, the window feature vector is taken as the node feature of the corresponding node, and the interaction relationship between the communication entities in the time window is taken as the edge, to construct a space-time interaction graph.

[0029] In the embodiments of the present application, the construction of the space-time interaction graph in each time window can be completed according to the following steps: Node definition: taking network entities as graph nodes The network entities can include at least one of IP address, port number, and certificate subject / fingerprint. For each node, the obtained window feature vector of the entity in the current time window is assigned to the node as the node feature, to represent the statistical and behavior characteristics of the entity in the time window.

[0030] Edge definition and interaction judgment: for any two nodes and , check whether there is communication interaction (such as TCP connection, DNS query-response, etc.) between them in the time window [t-Δt, t]. Let and be the packet set between the nodes in the time window, where t represents the time point, If , it is determined that there is interaction, an edge is established, and the edge weight is calculated to reflect the interaction intensity:

[0031] wherein, denotes the number of bytes of the data packet p; if , the edge is not established, or the edge weight is regarded as 0.

[0032] Dynamic adjacency matrix construction: finally, according to the edge set and the interaction intensity , a time-dependent adjacency matrix is constructed . An N*N zero matrix (N is the number of nodes in the node set V) is initialized, and then each edge is assigned a value:

[0033] The node pair that does not interact remains 0. The time-dependent adjacency matrix is defined as:

[0034] wherein, the rows and columns of the matrix correspond to the nodes in the node set V, and the element value represents the interaction intensity between the nodes in the time window.

[0035] Precedence and dependency relationship: first, node definition is performed to clarify the basic elements of the graph; on this basis, edge definition is performed and the interaction intensity is calculated; finally, the dynamic adjacency matrix is constructed according to the edge set and the intensity thereof. The edge definition depends on the result of the node definition, and the construction of the adjacency matrix depends on the edge set and the interaction intensity information. Thus, in each time window, a space-time interaction graph containing “node set V-edge set -dynamic adjacency matrix -node characteristics (window feature vector)” is obtained, and can be updated in a rolling manner with the time window to depict the space-time evolution of the encrypted traffic protocol behavior.

[0036] In summary, the construction of the space-time interaction graph fully maps the behavior characteristics and interaction intensity of the communication entities in the encrypted traffic in the time dimension through the step-by-step definition process of “node-edge-adjacency matrix”, and realizes the dynamic representation of the state evolution of the encrypted traffic protocol. This method not only can finely depict the interaction mode between the communication entities, but also can continuously update in the rolling process of the time window, so as to effectively capture the potential malicious communication characteristics, provide structured and time-sequenced data support for subsequent continuous time modeling based on the neural differential model, and significantly improve the detection ability of complex attack behaviors in encrypted traffic.

[0037] 104. Perform continuous time modeling processing on the space-time interaction graph based on a preset neural differential model, to obtain a detection result of the encrypted traffic.

[0038] In the embodiment of the present application, the neural differential model is used to map the node features and edge interaction features in the spatio-temporal interaction graph into the dynamic evolution process of the protocol state.

[0039] Specifically, first, the node features and interaction relationships constructed by the spatio-temporal interaction graph can be directly input into the neural differential model. The model models the time sequence features of encrypted traffic through a differential equation form of continuous time, which can capture the dynamic rules of the change of communication patterns over time. By analyzing the protocol state output by the model, potential abnormal behaviors or malicious communications can be identified, thereby generating a detection result of the encrypted traffic.

[0040] This processing method can avoid the shortcomings of traditional discrete modeling in capturing sudden or long-distance dependence relationships, achieve fine-grained description of encrypted traffic behavior, and provide high timeliness and high precision discrimination basis for subsequent traffic security analysis and attack detection.

[0041] Alternatively, the spatio-temporal interaction graph can also be processed for feature fusion based on a spatio-temporal attention mechanism to obtain fused interaction features, and then the fused interaction features are provided to a preset neural differential model for continuous time modeling processing to obtain a detection result of the encrypted traffic.

[0042] 105、Based on the detection result of the encrypted traffic, a traffic detection result of the target network is determined.

[0043] In the embodiment of the present application, in the case that all the traffic in the target network is encrypted traffic, the detection result of the encrypted traffic can be directly used as the traffic detection result of the target network, achieving rapid discrimination.

[0044] When there are both encrypted traffic and non-encrypted traffic in the traffic data of the target network, the detection result of the non-encrypted traffic and the detection result of the encrypted traffic can be further collected, integrated and analyzed to form a comprehensive traffic detection result of the target network. This comprehensive detection result can reflect the potential malicious communication in the encrypted traffic and also contain the abnormal behavior features in the non-encrypted traffic, thereby comprehensively improving the perception ability of the security state of the target network.

[0045] In the embodiment of the present application, network traffic data is collected in a target network, the network traffic data including encrypted traffic; based on the visible metadata and transmission statistics of the encrypted traffic, statistical processing is performed within a preset time window to obtain a window feature vector corresponding to each time window; within each preset time window, a communication entity in the encrypted traffic is taken as a node, the window feature vector is taken as a node feature of the corresponding node, and an interaction relationship between communication entities within the time window is taken as an edge, to construct a space-time interaction graph; based on a preset neural differential model, continuous time modeling processing is performed on the space-time interaction graph to obtain a detection result of the encrypted traffic; and based on the detection result of the encrypted traffic, a traffic detection result of the target network is determined. By extracting the visible metadata and transmission statistics of the encrypted traffic within the time window and constructing the space-time interaction graph, and then combining the neural differential model for continuous time modeling processing, accurate detection of the encrypted traffic can be achieved without decrypting the traffic content, so that the complex timing and interaction features can be better captured while protecting user privacy, and the accuracy and robustness of the detection are significantly improved.

[0046] It can be understood that in the specific embodiments of the present application, data related to network traffic data, encrypted traffic, non-encrypted traffic, etc. are involved, and when the embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of related data and the construction and use of the neural differential model need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0047] Optionally, the network traffic data further includes non-encrypted traffic, and in the step of determining the traffic detection result of the target network based on the detection result of the encrypted traffic, statistical features of different granularities can also be extracted from the non-encrypted traffic to obtain multi-scale statistical features, the granularity of each scale being different from that of other scales; feature fusion processing is performed based on the multi-scale statistical features to obtain a fusion feature vector; the fusion feature vector is provided to a preset traffic classification model, so that the traffic classification model outputs a traffic classification result corresponding to the fusion feature vector; the fusion feature vector is provided to a preset anomaly detection model, so that the anomaly detection model outputs an anomaly detection result corresponding to the fusion feature vector; a detection result of the non-encrypted traffic is determined based on the traffic classification result and the anomaly detection result; and a traffic detection result of the target network is determined based on the detection result of the non-encrypted traffic and the detection result of the encrypted traffic.

[0048] In the embodiments of the present application, the network traffic data can also include non-encrypted traffic in addition to encrypted traffic. For non-encrypted traffic, statistical features of different granularities can be further extracted to obtain multi-scale statistical features. The multi-scale statistical features can depict the traffic content characteristics at three different levels of character level, word level and subword level, for example, character-level statistics can include character distribution and character entropy to measure the randomness of the traffic; word-level statistics can be calculated by position-weighted TF-IDF, considering the position information of the word in the traffic; subword-level statistics can be obtained by byte pair encoding (BPE) to reflect the multidimensional feature distribution of the traffic.

[0049] After obtaining the multi-scale statistical features, the feature fusion can be further combined with semantic hashing coding and nonlinear dimension reduction technology to form a fusion feature vector. The fusion feature vector can not only maintain the semantic similarity between features, but also reduce the storage and calculation complexity, thereby improving the analysis efficiency while improving the discrimination of traffic representation.

[0050] In the classification and detection stage, the fusion feature vector can be input into different deep learning models: on the one hand, it is provided to a preset convolutional neural network classification model, and the traffic feature pattern is extracted by convolution operation and pooling operation, and finally the classification result of the traffic is output by the softmax function, such as distinguishing normal traffic and various known attack traffic; on the other hand, the fusion feature vector is provided to an anomaly detection model based on autoencoder, and the reconstruction error is minimized to determine whether the traffic is abnormal, and if the reconstruction error exceeds the threshold, it is determined as abnormal traffic.

[0051] Finally, the traffic classification result output by the classification model and the anomaly detection result output by the anomaly detection model are fused to obtain the detection result of the non-encrypted traffic; and the detection result of the non-encrypted traffic and the detection result of the encrypted traffic are integrated to obtain the overall traffic detection result of the target network. In this way, not only can the potential threat of encrypted traffic be detected under the condition of protecting privacy, but also the non-encrypted traffic can be comprehensively analyzed through fine-grained feature extraction and deep learning model, so as to realize high-precision detection of the overall traffic of the target network.

[0052] Specifically, the statistical extraction of the above multi-scale statistical features can refer to the following steps: Character-level statistics: statistical analysis of the decoded text content of non-encrypted traffic at the character level. Including calculating the frequency distribution of different character types in the text (such as the proportion of uppercase letters, numbers, and special symbols), and calculating the character entropy Used to measure the randomness of the overall distribution:

[0053] where C is the character set, and p(c) is the probability of character c appearing. The character entropy can reflect the regularity and complexity of the traffic content.

[0054] Word-level statistics: On the basis of traditional TF-IDF, a position weighting mechanism is introduced to obtain position weighted TF-IDF (PW-TF-IDF) to consider the position information of words in traffic content (for example, the weight is higher at the head or tail of the data packet). The calculation formula is:

[0055] wherein, is the normalized position of word w in text D (the value range is 0-1), is the position weight coefficient (the empirical value is 0.5). By introducing the position information, the protocol content and potential attack payload can be more accurately distinguished.

[0056] Sub-word level statistics: The byte pair encoding (BPE) method is used to disassemble the traffic text into sub-word units, and the frequency distribution and co-occurrence relationship of the sub-words are counted. For example, “malware” is disassembled into “mal” and “ware”, and the co-occurrence times are counted. Sub-word level statistics can capture potential transformation characteristics in attack payloads, and improve the robustness of detection on obfuscated or variant attacks.

[0057] The traffic classification model outputs the fusion feature vector corresponding to the traffic classification result, which can refer to the following steps: Feature vector input: The fusion feature vector is arranged into a feature matrix X, and the size is m*n, where m is the feature dimension, and n is the sample number. The matrix is used as the input of the convolutional neural network (CNN).

[0058] Convolution layer processing: a plurality of convolution kernels The input matrix X is convoluted to obtain the feature map

[0059]

[0060] wherein, represents convolution operation, is the bias term is the activation function, usually ReLU:

[0061] The convolution operation can extract the local pattern in the traffic feature, and the weight sharing mechanism improves the calculation efficiency.

[0062] Pooling and feature extraction: Apply pooling operations on the feature maps output by the convolutional layers to downsample the features, reduce redundancy, and improve the robustness of the model. After multiple layers of convolution and pooling, the high-order features obtained are flattened into a one-dimensional vector.

[0063] Fully connected layer and classification output: Input the flattened feature vector into the fully connected layer to obtain the output vector z, and calculate the probability distribution of each class by the softmax function:

[0064] where, is the jth output element, and K is the total number of traffic classes. The final output probability distribution is the confidence of each sample under different traffic classes.

[0065] The above anomaly detection model outputs the anomaly detection result corresponding to the fusion feature vector, which can be specifically referred to the following steps: Encoding stage: Input the fusion feature vector x into the encoder part of the autoencoder. The encoder is composed of multiple layers of neural networks, which is used to learn the low-dimensional latent representation of traffic features:

[0066] where z represents the latent representation, θ is the encoder parameter, and f is the nonlinear mapping function. This process is equivalent to compressing the input features and extracting their core feature patterns.

[0067] Decoding stage: Input the encoded latent representation z into the decoder, which attempts to reconstruct a result close to the original input from z:

[0068] where, represents the reconstructed data, ϕ is the decoder parameter, and g is the decoding function. The patterns of normal traffic can be well reconstructed, while the reconstruction effect of abnormal traffic is poor.

[0069] Reconstruction error calculation: Calculate the difference between the input and the reconstructed result by the mean square error function:

[0070] where n is the number of samples, and are the ith elements of the input and reconstruction, respectively.

[0071] Anomaly determination: When the reconstruction error L is lower than the preset threshold, the traffic is determined to be normal; When the reconstruction error L is higher than the preset threshold, the traffic is determined to be abnormal.

[0072] Optionally, in the step of extracting statistical features of different granularities from the non-encrypted traffic to obtain multi-scale statistical features, protocol decoding processing can also be performed based on the non-encrypted traffic to obtain decoded traffic; and statistical features of different granularities are extracted from the decoded traffic to obtain multi-scale statistical features.

[0073] In the embodiment of the application, protocol decoding processing can be performed on the non-encrypted traffic, and the original data packet is parsed into structured information according to a preset protocol rule to obtain decoded traffic data. Through decoding, more abundant and more explicit protocol layer field information can be obtained, such as IP header field, TCP / UDP port information, HTTP request / response message, DNS query / response message, etc. These decoded protocol elements provide more accurate data sources for subsequent feature statistics.

[0074] On the basis of the decoded traffic data, statistical features can be further extracted from different granularities to obtain multi-scale statistical features.

[0075] Specifically, the above protocol decoding processing can refer to the following steps: Protocol identification: first, the protocol type of the collected data packet is identified.

[0076] The Ethernet layer can determine the upper layer protocol through the EtherType field, for example, 0x0800 represents the IPv4 protocol; in the IP data packet, the protocol type field of the IP header can be used to further distinguish TCP, UDP, ICMP and other protocol types.

[0077] Through this step, the protocol family to which the data packet belongs can be quickly locked, so as to enter the corresponding analysis process.

[0078] TCP protocol decoding: the TCP header is parsed, including the source port, destination port, sequence number, acknowledgement number, window size, checksum and other fields.

[0079] Among them, the port number can identify the specific application, the sequence number and the acknowledgement number ensure the reliability and orderliness of transmission, and the window size reflects the current flow control state.

[0080] UDP protocol decoding: the source port, destination port, message length and checksum fields of the UDP header are extracted.

[0081] Since UDP is a connectionless protocol, the decoding process is mainly used to identify the application end point of communication and the integrity of data transmission.

[0082] ICMP protocol decoding: the type, code and checksum fields of the ICMP message are parsed.

[0083] For example, type value 8 represents a ping request, and type value 0 represents a ping response. Decoding can assist in diagnosing network connectivity and error causes.

[0084] Application layer protocol decoding: after identifying that the upper layer protocol is an application layer protocol (such as HTTP, FTP, etc.), the application layer load is further parsed.

[0085] For example, in the HTTP protocol, the request method (GET, POST, etc.), request URL, request header field, response status code, and response body content can be extracted.

[0086] This step enables the decoding module to delve into the specific content of the application layer interaction behavior, thereby providing more detailed feature information.

[0087] Optionally, in the step of performing feature fusion processing based on the multi-scale statistical features to obtain a fusion feature vector, dimension reduction processing can also be performed based on the multi-scale statistical features to obtain a low-dimensional feature vector; a corresponding binary hash code is generated based on the low-dimensional feature vector; and the binary hash code and the multi-scale statistical features are spliced to form the fusion feature vector.

[0088] In the embodiments of the application, the multi-scale statistical features obtained at the character level, word level and subword level can be spliced to form a high-dimensional feature vector , where d is the feature dimension, representing the number of features contained in the vector x, and x is the spliced multi-scale statistical feature vector.

[0089] In order to reduce the storage and calculation overhead of high-dimensional features while preserving their semantic similarity, they can be mapped to a low-dimensional space by random projection to obtain a low-dimensional feature vector , the calculation formula of which is: , where z is the dimension-reduced traffic feature vector, and the low-dimensional vector is obtained by the above formula, with a dimension of k. is a bias term used to break symmetry and improve model expression ability, x is the original high-dimensional feature such as length and port number, and d is the original feature dimension, which needs to be reduced to solve the high-dimensional problem. is a random projection matrix used for dimension reduction, i.e., for mapping the d-dimensional high-dimensional feature x to a k-dimensional low-dimensional space.

[0090] Subsequently, the low-dimensional feature vector z is converted into a binary hash code by a sign function, as follows:

[0091] , where For the i-th hash bit, k is the length of the hash code, which is consistent with the dimension of the low-dimensional vector z. The obtained binary hash code h has the advantages of compact representation and high retrieval efficiency, and can be used to realize efficient indexing and similarity retrieval of features, thereby reducing the storage and computing cost while still retaining the semantic relevance between features.

[0092] Finally, the generated binary hash code h is spliced with the original multi-scale statistical feature vector x to form the final fusion feature vector f.

[0093] Specifically, the similarity between samples can be measured by calculating the binary hash code similarity: let the hash codes corresponding to two samples be h and h respectively. The similarity can be calculated as follows:

[0094] wherein, The Hamming distance between the hash codes is represented by h, and k is the number of hash bits. In order to balance the retrieval efficiency and discriminative ability, the hash code can be spliced with the original multi-scale statistical feature to form a fusion feature vector f: .

[0095] The fusion feature vector f not only retains the fine-grained representation of the original statistical feature, but also has the fast similarity calculation ability of the hash space, and can be directly used for subsequent traffic classification and anomaly detection.

[0096] In summary, by splicing, dimensionality reduction and semantic hash coding of multi-scale statistical features, and then splicing the obtained binary hash code with the original feature to form a fusion feature vector, the storage and computing overheads are significantly reduced, while the discriminability of feature representation and the efficiency of similarity calculation are balanced, thereby providing high-quality input features for subsequent traffic classification and anomaly detection.

[0097] Optionally, in the step of determining the traffic detection result of the target network based on the detection result of the non-encrypted traffic and the detection result of the encrypted traffic, the detection result of the non-encrypted traffic and the detection result of the encrypted traffic can be collected and integrated to form a comprehensive result record; the comprehensive result record is formatted to obtain structured result data in a preset format; and the structured result data is taken as the traffic detection result, which includes at least one of report output and database storage.

[0098] In the embodiments of the present application, after the detection result of the non-encrypted traffic and the detection result of the encrypted traffic are collected and integrated to form a comprehensive result record, the detection results can be uniformly organized, formatted and output to provide structured data support for subsequent security analysis and traceability.

[0099] Specifically, various analysis results such as traffic classification, anomaly detection and encryption protocol identification can be collected first, and the classification type (such as normal traffic or specific attack type) corresponding to each traffic sample, whether abnormal and abnormal confidence, and the identified encryption protocol type and other information are integrated according to the traffic sample dimension to form a complete comprehensive result record. To ensure efficient and reliable data transmission, a message queue mechanism can be used, and the results are pushed to the message queue by the analysis process, and then read and uniformly integrated.

[0100] On this basis, the comprehensive result record can be formatted and converted into a preset standardized structured format (such as JSON, XML, etc.). The structured result can include fields such as the five-tuple information of the data packet (source IP, destination IP, source port, destination port, protocol type), traffic classification result, anomaly detection result (whether abnormal, abnormal type, confidence), and encryption protocol identification result, making it easy to parse, store and call later.

[0101] In order to improve readability and intuitiveness, the formatted structured data can also be displayed in a visual way, such as presenting the proportion of different types of traffic, the trend of abnormal traffic quantity and the distribution of encryption protocol usage in the form of column chart, line chart, pie chart, etc., so that users can intuitively master the overall distribution and abnormal pattern of the detection results.

[0102] Finally, the structured result data can be output as a report file for manual viewing according to system requirements, or written into a database for subsequent retrieval and correlation analysis, or pushed to other security systems (such as intrusion detection systems, security information and event management systems, etc.) for use, so that the structured result data is provided to users or other systems as the traffic detection result of the target network.

[0103] Optionally, in the step of performing statistical processing on the visible metadata and transmission statistical information of the encrypted traffic to obtain a window feature vector corresponding to each time window, the visible metadata and transmission statistical information of the encrypted traffic in the time window can also be extracted as the window feature; and the window feature is normalized to obtain the window feature vector.

[0104] In the embodiments of the present application, the continuous encrypted traffic can be divided into preset time windows before the statistical processing of the visible metadata and transmission statistical information of the encrypted traffic, and the size of each time window can be flexibly set according to actual requirements and traffic characteristics, for example, it can be set to 1 second.

[0105] Within each time window, visible metadata (e.g. source / destination IP address, port number, TLS certificate fingerprint, etc.) and transmission statistics (e.g. number of packets, mean, variance, maximum, minimum of packet size, mean, variance of time interval of packet arrival, etc.) can be extracted from the packets within the time period as window features to characterize the overall features of the communication behavior within the time window. For example, when the sequence of packet sizes within a time window is , the mean of packet size can be calculated as

[0106] where n is the number of packets within the window, is the size of the i-th packet.

[0107] Considering that the value ranges of different features can be quite different, to avoid the influence of dimensional difference on model training, normalization can be performed on the extracted window features of various types to scale them to a unified numerical interval. Specifically, for any feature value x, normalization can be performed based on its minimum value and maximum value in the training data to obtain the normalized value:

[0108] Through the above processing, a window feature vector of unified dimension can be obtained, which is used as the node feature of the corresponding node in the subsequent spatio-temporal interaction graph to input the model, improving the comparability between features and the stability of model convergence.

[0109] Optionally, in the step of performing continuous time modeling on the spatio-temporal interaction graph based on the preset neural differential model to obtain the detection result of the encrypted traffic, the spatio-temporal attention mechanism can also be used to perform feature fusion processing on the spatio-temporal interaction graph to obtain fused interaction features; the fused interaction features are provided to the preset neural differential model for continuous time modeling processing to obtain the detection result of the encrypted traffic.

[0110] In the embodiments of the present application, before performing continuous time modeling on the spatio-temporal interaction graph, the spatio-temporal attention mechanism can be used to perform feature fusion on the interaction information therein to highlight the key dynamic interaction mode, thereby improving the accuracy and robustness of subsequent modeling.

[0111] Specifically, first, the attention weights can be calculated from the spatial dimension and the time dimension, respectively, which can be specifically referred to the following steps: Spatial attention: for any pair of nodes in the graph, the attention weight between them can be calculated based on their node features at the current time t and the interaction strength (i.e. interaction relationship) between them Constructing the comprehensive feature vector:

[0112] Then, the unnormalized attention scores are calculated by the learnable weight matrix and the nonlinear activation function:

[0113] Subsequently, normalization is performed to obtain the spatial attention weights between the node and its neighbor nodes :

[0114] where is the neighbor node set of the node , and reflects the importance of the node to the node in the spatial dimension, and k is the index of any neighbor node in the neighbor node set of the node , used for normalization processing.

[0115] Temporal attention: For a single node , its features and interactions at different time steps are considered, and the feature at time step is concatenated with the corresponding dynamic adjacency matrix row vector :

[0116] The unnormalized temporal attention scores are calculated using the learnable weight matrix :

[0117] Then, normalization is performed to obtain the temporal attention weights:

[0118] where represents the influence weight of time step on the node , used to depict the time evolution characteristics of its interaction mode.

[0119] Spatio-temporal attention fusion: The spatial attention weights and the temporal attention weights are jointly applied to the original interaction strength to obtain the fused interaction strength:

[0120] the fused interaction strength (i.e., the fused interaction feature) The importance and time evolution trend of the interaction between nodes are comprehensively considered, and the key interaction mode in the graph is more accurately reflected.

[0121] After obtaining the fused interaction feature, the interaction feature fused through the spatio-temporal attention mechanism can be taken as input and provided to a preset neural differential model to perform continuous time modeling on the evolution process of the protocol state of the encrypted traffic over time, thereby generating a detection result of the encrypted traffic.

[0122] Specifically, for each node (representing a network entity, such as a client or a server), the feature vector of the node at time t and the fused interaction strength between the node and its neighbor nodes are integrated to form the protocol state vector of the node at time t The specific algorithm is as follows

[0123] wherein, is used to represent the statistical and protocol-specific features of the node itself (such as the mean packet length, the variance of the time interval, the TLS version, the SNI field, etc.), is the interaction strength after the spatio-temporal attention fusion, reflecting the importance of the interaction between the node and its neighbor nodes , is the neighbor node set of the node , and j represents a neighbor node adjacent to the node i.

[0124] The protocol state vector of all nodes is spliced to obtain the global protocol state s(t) of the entire spatio-temporal interaction graph at time t, and the protocol state is continuously time-modeled through a neural differential equation to describe the dynamic evolution law of the protocol state over time:

[0125] wherein, is a function realized by a neural network, used to learn the variation law of the protocol state, θ is a trainable parameter thereof, and t is a time variable.

[0126] On the premise that the initial protocol state is known, the protocol state vector at any time point is obtained by iterative solution through a numerical integration method, specifically:

[0127] Wherein, Δt is a time step, used to control the solving precision and the amount of calculation. Through the continuous time modeling process, the smooth evolution trajectory of the protocol state over time can be obtained, and the potential malicious communication behavior is identified based on the abnormal pattern appearing in the evolution process, so as to output the accurate encrypted traffic detection result.

[0128] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0129] In an embodiment, a traffic detection device is provided, which corresponds to the traffic detection method in the above embodiment. As shown in the figure, the traffic detection device comprises a first acquisition module 201, a first statistical module 202, a first construction module 203, a first modeling module 204, and a first determination module 205. The detailed description of each functional module is as follows: Figure 2 The first acquisition module 201 is used for collecting network traffic data in a target network, wherein the network traffic data comprises encrypted traffic. The first statistical module 202 is used for performing statistical processing in a preset time window based on the visible metadata and transmission statistical information of the encrypted traffic, to obtain a window feature vector corresponding to each time window. The first construction module 203 is used for constructing a space-time interaction graph in each preset time window, taking the communication entities in the encrypted traffic as nodes, taking the window feature vector as the node feature of the corresponding node, and taking the interaction relationship between the communication entities in the time window as edges. The first modeling module 204 is used for performing continuous time modeling processing on the space-time interaction graph based on a preset neural differential model, to obtain the detection result of the encrypted traffic. The first determination module 205 is used for determining the traffic detection result of the target network based on the detection result of the encrypted traffic. Optionally, the network traffic data further comprises non-encrypted traffic, and the first determination module 205 is further used for:

[0130] extracting statistical features of different granularities from the non-encrypted traffic to obtain multi-scale statistical features, wherein the granularity of each scale is different from that of other scales; performing feature fusion processing based on the multi-scale statistical features to obtain a fusion feature vector; providing the fusion feature vector to a preset traffic classification model, so that the traffic classification model outputs a traffic classification result corresponding to the fusion feature vector; ​provide the fusion feature vector to a preset anomaly detection model, so that the anomaly detection model outputs an anomaly detection result corresponding to the fusion feature vector; determine a detection result of the non-encrypted traffic based on the traffic classification result and the anomaly detection result; determine a traffic detection result of the target network based on the detection result of the non-encrypted traffic and the detection result of the encrypted traffic.

[0131] Optionally, the first determining module 205 is further configured to: perform protocol decoding processing based on the non-encrypted traffic to obtain decoded traffic; extract statistical features of different granularities from the decoded traffic to obtain multi-scale statistical features.

[0132] Optionally, the first determining module 205 is further configured to: perform dimension reduction processing based on the multi-scale statistical features to obtain a low-dimensional feature vector; generate a corresponding binary hash code based on the low-dimensional feature vector; splice the binary hash code and the multi-scale statistical features to form the fusion feature vector.

[0133] Optionally, the first determining module 205 is further configured to: collect and integrate the detection result of the non-encrypted traffic and the detection result of the encrypted traffic to form a comprehensive result record; perform format processing on the comprehensive result record to obtain structured result data in a preset format; use the structured result data as the traffic detection result, and the traffic detection result includes at least one of report output and database storage.

[0134] Optionally, the first statistical module 202 is further configured to: extract visible metadata and transmission statistical information of encrypted traffic as window features within the time window; perform normalization processing on the window features to obtain the window feature vector.

[0135] Optionally, the first modeling module 204 is further configured to: perform feature fusion processing on the spatio-temporal interaction graph based on a spatio-temporal attention mechanism to obtain fusion interaction features; provide the fusion interaction features to a preset neural differential model for continuous time modeling processing to obtain the detection result of the encrypted traffic.

[0136] Specific limitations regarding the flow detection device can be found in the limitations of the flow detection method described above, and will not be repeated here. Each module in the aforementioned flow detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0137] In one embodiment, a computer device is provided, which may be a terminal device, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a readable storage medium storing computer-readable instructions. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer-readable instructions implement a traffic detection method. The readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.

[0138] In this application embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it implements the steps of the traffic detection method described above.

[0139] In one embodiment of the application, a readable storage medium is provided, which stores computer-readable instructions. When the computer-readable instructions are executed by a processor, they implement the steps of the traffic detection method described above.

[0140] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware through computer readable instructions, and the computer readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer readable instructions are executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0141] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0142] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of flow detection, characterized by, The method comprises: collecting network traffic data in a target network, the network traffic data comprising encrypted traffic; based on the visible metadata and transmission statistics of the encrypted traffic, performing statistical processing within a preset time window to obtain a window feature vector corresponding to each time window; within each preset time window, taking a communication entity in the encrypted traffic as a node, taking the window feature vector as a node feature of the corresponding node, and taking an interaction relationship between communication entities within the time window as an edge, to construct a space-time interaction graph; based on a preset neural differential model, performing continuous time modeling processing on the space-time interaction graph to obtain a detection result of the encrypted traffic; based on the detection result of the encrypted traffic, determining a traffic detection result of the target network.

2. The traffic detection method of claim 1, wherein, The network traffic data also includes non-encrypted traffic, and the determination of the traffic detection result of the target network based on the detection result of the encrypted traffic comprises: extracting statistical features of different granularities from the non-encrypted traffic to obtain multi-scale statistical features, wherein the granularity of each scale is different from that of other scales; performing feature fusion processing based on the multi-scale statistical features to obtain a fusion feature vector; providing the fusion feature vector to a preset traffic classification model to enable the traffic classification model to output a traffic classification result corresponding to the fusion feature vector; providing the fusion feature vector to a preset anomaly detection model to enable the anomaly detection model to output an anomaly detection result corresponding to the fusion feature vector; determining a detection result of the non-encrypted traffic based on the traffic classification result and the anomaly detection result; determining the traffic detection result of the target network based on the detection result of the non-encrypted traffic and the detection result of the encrypted traffic.

3. The traffic detection method of claim 2, wherein, The extraction of statistical features of different granularities from the non-encrypted traffic to obtain multi-scale statistical features comprises: performing protocol decoding processing on the non-encrypted traffic to obtain decoded traffic; extracting statistical features of different granularities from the decoded traffic to obtain multi-scale statistical features.

4. The traffic detection method of claim 2, wherein, The feature fusion processing based on the multi-scale statistical features to obtain a fusion feature vector comprises: performing dimensionality reduction processing based on the multi-scale statistical features to obtain a low-dimensional feature vector; generating a corresponding binary hash code based on the low-dimensional feature vector; concatenating the binary hash code and the multi-scale statistical features to form the fusion feature vector.

5. The traffic detection method of claim 2, wherein, The determination of the traffic detection result of the target network based on the detection result of the non-encrypted traffic and the detection result of the encrypted traffic comprises: collecting and integrating the detection result of the non-encrypted traffic and the detection result of the encrypted traffic to form a comprehensive result record; performing formatting processing on the comprehensive result record to obtain structured result data in a preset format; taking the structured result data as the traffic detection result, wherein the traffic detection result comprises at least one of report output and database storage.

6. The flow rate detection method according to any one of claims 1 to 5, wherein The visible metadata and transmission statistics of the encrypted traffic are statistically processed in a preset time window to obtain a window feature vector corresponding to each time window, including: Visible metadata and transmission statistics of the encrypted traffic are extracted in the time window as window features; The window features are normalized to obtain the window feature vector.

7. The flow rate detection method according to any one of claims 1 to 5, wherein The spatio-temporal interaction graph is continuously time modeled based on a preset neural differential model to obtain the detection result of the encrypted traffic, including: The spatio-temporal interaction graph is feature fused based on a spatio-temporal attention mechanism to obtain fused interaction features; The fused interaction features are provided to the preset neural differential model for continuous time modeling to obtain the detection result of the encrypted traffic.

8. A flow detecting device, characterized by The device includes: A first acquisition module configured to acquire network traffic data in a target network, the network traffic data including encrypted traffic; A first statistical module configured to statistically process visible metadata and transmission statistics of the encrypted traffic in a preset time window to obtain a window feature vector corresponding to each time window; A first construction module configured to, in each preset time window, take a communication entity in the encrypted traffic as a node, take the window feature vector as a node feature of the corresponding node, and take an interaction relationship between the communication entities in the time window as an edge to construct a spatio-temporal interaction graph; A first modeling module configured to continuously time model the spatio-temporal interaction graph based on a preset neural differential model to obtain a detection result of the encrypted traffic; A first determination module configured to determine a traffic detection result of the target network based on the detection result of the encrypted traffic. 9.A computer device, comprising a memory, a processor, and computer readable instructions stored on the memory and running on the processor, wherein, The processor executes the computer-readable instructions to implement the traffic detection method of any one of claims 1-7.

10. A readable storage medium, having stored thereon computer readable instructions, characterized in that, The computer-readable instructions are executed by the processor to implement the traffic detection method of any one of claims 1-7.

Images