A lateral movement flow detection method

Abstract

The application provides a transverse movement flow detection method for detecting whether transverse movement flow exists in a to-be-detected flow sequence generated in a target containerization cluster, and the method comprises a preprocessing stage, a first detection stage, a second detection stage and a detection result output stage; wherein the preprocessing stage is used for obtaining a value range corresponding to a target feature and a topology structure of the target containerization cluster; the first detection stage is used for performing maximum and minimum value detection and topology detection; the second detection stage is used for performing flow detection on the to-be-detected flow sequence by using a pre-trained transverse movement detection model; and the detection result output stage is used for performing deduplication processing to obtain a final detection result of the to-be-detected flow sequence. The technical scheme of the application screens transverse movement flow by setting two-stage detection, thereby improving the detection rate and reducing the false positive rate.

Description

Technical Field

[0001] This invention relates to the field of network security, specifically to lateral movement traffic detection technology in the field of network security, and more specifically, to a lateral movement traffic detection method. Background Technology

[0002] With the rapid development of containerization technology, containerized clusters are increasingly used in various fields. However, the widespread application of this technology has also attracted the attention of attackers, making timely detection of attack behaviors particularly important. Attacks against containerized clusters typically fall into five phases: reconnaissance, foothold, lateral movement, attack, maintaining access, and cleanup. Among these phases, lateral movement is the key and most feasible phase for attack detection.

[0003] Because containerized environments contain a variety of components and have a wide attack surface, vulnerabilities in any component can become security risks, posing security challenges. Therefore, in containerized environments, it is necessary to perform lateral movement detection by acquiring network traffic, thereby enabling the detection of attack behaviors.

[0004] Currently, there are two main problems with lateral movement detection in containerized environments. First, the features available for lateral movement detection are unclear, making it difficult to achieve lateral movement detection. Second, the classification of lateral movement detection tasks is difficult, and the samples are imbalanced, making it difficult to achieve the requirements of high detection rate and low false alarm rate.

[0005] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solutions of the present invention, and does not imply that the relevant information is necessarily prior art. In the absence of evidence indicating that the relevant information was disclosed before the filing date of this invention, the relevant information should not be considered prior art. Summary of the Invention

[0006] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a method for detecting lateral movement flow.

[0007] The objective of this invention is achieved through the following technical solutions.

[0008] According to a first aspect of the present invention, a method for detecting lateral movement traffic is provided, used to detect whether lateral movement traffic exists in a traffic sequence to be detected generated in a target containerized cluster, wherein the traffic sequence to be detected includes multiple network traffic flows that are sequentially continuous, and the lateral movement traffic is traffic data generated when the target containerized cluster is attacked. The method includes: a preprocessing stage: acquiring a benign traffic sequence containing multiple benign flows generated by the target containerized cluster, performing feature extraction on the benign traffic sequence to obtain a target feature set containing multiple target features, and traversing each target feature in each benign flow in the benign traffic sequence to obtain the value range corresponding to each target feature, and performing topology extraction based on the benign traffic sequence to obtain the topology of the target containerized cluster; a first detection stage: extreme value detection: analyzing whether each network flow in the traffic sequence to be detected is lateral movement traffic; wherein each network flow includes multiple target features. The detection process involves several steps: First, a target feature is defined. This involves identifying network traffic whose value falls outside the target feature's range as lateral movement traffic. Second, a topology detection model is used to analyze the transmission process of each network traffic segment within the target traffic sequence to determine if it conforms to the target containerized cluster's topology. If not, the network traffic is considered lateral movement traffic. Third, all lateral movement traffic segments obtained through topology detection within the target traffic sequence constitute the second detection result. Finally, a pre-trained lateral movement detection model is used to perform traffic detection on the target traffic sequence to analyze the lateral movement traffic and obtain the third detection result. Finally, the detection result output stage removes network traffic segments repeatedly identified as lateral movement traffic from the first, second, and third detection results, and uses the remaining lateral movement traffic segments as the final detection result for the target traffic sequence.

[0009] In some embodiments of the present invention, the method includes extracting features from a benign traffic sequence to obtain a target feature set containing multiple target features in the following manner: each benign traffic in the benign traffic sequence includes multiple data packets; based on all data packets of any benign traffic, data packet-level feature extraction is performed on the benign traffic to obtain multiple data packet-level traffic features corresponding to the benign traffic; and based on a packet connection protocol, all data packets in the benign traffic are aggregated into multiple sessions to extract multiple session-level traffic features corresponding to the benign traffic; wherein, all data packet-level traffic features and all session-level traffic features constitute an initial feature set; each traffic feature in the initial feature set is evaluated for importance using a preset evaluation method to obtain an importance evaluation result for each traffic feature; the importance evaluation results are sorted in descending order, and a preset number of traffic features ranked first are selected as target features.

[0010] In some embodiments of the present invention, the preset evaluation method is as follows: gradient boosting decision tree, random forest and mutual information are used to evaluate the importance of each traffic feature in the initial feature set, and the mean of the importance of each traffic feature is calculated to obtain the importance evaluation result of each traffic feature.

[0011] In some embodiments of the present invention, the preset number is 57.

[0012] In some embodiments of the present invention, the target features include: standard deviation of continuous idle time, average number of bytes transmitted in forward bulk data packets, minimum length of forward data packets, average rate of forward bulk data packets, forward RST flag, number of forward active data packets, minimum time to arrive between backward frames, backward PSH flag, minimum size of forward segment, initial value of backward window, total TCP stream time, PSH flag count, ACK flag count, average time to arrive between forward frames, FIN flag count, backward header length, minimum time to arrive between streams, minimum time to arrive between forward frames, number of forward data packets, average data packet arrival time interval, total length of backward data packets, SYN flag count, maximum data packet length, average time to arrive between backward frames, ratio of backward to forward data packets, number of backward data packets, forward header length, stream duration in milliseconds, and forward number of packets. The table lists the data types and their corresponding time intervals: total arrival time intervals of packets, total arrival time intervals of backward packets, average length of forward packets, average size of forward segments, forward PSH flag, standard deviation of arrival time intervals, standard deviation of arrival time intervals of forward packets, number of bytes in backward substreams, initial value of forward window, standard deviation of length of backward packets, average length of packets, average length of backward packets, change in length of packets, standard deviation of arrival time intervals of backward packets, maximum arrival time interval of packets, average size of backward segments, maximum arrival time interval of backward packets, maximum arrival time interval of forward packets, standard deviation of length of packets, average length of packets, forward flow rate, total length of forward packets, standard deviation of length of forward packets, number of bytes in forward substreams, total length of backward packets, flow rate of packets, byte flow rate, flow rate of backward packets, and maximum length of forward packets.

[0013] In some embodiments of the present invention, the method includes performing topology extraction based on a benign traffic sequence to obtain the topology of a target containerized cluster in the following manner: each benign traffic in the benign traffic sequence corresponds to a source node and a target node; wherein, the source node represents the node that sends the benign traffic in the target containerized cluster, and the target node represents the node that receives the benign traffic in the target containerized cluster; based on the node types of the source node and target node corresponding to each benign traffic in the benign traffic sequence, a load node set, an API server set, a load node set communicating with API servers, and a load node set communicating with external nodes are obtained; wherein, the load node set, the API server set, the load node set communicating with API servers, and the load node set communicating with external nodes constitute the topology of the target containerized cluster.

[0014] In some embodiments of the present invention, the method includes analyzing whether the transmission process of each network traffic in the traffic sequence to be detected satisfies the topology of the target containerized cluster in the following manner: if the source node or target node corresponding to the network traffic is a new load node and is not in the load node set, then the network traffic is lateral movement traffic; if the source node corresponding to the network traffic is not in the load node set communicating with the API server, and the target node corresponding to the network traffic is the API server, then the network traffic is lateral movement traffic; if the target node corresponding to the network traffic is not in the load node set communicating with the API server, and the source node corresponding to the network traffic is the API server, then the network traffic is lateral movement traffic; if the source node corresponding to the network traffic is not in the load node set communicating with external nodes, and the target node corresponding to the network traffic is an external node, then the network traffic is lateral movement traffic; if the target node corresponding to the network traffic is not in the load node set communicating with external nodes, and the source node corresponding to the network traffic is an external node, then the network traffic is lateral movement traffic.

[0015] In some embodiments of the present invention, the pre-trained lateral movement detection model is configured to perform traffic detection on the traffic sequence to be detected in the following manner: taking the traffic sequence to be detected as input, generating a target prediction sequence of the traffic sequence to be detected in a recursive prediction manner; wherein, the target prediction sequence includes multiple target prediction network traffic, and each target prediction network traffic corresponds to a network traffic in the traffic to be detected; calculating the error value between each target prediction network traffic and its corresponding actual network traffic, and if the error value is greater than or equal to a threshold, then determining that the network traffic is lateral movement traffic.

[0016] In some embodiments of the present invention, the pre-trained lateral movement detection model is a model trained as follows: a training benign traffic sequence is obtained, wherein the training benign traffic sequence includes multiple temporally consecutive benign traffic flows, each benign traffic flow including multiple traffic features; feature extraction is performed on the training benign traffic sequence to obtain a target feature set corresponding to each benign traffic flow in the training benign traffic sequence; multiple rounds of iterative training are performed with the feature-extracted benign traffic sequence as input and the target prediction sequence corresponding to the feature-extracted benign traffic sequence as output until the lateral movement detection model converges.

[0017] In some embodiments of the present invention, the pre-trained lateral movement detection model includes a Transformer model and a judgment module, wherein: the Transformer model is used to predict the target prediction sequence corresponding to the traffic sequence to be detected; the judgment module is used to calculate the error value between each target prediction network traffic and its corresponding actual network traffic, and when the error value is greater than or equal to a threshold, the network traffic is judged to be lateral movement traffic.

[0018] In some embodiments of the present invention, the pre-trained lateral movement detection model includes a spatial feature embedding module, a temporal feature embedding module, an encoding / decoding module, a sequence output module, and a judgment module; wherein: the spatial feature embedding module is used to extract spatial features of the traffic sequence to be detected to obtain a spatial feature embedding vector for each network traffic in the traffic sequence to be detected, and to concatenate the spatial feature embedding vector of each network traffic with the traffic features of that network traffic to obtain the initial features of the traffic sequence to be detected; the temporal feature embedding module is used to extract temporal features based on the initial features of the traffic sequence to be detected to obtain a temporal feature embedding vector of the traffic sequence to be detected; the encoding / decoding module includes an encoder, a first decoder, and a second decoder, wherein: the encoder is used to extract the dependencies between each network traffic in the traffic sequence to be detected based on the initial features of the traffic sequence to be detected, to obtain a first latent vector of the traffic sequence to be detected; and based on the initial features of the traffic sequence to be detected and the actual network traffic in the traffic sequence to be detected corresponding to each initially predicted network traffic obtained by the sequence output module, the encoder is used to extract the dependencies between each network traffic in the traffic sequence to be detected, to obtain a first latent vector of the traffic sequence to be detected; and based on the initial features of the traffic sequence to be detected and the actual network traffic in the traffic sequence to be detected corresponding to each initially predicted network traffic obtained by the sequence output module. The error between quantities is used to re-extract the dependencies between various network traffic in the traffic sequence to be detected, thereby obtaining the second latent vector of the traffic sequence to be detected; the first decoder is used to perform decoding processing based on the first latent vector and the temporal feature embedding vector of the traffic sequence to be detected, thereby obtaining the first decoded feature vector of the traffic sequence to be detected; the second decoder is used to perform decoding processing based on the second latent vector and the temporal feature embedding vector of the traffic sequence to be detected, thereby obtaining the second decoded feature vector of the traffic sequence to be detected; the sequence output module is used to generate an initial prediction sequence of the traffic sequence to be detected based on the first decoded feature vector of the traffic sequence to be detected, and to generate a target prediction sequence of the traffic sequence to be detected based on the second decoded feature vector of the traffic sequence to be detected; wherein, the initial prediction sequence includes multiple initial prediction network traffic, and each initial prediction network traffic corresponds to a network traffic in the traffic sequence to be detected; the judgment module is used to calculate the error value between each target prediction network traffic and its corresponding actual network traffic, and when the error value is greater than or equal to a threshold, the network traffic is judged to be lateral movement traffic.

[0019] Compared with the prior art, the advantages of the present invention are: (1) extracting data packet-level traffic features and session-level traffic features, and also eliminating traffic features with lower importance based on the importance ranking results of traffic features, thus solving the problem of unclear lateral movement detection features in containerized clusters; (2) setting up two-stage detection to filter lateral movement traffic, thereby improving the detection rate while reducing the false alarm rate; (3) the lateral movement detection model uses two-step prediction to minimize the prediction error of benign traffic and maximize the prediction error of lateral movement traffic, thereby achieving more accurate lateral movement traffic detection. Attached Figure Description

[0020] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0021] Figure 1 This is a schematic flowchart of a lateral movement flow detection method according to an embodiment of the present invention;

[0022] Figure 2 This is a schematic diagram illustrating the ranking of traffic feature importance according to an embodiment of the present invention;

[0023] Figure 3 This is a box plot of lateral movement traffic under active and idle class characteristics according to an embodiment of the present invention;

[0024] Figure 4 This is a data transmission density map of network traffic under length-type features according to an embodiment of the present invention;

[0025] Figure 5 This is a lateral movement detection model according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0027] To better understand this invention, we will first briefly introduce existing methods for detecting lateral movement.

[0028] Current technologies for lateral movement detection typically employ four approaches: network traffic, endpoint behavior, user behavior, and threat intelligence. Network traffic-based detection can detect lateral movement through abnormal port scans, large data transfers, and communication using non-standard protocols; however, this method only focuses on network traffic generated by user login activity and is unsuitable for containerized clusters. Endpoint behavior-based detection can detect lateral movement through abnormal process creation, abnormal file access, and abnormal registry modifications. For example, a network connectivity graph can be constructed using Windows systems and security events. When remote file execution occurs (a typical detection scenario) or an attacked node is detected (a forensic analysis scenario), the constructed network connectivity graph can be used to find possible lateral movement paths. However, this method is deeply tied to the Windows system and cannot be used for lateral movement detection in containerized clusters. User behavior-based detection: This can be achieved by detecting abnormal logins, access to unusual resources, and execution of unusual operations. For example, user operation logs (such as logins, web page visits, email sending, and file opening) can be modeled into a graph, and embedding vectors for each node can be generated. Then, clustering can be used to identify smaller clusters and these smaller clusters can be identified as abnormal operations. However, in containerized clusters, there is no user behavior information available, and TCP / IP traffic also lacks relevant user behavior information. Therefore, this detection method is not suitable for containerized clusters. Threat intelligence-based detection: Attackers often use known tools and techniques for lateral movement. Therefore, by analyzing threat intelligence, the tools and techniques that attackers may use can be identified, allowing for proactive defense. However, this detection method requires known threats, and the timeliness of lateral movement detection is relatively weak.

[0029] While lateral movement detection can be implemented in various ways, existing methods primarily rely on network traffic analysis. These methods focus solely on network traffic generated by login activity and are based on the crucial assumption that attackers typically perform lateral movement to access machines inaccessible to the initial victim. Furthermore, while the detection models differ, they all employ a login graph modeling approach, where nodes represent computers and edges represent traffic between them. Because these methods use different datasets and evaluation metrics, and are all implemented in non-containerized environments such as enterprise networks, they are not suitable for containerized clusters.

[0030] To better understand this invention, the following is a brief introduction to existing lateral movement detection methods based on network traffic.

[0031] Some researchers have proposed extracting features from the login graph and then using a logistic regression model to detect lateral movement. These features include the number of nodes, edges, and graph density. While this method can detect lateral movement, the feature extraction approach it uses seems outdated in today's world of graph neural networks, is cumbersome to implement, and yields unsatisfactory results.

[0032] Other researchers have proposed constructing triples ⟨a, b, d by statistically mining several of the most frequently communicating users, source computers, and target computers from the login graph. For a login record ⟨a, b, d, if the login record matches one of the triples (i.e., b ∈ b, b ∈ b, d ∈ b), then the login is considered normal; otherwise, it is considered a lateral movement. The advantage of this detection method is its strong interpretability; the disadvantage is that its triple mining method is relatively inefficient, requiring enumeration of all possible ⟨a, b, d, which is time-consuming.

[0033] Other researchers have proposed a machine learning-based detection method. This method uses a Node2vec model to learn the login graph and generate embedding vectors for each node. For each login record, it calculates the element-wise product of the two nodes in that record and uses a Logistic model to predict the probability that the login record represents lateral movement. The advantage of this method is that it uses machine learning, extracting graph features more effectively than previous methods. The disadvantage is that it treats the login graph as static, failing to reflect the dynamic nature of the network, making it difficult to detect lateral movement behavior in computers that were originally healthy but were later infected.

[0034] Researchers have also conducted in-depth studies on the causal relationships between paths in the login graph, defining a method for aggregating causal relationships between login records. Specifically, for two login records, 𝐿1=⟨𝑠1,d1> and 𝐿2=⟨𝑠2,d2>, if 𝐿1 occurred within 24 hours before 𝐿2, and the destination computer of 𝐿1 is equal to the source computer of 𝐿2 (i.e., d1=𝑠2), then 𝐿1 and 𝐿2 have a "causal relationship," meaning 𝐿1 is the cause of 𝐿2. Multiple causal login records are aggregated into login paths using this rule, and lateral movement detection is achieved by analyzing these login paths. In this detection method, lateral movement is detected by examining user information along the login path. If user information changes, it is considered lateral movement; if user information remains unchanged, it is not lateral movement. However, due to the imprecise nature of causal aggregation—for example, the cause of one login record might be one of multiple login records, making accurate determination impossible—this detection method infers the probability of lateral movement by using historical login records and historical alarm records for such ambiguous login paths that may involve changes in user information. The advantage of this method is that it considers the temporal relationship of login paths; the disadvantage is that its login path rule aggregation is cumbersome and requires exclusion of various whitelist scenarios (such as bots, new machines, etc.), resulting in a need to improve detection accuracy.

[0035] Other researchers have proposed using a method similar to the GCRN model, stacking graph deep learning networks and recurrent neural networks for lateral movement detection. In this method, the system first divides the data into time slices based on a set time interval, generating a login graph for each slice. Then, a graph deep learning network is used to generate embedding vectors for each node in the login graph. These vectors are then input into a recurrent neural network to predict the node's embedding vector for the next time step. Finally, the dot product of the embedding vectors of two nodes is performed to obtain the probability that these two nodes have a login record; the node with the lower probability is identified as having moved laterally. The advantage of this detection method is that it considers the dynamics of the network and combines deep learning methods, significantly improving its accuracy. The disadvantage is that it requires time slices, and the temporal relationships within each time slice are lost after the division.

[0036] Some researchers have proposed using a Time-Neural Network (TGN) model to detect lateral movement, thus avoiding time-slicing. This detection method primarily consists of a graph feature extraction module, a memory module, and an embedding module. The graph feature extraction module calculates the daily features of each node, such as the number of user logins and computers logged into that node that day. The memory module processes each login record chronologically, updating the memory vectors of the source and destination nodes using a recurrent neural network. The embedding module enumerates each historical neighbor node, passing the node's current memory vector, current features, and current memory vectors and features of its historical neighbors through an attention function to obtain the node's current embedding vector. Finally, the embedding vectors of two nodes are decoded to obtain the probability of an edge between them; the node with the lower probability is identified as having moved laterally. The advantage of this detection method is that it eliminates the need for time-slicing, thus avoiding the loss of temporal data and significantly improving its accuracy.

[0037] The common drawback of existing network traffic-based lateral movement detection methods is that they all employ graph link prediction to detect lateral movement, essentially determining the probability of login records between two nodes. However, in containerized clusters, no such login record information is available. This is because attackers do not log in based on any individual's identity when infiltrating the cluster; instead, they exploit load balancing vulnerabilities to gain access via remote code execution. Furthermore, while the API server in a containerized cluster is responsible for authentication and authorization, it primarily identifies the load and host identities, not the individuals, when dealing with internal cluster operations. Moreover, in containerized clusters, using network traffic information to detect lateral movement cannot simply rely on graph link prediction, as this method only checks the existence of edges without considering their characteristics; however, in containerized clusters, edge characteristics, i.e., network traffic characteristics, play a crucial role.

[0038] As mentioned in the background section, there are two problems with lateral movement detection in a containerized environment. On the one hand, the features that can be used for lateral movement detection are not clear, making it difficult to achieve lateral movement detection. On the other hand, the classification of lateral movement detection tasks is difficult and the samples are imbalanced, making it difficult to achieve the requirements of high detection rate and low false alarm rate.

[0039] Based on the analysis of existing detection methods and the problems mentioned in the background technology, the inventors propose a lateral movement detection method suitable for containerized clusters. This method includes a preprocessing stage, a first detection stage, a second detection stage, and a detection result output stage. The preprocessing stage extracts multiple target features and determines the value range of each target feature, as well as extracting the topology of the target containerized cluster. The first detection stage performs maximum / minimum value detection and topology detection. The second detection stage uses a pre-trained lateral movement detection model to detect the traffic sequence to be detected. The detection result output stage performs deduplication to obtain the final detection result for the traffic sequence to be detected. The target feature extraction in the preprocessing stage solves the problem of unclear lateral movement detection features. Furthermore, the two-stage detection design can fully detect lateral movement traffic, thereby improving the detection rate and reducing the false positive rate.

[0040] In summary, such as Figure 1 As shown, this invention provides a method for detecting lateral movement traffic, used to detect whether lateral movement traffic exists in a traffic sequence generated in a target containerized cluster. The traffic sequence includes multiple network traffic flows that are sequentially continuous, and the lateral movement traffic is traffic data generated when the target containerized cluster is attacked. The method includes: a preprocessing stage: acquiring a benign traffic sequence containing multiple benign flows generated by the target containerized cluster; performing feature extraction on the benign traffic sequence to obtain a target feature set containing multiple target features; traversing each target feature in each benign flow in the benign traffic sequence to obtain the value range corresponding to each target feature; and performing topology extraction based on the benign traffic sequence to obtain the topology of the target containerized cluster; a first detection stage: extreme value detection: analyzing whether each network flow in the traffic sequence is lateral movement traffic; wherein each network flow includes multiple target features. The first detection result is formed by identifying lateral movement traffic where any target feature value is outside the range of that feature. The second detection result is formed by analyzing whether the transmission process of each network traffic in the target traffic sequence satisfies the topology of the target containerized cluster. If not, the network traffic is considered lateral movement traffic. The third detection result is formed by removing network traffic repeatedly identified as lateral movement traffic from the first, second, and third detection results. The final detection result is formed by removing all remaining lateral movement traffic from the first, second, and third detection results.

[0041] To better understand the present invention, each stage will be described in detail below with reference to specific embodiments.

[0042] I. Preprocessing stage

[0043] In the preprocessing stage, a benign traffic sequence containing multiple benign traffic generated by the target containerized cluster is obtained. Feature extraction is performed on the benign traffic sequence to obtain a target feature set containing multiple target features. Each target feature in each benign traffic in the benign traffic sequence is traversed to obtain the value range corresponding to each target feature. Topology extraction is performed based on the benign traffic sequence to obtain the topology structure of the target containerized cluster.

[0044] 1.1 Target Feature Extraction and Value Range Extraction

[0045] According to one embodiment of the present invention, the method includes extracting features from a benign traffic sequence to obtain a target feature set containing multiple target features in the following manner: each benign traffic in the benign traffic sequence includes multiple data packets; based on all data packets of any benign traffic, data packet-level feature extraction is performed on the benign traffic to obtain multiple data packet-level traffic features corresponding to the benign traffic; and based on a packet connection protocol, all data packets in the benign traffic are aggregated into multiple sessions to extract multiple session-level traffic features corresponding to the benign traffic; wherein, all data packet-level traffic features and all session-level traffic features constitute an initial feature set; each traffic feature in the initial feature set is evaluated for importance using a preset evaluation method to obtain an importance evaluation result for each traffic feature; the importance evaluation results are sorted in descending order, and a preset number of traffic features ranked first are selected as target features.

[0046] The packet-level traffic features are features extracted at the packet level, such as the time difference between packets in a session, the size of packets in a session, and TCP tag value statistics in a session. To better understand packet-level feature extraction, let's take extracting packet protocol field information as an example. Protocol field information can be determined by reading specific bytes in the packet header. For example, in an Ethernet frame, the byte starting from the 9th byte of the IP packet header is used to indicate the protocol type; a value of 6 indicates TCP, and a value of 17 indicates UDP.

[0047] The session-level traffic features are extracted at the session level after aggregating all data packets in historical network traffic into multiple sessions based on packet connection protocols. Examples include session duration, total bytes transmitted in the session, and total number of data packets transmitted in the session. To better understand session-level feature extraction, let's take extracting session byte traffic as an example. Session byte traffic is determined by calculating the total number of bytes sent and received during the session. Specifically, for each HTTP request and response data packet in the session, the packet length field (usually in the packet header) is read, and the lengths of all data packets are summed. It's important to note that when aggregating data packets into sessions, the packet connection protocol must be followed. For example, for TCP connections, a session begins with one party sending the first handshake data packet and ends with one party sending the last handshake data packet, resetting the connection, or a timeout. For connectionless protocols like UDP, specific aggregation methods can be used for the application layer protocol. For example, for the DNS protocol, a single query request and its corresponding result can be considered as a session. It should also be noted that there is a special type of data packet set called bulk during the process of aggregating data packets into a session (when a TCP stream transmits a large amount of data, it needs to be divided into multiple data packets for transmission, and the collection of these data packets is called bulk). When extracting features from this special type of data packet set bulk, it is necessary to first determine whether the data packets can be aggregated into bulk, and then count the number of data packets. When there are 4 data packets, the features of bulk can be extracted. Among them, the features related to bulk are usually 0, and are only greater than 0 when the amount of transmitted data is large, indicating that the stream has been split into packets.

[0048] Through packet-level and session-level feature extraction, we can extract features in seven categories: flow duration, number of packets, packet length, flow rate, packet interval, TCP flag statistics, packet header bulk transmission, and TCP window activity and idleness, totaling 79 traffic features. The traffic characteristics include: Backward URG Flags, Backward RST Flags, CWR Flag Count, ECE Flag Count, Subflow Fwd Packets, Subflow Bwd Packets, Average Reverse Batch Packet Transmission Bytes / Bulk AVg, Average Reverse Batch Packet Transmission Bytes / Bulk Avg, Forward URG Flags, Average Reverse Batch Rate Avg, URG Flag Count, Maximum Continuous Active Duration (Active Max), Minimum Continuous Active Duration (Active Min), Minimum Packet Length (Packet Length Min), Minimum Reverse Packet Length (Bwd Packet Length Min), Average Continuous Idle Duration (ldle Mean), and Standard Deviation of Continuous Active Duration (Active Std), average number of packets transmitted in the forward bulk (Fwd Packet / Bulk Avg), minimum continuous idle time (ldle Min), average continuous active time, maximum continuous idle time (ldle Max), RST flag count (RST Flag Count), standard deviation of continuous idle time (IdleStd), average number of bytes transmitted in the forward bulk (Fwd Bytes / Bulk Avg), minimum forward packet length (Fwd Packet Length Min), average forward bulk rate (Fwd Bulk Rate Avg), forward RST flags (Fwd RST Flags), number of forward active data packets (Fwd Act Data Pkts), minimum backward inter-frame arrival time (Bwd IAT Min), backward PSH flags (Bwd PSH Flags), minimum forward segment size (Fwd Seg SizeMin), initial backward window size (Bwd Init Win Bytes), total TCP flow time (Total TCP Flow Time), PSH flag count (PSH).Flag Count, ACK Flag Count, Forward Inter-Frame Arrival Time Mean (Fwd IAT Mean), FIN Flag Count, Backward Header Length, Flow IAT Min, Forward Inter-Frame Arrival Time Min (Fwd IAT Min), Total Forward Packet Count, Average Packet Arrival Time Interval (Flow IAT Mean), Backward Packet Length Max, SYN Flag Count, Maximum Packet Length Max, Backward Inter-Frame Arrival Time Mean (Bwd IAT Mean), Down / Up Ratio, Total Backward Packet Count (Bwd Packets), Forward Header Length (Fwd Header Length), Flow Duration (milliseconds), Total Forward Packet Arrival Time Interval (Fwd IAT Total), Total Backward Packet Arrival Time Interval (Bwd IAT Total), Average Forward Packet Length (Fwd Packet Length) Mean, Average Forward Segment Size (Fwd Segment Size Avg), Forward PSH Flags (Fwd PSH Flags), Standard Deviation of Packet Arrival Time (Flow IAT Std), Standard Deviation of Forward Packet Arrival Time (Fwd IAT Std), Subflow Bwd Bytes, Initial Forward Window (FWD Init Win Bytes), Standard Deviation of Backward Packet Length (Bwd Packet Length Std), Average Packet Length (Average Packet Size), Average Backward Packet Length (Bwd Packet Length Mean), Packet Length Variation (Packet Length Variance), Standard Deviation of Backward Packet Arrival Time (Bwd IAT Std), Maximum Packet Arrival Time (Flow IAT Max), Average Backward Segment Size (Bwd Segment Size Avg), Maximum Backward Packet Arrival Time (Bwd IAT Max), Maximum Forward Packet Arrival Time (Fwd IAT Max), Standard Deviation of Packet Length (Packet Length Std), Average Packet Length (Packet Length Std), Standard Deviation of ...The parameters include: Length Mean, Forward Flow Rate (FwdPackets / s), Total Length of Forward Packet, Standard Deviation of Forward Packet Length (Fwd Packet Length Std), Subflow Fwd Bytes, Total Length of Backward Packet, Flow Packets / s, Flow Bytes / s, Backward Packet Flow Rate (Bwd Packets / s), and Maximum Forward Packet Length (Fwd Packet LengthMax).

[0049] According to an embodiment of the present invention, the preset evaluation method is as follows: Gradient boosting decision tree, random forest, and mutual information are used to evaluate the importance of each flow feature in the initial feature set, and the mean importance of each flow feature is calculated to obtain the importance evaluation result of each flow feature. Specifically, gradient boosting decision tree is used to calculate the importance of each flow feature in the initial feature set, and the calculated importance of each flow feature is scaled to the interval [0,1]; random forest is used to calculate the importance of each flow feature in the initial feature set, and the calculated importance of each flow feature is scaled to the interval [0,1]; mutual information is used to calculate the importance of each flow feature in the initial feature set, and the calculated importance of each flow feature is scaled to the interval [0,1]; the mean importance of each flow feature is calculated, and the flow features are sorted in descending order of importance. To better understand the importance of different flow features, Figure 2 The ranking results of traffic characteristics are presented for illustration, in which... Figure 2 The various traffic features are sorted in order of importance from low to high, from... Figure 2 It can be seen that the importance of the six traffic features—Backward URG Flags, Backward RST Flags, CWR Flag Count, ECE Flag Count, Subflow Fwd Packets, and Subflow Bwd Packets—is 0, and these six traffic features can be directly removed.

[0050] According to one embodiment of the present invention, the preset number is 57. It should be noted that, based on the ranking of importance from high to low, the top 57 traffic features can be selected to be retained, or more or fewer traffic features can be retained according to actual needs, such as retaining the top 60 features or the top 50 features. The present invention does not impose any special limitations.

[0051] According to an embodiment of the present invention, the target features include: standard deviation of continuous idle time, average number of bytes transmitted in forward bulk data packets, minimum length of forward data packets, average rate of forward bulk data packets, forward RST flag, number of forward active data packets, minimum time to arrive between backward frames, backward PSH flag, minimum size of forward segment, initial value of backward window, total TCP stream time, PSH flag count, ACK flag count, average time to arrive between forward frames, FIN flag count, backward header length, minimum time to arrive between streams, minimum time to arrive between forward frames, number of forward data packets, average data packet arrival time interval, total length of backward data packets, SYN flag count, maximum data packet length, average time to arrive between backward frames, ratio of backward to forward data packets, number of backward data packets, forward header length, stream duration in milliseconds, and forward number of packets. The table lists the data types and their corresponding time intervals: total arrival time intervals of packets, total arrival time intervals of backward packets, average length of forward packets, average size of forward segments, forward PSH flag, standard deviation of arrival time intervals, standard deviation of arrival time intervals of forward packets, number of bytes in backward substreams, initial value of forward window, standard deviation of length of backward packets, average length of packets, average length of backward packets, change in length of packets, standard deviation of arrival time intervals of backward packets, maximum arrival time interval of packets, average size of backward segments, maximum arrival time interval of backward packets, maximum arrival time interval of forward packets, standard deviation of length of packets, average length of packets, forward flow rate, total length of forward packets, standard deviation of length of forward packets, number of bytes in forward substreams, total length of backward packets, flow rate of packets, byte flow rate, flow rate of backward packets, and maximum length of forward packets.

[0052] After extracting the target feature set based on the aforementioned embodiments, each target feature in each benign traffic flow within the benign traffic sequence is traversed to obtain the value range corresponding to each target feature. The code shown in Table 1 can be used to obtain the value range corresponding to each target feature.

[0053] Table 1

[0054]

[0055] 1.2 Topology Extraction

[0056] According to one embodiment of the present invention, the method includes obtaining the topology of a target containerized cluster by performing topology extraction based on a benign traffic sequence as follows: each benign traffic in the benign traffic sequence corresponds to a source node and a target node; wherein the source node represents the node that sends the benign traffic in the target containerized cluster, and the target node represents the node that receives the benign traffic in the target containerized cluster; based on the node types of the source node and target node corresponding to each benign traffic in the benign traffic sequence, a load node set, an API server set, a load node set communicating with API servers, and a load node set communicating with external nodes are obtained; wherein the load node set, the API server set, the load node set communicating with API servers, and the load node set communicating with external nodes constitute the topology of the target containerized cluster. The topology of the target containerized cluster can be obtained using code as shown in Figure 2.

[0057] Table 2

[0058]

[0059] II. First Testing Phase

[0060] 2.1 Maximum / Minimum Value Detection

[0061] Before introducing the specific extreme value detection operation, let's first explain why extreme value detection can be used to identify lateral movement traffic.

[0062] Lateral movement attacks alter the traffic patterns of load, hosts, and API servers in containerized clusters, resulting in differences between the generated lateral movement traffic and benign traffic. These differences are primarily twofold: First, for a small but critical amount of lateral movement traffic, it transmits large amounts of data, causing some batch transmission characteristics and packet length characteristics to exceed the range of benign traffic (this portion of lateral movement traffic can be detected through extremum detection). This is because the load is under the control of the attacker's C&C server, where the attacker installs malware or scripts and steals critical information from the API server. Second, for most lateral movement traffic, although its characteristic distribution differs from benign traffic, its characteristic values ​​remain within the range of benign traffic, and lateral movement traffic often has lower data transmission requirements (this portion of lateral movement traffic cannot be detected through extremum detection).

[0063] To more intuitively understand the difference in value range between lateral movement traffic and benign traffic, we take the active and idle features as examples to analyze the value range distribution of lateral movement traffic and benign traffic in these features, and obtain the following results: Figure 3 The box plot shown is shown. Among them, Figure 3(a) is a box plot of the maximum duration of continuous activity, and Figure 3 (a) The upper and middle layers represent the maximum continuous active duration corresponding to benign traffic. Figure 3 (a) The middle and lower layers represent the maximum continuous active duration corresponding to lateral movement traffic; Figure 3 (b) is a box plot of the standard deviation of continuous active duration, and Figure 3 (b) The upper-middle layer represents the standard deviation of the continuous active duration corresponding to benign traffic. Figure 3 (b) The lower and middle layers represent the standard deviation of the continuous active duration corresponding to lateral movement flow. (From...) Figure 3 It can be seen that, regardless of whether it is benign traffic or lateral movement traffic, the data distribution under this characteristic exhibits a long-tailed distribution, and the boxes and lines of the box plot are concentrated on the left side of the graph, almost invisible. At the same time, most of the space in the graph is occupied by outliers of the box plot. Among these outliers, the rightmost outlier of lateral movement traffic is further to the right than that of benign traffic, indicating that the maximum value and standard deviation of the continuous active duration of lateral movement traffic are larger. This means that after the containerized cluster suffers a lateral movement attack, some nodes in the network may exchange data more frequently, which leads to a larger maximum value of the continuous active duration of the session, and also drives up its standard deviation.

[0064] To more intuitively understand the data transmission differences between lateral movement traffic and benign traffic, we will analyze the data transmission behavior of lateral movement traffic and benign traffic in length-related features, using packet length as an example, and obtain the following results: Figure 4 The density map shown. Among them, Figure 4 (a) is a density map of the maximum forward packet length; Figure 4 (b) is a density plot of the average forward packet length; Figure 4 (c) is the density plot of the standard deviation of the forward packet length; and Figure 4 The blue area represents benign flow, and the yellow area represents lateral movement flow. Figure 4 It can be seen that, compared with benign traffic, lateral movement traffic has lower data transmission requirements, and lateral movement traffic is hidden among benign traffic.

[0065] Based on the above analysis, it can be seen that lateral movement traffic generated after a containerized cluster suffers a mobile attack has some feature values ​​exceeding those of benign traffic. Therefore, the inventors propose using extremum detection to detect lateral movement traffic. Extremum detection includes: analyzing whether each network traffic in the traffic sequence to be detected is lateral movement traffic; that is, network traffic whose target feature value is outside the range of that target feature is considered lateral movement traffic; and using all lateral movement traffic obtained from the extremum detection of the traffic sequence to be detected as the first detection result. The code shown in Figure 3 can be used for extremum detection.

[0066] Table 3

[0067]

[0068] 2.2 Topology Detection

[0069] Before introducing the specific topology detection operation, let's first explain why topology detection can be used to identify lateral movement traffic.

[0070] A containerized cluster is an architecture that combines multiple containers for management and orchestration. There are two common topologies for containerized clusters: a master-slave structure, which includes one master node and multiple slave nodes. The master node is responsible for managing the overall cluster status and scheduling tasks, while the slave nodes receive instructions from the master node and run containerized applications. The other is a peer-to-peer (P2P) structure, where all nodes are equal, there is no master-slave distinction, and each node can independently receive task requests, run containers, and communicate and collaborate with other nodes. Node types in a containerized cluster include load balancers, API servers, and external nodes.

[0071] Lateral movement attacks alter the topology of a containerized cluster, specifically the connections between nodes. These changes include: the addition of load balancers; the emergence of communication between load balancers that previously did not require it; the addition of nodes communicating with the API server, or the emergence of communication between nodes that had never communicated with the API server; and the emergence of communication between load balancers that had never accessed external nodes. Specifically, the appearance of a new load balancer unrelated to the cluster's scaling strategy indicates that the attacker illegally created a new load balancer via the API server. This load balancer may have special privileges, allowing the attacker to move it from the load balancer to a host. The emergence of communication between load balancers that previously did not require it indicates lateral movement between load balancers. The addition of nodes communicating with the API server, or the emergence of communication between nodes that had never communicated with the API server, indicates that the load balancer started communicating with the API server after the attack to obtain the cluster's configuration files and perform resource operations. The emergence of communication between load balancers that had never accessed external nodes indicates that the load balancer has been controlled by the attacker and is communicating with the attacker's server.

[0072] Based on the above analysis, it is known that lateral movement attacks will cause changes in the topology of containerized clusters. Therefore, the inventors propose using topology detection to detect lateral movement traffic. Topology detection includes: analyzing whether the transmission process of each network traffic in the traffic sequence to be detected conforms to the topology of the target containerized cluster. If it does not conform, the network traffic is considered lateral movement traffic, and all lateral movement traffic obtained from the topology detection of the traffic sequence to be detected is used as a second detection result.

[0073] According to an embodiment of the present invention, the method includes analyzing whether the transmission process of each network traffic in the traffic sequence to be detected satisfies the topology of the target containerized cluster in the following manner: if the source node or target node corresponding to the network traffic is a new load node and is not in the load node set, then the network traffic is lateral movement traffic; if the source node corresponding to the network traffic is not in the load node set communicating with the API server, and the target node corresponding to the network traffic is the API server, then the network traffic is lateral movement traffic; if the target node corresponding to the network traffic is not in the load node set communicating with the API server, and the source node corresponding to the network traffic is the API server, then the network traffic is lateral movement traffic; if the source node corresponding to the network traffic is not in the load node set communicating with external nodes, and the target node corresponding to the network traffic is an external node, then the network traffic is lateral movement traffic; if the target node corresponding to the network traffic is not in the load node set communicating with external nodes, and the source node corresponding to the network traffic is an external node, then the network traffic is lateral movement traffic. The topology detection can be performed using the code shown in Table 4.

[0074] Table 4

[0075]

[0076] To verify the feasibility of extremum detection and topology detection, the Kubernetes-dataset was used as the dataset for these tests. The extremum detection results are shown in Table 5, and the topology detection results are shown in Table 6. In the Kubernetes-dataset, benign traffic and lateral movement traffic are recorded separately. Therefore, lateral movement traffic was injected at the end of the benign traffic sequence, while preserving the temporal relationships between lateral movement traffic and benign traffic, and aligning the end times of lateral movement traffic and benign traffic.

[0077] As shown in Table 5, the extreme value detection can detect 5 lateral movement flows, while 3 other benign flows are falsely reported as lateral movement flows. As shown in Table 6, the topology detection detects 8 lateral movement flows without generating any false alarms.

[0078] Table 5

[0079]

[0080] Table 6

[0081]

[0082] III. Second Testing Phase

[0083] The detection phase of the flow sequence to be detected can detect some lateral movement flow, but some lateral movement flow still goes undetected. To improve the detection rate of lateral movement flow, the inventors propose using a pre-trained lateral movement detection model to further process the flow sequence to be detected. Based on this, in the second detection phase, the pre-trained lateral movement detection model is used to detect the flow in the flow sequence to be detected in order to analyze the lateral movement flow in the flow sequence and obtain the third detection result.

[0084] 3.1 Model Detection

[0085] According to one embodiment of the present invention, a pre-trained lateral movement detection model is configured to perform traffic detection on a traffic sequence to be detected as follows: taking the traffic sequence to be detected as input, a target prediction sequence of the traffic sequence to be detected is generated according to a recursive prediction method; wherein, the target prediction sequence includes multiple target prediction network traffic, and each target prediction network traffic corresponds to a network traffic in the traffic to be detected; the error value between each target prediction network traffic and its corresponding actual network traffic is calculated, and if the error value is greater than or equal to a threshold, the network traffic is determined to be lateral movement traffic. It should be noted that generating the target prediction sequence of the traffic sequence to be detected according to a recursive prediction method, for example, assumes a traffic sequence to be detected of length n. The lateral movement detection model processes the traffic sequence to be detected. At that time, with network traffic Predicting network traffic With network traffic and Predicting network traffic With network traffic to Predicting network traffic Based on this, the target predicted network traffic sequence is obtained as follows: It should also be noted that this is based on the traffic sequence to be detected. The predicted target prediction network sequence does not contain network traffic. The corresponding target is to predict network traffic. There are two scenarios: one is that there is a preceding traffic sequence to be detected, and the network traffic has already been predicted in that preceding traffic sequence. The corresponding target is to predict network traffic; another approach is to predict network traffic by applying padding. The corresponding target is predicted network traffic.

[0086] 3.2 Lateral Movement Detection Model

[0087] According to one embodiment of the present invention, the pre-trained lateral movement detection model is a model trained as follows: obtaining a training benign traffic sequence, wherein the training benign traffic sequence includes multiple temporally consecutive benign traffic sequences, and each benign traffic sequence includes multiple traffic features; performing feature extraction on the training benign traffic sequence to obtain a target feature set corresponding to each benign traffic sequence in the training benign traffic sequence; and performing multiple rounds of iterative training until the lateral movement detection model converges, using the feature-extracted benign traffic sequence as input and the target prediction sequence corresponding to the feature-extracted benign traffic sequence as output.

[0088] According to one embodiment of the present invention, the pre-trained lateral movement detection model includes a Transformer model and a judgment module, wherein: the Transformer model is used to predict the target prediction sequence corresponding to the traffic sequence to be detected; the judgment module is used to calculate the error value between each target prediction network traffic and its corresponding actual network traffic, and when the error value is greater than or equal to a threshold, the network traffic is judged to be lateral movement traffic. It should be noted that the lateral movement traffic detection model can also use other neural network models to replace the Transformer model, such as an RNN model.

[0089] According to one embodiment of the present invention, such as Figure 5 As shown ( Figure 5 (The judgment module is omitted in the text). The pre-trained lateral movement detection model includes a spatial feature embedding module, a temporal feature embedding module, an encoding / decoding module, a sequence output module, and a judgment module. The following sections describe each module in the lateral movement detection model.

[0090] The spatial feature embedding module is used to extract spatial features from the traffic sequence to be detected, obtaining a spatial feature embedding vector for each network traffic in the sequence. This spatial feature embedding vector is then concatenated with the traffic features of that network traffic to obtain the initial features of the traffic sequence. It should be noted that spatial feature extraction is performed on the traffic sequence to be detected because network traffic in a containerized cluster follows certain patterns, from which spatial features can be extracted. If network traffic is modeled as a graph, the nodes represent combinations of IP addresses and ports corresponding to nodes in the target containerized cluster, and the edges represent network traffic generated between nodes.

[0091] According to one embodiment of the present invention, the spatial feature embedding module is a graph convolutional network (GCN). It should be noted that the spatial feature embedding module can also employ other network models for processing graph-structured data, such as GraphSAGE, GAT, DeepWalk, and Node2vec. The present invention does not impose any special limitations on the spatial feature embedding module.

[0092] To better understand the working mode of the spatial feature embedding module, we will use a Graph Convolutional Network (GCN) as an example to illustrate how to extract spatial features from the traffic sequence to be detected. Figure 5 As shown, the spatial feature embedding module takes the graph data (containing multiple nodes and edges connecting two nodes) constructed from the traffic sequence to be detected as input. It uses multiple network layers for information propagation to extract the spatial embedding vector of each node in the graph data. During information propagation at each layer, it extracts the weighted average of all neighbor features and the features of each node itself to obtain the spatial embedding vector of each node corresponding to that layer. This spatial embedding vector is then passed to the next network layer for processing until all network layers have completed information propagation and the final spatial embedding vector of each node is obtained. After obtaining the spatial embedding vectors of all nodes, they are concatenated with the traffic features of each network traffic in the traffic sequence to be detected to obtain the initial feature output of the traffic sequence. The initial features of the traffic sequence to be detected include the initial features corresponding to each network traffic, and each initial feature corresponding to a network traffic is formed by concatenating the spatial embedding vector of the source node, the spatial embedding vector of the target node, and the traffic features corresponding to that network traffic.

[0093] It should be noted that the spatial feature embedding module does not participate in updates during multi-round iterative training; it is trained independently using benign traffic sequences. It should also be noted that during lateral movement detection, the traffic sequence to be detected may contain new nodes not seen by the spatial feature embedding module, therefore the spatial embedding vectors of these new nodes need to be considered. Depending on the containerized cluster network structure, new nodes could be new API servers, new load balancers, new hosts, or new external nodes. New hosts require configuration by the containerized cluster administrator, making them less likely to be created by attackers and therefore not considered by the spatial feature embedding module. New API servers depend on new load balancers, which were already filtered out in the previous topology detection stage and are also not considered by the spatial feature embedding module. Therefore, new nodes in the traffic sequence to be detected can only be new external nodes. For new external nodes, the spatial feature embedding module randomly generates a vector as the spatial embedding vector for that node.

[0094] The time feature embedding module is used to extract time features based on the initial features of the traffic sequence to be detected to obtain the time feature embedding vector of the traffic sequence to be detected. According to one embodiment of the present invention, the time feature embedding module is a Long Short-Term Memory (LSTM) network, and the number of network layers of the LSTM network can be set to 3. It should be noted that the number of network layers of the LSTM network is determined by actual needs, and the present invention does not impose any special restrictions.

[0095] like Figure 5 As shown, the encoding / decoding module includes an encoder, a first decoder, and a second decoder; wherein: the encoder is used to extract the dependencies between each network traffic in the traffic sequence to be detected based on the initial features of the traffic sequence to be detected, to obtain a first latent vector of the traffic sequence to be detected; and to re-extract the dependencies between each network traffic in the traffic sequence to be detected based on the initial features of the traffic sequence to be detected and the error between each initial predicted network traffic obtained by the sequence output module and its corresponding actual network traffic in the traffic sequence to be detected, to obtain a second latent vector of the traffic sequence to be detected; the first decoder is used to perform decoding processing based on the first latent vector of the traffic sequence to be detected and the temporal feature embedding vector to obtain a first decoded feature vector of the traffic sequence to be detected; the second decoder is used to perform decoding processing based on the second latent vector of the traffic sequence to be detected and the temporal feature embedding vector to obtain a second decoded feature vector of the traffic sequence to be detected; the sequence output module is used to generate an initial prediction sequence of the traffic sequence to be detected based on the first decoded feature vector of the traffic sequence to be detected, and to generate a target prediction sequence of the traffic sequence to be detected based on the second decoded feature vector of the traffic sequence to be detected; wherein, the initial prediction sequence includes multiple initial predicted network traffic, and each initial predicted network traffic corresponds to a network traffic in the traffic sequence to be detected.

[0096] Among them, such as Figure 5 As shown, the decoder includes a multi-head attention module, a first summation and regularization module, a feedforward network, and a second summation and regularization module. The multi-head attention module is used to mine the correlation and dependency relationships between network traffic. The first summation and regularization module is used to integrate the network traffic and the network traffic processed by the multi-head attention module into a unified representation and pass it to the feedforward network. The feedforward network is used to further process the processing result of the first summation and regularization module to learn the complex semantic relationships in each network traffic. The second summation and regularization module is used to integrate the processing result of the feedforward network and the processing result of the first summation and regularization module into a unified representation and pass it to the first decoder or the second decoder.

[0097] like Figure 5 As shown, both the first and second decoders contain a multi-head attention module and an additive regularization module. The multi-head attention modules in both the first and second decoders are used to mine the correlation between the output of the temporal feature embedding module and the decoder output; the additive regularization module is used to integrate the processing results of the multi-head attention modules into a unified representation and pass it to the sequence output module.

[0098] like Figure 5As shown, the sequence output module includes a feedforward network and a sigmoid activation function. The feedforward network is used to further process the output of the first decoder or the second decoder, and after processing by the sigmoid function, the corresponding initial prediction sequence and target prediction sequence are obtained.

[0099] The judgment module is used to calculate the error value between the predicted network traffic of each target and its corresponding actual network traffic, and when the error value is greater than or equal to the threshold, the network traffic is judged to be lateral movement traffic.

[0100] To better understand how pre-trained lateral movement detection models work, the following will use... Figure 5 Taking the lateral movement detection model shown as an example, the specific detection process will be explained. Specifically, assume the traffic sequence to be detected is... ( for Current network traffic for (Network traffic at any given time). During the detection process, graph data constructed based on the traffic sequence to be detected is first input into the lateral movement detection model. The spatial feature embedding module in the lateral movement detection model extracts spatial features from the graph data corresponding to the traffic sequence to be detected, thereby obtaining the traffic sequence to be detected. The corresponding spatial feature embedding vector is obtained, and the spatial feature embedding vector of each network traffic is concatenated with the traffic features of that network traffic to obtain the initial features of the traffic sequence to be detected. ,in, Includes network traffic corresponding source node Spatial embedding vector, target node Spatial embedding vectors and flow features, Includes network traffic corresponding source node Spatial embedding vector, target node The spatial embedding vector and traffic features are then used; then, the initial features of the traffic sequence to be detected are... The data is passed to the temporal feature embedding module and the encoding / decoding module, whereby the temporal feature embedding module is used to obtain initial features based on the traffic sequence to be detected. Temporal features are extracted to obtain the temporal feature embedding vector of the traffic sequence to be detected, and the temporal feature embedding vector of the traffic sequence to be detected is passed to the encoding and decoding module; then, a two-step prediction is performed, wherein in the first prediction step, the encoder, the first decoder, and the sequence output module are used to predict the network traffic. Corresponding initial predicted network traffic (Simplified description, only based on network traffic) Predicting network traffic The process of calculating the initial predicted network traffic is as follows. With network traffic Error between In the second step of prediction, the error is... Initial features of the flow sequence to be detected The input is then fed back into the encoder (at this point, the neurons in the encoder connected to the error vector will be activated, and the self-attention mechanism will focus more on the part of the prediction that contained errors in the first step), and the network traffic is predicted using the encoder, the second decoder, and the sequence output module. Corresponding target predicted network traffic Finally, the judgment module calculates the target predicted network traffic. The corresponding actual network traffic The error value between the two thresholds is used to determine whether the network traffic is lateral movement traffic. If the error value is greater than or equal to the threshold, the network traffic is considered to be benign traffic.

[0101] It should be noted that, Figure 5 The lateral movement detection model shown can use a loss function during multiple rounds of iterative training. To calculate the loss, and update the model parameters by minimizing the loss, where, This represents the error between the network traffic and its corresponding initial predicted network traffic. This represents the error between the network traffic and its corresponding target predicted network traffic. Indicates weight, and It decreases with each training iteration.

[0102] It should also be noted that, Figure 5 The lateral movement detection model shown uses the error between the target predicted network traffic and its corresponding actual network traffic to determine whether the corresponding network traffic is lateral movement traffic. Since the error between the target predicted network traffic and its corresponding actual network traffic is the final error formed by two prediction steps, it can minimize the prediction error of benign traffic, thereby making the prediction error of lateral movement traffic relatively large, thus reducing the false alarm rate of the model and improving the detection rate of the model.

[0103] IV. Test Result Output Stage

[0104] In the detection result output stage, after removing the network traffic that was repeatedly identified as lateral movement traffic in the first, second, and third detection results, all the remaining lateral movement traffic is taken as the final detection result of the traffic sequence to be detected.

[0105] To improve the performance of the lateral movement detection model proposed in this invention in a containerized environment, the model proposed in this invention will be used as follows: Figure 5The lateral movement detection model shown is compared with other existing lateral movement detection models proposed by researchers. These models included those proposed by Bowman, the Euler model, the TGN model, the USAD model, the OmniAnomaly model, and the TranAD model.

[0106] In the comparative experiment, the parameters of each model were set as shown in Table 7, and the same training data and the same test data were used to evaluate the TPR, FPR, F1 and AUC scores of each model, and the experimental results are shown in Table 8.

[0107] In Table 7, the meanings of each parameter are as follows: For the Bowman model: p represents the probability of repeatedly visiting a node, q represents the control parameter for interpolation between breadth-first and depth-first strategies, walk_length represents the walk length, context_size represents the actual context size considering the samples, improving the effective sampling rate by reusing samples between different source nodes; walks_per_node represents the number of samples per node. For the Euler model: delta: euler represents the time-dependent parameter of the model, gnn represents the graph convolutional network, and rnn represents the gated recurrent unit. For the TGN model, Batch_size represents the number of data (samples) passed to the program for training in a single batch. For the USAD model, n_hidden represents the number of hidden parameters, n_latent represents the latent space dimension, and n_window represents the window size. For the OmniAnomaly model: Beta represents the scale parameter of the tail of the fitted probability distribution of the generalized Pareto distribution, n_hidden represents the number of hidden parameters, n_latent represents the latent space dimension, and n_window represents the window size. In the TranAD model, n_window represents the window size, rnn represents a three-layer LSTM, and gnn represents a graph convolutional network. The model proposed in this invention uses n_window to represent the window size, rnn to represent a three-layer LSTM, and gnn to represent a graph convolutional network.

[0108] Table 8 shows that the model proposed by Bowman achieved the lowest score because it did not utilize temporal information and only performed link prediction on a static graph. The Euler model, employing a dynamic graph-based link prediction method, achieved a slightly higher AUC score; however, its performance was lower because it required dividing continuous time into discrete time slices, resulting in coarse temporal information. The TGN model modeled continuous time without the need for time slices, thus achieving the highest AUC score compared to both Bowman's and Euler models. However, the AUC scores of these three models were all lower than the others, indicating that graph-based models are not suitable for containerized environments. Furthermore, experimental results from the TGN model suggest that lateral movement detection should be considered from a global perspective, rather than just considering local information between adjacent nodes on the graph.

[0109] The USAD, OmniAnomaly, and TranAD models all achieved AUC scores above 0.8, indicating that encoder-decoder architecture and adversarial training methods can improve detection performance. Among these three models, the TranAD model achieved a significantly higher AUC score of 0.91, demonstrating that the self-attention mechanism is particularly suitable for lateral movement detection scenarios.

[0110] The model proposed in this invention has the highest AUC score of 0.96, and its detection rate and false alarm rate are both superior to the TranAD model.

[0111] Regarding training time overhead, the models proposed by Bowman and Euler have the shortest training times because they do not consider fine-grained temporal information, resulting in insufficient model capacity. Among the remaining models, the model proposed in this invention has the shortest training time. This is because, compared to TGN, this paper does not couple temporal information with spatial structure during training; and compared to USAD, OmniAnomaly, and TranAD, the method proposed in this paper removes the adversarial training mechanism, thus shortening the training time.

[0112] Table 7

[0113]

[0114] Table 8

[0115]

[0116] The beneficial effects of the present invention are as follows: (1) It extracts data packet-level traffic features and session-level traffic features, and also eliminates traffic features with lower importance based on the importance ranking results of traffic features, thus solving the problem of unclear lateral movement detection features in containerized clusters; (2) It sets up two-stage detection to screen lateral movement traffic, thereby improving the detection rate while reducing the false alarm rate; (3) The lateral movement detection model uses two-step prediction to minimize the prediction error of benign traffic and maximize the prediction error of lateral movement traffic, thereby achieving more accurate lateral movement traffic detection.

[0117] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0118] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0119] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0120] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for detecting lateral movement traffic, used to detect whether lateral movement traffic exists in a sequence of traffic to be detected generated in a target containerized cluster, wherein, The traffic sequence to be detected includes multiple network traffic flows that are sequentially continuous, and the laterally moving traffic is traffic data generated when the target containerized cluster is attacked. The method is characterized by comprising: Preprocessing stage: Obtain a benign traffic sequence containing multiple benign traffic generated by the target containerized cluster, perform feature extraction on the benign traffic sequence to obtain a target feature set containing multiple target features, traverse each target feature in each benign traffic in the benign traffic sequence to obtain the value range corresponding to each target feature, and perform topology extraction based on the benign traffic sequence to obtain the topology structure of the target containerized cluster. First testing phase: Maximum / minimum value detection: Analyze whether each network traffic in the traffic sequence to be detected is lateral movement traffic; where each network traffic includes multiple target features, and network traffic whose value of any target feature is not within the range of the target feature is lateral movement traffic; all lateral movement traffic obtained by the maximum / minimum value detection of the traffic sequence to be detected constitutes the first detection result; Topology detection: Analyze whether the transmission process of each network traffic in the traffic sequence to be detected satisfies the topology of the target containerized cluster. If it does not, the network traffic is lateral movement traffic. Among them, all lateral movement traffic obtained by topology detection of the traffic sequence to be detected constitutes the second detection result. Second testing phase: A pre-trained lateral movement detection model is used to perform flow detection on the flow sequence to be detected in order to analyze the lateral movement flow in the flow sequence to obtain a third detection result; Detection result output stage: After removing network traffic that was repeatedly identified as lateral movement traffic in the first, second, and third detection results, all remaining lateral movement traffic is taken as the final detection result of the traffic sequence to be detected.

2. The method according to claim 1, characterized in that, The method includes extracting features from a benign flow sequence to obtain a target feature set containing multiple target features in the following manner: Each benign traffic sequence contains multiple data packets. Based on all the data packets of any benign traffic, data packet-level feature extraction is performed on that benign traffic to obtain multiple data packet-level traffic features corresponding to that benign traffic. And based on the packet connection protocol, all packets in the benign traffic are aggregated into multiple sessions to extract multiple session-level traffic features corresponding to the benign traffic; wherein, all packet-level traffic features and all session-level traffic features constitute the initial feature set; The importance of each traffic feature in the initial feature set is evaluated using a preset evaluation method to obtain the importance evaluation result of each traffic feature; The importance assessment results are sorted from highest to lowest, and a preset number of traffic features at the top of the list are selected as target features.

3. The method according to claim 2, characterized in that, The preset evaluation method is as follows: Gradient boosting decision tree, random forest, and mutual information are used to evaluate the importance of each flow feature in the initial feature set, and the mean importance of each flow feature is calculated to obtain the importance evaluation result of each flow feature.

4. The method according to claim 3, characterized in that, The preset number is 57.

5. The method according to claim 4, characterized in that, The target characteristics include: standard deviation of continuous idle time, average number of bytes transmitted in forward batch packets, minimum forward packet length, average forward batch packet rate, forward RST flag, number of forward active packets, minimum backward inter-frame arrival time, backward PSH flag, minimum forward segment size, initial backward window value, total TCP stream time, PSH flag count, ACK flag count, average forward inter-frame arrival time, FIN flag count, backward header length, minimum inter-stream arrival time, minimum forward inter-frame arrival time, number of forward packets, average packet arrival time interval, total backward packet length, SYN flag count, maximum packet length, average backward inter-frame arrival time, ratio of backward to forward packets, number of backward packets, forward header length, stream duration in milliseconds, and forward packet arrival time. Total intervals, total backward packet arrival time intervals, average forward packet length, average forward segment size, forward PSH flag, standard deviation of packet arrival time intervals, standard deviation of forward packet arrival time intervals, number of bytes in backward sub-stream, initial value of forward window, standard deviation of backward packet length, average packet length, average backward packet length, packet length variation, standard deviation of backward packet arrival time intervals, maximum packet arrival time interval, average backward segment size, maximum backward packet arrival time interval, maximum forward packet arrival time interval, standard deviation of packet length, average packet length, forward flow rate, total forward packet length, standard deviation of forward packet length, number of bytes in forward sub-stream, total backward packet length, packet flow rate, byte flow rate, backward packet flow rate, and maximum forward packet length.

6. The method according to claim 5, characterized in that, The method includes obtaining the topology of the target containerized cluster by performing topology extraction based on benign traffic sequences in the following manner: Each benign traffic in a benign traffic sequence corresponds to a source node and a target node; where the source node represents the node that sends the benign traffic in the target containerized cluster, and the target node represents the node that receives the benign traffic in the target containerized cluster. Based on the node types of the source and target nodes corresponding to each benign traffic in the benign traffic sequence, the load node set, API server set, load node set communicating with API servers, and load node set communicating with external nodes are obtained; among them, the load node set, API server set, load node set communicating with API servers, and load node set communicating with external nodes constitute the topology of the target containerized cluster.

7. The method according to claim 6, characterized in that, The method includes analyzing whether the transmission process of each network traffic in the traffic sequence to be detected satisfies the topology of the target containerized cluster in the following manner: If the source or destination node corresponding to network traffic is a new load node and is not in the load node set, then the network traffic is lateral movement traffic. If the source node of the network traffic is not in the set of load nodes communicating with the API server, and the target node of the network traffic is the API server, then the network traffic is lateral movement traffic. If the target node of the network traffic is not in the set of load nodes communicating with the API server, and the source node of the network traffic is the API server, then the network traffic is lateral movement traffic. If the source node corresponding to the network traffic is not in the set of load nodes communicating with external nodes, and the target node corresponding to the network traffic is an external node, then the network traffic is lateral movement traffic. If the target node of the network traffic is not in the set of load nodes communicating with external nodes, and the source node of the network traffic is an external node, then the network traffic is lateral movement traffic.

8. The method according to claim 7, characterized in that, The pre-trained lateral movement detection model is configured to perform traffic detection on the traffic sequence to be detected as follows: Using the traffic sequence to be detected as input, a target prediction sequence is generated for the traffic sequence to be detected in a recursive prediction manner; wherein, the target prediction sequence includes multiple target prediction network traffic, and each target prediction network traffic corresponds to one network traffic in the traffic to be detected. Calculate the error value between the predicted network traffic for each target and its corresponding actual network traffic. If the error value is greater than or equal to the threshold, the network traffic is determined to be lateral movement traffic.

9. The method according to claim 8, characterized in that, The pre-trained lateral movement detection model is a model trained as follows: Obtain a benign traffic sequence for training, wherein the benign traffic sequence for training includes multiple benign traffic sequences that are consecutive in time, and each benign traffic sequence includes multiple traffic features; Feature extraction is performed on the benign traffic sequence used for training to obtain the target feature set corresponding to each benign traffic in the benign traffic sequence used for training; The model takes the benign traffic sequence after feature extraction as input and the target prediction sequence corresponding to the benign traffic sequence after feature extraction as output, and performs multiple rounds of iterative training until the lateral movement detection model converges.

10. The method according to claim 9, characterized in that, The pre-trained lateral movement detection model includes a Transformer model and a decision module, wherein: The Transformer model is used to predict the target prediction sequence corresponding to the traffic sequence to be detected. The judgment module is used to calculate the error value between each target predicted network traffic and its corresponding actual network traffic, and when the error value is greater than or equal to the threshold, the network traffic is judged to be lateral movement traffic.

11. The method according to claim 9, characterized in that, The pre-trained lateral movement detection model includes a spatial feature embedding module, a temporal feature embedding module, an encoding / decoding module, a sequence output module, and a judgment module; wherein: The spatial feature embedding module is used to extract the spatial features of the traffic sequence to be detected to obtain the spatial feature embedding vector of each network traffic in the traffic sequence to be detected, and to connect the spatial feature embedding vector of each network traffic with the traffic features of the network traffic to obtain the initial features of the traffic sequence to be detected. The time feature embedding module is used to extract time features based on the initial features of the traffic sequence to be detected to obtain the time feature embedding vector of the traffic sequence to be detected. The encoding / decoding module includes an encoder, a first decoder, and a second decoder, wherein: The encoder is used to extract the dependencies between network traffic in the traffic sequence to be detected based on the initial features of the traffic sequence to be detected, so as to obtain the first latent vector of the traffic sequence to be detected; and to re-extract the dependencies between network traffic in the traffic sequence to be detected based on the initial features of the traffic sequence to be detected and the error between each initial predicted network traffic obtained by the sequence output module and the actual network traffic in its corresponding traffic sequence to be detected, so as to obtain the second latent vector of the traffic sequence to be detected. The first decoder is used to perform decoding processing based on the first latent vector and the temporal feature embedding vector of the traffic sequence to be detected to obtain the first decoded feature vector of the traffic sequence to be detected. The second decoder is used to perform decoding processing based on the second latent vector and the temporal feature embedding vector of the traffic sequence to be detected to obtain the second decoded feature vector of the traffic sequence to be detected. The sequence output module is used to generate an initial prediction sequence of the traffic sequence to be detected based on a first decoded feature vector of the traffic sequence to be detected, and to generate a target prediction sequence of the traffic sequence to be detected based on a second decoded feature vector of the traffic sequence to be detected; wherein, the initial prediction sequence includes multiple initial prediction network traffic, and each initial prediction network traffic corresponds to a network traffic in the traffic sequence to be detected. The judgment module is used to calculate the error value between each target predicted network traffic and its corresponding actual network traffic, and when the error value is greater than or equal to the threshold, the network traffic is judged to be lateral movement traffic.

12. The method according to claim 11, characterized in that, The spatial feature embedding module is a graph convolutional network, and the temporal feature embedding module is a long short-term memory network.

13. A computer-readable storage medium, characterized in that, It contains a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1-12.

14. An electronic device, characterized in that, include: One or more processors, and memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1-12 by executing the executable instructions.

Images