A training method for a horizontal moving traffic detection model of containerized clusters
By extracting and filtering packet-level and session-level traffic features in containerized clusters, a lateral movement traffic detection model is constructed, which solves the problem of unclear detection features in containerized clusters and achieves more accurate lateral movement detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INST OF COMPUTING TECH CHINESE ACAD OF SCI
- Filing Date
- 2025-01-23
- Publication Date
- 2026-07-31
AI Technical Summary
Existing lateral movement detection technologies are difficult to apply directly to containerized clusters because containerized clusters lack user authentication information and feature importance differences, resulting in unclear and inaccurate detection features.
By extracting packet-level and session-level traffic features from containerized clusters, gradient boosting decision trees, random forests, and mutual information are used to evaluate feature importance, important features are selected, a lateral movement traffic detection model is constructed, and the Transformer model is used for iterative training.
It enables more accurate lateral movement detection in containerized clusters, improving detection accuracy and efficiency.
Smart Images
Figure CN120074879B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of network security, specifically to lateral movement traffic detection technology in the field of network security, and more specifically, to a training method for a lateral movement traffic detection model for containerized clusters. Background Technology
[0002] With the rapid advancement of cloud computing technology, cloud-native container platforms have become an indispensable driving force for enterprise digital transformation. However, the widespread application of containerized clusters has also brought unprecedented security risks, with various attack incidents emerging one after another, making security issues increasingly prominent. Containerized attacks often follow a set process: reconnaissance, establishing a foothold, lateral movement, attack, and cleanup. The lateral movement phase is particularly critical because at this point, attackers have infiltrated the cluster, begun gathering information, and are searching for their next target.
[0003] To achieve lateral movement detection, researchers have proposed various techniques, primarily focusing on four dimensions: network traffic, endpoint behavior, user behavior, and threat intelligence. However, most existing techniques have limitations: they either rely too heavily on user behavior information, require known threat intelligence as support, or are only applicable to specific system environments, making them difficult to directly apply to containerized clusters.
[0004] Furthermore, existing lateral movement detection technologies typically identify and detect lateral movement behavior by analyzing data extracted from datasets. These datasets mostly originate from enterprise internal networks and record various events within specific time periods, such as Windows authentication events, process events, network traffic events, and DNS events, or more detailed event types, such as network traffic, authentication, DHCP protocols, DNS protocols, file operations, HTTP protocols, and SSL protocols. However, these datasets are based on host events collected within the enterprise network, which differs significantly from network traffic-based data in containerized clusters, and therefore cannot be directly used for lateral movement detection in containerized environments.
[0005] While lateral movement detection is feasible in environments such as enterprise networks, lateral movement detection technologies for containerized clusters remain relatively scarce. Currently, lateral movement detection in containerized clusters suffers from two main shortcomings: firstly, unlike enterprise network environments, lateral movement in containerized clusters typically does not include user authentication information, resulting in unclear features suitable for lateral movement detection; secondly, different features have varying degrees of importance, requiring the filtering and removal of less important features to achieve more accurate lateral movement detection.
[0006] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solutions of the present invention, and does not imply that the relevant information is necessarily prior art. In the absence of evidence indicating that the relevant information was disclosed before the filing date of this invention, the relevant information should not be considered prior art. Summary of the Invention
[0007] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a training method for a lateral movement traffic detection model for containerized clusters, a network traffic detection system, and a network traffic detection method.
[0008] The objective of this invention is achieved through the following technical solutions.
[0009] According to a first aspect of the present invention, a training method for a lateral movement traffic detection model for containerized clusters is provided for training a model capable of detecting lateral movement traffic in a containerized cluster, wherein the lateral movement traffic is traffic data generated after the containerized cluster is attacked. The method includes: a feature analysis phase: acquiring historical network traffic generated in the containerized cluster, wherein the historical network traffic includes multiple data packets; performing data packet-level feature extraction on the historical network traffic based on all data packets to obtain multiple data packet-level traffic features corresponding to the historical network traffic; and aggregating all data packets in the historical network traffic into multiple sessions based on a packet connection protocol to extract multiple session-level traffic features corresponding to the historical network traffic; wherein all data packet-level traffic features and session-level traffic features constitute an initial feature set of the historical network traffic; and evaluating the initial features using a preset evaluation method. Each traffic feature in the set is evaluated for importance to obtain the importance evaluation result of each traffic feature; the importance evaluation results are sorted in descending order, and a predetermined number of traffic features at the top of the sort are selected as target features; in the data preprocessing stage: network traffic sequences are obtained, which include multiple time-series continuous training network traffic and the traffic category corresponding to each training network traffic; target features of each training network traffic are extracted to obtain the target feature set of each training network traffic, and a training set is constructed using the target feature set of each training network traffic as samples and the traffic category corresponding to each training network traffic as labels; in the model training stage: the lateral movement traffic detection model is iteratively trained multiple times according to a predetermined objective function, using the samples in the training set as input and the predicted traffic category of the samples as output, until the model converges.
[0010] In some embodiments of the present invention, the preset evaluation method is as follows: gradient boosting decision tree, random forest and mutual information are used to evaluate the importance of each traffic feature in the initial feature set, and the mean of the importance of each traffic feature is calculated to obtain the importance evaluation result of each traffic feature.
[0011] In some embodiments of the present invention, the preset number is 57.
[0012] In some embodiments of the present invention, the lateral movement traffic detection model includes a Transformer model and a judgment module, wherein: the Transformer model is used to generate a target prediction sequence corresponding to a network traffic sequence composed of multiple time-series continuous network traffic sequences according to a recursive prediction method, wherein the target prediction sequence includes multiple target prediction network traffic sequences, and each target prediction network traffic sequence corresponds to one network traffic sequence; the judgment module is used to calculate the error value between each target prediction network traffic sequence and its corresponding actual network traffic sequence, and when the error value is greater than or equal to a threshold, the network traffic sequence is judged to be lateral movement traffic.
[0013] In some embodiments of the present invention, the preset objective function is a cross-entropy loss function.
[0014] In some embodiments of the present invention, the target features include: standard deviation of continuous idle time, average number of bytes transmitted in forward bulk data packets, minimum length of forward data packets, average rate of forward bulk data packets, forward RST flag, number of forward active data packets, minimum time to arrive between backward frames, backward PSH flag, minimum size of forward segment, initial value of backward window, total TCP stream time, PSH flag count, ACK flag count, average time to arrive between forward frames, FIN flag count, backward header length, minimum time to arrive between streams, minimum time to arrive between forward frames, number of forward data packets, average data packet arrival time interval, total length of backward data packets, SYN flag count, maximum data packet length, average time to arrive between backward frames, ratio of backward to forward data packets, number of backward data packets, forward header length, stream duration in milliseconds, and forward number of packets. The table lists the data types and their corresponding time intervals: total arrival time intervals of packets, total arrival time intervals of backward packets, average length of forward packets, average size of forward segments, forward PSH flag, standard deviation of arrival time intervals, standard deviation of arrival time intervals of forward packets, number of bytes in backward substreams, initial value of forward window, standard deviation of length of backward packets, average length of packets, average length of backward packets, change in length of packets, standard deviation of arrival time intervals of backward packets, maximum arrival time interval of packets, average size of backward segments, maximum arrival time interval of backward packets, maximum arrival time interval of forward packets, standard deviation of length of packets, average length of packets, forward flow rate, total length of forward packets, standard deviation of length of forward packets, number of bytes in forward substreams, total length of backward packets, flow rate of packets, byte flow rate, flow rate of backward packets, and maximum length of forward packets.
[0015] According to a second aspect of the present invention, a network traffic detection system is provided, the system comprising: a feature extraction module for extracting target features of network traffic to obtain a target feature set of network traffic; and a lateral movement traffic detection model trained by the method described in the first aspect of the present invention for processing the target feature set of network traffic to detect the traffic category of network traffic.
[0016] According to a third aspect of the present invention, a network traffic detection method is provided. The method includes: step T1, acquiring network traffic to be detected; step T2, detecting the network traffic to be detected using the system described in the second aspect of the present invention.
[0017] Compared with the prior art, the advantages of the present invention are: (1) extracting data packet level traffic features and session level traffic features, solving the problem of unclear lateral detection features in containerized clusters; (2) eliminating traffic features with lower importance based on the importance ranking results of traffic features, achieving more accurate lateral movement detection. Attached Figure Description
[0018] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:
[0019] Figure 1 This is a flowchart illustrating the training method of the lateral movement traffic detection model according to an embodiment of the present invention.
[0020] Figure 2 This is a schematic diagram illustrating the importance ranking of network traffic features according to an embodiment of the present invention;
[0021] Figure 3 This is a line graph showing a comparative experiment according to an embodiment of the present invention. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.
[0023] As mentioned in the background section, lateral movement detection in containerized clusters has two shortcomings: First, unlike enterprise network environments, lateral movement in containerized clusters usually does not include user authentication information, resulting in unclear features that can be used for lateral movement detection; second, different features have different levels of importance, requiring the screening and removal of less important features to achieve more accurate lateral movement detection.
[0024] To address the aforementioned issues, the inventors propose a training method to train a detection model applicable to containerized clusters. This model can detect lateral movement traffic generated when a containerized cluster is under attack, thereby achieving lateral movement detection for containerized clusters. This training method comprises three stages: feature analysis, data preprocessing, and model training. The feature analysis stage extracts packet-level and session-level traffic features from network traffic to address the issue of ambiguous features. These features are then ranked according to importance to eliminate less important traffic features, resulting in more accurate lateral movement detection. The data preprocessing stage extracts target features from the training network traffic to obtain a target feature set. The model training stage trains the detection model based on the target feature set of the training network traffic until the model converges.
[0025] In summary, such as Figure 1As shown, this invention provides a training method for a lateral movement traffic detection model for containerized clusters, used to train a model capable of detecting lateral movement traffic in containerized clusters. The lateral movement traffic refers to traffic data generated after the containerized cluster is attacked. The method includes: a feature analysis phase: acquiring historical network traffic generated in the containerized cluster, wherein the historical network traffic includes multiple data packets; performing data packet-level feature extraction on the historical network traffic based on all data packets to obtain multiple data packet-level traffic features corresponding to the historical network traffic; and aggregating all data packets in the historical network traffic into multiple sessions based on a packet connection protocol to extract multiple session-level traffic features corresponding to the historical network traffic; wherein all data packet-level traffic features and session-level traffic features constitute an initial feature set of the historical network traffic; and using a preset evaluation method to evaluate the initial feature set. Each traffic feature in the dataset is evaluated for importance to obtain the importance evaluation result for each traffic feature; the importance evaluation results are sorted in descending order, and a predetermined number of traffic features at the top of the sort are selected as target features; in the data preprocessing stage: network traffic sequences are obtained, which include multiple time-series continuous training network traffic and the traffic category corresponding to each training network traffic; target features of each training network traffic are extracted to obtain the target feature set of each training network traffic, and a training set is constructed using the target feature set of each training network traffic as samples and the traffic category corresponding to each training network traffic as labels; in the model training stage: the lateral movement traffic detection model is iteratively trained multiple times according to a predetermined objective function, using the samples in the training set as input and the predicted traffic category of the samples as output, until the model converges.
[0026] To better understand the present invention, each stage will be described in detail below with reference to specific embodiments.
[0027] I. Feature Analysis Stage
[0028] The feature analysis stage includes: acquiring historical network traffic generated in the containerized cluster, wherein the historical network traffic includes multiple data packets; performing data packet-level feature extraction on the historical network traffic based on all data packets to obtain multiple data packet-level traffic features corresponding to the historical network traffic; and aggregating all data packets in the historical network traffic into multiple sessions based on the data packet connection protocol to extract multiple session-level traffic features corresponding to the historical network traffic; wherein all data packet-level traffic features and session-level traffic features constitute an initial feature set of the historical network traffic; using a preset evaluation method to evaluate the importance of each traffic feature in the initial feature set to obtain the importance evaluation result of each traffic feature; sorting the importance evaluation results in descending order, and selecting a preset number of traffic features at the top of the sort as target features.
[0029] The packet-level traffic features are features extracted at the packet level, such as the time difference between packets in a session, the size of packets in a session, and TCP tag value statistics in a session. To better understand packet-level feature extraction, let's take extracting packet protocol field information as an example. Protocol field information can be determined by reading specific bytes in the packet header. For example, in an Ethernet frame, the byte starting from the 9th byte of the IP packet header is used to indicate the protocol type; a value of 6 indicates TCP, and a value of 17 indicates UDP.
[0030] The session-level traffic features are extracted at the session level after aggregating all data packets in historical network traffic into multiple sessions based on packet connection protocols. Examples include session duration, total bytes transmitted in the session, and total number of data packets transmitted in the session. To better understand session-level feature extraction, let's take extracting session byte traffic as an example. Session byte traffic is determined by calculating the total number of bytes sent and received during the session. Specifically, for each HTTP request and response data packet in the session, the packet length field (usually in the packet header) is read, and the lengths of all data packets are summed. It's important to note that when aggregating data packets into sessions, the packet connection protocol must be followed. For example, for TCP connections, a session begins with one party sending the first handshake data packet and ends with one party sending the last handshake data packet, resetting the connection, or a timeout. For connectionless protocols like UDP, specific aggregation methods can be used for the application layer protocol. For example, for the DNS protocol, a single query request and its corresponding result can be considered as a session. It should also be noted that there is a special type of data packet set called bulk during the process of aggregating data packets into a session (when a TCP stream transmits a large amount of data, it needs to be divided into multiple data packets for transmission, and the collection of these data packets is called bulk). When extracting features from this special type of data packet set bulk, it is necessary to first determine whether the data packets can be aggregated into bulk, and then count the number of data packets. When there are 4 data packets, the features of bulk can be extracted. Among them, the features related to bulk are usually 0, and are only greater than 0 when the amount of transmitted data is large, indicating that the stream has been split into packets.
[0031] Through packet-level and session-level feature extraction, we can extract features in seven categories: flow duration, number of packets, packet length, flow rate, packet interval, TCP flag statistics, packet header bulk transmission, and TCP window activity and idleness, totaling 79 traffic features. The traffic characteristics include: Backward URG Flags, Backward RST Flags, CWR Flag Count, ECE Flag Count, Subflow Fwd Packets, Subflow Bwd Packets, Average Reverse Batch Packet Transmission Bytes / Bulk AVg, Average Reverse Batch Packet Transmission Bytes / Bulk Avg, Forward URG Flags, Average Reverse Batch Rate Avg, URG Flag Count, Maximum Continuous Active Duration (Active Max), Minimum Continuous Active Duration (Active Min), Minimum Packet Length (Packet Length Min), Minimum Reverse Packet Length (Bwd Packet Length Min), Average Continuous Idle Duration (ldle Mean), and Standard Deviation of Continuous Active Duration (Active Std), average number of packets transmitted in the forward bulk (Fwd Packet / Bulk Avg), minimum continuous idle time (ldle Min), average continuous active time (Active Mean), maximum continuous idle time (ldle Max), RST flag count (RST Flag Count), standard deviation of continuous idle time (Idle Std), average number of bytes transmitted in the forward bulk (Fwd Bytes / Bulk Avg), minimum forward packet length (Fwd Packet Length Min), average forward bulk rate (Fwd BulkRate Avg), forward RST flags (Fwd RST Flags), number of forward active data packets (Fwd Act Data Pkts), minimum backward inter-frame arrival time (Bwd IAT Min), backward PSH flags (Bwd PSH Flags), minimum forward segment size (Fwd Seg Size Min), backward window initial value (Bwd Init Win Bytes), Total TCP Flow.Time, PSH Flag Count, ACK Flag Count, Forward Inter-Frame Arrival Time Mean (Fwd IAT Mean), FIN Flag Count, Backward Header Length (Bwd Header Length), Flow IAT Min, Forward Inter-Frame Arrival Time Min (Fwd IAT Min), Total Forward Packet Count, Average Packet Arrival Time Interval (Flow IAT Mean), Backward Packet Length Max (Bwd Packet Length Max), SYN Flag Count, Maximum Packet Length (Packet Length Max), Backward Inter-Frame Arrival Time Mean (Bwd IAT Mean), Down / Up Ratio, Total Backward Packet Count (Bwd Packets), Forward Header Length (Fwd Header Length), Flow Duration (milliseconds), Total Forward Packet Arrival Time Interval (Fwd IAT Total), Total Backward Packet Arrival Time Interval (Bwd IAT) Total, Average Forward Packet Length Mean, Average Forward Segment Size Avg, Forward PSH Flags, Standard Deviation of Packet Arrival Time (Flow IAT Std), Standard Deviation of Forward Packet Arrival Time (Fwd IAT Std), Subflow Bwd Bytes, Initial Forward Window (FWDInit Win Bytes), Standard Deviation of Backward Packet Length (Bwd Packet Length Std), Average Packet Size, Average Backward Packet Length Mean, Packet Length Variance, Standard Deviation of Backward Packet Arrival Time (Bwd IAT Std), Maximum Packet Arrival Time (Flow IAT Max), Average Backward Segment Size Avg, Maximum Backward Packet Arrival Time (Bwd IAT Max), Maximum Forward Packet Arrival Time (Fwd IAT Max), Standard Deviation of Packet Length (Packet Length)The table lists the following parameters: Std, Packet LengthMean, Forward Flow Rate (Fwd Packets / s), Total Length of Fwd Packet, Standard Deviation of Forward Packet Length (Fwd Packet Length Std), Subflow Fwd Bytes, Total Length of Bwd Packet, Flow Packets / s, Flow Bytes / s, Bwd Packets / s, and Maximum Forward Packet Length (FwdPacket Length Max).
[0032] According to an embodiment of the present invention, the preset evaluation method is as follows: Gradient boosting decision tree, random forest, and mutual information are used to evaluate the importance of each flow feature in the initial feature set, and the mean importance of each flow feature is calculated to obtain the importance evaluation result of each flow feature. Specifically, gradient boosting decision tree is used to calculate the importance of each flow feature in the initial feature set, and the calculated importance of each flow feature is scaled to the interval [0,1]; random forest is used to calculate the importance of each flow feature in the initial feature set, and the calculated importance of each flow feature is scaled to the interval [0,1]; mutual information is used to calculate the importance of each flow feature in the initial feature set, and the calculated importance of each flow feature is scaled to the interval [0,1]; the mean importance of each flow feature is calculated, and the flow features are sorted in descending order of importance. To better understand the importance of different flow features, Figure 2 The results of the traffic characteristic ranking are used to illustrate this, where... Figure 2 The various traffic features are sorted in order of importance from low to high, from... Figure 2 It can be seen that the importance of the six traffic features—Backward URG Flags, Backward RST Flags, CWR Flag Count, ECE Flag Count, Subflow Fwd Packets, and Subflow Bwd Packets—is 0, and these six traffic features can be directly removed.
[0033] According to one embodiment of the present invention, the preset number is 57. It should be noted that, based on the ranking of importance from high to low, the top 57 traffic features can be selected to be retained, or more or fewer traffic features can be retained according to actual needs, such as retaining the top 60 features or the top 50 features. The present invention does not impose any special limitations.
[0034] According to an embodiment of the present invention, the target features include: standard deviation of continuous idle time, average number of bytes transmitted in forward bulk data packets, minimum length of forward data packets, average rate of forward bulk data packets, forward RST flag, number of forward active data packets, minimum time to arrive between backward frames, backward PSH flag, minimum size of forward segment, initial value of backward window, total TCP stream time, PSH flag count, ACK flag count, average time to arrive between forward frames, FIN flag count, backward header length, minimum time to arrive between streams, minimum time to arrive between forward frames, number of forward data packets, average data packet arrival time interval, total length of backward data packets, SYN flag count, maximum data packet length, average time to arrive between backward frames, ratio of backward to forward data packets, number of backward data packets, forward header length, stream duration in milliseconds, and forward number of packets. The table lists the data types and their corresponding time intervals: total arrival time intervals of packets, total arrival time intervals of backward packets, average length of forward packets, average size of forward segments, forward PSH flag, standard deviation of arrival time intervals, standard deviation of arrival time intervals of forward packets, number of bytes in backward substreams, initial value of forward window, standard deviation of length of backward packets, average length of packets, average length of backward packets, change in length of packets, standard deviation of arrival time intervals of backward packets, maximum arrival time interval of packets, average size of backward segments, maximum arrival time interval of backward packets, maximum arrival time interval of forward packets, standard deviation of length of packets, average length of packets, forward flow rate, total length of forward packets, standard deviation of length of forward packets, number of bytes in forward substreams, total length of backward packets, flow rate of packets, byte flow rate, flow rate of backward packets, and maximum length of forward packets.
[0035] II. Data Preprocessing Stage
[0036] The data preprocessing stage includes obtaining a network traffic sequence, wherein the network traffic sequence includes multiple time-series continuous training network traffic and the traffic category corresponding to each training network traffic; extracting target features of each training network traffic to obtain a target feature set of each training network traffic; and constructing a training set using the target feature set of each training network traffic as a sample and the traffic category corresponding to each training network traffic as a label.
[0037] It should be noted that the data preprocessing stage is mainly used to extract features from the network traffic used for training in order to build a training set for training the detection model.
[0038] III. Model Training Phase
[0039] The model training phase takes samples from the training set as input and the predicted traffic category of the samples as output, and performs multiple rounds of iterative training on the lateral movement traffic detection model according to the preset objective function until the model converges.
[0040] According to one embodiment of the present invention, the lateral movement traffic detection model includes a Transformer model and a judgment module, wherein: the Transformer model is used to generate a target prediction sequence corresponding to a network traffic sequence composed of multiple time-series continuous network traffic sequences according to a recursive prediction method, wherein the target prediction sequence includes multiple target prediction network traffic sequences, and each target prediction network traffic sequence corresponds to one network traffic sequence; the judgment module is used to calculate the error value between each target prediction network traffic sequence and its corresponding actual network traffic sequence, and when the error value is greater than or equal to a threshold, the network traffic sequence is judged to be lateral movement traffic. It should be noted that the lateral movement traffic detection model can also use other neural network models to replace the Transformer model, such as an RNN model; the present invention does not impose specific limitations on the lateral movement traffic detection model. It should also be noted that the training network traffic used in the training of the lateral movement traffic detection model is all benign network traffic.
[0041] According to one embodiment of the present invention, the preset objective function is a cross-entropy loss function.
[0042] The lateral movement traffic detection model trained in the foregoing embodiments can be used to detect lateral movement traffic in containerized clusters. Based on this, the present invention also proposes a network traffic detection system, the system comprising: a feature extraction module for extracting target features of network traffic to obtain a target feature set of network traffic; and a lateral movement traffic detection model trained by the method described in the foregoing embodiments for processing the target feature set of network traffic to detect the traffic category of network traffic.
[0043] Based on the foregoing embodiments, the present invention also proposes a network traffic detection method, the method comprising: step T1, acquiring network traffic to be detected; step T2, using the system described in the foregoing embodiments to detect the network traffic to be detected.
[0044] To verify the effectiveness of the importance-based traffic feature selection in the feature analysis stage of the training method proposed in this invention, a GRU model was used for comparative experiments. In the comparative experiments, three different methods were used to process the experimental training set (including multiple experimental network traffic sets and the traffic category label for each experimental network traffic set): first, directly retaining 79 traffic features for each experimental network traffic set; second, compressing the 79 traffic features for each experimental network traffic set into 57 traffic features using Principal Component Analysis (PCA); and third, selecting the top 57 traffic features from the 79 traffic features for each experimental network traffic set based on their importance ranking from high to low. The GRU model was then trained using these three processed experimental training sets, and the comparative results are shown in Table 1.
[0045] Table 1 shows that when all 79 traffic features are retained, the TPR (the proportion correctly classified as positive) is 0.7239; the FPR (the proportion misclassified as positive) is 0.2814; the F1 score (a comprehensive indicator considering precision and recall) is 0.0934; and the AUC (the area under the ROC curve) is 0.8080. When PCA is used to compress the 79 traffic features into 57 traffic features, the TPR (the proportion correctly classified as positive) is 0.7178, slightly lower than the full feature count; the FPR (the proportion misclassified as positive) is 0.2795, similar to the full feature count; the F1 score (a comprehensive indicator considering precision and recall) is 0.0932, similar to the full feature count; and the AUC (the area under the ROC curve) is 0.8038, slightly lower than the full feature count. When the 79 traffic features were reduced to 57 based on importance ranking, the following results were observed: TPR (Target Retention Rate): The proportion correctly classified as positive was 0.7423, an improvement compared to the full feature set; FPR (False Positive Retention Rate): The proportion incorrectly classified as positive was 0.2543, a decrease compared to the full feature set; F1 score (a comprehensive indicator considering precision and recall) was 0.1046, also an improvement compared to the full feature set; AUC (Area Under the ROC Curve) was 0.8113, a slight improvement compared to the full feature set. Table 1 shows that removing low-importance traffic features improved the performance of the GRU model, achieving an AUC of 0.8113. However, using PCA for dimensionality reduction resulted in a decrease in GRU model performance, and PCA also lost the interpretability of the dimensionality reduction method.
[0046] Table 1
[0047]
[0048] To further verify the effectiveness of the importance-based traffic feature selection adopted in the feature analysis stage of the training method proposed in this invention, the Transformer model was trained by gradually increasing the number of traffic features. The experimental results are as follows: Figure 3 As shown, Figure 3 This demonstrates how the training time and AUC score of the Transformer model change when the number of traffic features increases from 2 to 79. Figure 3 In the graph, the red curve represents time, the blue curve represents the AUC score, the left ordinate represents the AUC score, the horizontal axis represents the number of traffic features, and the right ordinate represents the training time. Figure 3 It can be seen that the Transformer model achieves optimal performance with an AUC score of 0.9640 when retaining the 57 top-ranked traffic features, while simultaneously reducing training time by 27%. In contrast, when all 79 features are retained, the AUC score is 0.9615, slightly lower than the performance when retaining the 57 top-ranked traffic features.
[0049] The beneficial effects of the present invention are: (1) extracting data packet-level traffic features and session-level traffic features, solving the problem of unclear lateral detection features in containerized clusters; (2) eliminating traffic features with lower importance based on the importance ranking results of traffic features, achieving more accurate lateral movement detection.
[0050] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.
[0051] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.
[0052] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can include, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.
[0053] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A training method for a model for detecting lateral moving traffic in containerized clusters, for training a model that can detect lateral moving traffic in containerized clusters, wherein, Lateral movement traffic refers to traffic data generated after a containerized cluster is attacked. The method is characterized by comprising: Feature analysis phase: Retrieve historical network traffic generated in the containerized cluster, where the historical network traffic includes multiple data packets; Based on all data packets, perform data packet-level feature extraction on historical network traffic to obtain multiple data packet-level traffic features corresponding to historical network traffic; and based on the data packet connection protocol, aggregate all data packets in historical network traffic into multiple sessions to extract multiple session-level traffic features corresponding to historical network traffic; wherein, all data packet-level traffic features and session-level traffic features constitute the initial feature set of historical network traffic. The importance of each traffic feature in the initial feature set is evaluated using a preset evaluation method to obtain the importance evaluation result of each traffic feature. The preset evaluation method is as follows: gradient boosting decision tree, random forest and mutual information are used to evaluate the importance of each traffic feature in the initial feature set respectively, and the mean of the importance of each traffic feature is calculated to obtain the importance evaluation result of each traffic feature. The importance assessment results are sorted from highest to lowest, and a preset number of traffic features at the top of the list are selected as target features. Data preprocessing stage: Obtain the network traffic sequence, which includes multiple time-series consecutive training network traffic and the traffic category corresponding to each training network traffic; Extract the target features of each training network traffic to obtain the target feature set of each training network traffic, and construct a training set with the target feature set of each training network traffic as the sample and the traffic category corresponding to each training network traffic as the label; Model training phase: Using samples from the training set as input and the predicted traffic category of the samples as output, the lateral movement traffic detection model is iteratively trained multiple times according to a preset objective function until the model converges. The lateral movement traffic detection model includes a Transformer model and a judgment module. The Transformer model generates a target prediction sequence corresponding to a network traffic sequence composed of multiple time-series continuous network traffic sequences using a recursive prediction method. The target prediction sequence includes multiple target predicted network traffic sequences, and each target predicted network traffic sequence corresponds to one network traffic sequence. The judgment module calculates the error value between each target predicted network traffic sequence and its corresponding actual network traffic. When the error value is greater than or equal to a threshold, the network traffic is judged to be lateral movement traffic.
2. The method according to claim 1, characterized in that, The preset number is 57.
3. The method according to claim 2, characterized in that, The preset objective function is the cross-entropy loss function.
4. The method according to claim 1, characterized in that, The target characteristics include: standard deviation of continuous idle time, average number of bytes transmitted in forward batch packets, minimum forward packet length, average forward batch packet rate, forward RST flag, number of forward active packets, minimum backward inter-frame arrival time, backward PSH flag, minimum forward segment size, initial backward window value, total TCP stream time, PSH flag count, ACK flag count, average forward inter-frame arrival time, FIN flag count, backward header length, minimum inter-stream arrival time, minimum forward inter-frame arrival time, number of forward packets, average packet arrival time interval, total backward packet length, SYN flag count, maximum packet length, average backward inter-frame arrival time, ratio of backward to forward packets, number of backward packets, forward header length, stream duration in milliseconds, and forward packet arrival time. Total intervals, total backward packet arrival time intervals, average forward packet length, average forward segment size, forward PSH flag, standard deviation of packet arrival time intervals, standard deviation of forward packet arrival time intervals, number of bytes in backward sub-stream, initial value of forward window, standard deviation of backward packet length, average packet length, average backward packet length, packet length variation, standard deviation of backward packet arrival time intervals, maximum packet arrival time interval, average backward segment size, maximum backward packet arrival time interval, maximum forward packet arrival time interval, standard deviation of packet length, average packet length, forward flow rate, total forward packet length, standard deviation of forward packet length, number of bytes in forward sub-stream, total backward packet length, packet flow rate, byte flow rate, backward packet flow rate, and maximum forward packet length.
5. A network traffic detection system, characterized in that, The system includes: The feature extraction module is used to extract target features from network traffic to obtain a set of target features for network traffic. The lateral movement traffic detection model trained by any one of the methods described in claims 1-4 is used to process the target feature set of network traffic to detect the traffic category of network traffic.
6. A method for detecting network traffic, characterized in that, The method includes: Step T1: Obtain the network traffic to be detected; Step T2: Use the system described in claim 5 to detect the network traffic to be detected.
7. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1-4, 6.
8. An electronic device, characterized in that, include: One or more processors, and memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1-4, 6 by executing the executable instructions.