Training method of transverse movement flow detection model for containerized cluster

Abstract

The invention provides a training method of a lateral movement traffic detection model for a containerized cluster, and the method comprises a feature analysis stage, a data preprocessing stage and a model training stage, in the feature analysis stage, data packet-level traffic features and session-level traffic features in network traffic are extracted, and the data packet-level traffic features and the session-level traffic features are extracted; the flow characteristics with low importance are screened out according to the importance sequence; in the data preprocessing stage, a target feature set of network traffic for training is extracted based on the target features obtained in the feature analysis stage to construct a training set; in the model training stage, a lateral movement flow detection model is trained based on the training set. According to the technical scheme of the invention, the problem that the transverse detection feature is not clear is solved by extracting the traffic features of the data packet level and the session level; and meanwhile, the flow characteristics with low importance are eliminated according to the importance so as to realize more accurate transverse movement detection.

Description

Technical Field

[0001] The present invention relates to the field of network security, specifically to the horizontal movement traffic detection technology in the field of network security, and more specifically, to a training method for a horizontal movement traffic detection model for a containerized cluster. Background Art

[0002] With the rapid progress of cloud computing technology, cloud-native container platforms have become an indispensable important driving force for enterprise digital transformation. However, the wide application of containerized clusters has also brought unprecedented security risks, and various attack events have emerged in an endless stream, making the security issue increasingly prominent. Containerized attacks often follow a fixed process: reconnaissance, foothold, horizontal movement, attack, and cleanup. Among them, the horizontal movement stage is particularly critical because at this time, the attacker has infiltrated the cluster and started to collect information and look for the next attack target.

[0003] To achieve horizontal movement detection, researchers have proposed a variety of horizontal movement detection technologies, which mainly focus on four dimensions: network traffic, endpoint behavior, user behavior, and threat intelligence. However, most of the existing technologies have limitations. They either rely too much on user behavior information, or require known threat intelligence as support, or are only applicable to specific system environments. Therefore, it is difficult to directly apply them to containerized clusters.

[0004] In addition, existing horizontal movement detection technologies usually identify and detect horizontal movement behaviors by analyzing the data extracted from datasets. Most of these datasets come from enterprise internal networks and record various events within a specific time period, such as Windows authentication events, process events, network traffic events, and DNS events, or more detailed event types, such as network traffic, authentication, DHCP protocol, DNS protocol, file operations, HTTP protocol, and SSL protocol. However, these datasets are collected based on host events within enterprise networks, which are significantly different from the network traffic-based data in containerized clusters. Therefore, they cannot be directly used for horizontal movement detection in containerized environments.

[0005] Although horizontal movement detection can be achieved in environments such as enterprise networks, the horizontal detection technology for containerized clusters is still relatively scarce. Currently, there are two deficiencies in horizontal movement detection under containerized clusters: on the one hand, different from the enterprise network environment, horizontal movement in containerized clusters usually does not contain user authentication information, resulting in unclear features available for horizontal movement detection; on the other hand, there are differences in the importance of different features, and it is necessary to screen and eliminate features with lower importance to achieve more accurate horizontal movement detection.

[0006] It should be noted that: This background technology is only used to introduce relevant information of the present invention to facilitate understanding of the technical solution of the present invention, but it does not necessarily mean that the relevant information is prior art. Without evidence showing that the relevant information has been made public before the filing date of the present invention, the relevant information should not be regarded as prior art. Summary of the Invention

[0007] Therefore, the object of the present invention is to overcome the defects of the above-mentioned prior art and provide a training method for a lateral movement traffic detection model for a containerized cluster, a network traffic detection system, and a network traffic detection method.

[0008] The object of the present invention is achieved by the following technical solutions.

[0009] According to a first aspect of the present invention, a training method for a lateral movement traffic detection model for a containerized cluster is used to train a model capable of detecting lateral movement traffic in a containerized cluster, wherein the lateral movement traffic is traffic data generated after the containerized cluster is attacked. The method includes: Feature analysis stage: Obtain historical network traffic generated in the containerized cluster, where the historical network traffic includes a plurality of data packets; perform packet-level feature extraction on the historical network traffic based on all the data packets to obtain a plurality of packet-level traffic features corresponding to the historical network traffic; and aggregate all the data packets in the historical network traffic into a plurality of sessions based on the packet connection protocol to extract a plurality of session-level traffic features corresponding to the historical network traffic; wherein, all the packet-level traffic features and session-level traffic features form an initial feature set of the historical network traffic; use a preset evaluation method to evaluate the importance of each traffic feature in the initial feature set to obtain the importance evaluation result of each traffic feature; sort the importance evaluation results in descending order and select a preset number of traffic features ranked at the front as target features; Data preprocessing stage: Obtain a network traffic sequence, where the network traffic sequence includes a plurality of temporally consecutive training network traffic and the traffic category corresponding to each training network traffic; extract the target features of each training network traffic to obtain the target feature set of each training network traffic, and construct a training set with the target feature set of each training network traffic as a sample and the traffic category corresponding to each training network traffic as a label; Model training stage: Use the samples in the training set as input and the predicted traffic category of the samples as output, and perform multiple rounds of iterative training on the lateral movement traffic detection model according to a preset objective function until the model converges.

[0010] In some embodiments of the present invention, the preset evaluation method is as follows: the importance of each traffic feature in the initial feature set is evaluated using gradient boosting decision trees, random forests, and mutual information, respectively, and the mean value of the importance of each traffic feature is calculated to obtain the importance evaluation result of each traffic feature.

[0011] In some embodiments of the present invention, the preset number is 57.

[0012] In some embodiments of the present invention, the lateral movement traffic detection model includes a Transformer model and a judgment module, where: the Transformer model is used to generate a corresponding target prediction sequence for the network traffic sequence composed of multiple temporally continuous network traffics in a recursive prediction manner, where the target prediction sequence includes multiple target prediction network traffics, and each target prediction network traffic corresponds to one network traffic in the network traffic sequence; the judgment module is used to calculate the error value between each target prediction network traffic and its corresponding actual network traffic, and when the error value is greater than or equal to the threshold, it is determined that the network traffic is lateral movement traffic.

[0013] In some embodiments of the present invention, the preset objective function is the cross-entropy loss function.

[0014] In some embodiments of the present invention, the target features include: standard deviation of consecutive idle durations, average number of bytes transmitted in forward bulk data packets, minimum forward data packet length, average forward bulk data packet rate, forward RST flag, number of forward active data packets, minimum backward inter-frame arrival time, backward PSH flag, minimum forward segment size, initial backward window value, total TCP flow time, PSH flag count, ACK flag count, average forward inter-frame arrival time, FIN flag count, backward header length, minimum inter-flow arrival time, minimum forward inter-frame arrival time, number of forward data packets, average data packet arrival time interval, total backward data packet length, SYN flag count, maximum data packet length, average backward inter-frame arrival time, backward-to-forward data packet ratio, number of backward data packets, forward header length, flow duration in milliseconds, sum of forward data packet arrival time intervals, sum of backward data packet arrival time intervals, average forward data packet length, average forward segment size, forward PSH flag, standard deviation of data packet arrival time intervals, standard deviation of forward data packet arrival time intervals, backward sub-flow bytes, initial forward window value, standard deviation of backward data packet length, average data packet length, average backward data packet length, data packet length variation, standard deviation of backward data packet arrival time intervals, maximum data packet arrival time interval, average backward segment size, maximum backward data packet arrival time interval, maximum forward data packet arrival time interval, standard deviation of data packet length, average data packet length, forward flow rate, total forward data packet length, standard deviation of forward data packet length, forward sub-flow bytes, total backward data packet length, data packet flow rate, byte flow rate, backward data packet flow rate, and maximum forward data packet length.

[0015] According to a second aspect of the present invention, a network traffic detection system includes: a feature extraction module configured to extract target features of network traffic to obtain a set of target features of network traffic; and a lateral movement traffic detection model trained by the method according to the first aspect of the present invention, configured to process the set of target features of network traffic to detect the traffic category of the network traffic.

[0016] According to a third aspect of the present invention, the method includes: Step T1, obtaining network traffic to be detected; Step T2, detecting the network traffic to be detected by using the system according to the second aspect of the present invention.

[0017] Compared with the prior art, the advantages of the present invention are as follows: (1) By extracting packet-level traffic features and session-level traffic features, the problem of unclear lateral detection features in containerized clusters is solved; (2) Based on the sorting result of the importance of traffic features, traffic features with lower importance are eliminated, achieving more accurate lateral movement detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The embodiments of the present invention will be further described below with reference to the accompanying drawings, where:

[0019] Figure 1 FIG. is a schematic flowchart of a training method for a lateral movement traffic detection model according to an embodiment of the present invention;

[0020] Figure 2 FIG. is a schematic diagram of the importance ranking of traffic characteristics of network traffic according to an embodiment of the present invention;

[0021] Figure 3 FIG. is a line graph of a comparative experiment according to an embodiment of the present invention. Specific Embodiments

[0022] In order to make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below through specific embodiments with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0023] As mentioned in the background art section, there are two deficiencies in lateral movement detection under a containerized cluster: on the one hand, different from the enterprise network environment, lateral movement in a containerized cluster usually does not contain user authentication information, resulting in unclear features available for lateral movement detection; on the other hand, there are differences in the importance of different features, and it is necessary to screen and eliminate less important features to achieve more accurate lateral movement detection.

[0024] To solve the above problems, the inventor proposes a training method to train a detection model applicable to a containerized cluster. This detection model can detect the lateral movement traffic generated when the containerized cluster is attacked, thereby realizing the lateral movement detection of the containerized cluster. In this training method, there are three stages, namely, the feature analysis stage, the data preprocessing stage, and the model training stage. Among them, the feature analysis stage extracts packet-level traffic features and session-level traffic features of network traffic to solve the problem of unclear features, and sorts the packet-level traffic features and session-level traffic features according to importance to eliminate unimportant traffic features, thereby achieving more accurate lateral movement detection; the data preprocessing stage extracts target features from the training network traffic to obtain a target feature set of the training network traffic; the model training stage trains the detection model based on the target feature set of the training network traffic until the model converges.

[0025] Generally speaking, as Figure 1As shown in the figure, the present invention provides a training method for a lateral movement traffic detection model for a containerized cluster, which is used to train a model capable of detecting lateral movement traffic in a containerized cluster. Among them, the lateral movement traffic is traffic data generated after the containerized cluster is attacked. The method includes: Feature analysis stage: Obtain the historical network traffic generated in the containerized cluster, where the historical network traffic includes multiple data packets; perform packet-level feature extraction on the historical network traffic based on all data packets to obtain multiple packet-level traffic features corresponding to the historical network traffic; and aggregate all data packets in the historical network traffic into multiple sessions based on the packet connection protocol to extract multiple session-level traffic features corresponding to the historical network traffic; among them, all packet-level traffic features and session-level traffic features form the initial feature set of the historical network traffic; use a preset evaluation method to evaluate the importance of each traffic feature in the initial feature set to obtain the importance evaluation result of each traffic feature; sort the importance evaluation results in descending order, and select the preset number of traffic features ranked at the top as target features; Data preprocessing stage: Obtain a network traffic sequence, where the network traffic sequence includes multiple temporally consecutive training network traffics and the traffic category corresponding to each training network traffic; extract the target features of each training network traffic to obtain the target feature set of each training network traffic, and construct a training set with the target feature set of each training network traffic as samples and the traffic category corresponding to each training network traffic as labels; Model training stage: Use the samples in the training set as inputs and the predicted traffic categories of the samples as outputs, and perform multiple rounds of iterative training on the lateral movement traffic detection model according to a preset objective function until the model converges.

[0026] To better understand the present invention, the following will separately elaborate on each stage in detail in combination with specific embodiments.

[0027] I. Feature analysis stage

[0028] The feature analysis stage includes: Obtain the historical network traffic generated in the containerized cluster, where the historical network traffic includes multiple data packets; perform packet-level feature extraction on the historical network traffic based on all data packets to obtain multiple packet-level traffic features corresponding to the historical network traffic; and aggregate all data packets in the historical network traffic into multiple sessions based on the packet connection protocol to extract multiple session-level traffic features corresponding to the historical network traffic; among them, all packet-level traffic features and session-level traffic features form the initial feature set of the historical network traffic; use a preset evaluation method to evaluate the importance of each traffic feature in the initial feature set to obtain the importance evaluation result of each traffic feature; sort the importance evaluation results in descending order, and select the preset number of traffic features ranked at the top as target features.

[0029] Among them, the packet-level traffic characteristics are the characteristics extracted at the traffic packet level. For example, the time difference between packets in a session, the size of packets in a session, the TCP flag value statistics in a session, etc. To better understand the feature extraction at the packet level, the extraction of packet protocol field information is taken as an example for illustration. Among them, the protocol field information can be determined by reading specific bytes in the packet header. For example, in an Ethernet frame, one byte starting from the 9th byte of the IP packet header is used to indicate the protocol type. When the value of this byte is 6, it represents the TCP protocol, and when it is 17, it represents the UDP protocol.

[0030] The session-level traffic characteristics are the characteristics extracted at the session level after aggregating all packets in the historical network traffic into multiple sessions based on the packet connection protocol. For example, session duration, the total number of bytes transmitted in a session, the total number of packets transmitted in a session, etc. To better understand the feature extraction at the session level, the extraction of the byte traffic of a session is taken as an example for illustration. Among them, the byte traffic of a session is determined by calculating the total number of bytes sent and received during the session. Specifically, for each HTTP request and response packet in the session, read the packet length field (usually in the packet header) and add up the lengths of all packets. It should be noted that when aggregating packets into sessions, the packet connection protocol needs to be followed. For example, for a TCP connection, the establishment of a session starts with one party sending the first handshake packet to the other party and ends with one party sending the last wave packet or reset connection packet or timeout to the other party; for packets of connectionless protocols such as UDP, a certain aggregation method can be adopted according to the specific application layer protocol. For example, for the DNS protocol, a query request and its corresponding result return can be regarded as a session. It should also be noted that there is a special packet set bulk in the process of aggregating packets into sessions (when a TCP stream transmits a large amount of data and needs to be divided into multiple packets for transmission, the set of these packets is called bulk). When extracting features from this special packet set bulk, it is necessary to first determine whether the packets can be aggregated into bulk, and then count the number of packets. When the number of packets reaches 4, the features of bulk can be extracted. Among them, the features related to bulk are usually 0 and are greater than 0 only when the amount of transmitted data is large, indicating that packet splitting transmission has occurred in this stream.

[0031] Through packet-level feature extraction and session-level feature extraction, seven categories of features can be extracted, including flow duration, number of packets, packet length, flow rate, packet interval, TCP flag statistics, bulk transfer of packet headers, and TCP window activity and idleness, for a total of 79 traffic features. Among them, the traffic features include: Backward URG Flags, Backward RST Flags, CWR Flag Count, ECE Flag Count, Subflow Fwd Packets, Subflow Bwd Packets, Average of Backward Bulk Transfer Bytes, Average of Backward Bulk Transfer Packets, Forward URG Flags, Average of Backward Bulk Transfer Rate, URG Flag Count, Maximum of Consecutive Active Duration, Minimum of Consecutive Active Duration, Minimum of Packet Length, Minimum of Backward Packet Length, Mean of Consecutive Idle Duration, Standard Deviation of Consecutive Active Duration, Average of Forward Bulk Transfer Packets, Minimum of Consecutive Idle Duration, Mean of Consecutive Active Duration, Maximum of Consecutive Idle Duration, RST Flag Count, Standard Deviation of Consecutive Idle Duration, Average of Forward Bulk Transfer Bytes, Minimum of Forward Packet Length, Average of Forward Bulk Transfer Rate, Forward RST Flags, Number of Forward Active Data Packets, Minimum of Backward Inter-Arrival Time, Backward PSH Flags, Minimum of Forward Segment Size, Backward Initial Window Bytes, Total TCP Flow TimeTime), PSH Flag Count, ACK Flag Count, Fwd IAT Mean, FIN Flag Count, BwdHeader Length, Flow IAT Min, FwdIAT Min, Total Fwd Packet, Flow IATMean, Bwd Packet Length Max, SYN Flag Count, Packet Length Max, Bwd IAT Mean, Down / Up Ratio, Total Bwd packets, Fwd Header Length, Flow Duration, Fwd IAT Total, Bwd IAT Total, Fwd Packet Length Mean, Fwd Segment Size Avg, Fwd PSH Flags, Flow IAT Std, Fwd IAT Std, Subflow Bwd Bytes, FWDInit Win Bytes, Bwd Packet Length Std, Average Packet Size, Bwd Packet Length Mean, Packet Length Variance, Bwd IAT Std, Flow IAT Max, Bwd Segment Size Avg, Bwd IAT Max, FwdIAT Max, Packet Length(Std), average packet length (Packet Length Mean), forward flow rate (Fwd Packets / s), total length of forward packets (Total Length of Fwd Packet), standard deviation of forward packet length (Fwd Packet Length Std), number of bytes in forward sub-flow (Subflow Fwd Bytes), total length of backward packets (Total Length of Bwd Packet), packet flow rate (Flow Packets / s), byte flow rate (Flow Bytes / s), backward packet flow rate (Bwd Packets / s), and maximum forward packet length (FwdPacket Length Max).

[0032] According to an embodiment of the present invention, the preset evaluation method is as follows: using gradient boosting decision trees, random forests, and mutual information to evaluate the importance of each traffic feature in the initial feature set respectively, and calculating the mean of the importance of each traffic feature to obtain the importance evaluation result of each traffic feature. Specifically, use gradient boosting decision trees to calculate the importance of each traffic feature in the initial feature set, and scale the importance of each calculated traffic feature to the interval [0,1]; use random forests to calculate the importance of each traffic feature in the initial feature set, and scale the importance of each calculated traffic feature to the interval [0,1]; use mutual information to calculate the importance of each traffic feature in the initial feature set, and scale the importance of each calculated traffic feature to the interval [0,1]; calculate the mean of the importance of each traffic feature, and sort the traffic features in descending order of importance. To better understand the importance of different traffic features, Figure 2 the sorted results of the traffic feature display are used for illustration, where Figure 2 each traffic feature in is sorted in ascending order of importance, and from Figure 2 it can be seen that the importance of the six traffic features of backward URG flag (Bwd URGFlags), backward RST flag (Bwd RST Flags), CWR flag count (CWR Flag Count), ECE flag count (ECE Flag Count), forward sub-flow packet count (Subflow Fwd Packets), and backward sub-flow packet count (Subflow Bwd Packets) is 0, and these six traffic features can be directly removed.

[0033] According to an embodiment of the present invention, the preset number is 57. It should be noted that according to the sorting result of importance from high to low, 57 traffic features ranked at the top can be selected for retention, or more or fewer traffic features can be retained according to actual needs. For example, 60 features ranked at the top can be retained, or 50 features ranked at the top can be retained. The present invention does not impose special restrictions.

[0034] According to an embodiment of the present invention, the target features include: standard deviation of consecutive idle duration, average value of forward bulk data packet transmission byte count, minimum value of forward data packet length, average value of forward bulk data packet rate, forward RST flag, number of forward active data packets, minimum value of backward inter-frame arrival time, backward PSH flag, minimum value of forward segment size, initial value of backward window, total time of TCP flow, PSH flag count, ACK flag count, average value of forward inter-frame arrival time, FIN flag count, backward header length, minimum value of inter-flow arrival time, minimum value of forward inter-frame arrival time, number of forward data packets, average value of data packet arrival time interval, total length of backward data packets, SYN flag count, maximum value of data packet length, average value of backward inter-frame arrival time, ratio of backward to forward data packets, number of backward data packets, forward header length, duration of flow in milliseconds, sum of forward data packet arrival time intervals, sum of backward data packet arrival time intervals, average value of forward data packet length, average value of forward segment size, forward PSH flag, standard deviation of data packet arrival time interval, standard deviation of forward data packet arrival time interval, backward sub-flow byte count, initial value of forward window, standard deviation of backward data packet length, average value of data packet length, average value of backward data packet length, change in data packet length, standard deviation of backward data packet arrival time interval, maximum value of data packet arrival time interval, average value of backward segment size, maximum value of backward data packet arrival time interval, maximum value of forward data packet arrival time interval, standard deviation of data packet length, average value of data packet length, forward flow rate, total length of forward data packets, standard deviation of forward data packet length, forward sub-flow byte count, total length of backward data packets, data packet flow rate, byte flow rate, backward data packet flow rate, and maximum value of forward data packet length.

[0035] II. Data preprocessing stage

[0036] The data preprocessing stage includes obtaining a network traffic sequence, where the network traffic sequence includes a plurality of temporally consecutive training network traffic and the traffic category corresponding to each training network traffic; extracting the target features of each training network traffic to obtain the target feature set of each training network traffic, and constructing a training set with the target feature set of each training network traffic as a sample and the traffic category corresponding to each training network traffic as a label.

[0037] It should be noted that the data preprocessing stage is mainly used to extract features from the training network traffic to construct a training set for training the detection model.

[0038] III. Model Training Stage

[0039] In the model training stage, the samples in the training set are used as inputs, and the predicted traffic categories of the samples are used as outputs. The lateral movement traffic detection model is iteratively trained multiple times according to a preset objective function until the model converges.

[0040] According to an embodiment of the present invention, the lateral movement traffic detection model includes a Transformer model and a judgment module, wherein: the Transformer model is used to generate a target prediction sequence corresponding to the network traffic sequence based on a network traffic sequence composed of multiple temporally consecutive network traffics according to a recursive prediction method. The target prediction sequence includes multiple target prediction network traffics, and each target prediction network traffic corresponds to a network traffic in the network traffic sequence; the judgment module is used to calculate the error value between each target prediction network traffic and its corresponding actual network traffic, and when the error value is greater than or equal to the threshold, it is determined that the network traffic is lateral movement traffic. It should be noted that other neural network models can also be set in the lateral movement traffic detection model to replace the Transformer model, such as the RNN model. The present invention does not specifically limit the lateral movement traffic detection model. It should also be noted that when the training network traffic used in the training of the lateral movement traffic detection model is all benign network traffic.

[0041] According to an embodiment of the present invention, the preset objective function is the cross-entropy loss function.

[0042] The lateral movement traffic detection model trained in the foregoing embodiment can be used to detect the lateral movement traffic in the containerized cluster. Based on this, the present invention also proposes a network traffic detection system, which includes: a feature extraction module for extracting the target features of the network traffic to obtain a set of target features of the network traffic; a lateral movement traffic detection model trained by the method as described in the foregoing embodiment for processing the set of target features of the network traffic to detect the traffic category of the network traffic.

[0043] Based on the foregoing embodiment, the present invention also proposes a network traffic detection method, which includes: Step T1, obtaining the network traffic to be detected; Step T2, using the system as described in the foregoing embodiment to detect the network traffic to be detected.

[0044] To verify the effectiveness of the importance-based traffic feature screening adopted in the feature analysis stage of the training method proposed in the present invention, a GRU model is used for comparative experiments. In the comparative experiments, three different methods are adopted to process the experimental training set (including multiple experimental network traffic and the traffic category labels of each experimental network traffic): one is to directly retain 79 traffic features of each experimental network traffic; the second is to compress 79 traffic features of each experimental network traffic into 57 traffic features through Principal Component Analysis (PCA) technology; the third is to select the top 57 traffic features from the 79 traffic features of each experimental network traffic based on the sorting result of traffic feature importance from high to low; then the GRU model is trained respectively using these three processed experimental training sets, and the comparative results shown in Table 1 are obtained.

[0045] As can be seen from Table 1, when all 79 traffic features are retained, TPR: the proportion of correctly classified positive classes is 0.7239; FPR: the proportion of misclassified positive classes is 0.2814; F1 score: the index considering both precision and recall is 0.0934; AUC: the area value under the ROC curve is 0.8080. When 79 traffic features are compressed into 57 traffic features using PCA technology, TPR: the proportion of correctly classified positive classes is 0.7178, slightly lower than that of the full set of features; FPR: the proportion of misclassified positive classes is 0.2795, similar to that of the full set of features; F1 score: the index considering both precision and recall is 0.0932, similar to that of the full set of features; AUC: the area value under the ROC curve is 0.8038, slightly lower than that of the full set of features. When 79 traffic features are screened into 57 traffic features based on the importance sorting result, TPR: the proportion of correctly classified positive classes is 0.7423, higher than that of the full set of features; FPR: the proportion of misclassified positive classes is 0.2543, lower than that of the full set of features; F1 score: the index considering both precision and recall is 0.1046, also higher than that of the full set of features; AUC: the area value under the ROC curve is 0.8113, slightly higher than that of the full set of features. Summarizing Table 1, after removing the traffic features with low importance, the performance of the GRU model is improved, and the AUC reaches 0.8113; while using PCA for dimensionality reduction, the performance of the GRU model decreases instead, and PCA loses the interpretability of the dimensionality reduction method.

[0046] Table 1

[0047] To further verify the effectiveness of the importance-based traffic feature screening adopted in the feature analysis stage of the training method proposed in the present invention, the Transformer model was trained by gradually increasing the traffic features. The experimental results are as Figure 3 shown, Figure 3 which shows the changes in the training time and AUC score of the Transformer model when the number of traffic features is increased from 2 to 79. Among them, in Figure 3 , the red curve represents the time, the blue curve represents the AUC score, the left vertical axis represents the AUC score, the horizontal axis is the number of traffic features, and the right vertical axis represents the training time. As Figure 3 can be seen, when 57 traffic features ranked in the front are retained, the Transformer model achieves the best performance, with an AUC score as high as 0.9640. At the same time, the training time is shortened by 27%. In contrast, when all 79 features are retained, the AUC score is 0.9615, slightly lower than the performance when the top 57 traffic features are retained.

[0048] The beneficial effects of the present invention are as follows: (1) Extracting packet-level traffic features and session-level traffic features solves the problem of unclear horizontal detection features in containerized clusters; (2) Based on the importance ranking results of traffic features, traffic features with lower importance are eliminated, realizing more accurate lateral movement detection.

[0049] It should be noted that although the above steps are described in a specific order, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even the order can be changed as long as the required functions can be achieved.

[0050] The present invention can be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement various aspects of the present invention.

[0051] A computer-readable storage medium can be a tangible device that retains and stores instructions for use by an instruction execution device. A computer-readable storage medium may include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punched card or raised structures in grooves having instructions stored thereon, and any suitable combination of the foregoing.

[0052] The embodiments of the present invention have been described above. The above description is exemplary and not exhaustive, and is also not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the embodiments disclosed herein.

Claims

1. A method for training a lateral movement traffic detection model for a containerized cluster, for training a model capable of detecting lateral movement traffic in a containerized cluster, wherein: The lateral movement traffic is the traffic data generated after the containerized cluster is attacked, and the method includes: Feature analysis phase: Obtain historical network traffic generated in the containerized cluster, where the historical network traffic includes multiple data packets; Extracting packet-level features of historical network traffic based on all data packets to obtain multiple packet-level traffic features corresponding to the historical network traffic; and aggregating all data packets in the historical network traffic into multiple sessions based on the packet connection protocol to extract multiple session-level traffic features corresponding to the historical network traffic; wherein all packet-level traffic features and session-level traffic features constitute an initial feature set of the historical network traffic; The importance of each traffic feature in the initial feature set is evaluated using a preset evaluation method to obtain an importance evaluation result of each traffic feature; The importance evaluation results are sorted in descending order, and a preset number of traffic features ranked first are selected as target features; Data preprocessing stage: Acquire a network traffic sequence, wherein the network traffic sequence includes a plurality of time-series continuous training network traffic and a traffic category corresponding to each training network traffic; Extracting target features of each training network flow to obtain a target feature set of each training network flow, and constructing a training set with the target feature set of each training network flow as a sample and the traffic category corresponding to each training network flow as a label; Model training phase: With the samples in the training set as input and the predicted traffic category of the samples as output, the lateral mobile traffic detection model is iteratively trained for multiple rounds according to the preset objective function until the model converges.

2. The method according to claim 1, characterized in that The preset evaluation method is: Gradient boosting decision tree, random forest and mutual information are used to evaluate the importance of each traffic feature in the initial feature set, and the mean of the importance of each traffic feature is calculated to obtain the importance evaluation result of each traffic feature.

3. The method according to claim 2, characterized in that The preset number is 57.

4. The method according to claim 3, characterized in that The lateral movement traffic detection model includes a Transformer model and a judgment module, wherein: The Transformer model is used to generate a target prediction sequence corresponding to the network flow sequence based on a network flow sequence composed of multiple time-series continuous network flows in a recursive prediction manner, wherein the target prediction sequence includes multiple target predicted network flows, and each target predicted network flow corresponds to a network flow in the network flow sequence; The judgment module is used to calculate the error value between each target predicted network flow and its corresponding actual network flow, and when the error value is greater than or equal to a threshold, the network flow is judged to be lateral movement flow.

5. The method according to claim 4, characterized in that The preset objective function is a cross entropy loss function.

6. The method according to claim 2, characterized in that The target features include: standard deviation of continuous idle time, average value of forward bulk data packet transmission bytes, minimum value of forward data packet length, average value of forward bulk data packet rate, forward RST flag, number of forward active data packets, minimum value of backward inter-frame arrival time, backward PSH flag, minimum value of forward message segment size, initial value of backward window, total TCP flow time, PSH flag count, ACK flag count, average value of forward inter-frame arrival time, FIN flag count, backward header length, minimum value of inter-flow arrival time, minimum value of forward inter-frame arrival time, number of forward data packets, average value of data packet arrival time interval, total length of backward data packets, SYN flag count, maximum value of data packet length, average value of backward inter-frame arrival time, ratio of backward to forward data packets, number of backward data packets, forward header length, flow duration in milliseconds, forward data packet arrival time Sum of intervals, Sum of backward packet arrival intervals, Mean forward packet length, Mean forward segment size, Forward PSH flag, Standard deviation of packet arrival intervals, Standard deviation of forward packet arrival intervals, Number of bytes in backward subflow, Forward window initial value, Standard deviation of backward packet lengths, Mean packet lengths, Mean backward packet lengths, Packet length variation, Standard deviation of backward packet arrival intervals, Maximum packet arrival intervals, Mean backward segment size, Maximum backward packet arrival intervals, Maximum forward packet arrival intervals, Standard deviation of packet lengths, Mean packet lengths, Forward flow rate, Total forward packet length, Standard deviation of forward packet lengths, Number of bytes in forward subflow, Total backward packet length, Packet flow rate, Byte flow rate, Backward packet flow rate, and Maximum forward packet length.

7. A network traffic detection system, characterized in that: The system comprises: A feature extraction module, used for extracting target features of network traffic to obtain a target feature set of network traffic; The lateral movement traffic detection model trained by the method as described in any one of claims 1-6 is used to process the target feature set of network traffic to detect the traffic category of the network traffic.

8. A network traffic detection method, characterized in that: The method comprises: Step T1, obtaining the network traffic to be detected; Step T2: using the system as claimed in claim 7 to detect the network traffic to be detected.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of any method described in claims 1-6 and 8.

10. An electronic device, characterized in that: include: one or more processors, and A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1-6 and 8 by executing the executable instructions.

Images