A Malicious Traffic Reshaping Method Based on Feature Serialization

Abstract

This invention relates to the fields of network security and network simulation technology, and discloses a method for reshaping malicious traffic based on feature serialization. By collecting traffic data from the target network environment, serialized features of load and time intervals are extracted, and session flow features are further statistically obtained. Then, based on Pareto, normal, log-normal, and Poisson probability distribution models, serialized features conforming to the target network traffic characteristics are generated, and the probability distribution model with the highest similarity to the target network traffic is selected by calculating Euclidean distance. Finally, based on this model, the load and time interval sequences of malicious traffic are reshaped to generate network attack traffic that is difficult to detect. This invention effectively improves the concealment of malicious traffic, making it more difficult for existing detection systems to identify; the generated malicious traffic can effectively test and evaluate the capabilities of network intrusion detection systems.

Description

Technical Field

[0001] This invention relates to the fields of network security and network simulation technology, specifically a method for reshaping malicious traffic based on feature serialization. Background Technology

[0002] With the rapid development of the internet and the widespread application of information technology, the types and methods of cyberattacks are constantly evolving and upgrading. Traditional Network Intrusion Detection Systems (NIDS), as a core component of network security defense, primarily rely on real-time monitoring and analysis of network traffic to identify potential security threats. However, as attack techniques become increasingly complex, traditional signature-based and rule-matching NIDS are gradually proving inadequate. While these systems perform well against known threats, their detection capabilities are severely challenged when dealing with emerging threats such as Advanced Persistent Threat (APT), zero-day attacks, and hybrid attacks.

[0003] Existing NIDS primarily rely on two detection mechanisms: feature-based detection and behavior-based detection. Feature-based detection methods identify malicious traffic by matching predefined attack signatures, but this approach has significant limitations, especially in the face of constantly evolving attack methods. Attackers can bypass these signatures using simple obfuscation and transformation techniques, rendering NIDS ineffective. Behavior-based detection identifies potential threats by monitoring abnormal network traffic behavior. While it has some generalization capabilities, it is not effective at identifying highly covert attack behaviors.

[0004] In the context of network attack and defense, malicious traffic generation technology has become a research hotspot. The purpose of malicious traffic generation is to simulate or forge normal traffic characteristics to generate offensive malicious traffic, in order to test and evaluate the robustness of NIDS (Network Intelligence Detection and Distributed System). However, existing malicious traffic generation technologies face several technical challenges. First, the generated malicious traffic often lacks sufficient stealth, making it easily detectable by NIDS and negating the purpose of simulating real attacks. Second, the characteristics of the generated traffic are difficult to match with the real traffic characteristics of the target network, making it difficult for the generated traffic to disguise as legitimate traffic in the network environment, thus increasing the risk of detection. Furthermore, existing malicious traffic generation methods typically rely on predefined traffic distribution models, lacking flexibility and adaptability. This fixed-pattern-dependent generation method performs poorly in complex and ever-changing network environments, struggling to cope with diverse traffic characteristics under different network conditions. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to provide a malicious traffic reshaping method based on feature serialization. By optimizing the network traffic generator through sampling and testing, malicious network traffic with stealth and offensive capabilities can be generated, thereby enhancing the ability to resist Network Intrusion Detection Systems (NIDS). The technical solution is as follows:

[0006] A malicious traffic reshaping method based on feature serialization includes the following steps:

[0007] Step 1: Collect traffic data during peak and off-peak hours from the target network environment, and capture data packets at key nodes using a network sniffer; preprocess and extract features from the collected traffic data to obtain the serialization features and session flow features of the target network; the serialization features include load serialization features and time serialization features;

[0008] Step 2: Generate several serialization features and session stream features that satisfy the given stream statistical characteristics based on several probability distribution models;

[0009] Step 3: Measure the similarity between the serialized features and session flow features generated under different probability distribution models and the serialized features and session flow features of the target network traffic by Euclidean distance, and select the probability distribution model with the minimum Euclidean distance as the probability distribution model when generating the target network traffic;

[0010] Step 4: Calculate the session flow characteristics of the malicious traffic that needs to be reshaped, and then generate serialization characteristics according to the selected probability distribution model; based on the generated load serialization characteristics, extract data sequentially from the application layer load of the merged original malicious traffic to construct new data packets; at the same time, based on the generated time serialization characteristics, assign corresponding timestamps according to the order of the data packets; finally, generate the corresponding network traffic through Scapy.

[0011] Furthermore, the extraction of serialization features in step 1 specifically involves:

[0012] Step 1.1: Extract Load Serialization Features: Statistically analyze the size of each data packet in the traffic data to form a load feature sequence. Let X = {x1, x2, x3, ..., x...} m} represents the set of packet payload sizes, where x i This represents the size of the i-th data packet, where i = 1, 2, ..., m;

[0013] Step 1.2: Time Series Characteristics: Record the transmission time of each data packet in the traffic data and calculate the time difference between adjacent data packets to form a time interval sequence; let T = {t1, t2, t3, ..., t...} n} represents the set of time intervals for data packets, where t jThis represents the time interval between the j-th data packet and the (j-1)-th data packet, where j = 1, 2, ..., n.

[0014] Furthermore, in step 2, several probability distribution models include: Pareto distribution, normal distribution, log-normal distribution, and Poisson distribution, and the generation process is as follows:

[0015] 1) The process of generating data conforming to the Pareto distribution is as follows: First, estimate the number of data packets based on the total data volume and the average data packet size; then, use the Pareto distribution to generate the initial data packet size; next, define a function to adjust these data packet sizes to meet specific statistical requirements, including total size, average size, and variance; and optimize to determine the optimal data packet size; finally, ensure that the adjusted data packet size is within the set minimum and maximum range, and return the adjusted data packet sequence and its new characteristics.

[0016] 2) The log-normal distribution is equivalent to the normal distribution on a logarithmic scale. The process of generating a log-normal distribution is as follows: First, determine the length of the sequence based on the specified sum and target mean; then initialize a basic array using the target mean and adjust each element by multiplying it by an exponential adjustment factor generated from the normal distribution, so that the array conforms to the log-normal distribution; prune the adjusted values ​​to ensure that the generated sequence meets the minimum and maximum value constraints; then calculate an adjustment factor to ensure that the sum of the array is consistent with the target sum, and prune again to meet the minimum and maximum value constraints; finally, return the adjusted log-normal distribution sequence that conforms to the specified statistical properties.

[0017] 3) The process of generating a Poisson distribution is as follows: First, an initial time interval sequence is generated based on the average occurrence rate of the Poisson distribution; then, the initial time interval sequence is adjusted to meet the requirements of sum, mean, variance, maximum and minimum values.

[0018] Furthermore, step 3 specifically includes:

[0019] Calculate the Euclidean distance between the serialization features and session flow features generated by different probability distribution models and the serialization features and session flow features of the target network traffic:

[0020] D A =α*D se +β*D sa

[0021] In the formula, s A For the total Euclidean distance, s se D is the Euclidean distance between the generated serialized features and the serialized features of the target network traffic. saLet α and β be the Euclidean distance between the generated session flow features and the session flow features of the target network traffic, respectively. se and D sa The weighting coefficient is used to adjust its contribution ratio.

[0022] Furthermore, step 4 specifically includes:

[0023] The process of reshaping threat traffic using generated load serialization features is as follows: First, the application layer loads in the original threat traffic session stream are concatenated together. Then, the application layer loads are re-divided according to the generated load serialization features, and finally inserted into the data packets. During this process, the TCP checksum, IP header checksum, and Ethernet frame checksum are modified accordingly.

[0024] The beneficial effects of this invention are:

[0025] 1) To address the problem of accurately determining the probability distribution model of the target network, this invention proposes a probability distribution model selection method based on sampling comparison. By generating network traffic with various common distributions and comparing the similarity with the sampled network traffic, the probability distribution model closest to the target network can be effectively determined. This method draws on the idea of ​​selecting the best option, providing a scientific basis for generating traffic with the same characteristics as the target network traffic, thereby improving the similarity and concealment between the generated traffic and the actual target network traffic, and enhancing the concealment and difficulty in detecting malicious traffic.

[0026] 2) This invention also proposes a malicious traffic generation method based on feature serialization. By reshaping the load and time interval sequence of malicious traffic, this method ensures that the generated malicious traffic has the same probability distribution model characteristics as the target network, and is difficult to distinguish from normal traffic in terms of time and load characteristics, thereby improving the concealment of malicious traffic. Attached Figure Description

[0027] Figure 1 This is a flowchart of the malicious traffic reshaping method based on feature serialization in this invention.

[0028] Figure 2 This is a schematic diagram of the probability distribution model selection method based on sampling comparison in this invention. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0030] This invention proposes a malicious traffic reshaping method based on feature serialization. It collects traffic data from the target network environment, extracts serialized features of load and time intervals, and further statistically obtains session flow features. Next, based on Pareto, normal, log-normal, and Poisson probability distribution models, serialized features conforming to the target network traffic characteristics are generated. The probability distribution model with the highest similarity to the target network traffic is selected by calculating Euclidean distance. Finally, based on this model, the load and time interval sequences of malicious traffic are reshaped to generate network attack traffic that is difficult to detect. The flowchart is as follows. Figure 1 As shown, the specific steps are as follows:

[0031] Step 1: Collect traffic data during peak and off-peak hours from the target network environment, and capture data packets at key nodes using a network sniffer; preprocess and extract features from the collected traffic data to obtain the serialization features and session flow features of the target network; the serialization features include load serialization features and time serialization features.

[0032] Traffic data can be obtained from the target network environment by using network sniffing tools such as Wireshark or tcpdump to capture packets at key nodes. To ensure the representativeness and comprehensiveness of the sample, the collected data should cover peak and off-peak periods.

[0033] Collected network traffic data typically contains a large amount of raw information, including many unnecessary details that can interfere with analysis and simulation. Therefore, it is necessary to preprocess the data to extract useful information and remove noise. Data preprocessing includes: removing corrupted, incomplete, or irrelevant data packets; reassembling the filtered data packets by flow, using network communication session flows as the basic unit of processing; and removing session flows containing no more than six data packets.

[0034] After preprocessing and feature extraction of the collected data, the serialization features and session stream features of the target network are obtained.

[0035] Serialization characteristics refer to the set of packet payload sizes and the set of time intervals between packets. The former is called payload serialization characteristics, and the latter is called time serialization characteristics. These two are collectively referred to as serialization characteristics.

[0036] Session flow characteristics: These refer to the sum, average, variance, maximum, and minimum number of bytes of payload for all packets in a session flow. They also include the sum, average, variance, maximum, and minimum session durations.

[0037] Load feature extraction involves analyzing the size of each data packet in the statistical traffic, thus forming a load feature sequence. Let X = {x1, x2, x3, ..., x...} m} represents the set of packet payload sizes, where x i This represents the size of the i-th data packet.

[0038] Extracting time interval features involves recording the transmission time of each data packet in the traffic and calculating the time difference between adjacent data packets, thereby generating a time interval sequence. Let T = {t1, t2, t3, ..., t...} n} represents the set of time intervals for data packets, where t j This represents the time interval between the j-th data packet and the (j-1)-th data packet.

[0039] Step 2: Generate various serialization features that conform to the given flow statistics based on a variety of common probability distribution models (such as Pareto distribution, normal distribution, log-normal distribution, and Poisson distribution).

[0040] The Pareto distribution is known for its "long tail" characteristic. The process of generating a Pareto distribution is as follows: First, estimate the number of data packets based on the total data volume and the average packet size; then, generate initial packet sizes using the Pareto distribution; next, define a function to adjust these packet sizes to meet specific statistical requirements, such as total size, mean, and variance; then, optimize to determine the optimal packet size; finally, ensure the adjusted packet size is within a set minimum and maximum range, and return the adjusted packet sequence and its characteristics.

[0041] The normal distribution, also known as the Gaussian distribution, is a common probability distribution model characterized by a symmetrical bell-shaped curve. The log-normal distribution is equivalent to the normal distribution on a logarithmic scale. The steps to generate a log-normal distribution are as follows: First, determine the sequence length based on a specified sum and a target mean. Next, initialize a base array using the target mean and adjust each element by multiplying it by an exponential adjustment factor generated from the normal distribution, thus making the array conform to a log-normal distribution. To ensure the generated sequence meets the minimum and maximum value requirements, the adjusted values ​​are pruned. Subsequently, an adjustment factor is calculated to ensure the sum of the array matches the target sum, and pruning is performed again to meet the minimum and maximum value constraints. Finally, the algorithm returns a log-normal distribution sequence that has been adjusted and conforms to the specified statistical properties.

[0042] The Poisson distribution is used to describe the frequency of a random event occurring within a specific time period or spatial range. When using the Poisson distribution to generate time series data, it can simulate the random occurrence of events over continuous time periods. The steps to generate a Poisson distribution are as follows: First, generate an initial time interval sequence based on the average occurrence rate of the Poisson distribution. Then, adjust the sequence to meet the requirements for sum, mean, variance, maximum, and minimum values.

[0043] Step 3: Calculate the Euclidean distance to evaluate the similarity between the serialization features and session flow features generated by different probability distribution models and the serialization features and session flow features of the target network traffic. Finally, select the probability distribution model with the smallest Euclidean distance as the probability distribution model for generating the target network traffic.

[0044] Specifically, calculate the Euclidean distance between the features generated by different probability distribution models and the features of the target network traffic:

[0045] D A =α*D se +β*D sa

[0046] In the formula, D A For the total Euclidean distance, D se D is the Euclidean distance between the generated serialized features and the serialized features of the target network traffic. sa Let α and β be the Euclidean distance between the generated session flow features and the session flow features of the target network traffic, respectively. se and D sa The weighting coefficient is used to adjust its contribution ratio.

[0047] A schematic diagram of the probability distribution model selection method based on sampling comparison is shown below. Figure 2 As shown.

[0048] Step 4: First, calculate the session flow characteristics of the malicious traffic to be reshaped, and then generate corresponding serialization features based on the selected probability distribution model. Based on the generated load serialization features, extract data one by one from the application layer load of the merged original malicious traffic to construct new data packets. Simultaneously, assign corresponding timestamps to these data packets using the generated time serialization features. Finally, generate the corresponding network traffic using Scapy.

[0049] In reshaping malicious traffic, the application layer payload of the original threat traffic is first spliced ​​together. Then, these application layer payloads are re-divided according to the generated payload serialization characteristics and inserted into new data packets. During this process, necessary adjustments need to be made to the TCP checksum, IP header checksum, and Ethernet frame checksum.

Claims

1. A method for reshaping malicious traffic based on feature serialization, characterized in that, Includes the following steps: Step 1: Collect traffic data during peak and off-peak hours from the target network environment, and capture data packets at key nodes using a network sniffer; The collected traffic data is preprocessed and features are extracted to obtain the serialization features and session flow features of the target network; the serialization features include load serialization features and time serialization features. Step 2: Generate several serialization features and session stream features that satisfy the given stream statistical characteristics based on several probability distribution models; Step 3: Measure the similarity between the serialized features and session flow features generated under different probability distribution models and the serialized features and session flow features of the target network traffic by Euclidean distance, and select the probability distribution model with the minimum Euclidean distance as the probability distribution model when generating the target network traffic; Step 4: Calculate the session flow characteristics of the malicious traffic that needs to be reshaped, and then generate serialized features based on the selected probability distribution model; Based on the generated load serialization characteristics, data is sequentially extracted from the application layer load of the merged original malicious traffic to construct new data packets; at the same time, based on the generated time serialization characteristics, corresponding timestamps are assigned according to the order of the data packets; finally, the corresponding network traffic is generated through Scapy. The extraction of serialization features in step 1 specifically involves: Step 1.1: Extract load serialization features: Statistically analyze the size of each data packet in the traffic data and form a load feature sequence. Let... Let be the set of packet payload sizes, where This represents the size of the i-th data packet, where i = 1, 2, ..., m; Step 1.2: Time-series data processing: Record the transmission time of each data packet in the traffic data and calculate the time difference between adjacent data packets to form a time interval sequence; assuming... Let be the set of time intervals for data packets, where This represents the time interval between the j-th data packet and the (j-1)-th data packet, where j = 1, 2, ..., n; In step 2, several probability distribution models are used, including Pareto distribution, normal distribution, log-normal distribution, and Poisson distribution. The generation process is as follows: 1) The process of generating data conforming to the Pareto distribution is as follows: First, estimate the number of data packets based on the total data volume and the average data packet size; then, use the Pareto distribution to generate the initial data packet size; next, define a function to adjust these data packet sizes to meet specific statistical requirements, including total size, average size, and variance; and optimize to determine the optimal data packet size; finally, ensure that the adjusted data packet size is within the set minimum and maximum range, and return the adjusted data packet sequence and its new characteristics. 2) The log-normal distribution is equivalent to the normal distribution on a logarithmic scale. The process of generating a log-normal distribution is as follows: First, determine the length of the sequence based on the specified sum and target mean; then, initialize a basic array using the target mean and adjust each element by multiplying it by an exponential adjustment factor generated from the normal distribution, so that the array conforms to the log-normal distribution; prune the adjusted values ​​to ensure that the generated sequence meets the minimum and maximum value constraints; then calculate an adjustment factor to ensure that the sum of the array is consistent with the target sum, and prune again to meet the minimum and maximum value constraints; finally, return the adjusted log-normal distribution sequence that conforms to the specified statistical properties. 3) The process of generating a Poisson distribution is as follows: First, an initial time interval sequence is generated based on the average occurrence rate of the Poisson distribution; then, the initial time interval sequence is adjusted to meet the requirements of sum, mean, variance, maximum and minimum values. Step 3 specifically involves: Calculate the Euclidean distance between the serialization features and session flow features generated by different probability distribution models and the serialization features and session flow features of the target network traffic: ； In the formula, D A For the total Euclidean distance, D se D is the Euclidean distance between the generated serialized features and the serialized features of the target network traffic. sa Let α and β be the Euclidean distance between the generated session flow features and the session flow features of the target network traffic, respectively. se and D sa The weighting coefficient is used to adjust its contribution ratio; Step 4 specifically involves: The process of reshaping threat traffic using generated load serialization features is as follows: First, the application layer loads in the original threat traffic session stream are concatenated together. Then, the application layer loads are re-divided according to the generated load serialization features, and finally inserted into the data packets. During this process, the TCP checksum, IP header checksum, and Ethernet frame checksum are modified accordingly.

Images