A data packet filtering method, device, equipment and medium

Abstract

The application relates to the technical field of network security, and in particular to a data packet filtering method and device, equipment and a medium. In the embodiment of the application, the time mode of data packet arrival is analyzed to determine the frequency and persistence of the target identifier of the data packet, without complex flow aggregation or feature extraction. According to the activation function obtained by training based on the historical frequency count, the historical persistence count and the historical classification information of whether the historical data packet is a data packet of a DDoS attack, and the preset threshold, data packets belonging to DDoS attack traffic are screened out, the intensity of the DDoS attack is greatly reduced, and normal business traffic is protected from the influence of the attack.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, and particularly relates to a data packet filtering method and device, equipment and a medium. BACKGROUND

[0002] Under the background of rapid development of the Internet, a distributed denial of service (DDoS) attack has become a common network security threat. It floods a target website or service with a large number of requests, so that it cannot process the requests of legitimate users, thereby causing service interruption.

[0003] Therefore, it is necessary to detect DDoS attacks to improve the DDoS attack detection accuracy and real-time performance, thereby improving the ability to defend against DDoS attacks, helping network administrators to discover abnormal traffic patterns in time, and taking defensive measures quickly to minimize the negative impact of attacks. SUMMARY

[0004] The present application provides a data packet filtering method, device, equipment and medium to solve the problem that the existing DDoS attack causes a target website or service to be flooded with a large number of requests, and cannot process the requests of legitimate users, thereby causing service interruption.

[0005] In a first aspect, the embodiments of the present application provide a data packet filtering method, which comprises:

[0006] The source address, destination address, source port, destination port and protocol information carried in the target data packet to be processed are determined as the target identifier of the target data packet;

[0007] The frequency count of the data packet of the target identifier appearing before the current time of the monitoring statistics and the persistence count of the data packet of the target identifier appearing in a set time window are obtained;

[0008] According to the frequency count, the persistence count, an activation function and a preset threshold, it is determined whether the target data packet is a first classification information of a data packet of a DDoS attack; wherein the activation function is obtained based on a historical frequency count of a historical data packet, a historical persistence count of the historical data packet and historical first classification information of whether the historical data packet is a data packet of a DDoS attack.

[0009] In a second aspect, the embodiments of the present application also provide a data packet filtering device, which comprises:

[0010] The processing module is configured to determine source address, destination address, source port, destination port and protocol information carried in the target data packet to be processed as a target identifier of the target data packet; obtain a frequency count of data packets of the target identifier before a current time of monitoring and statistics, and a persistence count of data packets of the target identifier within a set time window;

[0011] The classification module is configured to determine, according to the frequency count, the persistence count, an activation function and a preset threshold, whether the target data packet is a first classification information of a data packet of DDoS attack; wherein the activation function is obtained based on a historical frequency count, a historical persistence count of historical data packets and historical first classification information of whether the historical data packets are data packets of DDoS attack.

[0012] In a third aspect, an electronic device is provided, and the electronic device includes at least a processor and a memory. The processor is configured to implement the steps of the data packet filtering method according to any of the above embodiments when executing a computer program stored in the memory.

[0013] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is configured to implement the steps of the data packet filtering method according to any of the above embodiments when executed by a processor.

[0014] In the embodiments of the present application, the electronic device determines source address, destination address, source port, destination port and protocol information carried in the target data packet to be processed as a target identifier of the target data packet; obtains a frequency count of data packets of the target identifier before a current time of monitoring and statistics, and a persistence count of data packets of the target identifier within a set time window; determines, according to the frequency count, the persistence count, an activation function and a preset threshold, whether the target data packet is a first classification information of a data packet of DDoS attack; wherein the activation function is obtained based on a historical frequency count, a historical persistence count of historical data packets and historical first classification information of whether the historical data packets are data packets of DDoS attack. In the embodiments of the present application, the time mode of data packet arrival is analyzed to determine the frequency and persistence of the target identifier of the data packet, without complex flow aggregation or feature extraction. The data packets belonging to DDoS attack traffic are screened according to the activation function obtained based on the historical frequency count, the historical persistence count of the historical data packets and the historical classification information of whether the historical data packets are data packets of DDoS attack, and the preset threshold, which greatly reduces the intensity of DDoS attack and protects normal business traffic from being affected by the attack. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art without creative labor based on the embodiments in the present application shall fall within the scope of protection of the present application.

[0016] Figure 1 A data packet filtering process schematic diagram provided for the embodiments of the present application;

[0017] Figure 2 A DDoS attack detection flowchart combining online detection and offline detection provided for the embodiments of the present application;

[0018] Figure 3 A filtering rule generation and classification model training flowchart provided for the embodiments of the present application;

[0019] Figure 4 A data packet filtering device structure schematic diagram provided for the embodiments of the present application;

[0020] Figure 5 An electronic device structure schematic diagram provided for the embodiments of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor shall fall within the scope of protection of the present application.

[0022] The existing detection method for DDoS attacks is to collect and analyze multi-dimensional attributes in network traffic using machine learning and data mining techniques to ensure efficient and accurate real-time detection of large flow DDoS attacks on the Internet. For example, machine learning models such as decision trees, random forests, and XGBoost are used, and the performance is optimized by tuning, and finally the detection accuracy is improved through model fusion. However, as DDoS attack methods continue to evolve, traditional detection methods based on static feature rules (such as traffic size, packet rate, protocol type, etc.) have been unable to meet the requirements. Modern DDoS attack detection requires more intelligent and efficient techniques to cope with changing attack patterns. The development of machine learning and data mining techniques provides new possibilities for real-time traffic detection. By analyzing the behavioral characteristics of network traffic, abnormal patterns that differ from normal traffic can be automatically identified, enabling real-time detection of DDoS attacks. The current latest attack detection algorithm based on machine learning can achieve almost 99.99% precision at the flow level, and its classification performance is quite good, but most of them can only be detected after the attack flow is completed. Online models are mostly single rules, although the time performance is sufficient to detect and handle during the attack flow, but the classification performance is limited.

[0023] In order to accurately identify DDoS attacks and improve the security of the network, the embodiments of the present application provide a data packet filtering method, device, equipment and medium.

[0024] In the embodiments of the present application, the electronic device determines the source address, destination address, source port, destination port and protocol information carried in the target data packet to be processed as the target identifier of the target data packet; obtains the frequency count of the data packet of the target identifier before the current time of the monitoring statistics, and the persistence count of the data packet of the target identifier within the set time window; determines the first classification information of whether the target data packet is a data packet of DDoS attack according to the frequency count, the persistence count, the activation function and the preset threshold; wherein the activation function is obtained by training based on the historical frequency count of the historical data packet, the historical persistence count and the historical first classification information of whether the historical data packet is a data packet of DDoS attack.

[0025] Embodiment 1:

[0026] Figure 1 A data packet filtering process diagram is provided for the embodiments of the present application, the process includes:

[0027] S101: Determine the source address, destination address, source port, destination port and protocol information carried in the target data packet to be processed as the target identifier of the target data packet; obtain the frequency count of the data packet of the target identifier before the current time of monitoring and statistics, and the persistence count of the data packet of the target identifier within a set time window.

[0028] The data packet filtering method provided by the embodiment of the application is applied to an electronic device, which can be a PC or a server.

[0029] It is found through investigation that the data flow statistical summary method based on Sketch is a hot topic in recent years. The Sketch method uses limited memory and can accelerate the measurement of data flow. By using Sketch to evaluate statistical indicators in data flow, rough classification can be performed. As a kind of Sketch method, LTC (Long-Tail Clock) only analyzes the time mode of data packet arrival in network flow, without performing time-consuming flow aggregation and statistical feature extraction, so as to complete the evaluation of the frequency and persistence of element arrival. Such processing has certain rationality for the detection of DDoS attacks, because DDoS attack flow is not only very frequent (high frequency) in a very short time, but also such high-frequency flow continuously appears in multiple time periods (high persistence).

[0030] Therefore, compared with the existing method using a counter method based on a traditional Bloom Sketch, Counting Bloom Filter and the like, only the frequency is considered, the embodiment of the application can simultaneously measure the frequency and persistence of elements, and can more flexibly define and screen more complex DDoS attack modes. In the embodiment of the application, the electronic device can design an improved algorithm based on the LTC capability, realize rough-grained online screening of DDoS attack flow at the data packet level based on the distribution mode of data packet arrival time, and greatly reduce the proportion of DDoS attack flow.

[0031] Specifically, for each data packet in the flow, the header five-tuple (source address, destination address, source port, destination port and protocol information) is extracted as the identifier of the data packet, denoted as f ID wherein the protocol information is L4 protocol.

[0032] For each identifier, the electronic device uses the LTC algorithm to monitor and count the frequency of the target identifier data packet and the persistence of the target identifier data packet within a set time window. Wherein the frequency count is the frequency count of the LTC algorithm by combining the long tail recovery technology and the improved CLOCK algorithm, and the persistence count is the persistence count according to a certain observation period.

[0033] When the electronic device identifies the target data packet to be processed, the electronic device determines the source address, destination address, source port, destination port and protocol information carried in the target data packet as the target identifier of the target data packet, and based on the lookup algorithm of the LCT algorithm, the frequency of the target identifier data packet before the current time is found, and the persistence of the target identifier data packet within a set time window is found.

[0034] Wherein, the LTC algorithm is an advanced pattern mining algorithm for stream data, which is specially designed to quickly and accurately identify significant items in data streams under memory constraints. For network traffic, the only feature considered and used by the LTC algorithm is the arrival time interval of the data packet, without complex stream aggregation or feature extraction and other statistical work, which has a significant significance for early detection in high-speed traffic processing. High throughput and memory efficiency make it an ideal choice for preliminary screening of large amounts of network traffic.

[0035] S102: According to the frequency count, the persistence count, the activation function and the preset threshold, determine whether the target data packet is the first classification information of the DDoS attack data packet; wherein the activation function is obtained based on the historical frequency count of the historical data packet, the historical persistence count and the historical first classification information of whether the historical data packet is the DDoS attack data packet.

[0036] The original LTC algorithm aims to extract the top N most important elements from the stream, where the importance is the weighted sum of the frequency count and the persistence count. According to the mode of DDoS attack, by setting the importance threshold, the DDoS attack traffic can be screened out after giving equal weight to frequency and persistence. However, in the processing of complex DDoS attack patterns, simple weighted sum method is not enough to accurately screen out attack traffic. For example, some attack traffic may show abnormally high level in persistence, even if its frequency is not outstanding. Similarly, some attacks may suddenly increase their frequency within a certain period of time, including but not limited to the burstiness, intermittency and ability to disguise as normal traffic.

[0037] Based on this, in order to improve the accuracy and robustness of detection, it is necessary to design more complex filtering rules FP-Rule based on frequency and persistence, to filter out most of the DDoS attack traffic that meets the feature rules, and to keep only benign traffic and a small part of DDoS attack traffic, so that the original traffic is filtered into lower unbalance degree traffic data. Among them, the electronic device can use a fitting algorithm based on logistic regression to derive the filtering rule FP-Rule. FP-Rule falls into a function f(f, p) with f and p as inputs (its return value is 0 or 1) to determine whether the target packet is a DDoS attack. If the return value is 1, it is determined that the target packet is a DDoS attack, and if the return value is 0, it is determined that the target packet is not a DDoS attack.

[0038] Specifically, in the embodiment of the present application, the electronic device trains an activation function based on the historical frequency count of the historical packet, the historical persistence count of the historical packet, and the historical first classification information of whether the historical packet is a DDoS attack packet. The electronic device uses the trained activation function and a preset threshold to construct a filtering rule, and filters the target packet based on the filtering rule.

[0039] Among them, in the embodiment of the present application, the filtering rule derived by the electronic device contains two parts, one is to determine the target probability of the target packet being a DDoS attack packet based on the frequency count, the persistence count of the target packet and the trained activation function, and the other part is to determine the first classification information of whether the target packet is a DDoS attack packet according to the target probability and the preset threshold.

[0040] Among them, the activation function can map a real number to the interval (0, 1), and can be used for binary classification. It is effective when the characteristics are quite different or not very different.

[0041] Specifically, after the electronic device obtains the frequency count corresponding to the target packet for representing the frequency and the persistence count for representing the persistence, the electronic device can determine the target probability of the target packet being a DDoS attack packet according to the frequency count and the persistence count, and the trained activation function.

[0042] The electronic device is also preconfigured with a threshold value, which is adjusted according to the detected Precision value to ensure a low false negative rate, and the electronic device can determine the first classification information of whether the target data packet is a DDoS attack data packet according to the threshold value and the target probability. If the target probability exceeds the threshold value, the determined first classification information is that the target data packet is a DDoS attack data packet; if the target probability does not exceed the threshold value, the determined first classification information is that the target data packet is not a DDoS attack data packet. In this way, the fitting process is relatively "loose", which can filter out most of the DDoS attack traffic while not mistakenly intercepting benign traffic

[0043] Specifically, the electronic device can determine the target probability of whether the target data packet is a DDoS attack data packet through the following formula:

[0044] P(label=1|x)=σ(w T x+b),x=[f,p]

[0045] Wherein, σ(z) represents an activation function, x is a feature vector composed of the frequency count f and the persistence count p, w and b are parameters obtained through training, wherein w is a weight vector and b is a bias term.

[0046] The electronic device can determine the first classification information of whether the target data packet is a DDoS attack data packet through the following formula:

[0047]

[0048] Wherein, σ(w T x+b) is the target probability, and θ is a preset threshold value.

[0049] On the basis of the above embodiment, the electronic device determines the first classification information of whether the target data packet is a DDoS attack data packet as 1 or 0. If the electronic device determines that the first classification information is 1, the electronic device determines that the target data packet is a DDoS attack; if the electronic device determines that the first classification information is 0, the electronic device determines that the target data packet is not a DDoS attack.

[0050] In addition, in the embodiments of the present application, the method for generating FP-Rule in the embodiments of the present application includes but is not limited to logistic regression, linear fitting, density estimation and all methods that can dynamically adjust the decision rule of the classification plane. The activation function can be a sigmoid function or other functions, which are not limited here.

[0051] In the embodiment of the present application, by analyzing the time mode of the data packet arrival, the target identifier corresponding frequency count and persistence count of the data packet are determined, without complex flow aggregation or feature extraction, and the data packets belonging to DDoS attack traffic are screened out according to the historical frequency count, historical persistence count and historical classification information of whether the historical data packet is a DDoS attack packet of the historical data packet, the activation function trained based on the historical data packet, and the preset threshold, which greatly reduces the DDoS attack intensity and protects the normal business traffic from the influence of the attack.

[0052] Embodiment 2

[0053] In order to accurately identify DDoS attacks and improve the security of the network, on the basis of the above-mentioned embodiments, in the embodiment of the present application, before obtaining the frequency count of the data packet of the target identifier appearing before the current time of the monitoring statistics, the method further comprises:

[0054] Obtaining a configured data packet whitelist, wherein the data packet whitelist saves the identifier of the data packet that has been determined to be not a DDoS attack;

[0055] If the target identifier is not included in the data packet whitelist, the subsequent process of obtaining the frequency count of the data packet of the target identifier appearing before the current time of the monitoring statistics is performed.

[0056] In order to improve the efficiency of DDoS attack detection, in the embodiment of the present application, the electronic device is configured with a data packet whitelist, which saves the identifier of the data packet that has been determined to be not a DDoS attack, and the data packet whitelist is configured according to the historical DDoS attack detection result.

[0057] The data packet whitelist can quickly identify and allow those data packets with historical and priori knowledge to pass. Based on this, before filtering the target data packet based on the filtering rule, the electronic device obtains the configured data packet whitelist, if the electronic device determines that the target identifier is not included in the data packet whitelist, the electronic device continues to filter the target data packet; if the electronic device determines that the target identifier is included in the data packet whitelist, the electronic device processes the target data packet.

[0058] In the embodiment of the present application, the implementation method of the data packet whitelist includes but is not limited to various hash tables, sequential tables, red-black trees, Bloom filters and all data structures and algorithms supporting fast member lookup.

[0059] Embodiment 3

[0060] In order to accurately identify the DDoS attack and improve the security of the network, on the basis of the above embodiments, in the embodiments of the present application, if it is determined that the target data packet is not a data packet of DDoS attack, the method further comprises:

[0061] obtaining feature data of a target flow corresponding to the target data packet, wherein the feature data comprises a flow duration, a length standard deviation of each data packet contained in the target flow, and a total number of data packets transmitted per second of the target flow;

[0062] inputting the feature data into a pre-trained classifier to obtain second classification information of whether the target flow is a DDoS attack output by the classifier;

[0063] if the second classification information is that the target flow is a DDoS attack, it is determined that the target data packet is a data packet of DDoS attack.

[0064] In order to further improve the accuracy of DDoS attack detection, in the embodiments of the present application, the electronic device can also use a machine learning classifier to perform flow level detection on the traffic after reducing the non-equilibrium rate of DDoS attack and normal traffic, and complete further accurate discrimination.

[0065] Specifically, the electronic device obtains feature data of a target flow corresponding to a target data packet, wherein the feature data comprises a flow duration, a length standard deviation of each data packet contained in the target flow, and a total number of data packets transmitted per second of the target flow. The electronic device constructs a feature vector corresponding to the target flow based on the feature data, and inputs the feature vector into a pre-trained classifier to obtain second classification information of whether the target flow is a DDoS attack output by the classifier.

[0066] If the electronic device determines that the second classification information output by the classifier is that the target flow is a DDoS attack, the electronic device determines that the target data packet is a data packet of DDoS attack.

[0067] It should be noted that, in the embodiments of the present application, the second classification information output by the classifier is 1 or 0, if the second classification information is 1, it is determined that the target flow is a DDoS attack, and if the second classification information is 0, it is determined that the target flow is not a DDoS attack.

[0068] On the basis of the above embodiments, the data packet filtering method provided in the embodiments of the present application is divided into online monitoring and offline detection. The online detection is implemented by data packet whitelist matching and filtering rule screening, and belongs to the data packet level; the offline detection is implemented by a classifier, and belongs to the data flow level. Through the hierarchical method combining online detection and offline detection, on the one hand, based on the LTC capability, an improved algorithm is designed, the DDoS attack traffic is coarsely screened at the data packet level based on the distribution mode of the data packet arrival time, and the proportion of the DDoS attack traffic is greatly reduced; on the other hand, the machine learning classifier is used at the second layer to detect the flow at the flow level after the non-balanced rate of the DDoS attack and normal service traffic is reduced, and further accurate discrimination is completed. Through the embodiments of the present application, the DDoS attack intensity of the equipment in the protection detection range can be weakened after the DDoS attack occurs, and the normal service traffic is protected from the influence of the attack.

[0069] In order to accurately identify the DDoS attack and improve the security of the network, on the basis of the above embodiments, in the embodiments of the present application, the classifier is a CatBoost classifier.

[0070] In the embodiments of the present application, the classifier used by the electronic device can be a CatBoost classifier.

[0071] It should be noted that the classifier used in the embodiments of the present application includes but is not limited to CatBoost, nearest neighbor, random forest and all machine learning-based methods, and the CatBoost is selected based on the principle that the experimental effect is optimal.

[0072] Figure 2 The DDoS attack detection process diagram combining online detection and offline detection provided in the embodiments of the present application is shown in FIG. 1. Figure 2 As shown in the figure, the process includes:

[0073] 1. Model loading and preparation. Run the program, load the generated filtering rule FP-Rule and CatBoost classifier, and start to listen to the original network traffic from the specified port / queue / api.

[0074] 2. Data packet whitelist (whitelist) matching. Through the whitelist, those data packets with historical and priori knowledge can be quickly identified and allowed to pass. If the target identifier f ID On the whitelist, it is directly returned as benign.

[0075] It should be emphasized that the implementation of the whitelist includes but is not limited to all kinds of hash tables, order tables, red-black trees, BloomFilter and all data structures and algorithms supporting fast member lookup.

[0076] 3. FP-Rule screening. If the f II If not found in the whitelist, calculate f ID The corresponding frequency count (frequency) and persistence count (persistence), and use FP-Rule screening. If the FP-Rule returns a judgment of DDoS attack, filter out the data packet, otherwise add the data packet to the classification buffer.

[0077] 4. CatBoost classification. When the data packets in the classification buffer meet the conditions (the conditions are freely defined according to requirements, including but not limited to limiting the number of packets, limiting the refresh time, limiting the total size, etc.), the buffer data packets are aggregated and flow features are extracted as items, and a CatBoost classifier is used for classification. If it is not a DDoS attack, it is recorded in the whitelist, otherwise it is output as a DDoS attack.

[0078] Embodiment 4:

[0079] In order to accurately identify DDoS attacks and improve network security, on the basis of the above embodiments, in the embodiments of the present application, the training process of the CatBoost classifier includes:

[0080] Obtaining a pre-configured sample flow, sample feature data corresponding to the sample data packet, and sample second classification information of whether the sample flow is a DDoS attack;

[0081] Inputting the sample feature data into the CatBoost classifier to be trained to obtain predicted second classification information output by the CatBoost classifier;

[0082] According to the sample second classification information, the predicted second classification information, and the F1-score algorithm, a loss value is determined, and the parameters of the CatBoost classifier are adjusted according to the loss value.

[0083] In the embodiments of the present application, the electronic device can train the CatBoost classifier based on historical traffic data to improve the classification accuracy of the CatBoost classifier.

[0084] Specifically, the electronic device obtains pre-configured sample flow, sample feature data corresponding to sample data packet, and sample second classification information whether the sample flow is a DDoS attack; the electronic device inputs the sample feature data into the CatBoost classifier to be trained, and obtains the predicted second classification information output by the CatBoost classifier; the electronic device determines the loss value according to the sample second classification information, the predicted second classification information and the F1-score algorithm, and adjusts the parameters of the CatBoost classifier according to the loss value.

[0085] The electronic device can determine the loss value based on the following formula:

[0086]

[0087] Wherein, TP represents the number of sample flows corresponding to the predicted second classification information and the sample second classification information both being 1; FP represents the number of sample flows corresponding to the predicted second classification information being 1 and the sample second classification information being 0; FN represents the number of sample flows corresponding to the predicted second classification information being 0 and the sample second classification information being 1.

[0088] Embodiment 5:

[0089] In order to accurately identify the DDoS attack and improve the security of the network, on the basis of the above embodiments, in the embodiment of the application, if it is determined that the target data packet is not a data packet of DDoS attack, the method further comprises:

[0090] Adding the target identifier to the data packet whitelist.

[0091] In the embodiment of the application, if the electronic device determines that the target data packet is not a data packet of DDoS attack, the electronic device adds the target identifier corresponding to the target data packet to the data packet whitelist, so that when other data packets of the target identifier are received subsequently, the other data packets are directly passed.

[0092] Embodiment 6:

[0093] In order to accurately identify the DDoS attack and improve the security of the network, on the basis of the above embodiments, in the embodiment of the application, the training process of the activation function comprises:

[0094] Obtaining pre-configured sample data packet, sample frequency count corresponding to the sample data packet, sample persistence count, and sample first classification information whether the sample data packet is a data packet of DDoS attack;

[0095] input the sample frequency count and the sample persistence count into the activation function to be trained, obtain a predicted probability based on an output of the activation function, and the preset threshold, obtain a prediction first classification information of whether the sample data packet is a DDoS attack packet based on the predicted probability based on the output of the activation function and the preset threshold;

[0096] According to the sample first classification information, the predicted first classification information, and the maximum log-likelihood function, a loss value is determined, and parameters of the activation function are adjusted according to the loss value.

[0097] In the embodiments of the present application, when the electronic device trains the activation function based on the historical frequency count, the historical persistence count of the historical data packet, and the historical first classification information of whether the historical data packet is a DDoS attack packet, the electronic device can first obtain a pre-configured sample data packet, a sample frequency count corresponding to the sample data packet, a sample persistence count, a threshold, and a sample first classification information of whether the sample data packet is a DDoS attack packet. The electronic device inputs the sample frequency count and the sample persistence count into the activation function to be trained, obtains a predicted probability of whether the sample data packet is a DDoS attack packet based on the activation function, and determines a prediction first classification information according to the predicted probability and the threshold. The electronic device determines a loss value according to the sample first classification information, the predicted first classification information, and the maximum log-likelihood function, and adjusts parameters of the activation function according to the loss value.

[0098] Wherein, when DDoS attack occurs, the traffic is mainly attack traffic, and only a small part is benign traffic. According to this feature, the electronic device uses a simulation platform or obtains network traffic pcap files and label files labels.file containing DDoS attacks and normal behaviors from a network public data set. The electronic device obtains each sample data packet from the network traffic pcap file and determines the sample first classification information of whether each data packet is a DDoS attack packet according to the label file labels.file.

[0099] The electronic device replays the sample data packets according to the arrival time of the sample data packets, and measures using LTC to obtain the frequency count representing the frequency and the persistence count representing the persistence corresponding to the target identifier of each sample data packet.

[0100] Due to the principle of LTC, at the initial stage, all the counters of LTC are 0, and there is no discrimination, and the output result has no practical significance and cannot be used to generate filtering rules. In order to solve this "cold start" problem, the embodiment of the application designs to start recording when a certain amount (the application sets it to 100000 according to experience) of data packets pass through the LTC, at this time the counters will not be all zero, and these are regarded as valid fitting samples. After the cold start waiting, when the marked traffic passes through the LTC, the frequency count f corresponding to each data packet indicating the frequency and the persistence count p indicating the persistence are determined, and the label of this data packet in the labels.file is merged and added to the fitting data set (FP-set) in the format of <f, p | label>.

[0101] The electronic device uses <f, p | label>, and derives the FP-Rule based on the fitting algorithm activation function of the logistic regression. The FP-Rule is landed as a function f(f, p) with f and p as inputs (its return value is 0 or 1) to determine whether the traffic is a DDoS attack. And based on maximizing the log-likelihood function to determine the loss value to adjust the parameters of the activation function.

[0102] Wherein, the electronic device can determine the loss value according to the following formula:

[0103]

[0104] Wherein, y i represents the sample first classification information of the sample data packet i, σ(w T x i +b) represents the predicted first classification information of the sample data packet i, w and b are parameters to be adjusted, x i is the sample feature vector composed of the sample frequency count f i and the sample persistence count p i of the sample data packet i, that is, x i =[f i ,p i ].

[0105] Figure 3 The filtering rule generation and classification model training process schematic diagram provided by the embodiment of the application is as shown in the figure. Figure 3 As shown in the figure, the process of generating filtering rules includes:

[0106] 1-1, replay the marked traffic. Use the marked DDoS attack traffic, replay according to the arrival time of the data packets in the data set, use LTC to measure, and get the frequency count and persistence count corresponding to each target identifier f ID .

[0107] 1-2, Cold start waiting. In the initial stage, all the counters of LTC are 0, which has no distinguishing degree, and the output result has no practical significance and cannot be used to generate FP-Rule. In order to solve this "cold start" problem, the application is designed to start recording when a certain amount (the application is set to 100000 according to experience) of data packets pass through LTC. At this time, the counters will not be all zeros, and these are regarded as valid fitting samples.

[0108] 1-3, Construction of fitting data set. After cold start waiting, when the marked traffic passes through LTC, the frequency f and persistence p of each data packet are calculated using step 2, and the label of this data packet in labels.file is merged and added to the fitting data set (FP-set) in the format <persistence count f, p persistence count | persistence count label>.

[0109] 1-4, Fitting of FP-Rule. The fitting algorithm activation function based on logistic regression is used to derive FP-Rule. FP-Rule falls to a function f(f, p) with f and p as inputs (its return value is 0 or 1) to determine whether the traffic is a DDoS attack.

[0110] 1-5, Adjustment of threshold θ. Adjust θ according to the detected Precision value to ensure a low false negative rate.

[0111] The training process of the CatBoost classifier includes:

[0112] 2-1, Obtaining filtered traffic. After completing the derivation of FP-Rule, the original traffic is replayed and the corresponding persistence count and frequency count of each data packet are calculated.

[0113] 2-2, According to the trained FP-Rule, each data packet is discriminated, the data packets discriminated as DDoS by the rule are filtered, and the data packets discriminated as non-DDoS attack by the rule are added to train_pkt.

[0114] 2-3, When all data packets are played back, the data packets in train_pkt are extracted to obtain the feature vector And its label in labels.file is inserted into CatBoost-trainset, where the data items are packaged in the format

[0115] 2-4, Train CatBoost classifier, take F1-score as loss, start 10-fold cross-training iteration.

[0116] 2-5, save the classifier as model(item), and export a model file for fast loading.

[0117] The embodiment of the application provides a hierarchical method combining online detection and offline detection, and the following contributions are innovatively completed:

[0118] 1. The LTC algorithm is used to analyze the time mode of data packet arrival, without complex flow aggregation or feature extraction, and on this basis, the application innovatively designs a fast filtering rule based on a logistic regression model to screen out DDoS attack traffic, greatly reduces the intensity of DDoS attack, and protects normal service traffic from being affected by the attack.

[0119] 2. A hierarchical detection method is adopted, and an online and offline screening mechanism is combined: the LTC algorithm is used for online coarse screening, DDoS traffic is quickly identified and screened based on the time mode of data packet arrival. The offline fine screening utilizes a machine learning classifier (such as CatBoost), and flow level detection and classification are performed according to obtained data with a lower unbalance rate, and the classification performance and time performance can be effectively improved.

[0120] 3. Compared with the current measurement method that only focuses on the frequency of elements, the application quickly evaluates the frequency and persistence of elements in the network flow, and through double measurement, the embodiment of the application can more accurately identify complex DDoS attack modes. Based on this, the embodiment of the application can realize high-speed identification and coarse screening of DDoS traffic with extremely low resource consumption, and is suitable for real-time analysis of network traffic in a real environment.

[0121] Embodiment 7:

[0122] On the basis of the above-mentioned embodiments, Figure 4 A structural schematic diagram of a data packet filtering device provided by the embodiment of the application is shown in the figure, and the device comprises:

[0123] The processing module 401 is configured to determine source address, destination address, source port, destination port and protocol information carried in a target data packet to be processed as a target identifier of the target data packet, acquire a frequency degree count of data packets of the target identifier before a current time of monitoring and statistics, and acquire a persistence degree count of the data packets of the target identifier within a set time window.

[0124] The classification module 402 is configured to determine whether the target data packet is a data packet of DDoS attack according to the frequency count, the persistence count, an activation function, and a preset threshold value; the activation function is obtained based on historical frequency count, historical persistence count of historical data packets, and historical first classification information of whether the historical data packets are data packets of DDoS attack.

[0125] In a possible implementation, the processing module 401 is further configured to obtain a configured data packet white list, wherein the data packet white list stores the identifiers of data packets that have been determined to be not data packets of DDoS attack; if the target identifier is not included in the data packet white list, the process of obtaining the frequency count of the target identifier is performed before the current time of obtaining the monitoring statistics.

[0126] In a possible implementation, the processing module 401 is further configured to, if it is determined that the target data packet is not a data packet of DDoS attack, obtain feature data of a target flow corresponding to the target data packet, wherein the feature data includes a flow duration, a length standard deviation of each data packet included in the target flow, and a total number of data packets transmitted per second of the target flow.

[0127] The classification module 402 is further configured to input the feature data into a pre-trained classifier to obtain second classification information output by the classifier, the second classification information indicating whether the target flow is a DDoS attack; and if the second classification information indicates that the target flow is a DDoS attack, it is determined that the target data packet is a data packet of DDoS attack.

[0128] In a possible implementation, the classifier is a CatBoost classifier.

[0129] In a possible implementation, the apparatus further includes:

[0130] The training module 403 is configured to obtain a pre-configured sample flow, sample feature data corresponding to the sample data packet, and sample second classification information indicating whether the sample flow is a DDoS attack; input the sample feature data into the CatBoost classifier to be trained to obtain predicted second classification information output by the CatBoost classifier; determine a loss value according to the sample second classification information, the predicted second classification information, and an F1-score algorithm; and adjust parameters of the CatBoost classifier according to the loss value.

[0131] In a possible implementation, the processing module 401 is further configured to, if it is determined that the target data packet is not a data packet of DDoS attack, add the target identifier to a data packet white list.

[0132] In a possible implementation, the training module 403 is further configured to obtain a preconfigured sample data packet, sample frequency degree count corresponding to the sample data packet, sample persistence degree count, and sample first classification information of whether the sample data packet is a DDoS attack data packet; input the sample frequency degree count and the sample persistence degree count into the activation function to be trained, obtain a prediction probability output based on the activation function and a prediction first classification information of whether the sample data packet is a DDoS attack data packet determined based on the preset threshold value; determine a loss value based on the sample first classification information, the prediction first classification information, and a maximum log-likelihood function, and adjust parameters of the activation function based on the loss value.

[0133] Embodiment 8:

[0134] On the basis of the above-mentioned embodiments, the embodiments of the present application further provide an electronic device, Figure 5 A structural schematic diagram of an electronic device provided by the embodiments of the present application is shown in FIG. 5, which includes a processor 501, a communication interface 502, a memory 503, and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 complete mutual communication through the communication bus 504. Figure 5

[0135] The memory 503 stores a computer program, and when the program is executed by the processor 501, the processor 501 executes the following steps:

[0136] The source address, the destination address, the source port, the destination port, and the protocol information carried in the target data packet to be processed are determined as a target identifier of the target data packet;

[0137] A frequency degree count of the data packet of the target identifier appearing before a current time of monitoring and statistics is obtained, and a persistence degree count of the data packet of the target identifier appearing in a set time window is obtained;

[0138] First classification information of whether the target data packet is a DDoS attack data packet is determined based on the frequency degree count, the persistence degree count, an activation function, and a preset threshold value; wherein the activation function is obtained based on historical frequency degree count, historical persistence degree count of historical data packets, and historical first classification information of whether the historical data packets are DDoS attack data packets.

[0139] In a possible implementation, before the frequency degree count of the data packet of the target identifier appearing before the current time of monitoring and statistics is obtained, the method further includes:

[0140] ​obtaining a configured data packet whitelist, wherein the data packet whitelist stores an identification of a data packet that has been determined as not being a DDoS attack packet;

[0141] If the target identification is not included in the data packet whitelist, a process of counting a frequency of occurrence of the target identification before a current time of obtaining monitoring statistics is performed.

[0142] In a possible implementation, if it is determined that the target data packet is not a DDoS attack packet, the method further includes:

[0143] obtaining feature data of a target flow corresponding to the target data packet, wherein the feature data includes a flow duration, a length standard deviation of each data packet included in the target flow, and a total number of data packets transmitted per second of the target flow;

[0144] inputting the feature data into a pre-trained classifier to obtain second classification information output by the classifier, the second classification information indicating whether the target flow is a DDoS attack;

[0145] If the second classification information indicates that the target flow is a DDoS attack, it is determined that the target data packet is a DDoS attack packet.

[0146] In a possible implementation, the classifier is a CatBoost classifier.

[0147] In a possible implementation, a training process of the CatBoost classifier includes:

[0148] obtaining a preconfigured sample flow, sample feature data corresponding to the sample data packet, and sample second classification information indicating whether the sample flow is a DDoS attack;

[0149] inputting the sample feature data into the CatBoost classifier to be trained to obtain predicted second classification information output by the CatBoost classifier;

[0150] determining a loss value according to the sample second classification information, the predicted second classification information, and an F1-score algorithm, and adjusting parameters of the CatBoost classifier according to the loss value.

[0151] In a possible implementation, if it is determined that the target data packet is not a DDoS attack packet, the method further includes:

[0152] adding the target identification to the data packet whitelist.

[0153] In a possible implementation, the training process of the activation function includes:

[0154] obtaining preconfigured sample data packets, sample frequency degree counts corresponding to the sample data packets, sample persistence degree counts, and sample first classification information of whether the sample data packets are data packets of DDoS attacks;

[0155] inputting the sample frequency degree counts and the sample persistence degree counts into the activation function to be trained, obtaining a prediction probability output based on the activation function and a prediction first classification information of whether the sample data packets are data packets of DDoS attacks determined based on the preset threshold value;

[0156] According to the sample first classification information, the prediction first classification information, and the maximum log-likelihood function, a loss value is determined, and the parameters of the activation function are adjusted according to the loss value.

[0157] Since the principle of solving the problem of the above electronic device is similar to the packet filtering method, the implementation of the above electronic device can refer to the embodiments of the method, and the repeated parts will not be described again.

[0158] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface 502 is used for communication between the above electronic device and other devices. The memory can include a Random Access Memory (RAM) and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0159] The above processor can be a general-purpose processor, including a central processing unit, a network processing unit (NP), etc.; it can also be a digital signal processing (DSP) processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0160] Embodiment 9:

[0161] On the basis of the above embodiments, the embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium storing a computer program executable by a processor, when the program is run on the processor, the processor is caused to execute the following steps:

[0162] determining source address, destination address, source port, destination port and protocol information carried in the target data packet to be processed as a target identifier of the target data packet;

[0163] obtaining a frequency count of the target identifier of the data packet before a current time of monitoring statistics and a persistence count of the target identifier of the data packet within a set time window;

[0164] determining first classification information of whether the target data packet is a data packet of DDoS attack according to the frequency count, the persistence count, an activation function and a preset threshold value; wherein the activation function is obtained based on historical frequency count, historical persistence count of historical data packet and historical first classification information of whether the historical data packet is a data packet of DDoS attack.

[0165] In a possible implementation, before the obtaining the frequency count of the target identifier of the data packet before the current time of monitoring statistics, the method further comprises:

[0166] obtaining a configured data packet whitelist, wherein the data packet whitelist saves identifiers of data packets that have been determined to be not DDoS attack;

[0167] if the target identifier is not included in the data packet whitelist, then the subsequent process of obtaining the frequency count of the target identifier of the data packet before the current time of monitoring statistics is executed.

[0168] In a possible implementation, if it is determined that the target data packet is not a data packet of DDoS attack, the method further comprises:

[0169] obtaining feature data of a target flow corresponding to the target data packet, wherein the feature data includes flow duration, length standard deviation of each data packet contained in the target flow, and total number of data packets transmitted per second of the target flow;

[0170] inputting the feature data into a pre-trained classifier to obtain second classification information of whether the target flow is DDoS attack output by the classifier;

[0171] if the second classification information is that the target flow is DDoS attack, then determining that the target data packet is a data packet of DDoS attack.

[0172] In a possible implementation, the classifier is a CatBoost classifier.

[0173] In a possible implementation, the training process of the CatBoost classifier comprises the following steps.

[0174] obtaining a preconfigured sample stream, sample feature data corresponding to the sample data packet, and sample second classification information about whether the sample stream is a DDoS attack;

[0175] inputting the sample feature data into the CatBoost classifier to be trained, and obtaining predicted second classification information output by the CatBoost classifier;

[0176] determining a loss value according to the sample second classification information, the predicted second classification information, and an F1-score algorithm, and adjusting parameters of the CatBoost classifier according to the loss value.

[0177] In a possible implementation, if it is determined that the target data packet is not a data packet of a DDoS attack, the method further comprises the following steps.

[0178] adding the target identifier to a data packet whitelist.

[0179] In a possible implementation, the training process of the activation function comprises the following steps.

[0180] obtaining a preconfigured sample data packet, sample frequency degree count corresponding to the sample data packet, sample persistence degree count, and sample first classification information about whether the sample data packet is a data packet of a DDoS attack;

[0181] inputting the sample frequency degree count and the sample persistence degree count into the activation function to be trained, obtaining predicted first classification information about whether the sample data packet is a data packet of a DDoS attack, which is determined according to a predicted probability output by the activation function and the preset threshold;

[0182] determining a loss value according to the sample first classification information, the predicted first classification information, and a maximum log-likelihood function, and adjusting parameters of the activation function according to the loss value.

[0183] Since the principle of solving the problem of the above computer program product is similar to that of the packet filtering method, implementation of the above computer program product can be referred to the implementation of the method, and details are not repeated.

[0184] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In a software embodiment, various software modules in accordance with embodiments of the present application are stored in a memory such as a computer memory or disk storage for use by, or in connection with, the software on the computer system. The software can provide for programs to be transferred to another computer readable medium (e.g., a removable medium, or a medium conveyed through a computer network) for use in a different system.

[0185] The present application is described in reference to the flow diagrams and / or block diagrams of the methods, apparatus (systems) and computer program products according to this application. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0186] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks.

[0187] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Figure 1 one or more functions specified in the flow diagram and / or block diagram block or blocks. Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A data packet filtering method, characterized in that, The method includes: The source address, destination address, source port, destination port, and protocol information carried in the target data packet to be processed are determined as the target identifier of the target data packet; Obtain the frequency count of the data packets of the target identifier before the current time in the monitoring statistics, and the persistence count of the data packets of the target identifier within a set time window; Based on the frequency count, the persistence count, the activation function, and a preset threshold, the first classification information of whether the target data packet is a DDoS attack data packet is determined; wherein, the activation function is trained based on the historical frequency count, historical persistence count, and historical first classification information of whether the historical data packet is a DDoS attack data packet; If it is determined that the target data packet is not a DDoS attack data packet, the method further includes: Obtain the feature data of the target stream corresponding to the target data packet, wherein the feature data includes the stream duration, the length standard deviation of each data packet contained in the target stream, and the total number of data packets transmitted per second in the target stream; The feature data is input into a pre-trained classifier to obtain a second classification information, output by the classifier, indicating whether the target stream is a DDoS attack. If the second classification information indicates that the target flow is a DDoS attack, then the target data packet is determined to be a DDoS attack data packet.

2. The method according to claim 1, characterized in that, Before obtaining the frequency count of the target identifier's data packets before the current time of the monitoring statistics, the method further includes: Obtain the configured packet whitelist, which contains identifiers of packets that have been determined not to be DDoS attacks; If the target identifier is not included in the data packet whitelist, then the process of obtaining the frequency count of data packets of the target identifier that appeared before the current time in the monitoring statistics is executed.

3. The method according to claim 1, characterized in that, The classifier is a CatBoost classifier.

4. The method according to claim 3, characterized in that, The training process of the CatBoost classifier includes: Obtain pre-configured sample streams, sample feature data corresponding to sample data packets, and sample second classification information of whether the sample stream is a DDoS attack; The sample feature data is input into the CatBoost classifier to be trained, and the predicted second classification information output by the CatBoost classifier is obtained. Based on the sample's second classification information, the predicted second classification information, and the F1-score algorithm, a loss value is determined, and the parameters of the CatBoost classifier are adjusted according to the loss value.

5. The method according to claim 1, characterized in that, If it is determined that the target data packet is not a DDoS attack data packet, the method further includes: Add the target identifier to the data packet whitelist.

6. The method according to claim 1, characterized in that, The training process of the activation function includes: Obtain pre-configured sample data packets, the sample frequency count corresponding to the sample data packets, the sample persistence count, and the first classification information of whether the sample data packets are DDoS attack data packets; The sample frequency count and the sample persistence count are input into the activation function to be trained to obtain the first classification information of whether the sample data packet is a DDoS attack data packet based on the prediction probability output by the activation function and the preset threshold. Based on the sample's first classification information, the predicted first classification information, and the maximized log-likelihood function, a loss value is determined, and the parameters of the activation function are adjusted according to the loss value.

7. A data packet filtering device, characterized in that, The device includes: The processing module is used to determine the source address, destination address, source port, destination port and protocol information carried in the target data packet to be processed as the target identifier of the target data packet; and to obtain the frequency count of the data packets of the target identifier that appeared before the current time and the persistence count of the data packets of the target identifier that appeared within a set time window, based on monitoring statistics. Based on the frequency count, the persistence count, the activation function, and a preset threshold, the first classification information of whether the target data packet is a DDoS attack data packet is determined; wherein, the activation function is trained based on the historical frequency count, historical persistence count, and historical first classification information of whether the historical data packet is a DDoS attack data packet; The processing module is further configured to, if it is determined that the target data packet is not a DDoS attack data packet, obtain the characteristic data of the target flow corresponding to the target data packet, wherein the characteristic data includes the flow duration, the length standard deviation of each data packet contained in the target flow, and the total number of data packets transmitted per second in the target flow; The classification module is further configured to input the feature data into a pre-trained classifier to obtain second classification information on whether the target stream is a DDoS attack, as output by the classifier; if the second classification information indicates that the target stream is a DDoS attack, then the target data packet is determined to be a DDoS attack data packet.

8. An electronic device, characterized in that, The electronic device includes at least a processor and a memory, wherein the processor is configured to implement the steps of the packet filtering method as described in any one of claims 1-6 when executing a computer program stored in the memory.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the packet filtering method as described in any one of claims 1-6.

Images