Network anomaly detection method and device based on data augmentation

Abstract

The invention provides a network anomaly detection method based on data augmentation, and relates to the technical field of network anomaly detection, and the method comprises the steps: obtaining original network flow data; respectively augmenting the original network traffic data through an augmentation algorithm based on interpolation and an augmentation algorithm based on convolution to obtain first attack augmentation data and second attack augmentation data; and constructing augmented data based on the original network flow data, the first attack augmented data and the second attack augmented data, and realizing network anomaly detection through the augmented data. By the adoption of the scheme, accurate network anomaly detection can be achieved when network data is insufficient.

Description

Technical Field

[0001] The present application relates to the technical field of network anomaly detection, and in particular to a network anomaly detection method and device based on data augmentation. Background Art

[0002] The network attack situation is changing rapidly, and new attacks emerge in an endless stream. The serious imbalance between attack samples and benign traffic will affect the training of anomaly detection models. Detection models are generally divided into two scenarios: binary classification and multi-classification. In addition, security personnel hope that when there are insufficient attack samples, there will still be attack data for sharing to facilitate subsequent analysis tasks. Summary of the invention

[0003] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0004] To this end, the first purpose of the present application is to propose a data augmented network anomaly detection method that can achieve accurate network anomaly detection when network data is insufficient.

[0005] The second objective of the present application is to provide a network anomaly detection device with data augmentation.

[0006] To achieve the above-mentioned purpose, the first aspect embodiment of the present application proposes a network anomaly detection method with data augmentation, including: obtaining original network traffic data; augmenting the original network traffic data by an interpolation-based augmentation algorithm and a convolution-based augmentation algorithm, respectively, to obtain first attack augmented data and second attack augmented data; constructing augmented data based on the original network traffic data, the first attack augmented data and the second attack augmented data, and realizing network anomaly detection through the augmented data.

[0007] Optionally, in one embodiment of the present application, network anomaly detection includes binary classification detection and multi-classification detection.

[0008] Optionally, in one embodiment of the present application, when performing binary classification detection, the method includes:

[0009] Obtain the original network traffic data, perform feature extraction and normalization on the original network traffic data, and obtain benign samples and attack samples;

[0010] SMOTE is used as a binary classification interpolation algorithm to augment the attack sample and obtain the first augmented sample;

[0011] Use the attack samples to train the DCGAN model, use the generator in the model to generate attack augmented samples, and use the discriminator in the model to filter out samples with a malicious attack probability greater than a preset threshold to obtain the second augmented samples;

[0012] A training set is constructed based on benign samples, attack samples, first augmented samples and second augmented samples, and a binary classification anomaly detection model is trained using the training set.

[0013] Optionally, in one embodiment of the present application, when performing multi-classification detection, the method includes:

[0014] Obtain the original network traffic data, perform feature extraction and normalization on the original network traffic data, obtain benign samples and attack samples, and determine the category labels of the attack samples;

[0015] Bordline-SMOTE is used as a multi-classification interpolation algorithm to augment the attack samples. During augmentation, the attack data is distinguished according to the category label. For each attack category, a sample set that meets the Danger attribute is found. The SMOTE algorithm is used to perform interpolation augmentation in the set to obtain the third augmented sample, in which more than half of the surrounding samples of the attack sample that meets the Danger attribute have the same attack category as the attack sample itself.

[0016] Use any benign samples and all attack samples to train the ACGAN model, use the generator in the model to generate augmented samples corresponding to all category labels, and use the discriminator in the model to filter out samples with a malicious attack probability greater than a preset threshold to obtain the fourth augmented sample;

[0017] ACGAN is used to supplement and augment the attack categories that do not have Danger attributes in the third augmented sample to obtain the fifth augmented sample;

[0018] A training set is constructed based on the benign samples, the third augmented samples, the fourth augmented samples and the fifth augmented samples, and a multi-classification anomaly detection model is trained through the training set.

[0019] Optionally, in one embodiment of the present application, network anomaly detection further includes network attack data sharing, and the method further includes:

[0020] After the attack augmented data is generated, a constrained heuristic search algorithm is used to search the attack augmented data, and the searched attack augmented data is shared as network attack data;

[0021] Among them, a constrained heuristic search algorithm is used to search the attack augmented data, including:

[0022] The fields corresponding to all columns in the two-dimensional data table formed by network packets or network flows in the original network traffic data are divided into numerical type and string type;

[0023] Process the character segment so that the character segment can be traversed, wherein the character segment is processed, including: for a string type, the corresponding field is saved to a specified database according to the historical record, and the records are deduplicated and numbered; for a numeric type, the upper and lower bounds are given;

[0024] A genetic algorithm is used for search. The fitness function of the genetic algorithm is the minimum distance between the feature space vector of the search data and the feature space vector of the attack augmented data. When searching, all fields are traversed and used as variables. The searched attack augmented data are constrained to meet the requirement of repeatable network traffic.

[0025] To achieve the above-mentioned purpose, a second aspect of the present invention provides a data augmented network anomaly detection device, comprising:

[0026] A data acquisition module, used to acquire raw network traffic data;

[0027] An augmentation module, used to augment the original network traffic data by using an interpolation-based augmentation algorithm and a convolution-based augmentation algorithm to obtain first attack augmented data and second attack augmented data;

[0028] The network anomaly detection module is used to construct augmented data based on the original network traffic data, the first attack augmented data and the second attack augmented data, and realize network anomaly detection through the augmented data.

[0029] Optionally, in one embodiment of the present application, network anomaly detection includes binary classification detection and multi-classification detection.

[0030] Optionally, in one embodiment of the present application, when performing binary classification detection,

[0031] The data acquisition module is also used to obtain the original network traffic data, and perform feature extraction and normalization on the original network traffic data to obtain benign samples and attack samples;

[0032] The augmentation module is also used to augment the attack sample using SMOTE as a binary classification interpolation algorithm to obtain a first augmented sample;

[0033] The augmentation module is also used to train the DCGAN model using the attack samples, generate attack augmented samples using the generator in the model, and screen out samples with a malicious attack probability greater than a preset threshold through the discriminator in the model to obtain a second augmented sample;

[0034] The network anomaly detection module is also used to construct a training set based on benign samples, attack samples, first augmented samples and second augmented samples, and train a binary classification anomaly detection model through the training set.

[0035] Optionally, in one embodiment of the present application, when performing multi-classification detection,

[0036] The data acquisition module is also used to obtain the original network traffic data, perform feature extraction and normalization on the original network traffic data, obtain benign samples and attack samples, and determine the category label of the attack sample;

[0037] The augmentation module is also used to augment the attack samples by using Bordline-SMOTE as a multi-classification interpolation algorithm. During augmentation, the attack data is distinguished according to the category label. For each attack category, a sample set that meets the Danger attribute is found. The SMOTE algorithm is used to perform interpolation augmentation in the set to obtain a third augmented sample, in which more than half of the surrounding samples of the attack sample that meets the Danger attribute have the same attack category as the attack sample itself.

[0038] The augmentation module is also used to train the ACGAN model using any benign samples and all attack samples, use the generator in the model to generate augmented samples corresponding to all category labels, and use the discriminator in the model to filter out samples with a malicious attack probability greater than a preset threshold to obtain a fourth augmented sample;

[0039] The augmentation module is also used to use ACGAN to supplement and augment the attack categories that do not have the Danger attribute in the third augmented sample to obtain the fifth augmented sample;

[0040] The network anomaly detection module is also used to construct a training set based on the benign sample, the third augmented sample, the fourth augmented sample and the fifth augmented sample, and train the multi-classification anomaly detection model through the training set.

[0041] Optionally, in one embodiment of the present application, network anomaly detection also includes network attack data sharing.

[0042] The network anomaly detection module is further used to search the attack augmented data using a constrained heuristic search algorithm after the attack augmented data is generated, and share the searched attack augmented data as network attack data;

[0043] Among them, a constrained heuristic search algorithm is used to search the attack augmented data, including:

[0044] The fields corresponding to all columns in the two-dimensional data table formed by network packets or network flows in the original network traffic data are divided into numerical type and string type;

[0045] Process the character segment so that the character segment can be traversed, wherein the character segment is processed, including: for a string type, the corresponding field is saved to a specified database according to the historical record, and the records are deduplicated and numbered; for a numeric type, the upper and lower bounds are given;

[0046] A genetic algorithm is used for search. The fitness function of the genetic algorithm is the minimum distance between the feature space vector of the search data and the feature space vector of the attack augmented data. When searching, all fields are traversed and used as variables. The searched attack augmented data are constrained to meet the requirement of repeatable network traffic.

[0047] The network anomaly detection method and device with data augmentation in the embodiments of the present application improve the performance of a binary or multi-classification detection model through oversampling augmentation when attack data is insufficient; at the same time, for the samples generated by augmentation, the embodiments can find the corresponding solution in the problem space to facilitate data sharing and analysis.

[0048] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0050] Figure 1 A flowchart of a network anomaly detection method for data augmentation provided in Example 1 of the present application;

[0051] Figure 2 This is the overall design logic diagram of the HeteAUG model of the embodiment of the present application;

[0052] Figure 3 This is a structural diagram of HeteAUG-bi according to an embodiment of the present application;

[0053] Figure 4 This is a structural diagram of HeteAUG-multi according to an embodiment of the present application;

[0054] Figure 5 This is a structural diagram of HeteAUG-P according to an embodiment of the present application;

[0055] Figure 6 A schematic diagram of the structure of a network anomaly detection device with data augmentation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0057] The following describes the network anomaly detection method and device for data augmentation according to an embodiment of the present application with reference to the accompanying drawings.

[0058] Figure 1 A flowchart of a network anomaly detection method with data augmentation provided in Example 1 of the present application.

[0059] like Figure 1 As shown, the network anomaly detection method with data augmentation includes the following steps:

[0060] Step 101, obtaining original network traffic data;

[0061] Step 102, augmenting the original network traffic data by using an interpolation-based augmentation algorithm and a convolution-based augmentation algorithm respectively, to obtain first attack augmented data and second attack augmented data;

[0062] Step 103: construct augmented data based on the original network traffic data, the first attack augmented data, and the second attack augmented data, and implement network anomaly detection through the augmented data.

[0063] The network anomaly detection method and device with data augmentation in the embodiments of the present application improve the performance of a binary or multi-classification detection model through oversampling augmentation when attack data is insufficient; at the same time, for the samples generated by augmentation, the embodiments can find the corresponding solution in the problem space to facilitate data sharing and analysis.

[0064] Embodiment 2 of the present application proposes a HeteAUG model, through which network anomaly detection is implemented.

[0065] Figure 2 The overall design logic diagram of the HeteAUG model is as follows: Figure 2As shown, the overall logic of the HeteAUG model is as follows: In the feature space, this embodiment fuses the augmentation results based on interpolation and convolutional generative adversarial networks in a complementary manner. Based on the effective feature space augmented samples, this embodiment uses a constrained heuristic search algorithm to find the corresponding original traffic space augmented samples. This embodiment selects SMOTE as the binary classification interpolation method, and selects BordlineSMOTE as the multi-classification interpolation method. Multi-classification is more sensitive to category boundaries, so this embodiment must consider fine-grained labels and try to alleviate the problem of category overlap. The choice of GAN can be analyzed similarly. In general, HeteAUG-multi takes into account the problem of category overlap based on HeteAUG-bi. The goal of HeteAUG-P is to find a solution to the problem space based on the effective feature space augmented solution.

[0066] (1) HeteAUG-bi sub-model (two-class augmentation)

[0067] Figure 3 is the structure diagram of HeteAUG-bi, such as Figure 3 As shown in the figure, the model consists of DCGAN and SMOTE. First, SMOTE is augmented by interpolation to obtain AUG-bi-1 (augmented sample 1, yellow dots). Secondly, the DCGAN model is trained with attack data. Then the generator is used to generate the corresponding attack augmented data. These directly generated samples need to pass through the discriminator. If the probability of being judged as a malicious attack is greater than a certain threshold, it is selected as an augmented sample. The sample augmented by DCGAN is called AUG-bi-2 (augmented sample 2, blue dots). AUG-bi-1 and AUG-bi-2 are used together as the final augmented samples.

[0068] (2) HeteAUG-multi sub-model (multi-classification augmentation)

[0069] In order to better deal with the intra-class overlap problem of multi-classification, this embodiment selects BorderlineSMOTE that interpolates within the attack class and ACGAN that considers fine-grained labels. Figure 4 As shown, this embodiment describes the HeteAUG-multi model in detail. The model consists of ACGAN and BordlineSMOTE. Since benign traffic will affect the classification boundary of multi-classification, the HeteAUG-multi augmented model uses benign samples and attack samples as input. The BordlineSMOTE here is designed for multi-classification and is called multi-BordlineSMOTE. The core idea of ​​multi-BordlineSMOTE is: first, distinguish attack data according to fine-grained labels. Secondly, for the attack category attacki , find the set of these samples that satisfy the Danger property. Then, use the SMOTE algorithm to perform interpolation augmentation in this set. The label of these augmented data is the attack category currently analyzed. The Danger property here means that more than half (but not all) of the surrounding samples of the currently traversed sample are majority class samples. This embodiment uses multi-BordlineSMOTE to obtain the augmented data AUG-multi-1 (yellow dots). Then, this embodiment uses part of the benign data and all the attack data as the training data of ACGAN. After ACGAN training is completed, the generator is used to generate augmented samples for each category to obtain AUG-multi-2 (blue dots). Here, the input of the generator needs to be labeled to specify the augmented category. Next, check the augmented data in AUG-multi-1. If there are attack categories that do not have the Danger attribute, then use ACGAN to further supplement and augment these categories, and the augmented samples are named AUG-multi-3 (blue dots). AUG-multi-1, AUG-multi-2 and AUG-multi-3 are together as the final augmented samples.

[0070] (3) HeteAUG-P sub-model (problem space augmentation)

[0071] Figure 5 This is the structure diagram of HeteAUG-P, such as Figure 5 As shown, in the binary classification augmented model, this embodiment obtains the final augmented samples AUG-bi-1 (a1∈AUG-bi-1) and AUG-bi-2 (a2∈AUG-bi-2) in the feature space. This embodiment divides the fields corresponding to all columns in the two-dimensional data table formed by the network data packet (pcap) or network traffic (flow) in the problem space into numerical type and string type. For the string type, this embodiment saves the corresponding field to the specified database according to the historical records, and deduplicates and numbers the records, such as the formation of the source IP address database. For the numerical type, this embodiment gives reasonable upper and lower bounds, such as the source port range of 1-65535. After the above processing, all fields are traversable. Then, this embodiment uses a genetic algorithm to search, and each field corresponds to a variable. The fitness function of the genetic algorithm is the minimum distance between the vector after the searched sample feature extraction and the existing valid augmented samples (AUG-bi-1 and AUG-bi-2) in the feature space. During the search process, the searched samples need to be constrained so that they meet the requirements of network traffic replay. As Figure 5As shown, the constraint can be: if the protocol is TCP, then the UDP protocol related fields are empty. A record in the problem space is "1487598190,192.168.3.2,192.168.100.2,2,...".

[0072] This example conducts an experimental comparison between the HeteAUG model and the existing method, and the results are as follows:

[0073] (1) Experimental results of HeteAUG-bi submodel

[0074] Table 1 Performance comparison of binary classification oversampling methods in IDS field

[0075]

[0076]

[0077] △ indicates the top 3 algorithms.

[0078] This embodiment is compared with the SOTA two-classification augmentation method based on IDS, and the compared algorithms mainly include GAN-based and SMOTE-based related algorithms. As shown in Table 1, the algorithm of this embodiment has achieved the best augmentation effect on each data set. In particular, on the complex Kitsune data set, the accuracy of the augmentation algorithm of this embodiment is 0.871. The accuracy of the second-ranked algorithm WCGAN is 0.810. The model of this embodiment improves the accuracy by at least 6.1% on this data set. In order to better analyze the effect of augmentation, this embodiment first observes the detection of balanced and unbalanced states. It can be seen that the TON data set and the Kitsune data set are seriously affected by the imbalance, while the UKM data set is not greatly affected. This is strongly related to the collection form of the data set and the data distribution after feature extraction. Analysis shows that the GAN-based algorithm performs better on the Kitsune-based data set. The SMOTE-based algorithm performs better on the flow-based TON data set and UKM data set. This shows that different augmentation algorithms have differences in performance for different distributions. Secondly, the performance of each algorithm is analyzed. With the continuous improvement of SMOTE-related algorithms, their detection effects on the TON dataset have been significantly improved. For example, the detection effect of BordlineSMOTE is better than SMOTE. However, the opposite is true on the Kitsune dataset. The dimension of the TON dataset is 21 dimensions, while the feature extraction algorithm of the Kitsune dataset is more complex, and the dimension of the features has increased to 100 dimensions. In contrast, the types and number of attacks against Kitsune have also increased. G-IDS is essentially a linear GAN. It is obvious that the detection effect after augmentation with G-IDS has improved compared to imbalance, but it is still limited. ODDS uses two generators to learn a more optimized distribution, and the effect is significantly improved. WCGAN incorporates benign data into model training, which is effective for the Kitsune dataset, but not for the TON dataset and UKM dataset. TMG-GAN uses cosine similarity to provide diversity for the generated samples, but does not supplement the missing distribution outside of GAN. RAPIER first estimates the distribution using MADE and then augments it in two types of abnormal and benign traffic. MADE has a large deviation in the estimated distribution in an unbalanced data set, which affects the final detection results. Various improved forms of GAN and SMOTE always fail to achieve the detection effect of the model in this chapter, which illustrates the problem of missing distribution, thus illustrating the importance of distribution supplementation and the effectiveness of HeteAUG-bi.

[0079] (2) Experimental results of HeteAUG-multi sub-model

[0080] Table 2 Performance comparison of oversampling methods in multi-classification (accuracy)

[0081]

[0082]

[0083] HeteAUG-multi means ACGAN with JDA+multi=BorderlineSMOTE.

[0084] θ indicates that the corresponding ACGAN training does not use benign data.

[0085] This embodiment conducts comparative experiments on three data sets. As shown in Table 2, the augmented model HeteAUG-multi of this embodiment performs best on each data set. Specifically, the accuracy of HeteAUG-multi on the Kitsune data set is 0.623, while the accuracy of ACGAN with JDAθ is 0.502. Compared with the comparative algorithm, the augmented model proposed in this embodiment is improved by 12.1%. Through analysis, it is found that the SMOTE-related algorithm performs better on the TON data set, and the ACGAN trained with benign traffic performs better on the Kitsune data set. It is worth noting that on the UKM data set, there are only 2 attack samples in each category, and many augmented algorithms have side effects. The EOS algorithm, which is essentially an improved algorithm of SMOTE, performs generally on the Kitsune data set, but achieves good detection results on the UKM data set, which shows that the distribution of the UKM data set is more matched with the EOS augmented algorithm. HeteAUG-multi has a better augmentation effect than HeteAUG-multiθ, which shows that adding an appropriate amount of benign data is helpful for learning better classification boundaries.

[0086] (3) Experimental results of HeteAUG-P submodel

[0087] Table 3. Single dataset augmentation performance in the problem space

[0088]

[0089]

[0090] SSL Renegotiation(SSL), SYN DoS(SYN), SSDP Flood(SSDP).

[0091] Table 4 Performance of the problem space augmentation algorithm on the complex Kitsune dataset (binary classification scenario)

[0092] Model TPR FPR F1 ACC AUC unbalanced 0.168 0.028 0.283 0.513 0.570 HeteAUG-P-1 0.802 0.317 0.786 0.751 0.742 HeteAUG-P-10 0.898 0.220 0.871 0.848 0.839

[0093] In order to verify the effectiveness of the problem space augmentation algorithm, two groups of experiments are set up in this embodiment. First, the performance of problem space augmentation under a single data set is verified. As shown in Table 3, the proposed HeteAUG-P model can find the corresponding solution in the problem space very well. Next is the second group of experiments. This group of experiments uses the complex Kitsune data set. This experiment uses the configuration of the binary classification comparison experiment in the feature space. As shown in Table 4, the accuracy of the Hete-AUG-10 model is improved by 33.5% compared with the unbalanced state. The FPR increased in the unbalanced state, and it can be seen that data augmentation introduced some false positives to the model, but as the number of iterations increased, the effect of data augmentation became better and better, and the FPR decreased accordingly. This shows that the proposed HeteAUG-P model is also effective for the augmentation of complex data sets, but more iterations are required.

[0094] Network anomaly detection is an important network management task. IDS based on machine learning / deep learning faces the impact of imbalanced training data. Oversampling, as a promising technology to solve imbalance, also has the following two problems: first, it is impossible to learn a more complete attack distribution; second, the augmented data cannot correspond to the problem space. To this end, this embodiment proposes the HeteAUG model. In the feature space, this embodiment combines the augmentation results based on interpolation and convolution-based generative adversarial networks in a complementary manner. Based on the effective feature space augmented samples, this embodiment uses a constrained heuristic search algorithm to find the corresponding original traffic space augmented samples. Experiments show that the augmented model proposed in this embodiment is superior to other SOTA models and can be used well in practical scenarios. Specifically, taking the complex Kitsune dataset as an example, taking accuracy as the evaluation indicator, the algorithm proposed in this embodiment is at least 6.1% higher than the SOTA algorithm in binary augmentation. In multi-class augmentation, the proposed algorithm is at most 12.1% higher than the SOTA algorithm. In the augmentation of the binary problem space, the proposed augmentation algorithm is 33.5% higher than the unbalanced state.

[0095] In order to implement the above embodiment, the present application also proposes a network anomaly detection device with data augmentation.

[0096] Figure 6 A schematic diagram of the structure of a network anomaly detection device with data augmentation provided in an embodiment of the present application.

[0097] like Figure 6 As shown, the data augmented network anomaly detection device includes:

[0098] A data acquisition module, used to acquire raw network traffic data;

[0099] An augmentation module, used to augment the original network traffic data by using an interpolation-based augmentation algorithm and a convolution-based augmentation algorithm to obtain first attack augmented data and second attack augmented data;

[0100] The network anomaly detection module is used to construct augmented data based on the original network traffic data, the first attack augmented data and the second attack augmented data, and realize network anomaly detection through the augmented data.

[0101] Optionally, in one embodiment of the present application, network anomaly detection includes binary classification detection and multi-classification detection.

[0102] Optionally, in one embodiment of the present application, when performing binary classification detection,

[0103] The data acquisition module is also used to obtain the original network traffic data, and perform feature extraction and normalization on the original network traffic data to obtain benign samples and attack samples;

[0104] The augmentation module is also used to augment the attack sample using SMOTE as a binary classification interpolation algorithm to obtain a first augmented sample;

[0105] The augmentation module is also used to train the DCGAN model using the attack samples, generate attack augmented samples using the generator in the model, and screen out samples with a malicious attack probability greater than a preset threshold through the discriminator in the model to obtain a second augmented sample;

[0106] The network anomaly detection module is also used to construct a training set based on benign samples, attack samples, first augmented samples and second augmented samples, and train a binary classification anomaly detection model through the training set.

[0107] Optionally, in one embodiment of the present application, when performing multi-classification detection,

[0108] The data acquisition module is also used to obtain the original network traffic data, perform feature extraction and normalization on the original network traffic data, obtain benign samples and attack samples, and determine the category label of the attack sample;

[0109] The augmentation module is also used to augment the attack samples by using Bordline-SMOTE as a multi-classification interpolation algorithm. During augmentation, the attack data is distinguished according to the category label. For each attack category, a sample set that meets the Danger attribute is found. The SMOTE algorithm is used to perform interpolation augmentation in the set to obtain a third augmented sample, in which more than half of the surrounding samples of the attack sample that meets the Danger attribute have the same attack category as the attack sample itself.

[0110] The augmentation module is also used to train the ACGAN model using any benign samples and all attack samples, use the generator in the model to generate augmented samples corresponding to all category labels, and use the discriminator in the model to filter out samples with a malicious attack probability greater than a preset threshold to obtain a fourth augmented sample;

[0111] The augmentation module is also used to use ACGAN to supplement and augment the attack categories that do not have the Danger attribute in the third augmented sample to obtain the fifth augmented sample;

[0112] The network anomaly detection module is also used to construct a training set based on the benign sample, the third augmented sample, the fourth augmented sample and the fifth augmented sample, and train the multi-classification anomaly detection model through the training set.

[0113] Optionally, in one embodiment of the present application, network anomaly detection also includes network attack data sharing.

[0114] The network anomaly detection module is further used to search the attack augmented data using a constrained heuristic search algorithm after the attack augmented data is generated, and share the searched attack augmented data as network attack data;

[0115] Among them, a constrained heuristic search algorithm is used to search the attack augmented data, including:

[0116] The fields corresponding to all columns in the two-dimensional data table formed by network packets or network flows in the original network traffic data are divided into numerical type and string type;

[0117] Process the character segment so that the character segment can be traversed, wherein the character segment is processed, including: for a string type, the corresponding field is saved to a specified database according to the historical record, and the records are deduplicated and numbered; for a numeric type, the upper and lower bounds are given;

[0118] A genetic algorithm is used for search. The fitness function of the genetic algorithm is the minimum distance between the feature space vector of the search data and the feature space vector of the attack augmented data. When searching, all fields are traversed and used as variables. The searched attack augmented data are constrained to meet the requirement of repeatable network traffic.

[0119] It should be noted that the above explanations of the data augmented network anomaly detection method embodiment are also applicable to the data augmented network anomaly detection device of this embodiment, and will not be repeated here.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0121] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0122] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0123] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0124] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0125] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0126] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0127] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A network anomaly detection method based on data augmentation, characterized in that: include: Get raw network traffic data; The original network traffic data is augmented by an interpolation-based augmentation algorithm and a convolution-based augmentation algorithm respectively, to obtain first attack augmented data and second attack augmented data; Augmented data is constructed based on the original network traffic data, the first attack augmented data, and the second attack augmented data, and network anomaly detection is implemented through the augmented data.

2. The method according to claim 1, characterized in that The network anomaly detection includes binary classification detection and multi-classification detection.

3. The method according to claim 2, characterized in that When performing binary classification detection, the method includes: Acquire original network traffic data, and perform feature extraction and normalization processing on the original network traffic data to obtain benign samples and attack samples; SMOTE is used as a binary classification interpolation algorithm to augment the attack sample and obtain the first augmented sample; Use the attack samples to train the DCGAN model, use the generator in the model to generate attack augmented samples, and use the discriminator in the model to filter out samples with a malicious attack probability greater than a preset threshold to obtain the second augmented samples; A training set is constructed based on the benign sample, the attack sample, the first augmented sample, and the second augmented sample, and a binary classification anomaly detection model is trained by using the training set.

4. The method according to claim 2, characterized in that When performing multi-classification detection, the method includes: Obtaining original network traffic data, performing feature extraction and normalization processing on the original network traffic data, obtaining benign samples and attack samples, and determining category labels of attack samples; Bordline-SMOTE is used as a multi-classification interpolation algorithm to augment the attack samples. During augmentation, the attack data is distinguished according to the category label. For each attack category, a sample set that meets the Danger attribute is found. The SMOTE algorithm is used to perform interpolation augmentation in the set to obtain the third augmented sample, in which more than half of the surrounding samples of the attack sample that meets the Danger attribute have the same attack category as the attack sample itself. Use any benign samples and all attack samples to train the ACGAN model, use the generator in the model to generate augmented samples corresponding to all category labels, and use the discriminator in the model to filter out samples with a malicious attack probability greater than a preset threshold to obtain the fourth augmented sample; ACGAN is used to supplement and augment the attack category without Danger attribute in the third augmented sample to obtain a fifth augmented sample; A training set is constructed based on the benign sample, the third augmented sample, the fourth augmented sample and the fifth augmented sample, and a multi-classification anomaly detection model is trained by the training set.

5. The method according to claim 1, characterized in that The network anomaly detection also includes network attack data sharing, and the method further includes: After the attack augmented data is generated, a constrained heuristic search algorithm is used to search the attack augmented data, and the searched attack augmented data is shared as network attack data; The method of using a constrained heuristic search algorithm to search for attack augmented data includes: Classify the fields corresponding to all columns in the two-dimensional data table formed by the network packets or network flows in the original network traffic data into numerical type and string type; Processing the character segment so that the character segment can be traversed, wherein the processing of the character segment includes: for a string type, saving the corresponding field into a specified database according to the historical record, and removing duplicates and numbering the records, and for a numeric type, providing upper and lower bounds; A genetic algorithm is used for searching, and the fitness function of the genetic algorithm is the minimum distance between the feature space vector of the search data and the feature space vector of the attack augmented data. When searching, all fields are traversed and used as variables, and the searched attack augmented data are constrained to meet the requirement of repeatability of network traffic.

6. A network anomaly detection device based on data augmentation, characterized in that: include: A data acquisition module, used to acquire raw network traffic data; An augmentation module, used to augment the original network traffic data by using an interpolation-based augmentation algorithm and a convolution-based augmentation algorithm to obtain first attack augmented data and second attack augmented data; A network anomaly detection module is used to construct augmented data based on the original network traffic data, the first attack augmented data and the second attack augmented data, and implement network anomaly detection through the augmented data.

7. The device according to claim 6, characterized in that The network anomaly detection includes binary classification detection and multi-classification detection.

8. The device according to claim 7, characterized in that When performing binary classification detection, The data acquisition module is also used to acquire original network traffic data, and perform feature extraction and normalization processing on the original network traffic data to obtain benign samples and attack samples; The augmentation module is further used to augment the attack sample using SMOTE as a binary classification interpolation algorithm to obtain a first augmented sample; The augmentation module is further used to train the DCGAN model using the attack sample, generate the attack augmented sample using the generator in the model, and screen out the sample whose malicious attack probability is greater than a preset threshold through the discriminator in the model to obtain the second augmented sample; The network anomaly detection module is further used to construct a training set based on the benign sample, the attack sample, the first augmented sample and the second augmented sample, and train a binary classification anomaly detection model through the training set.

9. The device according to claim 7, characterized in that When performing multi-classification detection, The data acquisition module is also used to acquire original network traffic data, perform feature extraction and normalization processing on the original network traffic data, obtain benign samples and attack samples, and determine the category labels of the attack samples; The augmentation module is further used to augment the attack sample by using Bordline-SMOTE as a multi-classification interpolation algorithm. During augmentation, the attack data is distinguished according to the category label, and for each attack category, a sample set satisfying the Danger attribute is found, and the SMOTE algorithm is used to perform interpolation augmentation in the set to obtain a third augmented sample, wherein more than half of the surrounding samples of the attack sample satisfying the Danger attribute have the same attack category as the attack sample itself; The augmentation module is further used to train the ACGAN model using any benign samples and all attack samples, generate augmented samples corresponding to all category labels using the generator in the model, and screen out samples with a malicious attack probability greater than a preset threshold through the discriminator in the model to obtain a fourth augmented sample; The augmentation module is further used to use ACGAN to supplement and augment the attack category without Danger attribute in the third augmented sample to obtain a fifth augmented sample; The network anomaly detection module is further used to construct a training set based on the benign sample, the third augmented sample, the fourth augmented sample and the fifth augmented sample, and train the multi-classification anomaly detection model through the training set.

10. The device according to claim 6, characterized in that The network anomaly detection also includes network attack data sharing, The network anomaly detection module is further used to search the attack augmented data using a constrained heuristic search algorithm after generating the attack augmented data, and share the searched attack augmented data as network attack data; The method of using a constrained heuristic search algorithm to search for attack augmented data includes: Classify the fields corresponding to all columns in the two-dimensional data table formed by the network packets or network flows in the original network traffic data into numerical type and string type; Processing the character segment so that the character segment can be traversed, wherein the processing of the character segment includes: for a string type, saving the corresponding field into a specified database according to the historical record, and removing duplicates and numbering the records, and for a numeric type, providing upper and lower bounds; A genetic algorithm is used for searching, and the fitness function of the genetic algorithm is the minimum distance between the feature space vector of the search data and the feature space vector of the attack augmented data. When searching, all fields are traversed and used as variables, and the searched attack augmented data are constrained to meet the requirement of repeatability of network traffic.

Images