Sensitivity-based network intrusion detection resampling method, system, equipment and medium

Abstract

The invention discloses a sensitivity-based network intrusion detection resampling method, system and device and a medium, and relates to the technical field of data detection, and the method comprises the steps: obtaining network data containing normal traffic and attack traffic; prior probabilities and class condition probabilities of the two classes of data are calculated respectively, then posterior probabilities are obtained, and the sensitivity weight of each sample is calculated; carrying out putting-back undersampling on a normal flow sample according to the sensitivity weight; carrying out replacement oversampling on the attack traffic sample according to the sensitivity weight; a sampling process is repeated to generate a plurality of balance training subsets, and a base classifier is trained based on each subset; and calculating voting weights based on the performance evaluation indexes of the base classifiers on the verification set, and obtaining weighted voting results based on the voting weights to classify new data. Random undersampling neglects the importance of boundary samples to decision boundary learning, and sensitivity weighted sampling can guide the model to sample the boundary samples, so that important information is prevented from being lost.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data detection, and in particular to a network intrusion detection resampling method and system based on sensitivity, a device and a medium. BACKGROUND

[0002] Building an efficient and timely responsive network intrusion detection system is a key link to protect information security. The core of the system is to accurately analyze massive network data to effectively identify and prevent potential network attack behavior. However, network intrusion detection data sets often show obvious class imbalance, that is, the order of magnitude of normal traffic data is much higher than that of attack samples, which is a major challenge for current research work. Because the attack samples are much less than the normal traffic samples, the network intrusion detection system is difficult to learn enough attack features from the data, which can easily lead to poor detection performance, especially in identifying rare attack patterns. Data imbalance can also increase the risk of false positives (mistaking normal traffic as attacks) and false negatives (failing to detect real attacks) and other limitations. There are many mature methods to deal with the problem of class imbalance. The most common approach is resampling, such as randomly undersampling majority class samples to balance the class distribution, or oversampling minority class samples by interpolation to balance the class distribution. However, random undersampling directly deletes normal samples, which can destroy the statistical rules of traffic and cause information loss and mislead model training; interpolation-based oversampling methods generate synthetic samples in high-dimensional sparse space, which can easily introduce noise and amplify attack feature differences, making it difficult to learn effective minority class sample patterns. In addition, both of these methods use random methods to select samples, ignoring the fact that different samples have different sensitivities to class imbalance, that is, different samples have different importance for learning the decision boundary. The existing methods ignore this feature, which limits the generalization ability of the classifier trained. SUMMARY

[0003] In view of the above problems, the present application is proposed.

[0004] Therefore, the problem to be solved by the present application is that existing methods use random methods to select samples, ignoring the fact that different samples have different sensitivities to class imbalance, that is, different samples have different importance for learning the decision boundary. The existing methods ignore this feature, which limits the generalization ability of the classifier trained.

[0005] To solve the above technical problems, the application provides the following technical scheme: a network intrusion detection resampling method based on sensitivity, which comprises the following steps: acquiring network data containing normal traffic and attack traffic, performing feature extraction on the network data, and dividing the network data into normal traffic samples and attack traffic samples according to the extracted features; calculating the prior probability and the class conditional probability of each sample respectively, obtaining the posterior probability, and calculating the sensitivity weight of each sample under the condition of class imbalance based on the Bayesian method; for the normal traffic samples, undersampling with replacement is performed according to the sensitivity weight, and the samples with a sensitivity higher than a threshold are retained; for the attack traffic samples, oversampling with replacement is performed according to the sensitivity weight, and new synthetic samples are generated through k-nearest neighbor interpolation; a plurality of balanced training subsets are generated by repeating the sampling process, and a base classifier is trained based on each balanced training subset; the voting weight is calculated based on the performance evaluation index of each base classifier on the verification set, and the weighted voting result is obtained based on the voting weight, so as to finally classify the unknown network traffic samples.

[0006] As a preferred scheme of the network intrusion detection resampling method based on sensitivity, before the feature extraction is performed on the network data, the network data collected is subjected to the operations of de-duplication, missing value filling and abnormal value processing; and the features extracted from the network data include but are not limited to spatiotemporal features, protocol features, statistical features and session-level features.

[0007] As a preferred scheme of the network intrusion detection resampling method based on sensitivity, the calculation of the prior probability and the class conditional probability of the two categories of samples respectively comprises the following steps: the number of normal traffic samples and the number of attack traffic samples are counted from the preprocessed feature data, and the prior probability under the unbalanced state is calculated; the prior probability includes the normal traffic prior probability and the attack traffic prior probability; the normal traffic prior probability is the proportion of the number of normal traffic samples to the total number of samples; the attack traffic prior probability is the proportion of the number of attack traffic samples to the total number of samples; the mean and variance of each feature are calculated for each category of samples, and the class conditional probability of each category of samples is estimated.

[0008] As a preferred scheme of the network intrusion detection resampling method based on sensitivity, the posterior probability is obtained by multiplying the prior probability and the class conditional probability; the calculation of the sensitivity weight of each sample under the condition of class imbalance based on the Bayesian method comprises the following steps: the posterior probability is combined and subtracted to obtain the sensitivity score of each sample, and the sensitivity score is normalized to 0 to 1 to obtain the sensitivity weight.

[0009] As a preferred scheme of the network intrusion detection resampling method based on sensitivity, the new synthetic sample generated by the k-nearest neighbor interpolation method comprises: for the sampled attack traffic sample, a neighbor is randomly selected from the k nearest neighbors, linear interpolation is performed between the sample and the neighbor at a random ratio to generate a new attack traffic sample, and the interpolation ratio is uniformly distributed in [0, 1].

[0010] As a preferred scheme of the network intrusion detection resampling method based on sensitivity, the repeated sampling process generates several balanced training subsets, which comprises: the under-sampling result of the normal traffic sample and the over-sampling result of the attack traffic sample are combined to form a balanced training subset, and the process is repeated for several rounds to generate several balanced training subsets.

[0011] As a preferred scheme of the network intrusion detection resampling method based on sensitivity, the calculation formula of the weighted voting result is represented as,

[0012]

[0013] wherein, is represented as y that makes the value of the expression maximum, T is represented as the total number of base classifiers, C t (x) is represented as the output of the tth base classifier, w t is represented as the voting weight of the tth base classifier, δ(·) is represented as an indicator function, which takes the value of 1 when the condition in the parentheses is true, and otherwise takes the value of 0, y is represented as the current voting candidate class label including the normal traffic class and the attack traffic class, and is used to traverse all possible classification results.

[0014] To solve the above technical problems, the application provides the following technical scheme: a network intrusion detection resampling system based on sensitivity, comprising a data preprocessing module, a sensitivity calculation module and an integrated sampling and classification module; the data preprocessing module acquires network data containing normal traffic and attack traffic, performs feature extraction on the network data, and divides the network data into normal traffic samples and attack traffic samples according to the extracted features; the sensitivity calculation module calculates the prior probability and the class conditional probability of each sample respectively, obtains the posterior probability, and calculates the sensitivity weight of each sample under the condition of class imbalance based on the Bayesian method; the integrated sampling and classification module performs undersampling with replacement on the normal traffic samples according to the sensitivity weight, and retains samples with a sensitivity higher than a threshold; performs oversampling with replacement on the attack traffic samples according to the sensitivity weight, and generates new synthetic samples through k-nearest neighbor interpolation; generates a plurality of balanced training subsets by repeating the sampling process, trains a base classifier based on each balanced training subset respectively, calculates the voting weight based on the performance evaluation index of each base classifier on the validation set, obtains the weighted voting result based on the voting weight, and finally classifies unknown network traffic samples.

[0015] A computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the network intrusion detection resampling method based on sensitivity as described above when executing the computer program.

[0016] A computer readable storage medium stores a computer program, and the computer program implements the steps of the network intrusion detection resampling method based on sensitivity as described above when executed by a processor.

[0017] The application has the beneficial effects that: the application measures the importance of samples by calculating the sensitivity of samples, and the sensitivity calculation only needs to simply estimate the prior and class conditional probability density, which does not significantly increase the complexity compared with other random sampling methods, which is extremely important for large-scale network traffic data scenarios.

[0018] Random undersampling ignores the importance of boundary samples for decision boundary learning, and sensitivity weighted sampling can guide the sampling of boundary samples by the model, thereby avoiding the loss of important information.

[0019] The interpolation-based oversampling method is prone to introduce new noise in the presence of noise, and sensitivity weighted sampling can reduce the probability of noise sampling, thereby avoiding the introduction of new noise. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0021] Figure 1 The flow chart of the network intrusion detection resampling method based on sensitivity in the embodiment 1.

[0022] Figure 2 The module flow chart of the network intrusion detection resampling system based on sensitivity in the embodiment 3. DETAILED DESCRIPTION

[0023] In order to make the above-mentioned objects, features and advantages of the present application more apparent and understandable, the specific embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0024] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced in other manners different from those described herein without departing from the scope of the present application, and those skilled in the art can make similar generalizations without departing from the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0025] Embodiment 1, refer to Figure 1 The embodiment 1 provides a network intrusion detection resampling method based on sensitivity, which comprises the following steps. Figure 1 As shown in the following figure:

[0026] S1, obtaining network data containing normal traffic and attack traffic, performing feature extraction on the network data, and dividing the network data into normal traffic samples and attack traffic samples according to the extracted features.

[0027] S2, calculating the prior probability and the class conditional probability of each sample respectively, obtaining the posterior probability, and calculating the sensitivity weight of each sample under the condition of class imbalance based on the Bayesian method.

[0028] S3, for the normal traffic samples, performing undersampling with replacement according to the sensitivity weight, and retaining the samples with sensitivity higher than the threshold.

[0029] S4, for the attack traffic samples, performing oversampling with replacement according to the sensitivity weight, and generating new synthetic samples through k-nearest neighbor interpolation method.

[0030] S5, generating several balanced training subsets through the resampling process, and training the base classifier based on each balanced training subset respectively.

[0031] S6, based on the performance evaluation index of each base classifier on the validation set, the voting weight is calculated, and the weighted voting result is obtained based on the voting weight, and the unknown network traffic sample is finally classified.

[0032] It should be noted that the existing random undersampling has the problem of information loss. The class boundary samples play an important role in learning the decision boundary, but the random undersampling does not make a distinction, and important samples may be deleted, which is easy to cause information loss. The interpolation-based oversampling method is easy to introduce noise, and noise is inevitable in network traffic data. Normal samples may be labeled as attack samples, and attack samples may be mislabeled as normal samples. The interpolation-based oversampling method interpolates between minority class samples to obtain new samples. If one of the samples is noise, new noise will inevitably be introduced, making it more difficult to learn the classification boundary.

[0033] Therefore, in view of the above problems, through the steps of S1-S6, different sampling weights are given to different samples by calculating the sensitivity of the samples to class imbalance. The higher the sensitivity of the sample, the more likely it is a boundary sample, and the lower the sensitivity of the sample, the more likely it is a safe sample or a noise sample away from the boundary. According to this characteristic, the higher the sensitivity of the sample, the greater the sampling weight. Based on this, undersampling of the majority class and oversampling of the minority class can effectively avoid information loss and noise introduction. At the same time, the learning ability of the model is further improved through ensemble learning, so as to effectively cope with the class imbalance problem in network intrusion detection.

[0034] Embodiment 2 is a second embodiment of the present application, which is different from the first embodiment: a network intrusion detection resampling method based on sensitivity further comprises, in the embodiment of the present application, the network data containing normal traffic and attack traffic is obtained in step S1, the network data is feature extracted, and the network data is divided into normal traffic samples and attack traffic samples according to the extracted features.

[0035] Before feature extraction of the network data, the collected network data is cleaned, including operations such as de-duplication, missing value filling, and outlier processing, to improve data quality and consistency.

[0036] The extracted features of the network data include but are not limited to:

[0037] Space-time features: such as IP packet arrival interval, connection duration, payload entropy value, TCP window size change, connection frequency, and packet quantity per second.

[0038] Protocol features: such as protocol type (TCP / UDP / ICMP) used, port distribution, and flag information (such as SYN / ACK / RST).

[0039] Statistical features: such as mean, standard deviation, maximum, minimum, quantile, median, kurtosis, and skewness of flow size.

[0040] Session-level features: such as the number of packets in each session, average packet length, flow direction (inbound / outbound) ratio, etc.

[0041] In an optional embodiment, the extracted features need to be pre-processed in step S1, which can eliminate the dimensional differences between features, making all input features comparable in numerical scale, thereby avoiding excessive dependence on features with a larger value range during model training, including the following steps S111-S114:

[0042] S111: Calculate the maximum and minimum values of each feature. Traverse the entire training data set, and for each feature, record the maximum value (denoted as "max") and the minimum value (denoted as "min").

[0043] S112: Standardize each feature value. For each feature value in each sample data, perform the following operations:

[0044] Subtract the "min" value from the feature value, then divide by the difference between the "max" and "min" values. In simple terms: current value minus min, then divided by the difference between max and min

[0045] If the maximum value of a feature is 200 and the minimum value is 100, then a sample with an original value of 150 will be standardized to (150-100) / (200-100) = 0.5.

[0046] A sample with an original value of 200 will be standardized to 1.0.

[0047] A sample with an original value of 100 will be standardized to 0.0.

[0048] After processing, the values of all samples for this feature will be within the range of 0 to 1.

[0049] S113: Handle special cases. If the maximum and minimum values of a feature are the same (indicating that the feature takes the same value in all samples), assign the feature a value of 0 or delete the feature in subsequent steps, as it has no distinguishing ability.

[0050] S114: Save the maximum and minimum values. Save the maximum and minimum values of each feature for subsequent standardization operations on test data or actual application data to maintain consistency in processing.

[0051] In another optional embodiment, the preprocessing can also reduce the computational complexity of model training, reduce redundancy and noise, improve the training efficiency and generalization ability of the classifier by reducing the number of features while retaining the main information of the data, including the following steps S121-S124:

[0052] S121: Dimensionality reduction processing is performed using a principal component analysis method (PCA, Principal Component Analysis), the PCA algorithm is used to analyze the feature distribution of the normalized samples, and the "principal component direction" that best represents the sample difference is identified. Each principal component represents a combination of original features in a specific direction, and the earlier the principal component, the more information it can explain.

[0053] S122: The information retention threshold is determined, and the cumulative 95% information amount (also referred to as the cumulative variance contribution rate) is generally selected. This means that the first several principal components are selected, so that the total amount of information represented by these principal components accounts for 95% of the original data, so as to ensure that the key feature content is not lost as much as possible.

[0054] S123: A new feature space is constructed, and the data of each sample is projected onto the new coordinate axes formed by the selected principal components, thereby obtaining "dimensionality-reduced sample feature vectors". These new feature dimensions are fewer, but still retain the most important discriminant information in the original data.

[0055] S124: Save the conversion model parameters, save the transformation matrix (i.e. the principal component direction) generated in the dimensionality reduction process, so as to perform consistent dimensionality reduction processing on the test data or actual application data in the future.

[0056] It should be noted that the cleaning process reduces invalid values and outliers to reduce the interference of subsequent models and improve the overall training data reliability; the multi-type and multi-dimensional feature extraction method can describe network connection behavior from different angles, facilitating more comprehensive identification of attack features by the model; standardization and dimensionality reduction effectively reduce the convergence time and resource consumption during model training.

[0057] In the embodiments of the present application, the prior probability and the class conditional probability of each sample are calculated in step S2, the posterior probability is obtained, and the sensitivity weight of each sample under the condition of class imbalance is calculated based on the Bayesian method.

[0058] The number of normal flow and attack flow samples is counted from the preprocessed feature data, and the prior probability under the unbalanced state is calculated, including

[0059] The normal flow prior probability is the ratio of the number of normal flow samples to the total number of samples P(y j =c - ).

[0060] The attack flow prior probability is the proportion of the number of attack flow samples to the total number of samples P(y j = c + ).

[0061] Wherein, P represents the prior probability, y j represents the category to which the jth sample belongs, c + represents the attack flow sample, and c - represents the normal flow sample.

[0062] The mean and variance of each feature are calculated for the samples of each category, and then the class conditional probability of the samples of each category is estimated, which is expressed as,

[0063]

[0064] Wherein, p represents the class conditional probability, represents the ith feature of the jth sample, μ ij represents the mean of , and represents the variance of .

[0065] The class conditional probability is simplified as p(x j |y j = c + ), that is, the class conditional probability of the jth sample under the attack flow sample; p(x j |y j = c - ) is the class conditional probability of the jth sample under the normal flow sample.

[0066] The posterior probability is obtained by multiplying the prior probability and the class conditional probability;

[0067] Γ + = p(x j |y j = c + )P(y j = c + ) is the posterior probability density of the attack flow sample under the original unbalanced condition.

[0068] Γ - = p(x j |y j = c - )P(y j = c - ) is the posterior probability density of the normal flow sample under the original unbalanced condition.

[0069] Γ' - = p(x j |yj = c - )P(y j = c + ) is the calibrated posterior probability density.

[0070] Based on the Bayesian sensitivity evaluation, the obtained posterior probability is combined to obtain the sensitivity score S(x j ,y j ) of each sample, which is expressed by the formula,

[0071]

[0072] The sensitivity score is normalized to 0 to 1 to obtain the sensitivity weight, and low sensitivity samples are assigned low sampling priority, and high sensitivity samples are assigned high sampling priority. For normal flow samples, the weight is calculated as follows:

[0073]

[0074] And for attack flow samples, the weight is calculated as follows:

[0075]

[0076] Wherein, W - (x j ) represents the weight of the jth normal flow sample, W + (x j ) represents the weight of the jth attack flow sample, N - represents the number of normal flow samples, and N + represents the number of attack flow samples.

[0077] It should be noted that the model pays more attention to the data distribution in the “fuzzy” area, enhances the recognition ability of the classifier to unknown attack samples, and improves the generalization performance of the model. Through the normalization and weight generation strategy, different data sets and intrusion detection tasks can be flexibly adapted, and the controllability and adaptability are strong. Based on the Bayesian method, without introducing complex models, it is convenient for engineering landing and large-scale deployment, and the mathematical modeling is simple and can be realized.

[0078] In the embodiments of the present application, for normal flow samples in step S3, the sensitivity weight is used for replacement undersampling, and samples with sensitivity higher than the threshold are retained.

[0079] Based on the sensitivity weight of the majority class samples generated in step S2, the majority class samples are weighted undersampled with replacement.

[0080] The samples with high weight (i.e., high sensitivity, close to the classification boundary) are retained with a higher probability, and the samples with low weight (far from the boundary or potential noise) are selected with a lower probability or even not selected; the determination criterion of high and low weight is determined by a preset threshold.

[0081] Finally, a fixed number of high-sensitivity samples are selected from the normal traffic samples to constitute the "normal class" part in the balanced subset.

[0082] It should be noted that the majority class samples with high sensitivity scores are preferentially retained, enhancing the model's learning ability for the key decision boundary and avoiding the loss of boundary samples; the samples with low sensitivity are given low weight in under / oversampling, reducing the addition of invalid or misleading samples and reducing the noise sampling probability.

[0083] In the embodiment of the application, the attack traffic samples in step S4 are oversampled with replacement according to the sensitivity weight, and new synthetic samples are generated by k-nearest neighbor interpolation.

[0084] For the sampled attack traffic samples, a neighbor is randomly selected from the k nearest neighbors, and a new attack traffic sample is generated by linear interpolation between the sample and the neighbor at a random ratio, and the interpolation ratio is uniformly distributed in [0, 1].

[0085] Specifically, the new sample x' is generated as follows:

[0086] x' = x z +r*(x k -x z )

[0087] where x z represents the selected attack traffic sample, x k ∈KNN(x z ) represents the k-nearest neighbor of x z , and r represents a random number uniformly distributed in U(0, 0.5).

[0088] It should be noted that by sensitivity-weighted sampling, the key samples that have the most influence on the classifier training can be focused on, and the boundary sample retention rate can be improved; the new samples generated by interpolation in the boundary region help to maintain the authenticity of the data distribution and reduce the misclassification caused by noise interference; the synthetic new samples improve the density of the minority class data, which helps the model to learn the attack class features.

[0089] In the embodiment of the application, the sampling process is repeated in step S5 to generate several balanced training subsets, and a base classifier is trained based on each balanced training subset.

[0090] The under-sampling result of the normal traffic sample and the over-sampling result of the attack traffic sample are combined to form a balanced training subset, and several rounds of repetition are performed to generate several balanced training subsets

[0091] Each subset contains majority class samples obtained by sensitivity weighted sampling and minority class samples obtained by over-sampling, and a same-type or different-type classification model is selected as a base classifier for each balanced training subset. The base classifier can be a decision tree (such as CART), a multi-layer perceptron (MLP), a convolutional neural network (CNN), a support vector machine (SVM), or other supervised learning models that can be used for binary classification. A single model can be selected for multiple training, or multiple models can be selected to form a heterogeneous ensemble.

[0092] Each balanced training subset is trained separately to obtain T independent base classifiers, ensuring that each model can learn features and decision boundaries under different sample configurations.

[0093] The independently divided validation set is used to evaluate the performance of each base classifier, and evaluation indicators are calculated for subsequent voting weight allocation.

[0094] The T trained base classifiers and their performance records on the validation set are saved as the basis for the final model weighted voting.

[0095] It should be noted that each classifier learns data from different perspectives, effectively avoiding the bias of a single model; multiple base models are trained to enable the system to capture more diverse attack feature representations; and each model is trained on balanced data to avoid the phenomenon of "biasing towards the majority class".

[0096] In the embodiments of the present application, the voting weight is calculated based on the performance evaluation indicators of each base classifier on the validation set in step S6, and the weighted voting result is obtained based on the voting weight to perform final classification on unknown network traffic samples.

[0097] In an optional embodiment, the performance evaluation indicator can be the F1 score, including the following steps S611-S613:

[0098] S611: A part of samples not involved in training is divided from the original sample set to form a validation set; each base classifier is used to predict the validation set samples, and the classification results are counted; for each base classifier, the F1 score on the validation set is calculated as the model performance indicator.

[0099] S612: Calculate the voting weight, normalize the F1 scores of all T classifiers so that their sum is 1; the normalized F1 score of each classifier is the voting weight, and the higher the performance, the greater the weight. The weight formula w of voting is t is represented as,

[0100]

[0101] F1score of the t-th base classifier. t F1score of the t-th base classifier.

[0102] S613: The unknown network traffic sample to be classified is input to all T base classifiers in a sample prediction and voting integration; each classifier outputs a class label (normal or attack);

[0103] The weighted voting is performed as follows: if a certain classifier predicts an "attack class", it votes for the attack class with its weight value; if it predicts a "normal class", it votes for the normal class with its weight value; the weighted scores of all classifiers for the two classes are aggregated to obtain a weighted voting result; the class with a higher score is taken as the final prediction label of the unknown sample.

[0104] In another optional embodiment, the performance evaluation indicator can be AUC (Area Under Curve), including the following steps S621-S625:

[0105] S621: A base classifier is trained for each training subset respectively; the classifiers used can be the same model (such as a lightweight neural network) or different types (such as random forest, support vector machine, etc.); each base classifier outputs a confidence probability of binary classification prediction (i.e. the prediction probability value of the sample belonging to the "attack class", ranging from 0 to 1).

[0106] S622: Divide the validation set and evaluate the model performance; a part of data is divided from the original sample as a validation set (without intersection with the training subset); each base classifier is used to predict the validation set sample, and the prediction probability is recorded; based on the prediction probability, an ROC (Receiver Operating Characteristic) curve is drawn, and the AUC value of each classifier is calculated; the obtained AUC value is taken as the performance indicator of the classifier, which is used for subsequent weighting.

[0107] S623: Calculate the classifier voting weight; normalize the AUC value of each classifier (for example, the sum of all AUC values is set to 1); the normalized AUC value is the weight of the classifier in soft voting; the greater the weight, the better the performance of the classifier in the entire classification task, and the greater the influence of its result on the final judgment.

[0108] S624: Unknown sample prediction and probability fusion; the unknown network traffic sample x to be predicted is input to all T base classifiers; each classifier outputs the probability of the sample belonging to the "attack class" (such as 0.82 indicating an attack probability of 82%); the probability output by each classifier is multiplied by its corresponding AUC weight.

[0109] All weighted probabilities are added together as the final weighted voting result of the sample, and compared with a threshold value (such as 0.5): if the weighted score ≥ threshold value, it is predicted as "attack class", otherwise it is predicted as "normal class".

[0110] S625: output the final prediction result, and take the predicted class as the output result for system calling, which can trigger subsequent security protection actions such as alarm, log recording, and blocking connection.

[0111] The calculation formula of the weighted voting result is expressed as,

[0112]

[0113] wherein, is the maximum y that makes the value of the expression, T is the total number of base classifiers, C t (x) is the output of the t-th base classifier, w t is the voting weight of the t-th base classifier, δ(·) is an indicator function that takes the value of 1 when the condition in the parentheses is true, and 0 otherwise, y is the current voting candidate class label including normal traffic class and attack traffic class, and is used to traverse all possible classification results.

[0114] Embodiment 3, refer to Figure 2 , which is different from the first two embodiments: a network intrusion detection resampling system based on sensitivity, characterized by: comprising a data preprocessing module, a sensitivity calculation module and an integrated sampling and classification module; the data preprocessing module obtains network data containing normal traffic and attack traffic, extracts features from the network data, and pre-processes the extracted features, and divides them into normal traffic samples and attack traffic samples; the sensitivity calculation module calculates the prior probability and class conditional probability of the two categories of samples respectively, and then obtains the posterior probability, and calculates the sensitivity weight of each sample under the condition of class imbalance based on the Bayesian method; the integrated sampling and classification module performs undersampling with replacement on the normal traffic samples according to the sensitivity weight, and retains the samples with sensitivity higher than the threshold value; for attack traffic samples, oversampling with replacement is performed according to the sensitivity weight, and new synthetic samples are generated through k-nearest neighbor interpolation; repeat the sampling process to generate several balanced training subsets, and train base classifiers based on each subset; calculate the voting weight based on the performance evaluation index of each base classifier on the validation set, and obtain the weighted voting result based on the voting weight to finally classify the unknown network traffic sample.

[0115] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0116] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logic functions, which can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus or device, such as a computer-based system, a system including a processor or other system that can fetch the instructions from the instruction execution system, apparatus or device and execute the instructions, or in conjunction with these instructions execution systems, apparatus or devices. For the purpose of this specification, the "computer-readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by or in connection with an instruction execution system, apparatus or device, or in conjunction with these instruction execution systems, apparatus or devices.

[0117] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by editing, interpreting or otherwise processing, if necessary, in other suitable ways, to be electronically obtained and then stored in the computer memory.

[0118] It should be understood that portions of the present application can be implemented in hardware, software, firmware, or combinations thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, implementation can be with a combination of any of the following technologies, which are all well known in the art: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0119] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. A sensitivity-based network intrusion detection resampling method, characterized in that: include, Obtain network data containing normal traffic and attack traffic, extract features from the network data, and divide the network data into normal traffic samples and attack traffic samples based on the extracted features. The prior probability and class-conditional probability of each sample are calculated separately to obtain the posterior probability. The sensitivity weight of each sample under class imbalance is calculated based on the Bayesian method. For normal traffic samples, undersampling with replacement is performed according to sensitivity weights, and samples with sensitivity higher than the threshold are retained. For attack traffic samples, oversampling with replacement is performed according to sensitivity weights, and new synthetic samples are generated by k-nearest neighbor interpolation. The repeated sampling process generates several balanced training subsets, and a base classifier is trained based on each balanced training subset. Voting weights are calculated based on the performance evaluation metrics of each base classifier on the validation set. Weighted voting results are obtained based on these voting weights, and the unknown network traffic samples are finally classified.

2. The sensitivity-based network intrusion detection resampling method as described in claim 1, characterized in that: Before performing feature extraction on the network data, the method further includes: Perform deduplication, missing value imputation, and outlier handling operations on the collected network data; The features extracted from the network data include, but are not limited to, spatiotemporal features, protocol features, statistical features, and session-level features.

3. The sensitivity-based network intrusion detection resampling method as described in claim 2, characterized in that: The calculation of the prior probability and class-conditional probability of samples from the two categories includes: The number of normal traffic and attack traffic samples is counted from the preprocessed feature data, and the prior probability under the imbalanced state is calculated. The prior probabilities include: the prior probability of normal traffic and the prior probability of attack traffic; the prior probability of normal traffic is the proportion of normal traffic samples to the total number of samples; the prior probability of attack traffic is the proportion of attack traffic samples to the total number of samples. The mean and variance of each feature are calculated for each category of samples, and then the class conditional probability of each category of samples is estimated.

4. The sensitivity-based network intrusion detection resampling method as described in claim 3, characterized in that: The posterior probability is obtained by multiplying the prior probability by the class conditional probability; The calculation of the sensitivity weight for each sample under class imbalance conditions based on the Bayesian method includes: The obtained posterior probabilities are combined and subtracted to obtain the sensitivity score for each sample. The sensitivity scores are then normalized to a range of 0 to 1 to obtain the sensitivity weight.

5. The sensitivity-based network intrusion detection resampling method as described in claim 4, characterized in that: The method of generating new synthetic samples by k-nearest neighbor interpolation includes: for the sampled attack traffic sample, randomly selecting a neighbor from the k nearest neighbors, and performing linear interpolation between the sample and the neighbor at a random ratio to generate a new attack traffic sample, wherein the interpolation ratio follows a uniform distribution of [0, 1].

6. The sensitivity-based network intrusion detection resampling method as described in claim 5, characterized in that: The repeated sampling process generates several balanced training subsets by merging the undersampling results of normal traffic samples with the oversampling results of attack traffic samples to form a balanced training subset, repeating this process several times to generate several balanced training subsets.

7. The sensitivity-based network intrusion detection resampling method as described in claim 6, characterized in that: The formula for calculating the weighted voting result is expressed as follows: in, Let y be the value that maximizes the expression, T be the total number of base classifiers, and C be the value that maximizes the expression. t (x) represents the output of the t-th base classifier, w t Let t represent the voting weight of the t-th base classifier, δ(·) represent the indicator function, which takes the value of 1 when the condition in parentheses is true and 0 otherwise, and y represents the current voting candidate category label, which includes normal traffic category and attack traffic category, and is used to traverse all possible classification results.

8. A sensitivity-based network intrusion detection resampling system, employing the sensitivity-based network intrusion detection resampling method as described in any one of claims 1 to 7, characterized in that: It includes a data preprocessing module, a sensitivity calculation module, and an integrated sampling and classification module; The data preprocessing module acquires network data containing normal traffic and attack traffic, extracts features from the network data, and divides the network data into normal traffic samples and attack traffic samples based on the extracted features. The sensitivity calculation module calculates the prior probability and class-conditional probability of each sample to obtain the posterior probability, and calculates the sensitivity weight of each sample under class imbalance conditions based on the Bayesian method. The integrated sampling and classification module performs undersampling with replacement on normal traffic samples according to sensitivity weights, and retains samples with sensitivity higher than the threshold. For attack traffic samples, oversampling with replacement is performed according to sensitivity weights, and new synthetic samples are generated by k-nearest neighbor interpolation. The repeated sampling process generates several balanced training subsets, and base classifiers are trained based on each balanced training subset. Voting weights are calculated based on the performance evaluation metrics of each base classifier on the validation set, and weighted voting results are obtained based on the voting weights to finally classify the unknown network traffic samples.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the sensitivity-based network intrusion detection resampling method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the sensitivity-based network intrusion detection resampling method according to any one of claims 1 to 7.

Images