A Network Fault Detection Method Based on Multi-Boosting Ensemble Learning

Abstract

This invention specifically relates to a network fault detection method based on multi-boost ensemble learning, comprising: inputting network connection records into a trained fault detection model and outputting corresponding network fault prediction values; during training, firstly, a training sample set is obtained; secondly, the training sample set is input into the convolutional neural network of the fault detection model for preliminary learning, resulting in several CNN base classifiers; then, an adaptive boosting algorithm is used to focus on training samples misclassified during the training of the CNN base classifiers, and a corresponding AB model is constructed based on the CNN base classifiers; then, corresponding sub-decision groups are constructed based on the AB model, and the outputs of each sub-decision group are weighted and summed using a multi-boost algorithm to generate corresponding network fault prediction values, thereby optimizing the fault detection model. This invention is well-suited for network fault detection and can guarantee the accuracy and generalization error of network fault detection, thus improving the effectiveness of network fault detection.

Description

Technical Field

[0001] This invention relates to the field of deep learning network technology, and specifically to a network fault detection method based on multi-enhancement ensemble learning. Background Technology

[0002] With the development of informatization and intelligentization, the services running on communication networks are becoming increasingly diversified, the network scale is growing rapidly, and the structure is becoming increasingly complex, leading to a continuous expansion in the probability and impact of network failures. The causes of network failures are constantly emerging. These network failures cause enormous economic losses and social impacts. The efficiency and accuracy of network failure detection directly affect whether the network operates normally and the quality of service; therefore, researching high-performance network fault diagnosis models to ensure the normal operation of the network is crucial.

[0003] As network connection record data exhibits low data value density, traditional network fault diagnosis models have emerged, such as expert system diagnosis, signal processing diagnosis, statistical analysis diagnosis, and state estimation diagnosis. However, these traditional methods struggle to address the hierarchical, diffuse, and uncertain characteristics of network faults, making them inadequate for diagnosing faults in today's complex networks within the context of big data. Traditional network monitoring systems and manual fault diagnosis methods are insufficient to mitigate the impact of irrelevant or redundant features. Therefore, it is necessary to explore new fault diagnosis technologies capable of handling today's complex network systems.

[0004] The applicant discovered that machine learning, with its strong learning ability, is frequently used in network fault diagnosis, such as backpropagation (BP), backpropagation (RBF), and CNN algorithms. However, network faults are characterized by high feature dimensionality, low data value density, and large data volume, making existing BP and RBF algorithms unsuitable for high-precision identification. Furthermore, a single base learner struggles to guarantee accuracy and generalization error in fault diagnosis, similarly resulting in low accuracy in network fault detection. Therefore, designing a method suitable for network fault detection that can guarantee both detection accuracy and generalization error is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is: how to provide a network fault detection method based on multiple boosting ensemble learning, which can be well applied to network fault detection and can guarantee the accuracy and generalization error of network fault detection, thereby improving the effect of network fault detection.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A network fault detection method based on multi-boost ensemble learning includes:

[0008] S1: Obtain the network connection records to be detected;

[0009] S2: Input the network connection records into the trained fault detection model and output the corresponding network fault prediction values;

[0010] The fault detection model consists of T sub-decision groups, each containing I... t There are K AB models, and each AB model contains K CNN base classifiers;

[0011] During training, a training sample set is first constructed by acquiring several network connection records labeled with network faults. Next, the training sample set is input into the convolutional neural network of the fault detection model for initial learning, resulting in several CNN base classifiers. Then, an adaptive boosting algorithm is used to focus on training samples misclassified during the training of the CNN base classifiers, and a corresponding A / B model is constructed based on the CNN base classifiers. Following this, corresponding sub-decision groups are constructed based on the A / B model, and the outputs of T sub-decision groups are weighted and summed using a multiple boosting algorithm to generate corresponding network fault prediction values. Finally, the training loss is calculated based on the network fault prediction values ​​and the corresponding network fault labels to optimize the fault detection model.

[0012] S3: Use the network fault prediction value output by the fault detection model as the network fault detection result of the network connection record.

[0013] Preferably, in step S2, the convolutional neural network of the fault detection model is a LetNet5 model, which includes an input layer, a convolutional layer, a sampling layer, a fully connected layer, and an output layer.

[0014] Preferably, the fault detection model is trained through the following steps:

[0015] S201: Obtain several network connection records carrying network fault labels to construct a training sample set; then preprocess the training sample set to obtain a preprocessed dataset, and set the weights of each training sample in the preprocessed dataset;

[0016] S202: Initialize flag variable I t Let t = 1, and set the index of t.

[0017] S203: Initialize the number of AB models in the sub-decision group q = 1;

[0018] S204: Initialize the number of CNN base classifiers in the AB model, k = 1;

[0019] S205: Based on the weights of each training sample, perform N samplings with replacement to obtain the base classifier dataset of the k-th CNN base classifier in the q-th AB model of the t-th sub-decision group. It is then fed into a convolutional neural network, which outputs the corresponding CNN base classifier.

[0020] S206: If k < K, then k = k + 1, and return to step S205; otherwise, construct the q-th AB model AB(t) in the t-th sub-decision group based on the obtained K CNN base classifiers and the following formula. t D q ), and calculate the AB model AB( t D q Output error t e q Therefore, based on the output error t e q Adjust the weights of each training sample in the preprocessed dataset; simultaneously, obtain the corresponding AB model dataset vector.

[0021]

[0022] In the formula: AB( t D q ) represents the q-th AB model in the t-th sub-decision group; This represents the k-th CNN base classifier in the q-th AB model of the t-th sub-decision group; This represents the output weight of the k-th CNN base classifier in the q-th AB model of the t-th sub-decision group;

[0023] in,

[0024] In the formula: This represents the ratio of the number of samples that were not correctly classified in the k-th CNN base classifier of the q-th AB model in the t-th sub-decision group to the total number of samples.

[0025] S207: If q < I t If q = q + 1, then return to step S204; otherwise, based on the obtained I t An AB model is used to construct the t-th sub-decision group, and the output of the t-th sub-decision group is calculated using the following formula. and weight α t Simultaneously, the corresponding sub-decision group dataset vector is obtained.

[0026] α t =log[(1- t e q ) / t e q ];

[0027] In the formula: α t This represents the weight of the t-th sub-decision group; t e q This represents the output error of the q-th AB model in the t-th sub-decision group;

[0028] S208: If t < T, then t = t + 1, and return to step S203 to enter the next sub-decision group; otherwise, classify the training samples through T sub-decision groups respectively, and calculate the network fault category with the largest weight as the network fault prediction value using the following formula.

[0029]

[0030] t β q = t e q / (1- t e q );

[0031] In the formula: MB * ( t D q Y represents the T sub-decision groups, each classifying the training samples and using the network fault category with the highest weight as the predicted network fault value for that network connection record; t Represents the AB model dataset vector The training samples correspond to a label vector composed of network fault labels, which is the true value of the network fault; α t This represents the weight of the t-th sub-decision group; t D q This represents the input dataset vector to the q-th AB model in the t-th sub-decision group, which is the dataset of the K CNN base classifiers in the AB model; t β q This represents the output weight of the q-th AB model in the t-th sub-decision group on the fault detection model.

[0032] Preferably, in step S201, the training sample set is preprocessed through the following steps:

[0033] S2011: Training sample set D = {(X1,Y1),…,(X i ,Y i ),…,(X N ,Y N The set X contains N training samples, i = 1, 2, ..., N. i ={x ij{j = 1, 2, ..., M} represents the i-th training sample, and Y = {Y} i ,i=1,2,…,N} represents the label vector of the training sample, and M represents the original feature attribute dimension of the training sample;

[0034] S2012: For the feature attribute vector X = {X1,…,X...} j ,…,X M}, j = 1, 2, ..., M are processed into numerical values;

[0035] S2013: Normalize the numerically processed feature attribute vector X to obtain the normalized feature attribute vector. This results in a preprocessed dataset after numerical and normalization processing.

[0036] Preferably, in step S2013, the feature attribute vector is normalized using the following formula:

[0037]

[0038] in,

[0039] In the formula: X represents the feature attribute vector of the j-th dimension after normalization; j x represents the j-th feature attribute vector after numerical processing; ij A represents the feature attribute vector of the j-th dimension of the i-th training sample; j S j X jmin X jmax Let X represent the j-th feature attribute respectively. j The mean, variance, minimum and maximum values.

[0040] Preferably, in step S205, the flag variable I is calculated using the following formula. t : if I t =t, then the dataset will be preprocessed. The weights of each training sample are set to Otherwise, based on the output error t e q Adjust the weights of each training sample; then perform sampling with replacement based on the weights of each training sample in the preprocessed dataset.

[0041]

[0042] Preferably, in step S206, the output error of the AB model is calculated using the following formula. t e q :

[0043]

[0044] In the formula: t e q The weight(X) represents the output error of the q-th AB model in the t-th sub-decision group; t ) represents the training sample X t The weights; AB(X) t ) represents the output of the AB model.

[0045] Preferably, in step S206, the weights of each training sample in the preprocessed dataset are adjusted in the following manner:

[0046] 1) If t e q If the value is greater than 0.5, the corresponding AB model is discarded, and the weights of each training sample are calculated based on the continuous Poisson distribution.

[0047] 2) If t e q =0, then set the output weight. t β q =10 -10 And calculate the weights of each training sample based on a continuous Poisson distribution;

[0048] 3) If 0 < t e q If the value is less than 0.5, then set the output weight. t β q = t e q / (1- t e q ), and for the AB model dataset vector t D q For each training sample, the weight of the misclassified sample is divided by 2. t e q The weights of the positive samples are divided by 2(1- t e q ), and the minimum weight is 10. -8 .

[0049] Preferably, the continuous Poisson distribution is represented by the following formula:

[0050]

[0051] In the formula: p represents the probability value, which is the weight of the training sample; Random(1,2,…,999) means that an integer is randomly generated from 1 to 999.

[0052] Preferably, after training the fault detection model, the performance of the fault detection model is evaluated by the accuracy, F1 score, detection rate (TPR), and false alarm rate (FPR) of various training samples.

[0053] 1) Calculate the accuracy of various training samples using the following formula:

[0054]

[0055] In the formula: Acc test N represents the precision of various training samples; a This represents the number of samples that were correctly classified; N represents the total number of samples.

[0056] 2) Calculate the F1 value using the following formula:

[0057]

[0058] In the formula: F1 represents the F1 score; P represents the precision; R represents the recall.

[0059] 3) Calculate the detection rate (TPR) and false alarm rate (FPR) using the following formulas:

[0060]

[0061] In the formula: TP represents the number of samples that are actually positive and the fault diagnosis model predicts them to be positive; FN represents the number of samples that are actually positive and the fault diagnosis model predicts them to be negative; FP represents the number of samples that are actually negative and the fault diagnosis model predicts them to be positive; TN represents the number of samples that are actually negative and the fault diagnosis model predicts them to be negative.

[0062] The network fault detection method based on multiple boosting ensemble learning in this invention has the following beneficial effects:

[0063] This invention employs a convolutional neural network (CNN) to perform initial learning by inputting network connection records. This leverages the advantages of CNNs, such as kernel sharing, scale invariance, and powerful learning capabilities, to adapt to the characteristics of high-dimensional network fault features, low data value density, and large data volume. This makes it well-suited for network fault detection and improves its accuracy. Simultaneously, this invention uses an adaptive boosting algorithm to address misclassified training samples during the training of the CNN base classifier. This leverages the strong bias reduction ability of the adaptive boosting algorithm to reduce network fault classification errors, further improving network fault detection accuracy. Furthermore, this invention uses a multi-boosting algorithm to weightedly sum the outputs of each sub-decision group to generate corresponding network fault prediction values. This leverages the multi-boosting algorithm's ability to effectively reduce variance, improving classification accuracy and addressing the low fault data value density issue, thereby reducing the generalization error in network fault detection. Therefore, this invention is well-suited for network fault detection, ensuring both accuracy and low generalization error, thus improving the effectiveness of network fault detection. Attached Figure Description

[0064] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0065] Figure 1 This is a logical block diagram of a network fault detection method based on multiple boosting ensemble learning.

[0066] Figure 2 This is a diagram of the network structure of a convolutional neural network;

[0067] Figure 3 This is a diagram of the network structure during the training of the fault detection model.

[0068] Figure 4 This is a network structure diagram of the sub-decision groups in the fault detection model;

[0069] Figure 5 A box diagram of the characteristic attributes of network faults;

[0070] Figure 6 This is a schematic diagram comparing the fault detection model of the present invention with other existing models. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0072] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figures, or the orientation or positional relationship commonly used when the product is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. In addition, the terms "horizontal," "vertical," etc., do not indicate that the component is required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0073] The following detailed explanation illustrates the specific implementation methods:

[0074] Example:

[0075] This embodiment discloses a network fault detection method based on multiple boosting ensemble learning.

[0076] like Figure 1As shown, the network fault detection method based on multi-enhancement ensemble learning includes:

[0077] S1: Obtain the network connection records to be detected;

[0078] S2: Input the network connection records into the trained fault detection model and output the corresponding network fault prediction values;

[0079] The fault detection model consists of T sub-decision groups, each containing I... t There are K AB models, and each AB model contains K CNN base classifiers;

[0080] During training, a training sample set is first constructed by acquiring several network connection records labeled with network faults. Next, the training sample set is input into the convolutional neural network of the fault detection model for initial learning, resulting in several CNN base classifiers. Then, an adaptive boosting algorithm (AdaBoost, AB) is used to monitor training samples misclassified during the training of the CNN base classifiers, and a corresponding AB model is constructed based on the CNN base classifiers. Following this, corresponding sub-decision groups are constructed based on the AB model, and the outputs of T sub-decision groups are weighted and summed using a multi-boost algorithm (MultiBoost, MB) to generate corresponding network fault prediction values. Finally, the training loss is calculated based on the network fault prediction values ​​and the corresponding network fault labels to optimize the fault detection model.

[0081] In this embodiment, the network fault labels carried by the training samples serve two purposes: first, to determine whether the input sample is correctly classified or misclassified. If it is misclassified, its weight will be increased in the next iteration of the algorithm; otherwise, its weight will be decreased, thereby optimizing the fault detection model. Second, the training loss (such as cross-entropy loss) is calculated using the network fault prediction value and the corresponding network fault label, and then the fault detection model is optimized by combining the training loss with existing mature methods. In the test samples, the labels are used to calculate the performance metrics of the fault detection model, such as accuracy, F1 score, detection rate (TPR), and false positive rate (FPR).

[0082] S3: Use the network fault prediction value output by the fault detection model as the network fault detection result of the network connection record.

[0083] In this embodiment, a network connection record sequence for a certain period of time can be input into the fault detection model. The fault detection model can then determine whether a network fault has occurred and detect the corresponding network fault type (i.e., network fault prediction value), such as normal records, denial-of-service (DOS) types, monitoring and other probing activities (Probing), unauthorized access from remote machines (R2L), and unauthorized access by ordinary users to local superuser privileges (U2R).

[0084] This invention employs a convolutional neural network (CNN) to perform initial learning by inputting network connection records. This leverages the advantages of CNNs, such as kernel sharing, scale invariance, and powerful learning capabilities, to adapt to the characteristics of high-dimensional network fault features, low data value density, and large data volume. This makes it well-suited for network fault detection and improves its accuracy. Simultaneously, this invention uses an adaptive boosting algorithm to address misclassified training samples during the training of the CNN base classifier. This leverages the strong bias reduction ability of the adaptive boosting algorithm to reduce network fault classification errors, further improving network fault detection accuracy. Furthermore, this invention uses a multi-boosting algorithm to weightedly sum the outputs of each sub-decision group to generate corresponding network fault prediction values. This leverages the multi-boosting algorithm's ability to effectively reduce variance, improving classification accuracy and addressing the low fault data value density issue, thereby reducing the generalization error in network fault detection. Therefore, this invention is well-suited for network fault detection, ensuring both accuracy and low generalization error, thus improving the effectiveness of network fault detection.

[0085] In practical implementation, the Convolutional Neural Network (CNN) algorithm possesses certain rotation and translation invariance characteristics. Its convolution and pooling operations both utilize shared convolution kernels, which reduces the dimensionality of the input data. Considering that the feature attribute dimension of network record connections is 41, this invention employs CNN as the base classifier model. Furthermore, to reduce the algorithm's memory usage and computational requirements, this invention uses the mature and widely used LetNet5 model and employs an improved method of post-padding convolution to avoid information loss due to the reduction in the dimensionality of the input data during convolution.

[0086] like Figure 2 As shown, the LetNet5 model includes an input layer, a convolutional layer, a sampling layer, a fully connected layer, and an output layer.

[0087] The expression function of the output layer is represented by the following formula:

[0088] y = soft max(ωP + b);

[0089] In the formula: ω represents the weight matrix of the fully connected layer; P represents the pooling feature matrix; b represents the bias term.

[0090] K balanced datasets are sampled from the preprocessed dataset as training samples and fed into the CNN model to train the corresponding CNN base classification model. Specifically, from the preprocessed dataset... Mid-sampling is performed to obtain K balanced datasets, and these datasets constitute a dataset vector. in These data are used as the training sample set and input into the LeNet5 CNN model until all K sets of data have been trained. The output layer value Z is then obtained by combining the pooling feature matrix P, the fully connected layer weight matrix ω, and the bias term b. k =soft max(ω) k P k +b k This leads to K corresponding CNN base classification models {Z1, Z2, ..., Z...}. K}. The data within... From preprocessed dataset Total number of base classification samples The number of groups is K≤N.

[0091] This invention uses network connection records as input for preliminary learning in a convolutional neural network. This allows the advantages of convolutional neural networks, such as kernel sharing, scale invariance, and powerful learning capabilities, to be adapted to the characteristics of high dimensionality of network fault features, low data value density, and large data volume. As a result, it can be well applied to network fault detection and improve the accuracy of network fault detection.

[0092] In its implementation, the MultiBoost (MB) ensemble algorithm is a serial ensemble algorithm consisting of several sub-decision groups. The MultiBoost ensemble algorithm defines a flag variable I to determine the iteration number of a sub-decision group. t (t=1,2,…,∞) represents the sub-decision group t containing I. t K base classifiers are constructed using the A / B algorithm. The obtained CNN base classification models are then input into the MB model to obtain K CNN base classification models. The input is fed into the q-th AB algorithm model in the t-th sub-decision group to obtain the q-th AB algorithm model. This model is then continuously trained using a CNN model and the corresponding AB model until all T sub-decision groups are trained. Finally, all sub-decision groups are summed according to their corresponding weights to obtain the MB ensemble model.

[0093] The multi-boost ensemble algorithm based on CNN base classifiers is implemented in four steps: obtaining the training sample set, inputting data, the ensemble model learning process, and outputting diagnostic results. Figure 3 and Figure 4 As shown, the fault detection model is trained through the following steps:

[0094] Among them, dataset A dataset containing N preprocessed training samples.

[0095] Number of iterations T: Number of sub-decision groups.

[0096] Number of iterations q: The number of AB models in the sub-decision group.

[0097] Number of iterations K: The number of CNN base classifiers in the AB model.

[0098] CNN base classifier function: M = CNN(·).

[0099] Integer I t : Sub-decision group iteration termination flag variable.

[0100] S201: Obtain several network connection records with network fault labels to construct a training sample set; then preprocess the training sample set to obtain a preprocessed dataset, and set the weights of each training sample in the preprocessed dataset;

[0101] S202: Initialize flag variable I t Let t = 1, and set the index of t.

[0102] S203: Initialize the number of AB models in the sub-decision group q = 1;

[0103] S204: Initialize the number of CNN base classifiers in the AB model, k = 1;

[0104] S205: Based on the weights of each training sample, perform N samplings with replacement to obtain the base classifier dataset of the k-th CNN base classifier in the q-th AB model of the t-th sub-decision group. It is then fed into a convolutional neural network, which outputs the corresponding CNN base classifier.

[0105] The flag variable I is calculated using the following formula. t : if I t =t, then the dataset will be preprocessed. The weights of each training sample are set to Otherwise, based on the output error t e q Adjust the weights of each training sample; then perform sampling with replacement based on the weights of each training sample in the preprocessed dataset.

[0106]

[0107] S206: If k < K, then k = k + 1, and return to step S205; otherwise, construct the q-th AB model AB(t) in the t-th sub-decision group based on the obtained K CNN base classifiers and the following formula. t D q ), and calculate the AB model AB( t D q Output error t e q Therefore, based on the output error t eq Adjust the weights of each training sample in the preprocessed dataset; simultaneously, obtain the corresponding AB model dataset vector (or average dataset).

[0108]

[0109] In the formula: AB( t D q ) represents the q-th AB model in the t-th sub-decision group; This represents the k-th CNN base classifier in the q-th AB model of the t-th sub-decision group; This represents the output weight of the k-th CNN base classifier in the q-th AB model of the t-th sub-decision group;

[0110] in,

[0111] In the formula: This represents the ratio of the number of samples that were not correctly classified in the k-th CNN base classifier of the q-th AB model in the t-th sub-decision group to the total number of samples.

[0112] The output error of the AB model is calculated using the following formula. t e q :

[0113]

[0114] In the formula: t e q The weight(X) represents the output error of the q-th AB model in the t-th sub-decision group; t ) represents the training sample X t The weights; AB(X) t ) represents the output of the AB model.

[0115] Adjust the weights of each training sample in the preprocessed dataset as follows:

[0116] 1) If t e q If the value is greater than 0.5, the corresponding AB model is discarded, and the weights of each training sample are calculated based on the continuous Poisson distribution.

[0117] 2) If t e q =0, then set the output weight. t β q =10 -10 And calculate the weights of each training sample based on a continuous Poisson distribution;

[0118] 3) If 0 < t eq If the value is less than 0.5, then set the output weight. t β q = t e q / (1- t e q ), and for the AB model dataset vector t D q For each training sample, the weight of the misclassified sample is divided by 2. t e q The weights of the positive samples are divided by 2(1- t e q ), and the minimum weight is 10. -8 .

[0119] The continuous Poisson distribution can be represented by the following formula:

[0120]

[0121] In the formula: p represents the probability value, which is the weight of the training sample; Random(1,2,…,999) means that an integer is randomly generated from 1 to 999.

[0122] S207: If q < I t If q = q + 1, then return to step S204; otherwise, based on the obtained I t An AB model is used to construct the t-th sub-decision group, and the output of the t-th sub-decision group is calculated using the following formula. and weight α t Simultaneously, the corresponding sub-decision group dataset vector is obtained.

[0123] α t =log[(1- t e q ) / t e q ];

[0124] In the formula: α t This represents the weight of the t-th sub-decision group; t e q This represents the output error of the q-th AB model in the t-th sub-decision group;

[0125] S208: If t < T, then t = t + 1, and return to step S203 to enter the next sub-decision group; otherwise, classify the training samples through T sub-decision groups respectively, and calculate the network fault category with the largest weight as the network fault prediction value using the following formula.

[0126]

[0127] t β q = t e q / (1- t e q );

[0128] In the formula: MB * ( t D q Y represents the network fault prediction value for a network connection record, where each of the T sub-decision groups classifies the training samples. t Represents a dataset vector The training samples correspond to a label vector composed of network fault labels, which is the true value of the network fault; α t This represents the weight of the t-th sub-decision group; t D q This represents the input dataset vector to the q-th AB model in the t-th sub-decision group, which is the dataset of the K CNN base classifiers in the AB model; t β q This represents the output weight of the q-th AB model in the t-th sub-decision group on the fault detection model.

[0129] This invention focuses on misclassified training samples during the training of CNN base classifiers using an adaptive boosting algorithm. This leverages the strong bias reduction capability of the adaptive boosting algorithm to reduce network fault classification errors, thereby further improving the accuracy of network fault detection. Simultaneously, this invention uses a multi-boosting algorithm to weightedly sum the outputs of each sub-decision group to generate corresponding network fault prediction values. This leverages the multi-boosting algorithm's ability to effectively reduce variance, thereby improving classification accuracy and addressing the problem of low fault data value density, thus reducing the generalization error of network fault detection.

[0130] In practical implementation, considering the large number of network connection records in the training sample set, coupled with the fact that this massive amount of network data contains many invalid feature attributes, irrelevant or redundant features increase the algorithm's space and time consumption, and may also lead to a decrease in fault diagnosis accuracy. On the other hand, from the overall statistical analysis of network operation data, the values ​​of feature attributes vary, and some attribute values ​​are even quite complex. If data mining algorithms are directly applied to the original network operation dataset, the process will be extremely complex, consuming a lot of manpower and resources, and the fault diagnosis results will not be ideal. Therefore, in order to reduce the dependence on measurement units and mitigate the impact of differences in feature attribute measurements on the diagnostic algorithm, it is necessary to quantify and standardize the feature attributes.

[0131] The training sample set is preprocessed using the following steps:

[0132] S2011: Training sample set D = {(X1,Y1),…,(X i ,Y i ),…,(X N ,Y N )}, i = 1, 2, ..., N contains N training samples, where X i ={x ij {j = 1, 2, ..., M} represents the i-th training sample, and Y = {Y} i ,i=1,2,…,N} represents the label vector of the training sample, and M represents the original feature attribute dimension of the training sample;

[0133] S2012: For the feature attribute vector X = {X1,…,X...} j ,…,X M}, j = 1, 2, ..., M are processed into numerical values;

[0134] S2013: Normalize the numerically processed feature attribute vector X to obtain the normalized feature attribute vector. This results in a preprocessed dataset after numerical and normalization processing.

[0135] The feature attribute vector is normalized using the following formula:

[0136]

[0137] in,

[0138] In the formula: X represents the feature attribute vector of the j-th dimension after normalization; j x represents the j-th feature attribute vector after numerical processing; ij A represents the feature attribute vector of the j-th dimension of the i-th training sample; j S j X jmin X jmax Let X represent the j-th feature attribute respectively. j The mean, variance, minimum and maximum values.

[0139] In this embodiment, after numerical and normalization processing, the preprocessed dataset undergoes one-hot encoding. One-hot encoding uses categorical variables for encoding. Assuming L labels, the integer values ​​from 0 to L-1 are first converted into binary data. The one-hot encoding for the L labels is a string of binary data, where all bits except the integer label are 0, with the integer label being 1. See Table 4 for details.

[0140] This invention reduces the dependence of the fault detection model on the unit of measurement by performing numerical processing and normalization on the training sample set, weakens the impact of differences in feature attribute measurements on the fault detection model, and thus improves the performance of network fault detection, thereby further improving the effectiveness of network fault detection.

[0141] In other preferred embodiments, a high-value-density dataset can also be generated using the XGBoost algorithm:

[0142] Preprocessed datasets were computed using the XGBoost algorithm. Various feature attributes Gain for j = 1, 2, ..., M;

[0143]

[0144] In the formula: G j Representing feature attributes Gain; (G jL +G jR ) 2 / (H jL +H jR +λ), and G represents the score of the left subtree, the score of the right subtree, and the score obtainable without splitting; jL H jL G jR H jR denoted as the gradient and second gradient of the left subtree, and the gradient and second gradient of the right subtree, respectively, in the Taylor second-order expansion of the XGBoost loss function; λ represents the desired simplicity of the obtained tree, and the larger the value, the simpler the tree structure is; γ represents the complexity cost of adding a new leaf node;

[0145] Set a threshold η: if G j If the value is less than η, then the corresponding feature attribute is removed; otherwise, the corresponding feature attribute is retained.

[0146] Repeat the above steps until the gains of M feature attributes are compared. If, during this process, the gains of b feature attributes are less than a set threshold η, then these b feature attributes are removed, resulting in a high-value density dataset. Wherein, the feature attribute vector is b represents the number of feature attributes that were removed.

[0147] This invention uses the XGBoost algorithm to evaluate the importance of feature attributes and filter features of training samples in the preprocessed dataset. This enables the evaluation and filtering of feature attributes in network fault data, thereby eliminating irrelevant or redundant feature attributes to obtain a high-value-density dataset for training fault detection models. This approach balances the efficiency and accuracy of fault detection model training.

[0148] In the specific implementation process, after training the fault detection model, the performance of the fault detection model is evaluated by the accuracy, F1 score, detection rate (TPR), and false alarm rate (FPR) of various training samples.

[0149] 1) Calculate the accuracy of various training samples using the following formula:

[0150]

[0151] In the formula: Acc test N represents the precision of various training samples; a This represents the number of samples that were correctly classified; N represents the total number of samples.

[0152] 2) The F1 score is calculated using the following formula (F1-score, which balances the model's recall and precision):

[0153]

[0154] In the formula: F1 represents the F1 score; P represents the precision; R represents the recall.

[0155] 3) Calculate the detection rate (TPR) and false alarm rate (FPR) using the following formulas:

[0156]

[0157] In the formula: TP represents the number of samples that are actually positive and the fault diagnosis model predicts them to be positive; FN represents the number of samples that are actually positive and the fault diagnosis model predicts them to be negative; FP represents the number of samples that are actually negative and the fault diagnosis model predicts them to be positive; TN represents the number of samples that are actually negative and the fault diagnosis model predicts them to be negative.

[0158]

[0159] This invention can effectively evaluate the performance of fault detection models by measuring the accuracy, F1 score, detection rate, and false alarm rate of various training samples, thereby obtaining the best-performing fault detection model and further improving the effectiveness of network fault detection.

[0160] To better illustrate the advantages of the technical solution of the present invention, the following experiments are disclosed in this embodiment.

[0161] This experiment uses the KDDCup99 dataset, a widely recognized dataset among cybersecurity researchers. This dataset contains TCP dump network connection data simulating the operation of the U.S. Air Force LAN. The test dataset contains 450,000 connection records, and the characteristic attributes of network failures and samples of specific network connection records are shown in Tables 1 and 2, respectively. Sampling was performed according to the percentages shown in Table 1, obtaining 10% of the test dataset as the training sample set and 5% as the test dataset. To verify the effectiveness of the proposed algorithm, this experiment utilizes the TensorFlow module and implements the algorithm in Python.

[0162] Table 1. Description of the characteristic attributes of network faults

[0163]

[0164] Table 2 Sample data for a network connection record

[0165]

[0166] (1) Feature attribute preprocessing

[0167] To simulate a real-world environment and test the robustness of the proposed algorithm, this experiment added 50dB of random noise to the original dataset and performed numerical and normalization processing on the data. The statistical analysis of each feature attribute is as follows: Figure 5 As shown in the figure, the boxplot reveals that the values ​​of some features are almost all zero, indicating that these features have little impact on the classification results and can be removed to achieve data dimensionality reduction. The removed feature attributes are shown in Table 3. In this experiment, the parameters were set to N=450000, M=41, b=5, so the preprocessed dataset... Where the feature attribute vector is

[0168] Table 3. Characteristic attributes of network faults that were removed.

[0169]

[0170] (2) Establish CNN base classifier

[0171] The categorical variables were encoded using one-hot encoding, converting integer values ​​from 0 to 4 into binary vectors. Except for the marker bit corresponding to the integer, which was 1, all other bits were 0. The results are shown in Table 4.

[0172] Table 4. Five Classification Categories of Network Faults

[0173]

[0174] The parameters for the input layer of the CNN base classifier are: W1 = 36, H1 = 36; the parameters for convolutional layers C1 and C2 are: kernel size 2×2, stride 1, no padding; the parameters for sampling layers S1 and S2 are: stride 1, no padding; the parameters for the output layer are: using the Softmax function, number of fault categories 5. In this experiment, the number of base classification models is set to K = 6, and the number of samples is... This yields K balanced datasets, and these datasets form a dataset vector.

[0175] (3) Fault diagnosis of MB serial ensemble model network based on CNN base classifier

[0176] The parameters for initializing the fault detection model (hereinafter also referred to as the CNN+HB model) in this invention are as follows: number of training samples. The number of sub-decision groups is T=5. Then, the number of AB algorithm models contained in each sub-decision group is calculated, and the number of base classification models contained in each AB model is K=6. After training, the accuracy and F1 score of the network fault diagnosis model proposed in this invention are calculated. The average diagnostic accuracy and F1 score for Normal, DOS, Probing, R2L, and U2R are shown in Table 5. The detection rate (TPR) and false alarm rate (FPR) of the fault detection model proposed in this invention are calculated to be 0.92 and 2.16, respectively.

[0177] Table 5. Fault diagnosis results based on CNN+HB model

[0178]

[0179] Table 5 shows the diagnostic accuracy of the CNN+HB model for each network type, ranging from a minimum of 90.43% to a maximum of 95.84%, with an F1 score of 0.964, thus proving the feasibility of the invention.

[0180] To demonstrate the effectiveness and high accuracy of the model, this experiment compares the proposed CNN+HB model, LSTM model, and VSM model. The comparison results are as follows: Figure 6 As shown.

[0181] As the number of iterations increases Figure 6 The results show a comparison of the network fault diagnosis accuracy of the CNN+HB model, LSTM model, and VSM model proposed in this patent. Figure 6It can be seen that when the number of training periods is 30, the average diagnostic accuracy of the CNN+HB model, LSTM model, and VSM model reaches a moderate level. When the number of training periods is 22, the diagnostic accuracy of the three models are 95.4%, 90.7%, and 89.1%, respectively. In addition, CNN+HB integrates multiple CNN base classification models, and the integrated model CNN+HB has a smaller diagnostic error, that is, it improves the fault diagnosis accuracy.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A network fault detection method based on multi-boost ensemble learning, characterized in that, Including: S1: Obtain the network connection records to be detected; S2: Input the network connection records into the trained fault detection model and output the corresponding network fault prediction value; The fault detection model consists of T sub-decision groups, each containing I... t There are K AB models, and each AB model contains K CNN base classifiers; During training, first obtain a number of network connection records with network fault labels to construct a training sample set; secondly, input the training sample set into the convolutional neural network of the fault detection model for preliminary learning to obtain a number of CNN base classifiers; then, through the adaptive boosting algorithm, focus on the training samples misclassified during the training process of the CNN base classifiers, and construct the corresponding AB model based on the CNN base classifiers; then construct the corresponding sub-decision groups based on the AB model, and then through the multiple boosting algorithm, perform weighted summation on the outputs of the T sub-decision groups to generate the corresponding network fault prediction value; finally, calculate the training loss based on the network fault prediction value and the corresponding network fault label to optimize the fault detection model; Train the fault detection model through the following steps: S201: Obtain a number of network connection records with network fault labels to construct a training sample set; then preprocess the training sample set to obtain a preprocessed data set, and set the weights of each training sample in the preprocessed data set; S202: Initialize flag variable I t Let t = 1, and set the index of t. S2'step 03: Initialize the number q = 1 of AB models in the sub-decision group; S204: Initialize the number k = 1 of CNN base classifiers in the AB model; S205: Based on the weights of each training sample, perform N samplings with replacement to obtain the base classifier dataset of the k-th CNN base classifier in the q-th AB model of the t-th sub-decision group. It is then fed into a convolutional neural network, which outputs the corresponding CNN base classifier. S206: If k < K, then k = k + 1, and return to step S205; otherwise, construct the q-th AB model AB(t) in the t-th sub-decision group based on the obtained K CNN base classifiers and the following formula. t D q ), and calculate the AB model AB( t D q Output error t e q Therefore, based on the output error t e q Adjust the weights of each training sample in the preprocessed dataset; simultaneously, obtain the corresponding AB model dataset vector. In the formula: AB( t D q ) represents the q-th AB model in the t-th sub-decision group; This represents the k-th CNN base classifier in the q-th AB model of the t-th sub-decision group; This represents the output weight of the k-th CNN base classifier in the q-th AB model of the t-th sub-decision group; in, In the formula: This represents the ratio of the number of samples that were not correctly classified in the k-th CNN base classifier of the q-th AB model in the t-th sub-decision group to the total number of samples. S207: If q < I t If q = q + 1, then return to step S204; otherwise, based on the obtained I t An AB model is used to construct the t-th sub-decision group, and the output of the t-th sub-decision group is calculated using the following formula. and weight α t Simultaneously, the corresponding sub-decision group dataset vector is obtained. α t log[(1- t e q ) / t e q ]4 In the formula: α t This represents the weight of the t-th sub-decision group; t e q This represents the output error of the q-th AB model in the t-th sub-decision group; S208: If t < T, then t = t + 1, and return to step S203 to enter the next sub-decision group; otherwise, classify the training samples through the T sub-decision groups respectively, and calculate and obtain the network fault category with the largest weight as the network fault prediction value according to the following formula; t β q = t And q / (1- t And q ); In the formula: MB * ( t D q Y represents the T sub-decision groups, each classifying the training samples and using the network fault category with the highest weight as the predicted network fault value for that network connection record; t Represents the AB model dataset vector The training samples correspond to a label vector composed of network fault labels, which is the true value of the network fault; α t This represents the weight of the t-th sub-decision group; t D q This represents the input dataset vector to the q-th AB model in the t-th sub-decision group, which is the dataset of the K CNN base classifiers in the AB model; t β q This represents the output weight of the q-th AB model in the t-th sub-decision group to the fault detection model; S3: Take the network fault prediction value output by the fault detection model as the network fault detection result of the network connection record.

2. The network fault detection method based on multiple boosting ensemble learning as described in claim 1, characterized in that: In step S2, the convolutional neural network of the fault detection model is the LetNet5 model, which includes an input layer, a convolutional layer, a sampling layer, a fully connected layer, and an output layer.

3. The network fault detection method based on multiple boosting ensemble learning as described in claim 1, characterized in that: In step S201, preprocess the training sample set through the following steps: S2011: Training sample set D = {(X1,Y1),…,(X i ,Y i ),…,(X N ,Y N The sequence X = 1, 2, ..., N contains N training samples. i ={x ij {j = 1, 2, ..., M} represents the i-th training sample, and Y = {Y} i ,i=1,2,…,N} represents the label vector of the training sample, and M represents the original feature attribute dimension of the training sample; S2012: For the feature attribute vector X = {X1,…,X...} j ,…,X M }, j = 1, 2, ..., M are processed into numerical values; S2013: Normalize the numerically processed feature attribute vector X to obtain the normalized feature attribute vector. j = 1, 2, ..., M, thus obtaining the preprocessed dataset after numerical and normalization processing.

4. The network fault detection method based on multiple boosting ensemble learning as described in claim 3, characterized in that: In step S2013, normalize the feature attribute vector according to the following formula: in, In the formula: X represents the feature attribute vector of the j-th dimension after normalization; j x represents the j-th feature attribute vector after numerical processing; ij A represents the feature attribute vector of the j-th dimension of the i-th training sample; j S j X jmin X jmax Let X and Y represent the j-th feature attribute respectively. j The mean, variance, minimum and maximum values.

5. The network fault detection method based on multiple boosting ensemble learning as described in claim 1, characterized in that: In step S205, the flag variable I is calculated using the following formula. t : if I t =t, then the dataset will be preprocessed. The weights of each training sample are set to Otherwise, based on the output error t e q Adjust the weights of each training sample; then perform sampling with replacement based on the weights of each training sample in the preprocessed dataset.

6. The network fault detection method based on multiple boosting ensemble learning as described in claim 1, characterized in that: In step S206, the output error of the AB model is calculated using the following formula. t e q : In the formula: t e q The weight(X) represents the output error of the q-th AB model in the t-th sub-decision group; t ) represents the training sample X t The weights; AB(X) t ) represents the output of the AB model.

7. The network fault detection method based on multiple boosting ensemble learning as described in claim 6, characterized in that: In step S206, adjust the weights of each training sample in the preprocessed data set in the following way: 1) If t e q If the value is greater than 0.5, the corresponding AB model is discarded, and the weights of each training sample are calculated based on the continuous Poisson distribution. 2) If t e q =0, then set the output weight. t β q =10 -10 And calculate the weights of each training sample based on a continuous Poisson distribution; 3) If 0 < t e q If the value is less than 0.5, then set the output weight. t β q = t e q / (1- t e q ), and for the AB model dataset vector t D q For each training sample, the weight of the misclassified sample is divided by 2. t e q The weights of the positive samples are divided by 2(1- t e q ), and the minimum weight is 10. -8 .

8. The network fault detection method based on multiple boosting ensemble learning as described in claim 7, characterized in that: Represent the continuous Poisson distribution by the following formula: In the formula: p represents the probability value, which is the training sample weight; Random(1, 2,..., 999) represents a random integer generated from 1 to 999.

9. The network fault detection method based on multiple boosting ensemble learning as described in claim 1, characterized in that: After training the fault detection model, evaluate the performance of the fault detection model through the accuracy, F1 value, detection rate TPR, and false alarm rate FPR of various training samples; 1) Calculate the accuracy of various training samples according to the following formula: In the formula: Acc test N represents the precision of various training samples; a This represents the number of samples that were correctly classified; N represents the total number of samples. 2) Calculate the F1 value according to the following formula: In the formula: F1 represents the F1 value; P represents the precision rate; R represents the recall rate; 3) Calculate the detection rate TPR and false alarm rate FPR according to the following formula: In the formula: TP represents the number of samples that are actually positive and the fault diagnosis model predicts them to be positive; FN represents the number of samples that are actually positive and the fault diagnosis model predicts them to be negative; FP represents the number of samples that are actually negative and the fault diagnosis model predicts them to be positive; TN represents the number of samples that are actually negative and the fault diagnosis model predicts them to be negative.

Images