5g network anomaly traffic detection method based on deep learning

By constructing a dataset using deep learning methods and performing feature extraction and data augmentation, the shortcomings of traditional methods in processing massive amounts of data and extracting temporal features are addressed, enabling efficient anomaly detection of 5G network traffic.

CN117113221BActive Publication Date: 2025-11-28XIAN UNIV OF TECH
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202311039327.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-08-17
Publication Date
2025-11-28
Estimated Expiration
2043-08-17

AI Technical Summary

Technical Problem

Traditional methods for detecting abnormal network traffic are insufficient in processing massive amounts of data and extracting time-series features, and cannot effectively handle complex network environments and the problem of explosive data growth.

Method used

We employ a deep learning-based approach, which involves constructing a dataset, performing feature extraction and data preprocessing, using a GAN model for data augmentation, combining BiGRU and Attention mechanisms for feature extraction, and finally utilizing a softmax classifier for abnormal traffic detection.

Benefits of technology

It improves the ability to process massive amounts of data, efficiently extracts the temporal characteristics of 5G network traffic, and enhances the accuracy and efficiency of anomaly detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117113221B_ABST
    Figure CN117113221B_ABST
Patent Text Reader

Abstract

The application discloses a 5G network abnormal traffic detection method based on deep learning, and specifically comprises the following steps: step 1, constructing an initial data set X of data traffic under a 5G environment; step 2, pre-processing the initial traffic data set X to obtain a data set Z; step 3, extracting features from the data set Z to obtain a feature data set F; step 4, dividing the features F obtained in step 3 into a data set A' and a data set B', inputting the data set A' into a GAN model for training to obtain a model LSCGAN, and placing the data set B' into the model LSCGAN to obtain a data set F'; and step 5, placing the feature set F' extracted in step 4 into a softmax classifier for classification to obtain an abnormal traffic classification result. The application solves the problems that current traditional methods cannot process massive data and extract data timing features.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of network security, and relates to a 5G network abnormal traffic detection method based on deep learning. BACKGROUND

[0002] The network abnormal traffic detection technology refers to using the traffic analysis technology in the computer network to monitor the traffic in the network in real time, identify and report the possible abnormal traffic conditions. This technology can effectively prevent and control network attacks, network congestion and other network security problems, and ensure the normal operation of the network. In recent years, in the network traffic anomaly detection, traditional machine learning algorithms have been widely applied, such as Naive Bayes, Random Forest, Support Vector Machine and K-Nearest Neighbor algorithm. These methods have certain improvement in the accuracy of identification, but there is a problem of limited ability to express complex functions and weak generalization ability. In the face of complex network environment and explosive growth of data, the performance of the model will be greatly reduced, so it cannot well handle the problem of mass data classification and cannot extract the time sequence characteristics of data. SUMMARY

[0003] The purpose of the application is to provide a 5G network abnormal traffic detection method based on deep learning, which solves the problem that the current traditional method cannot handle mass data and extract data time sequence characteristics.

[0004] The technical scheme adopted by the application is that the 5G network abnormal traffic detection method based on deep learning specifically comprises the following steps:

[0005] Step 1, constructing an initial data set X of data traffic under the 5G environment;

[0006] Step 2, preprocessing the initial traffic data set X to obtain a data set Z;

[0007] Step 3, extracting features from the data set Z to obtain a feature data set F;

[0008] Step 4, dividing the features F obtained in step 3 into a data set A' and a data set B', inputting the data set A' into a GAN model for training to obtain a model LSCGAN, and putting the data set B' into the model LSCGAN to obtain a data set F';

[0009] Step 5, putting the feature set F' extracted in step 4 into a softmax classifier for classification to obtain an abnormal traffic classification result.

[0010] The application also has the following characteristics:

[0011] The specific process of step 1 is: capture the data traffic generated by normal network activities and artificially initiated network attack behaviors in the 5G environment through wireshark software, store it in the form of pcap file, and use all the data traffic packets generated as the initial data set X.

[0012] The specific process of step 2 is:

[0013] Step 2.1, use the open source tool named SplitCap to split the pcap in the data set X according to the flow cutting method, get all the packets of the same five tuple in the network traffic, and manually label all the packets;

[0014] Step 2.2, replace the unique IP address and MAC address in all packets segmented in step 2.1 with randomized address;

[0015] Step 2.3, through the script, batch judge whether the packet size after replacing the random address in step 2.2 is 0, if it is 0, delete the empty packet; then use the script to calculate the hash value of each packet content, and store the hash value in a hash table, if there are the same hash value, it means that the two packet contents are the same, which can be marked as duplicate packet, finally output the path of all duplicate packets, and delete the duplicate packets through finddupe tool;

[0016] Step 2.4, based on the data set after deleting the empty packet and duplicate packet in step 2.3, use BorderlineSMOTE algorithm to oversample the abnormal traffic, use TomeLink algorithm to undersample the data set after oversampling, get the final sample X eq ;

[0017] Step 2.5, for the symbol type data appearing in the data packet, use one-hot method to convert all symbol type features in data set X eq into digital features, finally get the numerical data set X o ;

[0018] Step 2.6, based on the data set X o processed in step 2.5, use hot card filling algorithm to fill the missing numerical features, get the filled data set X p ;

[0019] Step 2.7, use max-min normalization method to normalize the filled data set X p , so that the range of data is compressed between [0, 1], the specific formula is as follows:

[0020]

[0021] where x finp is the normalized data, x min is the minimum value in the original data set, x max is the maximum value in the original data set, x p refers to any value in the data set X p , p = 1, 2,..., P, P is the total data amount of the data set X p ;

[0022] Step 2.8, using the normalized data x finp obtained in step 2.7, a data set X FIN is constructed, and the data set X FIN = {x fin1 , x fin2 ..., x finp} is subjected to data dimension reduction using the LDA method, and the dimension of the reduced data set is from a x a to b x b, wherein b < a;

[0023] Step 2.9, the data packet subjected to dimension reduction in step 2.8 is cropped, and the data packet of uniform length is generated into a grayscale image;

[0024] Step 2.10, the grayscale image generated in step 2.9 is converted into an idx format file, and a data set Z is obtained.

[0025] The specific process of step 3 is as follows:

[0026] Step 3.1, the data set Z is divided into a training set A and a test set B;

[0027] Step 3.2, the divided training set A = {a1, a2,..., a n} is sent into a BiGRU model, and the calculation formula of the hidden layer state h t of the BiGRU model is as follows:

[0028]

[0029]

[0030]

[0031] where h f t represents the forward output state obtained by inputting the t time through the forward propagation layer; h b t represents the reverse output state obtained by inputting the t time through the reverse propagation layer, f d is a forward output function, f b is a reverse output function, h t is a hidden state vector, and at denotes any value in the training set A, denotes the forward output state at the previous t time, denotes the backward output state at the next time;

[0032] Step 3.3, using the Attention mechanism to extract the features h from the BIGRU model t , according to the feature importance, and finally adding the features evaluated by the feature importance weight, that is, the secondary feature extraction feature F.

[0033] The specific process of step 4 is:

[0034] Step 4.1, randomly initialize the parameters of the generator G and the discriminator D, define Adam as the optimizer of the generator and the discriminator, use the rand() function in the numpy library to generate Gaussian random noise data z, define the mean as 0 and the standard deviation as 1;

[0035] Step 4.2, input the noise data z and the conditional information--network traffic category y into the generator network G to generate data G'(z,y), input G'(z,y) into the discriminator network D, and calculate the generator loss function according to the generated data G'(z,y) and the output of the discriminator D(G'(z,y)). Define the generator network generation loss function L gen :

[0036]

[0037] Step 4.3, input the original feature data t and the conditional information--network traffic category y into the discriminator network D and the generated data G'(z,y) to calculate the discriminator loss together, define the discriminator loss function L dis :

[0038]

[0039] Step 4.4, use the stochastic gradient descent algorithm to alternately update the parameters of the generator and the discriminator to minimize their respective loss functions. In each iteration, use the optimizer Adm to update the discriminator first, then update the generator, and finally obtain the trained data enhancement model--LSCGAN:

[0040]

[0041]

[0042] where, θ D(t) and θ G(t) represent the parameters of the discriminator and the generator at the tth round, respectively, θ D(t-1) and θ G(t-1)respectively are the parameters of the discriminator and the generator in the t-1th round, and respectively are the gradients of the loss function of the discriminator and the generator with respect to the parameters, obtained by the automatic differentiation function of the deep learning framework, and δ is the learning rate;

[0043] Step 4.5, input the data set B' into the trained generator in the model LSCGAN to obtain an expanded new data set F'.

[0044] The beneficial effects of the present application are as follows:

[0045] 1. The present application adopts the Borderline SMOTE algorithm to oversample the abnormal traffic samples with less sample quantity, and adopts the Tomek Link algorithm to undersample the oversampled sample set; for the problem of data lack, the hot card filling algorithm is used for filling,

[0046] 2. The present application adopts LDA data dimension reduction for the problem that too large data set dimension may cause high model complexity and poor generalization, which greatly reduces the model complexity, shortens the model training time and improves the model robustness.

[0047] 3. The BIGRU-Attention model proposed by the present application is more portable than the traditional feature extraction method, and for the time sequence feature which is particularly important in 5G network traffic, the model can efficiently and accurately extract the sequence feature of network traffic, and make the model pay more attention to the features more important for abnormal traffic detection, so that the abnormal detection is more efficient.

[0048] 4. In the data enhancement module, the LSCGAN model adopted by the present application uses the least square loss function to replace the binary cross entropy loss function in the original GAN, improves the training stability of the generator and the discriminator, and introduces the network traffic type as the conditional information, which can better generate data conforming to the network traffic classification. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is the total flowchart of the 5G network abnormal traffic detection method based on deep learning of the present application;

[0050] Figure 2 is the Borderline SMOTE oversampling flowchart of the 5G network abnormal traffic detection method based on deep learning of the present application;

[0051] Figure 3 is the Tomek Link undersampling flowchart of the 5G network abnormal traffic detection method based on deep learning of the present application;

[0052] Figure 4is a BIGRU-Attention model structure diagram of the 5G network abnormal traffic detection method based on deep learning of the application;

[0053] Figure 5 is an LSCGAN model structure diagram of the 5G network abnormal traffic detection method based on deep learning of the application. DETAILED DESCRIPTION

[0054] The application will be described in detail below in combination with the drawings and specific embodiments.

[0055] Embodiment 1

[0056] The 5G network abnormal traffic detection method based on deep learning of the application, as shown in Figure 1 , specifically includes the following steps:

[0057] Step 1, capture the data traffic generated by normal network activities and artificially initiated network attack behaviors in the 5G environment through the wireshark software, store it in the pcap file mode, and take all the data traffic packets generated as the initial data set X.

[0058] Step 2, pre-process the initial traffic data set X, and the specific steps are as follows:

[0059] Step 2.1, use the open source tool named SplitCap to split the pcap in the data set X according to the flow cutting mode, obtain all the packets of the same five-tuple in the network traffic, and manually label all the packets in packet units. The characteristics of abnormal traffic are: a large number of connection attempts, frequent port changes, frequent packet loss, protocol violations, malicious software communication, and large-scale repeated requests. According to the above characteristics, the abnormal traffic is labeled, and the rest is normal traffic. Finally, packets with normal category labels and abnormal category labels are obtained.

[0060] Step 2.2, replace the unique IP address and MAC address in all packets after step 2.1 with a randomized address.

[0061] Step 2.3, first, determine whether the size of the packet after replacing the random address in step 2.2 is 0 through a script in batches. If it is 0, the empty packet is deleted. Secondly, calculate the hash value of the content of each packet using a script, and store the hash value in a hash table. If there are the same hash values, it means that the contents of the two packets are the same, and they can be marked as duplicate packets. Finally, output the path of all duplicate packets, and delete the duplicate packets through the finddupe tool.

[0062] Step 2.4, based on step 2.3, the data set after deleting empty packets and repeated packets, the BorderlineSMOTE algorithm is used for oversampling of abnormal traffic, and the TomeLink algorithm is used for undersampling of the oversampled data set, the specific steps are as follows:

[0063] Step 2.4.1, first set the abnormal traffic sample in the data set X as x u , the sampling process is shown in Figure 2 , for each sample x u in the abnormal traffic sample set X u , search for the nearest m samples in the entire data set X u , record the number of abnormal traffic samples in the m samples as m', if m' = m, x u is noise, and no operation is performed; if 0≤m'≤m / 2, x u is very safe, and no operation is performed; if m / 2≤m'≤m, then x u is a dangerous point, according to this rule, the original data set X can be divided into three categories, as shown in the following table 1:

[0064] Table 1

[0065]

[0066] Step 2.4.2, after determining which abnormal samples belong to the dangerous set, the dangerous set is composed of all dangerous samples in the data set X, and the abnormal traffic sample point in the dangerous set is represented by x d . From the abnormal traffic sample set X u and the normal traffic sample set X n , select the K samples with the smallest Euclidean distance from x d to form sets X uk and X nk .

[0067] Step 2.4.3, set a proportion a, select a proportion of sample points in X uk and x d to do random linear interpolation to generate new sample points; select 1-a proportion of sample points in X nk and x d to do random interpolation to generate new sample points, where the a proportion is selected in the range of (0, 0.5), which can make the generated new sample points closer to the abnormal sample points. At the same time, the present application is aimed at linear and nonlinear relationships existing in the data set, and linear interpolation x new1 and quadratic interpolation x new2 are combined to calculate the new sample point x new , and the formula is as follows:

[0068] xnew1 = (x d + |x i - x d | * |y new - y d | ÷ |y i - y d |) (1)

[0069] x new2 = (x new1 + |y new - y d | * |y new - y i | * |x i - x d | / |y i - y d | (2)

[0070] x 2 = εx new + (1-ε) x new1 (3)

[0071] where x new2 is the new anomaly traffic sample point, x new is the anomaly traffic sample point belonging to the dangerous set, x d is the sample point in X i / X uk , y nk is the horizontal coordinate of the new anomaly traffic sample point, y new is the horizontal coordinate of the anomaly traffic sample point belonging to the dangerous set, and y d is the horizontal coordinate of the sample point in X i / X uk , the adjustment parameter ε takes the value of 0 when the distance d = (y nk - y i ) between x d and x i is greater than the threshold value (set to 0.5 in this paper), otherwise takes the value of 1.

[0072] Step 2.4.4, the processed sample set X d is cleaned up by Tomek Link algorithm, and the specific processing process is as follows: any two samples x new , x new in the sample set X newi belong to two different categories, if there is no third sample x1 such that distance(x newj , x1) < distance(x newi , x newi ), then x newj is removed.) or distance(x newj , x newi ) < distance(x newj , x eq ), then the two samples are called a Tomek Link, delete all Tomek Links (also called T-link pairs refer to the distance between two samples is very close, but they belong to different categories), and finally get the final sample X eq . The processing process is shown in Figure 3 , the left side of the arrow is the sample set before processing, and the right side of the arrow is the sample set after processing, wherein the white circle and the triangle represent two samples of different categories, and the black circle represents the Tomek Link.

[0073] The above steps 2.4.1-2.4.3 are oversampling processes, and step 2.4.4 is an undersampling process.

[0074] In step 2.5, for the symbol type data appearing in the data packet, all symbol type features in the data set X o are converted into digital features using the one-hot method, and finally the numerical data set X o is obtained. The specific processing of the symbol type feature is as follows: the Boolean type True represents 1, and False represents 0. For text type, since the text feature has multiple values, this time the ASCLL code value is selected to represent different text features, and each ASCLL code value is separated by “.”. In particular, for the protocol in the data packet, the definition is shown in the following table 2:

[0075] Table 2

[0076] TCP 1 UDP 2 SCTP 3 NGAP 4 HTTP2 5 NAS 6

[0077] In step 2.6, based on the data set X o processed in step 2.5, for the missing numerical features, a hot card filling algorithm is used to fill, and the specific steps are as follows:

[0078] In step 2.6.1, the position of the missing value in the data set X miss is determined by observing the data, the variable containing the missing value is defined as X miss , and the missing value of this variable is defined as x o .

[0079] In step 2.6.2, the correlation coefficient between all variables is calculated based on the data set X o . In selecting the correlation coefficient calculation method, in order to fully consider the linear and nonlinear relationship between data, the present application combines the Pierre coefficient and the Spearman coefficient, and the calculation formula is as follows:

[0080]

[0081] where r(X I , X J ) represents the correlation coefficient between variable X o and variable X I , whose value range is (0, 1), when r(X, Y) = 1, it represents complete positive correlation; when r(X, Y) = -1, it represents complete negative correlation; when r(X, Y) = 0, it represents no linear correlation; x J is the observation value (specific measurement value) in X I , x J is the observation value in X J , is the mean value of variable X I , is the mean value of variable X J , and n is the sample size. In this data set, most of them show linear relationship, so we set the weight values ω1, ω2 as 0.7, 0.3.

[0082] Step 2.6.2, according to the correlation coefficient r(X I , X J ) calculated, an m x m correlation coefficient matrix is constructed, where m is the number of variables. For each pair of variables (X I , X J ), the calculated correlation coefficient r(X I , X J ) is filled into the (I, J) and (J, I) positions of the correlation coefficient matrix (I, J = 1, 2,..., m). The elements on the diagonal represent the correlation of each variable X s (s = 1, 2,..., m) with itself, which is usually 1 (because the variable is completely correlated with itself), and the matrix pattern is constructed as follows.

[0083]

[0084] Step 2.6.3, according to the correlation coefficient of each variable X s (s = 1, 2,..., m) and X miss , select the observation values of the variables whose correlation coefficient is greater than the correlation coefficient threshold value б to form a candidate set S. Taking X miss = X2 as an example, the correlation coefficients of X s (s = 1, 2,..., m) and X miss are r(X2, X1), r(X2, X m-1 ), r(X2, X m ), etc.

[0085] Step 2.6.4, fill in the missing values based on the candidate set S using the following formula, repeat the above steps for all variables containing missing values until all missing values are filled in, and finally obtain the filled data set X p :

[0086]

[0087] where x miss is the missing value to be filled, |S| is the number of candidate samples, S i is an observation value in the candidate sample.

[0088] Steps 2.6.3-2.6.4 are consistent with the method of traditional hot filling algorithm;

[0089] Step 2.7, the present application uses the max-min normalization method to normalize the filled data set X p , so that the range of data is compressed between [0, 1], the specific formula is as follows:

[0090]

[0091] where x finp is the normalized data, x min is the minimum value in the original data set, x max is the maximum value in the original data set, x p (p = 1, 2,..., P, P is the total data amount of data set X p ) refers to any value in data set X p .

[0092] Step 2.8, in order to avoid the latitude of a large amount of data being too high and making the model training effect poor, the present application uses LDA method to reduce the dimension of the normalized data set X FIN ={x fin1 , x fin2 ..., x finp}, the dimension of the reduced data set is from a x a to b x b (b < a), through this step, the problem that the current 5G network traffic data is too large to be effectively processed by the model can be well handled.

[0093] Step 2.9, crop the processed data packet, and generate a gray scale image after the data packet is uniformly lengthened.

[0094] Step 2.10, convert the picture into an idx format file, and all IDX format files constitute an initial data set Z as the input of the model.

[0095] Step 3, feature extraction is performed on the preprocessed initial data set Z based on the BIGRU-Attention model to obtain a feature data set F. The model includes a BIGRU layer for time series feature extraction and an attention layer for secondary feature extraction. The specific model structure is shown in Figure 4 FIG. 6, where n = 6 is taken as an example for illustration.

[0096] Step 4, the features F obtained in step 3 are divided into A' and B'. Based on the GAN model, A' is input into the model for training and improvement to obtain a trained model LSCGAN. The LSCGAN model training process is shown in Figure 5 FIG. 7, and the specific process is shown in steps 4.1 to 4.4. Finally, B' is input into the model to obtain a new data set F'.

[0097] Step 5, the feature set F' extracted in step 4 is input into a softmax classifier for classification to obtain an abnormal traffic classification result.

[0098] Step 5.1, the feature set F' is input into the softmax classifier for classification training to obtain a trained classifier model;

[0099] Step 5.2, the test set B is input into the trained classifier for classification of normal traffic and abnormal traffic to obtain a classification result.

[0100] Example 2

[0101] Based on example 1, the specific process of step 3 is as follows:

[0102] Step 3.1, the preprocessed data set Z is divided into a training set A and a test set B;

[0103] Step 3.2, the divided training set A = {a1, a2,..., an} is input into the BiGRU model. The calculation formula of the hidden layer state h n of the BiGRU model is as follows: t

[0104]

[0105]

[0106]

[0107] where h f t represents the forward output state obtained by inputting t time into the forward propagation layer; h b t represents the reverse output state obtained by inputting t time into the reverse propagation layer, f​d is the forward output function, b is the backward output function. It is noted that the sequence data is input into both the forward propagation layer and the backward propagation layer, and after obtaining the forward hidden state output and the backward hidden state output, the two are spliced to obtain the hidden state vector h t , a t denotes any value in the training set A, is the forward output state at the previous t time, denotes the backward output state at the next time. The present application takes h t (t = 1, 2,..., n) as the initial state of the attention mechanism layer.

[0108] Step 3.3, using the attention mechanism to extract the features h t from the BIGRU model, assigning values according to the feature importance, and finally adding the features evaluated by the feature importance weight to obtain the secondary feature extraction feature F.

[0109] Step 3.3.1, define a sliding window of size L, according to the size of the sliding window, take a corresponding number of time series feature vectors in front of and behind h t (t = 1, 2,..., n) to obtain the flow front and back data u w .

[0110] Step 3.3.2, calculate the correlation importance between the feature vector h t (t = 1, 2,..., n) and the flow front and back data u w , h t T represents the transpose of h t , d k is the dimension of h t , then use the softmax function to normalize these scores to obtain the importance weight coefficient k t between the first feature extraction vector and the data:

[0111]

[0112] Step 3.3.3, multiply all the feature vectors ht by the corresponding weight coefficient kt, and finally sum to obtain the important feature evaluated by the feature importance weight, that is, the secondary feature extraction feature F.

[0113]

[0114] Example 3

[0115] On the basis of embodiment 2, the specific process of step 4 is as follows:

[0116] Step 4.1, randomly initialize the parameters of the generator G and the discriminator D. Define Adam as the optimizer of the generator and the discriminator. Use the rand() function in the numpy library to generate Gaussian random noise data z, define the mean as 0 and the standard deviation as 1;

[0117] Step 4.2, input the noise data z and the conditional information--network traffic category y(including: normal, abnormal) into the generator network G to generate data G'(z, y) and input it into the discriminator network D, and calculate the generator loss function according to the generated data G'(z, y) and the output D(G'(z, y)) of the discriminator. Define the generation loss function of the generator network as:

[0118]

[0119] Step 4.3, input the original feature data t and the conditional information--network traffic category y into the discriminator network D together with the generated data G'(z, y) to calculate the discriminator loss. Define the discriminator loss function as:

[0120]

[0121] Step 4.4, use the stochastic gradient descent(SGD) algorithm to alternately update the parameters of the generator and the discriminator. In each iteration, first update the discriminator using the optimizer Adam, and then update the generator. Finally, an optimized data enhancement model--LSCGAN is obtained.

[0122]

[0123]

[0124] wherein θ D(t) and θ G(t) represent the parameters of the tth round of the discriminator and the generator, respectively, θ D(t-1) and θ G(t-1) represent the parameters of the (t-1)th round of the discriminator and the generator, respectively. The parameter here is not a specific single value, but a process, and are the gradients of the loss functions of the discriminator and the generator with respect to the parameters, which are obtained by the automatic differentiation function of the deep learning framework. δ is the learning rate, which is initially set to 0.001. Subsequently, if the loss function reaches stability, the learning rate can be reduced until the learning rate is reduced and the loss function reaches stability, and the training is completed.

[0125] Step 4.5. Input B' into the trained LSCGAN in the generator to get the expanded new dataset F'.

Claims

1. A deep learning-based 5G network anomaly traffic detection method, characterized in that: Specifically comprising the following steps: Step 1, constructing the initial data set X of data traffic under the 5G environment; the specific process of step 1 is: capturing the data traffic generated by normal network activities and artificially initiated network attack behaviors under the 5G environment by using wireshark software, storing in the form of pcap file, and taking all the data traffic packets generated as the initial data set X; Step 2, preprocessing the initial traffic data set X to obtain data set Z; The specific process of step 2 is: Step 2.1, using the open source tool named SplitCap to split the pcap in data set X by flow, obtaining all the packets of the same five-tuple in network traffic, and manually labeling all the packets; Step 2.2, replacing the specific IP address and MAC address in all packets segmented in step 2.1 with randomized address; Step 2.3, judging by script whether the size of the packet with randomized address replaced in step 2.2 is 0 in batches, if it is 0, the empty packet is deleted; then the script is used to calculate the hash value of each packet content, and the hash value is stored in a hash table, if there is the same hash value, it means that the two packet contents are the same, which can be marked as duplicate packet, finally output the path of all duplicate packets, and delete the duplicate packets by finddupe tool; Step 2.4, based on step 2.3, the data set after deleting empty packets and repeated packets, using Borderline SMOTE algorithm to oversample abnormal traffic, using TomeLink algorithm to undersample the data set after oversampling, to get the final sample X eq ; Step 2.5, using one-hot method to convert the features of all symbol types in dataset X eq into digital features, and finally get the numerical dataset X o ; Step 2.

6. Based on the processed dataset X of step 2.5 o For the missing numerical features, hot encoding algorithm is used to fill in, and the filled dataset X is obtained p ; Step 2.

7. Normalization of the filled dataset X using max-min normalization method p Normalization is performed so that the range of data is compressed between [0, 1] with the following formula: (1); where x finp is the normalized data, x min is the minimum value in the original data set, x max is the maximum value in the original data set, x p refers to any value in the data set X p , p = 1, 2,..., P, P is the total data amount of the data set X p ; Step 2.8: Use the normalized data x obtained in step 2.

7. finp Construct dataset X FIN For dataset X FIN ={x fin1 x fin2 ...、x finp The LDA method is used to reduce the dimensionality of the data. The dimensionality of the dataset after dimensionality reduction changes from a×a to b×b, where b <a; Step 2.9, cutting the data packets processed by dimension reduction in step 2.8 to generate gray images of uniform length; Step 2.10, converting the gray images generated in step 2.9 into idx format file to obtain data set Z; Step 3, extracting features from data set Z to obtain feature data set F; The specific process of step 3 is: Step 3.1, dividing data set Z into training set A and test set B; Step 3.2, divide the training set A = {a1, a2,..., a n} into the BiGRU model, and the calculation formula of the hidden layer state h t of the BiGRU model is: (2); (3); (4); wherein h f t denotes the forward output state at time t obtained by inputting through the forward propagation layer; h b t denotes the backward output state at time t obtained by inputting through the backward propagation layer, f d is a forward output function, f b is a backward output function, h t is a hidden state vector, a t denotes any value in the training set A, is the forward output state at the previous time t, denotes the backward output state at the next time. Step 3.

3. Extracting features from the BIGRU model using the Attention mechanism According to the feature importance, the features after the feature importance weight evaluation are added, and the secondary feature extraction feature F is obtained. Step 4, dividing the features F obtained in step 3 into data set A' and data set B', inputting data set A' into GAN model for training to obtain model LSCGAN, and putting data set B' into model LSCGAN to obtain data set F'; Step 5, putting the feature set F' extracted in step 4 into the softmax classifier for classification to obtain the classification result of abnormal traffic. 2.The deep learning based 5G network abnormal traffic detection method of claim 1, wherein: The specific process of step 4 is: Step 4.1, randomly initializing the parameters of generator G and discriminator D, defining Adam as the optimizer of generator and discriminator, using the rand() function in numpy library to generate Gaussian random noise data z, defining the mean as 0 and the standard deviation as 1; Step 4.2, input the noise data z and the condition information - network traffic category y into the generator network G to generate data G'(z, y), input G'(z, y) into the discriminator network D, and calculate the generator loss function according to the generated data G'(z, y) and the output D(G'(z, y)) of the discriminator, define the generation loss function of the generator network is: (5); Step 4.3, the original feature data t and the condition information - network traffic class y are input into the discriminator network D to jointly calculate the discriminator loss with the generated data G'(z, y), and the discriminant loss function is defined as: is: (6); Step 4.4, using the stochastic gradient descent algorithm to alternately update the parameters of the generator and the discriminator to minimize their respective loss functions, in each iteration, using the optimizer Adam to update the discriminator first, then updating the generator, finally obtaining the trained data enhancement model--LSCGAN: (7); (8); wherein, and and and and and and is the learning rate. Step 4.5, inputting data set B' into the trained model LSCGAN to obtain the expanded new data set F'.

Citation Information

Patent Citations

  • Unsupervised intrusion traffic detection and identification method based on generative adversarial network

    CN115277189A