Abnormal network traffic detection method, system and device based on deep learning, and medium

Abstract

The invention provides an abnormal network flow detection method, system and device based on deep learning and a medium, and relates to the technical field of network security protection, and the method comprises the steps: setting a discrete data processing rule; capturing network traffic data, carrying out data splitting and length trimming, and then converting the data into a network traffic grayscale image; constructing and training a flow detection model by using a convolutional neural network; and inputting the network traffic grayscale image into the traffic detection model, and outputting the network traffic anomaly probability. According to the invention, the network flow data monitoring efficiency can be improved, the abnormal flow identification precision can be improved, the false report and missing report conditions of the abnormal flow can be reduced, the dependence on human resources can be reduced, and the overall performance and adaptability of an abnormal network flow detection system can be improved.

Description

Background Art

[0002] With the rapid development of network technology, the scale of network traffic has grown exponentially, and network attack means have become increasingly complex. New threats such as DDoS attacks and malware propagation emerge in an endless stream. Traditional security protection means are difficult to cope with the dynamically changing network environment, and there is an urgent need for an efficient and adaptive abnormal traffic detection technology to improve network security protection capabilities.

[0003] Existing technologies mostly use rule matching or shallow machine learning models (such as support vector machines, random forests) for abnormal traffic detection. The specific process is as follows: By manually defining features (such as traffic packet size, protocol type, port number, etc.), extracting the statistical characteristics of network traffic, and then inputting them into a classification model for abnormal determination. Some studies attempt to convert traffic data into time series signals and use recurrent neural networks (RNNs) or long short-term memory networks (LSTMs) to extract time series features.

[0004] However, the existing methods for abnormal traffic detection based on rule matching or shallow machine learning models often rely on manual feature extraction, making it difficult to capture deep associated features in traffic data, resulting in low detection accuracy, high false alarm rates. Among them, traditional machine learning models have insufficient generalization ability and cannot adapt to complex and changing network environments and unknown attack patterns; the methods based on time series models have low processing efficiency for unstructured traffic data and are difficult to meet the real-time detection requirements. Summary of the Invention

[0005] Aiming at the technical problems that the existing methods for abnormal traffic detection based on rule matching or shallow machine learning models rely on manual feature extraction, are difficult to capture deep associated features in traffic data, have low detection accuracy, high false alarm rates, insufficient generalization ability, and low data processing efficiency, the present invention provides a deep learning-based abnormal network traffic detection method, system, device, and medium, which can improve the monitoring efficiency of network traffic data, improve the recognition accuracy of abnormal traffic, reduce false alarms and missed reports of abnormal traffic, reduce the dependence on human resources, and improve the overall performance and adaptability of the abnormal network traffic detection system.

[0006] In the first aspect, the present invention provides a deep learning-based abnormal network traffic detection method, and the steps include: S1. Set discrete data processing rules, and the discrete data processing rules are to perform one-hot encoding on discrete data to generate continuous feature vectors; S2. Capture network traffic data and perform data splitting. Split the network traffic data into short flow data files, trim the lengths of the short flow data files, and after unifying the lengths of the short flow data files, convert all short flow data files belonging to the same network traffic data into a network traffic grayscale image; Among them, after capturing the network traffic data, discrete data in the network traffic data is processed using discrete data processing rules, and then data splitting is performed. S3. Use a convolutional neural network to construct and train a traffic detection model, where the traffic detection model includes an input module, a feature extraction module, and a classification decision module. The feature extraction module includes at least two serially connected feature extraction units, and each feature extraction unit contains an attention module, a residual convolution module, and a skip connection. The classification decision module includes a pooling layer and a classification layer, and the classification layer includes a fully connected layer and an activation function. The output of the traffic detection model is the probability of network traffic anomaly , [0, 1]; S4. Input the network traffic grayscale image into the traffic detection model to output the probability of network traffic anomaly.

[0007] It should be further noted that in step S1, the discrete data includes protocol type, port number, and traffic attributes encoded as integers.

[0008] It should be further noted that the specific operation of step S2 is as follows: S201. Use a packet capture function to capture network traffic data, process the discrete data in the network traffic data using discrete data processing rules, and then save the network traffic data as a pcap file. S202. Use a splitting tool to split the pcap file of the network traffic data in the form of a five-tuple, and split the pcap file of the network traffic data into short flow data files. The five-tuple includes source IP, source port, destination IP, destination port, and protocol type. S203. Trim the length of the short flow data files to make the number of bytes of all short flows the same, discard the extra bytes, and fill in the insufficient bytes with zeros. S204. Convert each byte in the short flow data file into a pixel, where the grayscale value of each pixel is 0 - 255. The higher the byte value, the higher the corresponding grayscale value, and the lower the byte value, the lower the corresponding grayscale value; finally, convert all short flow data files belonging to the same network traffic data into a network traffic grayscale image.

[0009] It should be further noted that the splitting tool includes one of splitcap tool and editcap tool.

[0010] It should be further noted that in the feature extraction unit of step S3, the attention mechanism description formula of the attention module is:

[0011] Wherein:

[0012] In the formula, , , are the linear mapping parameters of the input vector, and their function is to perform feature mapping on the original input vector; represents an alignment function, which is used to measure and the similarity between them; is used to convert the similarity into a weight form of 0 - 1; is the output of the attention module; The convolutional residual module includes three convolutional layers. The output of the attention module is used as the input of the first convolutional layer, and the input of the remaining convolutional layers is the output of the previous convolutional layer. The total expression of the three convolutional layers is:

[0013] In the formula, C is the output of the third convolutional layer; The skip connection is to directly add the input of the feature extraction unit to the output of the third convolutional layer as the output of the feature extraction unit to achieve identity mapping. The expression is: .

[0014] It should be further noted that the activation function is the Sigmoid function.

[0015] It should be further noted that it further includes step S5: determining whether the network traffic data belongs to abnormal network traffic through an artificially set abnormal probability threshold. The determination criterion is: if the network traffic abnormal probability corresponding to the network traffic data is greater than or equal to the abnormal probability threshold, then the network data traffic belongs to abnormal network traffic.

[0016] It should be further noted that the abnormal probability threshold is 0.5.

[0017] In the second aspect, the present invention provides an abnormal network traffic detection system based on deep learning for implementing the above-mentioned abnormal network traffic detection method based on deep learning, including: A data processing rule setting module, which is used to set discrete data processing rules; A network traffic data capture module, which is used to capture network traffic data; A data processing module, which is used to process discrete data in network traffic data using discrete data processing rules, and split the network traffic data into short flow data files, trim the lengths of the short flow data files to make their lengths uniform, and then convert all short flow data files belonging to the same network traffic data into a network traffic grayscale image; A model construction and training module, which is used to construct and train a traffic detection model using a convolutional neural network; A detection module, which is used to input the network traffic grayscale image into the traffic detection model and output the network traffic anomaly probability.

[0018] In a third aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor is used to implement the steps of the above-mentioned deep learning-based abnormal network traffic detection method when executing the computer program.

[0019] In a fourth aspect, the present invention provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned deep learning-based abnormal network traffic detection method are implemented.

[0020] The beneficial effects of the present invention are as follows: 1. For the deep learning-based abnormal network traffic detection method provided by the present invention, after setting the discrete data processing rules, network traffic data is captured and data splitting and length trimming are performed. The network traffic data is converted into a network traffic grayscale image, and then a traffic detection model is constructed and trained using a convolutional neural network. The network traffic grayscale image is input into the traffic detection model, and the network traffic anomaly probability is output. By utilizing the powerful feature extraction ability of the convolutional neural network, deep and complex feature patterns are mined from the visualized network traffic data, thereby improving the monitoring efficiency of network traffic data, enhancing the recognition accuracy of abnormal traffic, and reducing false alarms and missed alarms of abnormal traffic.

[0021] 2. The present invention uses a convolutional neural network to construct and train a traffic detection model for detecting network traffic data. The trained traffic detection model has strong robustness, can adapt to different network environments and different types of network traffic data, and can effectively detect unknown abnormal behavior patterns, which can improve the overall performance and adaptability of the abnormal network traffic detection system.

[0022] 3. Compared with the existing methods for detecting abnormal traffic based on rule matching or shallow machine learning models, the present invention can significantly reduce the dependence on human resources, reduce the burden on operation and maintenance personnel, and thus reduce the overall operation and maintenance costs.

[0023] 4. The present invention can promptly detect and respond to abnormal behaviors in the network, such as DDoS attacks, malware propagation, etc., effectively contain the spread of network threats, protect the security of user data and systems, and enhance the overall security protection level of the network.

[0024] 5. The configuration and parameters of the traffic detection model of the present invention can be adjusted according to the actual needs of users and the characteristics of the network environment, enabling the abnormal network traffic detection system to more flexibly adapt to different application scenarios and requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the present invention, the accompanying drawings required for description will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0026] Figure 1 is a flowchart of a method for detecting abnormal network traffic based on deep learning in an embodiment of the present invention.

[0027] Figure 2 is a grayscale image of network traffic formed according to different types of network traffic in an embodiment of the present invention.

[0028] Figure 3 is a schematic block diagram of an abnormal network traffic detection system based on deep learning in an embodiment of the present invention.

[0029] Figure 4 is a schematic diagram of the hardware structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] In order to make the objectives, features, and advantages of the present invention more obvious and understandable, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the specific embodiments of the present invention. Obviously, the embodiments described below are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0031] The abnormal network traffic detection method based on deep learning involved in this application mainly aims at the field of network security protection technology. The technical solution includes setting discrete data processing rules; capturing network traffic data, splitting and trimming the length of the data, and then converting it into a network traffic grayscale image; using a convolutional neural network to construct and train a traffic detection model; inputting the network traffic grayscale image into the traffic detection model to output the abnormal probability of the network traffic. Through setting discrete data processing rules, capturing network traffic data, splitting and trimming the length of the data, converting the network traffic data into a network traffic grayscale image, then using a convolutional neural network to construct and train a traffic detection model, and inputting the network traffic grayscale image into the traffic detection model to output the abnormal probability of the network traffic. By utilizing the powerful feature extraction ability of the convolutional neural network, deep and complex feature patterns are mined from the visualized network traffic data, thereby improving the monitoring efficiency of network traffic data, enhancing the recognition accuracy of abnormal traffic, reducing false alarms and missed alarms of abnormal traffic; using a convolutional neural network to construct and train a traffic detection model for detecting network traffic data, the trained traffic detection model has strong robustness, can adapt to different network environments and different types of network traffic data, and can effectively detect unknown abnormal behavior patterns, which can improve the overall performance and adaptability of the abnormal network traffic detection system; compared with the existing methods for detecting abnormal traffic based on rule matching or shallow machine learning models, this invention can significantly reduce the dependence on human resources, reduce the burden on operation and maintenance personnel, and thus reduce the overall operation and maintenance cost; it can timely discover and respond to abnormal behaviors in the network, such as DDoS attacks, malware propagation, etc., effectively contain the spread of network threats, protect the security of user data and systems, and improve the overall security protection level of the network; the configuration and parameters of the traffic detection model can be adjusted according to the actual needs of users and the characteristics of the network environment, enabling the abnormal network traffic detection system to more flexibly adapt to different application scenarios and requirements.

[0032] The abnormal network traffic detection method based on deep learning involved in this application mainly aims at the technical problems of the existing methods for detecting abnormal traffic based on rule matching or shallow machine learning models, which rely on manual feature extraction, are difficult to capture deep - level correlation features in traffic data, have low detection accuracy, high false alarm rate, insufficient generalization ability, and low data processing efficiency.

[0033] The abnormal network traffic detection method based on deep learning involved in this application will be described in detail below. For the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are proposed to thoroughly understand the embodiments of this application. However, those skilled in the art should clearly understand that this application can also be implemented in other embodiments without these specific details.

[0034] In the method for detecting abnormal network traffic based on deep learning involved in this application, the term "including" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations. The terms "including", "comprising", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0035] For the convenience of clearly describing the technical solutions of this application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. do not necessarily limit to being different.

[0036] The statements such as "in one embodiment" or "in some embodiments" described in this application mean that the specific features, structures, or characteristics described in the embodiment are included in one or more embodiments of this application. Thus, the statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" and the like that appear in different places in this application do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0037] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0038] The method for detecting abnormal network traffic based on deep learning provided by the embodiments of the present invention is executed by a computer device. Correspondingly, the system for detecting abnormal network traffic based on deep learning runs in the computer device.

[0039] Figure 1 is a flowchart of the method for detecting abnormal network traffic based on deep learning according to an embodiment of the present invention. Among them, Figure 1 The execution subject can be a system for detecting abnormal network traffic based on deep learning. According to different requirements, the order of the steps in this flowchart can be changed, and some can be omitted.

[0040] As Figure 1 shown, the method for detecting abnormal network traffic based on deep learning includes: Step S1, set the discrete data processing rule, where the discrete data processing rule is to perform one-hot encoding on the discrete data to generate a continuous feature vector.

[0041] Set the discrete data processing rule, perform one-hot encoding on the discrete data to generate a continuous feature vector, making the data more suitable for processing by deep learning models, capable of retaining the original data features, improving the usability of the data, laying a foundation for subsequent accurate analysis of network traffic data, and enhancing the accuracy of feature extraction.

[0042] In some specific embodiments, the discrete data includes protocol type, port number, and traffic attributes encoded as integers.

[0043] Determine the discrete data range, which can provide a clear basis for subsequent data processing, ensure the integrity and accuracy of data processing, contribute to more precisely extracting features from network traffic data, and thus improve the accuracy of abnormal traffic detection.

[0044] Step S2, capture network traffic data and perform data splitting. Split the network traffic data into short flow data files, trim the lengths of the short flow data files, and after unifying the lengths of the short flow data files, convert all short flow data files belonging to the same network traffic data into a network traffic grayscale image; Among them, after capturing the network traffic data, use the discrete data processing rule to process the discrete data in the network traffic data, and then perform data splitting.

[0045] Convert the network traffic data into an image form suitable for processing by a convolutional neural network, which can make full use of the advantages of the convolutional neural network in image feature extraction, mine deep and complex feature patterns, thereby improving the network traffic data monitoring efficiency and enhancing the recognition accuracy of abnormal traffic.

[0046] In some specific embodiments, the specific operations of Step S2 are as follows: S201. Use a packet capture function to capture network traffic data, use the discrete data processing rule to process the discrete data in the network traffic data, and then save the network traffic data as a pcap file; S202. Use a splitting tool to split the pcap file of the network traffic data in the form of a five-tuple, and split the pcap file of the network traffic data into short flow data files. The five-tuple includes source IP, source port, destination IP, destination port, and protocol type; S203. Trim the lengths of the short flow data files to make the number of bytes of all short flows the same, discard the extra bytes, and fill the insufficient bytes with zeros; S204. Convert each byte in the short flow data file into a pixel, with the grayscale value of each pixel ranging from 0 to 255. The higher the byte value, the higher the corresponding grayscale value; the lower the byte value, the lower the corresponding grayscale value. Finally, convert all the short flow data files belonging to the same network traffic data into a network traffic grayscale image.

[0047] Through the standardized operation process of capturing, splitting, trimming, and converting network traffic data into grayscale images, the standardization and consistency of data processing can be ensured, enabling the data to better adapt to the subsequent processing of convolutional neural networks, improving the quality of the input data of the detection model, and thus enhancing the reliability of abnormal traffic detection.

[0048] In some specific embodiments, the splitting tool includes one of the splitcap tool and the editcap tool.

[0049] Specifying the scope of the splitting tool provides a specific implementation method for the data splitting operation, facilitating technicians to select an appropriate tool for data processing, ensuring the efficient progress of the data splitting work, and contributing to improving the efficiency of the entire abnormal traffic detection process.

[0050] Step S3, use a convolutional neural network to construct and train a traffic detection model, where the traffic detection model includes an input module, a feature extraction module, and a classification decision module; The feature extraction module includes at least two serially connected feature extraction units, and each feature extraction unit contains an attention module, a residual convolution module, and a skip connection; The classification decision module includes a pooling layer and a classification layer, and the classification layer includes a fully connected layer and an activation function; The output of the traffic detection model is the probability of network traffic abnormality , [0,1] .

[0051] In some specific embodiments, in the feature extraction unit, the attention mechanism description formula of the attention module is:

[0052] Where:

[0053] In the formula, , , are the linear mapping parameters of the input vector, and their function is to perform feature mapping on the original input vector; represents an alignment function, which is used to measure and the similarity between them; Used to convert similarity into a weight form of 0-1; Is the output of the attention module; The convolutional residual module includes three convolutional layers. The output of the attention module is used as the input of the first convolutional layer, and the input of the remaining convolutional layers is the output of the previous convolutional layer. The total expression of the three convolutional layers is:

[0054] In the formula, C is the output of the third convolutional layer; The skip connection is to directly add the input of the feature extraction unit to the output of the third convolutional layer as the output of the feature extraction unit to achieve identity mapping. The expression is: .

[0055] By giving the formulas of the attention module, convolutional residual module and skip connection, and explaining in detail the working principle of the key components inside the model, it helps to deeply understand the feature extraction and processing mechanism of the model, provides theoretical support for optimizing the model structure and improving the model performance, and enables the detection model to more effectively mine the deep features in the network traffic data.

[0056] In some specific embodiments, the activation function is the Sigmoid function.

[0057] Sigmoid in the model can map the output to the interval [0,1], which is convenient to intuitively represent the probability of network traffic anomalies, and its characteristics contribute to the training and convergence of the model, improving the stability and detection accuracy of the model.

[0058] In some specific embodiments, it further includes step S5: determining whether the network traffic data belongs to abnormal network traffic through an artificially set anomaly probability threshold. The determination criterion is: if the network traffic anomaly probability corresponding to the network traffic data is greater than or equal to the anomaly probability threshold, then the network data traffic belongs to abnormal network traffic.

[0059] Determining abnormal network traffic by setting an anomaly probability threshold provides a clear criterion for the judgment of abnormal traffic, makes the detection result operable, facilitates the timely discovery and handling of abnormal traffic, and enhances the network security protection ability.

[0060] In some specific embodiments, the anomaly probability threshold is 0.5.

[0061] Among them, the attention module in the feature extraction module can automatically focus on key features and enhance the weights of important features; the residual convolution module gradually extracts features at different levels through multiple convolutional layers, which can deepen the mining of data features; the skip connection can avoid the problem of gradient disappearance, ensure the stability and accuracy of model training, and enable the model to learn the features of network traffic data more comprehensively and accurately; the pooling layer can reduce the data dimension and computational amount, the fully connected layer comprehensively processes features, and the activation function outputs the probability of network traffic anomalies, which is convenient for intuitively judging whether the traffic is abnormal and can improve the interpretability and practicality of the detection results; the trained model has strong robustness, can adapt to different network environments and types of network traffic data, effectively detect unknown abnormal behavior patterns, and improve the overall performance and adaptability of the abnormal network traffic detection system.

[0062] Step S4, input the network traffic grayscale image into the traffic detection model, and output the probability of network traffic anomalies.

[0063] Based on the feature learning and classification capabilities of the traffic detection model, it can quickly and accurately judge the abnormal situation of network traffic, timely discover and respond to abnormal behaviors in the network, protect the security of user data and systems, and improve the overall security protection level of the network.

[0064] In a specific embodiment, the abnormal network traffic detection method based on deep learning includes: Step S1, set the discrete data processing rule, and the discrete data processing rule is to perform one-hot encoding on discrete data to generate a continuous feature vector. The discrete data includes protocol type, port number, and traffic attributes encoded as integers; Step S2, capture network traffic data and perform data splitting. Split the network traffic data into short flow data files, trim the lengths of the short flow data files, and after unifying the lengths of the short flow data files, convert all short flow data files belonging to the same network traffic data into a network traffic grayscale image. The network traffic grayscale images formed according to different types of network traffic are as Figure 2 shown; Among them, after capturing the network traffic data, use the discrete data processing rule to process the discrete data in the network traffic data, and then perform data splitting; The specific operation is: S201. Use the packet capture function to capture network traffic data, use the discrete data processing rule to process the discrete data in the network traffic data, and then save the network traffic data as a pcap file; S202. Use a splitting tool to split the pcap file of network traffic data in the form of five-tuples, and split the pcap file of network traffic data into short flow data files. The five-tuples include source IP, source port, destination IP, destination port, and protocol type. The splitting tool includes one of the splitcap tool and the editcap tool.

[0065] S203. Trim the length of the short flow data files to make the number of bytes of all short flows the same. Discard the extra bytes and pad the insufficient bytes with zeros. S204. Convert each byte in the short flow data file into a pixel, and the grayscale value of each pixel is 0 - 255. The higher the byte value, the higher the corresponding grayscale value, and the lower the byte value, the lower the corresponding grayscale value. Finally, convert all short flow data files belonging to the same network traffic data into a network traffic grayscale image. Step S3. Use a convolutional neural network to construct and train a traffic detection model. The traffic detection model includes an input module, a feature extraction module, and a classification decision module. The feature extraction module includes four serially connected feature extraction units, and each feature extraction unit contains an attention module, a residual convolution module, and a skip connection. In the feature extraction unit, the attention mechanism description formula of the attention module is:

[0066] Where:

[0067] In the formula, , , are the linear mapping parameters of the input vector, and their function is to perform feature mapping on the original input vector. represents an alignment function, which is used to measure the similarity between and . is used to convert the similarity into a weight form of 0 - 1. is the output of the attention module. The convolutional residual module includes three convolutional layers, and the configuration of each layer is as follows: The first layer: 3×3 convolution, 64 filters, stride 1, and the activation function is ReLU. The second layer: 3×3 convolution, 64 filters, stride 1, and the activation function is ReLU. The third layer: 1×1 convolution, 64 filters, stride 1, which is used to adjust the number of channels. Use the output of the attention module as the input of the first convolutional layer, and the input of the remaining convolutional layers is the output of the previous convolutional layer. The total expression of the three convolutional layers is:

[0068] In the formula, C is the output of the third convolutional layer; The skip connection is to directly add the input of the feature extraction unit to the output of the third convolutional layer as the output of the feature extraction unit to achieve the identity mapping. The expression is: ; The classification decision module includes a pooling layer and a classification layer. The pooling layer performs average pooling on the output of the feature extraction unit. The classification layer includes a fully connected layer and an activation function, and the activation function is the Sigmoid function; The output of the traffic detection model is the probability of network traffic anomaly , [0,1] ; The training steps of the traffic detection model are as follows: S301. Use the method in step S2 to obtain a large number of network traffic grayscale images to form a network traffic grayscale image set. The network traffic grayscale image set contains network traffic grayscale images corresponding to normal traffic and abnormal traffic. Abnormal traffic includes network traffic generated when DDoS and / or port scanning occur; Perform random data augmentation operations on the network traffic grayscale images in the network traffic grayscale image set. The data augmentation operations include horizontal flipping, ±10% translation, and Gaussian noise injection; Divide the randomly augmented network traffic grayscale image set into a training set, a validation set, and a test set according to 7:2:1; S302. Set the optimizer to AdamW, the initial learning rate to 1×10 -4 , and the weight decay to 1×10 −5 ; Set the loss function to binary cross-entropy loss. The expression of the loss function is:

[0069] Set the batch size to 128, the training period to 100, and set the early stopping mechanism to terminate the training if the validation set loss does not decrease for 5 consecutive times; The learning rate scheduling adopts the cosine annealing strategy, with a period of 10 epochs and a minimum learning rate of 1×10 -6 ; S303. Initialize the model, input the network traffic grayscale images in the training set, and calculate layer by layer through the four feature extraction units in the feature extraction module to output the anomaly probability; S304. Calculate the loss using the loss function and update the parameters of the feature extraction unit, and set the threshold for gradient clipping to 5.0 during the process; S305. Evaluate the F1 score on the validation set after each epoch, and save the traffic detection model with the best evaluation result as the best model after all epochs are completed; S306. Load the best model, calculate the accuracy, recall rate, and AUC-ROC on the test set. If the artificially set threshold is not met, repeat steps S303 - S306 until the artificially set threshold is met and then stop to obtain the trained traffic detection model; Step S4. Input the grayscale image of network traffic into the traffic detection model to output the probability of network traffic anomaly; Step S5. Determine whether the network traffic data belongs to abnormal network traffic through the artificially set anomaly probability threshold. The anomaly probability threshold is 0.5. The determination criterion is: if the network traffic anomaly probability corresponding to the network traffic data is greater than or equal to the anomaly probability threshold, then the network data traffic belongs to abnormal network traffic.

[0070] The following is an embodiment of the abnormal network traffic detection system based on deep learning provided by the present disclosure. This active load shedding optimization system and the abnormal network traffic detection method based on deep learning in the above embodiments belong to the same inventive concept. For the details not described in detail in the embodiment of the abnormal network traffic detection system based on deep learning, reference can be made to the embodiments of the abnormal network traffic detection method based on deep learning.

[0071] Now, the mobile terminal implementing various embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, suffixes such as "module", "component", or "unit" used to represent elements are only for the convenience of description of the embodiments of the present invention, and they have no specific meaning by themselves. Therefore, "module" and "component" can be used interchangeably.

[0072] As Figure 3 shown, the abnormal network traffic detection system based on deep learning includes: A data processing rule setting module for setting discrete data processing rules; A network traffic data capture module for capturing network traffic data; A data processing module for processing the discrete data in the network traffic data using the discrete data processing rules, splitting the network traffic data into short flow data files, trimming the lengths of the short flow data files to make their lengths uniform, and then converting all the short flow data files belonging to the same network traffic data into a network traffic grayscale image; A model construction and training module for constructing and training a traffic detection model using a convolutional neural network; A detection module for inputting a grayscale image of network traffic into a traffic detection model and outputting the probability of network traffic anomaly.

[0073] The abnormal network traffic detection system of this embodiment is used to implement an abnormal network traffic detection method based on deep learning. The steps include: S1. Set the discrete data processing rule, which is to perform one-hot encoding on discrete data to generate a continuous feature vector; S2. Capture network traffic data and perform data splitting. Split the network traffic data into short flow data files, trim the lengths of the short flow data files to make their lengths uniform, and then convert all short flow data files belonging to the same network traffic data into a grayscale image of network traffic; Among them, after capturing the network traffic data, use the discrete data processing rule to process the discrete data in the network traffic data, and then perform data splitting; S3. Use a convolutional neural network to construct and train a traffic detection model. The traffic detection model includes an input module, a feature extraction module, and a classification decision module; The feature extraction module includes at least two serially connected feature extraction units, and each feature extraction unit includes an attention module, a residual convolution module, and a skip connection; The classification decision module includes a pooling layer and a classification layer. The classification layer includes a fully connected layer and an activation function; The output of the traffic detection model is the probability of network traffic anomaly , [0,1] ; S4. Input the grayscale image of network traffic into the traffic detection model and output the probability of network traffic anomaly.

[0074] This application also provides an electronic device for implementing each embodiment of the present invention. The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor.

[0075] Those skilled in the art can understand that the structure of the electronic device involved in the embodiments of the present invention does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0076] Figure 4 A schematic diagram of the hardware structure of an electronic device for implementing each embodiment of the present invention.

[0077] The electronic device includes, but is not limited to, components such as a processor and a memory. Those skilled in the art can understand that the structure of the electronic device involved in the embodiments of the present invention does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0078] In the embodiments of the present invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described herein and / or claimed.

[0079] In the embodiments of the present application, the processor may be implemented by using at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a processor, a controller, a microcontroller, a microprocessor, and an electronic unit designed to perform the functions described herein. In some cases, such an implementation may be implemented in the controller. For a software implementation, an implementation of a process or function may be implemented with a separate software module that allows performing at least one function or operation. The software code may be implemented by a software application (or program) written in any suitable programming language. The software code may be stored in the memory and executed by the controller.

[0080] In addition, the electronic device includes some functional modules not shown herein, which will not be elaborated further.

[0081] Those skilled in the art to which the present application pertains can understand that various aspects of the electronic device provided by the present application can be implemented as a system, a method, or a program product. Therefore, the various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to herein as "circuitry", "module", or "system".

[0082] The present application also provides a storage medium in which a program product capable of implementing the method for detecting abnormal network traffic based on deep learning is stored. In some possible implementation manners, various aspects of the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments described in the above "Exemplary Method" section of this specification.

[0083] The storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0084] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting abnormal network traffic based on deep learning, characterized in that the steps include: S1. Set discrete data processing rules. The discrete data processing rules are to perform one-hot encoding on discrete data to generate continuous feature vectors. S2. Capture network traffic data and split the data into short flow data files, trim the length of the short flow data files to make the length of the short flow data files uniform, and convert all short flow data files belonging to the same network traffic data into a network traffic grayscale image; After capturing the network traffic data, discrete data in the network traffic data is processed using discrete data processing rules, and then the data is split; S3. Use a convolutional neural network to build and train a traffic detection model, where the traffic detection model includes an input module, a feature extraction module, and a classification decision module; The feature extraction module includes at least two feature extraction units connected in series, each of which includes an attention module, a residual convolution module and a skip connection; The classification decision module includes a pooling layer and a classification layer. The classification layer includes a fully connected layer and an activation function. The output of the traffic detection model is the probability of network traffic anomaly , [0,1]; S4. Input the network traffic grayscale image into the traffic detection model and output the network traffic anomaly probability.

2. The abnormal network traffic detection method according to claim 1, characterized in that: In step S1, the discrete data includes the protocol type, the port number and the traffic attributes encoded in integers.

3. The abnormal network traffic detection method according to claim 1, characterized in that: The specific operations of step S2 are: S201 uses the packet capture function to capture network traffic data, uses discrete data processing rules to process discrete data in the network traffic data, and then saves the network traffic data as a pcap file; S202 using split tools, the network traffic data pcap file is split in five-tuple mode, the network traffic data pcap file is split into short stream data files, five-tuple includes source IP, source port, destination IP, destination port, protocol type; S203. The length of the short stream data file is trimmed so that the number of bytes of all short streams is consistent, the excess bytes are discarded, and the insufficient bytes are filled with zeros; S204. Convert each byte in the short flow data file into a pixel, and the grayscale value of each pixel is 0-255. The higher the byte value, the higher the corresponding grayscale value, and the lower the byte value, the lower the corresponding grayscale value; finally, convert all the short flow data files belonging to the same network traffic data into a network traffic grayscale image.

4. The abnormal network traffic detection method according to claim 3, characterized in that: The splitting tool includes one of the splitcap tool and the editcap tool.

5. The abnormal network traffic detection method according to claim 1, characterized in that: In the feature extraction unit of step S3, the attention mechanism description formula of the attention module is: in: In the formula, , , is the linear mapping parameter of the input vector, which is used to perform feature mapping on the original input vector; Represents the alignment function, which is used to measure and similarities between; Used to convert similarity into a weighted form of 0-1; is the output of the attention module; The convolution residual module consists of three convolutional layers. The output of the attention module is used as the input of the first convolutional layer, and the input of the remaining convolutional layers is the output of the previous convolutional layer. The total expression of the three convolutional layers is: Where C is the output of the third convolutional layer; The skip connection directly adds the input of the feature extraction unit to the output of the third convolutional layer as the output of the feature extraction unit to achieve an identity mapping, which is expressed as: 。 6. The abnormal network traffic detection method according to claim 5, characterized in that: The activation function is the Sigmoid function.

7. The abnormal network traffic detection method according to claim 1, characterized in that: It also includes step S5: determining whether the network traffic data belongs to abnormal network traffic through an artificially set abnormal probability threshold, and the determination standard is: if the network traffic abnormal probability corresponding to the network traffic data is greater than or equal to the abnormal probability threshold, then the network data traffic belongs to abnormal network traffic.

8. An abnormal network traffic detection system based on deep learning, characterized in that: The method for detecting abnormal network traffic according to any one of claims 1 to 7 comprises: A data processing rule setting module is used to set discrete data processing rules; A network traffic data capture module, used for capturing network traffic data; A data processing module, used for processing discrete data in network flow data using discrete data processing rules, splitting the network flow data into short flow data files, trimming the length of the short flow data files to make the lengths of the short flow data files uniform, and converting all short flow data files belonging to the same network flow data into a network flow grayscale image; Model building and training module, used to build and train traffic detection models using convolutional neural networks; The detection module is used to input the network traffic grayscale image into the traffic detection model and output the network traffic anomaly probability.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the steps of the abnormal network traffic detection method as described in any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that: A computer program is stored on the storage medium, and when the computer program is executed by the processor, the steps of the abnormal network traffic detection method as described in any one of claims 1 to 7 are implemented.

Images