Method and device for analyzing network traffic, electronic equipment and storage medium

Abstract

The invention relates to a method and a device for analyzing network traffic, electronic equipment and a storage medium. The method comprises the following steps: acquiring to-be-analyzed network traffic data; the to-be-analyzed network traffic data is input into the deep learning network to obtain an analysis result, the training data set comprises pre-acquired normal network traffic data and / or abnormal network traffic data, the high-level features are extracted from original fields in the pre-acquired normal network traffic data and / or abnormal network traffic data, and the high-level features are extracted from the original fields in the pre-acquired normal network traffic data and / or abnormal network traffic data; the analysis result comprises at least one of the following items: a statistical feature, a protocol feature, a time feature and a content feature, and the analysis result comprises the content of a field corresponding to the high-level feature of the to-be-analyzed network flow data and the protocol type of the to-be-analyzed network flow data. The problems that in the prior art, new and complex attack behaviors cannot be effectively handled, manual writing and maintenance of rules and signatures are tedious and time-consuming, and the maintenance cost is high can be solved.

Description

Technical Field

[0001] The present disclosure relates to the field of network security, and in particular to a method, device, electronic device and storage medium for parsing network traffic. Background Art

[0002] In the field of network security, traditional intrusion detection and traffic analysis methods usually rely on rules and signatures. These methods detect and analyze abnormal behaviors in network traffic through predefined rules and known attack patterns. However, with the continuous evolution of network attack methods, rule-based and signature-based methods have gradually exposed their limitations. On the one hand, these methods cannot effectively deal with emerging and complex attack behaviors; on the other hand, manually writing and maintaining rules and signatures is a tedious and time-consuming task with high maintenance costs. Summary of the invention

[0003] The method, device, electronic device and storage medium for parsing network traffic provided by the exemplary embodiments of the present disclosure can at least solve the above technical problems and other technical problems not mentioned above.

[0004] According to one aspect of the present disclosure, a method for parsing network traffic data is provided, the method comprising: obtaining network traffic data to be parsed, wherein the network traffic data to be parsed comprises normal network traffic data and / or abnormal network traffic data; inputting the network traffic data to be parsed into a deep learning network to obtain a parsing result, wherein the deep learning network is trained based on high-level features extracted from a pre-acquired training data set, the training data set comprises pre-acquired normal network traffic data and / or abnormal network traffic data, the high-level features are extracted from original fields in the pre-acquired normal network traffic data and / or abnormal network traffic data, and comprise at least one of the following items: statistical features, protocol features, time features, and content features, wherein the parsing result comprises the content of the field corresponding to the high-level features of the network traffic data to be parsed and the protocol type of the network traffic data to be parsed.

[0005] Optionally, the abnormal network traffic data is obtained by deleting and / or replacing at least one field in the normal network traffic data.

[0006] Optionally, the training data set is obtained by cleaning and diversifying the pre-acquired normal network traffic data and / or abnormal network traffic data, wherein the cleaning includes at least one of the following items: removing noise, removing irrelevant data, and removing data with a classification sample number greater than a first preset threshold; the diversification includes at least one of the following items: performing an undersampling operation on data of a protocol type whose sample size is greater than a second preset threshold, performing an oversampling operation on data of a protocol type whose sample size is less than the second preset threshold; and the pre-acquired abnormal network traffic data is obtained by deleting and / or replacing at least one field in the pre-acquired normal network traffic data.

[0007] Optionally, the pre-acquired normal network traffic data and / or abnormal network traffic data included in the training data set is collected from multiple network nodes and devices.

[0008] Optionally, the training data set is obtained by converting the format of the pre-acquired normal network traffic data and / or abnormal network traffic data into a parquent format.

[0009] Optionally, the statistical features include at least one of the following items: data packet length, inter-packet time, and the protocol features include at least one of the following items: TCP flag, HTTP header information, IP address, MAC address.

[0010] Optionally, the deep learning network is a Transformer-based T5-large model.

[0011] According to another aspect of the present disclosure, a device for parsing network traffic data is also provided, the device comprising: a data acquisition unit, configured to: acquire network traffic data to be parsed, wherein the network traffic data to be parsed comprises normal network traffic data and / or abnormal network traffic data; a data parsing unit, configured to: input the network traffic data to be parsed into a deep learning network to obtain a parsing result, wherein the deep learning network is trained based on high-level features extracted from a pre-acquired training data set, the training data set comprises pre-acquired normal network traffic data and / or abnormal network traffic data, the high-level features are extracted from original fields in the pre-acquired normal network traffic data and / or abnormal network traffic data, and comprise at least one of the following items: statistical features, protocol features, time features, and content features, wherein the parsing result comprises the content of the field corresponding to the high-level features of the network traffic data to be parsed and the protocol type of the network traffic data to be parsed.

[0012] According to another aspect of an embodiment of the present disclosure, an electronic device is also provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein when the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute any of the methods for parsing network traffic as described above.

[0013] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium storing instructions is also provided. When the instructions are executed by at least one processor, the at least one processor is prompted to execute any of the methods for parsing network traffic as described above.

[0014] According to another aspect of an embodiment of the present disclosure, a system is also provided, comprising at least one computing device and at least one storage device storing instructions, wherein when the instructions are executed by the at least one computing device, the at least one computing device is prompted to execute any of the methods for parsing network traffic as described above.

[0015] According to another aspect of an embodiment of the present disclosure, a computer program product is also provided, including a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the method for parsing network traffic as described in any one of the above is implemented.

[0016] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0017] According to the method, device, electronic device and storage medium for parsing network traffic disclosed in the present invention, by introducing deep learning technology, high-level features in network traffic data are automatically extracted and analyzed and an effective detection model is constructed, breaking through the limitations of traffic defined by artificial rules. In the transmission process of various network traffic data, efficient and accurate parsing of normal and / or abnormal traffic data under various protocols is achieved, and detailed parsing results are generated to assist in the judgment of network security. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0019] Figure 1 A flowchart showing a method of parsing network traffic in an exemplary embodiment of the present disclosure;

[0020] Figure 2 A schematic diagram of a data processing flow of a Transformer-based T5-large model in an exemplary embodiment of the present disclosure is shown;

[0021] Figure 3A schematic diagram showing the field contents of the parsing result in an exemplary embodiment of the present disclosure;

[0022] Figure 4 A schematic diagram showing the training and testing process of a deep learning network in an exemplary embodiment of the present disclosure is shown;

[0023] Figure 5 A schematic diagram showing a use process of deploying a trained model in an exemplary embodiment of the present disclosure is shown;

[0024] Figure 6 A block diagram showing an apparatus for parsing network traffic in an exemplary embodiment of the present disclosure;

[0025] Figure 7 A block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0028] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0029] With the continuous evolution of network attack methods, traditional detection methods based on rules and signatures can no longer meet the increasingly complex network security needs.

[0030] In recent years, with the rapid development of artificial intelligence technology, especially the successful application of deep learning algorithms in computer vision, natural language processing and other fields, researchers have begun to try to apply deep learning algorithms to the analysis of network traffic data. Deep learning algorithms can automatically learn and extract complex features in data without relying on artificial rules, and have high detection accuracy and generalization ability. Therefore, traffic data analysis methods based on deep learning have become a hot topic in current research.

[0031] In order to solve the problems of low network traffic data detection efficiency, high false alarm rate, high maintenance cost, etc. in related network security methods, the present disclosure provides a method, device, electronic device and storage medium for parsing network traffic. By introducing deep learning technology, high-level features in network traffic data are automatically extracted and analyzed and an effective detection model is constructed, breaking through the limitations of traffic defined by artificial rules. In the transmission process of various network traffic data, efficient and accurate parsing of normal and / or abnormal traffic data under various protocols is achieved, and detailed parsing results are generated to assist in the judgment of network security.

[0032] Next, we will refer to Figures 1 to 6 The method, device, electronic device and storage medium for parsing network traffic disclosed in the present invention are specifically described.

[0033] Figure 1 A flow chart showing a method for parsing network traffic in an exemplary embodiment of the present disclosure.

[0034] Reference Figure 1 In step 101, the network traffic data to be parsed is obtained, wherein the network traffic data to be parsed includes normal network traffic data and / or abnormal network traffic data.

[0035] Specifically, network traffic data refers to a collection of various data information transmitted in the network, which records the flow of data in the network. These data include but are not limited to the following aspects: source and destination information, data content (i.e. the actual transmitted data, such as text, pictures, videos on web pages, content of emails, etc.), traffic characteristics (including traffic size (number of bytes), transmission rate (number of bytes transmitted per second), number of data packets, size distribution of data packets, etc.), time information, etc.

[0036] In step 102, the network traffic data to be parsed is input into the deep learning network to obtain the parsing result.

[0037] Here, the deep learning network is trained based on high-level features extracted from a pre-acquired training data set, the training data set includes pre-acquired normal network traffic data and / or abnormal network traffic data, the high-level features are extracted from the original fields in the pre-acquired normal network traffic data and / or abnormal network traffic data, and include at least one of the following items: statistical features, protocol features, time features, and content features. In addition, the parsing result includes the content of the field corresponding to the high-level features of the network traffic data to be parsed and the protocol type of the network traffic data to be parsed.

[0038] According to an exemplary embodiment of the present disclosure, the pre-acquired normal network traffic data and / or abnormal network traffic data included in the training data set are collected from multiple network nodes and devices.

[0039] Specifically, collecting network traffic data from multiple network nodes and devices can ensure data diversity and coverage. The data can include both normal traffic and abnormal traffic for model training and verification.

[0040] Next, data preprocessing can be performed on the training dataset.

[0041] According to an exemplary embodiment of the present disclosure, the training data set is obtained by cleaning and diversifying the normal network traffic data and / or abnormal network traffic data acquired in advance. Here, cleaning includes at least one of the following items: removing noise, removing irrelevant data, and removing data whose number of classified samples is greater than a first preset threshold. In addition, the diversification processing includes at least one of the following items: performing an undersampling operation on data of a protocol type whose sample size is greater than a second preset threshold, performing an oversampling operation on data of a protocol type whose sample size is less than a second preset threshold, and obtaining the abnormal network traffic data acquired in advance by deleting and / or replacing at least one field in the normal network traffic data acquired in advance.

[0042] Specifically, the collected network traffic data is cleaned to remove noise and irrelevant data, and some data with a large number of classified samples (i.e., greater than the first preset threshold) in the training set are removed to ensure data balance. At the same time, the data is processed in a diversified manner to improve the generalization of the model. In the process of data processing, the protocol type data with a large sample size (i.e., greater than the second preset threshold) is under-sampled, and the data with a small sample size (i.e., less than the second preset threshold) is over-sampled to ensure data balance and diversification. At the same time, the diversity of samples can be improved through artificial simulation, that is, by deleting and replacing the content of some fields of normal protocols (normal network traffic data).

[0043] According to an exemplary embodiment of the present disclosure, abnormal network traffic data is obtained by deleting and / or replacing at least one field in normal network traffic data. That is, abnormal network traffic data can be obtained by performing diversified processing on the collected normal network traffic data. For example, the simulation of abnormal protocol types (abnormal network traffic data) can be completed by performing operations such as randomly adding segments, deleting some segments, and modifying segments on 14 common protocols such as HTTP(s), DNS, FTP, etc.

[0044] After preprocessing the training dataset, the data can be converted into a format suitable for processing by deep learning large model algorithms.

[0045] According to an exemplary embodiment of the present disclosure, the training data set is obtained by converting the format of pre-acquired normal network traffic data and / or abnormal network traffic data into a parquent format.

[0046] Specifically, converting the original data into parquent format can improve the efficiency of data reading. In addition, data can be stored in an external memory and streamed to reduce the pressure of memory usage during training and improve training efficiency.

[0047] Next, you can choose a suitable deep learning algorithm.

[0048] According to an exemplary embodiment of the present disclosure, the deep learning network may be a Transformer-based T5-large model.

[0049] Figure 2 A schematic diagram of the data processing flow of the Transformer-based T5-large model in an exemplary embodiment of the present disclosure is shown.

[0050] Reference Figure 2 The architecture of the Transformer-based T5-large model is basically symmetrical on the left and right sides, processing inputs (Inputs) and outputs (Outputs (shifted right)) respectively. Each side is composed of multiple layers of the same modules stacked together, with the number of layers being N.

[0051] From bottom to top, each layer contains the following main modules:

[0052] Input Embedding and Output Embedding: This is the first step in processing input and output. It converts the input words or tokens (from network traffic data) into vector representations. These vectors capture the semantic and grammatical information of the words.

[0053] Positional Encoding: Since Transformer does not have an inner loop like RNN to process sequence order, positional encoding can be added to represent the position of the word in the sentence. This is achieved by adding positional encoding to the embedding vectors of the input and output.

[0054] Multi-Head Attention and Masked Multi-Head Attention: The attention mechanism is the core part of Transformer, which allows the model to pay attention to other words in the input sentence when processing each word. The multi-head attention mechanism can capture semantic relationships at different levels by calculating the attention distribution of multiple "heads" in parallel and then splicing the results together. In the decoder part, Masked Multi-Head Attention is used, which masks future words when calculating attention, preventing the model from seeing future words when generating the current word.

[0055] Add & Normalization: This step includes residual connections (Add) and layer normalization (Norm). Residual connections help alleviate the gradient vanishing problem in deep networks, while layer normalization helps speed up the training process.

[0056] Feed Forward: This is a fully connected feedforward neural network that performs further nonlinear transformations on the vectors processed by the attention mechanism.

[0057] The final output part, including but not limited to linear layers and Softmax: After processing all layers, a linear layer can be used to map the vector to the output dimension, and then the Softmax function can be used to convert the output to probability distributions for predicting the next word or performing other NLP (natural language processing) tasks.

[0058] The advantage of the Transformer architecture is that it can process input sequences in parallel, which greatly improves the speed of training and inference.

[0059] The deep learning network can be pre-trained on a natural language dataset to retain some of the features learned from natural language. Through the encoder-decoder network structure of the T5-large model, it can more fully understand the information of the network traffic data.

[0060] Optionally, the self-attention mechanism is used to extract the context-related code stream data in the network traffic data, and the fixed segment data stream results are parsed according to the context-related code stream data, which can ensure the consistency and quality of the context data and improve the reliability of the parsing results. In the network flow (network traffic data), it may be difficult to fully parse the detailed features related to the code stream based on a single network flow. The context-related representation is to extract features based on a series of interactive processes, such as determining whether the network flow is HTTPs or FTP, etc. Since each network flow is composed of binary (hexadecimal), each segment represents fixed information, and the content of each segment can be parsed by feature extraction and converted into natural language expression.

[0061] The model is trained using preprocessed data and extracted features, and the model parameters are adjusted through the back-propagation algorithm and optimizer. It can converge to the optimal effect on a training data set of 14 different common protocol types that contain manually adjusted data, and then some deep features of the network flow can be extracted to achieve better content parsing effects for various protocols.

[0062] Deep learning technology can be used to automatically extract high-level features including but not limited to statistical features, protocol features, time features, content features, and other key information from the pre-processed data.

[0063] According to an exemplary embodiment of the present disclosure, the statistical features include at least one of the following items: data packet length, inter-packet time, and the protocol features include at least one of the following items: TCP flag, HTTP header information, IP address, MAC address.

[0064] Specifically, when parsing, the specific content of each segment can be parsed according to the high-level characteristics of the network traffic, and converted into JSON format for expression, including IP Header, protocol type, application layer information, etc. As an example only, for rtsp messages, the parsing results can include but are not limited to Figure 3 The contents of the fields are shown.

[0065] Figure 3 A schematic diagram showing the field contents of a parsing result in an exemplary embodiment of the present disclosure.

[0066] Reference Figure 3 , which is an example of rtsp message and label. Other contents can be parsed according to the payload. The payload is the input and the other parts are labels.

[0067] In addition to the above analysis content, the analysis results can also include the type of network traffic data. There are many different types of network flow transmission protocols, including udp, ntp, http, etc., and there are many private custom protocol types. By obtaining detailed analysis results, it can assist in judging network security.

[0068] According to the exemplary embodiment of the present disclosure, some normal protocols and private protocols in actual application, as well as some abnormal protocols simulated artificially, can be used for testing, the parsing results related to the normal protocol of the traffic can be identified, and the returned parsing results can be viewed to verify the accuracy of the parsing during the application process. In the exemplary embodiment of the present disclosure, the accuracy of protocol parsing and field parsing is about 95%.

[0069] Figure 4 A schematic diagram of the training and testing process of a deep learning network in an exemplary embodiment of the present disclosure is shown.

[0070] Reference Figure 4 , the deep learning network can be a large model based on Transformer. The training process and testing process can include but are not limited to the following:

[0071] In the training process, data preprocessing can be performed, and normal data flow (normal network traffic data) can be artificially simulated (such as randomly adding segments, deleting some segments, modifying segments, etc.) to obtain abnormal data flow (abnormal network traffic data); after data preprocessing, the preprocessed data can enter the Transformer module (large model), and execute the training process of the large model with the help of the task objectives contained in the prompt words, and output a string. The preprocessed label is used to compare with the output string, and the large model can be optimized by calculating the cross entropy loss.

[0072] In the test process, after obtaining the unknown data stream (network traffic data to be parsed), data preprocessing can be performed on it, and the preprocessed data can be input into the Transformer. After being processed by the Transformer, an output string is obtained, which includes the protocol type and field parsing.

[0073] It is understandable that since a single network flow may not necessarily parse out the complete protocol content, the prompt words are similar to, including but not limited to, the following: Please parse out the information of the third message based on the following interactive message flow.

[0074] In the experimental environment, the detection accuracy and false alarm rate of the model can be verified by simulating network attacks and normal traffic. By deploying the model in the actual network environment and monitoring network traffic, the analysis results of various abnormal traffic protocols and their correlation with normal protocols can be detected and analyzed.

[0075] Figure 5 A schematic diagram of the usage flow after the trained model is deployed in an exemplary embodiment of the present disclosure is shown.

[0076] Reference Figure 5 , the network traffic data to be parsed can be obtained from the data source (such as network flow, etc.), and then data preprocessing can be performed. Data preprocessing can include but is not limited to data cleaning, normalization, standardization and other operations, the purpose of which is to convert the original data into a format suitable for model training and application; the preprocessed data can then enter the data sorting stage, which can include but is not limited to adding prompt words, serial numbers and other operations to be better processed by the model. The trained model, that is, the deep learning network, can be integrated into the Web application service, which can be an online platform that provides model-based prediction or classification functions. Finally, users can access the Web application service through a browser. The browser is the interface for users to interact with the Web application service. Users can enter data in the browser to obtain the prediction results of the model, that is, the parsing results.

[0077] According to an exemplary embodiment of the present disclosure, the method for parsing network traffic may mainly include the following steps: data collection, data preprocessing, feature extraction, model construction and training, and detection and parsing. First, network traffic data can be collected through a variety of network nodes and devices to ensure the diversity and coverage of the data. Then, the collected data can be cleaned and format converted to remove noise and irrelevant data to ensure the quality and consistency of the data. Next, deep learning technology can be used to automatically extract high-level features from the preprocessed data, which include statistical features, protocol features, time features, and content features. After that, a suitable deep learning algorithm (e.g., Transformer) can be selected to build and train a model so that it can accurately parse abnormal traffic protocols. Finally, the trained model can be used to parse the network traffic data collected in real time to generate detailed parsing results to assist manual judgment and interception of complex network attacks.

[0078] Compared with the related technologies, the present invention has the following advantages: first, it has high detection efficiency and can quickly assist in monitoring and detecting abnormal behaviors in network traffic; second, it has low false alarm rate and reduces false alarms and missed alarms through the adaptive ability of deep learning algorithms; third, it has strong adaptability and can cope with ever-changing network attack methods and has strong generalization ability; fourth, it reduces maintenance costs and does not require frequent updates and maintenance of rules, which improves the maintainability and scalability of the system.

[0079] The present invention realizes automated feature extraction and model training through deep learning algorithms, which can greatly improve detection efficiency, accuracy and generalization, reduce false alarm rate, adapt to ever-changing network attack methods, enhance the adaptability and maintainability of the system, thereby improving the security of network communications, and has important practical application value and broad application prospects.

[0080] Figure 6 A block diagram showing an apparatus for parsing network traffic in an exemplary embodiment of the present disclosure.

[0081] Reference Figure 6 The exemplary embodiment of the present disclosure further provides a device 600 for parsing network traffic, which may include but is not limited to a data acquisition unit 601 and a data parsing unit 602 .

[0082] The data acquisition unit 601 can acquire network traffic data to be parsed, wherein the network traffic data to be parsed includes normal network traffic data and / or abnormal network traffic data.

[0083] The data parsing unit 602 can input the network traffic data to be parsed into the deep learning network to obtain the parsing result.

[0084] Among them, the deep learning network is trained based on high-level features extracted from a pre-acquired training data set, the training data set contains pre-acquired normal network traffic data and / or abnormal network traffic data, the high-level features are extracted from original fields in the pre-acquired normal network traffic data and / or abnormal network traffic data, and include at least one of the following items: statistical features, protocol features, time features and content features, wherein the parsing results include the content of the fields corresponding to the high-level features of the network traffic data to be parsed and the protocol type of the network traffic data to be parsed.

[0085] It is understandable that in the exemplary embodiment of the above-mentioned device 600 for parsing network traffic, the specific implementation process is roughly the same as the exemplary embodiment of the method for parsing network traffic, and will not be repeated here. The device 600 for parsing network traffic can be configured as software, hardware, firmware or any combination of the above items to perform specific functions. For example, these devices may correspond to dedicated integrated circuits, pure software codes, or modules that combine software and hardware. In addition, one or more functions implemented by these devices may also be uniformly executed by components in physical entity devices (for example, processors, clients or servers, etc.).

[0086] Figure 7 A block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure.

[0087] Reference Figure 7The electronic device 700 includes at least one memory 701 and at least one processor 702, wherein the at least one memory 701 stores a set of computer executable instructions. When the computer executable instruction set is executed by the at least one processor 702, a method for parsing network traffic according to an exemplary embodiment of the present disclosure is executed.

[0088] As an example, the electronic device 700 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instruction set. Here, the electronic device 700 is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instruction (or instruction set) individually or in combination. The electronic device 700 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0089] In the electronic device 700, the processor 702 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0090] The processor 702 may execute instructions or codes stored in the memory 701, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0091] The memory 701 may be integrated with the processor 702, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 701 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 701 and the processor 702 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 702 can read files stored in the memory.

[0092] In addition, the electronic device 700 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 700 may be connected to each other via a bus and / or a network.

[0093] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions may also be provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to execute the above-mentioned method for parsing network traffic.

[0094] Examples of computer-readable storage media include read-only memory (ROM), random-access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, nonvolatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers. It should be noted that the instructions can also be used to execute additional steps in addition to the above steps or to perform more specific processing when executing the above steps. These additional steps and further processing contents have been mentioned in the description of the relevant methods, so they will not be repeated here to avoid repetition.

[0095] Another embodiment of the present disclosure relates to a system including at least one computing device and at least one storage device storing instructions, wherein the instructions, when executed by the at least one computing device, prompt the at least one computing device to execute the above-mentioned method for parsing network traffic.

[0096] It should be noted that the system according to the exemplary embodiments of the present disclosure can completely rely on the execution of computer programs or instructions to realize corresponding functions, that is, each unit corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (e.g., lib library) to realize the corresponding functions.

[0097] On the other hand, when the above-mentioned system is implemented in software, firmware, middleware or microcode, the program code or code segment for performing the corresponding operation can be stored in a computer-readable medium such as a storage medium, so that at least one processor or at least one computing device can perform the corresponding operation by reading and running the corresponding program code or code segment.

[0098] According to an exemplary embodiment of the present disclosure, the storage device may be integrated with the computing device, for example, RAM or flash memory is arranged within an integrated circuit microprocessor, etc. In addition, the storage device may include an independent device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The storage device and the computing device may be operationally coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the computing device can read instructions stored in the storage device.

[0099] Another embodiment of the present disclosure relates to a computer program product, including a computer program / instruction, which implements any of the above methods for parsing network traffic when executed by a processor.

[0100] According to the method, device, electronic device and storage medium for parsing network traffic provided by the present invention, by introducing deep learning technology, high-level features in network traffic data are automatically extracted and analyzed and an effective detection model is constructed, breaking through the limitations of traffic defined by artificial rules. In the transmission process of various network traffic data, efficient and accurate parsing of normal and / or abnormal traffic data under various protocols is achieved, and detailed parsing results are generated to assist in the judgment of network security.

[0101] The above describes various exemplary embodiments of the present disclosure, and it should be understood that the above description is only exemplary and not exhaustive, and the present disclosure is not limited to the disclosed exemplary embodiments. Without departing from the scope and spirit of the present disclosure, many modifications and changes are obvious to those of ordinary skill in the art. Therefore, the scope of protection of the present disclosure should be based on the scope of the claims.

Claims

1. A method for parsing network traffic data, characterized in that: The method comprises: Acquire network traffic data to be parsed, wherein the network traffic data to be parsed includes normal network traffic data and / or abnormal network traffic data; Input the network traffic data to be analyzed into the deep learning network to obtain the analysis result, The deep learning network is trained based on high-level features extracted from a pre-acquired training data set, the training data set includes pre-acquired normal network traffic data and / or abnormal network traffic data, the high-level features are extracted from original fields in the pre-acquired normal network traffic data and / or abnormal network traffic data, and include at least one of the following items: statistical features, protocol features, time features, and content features. The parsing result includes the content of the field corresponding to the high-level features of the network traffic data to be parsed and the protocol type of the network traffic data to be parsed.

2. The method for parsing network traffic data according to claim 1, characterized in that: The abnormal network traffic data is obtained by deleting and / or replacing at least one field in the normal network traffic data.

3. The method for parsing network traffic data according to claim 1, characterized in that: The training data set is obtained by cleaning and diversifying the pre-acquired normal network traffic data and / or abnormal network traffic data, wherein the cleaning includes at least one of the following items: removing noise, removing irrelevant data, and removing data with a classification sample number greater than a first preset threshold; the diversification includes at least one of the following items: performing an undersampling operation on data of a protocol type whose sample size is greater than a second preset threshold, performing an oversampling operation on data of a protocol type whose sample size is less than the second preset threshold, and obtaining the pre-acquired abnormal network traffic data by deleting and / or replacing at least one field in the pre-acquired normal network traffic data.

4. The method for parsing network traffic data according to claim 1, characterized in that: The pre-acquired normal network traffic data and / or abnormal network traffic data included in the training data set are collected from multiple network nodes and devices.

5. The method for parsing network traffic data according to claim 1, characterized in that: The training data set is obtained by converting the format of the pre-acquired normal network traffic data and / or abnormal network traffic data into a parquent format.

6. The method for parsing network traffic data according to claim 1, characterized in that: The statistical features include at least one of the following items: data packet length, inter-packet time, and the protocol features include at least one of the following items: TCP flag, HTTP header information, IP address, and MAC address.

7. The method for parsing network traffic data according to claim 1, characterized in that: The deep learning network is a Transformer-based T5-large model.

8. A device for parsing network traffic data, characterized in that: The device comprises: The data acquisition unit is configured to: acquire network traffic data to be parsed, wherein the network traffic data to be parsed includes normal network traffic data and / or abnormal network traffic data; The data parsing unit is configured to: input the network traffic data to be parsed into the deep learning network to obtain a parsing result, The deep learning network is trained based on high-level features extracted from a pre-acquired training data set, the training data set includes pre-acquired normal network traffic data and / or abnormal network traffic data, the high-level features are extracted from original fields in the pre-acquired normal network traffic data and / or abnormal network traffic data, and include at least one of the following items: statistical features, protocol features, time features, and content features. The parsing result includes the content of the field corresponding to the high-level features of the network traffic data to be parsed and the protocol type of the network traffic data to be parsed.

9. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer executable instructions, When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the method for parsing network traffic according to any one of claims 1 to 7.

10. A computer-readable storage medium storing instructions, characterized in that: When the instructions are executed by at least one processor, the at least one processor is prompted to perform the method for parsing network traffic according to any one of claims 1 to 7.

Images