A method, device, equipment and medium for identifying abnormal packet flow

Abstract

The present invention relates to the field of network security, and discloses a method, device, equipment and medium for identifying packet flow anomalies. The method comprises: obtaining packet flow data, the packet flow data comprising: flow data corresponding to a plurality of sessions; parsing the packet flow data to obtain a grayscale image corresponding to each session; identifying the grayscale image by deep learning to obtain a feature vector of the grayscale image, and determining an abnormal probability distribution of a session corresponding to the grayscale image according to the feature vector; judging whether there is a session type with a probability greater than a preset value in the probability distribution; if not, determining whether the session corresponding to the grayscale image is an abnormal session by combining an anomaly detection model with historical sessions in a historical period, thereby effectively improving the recognition accuracy and comprehensiveness of abnormal sessions.

Description

Technical Field

[0001] The present invention relates to the technical field of network security, and in particular to a method, device, computer equipment and storage medium for identifying abnormal packet flow. Background Art

[0002] With the development of the Internet, network security is becoming more and more important, and network security threats are becoming more and more complex. In order to better protect users' network security, it is becoming more and more important to monitor network packet traffic for anomalies.

[0003] In the prior art, when performing anomaly monitoring and identification on network packet capture traffic, it is often necessary to determine whether each session in the packet capture traffic is abnormal based on preset fixed rules. When anomaly identification is performed on sessions in the packet capture traffic data in this way, the identification accuracy is often low and unknown threats cannot be identified, and a good anomaly identification effect cannot be achieved. Summary of the invention

[0004] In view of this, the present invention provides a method, device, computer equipment and storage medium for identifying packet flow anomalies to solve the problem of low recognition accuracy and inability to identify unknown threats when identifying anomalies in session in packet flow data.

[0005] In a first aspect, the present invention provides a method for identifying abnormal packet flow, the method comprising:

[0006] Acquire packet capture flow data, the packet capture flow data including: flow data corresponding to multiple sessions;

[0007] Parsing the captured packet flow data to obtain a grayscale image corresponding to each session;

[0008] Recognize the grayscale image through deep learning to obtain a feature vector of the grayscale image, and determine the probability distribution of the conversation type corresponding to the grayscale image according to the feature vector;

[0009] Determine whether there is a session type with a probability greater than a preset value in the probability distribution;

[0010] If not, the anomaly detection model is combined with historical sessions in the historical period to determine whether the session corresponding to the grayscale image is an abnormal session.

[0011] This method parses each session in the packet capture traffic data to obtain the corresponding grayscale image, and then recognizes the grayscale image through deep learning, determines the probability distribution of the grayscale image corresponding to the session type, and when the probability distribution cannot give an accurate judgment, uses the anomaly detection model to combine historical data to make an anomaly judgment, ensuring the effective judgment of unknown anomalies, thereby improving the comprehensiveness and accuracy of identifying session anomalies in the packet capture traffic data, so that network security personnel can promptly perform relevant processing on abnormal sessions to ensure the network security of users.

[0012] In an optional implementation, the session type includes: multiple threat types and multiple normal types, and the method further includes:

[0013] If there is a session type with a probability greater than a preset value in the probability distribution, and the session type is a threat type, determining that the session corresponding to the grayscale image is an abnormal session, and determining the threat type as the abnormal type of the abnormal session;

[0014] If there is no session with a probability greater than a preset value in the probability distribution, and the session corresponding to the grayscale image is determined to be an abnormal session through the anomaly detection model, a similar abnormal type of the session corresponding to the grayscale image is determined according to the probability value ranking of the probability distribution.

[0015] This implementation manner analyzes the specific distribution of the probability distribution so that when deep learning can give a highly reliable judgment, the session type of the session is directly determined according to the probability distribution, and when the session type is a threat type, the corresponding session is determined to be abnormal, thereby ensuring the efficiency of anomaly detection. At the same time, when deep learning cannot make an accurate judgment due to the lack of training samples for unknown threats, and the corresponding session is determined to be an abnormal session through the anomaly detection model, similar anomaly types are given for reference in combination with the probability distribution, thereby improving the efficiency of staff in handling anomalies.

[0016] In an optional implementation, the determining whether the session corresponding to the grayscale image is an abnormal session by combining the abnormality detection model with the historical sessions in the historical period includes:

[0017] Determine a plurality of first session types having probability values ​​ranked greater than a preset ranking in the probability distribution and source IP addresses and destination IP addresses of sessions corresponding to the grayscale image;

[0018] Based on the feature vector corresponding to the grayscale image, the source IP address and the destination IP address, and the multiple first session types, comparing the session corresponding to the grayscale image with the historical sessions through an anomaly detection model to determine the degree of abnormality of the session corresponding to the grayscale image;

[0019] Determining whether the conversation corresponding to the grayscale image is an abnormal conversation according to the relationship between the abnormality degree and the preset interval;

[0020] The determining, according to the probability value ranking of the probability distribution, the similar abnormal type of the session corresponding to the grayscale image includes:

[0021] Determining whether there is a threat type among the plurality of first session types whose probability value ranking in the probability distribution is greater than a preset ranking;

[0022] If yes, determining the threat type in the plurality of first session types as a similar abnormal type of the session corresponding to the grayscale image;

[0023] If not, the abnormality type of the session corresponding to the grayscale image is determined to be an unknown abnormality.

[0024] In this embodiment, by determining the feature vector corresponding to the session, multiple first session types with larger probability values ​​in the probability distribution, and the source IP and destination IP, and then comparing the session with the historical sessions based on the above multiple angles through the abnormality monitoring model, the specific abnormality degree of the session is determined, and then it is determined whether it is an abnormal session, and when it is an abnormal session, the multiple first abnormality classifications are used as the expected abnormality types of the session, so that when deep learning cannot give a clear judgment, the current session to be judged is judged abnormally according to historical conditions, and its possible abnormality type is determined in combination with the probability distribution.

[0025] In an optional implementation, the anomaly detection model is: an anomaly detection model constructed based on an isolation forest algorithm;

[0026] The comparing the session corresponding to the grayscale image with the historical session by using an anomaly detection model based on the feature vector corresponding to the grayscale image, the source IP address and the destination IP address, and the multiple first session types, to determine the degree of abnormality of the session corresponding to the grayscale image includes:

[0027] determining the feature vector, the source IP address and the destination IP address, and the plurality of first session types as a plurality of anomaly measurement dimensions;

[0028] The degree of alienation between the conversation corresponding to the grayscale image and the historical conversation is determined by using the anomaly detection model constructed based on the isolation forest algorithm and combining the multiple anomaly measurement dimensions, and the degree of alienation is used to characterize the degree of anomaly.

[0029] In this implementation, feature vectors, IP addresses, and session types with higher probabilities in probability distribution are used as dimensions for measuring anomalies, and an anomaly monitoring model built based on the isolation forest algorithm is used to compare the session to be judged with historical sessions on these dimensions, and the corresponding degree of alienation is obtained to measure the degree of abnormality of the session. This can effectively judge the degree of abnormality of the session by comparing it with historical sessions when deep learning cannot make a clear judgment due to a lack of training samples for unknown threats.

[0030] In an optional embodiment, the method further includes:

[0031] Aggregate the abnormal sessions according to the abnormal types and source IP addresses corresponding to the abnormal sessions within a preset period to obtain multiple aggregated data;

[0032] The plurality of aggregated data are output to a user, so that the user performs exception processing according to the aggregated data.

[0033] In this embodiment, the abnormal sessions within a preset time period are aggregated according to the abnormal type and source IP address, and then output to the user, so that the user can intuitively understand the specific situation of the abnormal sessions of different abnormal types identified by the residual network, so as to make corresponding processing.

[0034] In an optional implementation, aggregating the abnormal sessions according to the abnormal type and source IP address corresponding to each abnormal session within a preset time period includes:

[0035] Determine the corresponding exception label according to the exception type of each abnormal session;

[0036] Aggregate the packet capture traffic data corresponding to abnormal sessions with the same abnormal label and source IP address within a preset time period.

[0037] In this embodiment, by determining the abnormal label corresponding to each abnormal session, the traffic data corresponding to the sessions with the same abnormal type and the same source IP address are aggregated, which can help users understand the specific situation of the session under a certain source IP address and abnormal type, so as to make corresponding processing.

[0038] In an optional implementation manner, parsing the captured packet flow data to obtain a grayscale image corresponding to each session includes:

[0039] Parsing the flow data corresponding to each session in the captured packet flow data to obtain a binary file corresponding to each session;

[0040] Performing hexadecimal conversion on the binary file to obtain a session file in hexadecimal;

[0041] Convert the hexadecimal session file into a corresponding grayscale image.

[0042] This implementation method converts the session file in the captured packet traffic into a base, thereby generating a corresponding grayscale image based on the converted base. This can efficiently realize the conversion between session data and images, and carry specific information about the session in the image to ensure the accuracy of subsequent feature extraction and anomaly judgment.

[0043] In a second aspect, the present invention provides a device for identifying abnormal packet flow, the device comprising:

[0044] A traffic acquisition module, used to acquire traffic data, wherein the traffic data includes: traffic data corresponding to multiple sessions;

[0045] An image conversion module, used to parse the captured packet flow data to obtain a grayscale image corresponding to each session;

[0046] An image recognition module, configured to recognize a grayscale image through deep learning, obtain a feature vector of the grayscale image, and determine a probability distribution of a conversation type corresponding to the grayscale image based on the feature vector;

[0047] A probability judgment module, used to judge whether there is a session type with a probability greater than a preset value in the probability distribution;

[0048] The abnormality judgment module is used to determine whether the conversation corresponding to the grayscale image is an abnormal conversation by combining the abnormality detection model with the historical conversations in the historical period when there is no probability greater than a preset value in the probability distribution.

[0049] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the packet capture traffic anomaly identification method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0050] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the packet capture flow anomaly identification method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 It is a flow chart of a method for identifying abnormal packet flow according to an embodiment of the present invention;

[0053] Figure 2 is a flow chart of another method for identifying abnormal packet flow according to an embodiment of the present invention;

[0054] Figure 3 is an example diagram of an abnormal session identification process according to an embodiment of the present invention;

[0055] Figure 4 is a structural block diagram of a device for identifying abnormal packet flow according to an embodiment of the present invention;

[0056] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0058] With the development of the Internet, network security is becoming more and more important, and the traffic data in network security is becoming more and more complicated. In order to better protect users' network security, it is becoming more and more important to monitor the abnormality of network packet capture traffic.

[0059] In the prior art, when performing anomaly monitoring and identification on network packet capture traffic, it is often determined whether each session in the packet capture traffic is abnormal based on preset fixed rules. When anomaly identification of sessions in the packet capture traffic data is performed in this way, the identification accuracy is often low and a good anomaly identification effect cannot be achieved.

[0060] To this end, an embodiment of the present invention provides a method for identifying packet capture traffic anomalies, which identifies grayscale images through a degree residual network, determines the probability distribution of session types corresponding to the grayscale images, and finally determines the specific abnormal type of the session based on the probability distribution. This can effectively improve the accuracy of identifying session anomalies in packet capture traffic data, so that network security personnel can promptly perform relevant processing on abnormal sessions to ensure user network security.

[0061] According to an embodiment of the present invention, an embodiment of a method for identifying packet traffic anomalies is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0062] In this embodiment, a method for identifying abnormal packet flow is provided, which can be used for the above-mentioned session abnormality identification. Figure 1 is a flow chart of a method for identifying abnormal packet flow according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0063] Step S101, acquiring flow data, where the flow data includes: packet capture flow data corresponding to a plurality of sessions.

[0064] During operation, the network security protection system will capture the traffic data of the network transmission process to obtain the captured traffic data. By calling the captured traffic data obtained by the network security protection system in the recent period of time, the obtained traffic data includes the traffic data corresponding to multiple network sessions.

[0065] Step S102, parsing the captured packet flow data to obtain a grayscale image corresponding to each session.

[0066] After obtaining the packet capture traffic data, since it contains data corresponding to different sessions, the packet capture traffic data can be divided according to the session, that is, the traffic data corresponding to different sessions can be divided. Through common packet capture tools, such as wireshark, the traffic data corresponding to the session can be parsed to obtain its corresponding PCAP file. In the file under this format, the traffic data can be presented in binary format. On this basis, the traffic data of different sessions can be presented in image format to obtain the corresponding grayscale image. The grayscale image can carry the specific information of the session corresponding traffic data.

[0067] Specifically, the flow data in binary format may be converted into hexadecimal format, and then different values ​​may correspond to different grayscales to convert the flow data into a grayscale image.

[0068] Step S103, identifying the grayscale image through deep learning to obtain a feature vector of the grayscale image, and determining a probability distribution of the conversation type corresponding to the grayscale image based on the feature vector.

[0069] Grayscale images are identified through deep learning to obtain their corresponding probability distribution, which includes the probability that the grayscale image corresponds to various session types. For example, the probability distribution may include 20 session types, of which several are types corresponding to normal sessions and several are types corresponding to threat sessions. This can be achieved through various forms of deep learning methods, such as convolutional neural networks, deep residual networks and other artificial intelligence related technical means. For example, grayscale images can be identified through a trained deep residual network.

[0070] These technical means are all supervised learning methods, that is, they need to be trained based on known abnormal conversations and corresponding labels. Therefore, if the abnormal conversation to be identified is a known type of abnormal conversation, the output probability distribution can give a relatively reliable judgment, that is, there is a probability with a large probability value in the probability distribution, such as 99%. However, if some new types of abnormal conversations appear, deep learning cannot make accurate judgments. At this time, the output probability distribution may not have very high probabilities for various conversation types, such as 20%, 30%, 15%, etc.

[0071] The network framework of the deep residual network can adopt resnet34, and a deep residual network with image recognition function is constructed based on the framework. At the same time, the deep residual network is trained with a set of images with anomaly types labeled so that it has better anomaly recognition ability. After the training is completed, the specific content in the above step S103 can be executed. The specific training process can refer to the training method of the relevant model, which will not be repeated here.

[0072] When performing image recognition on the grayscale images of each session, the deep residual network extracts features of the image in multiple dimensions to obtain a feature vector of the image. The abnormal classification of the grayscale image is then determined based on the feature vector. Specifically, the residual network can have a number of classification layers, each of which corresponds to a session type. These session types can include multiple threat types and multiple normal types. The probability corresponding to each classification layer is determined based on the feature vector, thereby obtaining a session type probability distribution.

[0073] Exemplarily, the specific settings of the fully connected layer in the deep residual network can be: in_fetures = 512, out_fetures = 128, that is, the final extracted features are 128-dimensional features, and finally a 20-class classification layer is set. When the residual network is trained, the residual network training can be determined to be completed when the accuracy of the training set is 99% and the accuracy of the test set is 96%. The main purpose of the residual network model is prediction classification and feature extraction.

[0074] The deep residual network can output a probability distribution, which can represent the probabilities corresponding to different session types. For example, the probability distribution can include several threat types and several normal types.

[0075] For example, the number of classification layers can be 20, of which 1-10 are session types under normal circumstances, and 11-20 are session types under threat circumstances, i.e., threat types. When the session type with the highest probability in the deep residual network identification probability distribution is the session type under normal circumstances, and the probability is greater than a preset value, such as 99%, the session can be determined to be a normal session. Similarly, when the session type with the highest probability in the deep residual network identification probability distribution is the session type under threat circumstances, and the probability is greater than a preset value, the session can also be determined to be an abnormal session, and the corresponding threat type is output. The above classification number is only exemplary and is not limited here. It can be set according to actual conditions.

[0076] Step S104, determining whether there is a session type with a probability greater than a preset value in the probability distribution.

[0077] It can be understood that deep learning is usually a supervised learning method, that is, it needs to be trained based on known threat sessions or normal sessions and corresponding labels. Therefore, if the session to be identified is a known type of session, the output probability distribution can give a relatively reliable judgment, that is, there is a probability with a large probability value in the probability distribution, such as 99%. However, if the session type to be judged is a session corresponding to a new unknown threat type, since the deep learning model has not been trained for unknown threats, deep learning cannot make an accurate judgment. At this time, the probability of each session type in the output probability distribution may not be too high, such as 20%, 30%, 15%, etc.

[0078] Therefore, it is necessary to determine whether there is a probability greater than a preset value in the probability distribution, so as to determine whether deep learning can make a judgment with a high degree of certainty. If so, it means that the session type corresponding to the currently judged session is historically known, that is, it has been trained, and its abnormal type can be determined based on the probability situation in the specific probability distribution. If not, it means that the session type corresponding to the judged session is unknown, that is, a new threat type, and deep learning cannot make an accurate judgment.

[0079] Step S105: If not, determine whether the conversation corresponding to the grayscale image is an abnormal conversation by combining the anomaly detection model with the historical conversations in the historical period.

[0080] When there is no probability greater than the preset value in the probability distribution, it means that the deep residual network cannot give a high-certainty judgment result for the session. At this time, the anomaly detection model can be used to compare the relevant data of the session with the sessions in the historical period to determine whether the session is an abnormal session. Specifically, the anomaly detection model can compare the session with the historical sessions based on the specific features of the session, such as the feature vector, the session type with a higher probability in the probability distribution, the IP address, etc., to determine whether the session is a common session, thereby determining whether it is an abnormal session.

[0081] In some cases, due to the limitation of training samples and the emergence of new abnormal types, the deep residual network may sometimes be unable to determine the specific abnormal type, that is, there is no session type with a probability greater than the probability threshold in the probability distribution. This situation shows that the residual network cannot accurately determine the session type corresponding to the session. At this time, you can refer to the corresponding feature vector of the session, several abnormal types with larger probability values, and specific information of multiple dimensions such as IP address, and compare the session with the sessions in the historical period to determine whether sessions with similar features are common, so as to determine whether it is an abnormal session. If the session is determined to be an occasional session, it is determined as an abnormal session, and the possible abnormal type of the session is predicted in combination with several session types with larger probabilities in the probability distribution of the session to provide a reference for users.

[0082] The packet capture traffic anomaly identification method provided in this embodiment parses each session in the packet capture traffic data to obtain a corresponding grayscale image, thereby identifying the grayscale image based on deep learning, determining the probability distribution of the session corresponding to the grayscale image, and finally determining the specific anomaly type of the session based on the probability distribution. This can effectively improve the accuracy of identifying session anomalies in the packet capture traffic data, so that network security personnel can promptly perform relevant processing on abnormal sessions to ensure user network security.

[0083] According to an embodiment of the present invention, another embodiment of a method for identifying abnormal packet flow is provided, which can be used for the above-mentioned session abnormality identification. Figure 2 FIG. 4 is a flow chart of another method for identifying abnormal packet flow according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0084] Step S201, obtain flow data, the flow data includes: packet capture flow data corresponding to multiple sessions. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0085] Step S202: parse the captured packet flow data to obtain a grayscale image corresponding to each session.

[0086] Specifically, in step S202, the captured packet flow data is parsed to obtain a grayscale image corresponding to each session, including:

[0087] Parse the traffic data corresponding to each session in the captured packet traffic data to obtain the binary file corresponding to each session;

[0088] Convert the binary file into hexadecimal to obtain the session file in hexadecimal.

[0089] Convert the hexadecimal session file to the corresponding grayscale image.

[0090] It can be understood that, through common packet capture tools, such as wireshark, the flow data corresponding to each session can be parsed to obtain a corresponding PCAP file. In this file, the flow data of the session can be displayed in a binary format.

[0091] Then convert the file into hexadecimal, so that different grayscale values ​​correspond to different values ​​on each bit, and then convert it into a corresponding grayscale image. The grayscale image is composed of squares, each square corresponds to a grayscale value, and the grayscale image can be regarded as an information carrier of traffic data in image format.

[0092] Step S203, identify the grayscale image through deep learning to obtain a feature vector of the grayscale image, and determine the probability distribution of the conversation type corresponding to the grayscale image based on the feature vector. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0093] Step S204: determine whether there is a session type with a probability greater than a preset value in the probability distribution. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0094] Specifically, the session types in the probability distribution include: multiple threat types and multiple normal types.

[0095] It can be understood that when training the relevant network corresponding to deep learning, deep learning training is performed through training data corresponding to multiple normal session types and training data corresponding to multiple threat types. The probability distribution output in the trained deep learning model includes the probabilities corresponding to each of the multiple threat types and the probabilities corresponding to each of the multiple normal types.

[0096] Step S205: If there is a session type with a probability greater than a preset value in the probability distribution, and the session type is a threat type, then the session corresponding to the grayscale image is determined to be an abnormal session, and the threat type is determined as the abnormal type of the abnormal session.

[0097] It can be understood that when deep learning training is performed, its probability distribution includes the probabilities corresponding to multiple session types, which may include session types under threat conditions and session types under normal conditions. Therefore, in the probability distribution of session types, some session types have threats, that is, they belong to threat types, and some do not have threats, that is, they belong to normal types.

[0098] If there is a probability greater than a preset value in the probability distribution output by deep learning, such as greater than 99%, the deep learning artificial intelligence model can directly give a certain session type. At this time, if the session type is a threat type, the session is directly determined as an abnormal session, and the threat type is determined as the corresponding abnormal type. In some cases, if the session type with the highest probability is a normal type, it means that the session is a normal session and no relevant processing is required.

[0099] Step S206: If not, determine whether the conversation corresponding to the grayscale image is an abnormal conversation by combining the anomaly detection model with the historical conversations in the historical period.

[0100] Specifically, in step S206, the anomaly detection model is combined with the historical session information in the historical period to determine whether the session corresponding to the grayscale image is an abnormal session, including:

[0101] Step S206-1, determining a plurality of first session types having probability values ​​ranked greater than a preset ranking in the probability distribution and source IP addresses and destination IP addresses of sessions corresponding to the grayscale image.

[0102] It can be understood that when comparing the above-mentioned session to be judged with the historical sessions, it is necessary to first determine several session types with higher probability values ​​in the probability distribution corresponding to the session, for example, the three session types with the largest probability values; as well as the source IP address and destination IP address of the session.

[0103] Step S206-2, based on the feature vector corresponding to the grayscale image, the source IP address and the destination IP address, and the multiple first session types, the session corresponding to the grayscale image is compared with the historical session through an anomaly detection model to determine the degree of abnormality of the session corresponding to the grayscale image.

[0104] It can be understood that in the historical data, the feature vectors, abnormal classifications, IP addresses and other information corresponding to each historical session in the historical period will be recorded. After determining the feature vectors, multiple first session types, source IP addresses, destination IP addresses and other multiple dimensional information corresponding to the above session to be determined to be abnormal, the abnormality detection model can be used to compare the session with multiple sessions in the historical period based on the above multiple dimensional information to determine the abnormality of the session.

[0105] Specifically, in step S206-2, the anomaly detection model is: an anomaly detection model constructed based on the isolation forest algorithm.

[0106] The Isolation Forest Algorithm is a machine learning algorithm for anomaly detection. It is an unsupervised learning algorithm that can determine outliers based on a set decision tree. The specific knowledge about the algorithm will not be repeated here.

[0107] Furthermore, in step S206-2, it includes:

[0108] Step S206-2-a: determine the feature vector, the source IP address, the destination IP address, and the multiple first session types as multiple anomaly measurement dimensions.

[0109] The feature vector, the source IP address, the destination IP address, and the multiple first session types determined in the above related steps can be regarded as multiple measurement dimensions for measuring session abnormalities.

[0110] Step S205-2-b, using an anomaly detection model built based on the isolation forest algorithm, combined with multiple anomaly measurement dimensions, determines the alienation degree between the grayscale image corresponding session and the historical session, and the alienation degree is used to characterize the degree of abnormality.

[0111] The anomaly detection model built based on the isolation forest algorithm can determine the degree of alienation between the grayscale image corresponding session and the historical session in multiple measurement dimensions, and combine the alienation degrees of different dimensions to consider and quantify the degree of alienation between the session and the historical session as a whole. Exemplarily, different weights can be set for different measurement dimensions, and the overall degree of alienation can be measured by combining the degree of alienation corresponding to each measurement dimension. The greater the degree of alienation, the more it can be said that the session is not a frequently occurring session, so the greater the degree of alienation, the greater the abnormality of the session.

[0112] Exemplarily, historical sessions with the same source IP address and destination IP address as the current session may be obtained to determine whether the feature vectors and session types of these historical sessions are close to the corresponding information of the current session, thereby determining the alienation of the session.

[0113] Exemplarily, when determining the degree of abnormality, if the multiple first session types determined above are threat types, the IP addresses corresponding to the sessions of the same threat type in the historical sessions can be compared with the current session. If the IP addresses are the same, the session can be assigned a higher degree of abnormality. The above example is only an exemplary way to determine the degree of abnormality of a session. When determining the degree of abnormality for the abnormality detection model, it can be set in combination with actual conditions, and no limitation is made here.

[0114] Step S206-3: Determine whether the conversation corresponding to the grayscale image is an abnormal conversation based on the relationship between the abnormality degree and the preset interval.

[0115] When the abnormality of the session is greater than a preset interval, the session corresponding to the grayscale image is determined as an abnormal session. For example, when the abnormality is 0-0.5, the session is a normal session, when the abnormality is 0.6-0.7, the session is a session that requires a warning, and when the abnormality is 0.8-1, the session is determined as an abnormal session.

[0116] Specifically, in step S206-3, if there is no session with a probability greater than a preset value in the probability distribution, and the session corresponding to the grayscale image is determined to be an abnormal session through the anomaly detection model, a similar abnormal type of the session corresponding to the grayscale image is determined according to the probability value ranking of the probability distribution.

[0117] It can be understood that, since the anomaly is judged by comparing with historical sessions, it means that the judged session cannot be judged by deep learning, that is, it belongs to a new threat type. Therefore, it is impossible to accurately determine the specific threat that caused the anomaly, but the several session types with higher rankings in the anomaly probability distribution given by deep learning can be used as a reference, that is, as similar anomaly types of the judged session, to provide a reference for network security personnel.

[0118] Furthermore, as mentioned above, determining the similar anomaly type of the grayscale image corresponding to the session according to the probability value ranking of the probability distribution includes:

[0119] Determining whether there is a threat type among the plurality of first session types whose probability value ranking in the probability distribution is greater than a preset ranking;

[0120] If yes, determining the threat type in the plurality of first session types as a similar abnormal type of the session corresponding to the grayscale image;

[0121] If not, the abnormality type of the session corresponding to the grayscale image is determined to be an unknown abnormality.

[0122] It can be understood that after the above-mentioned session to be determined is determined as an abnormal session according to the anomaly detection model, the specific situation in the probability distribution can be referred to determine the abnormal type close to the abnormal session. If the several session types with higher rankings in the probability distribution are all normal sessions, it means that the current session belongs to an unknown threat, that is, there is no existing threat type to provide reference. If there is a threat type among the several session types with higher rankings in the probability distribution, the corresponding threat type will be used as a similar abnormal type of the session for reference by relevant staff.

[0123] Step S207, aggregating the abnormal sessions according to the abnormal types and source IP addresses corresponding to the abnormal sessions within a preset period to obtain a plurality of aggregated data; outputting the plurality of aggregated data to the user so that the user performs an exception handling according to the aggregated data.

[0124] It can be seen from the above steps that all sessions have corresponding abnormal types. Therefore, all abnormal sessions parsed in a certain time period can be aggregated and processed in a unified manner in combination with the abnormal type, so that the staff can analyze the abnormal sessions from a global perspective. Specifically, the preset time period can be the current day, the past week, the past month, etc. The abnormal sessions that are close to each other can be aggregated in combination with the abnormal type and source IP address, which can be the sessions with the same source IP address and abnormal type.

[0125] Specifically, in step S207, aggregating the abnormal sessions according to the abnormal type and source IP address corresponding to each abnormal session within a preset time period includes:

[0126] Determine the corresponding exception label according to the exception type of each abnormal session;

[0127] Aggregate the packet capture traffic data corresponding to abnormal sessions with the same abnormal label and source IP address within a preset time period.

[0128] It can be understood that after the abnormal session is determined through the above related steps, the corresponding abnormal type can be determined in combination with the probability distribution.

[0129] For example, in case 1, a clear anomaly type is given directly based on the threat type whose probability exceeds the preset threshold in the probability distribution given by deep learning; in case 2, it is determined to be an abnormal session based on the anomaly detection model, and a similar anomaly type is given in combination with the probability distribution; in case 3, it is determined to be an abnormal session based on the anomaly detection model, but there is no similar anomaly type in the probability distribution, and the session is determined to be an unknown anomaly type.

[0130] Therefore, corresponding abnormal labels can be set for abnormal sessions in different situations. For example, in case 1, the abnormal type of an abnormal session is A, and its corresponding label is A. In case 2, similar abnormal types of an abnormal session are A, B, and C, and its corresponding label is ABB. In case 3, since its corresponding abnormal type is unknown abnormality, its corresponding label is unknown.

[0131] On this basis, sessions with the same abnormal label and source IP address are aggregated to obtain multiple types of aggregated data.

[0132] The packet capture traffic anomaly identification method provided by the embodiment of the present invention parses each session in the packet capture traffic data to obtain a corresponding grayscale image, thereby identifying the grayscale image based on deep learning, determining the probability distribution of the session type corresponding to the grayscale image, and finally determining whether the session is abnormal and the specific anomaly type based on the probability distribution. When deep learning cannot give a definite judgment, the anomaly is judged by comparing the anomaly detection model with the historical sessions, which can effectively improve the comprehensiveness of identifying session anomalies in the packet capture traffic data, so that network security personnel can promptly perform relevant processing on the abnormal sessions to ensure the user's network security.

[0133] To facilitate understanding of the above embodiment, an exemplary embodiment of the present invention provides an example diagram of an abnormal session identification process, such as Figure 3 shown.

[0134] First, the packet capture traffic is acquired. After acquiring the packet capture traffic data, the packet capture traffic data is divided according to the session to obtain the data corresponding to each session. Then the data corresponding to the session is converted into a format to obtain the image corresponding to the session. The image is input into the trained neural network model, i.e., the deep residual network in the above embodiment, to perform the classification task to obtain the classification result and the final extracted features (128 dimensions), i.e., the feature vector in the above embodiment. At the same time, the classification probability distribution corresponding to each session and the top three categories of predicted probability are determined. The classification probability distribution includes multiple classifications under normal conditions and multiple classifications under abnormal conditions. The above results are stored in the database.

[0135] Perform classification prediction inference on the probability distribution corresponding to the session, that is, determine whether there is a classification greater than the preset threshold in the probability distribution. If so, directly output the corresponding classification result.

[0136] If not, use the network obtained above to extract features (128 dimensions), probability distribution, predict the top three category features, source IP, destination IP, and build an anomaly detection model (unsupervised). The model is based on the isolated forest algorithm, which can be trained with the data of the previous week of the day. Then filter out the sessions that the anomaly detection model determines to be abnormal, and infer their possible anomaly types based on the probability distribution. When the anomaly detection model makes an anomaly judgment, it is divided according to the anomaly score, for example: if 0.5 <x<=0.55:out='warning',elif 0.55<x<=0.6:out='error',else:out='alert'。

[0137] After obtaining the inference results, aggregation processing is performed, that is, multiple sessions are constituted into a data of the same type. First, data with a prediction probability greater than 95% is constructed, that is, the classified sessions are directly determined through the neural network model, and the types with predicted categories under abnormal conditions are filtered out. This part of the data is data_1.

[0138] Next, we construct data with a predicted probability less than 95%, that is, sessions detected as abnormal by the anomaly detection model. This part is data_2, and the predicted labels are modified to the top three most similar categories. We combine data_1 and data_2 to construct risk anomaly data as data_all.

[0139] The method provided by the present invention can adopt transfer learning to adjust the pre-trained ResNet34 model to a new model suitable for a specific task, combining supervised and unsupervised learning. At the same time, the network traffic data is converted into an image and processed using a deep learning model, so as to facilitate the use of the powerful feature extraction capabilities of the convolutional neural network. And by modifying the fully connected layer of ResNet34 to extract 128-dimensional features, input is provided for subsequent classification and anomaly detection; combined with unsupervised methods such as isolation forests and feature-based supervised methods, comprehensive anomaly detection is performed. On the basis of identifying the anomaly, the type of anomaly is further identified, more specific classification information is provided, and detailed level division is performed according to the anomaly score, providing a clear threshold judgment standard. By combining unsupervised and supervised learning, as well as the threshold division of the anomaly score, the false alarm rate is effectively reduced.

[0140] In this embodiment, a packet capture flow anomaly identification device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0141] This embodiment provides a device for identifying abnormal packet flow. Figure 4 As shown, including:

[0142] The traffic acquisition module 401 is used to acquire traffic data, where the traffic data includes: packet capture traffic data corresponding to multiple sessions.

[0143] The image conversion module 402 is used to parse the captured packet flow data to obtain a grayscale image corresponding to each session.

[0144] The image recognition module 403 is used to recognize the grayscale image through deep learning, obtain the feature vector of the grayscale image, and determine the probability distribution of the conversation type corresponding to the grayscale image according to the feature vector.

[0145] A probability determination module 404 is used to determine whether there is a session type with a probability greater than a preset value in the probability distribution;

[0146] The abnormality judgment module 405 is used to determine whether the session corresponding to the grayscale image is an abnormal session by combining the abnormality detection model with the historical sessions in the historical period when there is no probability greater than a preset value in the probability distribution.

[0147] In some optional implementations, the session type includes: multiple threat types and multiple normal types, and the probability judgment module 404 determines that the session corresponding to the grayscale image is an abnormal session when there is a session type with a probability greater than a preset value in the probability distribution and the session type is a threat type, and determines the threat type as an abnormal type of the abnormal session;

[0148] The abnormality judgment module 405 determines the similar abnormality type of the session corresponding to the grayscale image according to the probability value ranking of the probability distribution when there is no session with a probability greater than a preset value in the probability distribution and the session corresponding to the grayscale image is determined to be an abnormal session through the abnormality detection model.

[0149] In some optional implementations, the abnormality determination module 405, when determining whether the session corresponding to the grayscale image is an abnormal session by combining the abnormality detection model with the historical sessions in the historical period, includes:

[0150] Determine a plurality of first session types having probability values ​​ranked greater than a preset ranking in the probability distribution and source IP addresses and destination IP addresses of sessions corresponding to the grayscale image;

[0151] Based on the feature vector corresponding to the grayscale image, the source IP address and the destination IP address, and the multiple first session types, comparing the session corresponding to the grayscale image with the historical sessions through an anomaly detection model to determine the degree of abnormality of the session corresponding to the grayscale image;

[0152] Determining whether the conversation corresponding to the grayscale image is an abnormal conversation according to the relationship between the abnormality degree and the preset interval;

[0153] The abnormality judgment module 405, when determining the similar abnormality type of the conversation corresponding to the grayscale image according to the probability value ranking of the probability distribution, includes:

[0154] Determining whether there is a threat type among the plurality of first session types whose probability value ranking in the probability distribution is greater than a preset ranking;

[0155] If yes, determining the threat type in the plurality of first session types as a similar abnormal type of the session corresponding to the grayscale image;

[0156] If not, the abnormality type of the session corresponding to the grayscale image is determined to be an unknown abnormality.

[0157] In some optional implementations, the anomaly detection model is: an anomaly detection model constructed based on an isolation forest algorithm;

[0158] The abnormality judgment module 405, based on the feature vector corresponding to the grayscale image, the source IP address and the destination IP address, and the multiple first session types, compares the session corresponding to the grayscale image with the historical sessions through the abnormality detection model to determine the abnormality degree of the session corresponding to the grayscale image, includes:

[0159] determining the feature vector, the source IP address and the destination IP address, and the plurality of first session types as a plurality of anomaly measurement dimensions;

[0160] The degree of alienation between the conversation corresponding to the grayscale image and the historical conversation is determined by using the anomaly detection model constructed based on the isolation forest algorithm and combining the multiple anomaly measurement dimensions, and the degree of alienation is used to characterize the degree of anomaly.

[0161] In some optional implementations, the abnormality determination module 405 is further configured to aggregate the abnormal sessions according to the abnormality types and source IP addresses corresponding to the abnormal sessions within a preset period of time to obtain a plurality of aggregated data;

[0162] The plurality of aggregated data are output to a user, so that the user performs exception processing according to the aggregated data.

[0163] In some optional implementations, the abnormality determination module 405, when aggregating the abnormal sessions according to the abnormality type and source IP address corresponding to each abnormal session within a preset time period, includes:

[0164] Determine the corresponding exception label according to the exception type of each abnormal session;

[0165] Aggregate the packet capture traffic data corresponding to abnormal sessions with the same abnormal label and source IP address within a preset time period.

[0166] In some optional implementations, the image conversion module 402, when parsing the captured packet flow data to obtain the grayscale image corresponding to each session, includes:

[0167] Parse the traffic data corresponding to each session in the captured packet traffic data to obtain the binary file corresponding to each session;

[0168] Convert the binary file into hexadecimal to obtain the session file in hexadecimal.

[0169] Convert the hexadecimal session file to the corresponding grayscale image.

[0170] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0171] The packet capture traffic anomaly identification device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0172] The embodiment of the present invention also provides a computer device having the above Figure 4 The packet capture flow anomaly identification device shown.

[0173] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0174] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0175] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0176] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0177] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0178] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0179] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0180] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0181] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for identifying abnormal packet flow, characterized in that: The method comprises: Acquire flow data, the flow data including: packet capture flow data corresponding to multiple sessions; Parsing the captured packet flow data to obtain a grayscale image corresponding to each session; Recognize the grayscale image through deep learning to obtain a feature vector of the grayscale image, and determine the probability distribution of the conversation type corresponding to the grayscale image according to the feature vector; Determine whether there is a session type with a probability greater than a preset value in the probability distribution; If not, determine whether the session corresponding to the grayscale image is an abnormal session by combining the historical sessions in the historical period with the anomaly detection model; The determining whether the session corresponding to the grayscale image is an abnormal session by combining the abnormality detection model with the historical sessions in the historical period includes: Determine a plurality of first session types having probability values ​​ranked greater than a preset ranking in the probability distribution and source IP addresses and destination IP addresses of sessions corresponding to the grayscale image; Based on the feature vector corresponding to the grayscale image, the source IP address and the destination IP address, and the multiple first session types, comparing the session corresponding to the grayscale image with the historical sessions through an anomaly detection model to determine the degree of abnormality of the session corresponding to the grayscale image; According to the relationship between the abnormality degree and the preset interval, it is determined whether the conversation corresponding to the grayscale image is an abnormal conversation.

2. The method according to claim 1, characterized in that The session types include: multiple threat types and multiple normal types, and the method further includes: If there is a session type with a probability greater than a preset value in the probability distribution, and the session type is a threat type, determining that the session corresponding to the grayscale image is an abnormal session, and determining the threat type as the abnormal type of the abnormal session; If there is no session with a probability greater than a preset value in the probability distribution, and the session corresponding to the grayscale image is determined to be an abnormal session through the anomaly detection model, a similar abnormal type of the session corresponding to the grayscale image is determined according to the probability value ranking of the probability distribution.

3. The method according to claim 2, characterized in that The determining, according to the probability value ranking of the probability distribution, the similar abnormal type of the session corresponding to the grayscale image includes: Determining whether there is a threat type among the plurality of first session types whose probability value ranking in the probability distribution is greater than a preset ranking; If yes, determining the threat type in the plurality of first session types as a similar abnormal type of the session corresponding to the grayscale image; If not, the abnormality type of the session corresponding to the grayscale image is determined to be an unknown abnormality.

4. The method according to claim 3, characterized in that The anomaly detection model is: an anomaly detection model constructed based on the isolation forest algorithm; The comparing the session corresponding to the grayscale image with the historical session by using an anomaly detection model based on the feature vector corresponding to the grayscale image, the source IP address and the destination IP address, and the multiple first session types, to determine the degree of abnormality of the session corresponding to the grayscale image includes: determining the feature vector, the source IP address and the destination IP address, and the plurality of first session types as a plurality of anomaly measurement dimensions; The degree of alienation between the conversation corresponding to the grayscale image and the historical conversation is determined by using the anomaly detection model constructed based on the isolation forest algorithm and combining the multiple anomaly measurement dimensions, and the degree of alienation is used to characterize the degree of anomaly.

5. The method according to claim 2, characterized in that: The method further comprises: Aggregate the abnormal sessions according to the abnormal types and source IP addresses corresponding to the abnormal sessions within a preset period to obtain multiple aggregated data; The plurality of aggregated data are output to a user, so that the user performs exception processing according to the aggregated data.

6. The method according to claim 5, characterized in that: The aggregating the abnormal sessions according to the abnormal types and source IP addresses corresponding to the abnormal sessions within the preset time period includes: Determine the corresponding exception label according to the exception type of each abnormal session; Aggregate the packet capture traffic data corresponding to abnormal sessions with the same abnormal label and source IP address within a preset time period.

7. The method according to claim 1, characterized in that The step of parsing the captured packet flow data to obtain a grayscale image corresponding to each session includes: Parsing the flow data corresponding to each session in the captured packet flow data to obtain a binary file corresponding to each session; Performing hexadecimal conversion on the binary file to obtain a session file in hexadecimal; Convert the hexadecimal session file into a corresponding grayscale image.

8. A device for identifying abnormal packet flow, characterized in that: The device comprises: A flow acquisition module, used to acquire flow data, the flow data including: packet capture flow data corresponding to multiple sessions; An image conversion module, used to parse the captured packet flow data to obtain a grayscale image corresponding to each session; An image recognition module, configured to recognize a grayscale image through deep learning, obtain a feature vector of the grayscale image, and determine a probability distribution of a conversation type corresponding to the grayscale image based on the feature vector; A probability judgment module, used to judge whether there is a session type with a probability greater than a preset value in the probability distribution; An abnormality judgment module, used to determine whether the conversation corresponding to the grayscale image is an abnormal conversation by combining the abnormality detection model with the historical conversations in the historical period when there is no probability greater than a preset value in the probability distribution; The determining whether the session corresponding to the grayscale image is an abnormal session by combining the abnormality detection model with the historical sessions in the historical period includes: Determine a plurality of first session types having probability values ​​ranked greater than a preset ranking in the probability distribution and source IP addresses and destination IP addresses of sessions corresponding to the grayscale image; Based on the feature vector corresponding to the grayscale image, the source IP address and the destination IP address, and the multiple first session types, comparing the session corresponding to the grayscale image with the historical sessions through an anomaly detection model to determine the degree of abnormality of the session corresponding to the grayscale image; According to the relationship between the abnormality degree and the preset interval, it is determined whether the conversation corresponding to the grayscale image is an abnormal conversation.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the packet capture flow anomaly identification method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for identifying abnormal packet flow capture according to any one of claims 1 to 7.

Images