A message detection and analysis system and method based on deep flow detection

By using a message detection and analysis system based on deep flow detection, which utilizes flow feature extraction and classifiers to determine whether a message is normal, the system solves the problem that deep flow detection technology cannot accurately locate the application type, and achieves efficient and accurate analysis of encrypted messages.

CN117792701BActive Publication Date: 2024-11-19EXANDS INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311689217.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-11
Publication Date
2024-11-19
Estimated Expiration
2043-12-11

AI Technical Summary

Technical Problem

Deep flow detection technology cannot accurately pinpoint the specific application type, especially when HTTP and HTTPS traffic characteristics are similar, making it difficult to determine whether the business is operating normally.

Method used

A message detection and analysis system based on deep flow detection is adopted, which includes data acquisition, deep flow detection, message analysis and security modules. Through flow feature extraction, supervised and unsupervised classification, a 3×n matrix is ​​used to store the data flow probability and abnormality rate, and a Bayesian classifier is combined to determine whether the message is normal.

Benefits of technology

It enables efficient and accurate identification of encrypted messages, distinguishes between application types such as HTTP and HTTPS, and improves the accuracy and robustness of network traffic analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117792701B_ABST
    Figure CN117792701B_ABST
Patent Text Reader

Abstract

The invention discloses a message detection and analysis system and method based on deep flow detection, and relates to the technical field of message detection. The system comprises: a data acquisition module, a deep flow detection module, a message analysis module, a data storage module and a security module; the data acquisition module is used to collect data flow information produced by message transmission in a network; the deep flow detection module is used to extract flow features of message data flow and judge the application type; the message analysis module is used to judge whether the message is normal; the data storage module is used to store message flow data; the security module checks the message information and the network connection status; the invention uses the deep flow detection technology to identify the application type of the data flow, which is not affected by the encryption of application layer data; the Bayesian classifier is used to continue classification in the large category, and only relies on the statistical properties of the data itself, and has high efficiency and accuracy when judging whether the encrypted message is normal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of message detection, and in particular to a message detection and analysis system and method based on deep flow detection. Background Art

[0002] Deep flow detection technology is an application identification technology based on traffic behavior. It takes flow as the basic research object and extracts flow features, such as flow size and flow speed, from huge network flow data. Deep flow detection technology is mainly divided into three parts: flow feature selection, flow feature extraction, and classifier. Deep flow detection uses an application identification technology based on traffic behavior. This technology detects and identifies different application types by analyzing data packets in network traffic. Different application types have different manifestations in network traffic. Therefore, deep flow detection technology can identify different application types based on the traffic behavior of data packets. Deep flow detection technology can only classify application types in general and cannot accurately locate specific application types. This is because deep flow detection technology mainly focuses on the behavioral characteristics of network traffic. When a traffic meets a certain specific behavioral model, deep flow detection technology will classify it as the corresponding application type, such as P2P traffic or VOIP traffic.

[0003] To determine whether a service is normal, it is usually necessary to understand the specific type of application. Different application types have different traffic characteristics and behavior patterns. HTTP and HTTPS are both web browsing application types based on the TCP protocol, and their traffic characteristics and behavior patterns are very similar; however, the distribution of flow characteristics of HTTP and HTTPS is not exactly the same. The same traffic characteristics, when the application types are HTTP and HTTPS, are located in different positions in the flow characteristic distribution diagrams of HTTP and HTTPS, and need to be determined separately to determine whether the service is normal; therefore, when deep flow detection technology can only distinguish data flow application types by broad categories, how to determine whether the service is normal becomes a problem that needs to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a message detection and analysis system and method based on deep flow inspection to solve the problems raised in the above background technology.

[0005] In one aspect of the present invention, a message detection and analysis system based on deep flow detection is provided, comprising: a data acquisition module, a deep flow detection module, a message analysis module, a data storage module and a security module; the output end of the data acquisition module is interconnected with the input end of the deep flow detection module and the data storage module, and is used to collect data flow information produced by message transmission in the network; the output end of the deep flow detection module is interconnected with the input end of the message analysis module, and is used to extract flow features of the message data flow and determine the application type of the message data flow; the output end of the message analysis module is interconnected with the input end of the security module, and is used to determine whether the message is normal; the data storage module is interconnected with the deep flow detection module and the message analysis module, and is used to store message traffic data; when the message analysis module determines that the message data flow is abnormal, the security module checks the message information and network connection status, and notifies the administrator to take measures.

[0006] Specifically, the deep flow detection module also includes a data preprocessing unit, a flow feature extraction unit, a data flow classification unit and an optimization unit; the data preprocessing unit is used to preprocess the message flow data; the flow feature extraction unit is used to extract the flow features of the message flow from the message flow data; the data flow classification unit is used to perform supervised classification and identification of the application type of the message data flow; the optimization unit trains and optimizes the data flow classification unit according to the known message flow data and application type to improve the accuracy and robustness; the message analysis module also includes an unsupervised classification unit, an application type judgment unit and a detection unit; the unsupervised classification unit is used to find the historical data flow related to the message data flow, and the application type judgment unit is used to determine the message The probability that the data flow belongs to different application types; the monitoring unit is used to determine whether the message data flow is normal; the data storage module stores the probability that the data flow belongs to each application type and the abnormal rate of each application type through a 3×n-order matrix; the elements in the first row are 1, 2, ...n, representing n different application types; the elements in the second row represent the probability that the data flow belongs to each application type, and the elements in the third row represent the probability that the message data flow is abnormal when the message data flow belongs to each application type; each 3×n-order matrix corresponds to the flow feature data of a message. As the flow feature data of the message increases, it is only necessary to update the data at the corresponding position in the 3×n-order matrix. When the same flow feature data appears, it is determined whether the message is normal based on the data in the matrix, and there is no need to analyze the message traffic data.

[0007] In another aspect of the present invention, a packet detection and analysis method based on deep flow inspection is provided, comprising the following steps:

[0008] S5-1, obtaining flow characteristic data of a data flow, where the data flow refers to a data flow generated when a message is transmitted in a network;

[0009] S5-2, based on the flow characteristic data of the data flow, the application type of the data flow is first determined by a supervised classification model. If the application type of the data flow is not unique, the process proceeds to step S5-3; if the application type of the data flow is unique, the process proceeds to step S5-4;

[0010] If the flow characteristics of the data flow do not match the flow characteristics of any known application type, the message data flow is directly judged to be abnormal. The abnormal flow characteristics of the data flow may be caused by network problems or problems with the data itself. The security module will check the network connection status and message information.

[0011] S5-3, based on the data flow application type determined for the first time, determine the probability that the data flow belongs to each application type; determine the probability that the data flow is abnormal when it belongs to different application types according to the application type and flow characteristic data of the data flow; and determine whether the data flow is normal by combining all abnormal probabilities;

[0012] S5-4, judging whether the data flow is normal according to the application type and flow characteristic data of the data flow.

[0013] In step S5-2, the application type of the data stream is first determined by the supervised classification model, which specifically includes the following steps:

[0014] S6-1, obtaining historical data flow information, including flow characteristics and application types of data flows;

[0015] S6-2, for each application type, a binary classification model is trained with the flow features of the historical data flow as input and the application type as output, and a total of n binary classification models are obtained, where n is the number of application types;

[0016] S6-3, inputting the flow characteristics of the current data flow into n binary classification models to determine one or more application types that match the flow characteristics of the current data flow;

[0017] For data flows of the same application type, the flow characteristics are very similar, and it is difficult to distinguish the application types in detail through the classification model. Here, the supervised classification model is used to determine one or more application types that match the flow characteristics of the current data flow, and find the possible application types of the current data flow;

[0018] In step S5-3, determining the probability that the data flow belongs to each application type includes the following steps:

[0019] S7-1, with a1, a2, ...a mIndicates the element corresponding to the data stream application type determined for the first time, where m is the number of application types that match the flow characteristics of the current data stream; when m is 2, the elements corresponding to the data stream application type are a1 and a2;

[0020] S7-2, with X1, X2, ...X k The flow characteristics of the data flow are represented by Y, which is a random variable representing the application type. The value range of Y is {a1, a2, ...a m}, k is the number of flow features;

[0021] S7-3, calculates that the data flow belongs to application type a1, a2, ...a m The probability of Pa1, Pa2, ...Pa m ;

[0022] Calculate m conditional probabilities,

[0023]

[0024] Determine Pa1, Pa2, ...Pa m , The value range of i is a positive integer between the interval [1, m];

[0025] Flow characteristics X1, X2, ...X of data flow k It is known that, given the known flow features X1, X2, ...X k In the case of, the probability that the data flow belongs to each application type is calculated using the conditional probability formula. The higher the conditional probability, the higher the probability that the data flow belongs to a certain application type. Since the denominators in the calculation formula are all the same, only the numerators need to be compared. Here, the application type corresponding to the maximum conditional probability is not simply used as the current data flow application type, but the application type of the current data flow is expressed in the form of probability.

[0026] P{Y=a i} is the data flow belongs to application type a i The probability of applying type a in the data storage module is calculated i The ratio of the number of data flows to the total number of data flows in the data storage module is obtained; P{X1,X2,…X k |Y=a i} and P{X1,X2,…X k , Y = a i} is equal, determine the application type from the data storage module as a i And the flow characteristics are X1,X2,…X kThe number of data flows quantity is calculated, and the ratio of quantity to the total number of data flows in the data storage module is calculated to obtain P{X1,X2,…X k |Y=a i};

[0027] S7-4, Pa1, Pa2, ...Pa m Fill in the corresponding position of the second row in the 3×n matrix.

[0028] In step S5-3, determining the probability of abnormality when the data stream belongs to different application types includes the following steps:

[0029] S8-1, taking the flow characteristics of the current data flow and the flow characteristics of the historical data flow as input, performing unsupervised classification, and determining the classification cluster to which the flow characteristics of the current data flow belong;

[0030] S8-2, in the classification cluster to which the flow characteristics of the current data flow belong, find out the application types a1, a2, ...a m The flow characteristic data are recorded as The superscript indicates the application type of the flow characteristic data, and the subscript b1, b2, ... b m They respectively represent the classification cluster to which the flow characteristics of the current data flow belong, and the application types are a1, a2, ...a m The number of flow feature data;

[0031] S8-3, for each flow feature data in step S8-2, calculate the probability of abnormality, denoted as

[0032] In particular, in step S5-4, the flow characteristics of the current data flow only match one application type, and only the abnormal probability of one application type needs to be calculated, that is, the probability that the current data flow is abnormal. If it is greater than or equal to the threshold, it is judged that the data flow generated when the current message is transmitted in the network is abnormal. If it is less than the threshold, it is judged that the data flow generated when the current message is transmitted in the network is normal.

[0033] S8-4, determine each application type a1, a2, ...a m Abnormal probability In the formula is the connection weight of the stream feature data; Fill in the corresponding position of the third row in the 3×n-order matrix;

[0034] Each flow feature data is distributed in different locations in different application types, and has different effects on judging whether the message is normal. For example, HTTP and HTTPS have very similar traffic features and behavior patterns. Since HTTPS needs to be encrypted, additional information needs to be added to the beginning of the data packet, such as the identifier and key of the encryption algorithm. This information will increase the size of the entire data packet. For the same data packet size quan, which meets the traffic features of both HTTP and HTTPS, quan may be at the average value in the HTTP flow features, but in the HTTPS traffic features, quan will be in front of the average value, which has different effects on whether the message data is normal.

[0035] S8-5, calculate the probability PE of the current data flow being abnormal, If PE is greater than or equal to the threshold, the data flow generated when the current message is transmitted in the network is judged to be abnormal; if PE is less than the threshold, the data flow generated when the current message is transmitted in the network is judged to be normal;

[0036] The threshold is calculated by the flow feature data of the abnormal historical data flow in the data storage module according to the above steps to obtain the probability that the data flow is abnormal, and the 25% quantile of the abnormal probability corresponding to the flow feature data of all abnormal historical data flows is taken as the threshold. Other values ​​such as the average value, minimum value or median can also be used as the threshold according to needs.

[0037] In step S8-4, the connection weight of the stream feature data is determined by the following steps:

[0038] Calculate flow characteristic data The Euclidean distance between the stream feature data of the current data stream and the Euclidean distance is recorded as Map the Euclidean distance through the anti-correlation function f and get

[0039] Calculate the weight of each stream feature data

[0040] In the summation formula, j only represents a sequence number and has no practical meaning;

[0041] The anti-correlation function f can be e -x etc., where c is a non-zero positive constant, and x is the input of the anti-correlation function f; when c is 0 and the Euclidean distance between the existing stream feature data and the stream feature data of the current data stream is 0, The value of tends to infinity, so c is a positive constant greater than 0;

[0042] In particular, the weight of the flow feature data It can be used to calculate whether the data flow belongs to application type a1, a2, ...a m The probability of Pa1, Pa2, ...Pa m , execute steps S8-1 and S8-2 and obtain the weight of the flow feature data For each flow characteristic data, step S7-3 is executed to obtain a set of Pa1, Pa2, ...Pa for each flow characteristic data. m , Pa1, Pa2, ...Pa obtained from the flow feature data of the same application type m By connecting through weights, we can get that the current data stream belongs to application type a1, a2, ...a m probability.

[0043] Specifically, in step S8-3, for each flow feature data in step S8-2, calculating the probability of abnormality includes the following steps:

[0044] by Represents any stream feature data, the value range of x is [1, b i ] is a positive integer between ; when the application type is a i From the flow feature data of the historical data flow, determine The total number of the same stream feature data, The number of times the same flow feature data is abnormal is calculated as the ratio of the abnormal number to the total number as the flow feature data. abnormal probability.

[0045] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the deep flow detection technology identifies the application type of the data flow according to the characteristics of the flow, different application types are reflected in different states on the session connection or data flow, and the flow characteristics of the entire data flow are analyzed, such as the average packet length of each flow, the time interval for the arrival of each packet, etc., without the need to detect the application layer data, so the detection effect will not be affected by the encryption of the application layer data; after determining the major category of the application type, the Bayesian classifier is used to continue classification in the major category of the application program, which is insensitive to noise and outliers, and only relies on the statistical properties of the data itself without knowing the specific content of the data, and has high efficiency and accuracy in judging whether the encrypted message is normal. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0047] Figure 1It is a structural schematic diagram of a message detection and analysis system based on deep flow detection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] In the embodiments of the present invention, please refer to Figure 1 , provides a message detection and analysis system based on deep flow detection, including: a data acquisition module, a deep flow detection module, a message analysis module, a data storage module and a security module; the output end of the data acquisition module is interconnected with the input end of the deep flow detection module and the data storage module, and is used to collect the data flow information of the message transmitted in the network; the output end of the deep flow detection module is interconnected with the input end of the message analysis module, and is used to extract the flow characteristics of the message data flow and determine the application type of the message data flow; the output end of the message analysis module is interconnected with the input end of the security module, and is used to determine whether the message is normal; the data storage module is interconnected with the deep flow detection module and the message analysis module, and is used to store message traffic data; the security module, when the message analysis module determines that the message data flow is abnormal, checks the message information and network connection status, and notifies the administrator to take measures.

[0050] The deep flow detection module also includes a data preprocessing unit, a flow feature extraction unit, a data flow classification unit and an optimization unit; the data preprocessing unit is used to preprocess the message flow data; the flow feature extraction unit is used to extract the flow features of the message flow from the message flow data; the data flow classification unit is used to perform supervised classification and identification of the application type of the message data flow; the optimization unit trains and optimizes the data flow classification unit according to the known message flow data and application type to improve the accuracy and robustness.

[0051] The message analysis module also includes an unsupervised classification unit, an application type judgment unit and a detection unit; the unsupervised classification unit is used to find historical data flows related to message data flows, the application type judgment unit is used to determine the probability that the message data flows belong to different application types; the monitoring unit is used to determine whether the message data flows are normal.

[0052] The data storage module stores the probability of data flow belonging to each application type and the abnormal rate of each application type through a 3×n-order matrix; the elements in the first row are 1, 2, ...n, representing n different application types; the elements in the second row represent the probability of data flow belonging to each application type, and the elements in the third row represent the probability of abnormal message data flow when the message data flow belongs to each application type; each 3×n-order matrix corresponds to the flow feature data of a message, and as the flow feature data of the message increases, it is only necessary to update the data at the corresponding position in the 3×n-order matrix. When the same flow feature data appears, it is determined whether the message is normal based on the data in the matrix, and there is no need to analyze the message flow data.

[0053] In an embodiment of the present invention, a packet detection and analysis method based on deep flow inspection is provided, comprising the following steps:

[0054] S5-1, obtaining flow characteristic data of a data flow, wherein the data flow refers to a data flow generated when a message is transmitted in a network; the flow characteristic data is extracted from the data flow by a deep flow inspection module, including but not limited to a packet size, a number of packets, a flow rate, and a duration of the flow;

[0055] S5-2, based on the flow characteristic data of the data flow, the application type of the data flow is first determined by a supervised classification model. If the application type of the data flow is not unique, the process proceeds to step S5-3; if the application type of the data flow is unique, the process proceeds to step S5-4;

[0056] The steps to determine the application type of the data flow for the first time are as follows:

[0057] Obtain historical data flow information, including flow characteristics and data flow application types;

[0058] For each application type, a binary classification model is trained with the flow features of the historical data flow as input and the application type as output. A total of n binary classification models are obtained, where n is the number of application types.

[0059] Inputting the flow characteristics of the current data flow into n binary classification models to determine one or more application types that match the flow characteristics of the current data flow;

[0060] S5-3, based on the data flow application type determined for the first time, determine the probability that the data flow belongs to each application type; determine the probability that the data flow is abnormal when it belongs to different application types according to the application type and flow characteristic data of the data flow; and determine whether the data flow is normal by combining all abnormal probabilities;

[0061] First, determine the probability that the data flow belongs to each application type, which includes the following steps:

[0062] With a1, a2, ...a mIndicates the element corresponding to the data stream application type determined for the first time, where m is the number of application types that match the flow characteristics of the current data stream; when m is 2, the elements corresponding to the data stream application type are a1 and a2;

[0063] X1, X2, X3, and X4 represent the packet size, number of packets, traffic rate, and duration of the flow. Y represents the random variable of the application type. The value range of Y is {a1, a2, …a m};

[0064] Calculate that the data flow belongs to application type a1, a2, ...a m The probability of Pa1, Pa2, ...Pa m ;

[0065]

[0066] Pa1, Pa2, ...Pa m , The value range of i is a positive integer between the interval [1, m];

[0067] Set Pa1, Pa2, ...Pa m Fill in the corresponding position of the second row in the 3×n-order matrix;

[0068] Next, determining the probability that each current data flow is abnormal when belonging to each application type includes the following steps:

[0069] S8-1, taking the packet size, number of packets, flow rate and flow duration of the current data flow and the historical data flow as input, performing unsupervised classification to determine the classification cluster to which the flow feature of the current data flow belongs;

[0070] S8-2, in the classification cluster to which the flow characteristics of the current data flow belong, find out the application types a1, a2, ...a m The flow characteristic data are recorded as The superscript indicates the application type of the flow characteristic data, and the subscript b1, b2, ... b m They respectively represent the classification cluster to which the flow characteristics of the current data flow belong, and the application types are a1, a2, ...a m The number of flow feature data;

[0071] S8-3, for each flow feature data in step S8-2, calculate the probability of abnormality, denoted as

[0072] by Represents any stream feature data, the value range of x is [1, b i ] is a positive integer between ; when the application type is ai From the flow feature data of the historical data flow, determine The total number of the same stream feature data, The number of times the same flow feature data is abnormal is calculated as the ratio of the abnormal number to the total number as the flow feature data. The probability of abnormality;

[0073] S8-4, calculate the flow characteristic data The Euclidean distance between the stream feature data of the current data stream and the Euclidean distance is recorded as The Euclidean distance is calculated by the anti-correlation function e -x Mapping, we get Calculate the weight of each stream feature data

[0074] In the summation formula, j only represents a sequence number and has no practical meaning;

[0075] Determine each application type a1, a2, ...a m Abnormal probability

[0076] In the formula is the connection weight of the stream feature data; Fill in the corresponding position of the third row in the 3×n-order matrix;

[0077] S8-5, calculate the probability PE of the current data flow being abnormal, If PE is greater than or equal to the threshold, the data flow generated when the current message is transmitted in the network is judged to be abnormal; if PE is less than the threshold, the data flow generated when the current message is transmitted in the network is judged to be normal;

[0078] When the deep flow detection module determines through binary classification that the application types that match the flow characteristics of the current data flow are a1, a2, and a3, the historical flow characteristic data of application type a1 in the classification cluster to which the flow characteristics of the current data flow belong The Euclidean distances between the stream feature data of the current data stream are 0, 0, 0.1, and 0.1 respectively, and the historical stream feature data of application type a2 The Euclidean distances between the stream feature data of the current data stream are 0.3, 0.3, and 0.2 respectively, and the historical stream feature data of application type a3 When the Euclidean distances between the stream feature data of the current data stream are 0.5 and 0.4 respectively, the weight of each stream feature data in each application type is first calculated.

[0079]

[0080] When the application type of the current data flow is a1, a2, and a3, the probability of abnormal They are: The abnormal probability of the current data flow

[0081] S5-4, based on the unique application type and flow characteristic data determined by the current data flow, calculate the probability that the current data flow is abnormal. If it is greater than or equal to the threshold, the data flow generated when the current message is transmitted in the network is judged to be abnormal. If it is less than the threshold, the data flow generated when the current message is transmitted in the network is judged to be normal.

[0082] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0083] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A packet detection and analysis method based on deep flow inspection, characterized in that: The following steps are involved: S5-1, obtaining flow characteristic data of a data flow, where the data flow refers to a data flow generated when a message is transmitted in a network; S5-2, based on the flow characteristic data of the data flow, the application type of the data flow is first determined by a supervised classification model. If the application type of the data flow is not unique, the process proceeds to step S5-3; if the application type of the data flow is unique, the process proceeds to step S5-4; S5-3, based on the data flow application type determined for the first time, determine the probability that the data flow belongs to each application type; determine the probability that the data flow is abnormal when it belongs to different application types according to the application type and flow characteristic data of the data flow; and determine whether the data flow is normal by combining all abnormal probabilities; Determining the probability that the data stream is abnormal when it belongs to different application types includes the following steps: S8-1, taking the flow characteristics of the current data flow and the flow characteristics of the historical data flow as input, performing unsupervised classification, and determining the classification cluster to which the flow characteristics of the current data flow belong; S8-2, in the classification cluster to which the flow characteristics of the current data flow belong, find out the application types a1, a2, ...a m The flow characteristic data are recorded as The superscript indicates the application type of the flow characteristic data, and the subscript b1, b2, ... b m They respectively represent the classification cluster to which the flow characteristics of the current data flow belong, and the application types are a1, a2, ...a m The number of flow feature data; S8-3, for each flow feature data in step S8-2, calculate the probability of abnormality, denoted as S8-4, determine each application type a1, a2, ...a m Abnormal probability In the formula is the connection weight of the stream feature data; Fill in the corresponding position of the third row in the 3×n-order matrix; S8-5, calculate the probability PE of the current data flow being abnormal, If PE is greater than or equal to the threshold, the data flow generated when the current message is transmitted in the network is judged to be abnormal; if PE is less than the threshold, the data flow generated when the current message is transmitted in the network is judged to be normal; S5-4, judging whether the data flow is normal according to the application type and flow characteristic data of the data flow.

2. The method for packet detection and analysis based on deep flow inspection according to claim 1, characterized in that: In step S5-2, the application type of the data stream is first determined by the supervised classification model, which specifically includes the following steps: S6-1, obtaining historical data flow information, including flow characteristics and application types of data flows; S6-2, for each application type, a binary classification model is trained with the flow features of the historical data flow as input and the application type as output, and a total of n binary classification models are obtained, where n is the number of application types; S6-3, input the flow characteristics of the current data flow into n binary classification models to determine one or more application types that are consistent with the flow characteristics of the current data flow.

3. The method for packet detection and analysis based on deep flow inspection according to claim 2, characterized in that: In step S5-3, determining the probability that the data flow belongs to each application type includes the following steps: S7-1, with a1, a2, ...a m Indicates the element corresponding to the data stream application type determined for the first time, where m is the number of application types that match the flow characteristics of the current data stream; when m is 2, the elements corresponding to the data stream application type are a1 and a2; S7-2, with X1, X2, ...X k The flow characteristics of the data flow are represented by Y, which is a random variable representing the application type. The value range of Y is {a1, a2, ...a m }, k is the number of flow features; S7-3, calculates that the data flow belongs to application type a1, a2, ...a m The probability of Pa1, Pa2, ...Pa m ; Calculate m conditional probabilities, Determine Pa1, Pa2, ...Pa m , The value range of i is a positive integer between the interval [1, m]; S7-4, Pa1, Pa2, ...Pa m Fill in the corresponding position of the second row in the 3×n matrix.

4. The method for packet detection and analysis based on deep flow inspection according to claim 3, characterized in that: In step S8-4, the connection weight of the stream feature data is determined by the following steps: Calculate flow characteristic data The Euclidean distance between the stream feature data of the current data stream and the Euclidean distance is recorded as Map the Euclidean distance through the anti-correlation function f and get Calculate the weight of each stream feature data In the summation formula, j only represents a serial number and has no practical meaning.

5. The method for packet detection and analysis based on deep flow inspection according to claim 4, characterized in that: In step S8-3, for each flow feature data in step S8-2, calculating the probability of abnormality includes the following steps: by Represents any stream feature data, the value range of x is [1, b i ] is a positive integer between ; when the application type is a i From the flow feature data of the historical data flow, determine The total number of the same stream feature data, The number of times the same flow feature data is abnormal is calculated as the ratio of the abnormal number to the total number as the flow feature data. abnormal probability.

6. A packet detection and analysis system based on deep flow inspection, implementing a packet detection and analysis method based on deep flow inspection as described in any one of claims 1 to 5, characterized in that: include: Data acquisition module, deep flow detection module, message analysis module, data storage module and security module; The output end of the data acquisition module is connected to the input end of the deep flow detection module and the data storage module, and is used to collect data flow information produced by the transmission of the message in the network; The output end of the deep flow detection module is connected to the input end of the message analysis module, and is used to extract flow features from the message data flow and determine the application type of the message data flow; The output end of the message analysis module is interconnected with the input end of the security module to determine whether the message is normal; the data storage module is interconnected with the deep flow detection module and the message analysis module to store message flow data; When the message analysis module determines that the message data flow is abnormal, the security module checks the message information and the network connection status and notifies the administrator to take measures.

7. A packet detection and analysis system based on deep flow inspection according to claim 6, characterized in that: The deep flow detection module also includes a data preprocessing unit, a flow feature extraction unit, a data flow classification unit and an optimization unit; the data preprocessing unit is used to preprocess the message flow data; the flow feature extraction unit is used to extract the flow features of the message flow from the message flow data; The data flow classification unit is used to perform supervised classification and identification of application types of message data flows; the optimization unit trains and optimizes the data flow classification unit according to known message flow data and application types to improve accuracy and robustness.

8. A packet detection and analysis system based on deep flow inspection according to claim 6, characterized in that: The message analysis module also includes an unsupervised classification unit, an application type judgment unit and a detection unit; the unsupervised classification unit is used to find historical data flows related to message data flows, the application type judgment unit is used to determine the probability that the message data flows belong to different application types; the detection unit is used to determine whether the message data flows are normal.

9. A packet detection and analysis system based on deep flow inspection according to claim 6, characterized in that: The data storage module stores the probability of data flow belonging to each application type and the abnormal rate of each application type through a 3×n-order matrix; the elements in the first row are 1, 2, ...n, representing n different application types; the elements in the second row represent the probability of data flow belonging to each application type, and the elements in the third row represent the probability of abnormal message data flow when the message data flow belongs to each application type; each 3×n-order matrix corresponds to the flow feature data of a message, and as the flow feature data of the message increases, it is only necessary to update the data at the corresponding position in the 3×n-order matrix. When the same flow feature data appears, it is determined whether the message is normal based on the data in the matrix, and there is no need to analyze the message flow data.