Network traffic monitoring method, network traffic monitoring device, electronic equipment and readable medium

Abstract

The invention relates to a network flow monitoring method, a network flow monitoring device, electronic equipment and a readable medium, and relates to the technical field of flow monitoring. The method comprises the following steps: acquiring real-time network traffic, splitting the real-time network traffic into a plurality of data packets, and performing feature extraction on the plurality of data packets to obtain connection features of each data packet; wherein the connection features comprise connection identification features and time sequence features; performing feature tracking prediction, feature tracking matching and feature updating processing on the data packet according to a Kalman filtering algorithm, a Hungary algorithm and the connection features to obtain target connection features; and performing feature analysis on the target connection feature, and identifying an abnormal data packet according to a feature analysis result. According to the method, the behavior change of the traffic is dynamically identified, the accuracy and real-time performance of abnormal traffic detection and identification are improved, the network traffic monitoring effect is improved, and the application scene effect of the monitoring method is expanded.

Description

Technical Field

[0001] The present application relates to the technical field of flow monitoring, and in particular to a network flow monitoring method, a network flow monitoring device, an electronic device and a readable medium. Background Art

[0002] Network traffic monitoring is an important basis for summarizing network traffic characteristics and analyzing network behavior. It is an important means to obtain network performance parameters and is widely used in network management, intrusion monitoring, network security assessment and other fields.

[0003] The network traffic monitoring methods in related technologies are generally as follows: 1. Monitoring methods based on static rules: Detect abnormal behaviors in network traffic through a series of predefined rules or signatures. These rules usually include specific IP addresses, port numbers, protocol types, etc. However, it has poor flexibility and can only detect known threats. It is easy to cause false alarms, affecting the availability and reliability of the system. 2. Monitoring methods based on statistical analysis: Detect abnormal behaviors by analyzing the statistical characteristics of network traffic (such as traffic size, frequency, time interval, etc.) and combining them with set thresholds. However, since statistical methods are easily affected by normal traffic fluctuations, they are prone to false alarms or omissions, and corresponding to complex attack patterns, a single feature is difficult to accurately identify.

[0004] Currently, no effective solution has been proposed to address the problem that network traffic monitoring in related technologies is prone to false alarms or missed alarms, resulting in poor monitoring results, low accuracy in identifying abnormal traffic, and narrow applicable scenarios of monitoring methods. Summary of the invention

[0005] The embodiments of the present invention provide a network traffic monitoring method, a network traffic monitoring device, an electronic device and a readable medium, which at least solve the problems in the related art of poor monitoring effect due to false alarms or missed alarms, low accuracy in identifying abnormal traffic and narrow application scenarios of the monitoring method.

[0006] In order to solve the above problem, one aspect of an embodiment of the present invention provides a network traffic monitoring method, including: Acquire real-time network traffic, decompose the real-time network traffic into multiple data packets, and extract features from the multiple data packets to obtain connection features of each data packet; wherein the connection features include connection identification features and time series features; According to the Kalman filter algorithm, Hungarian algorithm and connection characteristics, feature tracking prediction, feature tracking matching and feature update processing are performed on the data packet to obtain the target connection characteristics; Perform feature analysis on the target connection features and identify abnormal data packets based on the feature analysis results.

[0007] In some embodiments, the step of performing feature tracking prediction, feature tracking matching and feature updating processing on the data packet according to the Kalman filter algorithm, the Hungarian algorithm and the connection feature to obtain the target connection feature includes: According to the Kalman filter algorithm and the connection identification features of the data packet, the time series features of the data packet are tracked and predicted to obtain the predicted time series features corresponding to the data packet at the next moment; Based on the Hungarian algorithm, the predicted time series features are tracked and matched with the actual time series features at the next moment; The time series characteristics of the data packet are updated according to the matching results and the Kalman filter algorithm to obtain the target time series characteristics.

[0008] In some of the embodiments, the method further comprises a parameter optimization step of a Kalman filter algorithm: The connection characteristics of data packets corresponding to historical network traffic are used as input data and verification data to initialize the Kalman filter, and feature tracking prediction and feature update are performed through the Kalman filter. The state parameters of the Kalman filter are optimized according to the prediction results and update results.

[0009] In some embodiments, the step of performing feature tracking prediction, feature tracking matching and feature updating processing on the data packet according to the Kalman filter algorithm, the Hungarian algorithm and the connection feature to obtain the target connection feature includes: According to the Kalman filter algorithm and the connection identification feature, the time series features of multiple data packets are respectively tracked and predicted to obtain the predicted time series features corresponding to the multiple data packets at the next moment; Based on the Hungarian algorithm, the predicted time series features corresponding to multiple data packets are tracked and matched with the actual time series features at the next moment; The time series features of multiple data packets are updated according to the matching results and the Kalman filter algorithm to obtain the target time series features corresponding to the multiple data packets.

[0010] In some of the embodiments, if the real-time network traffic is encrypted traffic, the method further includes: Based on the fingerprint analysis algorithm, the handshake message in the data packet corresponding to the encrypted traffic is extracted to obtain the connection characteristics of the data packet.

[0011] In some embodiments, the step of performing feature analysis on the target connection feature and identifying abnormal data packets according to the feature analysis result includes: The target connection feature is input into the machine learning model, and whether the data packet corresponding to the target connection feature is an abnormal data packet is determined based on the model output result; wherein the machine learning model is trained based on historical network traffic and is used to identify normal traffic or abnormal traffic.

[0012] In some of these embodiments, the method further comprises: The alarm mechanism is pre-configured, and after the abnormal traffic packet is identified, an alarm message is sent to the traffic monitoring party based on the alarm mechanism.

[0013] In order to solve the above problem, one aspect of an embodiment of the present invention provides a network traffic monitoring device, including: The data processing module is used to obtain real-time network traffic, disassemble the real-time network traffic into multiple data packets, and extract features from the multiple data packets to obtain connection features of each data packet; wherein the connection features include connection identification features and time series features; The object tracking module is used to perform feature tracking prediction, feature tracking matching and feature update processing on the data packet according to the Kalman filter algorithm, the Hungarian algorithm and the connection features to obtain the target connection features; The abnormal traffic identification module is used to perform feature analysis on target connection features and determine abnormal data packets based on the feature analysis results.

[0014] In order to solve the above problems, one aspect of an embodiment of the present invention provides an electronic device, including: a processor, and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes any one of the above network traffic monitoring methods.

[0015] In order to solve the above problems, one aspect of an embodiment of the present invention provides a non-transitory machine-readable medium storing computer instructions, where the computer instructions are used to enable a computer to execute any of the above network traffic monitoring methods.

[0016] The beneficial effects of the embodiments of the present invention are as follows: by acquiring real-time network traffic, the real-time network traffic is disassembled into multiple data packets, and feature extraction is performed on the multiple data packets to obtain connection features of each data packet; wherein the connection features include connection identification features and time series features; according to the Kalman filter algorithm, the Hungarian algorithm and the connection features, feature tracking prediction, feature tracking matching and feature update processing are performed on the data packets to obtain target connection features; feature analysis is performed on the target connection features, and a technical means for identifying abnormal data packets according to the feature analysis results overcomes the problems of poor monitoring effect, low accuracy of abnormal traffic identification and narrow application scenarios of the monitoring method in network traffic monitoring in the related technology due to easy false alarms or missed alarms, and realizes feature tracking prediction, feature tracking matching and feature update of the connection features of the data packets by using the Kalman filter algorithm and the Hungarian algorithm, thereby realizing continuous tracking of each data packet in the network traffic, and then being able to dynamically identify changes in traffic behavior, reduce or avoid false alarms and missed alarms, improve the accuracy and real-time performance of abnormal traffic detection and identification, improve the network traffic monitoring effect, and expand the application scenarios of the monitoring method.

[0017] The details of one or more embodiments of the invention are set forth in the following drawings and description so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a main flow chart of a network traffic monitoring method according to an embodiment of the present invention.

[0020] Figure 2 It is a schematic diagram of an enterprise intranet architecture and a network traffic monitoring system architecture according to an embodiment of the present invention.

[0021] Figure 3 It is a main flow chart of a network traffic monitoring method according to another embodiment of the present invention.

[0022] Figure 4 It is a main framework diagram of a network traffic monitoring device according to an embodiment of the present invention.

[0023] Figure 5 It is a schematic structural diagram of the electronic device of the present invention. DETAILED DESCRIPTION

[0024] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0025] In the field of network traffic monitoring, the monitoring methods in related technologies mainly rely on static rules and signature matching. Although these methods can detect known threats to a certain extent, they have obvious limitations when facing new attacks and encrypted traffic.

[0026] For example: 1. IDS (intrusion detection systems) such as Snort and Suricata use rule bases to match network traffic. Once a match is found, the corresponding alarm or blocking operation is triggered (a monitoring method based on static rules). Because the rules are clear, easy to understand and configure, and the rule matching speed is fast. However, it can only detect known abnormal traffic behaviors in the rule base, and cannot detect and identify new attacks and unknown threats; because the rules are too simple, a large number of false positives may occur, affecting the availability and reliability of the system; similar to encrypted traffic, the content cannot be directly parsed, resulting in many threats not being detected. 2. Threshold monitoring method: if the traffic of a certain IP address suddenly increases and exceeds the set threshold, it is considered that there may be an abnormality (a monitoring method based on statistical analysis). It has a wide range of applications and can detect multiple types of abnormal behaviors, not just specific attack modes. At the same time, it detects through statistical features, reducing the dependence on the rule base. However, the accuracy is low, because the statistical method is easily affected by normal traffic fluctuations, resulting in false positives and false negatives; and it is difficult to identify complex attacks. For complex attack modes (such as distributed denial of service attacks DDoS), a single statistical feature may not be sufficient for accurate identification. Therefore, the monitoring methods provided by related technologies are prone to false alarms or missed alarms, resulting in poor monitoring effects and low accuracy in identifying abnormal traffic.

[0027] In order to solve the above problems, the embodiment of the present invention provides a network traffic monitoring method, such as Figure 1 As shown, the network traffic monitoring method mainly includes: Step S101, obtain real-time network traffic, disassemble the real-time network traffic into multiple data packets, and extract features from the multiple data packets to obtain connection features of each data packet; wherein the connection features include connection identification features and time series features.

[0028] The above network traffic can be all traffic within the target network area; the target network area can be a specific network area (such as a corporate intranet, campus network, or other special LAN) or a specific boundary network device (a firewall or router deployed on the network boundary, a specific gateway), and the corresponding target network traffic can be captured by a traffic capture module deployed on a switch of a specific LAN, or the traffic can be copied to the traffic capture module using a SPAN port or a mirror port, or a traffic capture module can be deployed on an enterprise boundary firewall or router to capture traffic in and out of the network (such as Figure 2 As shown, the traffic collector can capture the traffic in and out of the network through the traffic capture module deployed on the firewall), or use protocols such as NetFlow or sFlow to collect traffic data from network devices. The corresponding network traffic is all the traffic within the specific network area or all the traffic in and out of the above-mentioned specific network devices. Based on this, the network traffic in a specific network area or in and out of a specific boundary network device can be fully monitored, so that potential security risks can be discovered and responded to in a timely manner through subsequent data processing. At the same time, the target network area can also refer to the entire network, and the traffic data in the entire network can be obtained by monitoring all network interfaces. It should be noted that the selection of the target network area and its corresponding network traffic acquisition range can be comprehensively considered based on multiple factors such as monitoring requirements (the type and range of traffic to be monitored), performance requirements (system processing capabilities and performance requirements), security (assets and sensitive information to be protected), and cost (deployment and operation and maintenance costs).

[0029] Among them, the connection identification features provided by the embodiment of the present invention include source IP, destination IP, source port, destination port and protocol type. The connection feature identification is used to uniquely identify the network connection, which helps to distinguish different network sessions and avoid mixing data from different sessions. Time series features include the timestamp, packet size, interval time, etc. of the data packet, which are used to analyze the behavior pattern of the traffic and help to find abnormal traffic behavior, such as sudden increase in traffic, irregular packet interval time, etc., which may be signs of attack; for example, DDoS attacks are usually manifested as a large number of small packets in a short period of time. This abnormal behavior can be effectively identified through time series features.

[0030] In some of the embodiments, the connection identification features also include TTL (Time To Live), TCP (Transmission Control Protocol) flag, etc.; the time series features also include the transmission rate of data packets, the arrival time distribution of data packets, etc.

[0031] Through the above settings, real-time network traffic is obtained, ensuring that the monitoring system can capture the traffic data in the network in real time, and disassemble the captured traffic data into individual data packets. Each data packet is used as an independent processing unit to facilitate subsequent analysis and processing. By extracting detailed connection identification features and time series features, each connection in the network can be more accurately identified and tracked, thereby improving the precision of monitoring.

[0032] In some embodiments, the method further includes: extracting features from real-time network traffic to obtain statistical features, the statistical features including the start time, end time, total number of packets, total number of bytes, and average packet size of the target network traffic connection. The statistical features can be used together with the connection identification features in the subsequent feature tracking process for the data packets to improve the accuracy of feature tracking.

[0033] In some of the embodiments, if the real-time network traffic is encrypted traffic, the method further comprises: extracting features of the handshake message in the data packet corresponding to the encrypted traffic based on a fingerprint analysis algorithm to obtain connection features of the data packet.

[0034] Encrypted traffic usually cannot be directly parsed, but some abnormal behaviors can be found through feature extraction of handshake messages. By extracting features from handshake messages in encrypted traffic through fingerprint analysis algorithms, some key information can be parsed, such as TLS version, cipher suite, certificate fingerprint, etc. This information helps to identify and analyze the behavior of encrypted traffic, improves visibility of encrypted traffic, and further expands the applicable scenarios of the above network traffic monitoring methods.

[0035] Step S102, performing feature tracking prediction, feature tracking matching and feature updating processing on the data packet according to the Kalman filter algorithm, the Hungarian algorithm and the connection characteristics to obtain the target connection characteristics.

[0036] Among them, Kalman filtering is a recursive minimum mean square error estimation method suitable for processing dynamic systems with noise. Through the Kalman filtering algorithm, the characteristics of the data packet at the next moment can be predicted in real time, the dynamic perception of network traffic behavior is improved, and the behavior changes of each data packet can be dynamically tracked.

[0037] The Hungarian algorithm is an algorithm for solving the assignment problem and is often used to solve the maximum matching problem of bipartite graphs. In this step, the Hungarian algorithm is used to match the predicted time series features with the actual time series features, reducing the probability of mismatching and improving the accuracy and efficiency of feature tracking. Especially in a high concurrent traffic environment, the Hungarian algorithm can quickly find the optimal match and ensure the real-time and stability of the system.

[0038] In some of the embodiments, the above-mentioned steps of performing feature tracking and prediction, feature tracking matching and feature updating processing on the data packet according to the Kalman filter algorithm, the Hungarian algorithm and the connection characteristics to obtain the target connection characteristics include: performing feature tracking and prediction on the time series characteristics of the data packet according to the Kalman filter algorithm and the connection identification characteristics of the data packet to obtain the predicted time series characteristics corresponding to the data packet at the next moment; performing feature tracking and matching on the predicted time series characteristics with the actual time series characteristics at the next moment based on the Hungarian algorithm; updating the time series characteristics of the data packet according to the matching results and the Kalman filter algorithm to obtain the target time series characteristics.

[0039] Through the above settings, based on the combination of Kalman filtering and Hungarian algorithm, the system can more accurately track and predict the behavior changes of network traffic, thereby improving the accuracy of anomaly detection. Through dynamic tracking and prediction, the system can accurately identify potential threats and abnormal behaviors in a complex network environment in a timely manner, reduce false positives and false negatives, and improve the overall security of the system.

[0040] In some of the embodiments, the above-mentioned step of performing feature tracking and prediction, feature tracking matching and feature updating processing on data packets according to the Kalman filter algorithm, the Hungarian algorithm and the connection characteristics to obtain the target connection characteristics includes: performing feature tracking and prediction on the time series characteristics of multiple data packets according to the Kalman filter algorithm and the connection identification characteristics, to obtain the predicted time series characteristics corresponding to the multiple data packets at the next moment; performing feature tracking and matching on the predicted time series characteristics corresponding to the multiple data packets with the actual time series characteristics at the next moment based on the Hungarian algorithm; updating the time series characteristics of the multiple data packets according to the matching results and the Kalman filter algorithm, to obtain the target time series characteristics corresponding to the multiple data packets.

[0041] The combination of Kalman filtering and the Hungarian algorithm is not only applicable to the tracking of a single data packet, but also to the tracking of multiple data packets. Through an efficient matching and updating mechanism, the system can track multiple network connections at the same time, which is suitable for high concurrent traffic environments. Through the above settings, the method provided by the embodiment of the present invention can track multiple data packets at the same time in a high concurrent traffic environment, improve the processing capacity and monitoring range of the system, and ensure that high performance and high accuracy can be maintained during peak network traffic periods.

[0042] According to a specific implementation of an embodiment of the present invention, for multiple data packets, a Kalman filter can be initialized for each data packet to perform prediction and update respectively; the Hungarian algorithm is used to batch match the predicted features and actual features of multiple data packets to ensure efficient processing of high concurrent traffic.

[0043] In some of the embodiments, the above method also includes a parameter optimization step of the Kalman filter algorithm: using the connection characteristics of the data packets corresponding to the historical network traffic as input data and verification data, initializing the Kalman filter, and performing feature tracking prediction and feature update through the Kalman filter, and optimizing the state parameters of the Kalman filter based on the prediction results and update results.

[0044] By optimizing the state parameters of the Kalman filter based on historical network traffic data, the prediction and tracking accuracy of the model can be improved. The optimized parameters can more accurately predict and track the behavior of network traffic, reduce prediction errors, and improve the overall performance of the monitoring system. At the same time, parameter optimization can reduce the computational complexity of the Kalman filter, increase processing speed, and ensure the real-time performance of the model in a high concurrent traffic environment.

[0045] According to an embodiment of the present invention, the parameter optimization of the Kalman filter is a dynamic adjustment process, and the model parameters can be continuously adjusted according to real-time data, so that the model can adapt to changes in network traffic more quickly, improve the model's adaptability and robustness, especially in complex and changeable network environments.

[0046] Step S103, performing feature analysis on the target connection features, and identifying abnormal data packets according to the feature analysis results.

[0047] By analyzing the target connection features, abnormal behaviors in network traffic can be identified more accurately. At the same time, feature analysis not only focuses on a single feature (time series feature), but can also conduct a comprehensive evaluation of multiple features such as the association and combination of time series features, connection identification features, and statistical features, so as to effectively distinguish normal traffic from abnormal traffic, reduce false positives and false negatives, and improve the accuracy of anomaly detection.

[0048] In some of the embodiments, the step of performing feature analysis on the target connection features and identifying abnormal data packets based on the feature analysis results includes: inputting the target connection features into a machine learning model, and judging whether the data packet corresponding to the target connection features is an abnormal data packet based on the model output results; wherein the machine learning model is trained based on historical network traffic and is used to identify normal traffic or abnormal traffic.

[0049] The machine learning model is trained based on historical data and can learn the characteristics of normal and abnormal traffic. The model can adapt to the ever-changing network environment, automatically learn and identify new threat patterns, and improve the system's adaptability and generalization capabilities. By inputting the target connection characteristics into the machine learning model, adaptive abnormal traffic detection can be achieved, which improves the adaptability and real-time performance of abnormal traffic detection. Compared with traditional rule-based or statistical methods, machine learning models can capture more complex patterns and features, reduce false positives and false negatives, and improve the accuracy of anomaly detection.

[0050] According to a specific implementation of an embodiment of the present invention, the feature analysis process can adopt an efficient algorithm (such as the K-means clustering algorithm or the DBSCAN clustering algorithm) to ensure that data can still be processed and analyzed in real time in a high-concurrency traffic environment, so as to improve the processing speed of the system while ensuring accuracy, and ensure real-time monitoring and response to changes in network traffic.

[0051] According to another specific implementation of the embodiment of the present invention, the feature analysis process can also be combined with a machine learning model (such as Isolation Forest, SVM (vector machine), random forest, etc.) to improve the accuracy of anomaly detection through the machine learning model.

[0052] In some of the embodiments, the method further includes: pre-configuring an alarm mechanism, and after identifying an abnormal traffic packet, sending an alarm message to a traffic monitoring party based on the alarm mechanism.

[0053] Through the pre-configured alarm mechanism, the system can send an alarm message immediately after identifying abnormal traffic packets, ensuring that network managers can understand and respond to abnormal situations in a timely manner. Through the above settings, the system's response speed is improved, and measures can be taken quickly when attacks or abnormal behaviors occur, preventing the attack from spreading or causing greater harm, and reducing potential losses.

[0054] The network traffic monitoring method provided by the embodiment of the present invention, by acquiring real-time network traffic, disassembling the real-time network traffic into multiple data packets, and extracting features from the multiple data packets to obtain connection features of each data packet; wherein the connection features include connection identification features and time series features; according to the Kalman filter algorithm, the Hungarian algorithm and the connection features, the data packets are subjected to feature tracking prediction, feature tracking matching and feature update processing to obtain target connection features; feature analysis is performed on the target connection features, and the technical means of identifying abnormal data packets according to the feature analysis results overcomes the problems of poor monitoring effect, low accuracy of abnormal traffic identification and narrow application scenarios of the monitoring method in the network traffic monitoring in the related technology due to the proneness to false alarms or omissions, and realizes feature tracking prediction, feature tracking matching and feature update of the connection features of the data packets by using the Kalman filter algorithm and the Hungarian algorithm, thereby realizing continuous tracking of each data packet in the network traffic, and then being able to dynamically identify changes in traffic behavior, reduce or avoid false alarms and omissions, improve the accuracy and real-time performance of abnormal traffic detection and identification, improve the network traffic monitoring effect, and expand the technical effect of the applicable scenarios of the monitoring method.

[0055] The embodiment of the present invention also provides a network traffic monitoring method, such as Figure 3 As shown, the network traffic monitoring method provided by the embodiment of the present invention mainly includes: 1. Traffic capture: The traffic capture module monitors the network interface and captures all inbound and outbound real-time network traffic; and disassembles the captured real-time network traffic into multiple data packets for subsequent analysis.

[0056] 2. Traffic preprocessing: Parse the data packets and extract connection identification features such as IP (source IP address, destination IP address), port (source port, destination port), protocol (type), traffic direction, and time series features such as the timestamp, packet size, and interval of the data packet. If there is encrypted traffic, the fingerprint analysis algorithm based on TLS is used to identify and record the common encryption handshake behaviors in the encrypted traffic and extract the characteristic information corresponding to the encrypted traffic.

[0057] 3. Object tracking: Each data packet and its associated connection are regarded as an "object", and its behavior changes are tracked through Kalman filtering and Hungarian algorithm, while the time series characteristics of the object are recorded.

[0058] (A) Kalman filter-based tracking algorithm: It is used to predict the next behavior pattern of a data packet. By tracking and predicting the historical traffic characteristics of the data packet, the predicted time series characteristics of the data packet are obtained.

[0059] Specifically, the Kalman filter includes the following two equations: (1) The expression of the state transition equation is as follows: In the above formula, is the state vector of the system at time k. A is the state transfer matrix, which is used to describe the state change of the system from time k-1 to time k. is the system input control vector, which is used to describe external effects. B is the control input matrix, which is used to describe the influence of the control vector on the system. is the process noise, which is usually assumed to be zero-mean Gaussian noise, and the covariance matrix equation is Q.

[0060] (2) The expression of the observation equation is as follows: In the above formula, is the predicted value at time k. H is the prediction matrix, which is used to describe how the system state is mapped to the predicted value. is the measurement noise, which is usually assumed to be zero-mean Gaussian noise with a covariance matrix of R.

[0061] (A1): The feature tracking prediction step (Predict Step) based on the Kalman filter algorithm is: at each time k, use the state at the previous time k-1 to predict the current state; or use the state at the current time k to predict the state at the next time.

[0062] Among them, the expression of state prediction (Predicted State Estimate) is as follows: In the above formula, is the state prediction value at time k. is the state estimate at the previous time step k−1.

[0063] Among them, the error covariance prediction (Predicted Error Covariance) expression is as follows: In the above formula, is the error covariance matrix of the k-state prediction at the current moment. is the covariance matrix of the state estimation error at the previous time k-1. Q is the covariance matrix of the process noise.

[0064] (A2): The feature update step (Update Step) based on the Kalman filter algorithm is: after obtaining the observation value, the predicted value is corrected by the observation value.

[0065] Among them, the expression of Kalman Gain is as follows: In the above formula, is the Kalman gain matrix, which is used to balance the influence of the prediction and the predicted value on the state estimate. R is the covariance equation of the prediction noise.

[0066] The expression of the updated state estimate is as follows: In the above formula, is the updated state estimate. is the error between the observed and predicted values, called the innovation or residual.

[0067] Among them, the expression of the updated error covariance is as follows: In the above formula, is the updated error covariance matrix.

[0068] (B) Multi-target tracking based on the Hungarian Algorithm: When multiple traffic packets change at the same time, this algorithm is used to efficiently track multiple traffic objects to ensure that no potential attack threats are missed.

[0069] Multi-target tracking technology based on the Hungarian algorithm can achieve efficient tracking through the following steps: (B1) Constructing the cost matrix Construct a cost matrix C, where the rows of the matrix represent the detected targets in the current frame and the columns of the matrix represent the existing tracking tracks. The element C(i,j) in the cost matrix represents the matching cost between the i-th detection and the j-th target track. The matching cost can usually be based on the following distance metric: (11) Euclidean distance: used for position-based matching, to calculate the distance between the center of the detection box and the trajectory prediction position. Its expression is: In the above formula, is the position of the detection box, is the predicted position of the target trajectory.

[0070] (12) Mahalanobis distance: It takes into account the variance of different dimensions and the detection noise. It is suitable for high-dimensional data. Its expression is: In the above formula, S is the covariance matrix.

[0071] (13) Intersection over Union (IOU): used to measure the overlap of detection boxes. Its expression is: In the above formula, and are the bounding boxes of the i-th detection box and the j-th tracked target, respectively.

[0072] (B2) Hungarian algorithm to find the optimal match The Hungarian algorithm is used to solve the optimal matching problem in a bipartite graph, with the goal of minimizing the total matching cost. The basic steps of the algorithm are as follows: (21) Input cost matrix C of size m×n, where m is the number of detections and n is the number of targets.

[0073] (22) Row and column normalization: Subtract the minimum value from each row and column and convert the minimum element in the cost matrix to 0 to facilitate subsequent matching.

[0074] (23) Find the match of zero elements: In the normalized matrix, find the only zero element as a preliminary matching solution.

[0075] (24) Optimize unmatched elements: adjust unmatched rows and columns and repeatedly search for zero elements until the optimal match is found.

[0076] Based on the Hungarian algorithm, the solution that minimizes the total matching cost in the cost matrix C is found through the above steps (21)-(24).

[0077] (B3) Matching results The matching result is an optimal allocation scheme, which indicates the matching relationship between the detection results in the current frame and the existing target trajectory. If some detections do not match the target, it may be a new target. If some targets do not match the detection, it may be that the target has disappeared.

[0078] (B4) Formula summary Define the cost matrix C and minimize the sum of matching costs: in, is the matching function between the observation and target trajectories. is the cost between the ith observation and the matched target trajectory.

[0079] 4. Behavior analysis: The tracked traffic behavior is passed to the behavior analysis module, and through time series analysis of the behavioral characteristics of the traffic object (such as using clustering algorithms or machine learning models to classify the traffic), abnormal behavior patterns are detected and identified, especially DDoS attacks, packet flood attacks, etc.

[0080] (41) Traffic clustering analysis: Based on the feature vector of the traffic, a clustering algorithm (such as K-means, DBSCAN) is used to classify the traffic to distinguish between normal traffic and abnormal traffic.

[0081] (42) Attack pattern recognition: Real-time attack pattern recognition is performed through machine learning models or rule-based models (such as SVM and random forest), and the behavioral characteristics of the traffic are combined to determine whether there is a network attack.

[0082] 5. Real-time response: When abnormal traffic is identified, the alarm mechanism is immediately activated, and the control module transmits the alarm information to the management end, and security operations personnel can view detailed traffic tracking and analysis results.

[0083] Based on the above-mentioned network traffic monitoring method, the embodiments of the present invention achieve the following technical effects: 1. Traffic analysis based on object tracking: Different from the traditional monitoring method based on static rules, the present invention adopts an object tracking algorithm to continuously track each "object" in the network traffic, and can dynamically identify the behavior changes of the traffic, especially the potential threats in encrypted traffic. 2. Application of multi-target tracking: Combining Kalman filtering and Hungarian algorithm, it can efficiently track multiple traffic objects in the network, which is suitable for high-concurrency traffic environments. 3. Time series analysis of traffic behavior: By analyzing the time series characteristics of traffic behavior, the present invention can discover abnormal behaviors hidden in surface traffic characteristics, which is particularly suitable for detecting complex attack patterns (such as DDoS). 4. Adaptive threat detection model: Combining machine learning and clustering algorithms, the detection model can be adaptively adjusted, which can adapt to a variety of complex network environments and improve the accuracy and real-time performance of threat detection.

[0084] Based on the above network traffic monitoring method provided by the embodiment of the present invention, the embodiment of the present invention also provides a network traffic monitoring device, such as Figure 4 As shown, the network traffic monitoring device 400 includes: The data processing module 401 is used to obtain real-time network traffic, disassemble the real-time network traffic into multiple data packets, and extract features from the multiple data packets to obtain connection features of each data packet; wherein the connection features include connection identification features and time series features.

[0085] The connection identification features provided in the embodiment of the present invention include source IP, destination IP, source port, destination port and protocol type, and the connection feature identification is used to uniquely identify the network connection. The time series features include the timestamp, packet size, interval time, etc. of the data packet, which are used to analyze the behavior pattern of the traffic and help discover abnormal traffic behavior.

[0086] Through the above settings, real-time network traffic is obtained, ensuring that the monitoring system can capture the traffic data in the network in real time, and disassemble the captured traffic data into individual data packets. Each data packet is used as an independent processing unit to facilitate subsequent analysis and processing. By extracting detailed connection identification features and time series features, each connection in the network can be more accurately identified and tracked, thereby improving the precision of monitoring.

[0087] In some of the embodiments, if the real-time network traffic is encrypted traffic, the data processing module 401 is further used to: extract features of the handshake message in the data packet corresponding to the encrypted traffic based on a fingerprint analysis algorithm to obtain connection features of the data packet.

[0088] By extracting features from handshake messages in encrypted traffic through a fingerprint analysis algorithm, some key information can be parsed out, such as TLS version, cipher suite, certificate fingerprint, etc. This information helps to identify and analyze the behavior of encrypted traffic, improves visibility into encrypted traffic, and further expands the applicable scenarios of the above network traffic monitoring methods.

[0089] The object tracking module 402 is used to perform feature tracking prediction, feature tracking matching and feature update processing on the data packet according to the Kalman filter algorithm, the Hungarian algorithm and the connection characteristics to obtain the target connection characteristics.

[0090] The Kalman filter algorithm can predict the characteristics of the data packet at the next moment in real time, improve the dynamic perception of network traffic behavior, and realize dynamic tracking of the behavior changes of each data packet. The predicted time series characteristics are matched with the actual time series characteristics through the Hungarian algorithm, which reduces the probability of mismatching and improves the accuracy and efficiency of feature tracking. Especially in a high-concurrency traffic environment, the Hungarian algorithm can quickly find the optimal match to ensure the real-time and stability of the system.

[0091] In some of the embodiments, the object tracking module 402 is further used to: perform feature tracking and prediction on the time series features of the data packet based on the Kalman filter algorithm and the connection identification features of the data packet to obtain the predicted time series features corresponding to the data packet at the next moment; perform feature tracking and matching on the predicted time series features with the actual time series features at the next moment based on the Hungarian algorithm; and update the time series features of the data packet based on the matching results and the Kalman filter algorithm to obtain the target time series features.

[0092] Through the above settings, based on the combination of Kalman filtering and Hungarian algorithm, the system can more accurately track and predict the behavior changes of network traffic, thereby improving the accuracy of anomaly detection. Through dynamic tracking and prediction, the system can accurately identify potential threats and abnormal behaviors in a complex network environment in a timely manner, reduce false positives and false negatives, and improve the overall security of the system.

[0093] In some of the embodiments, the object tracking module 402 is further used to: perform feature tracking and prediction on the time series features of multiple data packets according to the Kalman filter algorithm and the connection identification features, and obtain the predicted time series features corresponding to the multiple data packets at the next moment; perform feature tracking and matching on the predicted time series features corresponding to the multiple data packets with the actual time series features at the next moment based on the Hungarian algorithm; update the time series features of the multiple data packets according to the matching results and the Kalman filter algorithm, and obtain the target time series features corresponding to the multiple data packets.

[0094] The combination of Kalman filtering and the Hungarian algorithm is not only applicable to the tracking of a single data packet, but also to the tracking of multiple data packets. Through an efficient matching and updating mechanism, the system can track multiple network connections at the same time, which is suitable for high concurrent traffic environments. Through the above settings, the method provided by the embodiment of the present invention can track multiple data packets at the same time in a high concurrent traffic environment, improve the processing capacity and monitoring range of the system, and ensure that high performance and high accuracy can be maintained during peak network traffic periods.

[0095] According to a specific implementation of an embodiment of the present invention, for multiple data packets, a Kalman filter can be initialized for each data packet to perform prediction and update respectively; the Hungarian algorithm is used to batch match the predicted features and actual features of multiple data packets to ensure efficient processing of high concurrent traffic.

[0096] In some of the embodiments, the network traffic monitoring device 400 further includes a parameter optimization module, which is used to: use the connection characteristics of data packets corresponding to historical network traffic as input data and verification data, initialize the Kalman filter, perform feature tracking prediction and feature update through the Kalman filter, and optimize the state parameters of the Kalman filter according to the prediction results and update results.

[0097] By optimizing the state parameters of the Kalman filter based on historical network traffic data, the prediction and tracking accuracy of the model can be improved. The optimized parameters can more accurately predict and track the behavior of network traffic, reduce prediction errors, and improve the overall performance of the monitoring system. At the same time, parameter optimization can reduce the computational complexity of the Kalman filter, increase processing speed, and ensure the real-time performance of the model in a high concurrent traffic environment.

[0098] The abnormal traffic identification module 403 is used to perform feature analysis on target connection features and determine abnormal data packets according to the feature analysis results.

[0099] By analyzing the target connection features, abnormal behaviors in network traffic can be identified more accurately. At the same time, feature analysis not only focuses on a single feature (time series feature), but can also conduct a comprehensive evaluation of multiple features such as the association and combination of time series features, connection identification features, and statistical features, so as to effectively distinguish normal traffic from abnormal traffic, reduce false positives and false negatives, and improve the accuracy of anomaly detection.

[0100] In some of the embodiments, the abnormal traffic identification module 403 is also used to: input the target connection feature into the machine learning model, and determine whether the data packet corresponding to the target connection feature is an abnormal data packet based on the model output result; wherein the machine learning model is trained based on historical network traffic and is used to identify normal traffic or abnormal traffic.

[0101] The machine learning model is trained based on historical data and can learn the characteristics of normal and abnormal traffic. The model can adapt to the ever-changing network environment, automatically learn and identify new threat patterns, and improve the system's adaptability and generalization capabilities. By inputting the target connection characteristics into the machine learning model, adaptive abnormal traffic detection can be achieved, which improves the adaptability and real-time performance of abnormal traffic detection. Compared with traditional rule-based or statistical methods, machine learning models can capture more complex patterns and features, reduce false positives and false negatives, and improve the accuracy of anomaly detection.

[0102] In some of the embodiments, the network traffic monitoring device 400 further includes an alarm module, which is used to: pre-configure an alarm mechanism, and after identifying an abnormal traffic packet, send an alarm message to the traffic monitoring party based on the alarm mechanism.

[0103] Through the pre-configured alarm mechanism, the system can send an alarm message immediately after identifying abnormal traffic packets, ensuring that network managers can understand and respond to abnormal situations in a timely manner. Through the above settings, the system's response speed is improved, and measures can be taken quickly when attacks or abnormal behaviors occur, preventing the attack from spreading or causing greater harm, and reducing potential losses.

[0104] The network traffic monitoring device provided by the embodiment of the present invention uses a data processing module to obtain real-time network traffic, disassembles the real-time network traffic into multiple data packets, and extracts features from the multiple data packets to obtain connection features of each data packet; wherein the connection features include connection identification features and time series features; the object tracking module is used to perform feature tracking prediction, feature tracking matching and feature update processing on the data packets according to the Kalman filter algorithm, the Hungarian algorithm and the connection features to obtain the target connection features; the abnormal traffic identification module is used to perform feature analysis on the target connection features and determine the abnormal data packets according to the feature analysis results. The continuous tracking of each data packet in the network traffic is achieved, and the behavior changes of the traffic can be dynamically identified, the situations of false alarms and missed alarms can be reduced or avoided, the accuracy and real-time performance of abnormal traffic detection and identification can be improved, the network traffic monitoring effect can be improved, and the technical effect of expanding the applicable scenarios of the monitoring method can be expanded.

[0105] An embodiment of the present invention further provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present invention.

[0106] The embodiment of the present invention further provides a computer program product, including a computer program, wherein the computer program is used to make the computer perform the method of the embodiment of the present invention when executed by a processor of the computer. The computer program product should be understood as a software product that mainly implements the above method of the present application through a computer program.

[0107] An embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory in communication with the at least one processor. The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the electronic device executes the method of the embodiment of the present invention.

[0108] refer to Figure 5 , a block diagram of an electronic device that can be used as a server or client of an embodiment of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0109] like Figure 5 As shown, the electronic device includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0110] Multiple components in the electronic device are connected to the I / O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 can be any type of device that can input information to the electronic device, and the input unit 506 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 507 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 can include but is not limited to a disk, an optical disk. The communication unit 509 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, and / or a wireless communication transceiver, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0111] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a CPU, a graphics processing unit (GPU), various special artificial intelligence (AI) computing units, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present invention may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via a ROM 502 and / or a communication unit 509. In some embodiments, the computing unit 501 may be configured to perform the above-described method in any other appropriate manner (e.g., by means of firmware).

[0112] The computer programs for implementing the methods of the embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer programs are executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer programs may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0113] In the context of an embodiment of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, or infrared systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0114] It should be noted that the term "including" and its variations used in the embodiments of the present invention are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0115] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0116] The various steps described in the method implementation methods provided in the embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method implementation methods may include additional steps and / or omit the steps shown. The scope of protection of the present invention is not limited in this respect.

[0117] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0118] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the attached claims.

Claims

1. A network traffic monitoring method, characterized in that: include: Acquire real-time network traffic, decompose the real-time network traffic into multiple data packets, and extract features from the multiple data packets to obtain connection features of each data packet; wherein the connection features include connection identification features and time series features; According to the Kalman filter algorithm, the Hungarian algorithm and the connection feature, the data packet is subjected to feature tracking prediction, feature tracking matching and feature updating processing to obtain the target connection feature; Perform feature analysis on the target connection features, and identify abnormal data packets based on the feature analysis results.

2. The method according to claim 1, characterized in that The step of performing feature tracking prediction, feature tracking matching and feature updating processing on the data packet according to the Kalman filter algorithm, the Hungarian algorithm and the connection feature to obtain the target connection feature comprises: Perform feature tracking and prediction on the time series features of the data packet according to the Kalman filter algorithm and the connection identification features of the data packet to obtain the predicted time series features corresponding to the data packet at the next moment; Based on the Hungarian algorithm, the predicted time series features are tracked and matched with the actual time series features at the next moment; The time series characteristics of the data packet are updated according to the matching result and the Kalman filter algorithm to obtain the target time series characteristics.

3. The method according to claim 2, characterized in that The method also includes a parameter optimization step of the Kalman filter algorithm: The connection characteristics of data packets corresponding to historical network traffic are used as input data and verification data to initialize the Kalman filter, and feature tracking prediction and feature update are performed through the Kalman filter, and the state parameters of the Kalman filter are optimized according to the prediction results and update results.

4. The method according to claim 2, characterized in that: The step of performing feature tracking prediction, feature tracking matching and feature updating processing on the data packet according to the Kalman filter algorithm, the Hungarian algorithm and the connection feature to obtain the target connection feature comprises: Perform feature tracking and prediction on the time series features of the multiple data packets respectively according to the Kalman filter algorithm and the connection identification features to obtain predicted time series features corresponding to the multiple data packets at the next moment; Based on the Hungarian algorithm, the predicted time series features corresponding to the multiple data packets are tracked and matched with the actual time series features at the next moment; The time series features of the multiple data packets are updated according to the matching results and the Kalman filter algorithm to obtain target time series features corresponding to the multiple data packets respectively.

5. The method according to claim 1, characterized in that If the real-time network traffic is encrypted traffic, the method further includes: Based on the fingerprint analysis algorithm, feature extraction is performed on the handshake message in the data packet corresponding to the encrypted traffic to obtain the connection feature of the data packet.

6. The method according to claim 1, characterized in that The step of performing feature analysis on the target connection feature and identifying abnormal data packets according to the feature analysis result comprises: The target connection feature is input into a machine learning model, and whether the data packet corresponding to the target connection feature is an abnormal data packet is determined based on the output result of the model; wherein the machine learning model is trained based on historical network traffic and is used to identify normal traffic or abnormal traffic.

7. The method according to claim 1, characterized in that The method further comprises: An alarm mechanism is pre-configured, and after an abnormal traffic packet is identified, an alarm message is sent to a traffic monitoring party based on the alarm mechanism.

8. A network traffic monitoring device, characterized in that: include: A data processing module is used to obtain real-time network traffic, disassemble the real-time network traffic into multiple data packets, and extract features from the multiple data packets to obtain connection features of each data packet; wherein the connection features include connection identification features and time series features; An object tracking module is used to perform feature tracking prediction, feature tracking matching and feature update processing on the data packet according to the Kalman filter algorithm, the Hungarian algorithm and the connection feature to obtain the target connection feature; The abnormal traffic identification module is used to perform feature analysis on the target connection features and determine abnormal data packets according to the feature analysis results.

9. An electronic device, comprising: A processor, and a memory storing a program, wherein the program comprises instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory machine-readable medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

Images