Traffic monitoring method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a traffic monitoring method and device, equipment and a storage medium. The method comprises the following steps: acquiring current traffic data received by a target area and a current feature vector of the current traffic data; obtaining a KL divergence result according to the current feature vector of the current traffic data and a clustering result of a Gaussian mixture model corresponding to the target area on historical traffic data in the target area; the Gaussian mixture model is obtained through training according to historical feature vectors of historical flow data in the target area; and obtaining an abnormal monitoring result of the current traffic data according to the KL divergence result. The method is used for achieving the effect of improving the complexity of the optimized flow monitoring process.

Description

Technical Field

[0001] The present application relates to the fields of flow monitoring technology and computer technology, and in particular to a flow monitoring method, apparatus, device, and storage medium. Background Art

[0002] With the deepening of digital transformation and the increasing complexity of data center network architectures, security operations have become critical to ensuring business continuity and data security. In the field of network security, traffic monitoring and log analysis, as key tools for threat detection, incident response, and compliance auditing, are widely used in various security devices and systems, including firewalls, WAFs, and SIEMs. Faced with the continued growth of network traffic and the increasingly refined network segmentation, the precise configuration and efficient management of monitoring areas have become crucial issues in improving security operations efficiency.

[0003] In existing technologies, when network equipment is upgraded or the security architecture is adjusted, operations and maintenance personnel usually manually configure the regional affiliation of security equipment in the traffic monitoring system based on the network topology and security policy, and set corresponding log collection and parsing rules to monitor traffic.

[0004] However, this process relies on manual judgment and experience, and requires reference to change documents, configuration templates and related specifications to complete the writing, comparison and debugging of configuration files item by item. Although scripts can be used to assist in some scenarios, manual review and confirmation are still required, making the overall process more complicated. Summary of the Invention

[0005] The embodiments of the present application provide a flow monitoring method, apparatus, device, and storage medium to achieve the effect of improving the complexity of optimizing the flow monitoring process.

[0006] In a first aspect, an embodiment of the present application provides a flow monitoring method, comprising:

[0007] Obtaining the current traffic data received by the target area and the current feature vector of the current traffic data;

[0008] The KL divergence result is obtained based on the clustering result of the historical traffic data in the target area according to the current feature vector of the current traffic data and the Gaussian mixture model corresponding to the target area; the Gaussian mixture model is trained based on the historical feature vector of the historical traffic data in the target area;

[0009] According to the KL divergence results, the abnormal monitoring results of the current traffic data are obtained.

[0010] In a second aspect, an embodiment of the present application provides a flow monitoring device, comprising:

[0011] An acquisition module, used to acquire current traffic data received in a target area and a current feature vector of the current traffic data;

[0012] The first obtaining module is used to obtain a KL divergence result based on the current feature vector of the current traffic data and the Gaussian mixture model corresponding to the target area for clustering the historical traffic data in the target area; the Gaussian mixture model is trained based on the historical feature vector of the historical traffic data in the target area;

[0013] The second obtaining module is used to obtain the abnormal monitoring result of the current traffic data according to the KL divergence result.

[0014] In a third aspect, an embodiment of the present application provides a flow monitoring device, including: a memory, a processor;

[0015] Memory stores computer-executable instructions;

[0016] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0019] The traffic monitoring method, apparatus, device, and storage medium provided by the embodiments of the present application obtain the current traffic data received by the target area and the current feature vector of the current traffic data; cluster the historical traffic data in the target area according to the current feature vector of the current traffic data and the Gaussian mixture model corresponding to the target area to obtain the KL divergence result; the Gaussian mixture model is trained based on the historical feature vector of the historical traffic data in the target area; and the abnormal monitoring result of the current traffic data is obtained according to the KL divergence result. By introducing the Gaussian mixture model to cluster the historical traffic data of the target area, a fine-grained characterization of complex traffic behavior is achieved, and the KL divergence is calculated based on the feature vector of the current traffic data, and the difference between the current traffic data and the historical pattern is quantified from the probability distribution level, thereby transforming the monitoring process that originally relied on manual rules and static thresholds into a dynamic analysis mechanism based on statistical learning. As a result, not only the accuracy of abnormal traffic identification is improved, but also the traditional multi-step, multi-condition manual judgment logic is replaced by mathematical modeling, making the entire traffic monitoring process more automated and intelligent, thereby effectively simplifying the judgment chain in actual operation and maintenance, and improving the adaptability and efficiency of monitoring process optimization and configuration adjustment in the face of complex network environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] Figure 1 Schematic diagram of the flow monitoring method provided in this application;

[0022] Figure 2 A flow chart of a flow monitoring method provided in this application;

[0023] Figure 3 A flow chart of another flow monitoring method provided by this application;

[0024] Figure 4 A schematic diagram of the structure of the flow monitoring device provided in this application;

[0025] Figure 5 This is a schematic diagram of the structure of the flow monitoring device provided in this application.

[0026] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0028] First, let’s explain the terms involved in this application:

[0029] Traffic monitoring involves collecting, analyzing, and collecting statistics on network data transmission in real time or periodically to understand the state of network communications, identify abnormal behavior, evaluate network performance, and ensure network security. Traffic monitoring involves observing key information such as source / destination IP addresses, port numbers, protocol types, packet sizes, and transmission rates. It is widely used in scenarios such as network operations and maintenance, security protection, service quality assurance, and compliance auditing.

[0030] In the existing technology, during the process of network equipment upgrades, architecture adjustments, or the deployment of new security equipment, errors are prone to occur in the configuration of traffic monitoring areas of security equipment in security operation systems, which mainly manifest as two types of problems: missing log reports and log parsing errors. On the one hand, key traffic may not be collected due to incorrect access to the monitoring area, resulting in the inability to deliver logs to security systems such as WAF; on the other hand, incorrect configuration of traffic area labels (such as mistakenly labeling office area traffic as a test area) will mislead threat analysis and affect the accuracy of linkage blocking and alarm rules. The root cause of such problems is that the current network environment is becoming increasingly complex, with an increase in data center partitions and rapid traffic growth, while operation and maintenance still relies heavily on manual verification of configuration files and manual provision of parsing instructions, which is inefficient and prone to errors. In addition, there is a lack of unified regional parsing specifications and automated correction mechanisms. Most existing tools can only passively issue alarms and cannot actively output executable configuration optimization suggestions, further exacerbating the occurrence and continued impact of configuration errors.

[0031] The traffic monitoring method provided in the embodiment of the present application realizes the transition from "manual rule judgment" to "automated statistical analysis" by introducing a Gaussian mixture model to cluster model historical traffic and combining it with KL divergence to analyze the probability distribution differences of current traffic. It effectively solves the problems of easy configuration errors of traffic monitoring areas, low efficiency of manual inspection, and confusing log parsing in network change scenarios.

[0032] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0033] Figure 1 A schematic diagram of the flow monitoring method provided in this application, such as Figure 1 As shown, the specific application scenario of the present application is a traffic monitoring system, which can be a server, and the server can be a computer, mobile phone, tablet and other devices. The embodiment of the present application does not set the subject that executes the traffic monitoring method, as long as it can execute the acquisition of the current traffic data received by the target area and the current feature vector of the current traffic data; clustering results of historical traffic data in the target area according to the current feature vector of the current traffic data and the Gaussian mixture model corresponding to the target area, to obtain the KL divergence result; the Gaussian mixture model is trained according to the historical feature vector of the historical traffic data in the target area; according to the KL divergence result, the abnormal monitoring result of the current traffic data is obtained.

[0034] Figure 2 A flow chart of a flow monitoring method provided by this application, such as Figure 2 As shown, the method includes:

[0035] S201: Acquire current traffic data received in the target area and a current feature vector of the current traffic data.

[0036] The target zone can refer to a logical or physical area within the network that is used for traffic classification management and security policy control. For example, in a data center, the network can be divided into core business areas, test areas, office areas, DMZ areas, etc. based on function. Alternatively, it can be divided into Internet areas, third-party external areas, WAN areas, office areas, etc. Each area has different network service functions, and security devices will be configured with corresponding monitoring and protection policies based on the area they are located in.

[0037] Current traffic data may refer to the network communication data flow flowing into or out of the target area within a specific time period, including but not limited to information such as source IP address, destination IP address, port number, protocol type, packet size, and transmission frequency.

[0038] The current feature vector refers to key statistical features extracted from current traffic data, describing the traffic behavior in the area. This feature vector can include information such as IP segment distribution (derived through CIDR aggregation analysis), the top 10 most frequently accessed ports, and the proportion of various protocols (such as TCP / UDP / ICMP).

[0039] In the embodiment of the present application, obtaining the current traffic data received by the target area and the current feature vector of the current traffic data includes:

[0040] Get current traffic data and device configuration files of current traffic data;

[0041] Perform feature vector extraction processing on the current traffic data to obtain the current feature vector of the current traffic data;

[0042] Determine the target area to which the current traffic data belongs based on the device configuration file of the current traffic data.

[0043] The device profile for current traffic data refers to a collection of configuration information on a security or network device, describing the device's functional role and regional affiliation within the network. This profile can include access control list (ACL) rules, VLAN mappings, monitoring zone definitions, and IP address allocation tables, reflecting the source and destination of traffic processed by the device, as well as the basis for the network zone to which it belongs.

[0044] Determining the target area to which the current traffic data belongs based on the device configuration file of the current traffic data can refer to parsing the area definition information in the device configuration file, such as the security zone involved in the ACL rule, the logical partition to which the VLAN interface belongs, the monitoring area tag, or the deployment location corresponding to the IP address segment, and combining the overall network topology to determine the target area to which the traffic currently received or forwarded by the device should belong, such as the "production area", "DMZ area", or "office area".

[0045] Extracting feature vectors from current traffic data can refer to extracting representative statistical features from the collected current network traffic and converting them into structured numerical vectors. For example, unstructured or semi-structured network behavior data, such as IP segment distribution, top 10 ports, and protocol percentages, can be uniformly converted into a set of fixed-length, computable numerical sequences through statistical analysis, numerical encoding, and normalization. This can include quantifying the access frequency of each CIDR segment into a proportional value, converting port usage into a frequency or Boolean flag, and expressing the percentage of protocol types as a percentage. Features of different dimensions are then standardized and ultimately combined into a complete feature vector in a preset format. This vector is then output as an array or JSON format for subsequent analysis and decision-making.

[0046] In an embodiment of the present application, by parsing the field information related to network area division in the device configuration file and combining it with the preset area division rules, automatic identification of traffic ownership is achieved. For example, key information such as ACL rules, VLAN interface configuration, monitoring area tags, IP address allocation table, etc. are extracted from the device configuration file to analyze the location of the device in the network topology and the security domain to which it belongs; then, based on the defined area mapping strategy (such as VLAN ID corresponding to area name, IP segment belonging to business unit, etc.), the target area to which the received or sent traffic should belong is determined, such as "production area", "test area" or "DMZ area", thereby achieving automatic matching and classification of traffic areas without manual intervention.

[0047] S202. Clustering results of historical traffic data in the target area based on the current feature vector of the current traffic data and the Gaussian mixture model corresponding to the target area to obtain KL divergence results; the Gaussian mixture model is trained based on the historical feature vector of historical traffic data in the target area.

[0048] A Gaussian mixture model refers to an unsupervised machine learning model based on probability distribution, used for cluster analysis and distribution modeling of complex data sets. In the present embodiment, the model is trained based on historical feature vectors extracted from historical traffic data in the target area. It can learn and characterize the multimodal distribution characteristics under different traffic patterns, thereby providing a probabilistic attribution basis for new traffic features.

[0049] The clustering result of the Gaussian mixture model on the historical traffic data in the target area can refer to the division of traffic samples with similar communication behaviors into different Gaussian components after probabilistic modeling of the feature vectors extracted from the historical traffic in the target area through the Gaussian mixture model, thereby forming several representative traffic pattern clusters; each cluster corresponds to a Gaussian distribution, and its mean and covariance matrix can characterize the typical characteristics of this type of traffic.

[0050] The KL divergence result may refer to a value obtained by calculating the Kullback-Leibler divergence (KL divergence) between two probability distributions, which is used to quantify the degree of difference between the two distributions. In the embodiment of the present application, when the characteristic distribution of the current traffic data is compared with the output of a Gaussian mixture model trained based on historical traffic data, the KL divergence result indicates the degree of deviation between the current traffic pattern and the historical "normal" pattern.

[0051] In this embodiment of the present application, the KL divergence result is obtained based on the current feature vector of the current traffic data, the clustering result of the Gaussian mixture model corresponding to the target area on the historical traffic data in the target area, and the feature mean and covariance matrix of the Gaussian mixture model, including:

[0052] Determine the characteristic distribution of the current flow according to the current characteristic vector of the current flow data;

[0053] The KL divergence result is obtained based on the clustering results of the historical traffic data in the target area and the current feature vector of the current traffic data according to the Gaussian mixture model corresponding to the target area.

[0054] Among them, the KL divergence result satisfies:

[0055]

[0056] Where x is the current eigenvector, P(x) represents the characteristic distribution of the current traffic, and Q(x) represents the expected probability density of the eigenvector x output by the Gaussian mixture model in the clustering result based on the mean and covariance matrix.

[0057] P(x) can represent a probability distribution formed by normalizing the feature vector of the current traffic.

[0058] Q(x) satisfies:

[0059]

[0060] Wherein, K is the number of clusters in the Gaussian mixture model, for example, K can be 5; αk is the weight of the k-th Gaussian component; μk can be the characteristic mean of the k-th Gaussian component, Σk can be the covariance matrix of the k-th Gaussian component; N(x|μk, Σk) can represent the probability density value of x under the Gaussian distribution.

[0061] In this embodiment of the present application, the method for obtaining the Gaussian mixture model may include:

[0062] Obtaining historical traffic data, device configuration files of the historical traffic data, and historical feature vectors of the historical traffic data;

[0063] Segment the historical traffic data based on the device configuration file of the historical traffic data to determine the segmented traffic data belonging to different monitoring areas;

[0064] The segmented traffic data is input into the preset initial Gaussian mixture model for unsupervised model training to obtain a Gaussian mixture model;

[0065] According to the characteristic mean and covariance matrix of the Gaussian mixture model, the clustering result of the Gaussian mixture model on the historical traffic data in the target area is obtained.

[0066] The Gaussian Mixture Model (GMM) can be trained using the Expectation-Maximization (EM) algorithm. This algorithm estimates the parameters of multiple Gaussian distributions (including the mean, covariance matrix, and weight of each Gaussian distribution) based on the historical feature vectors of historical traffic data to better fit the overall data distribution. The Expectation-Maximization (EM) algorithm refers to an iterative optimization algorithm commonly used for parameter estimation in probabilistic models with latent variables. Its core concept is to gradually approximate the maximum likelihood estimate of the model parameters by alternating between two steps—the E-step (expectation step) and the M-step (maximization step)—when the observed data is incomplete or there are unknown variables. In the E-step, the expected values ​​of the latent variables are estimated based on the current parameters; in the M-step, these expected values ​​are used to re-estimate the model parameters to maximize the likelihood function of the data.

[0067] Based on this, the specific training process of the Gaussian mixture model can be:

[0068] First, the feature vectors extracted from the historical traffic of the target area are combined into a training dataset. Then, the number of Gaussian components in the Gaussian mixture model (i.e., the number of clusters K) is set, and the mean, covariance matrix, and weight of each Gaussian distribution are initialized.

[0069] Next, the model parameters are iteratively optimized using the EM algorithm: in the E-step (expectation step), for each sample feature vector in the training data set, the posterior probability (i.e., soft assignment) of the feature vector belonging to each Gaussian component is calculated based on the parameters of the current Gaussian distributions, that is, the probability that the sample was generated by the K-th Gaussian distribution; in the M-step (maximization step), based on the probability weights obtained in the E-step, the parameters of each Gaussian component are re-estimated, including updating the mean vector, covariance matrix, and weights to maximize the overall data likelihood function; the E-step and M-step are repeated until the model converges or the preset maximum number of iterations is reached;

[0070] Finally, a Gaussian mixture model with a multimodal distribution structure that can reflect the typical traffic pattern of the target area is obtained.

[0071] Therefore, a Gaussian mixture model is used to probabilistically model the feature vectors extracted from the historical traffic data in the target area. Using the characteristic means and covariance matrices of the trained Gaussian components, the historical traffic is divided into multiple clusters with similar communication characteristics. Each cluster represents a typical traffic pattern, thus forming a structured clustering result for the historical traffic data.

[0072] In this embodiment of the present application, the traffic data in the current traffic data and the traffic data in the historical traffic data both include the source IP address, the destination IP address, the source port, the destination port and the protocol type.

[0073] The source IP address refers to the IP address used by the party initiating data transmission during network communication. It identifies the origin of the data packet and is used to uniquely locate the sending device within the network. By analyzing the distribution of source IP addresses, it is possible to identify the source of traffic, detect abnormal access behavior, and track potential security threats.

[0074] The destination IP address refers to the IP address of the target device that receives data transmission in network communication. It is the final destination of the data packet transmission and determines which host or service the data should be routed to.

[0075] The source port may refer to the port number used by the sender when establishing a network connection. It is usually dynamically assigned by the operating system and is used to identify a specific communication process on the local host.

[0076] The Destination Port field indicates the port number on the receiving host that listens for and processes a specific service, such as port 80 for HTTP and port 443 for HTTPS. This field indicates the application or service to which the packet should be delivered for processing.

[0077] Protocol Type refers to the transport layer protocol used in network communications. Common protocols include TCP (Transmission Control Protocol), UDP (User Datagram Protocol), and ICMP (Internet Control Message Protocol). Different protocols are suitable for different communication scenarios, such as TCP for reliable connections, UDP for low-latency transmission, and ICMP for network diagnostics.

[0078] In the embodiment of the present application, the vector type in the current feature vector and the vector type in the historical feature vector both include IP segment distribution, top 10 ports, and protocol proportions.

[0079] IP Segment Distribution refers to the frequency of occurrence of different IP address segments (usually expressed in CIDR notation, such as 192.168.1.0 / 24) in network communications and their proportion in total traffic. By analyzing IP segment distribution, it is possible to identify which subnets or external networks frequently interact with the target area, helping to identify unusual access patterns or potential security threats, such as a large number of requests from unknown or suspicious IP segments.

[0080] Top 10 Ports may refer to the ten most frequently used port numbers in network traffic. These ports may be source ports or destination ports.

[0081] Protocol Proportion refers to the proportion of different types of transport layer protocols (such as TCP, UDP, and ICMP) in network communications. By calculating the relative number of these protocols in the overall traffic, we can assess the main types of network communications and their distribution.

[0082] S203. Obtain an abnormal monitoring result of the current traffic data according to the KL divergence result.

[0083] Among them, the abnormal monitoring results can refer to the quantitative analysis of the difference between the current traffic behavior and the historical normal pattern (such as calculating the KL divergence) to determine whether the current traffic data deviates from the expected behavior range, and output the judgment result of whether an abnormality has occurred accordingly.

[0084] If the KL divergence result representing the degree of difference exceeds the set threshold, it is marked as "abnormal", indicating that there may be configuration errors, illegal access or potential attack behavior; otherwise it is marked as "normal", indicating that the traffic behavior conforms to the typical pattern described by the historical model.

[0085] In this embodiment of the present application, the abnormal monitoring result of the current traffic data is obtained according to the KL divergence result, including:

[0086] When the KL divergence value corresponding to the KL divergence result is greater than the preset KL divergence value, the abnormal type of the current traffic data is determined based on the device configuration file of the current traffic data. The abnormal types of the current traffic data include configuration errors that indicate that newly added IP segments are not covered, traffic drift in the wrong monitoring area, and parsing errors with incorrect monitoring area naming.

[0087] According to the abnormal type of the current traffic data, the abnormal monitoring result of the current traffic data is obtained.

[0088] Among them, the preset KL divergence value can be a pre-set KL divergence value, which can be set arbitrarily as needed. For example, the preset KL divergence value can be 1.5. Then, if the KL divergence value corresponding to the KL divergence result is greater than 1.5, the abnormal monitoring result represents an abnormality. In any case, when the KL divergence value corresponding to the KL divergence result is less than or equal to 1.5, the abnormal monitoring result represents a normal state.

[0089] When the abnormal monitoring results indicate an abnormality, it is necessary to determine the specific abnormal situation and further analyze the specific abnormal situation to analyze and determine the specific abnormal type. By analyzing the ACL rules, VLAN mapping, monitoring area definition, and IP address allocation information in the device configuration, the current traffic characteristics are compared with the expected configuration to identify possible configuration or behavior deviations. For example:

[0090] 1. If an IP segment not defined in the target area configuration range appears in the current traffic, it is determined to be a configuration error that does not cover the newly added IP segment;

[0091] 2. If an IP segment not defined in the target area configuration range appears in the current traffic, it is determined to be a configuration error that does not cover the newly added IP segment;

[0092] 3. If the area label used in the device configuration does not match the actual traffic attribution, resulting in confusion in log classification, it is determined to be a parsing error due to incorrect monitoring area naming.

[0093] In this way, refined identification of abnormal types can be achieved to obtain more detailed and accurate abnormal monitoring results.

[0094] In the embodiment of the present application, the method further includes:

[0095] When the abnormal monitoring result of the current traffic data represents that the current traffic data is normal, the Gaussian mixture model is updated according to the current traffic data and the historical traffic data.

[0096] When the abnormal monitoring results of current traffic data indicate that the traffic is behaving normally, this indicates that its characteristics are consistent with the typical patterns described by the historical model or that the deviation is within an acceptable range. At this point, the feature vector of the traffic can be incorporated into the historical dataset, and the Gaussian mixture model can be incrementally updated regularly. By dynamically adjusting the original model's parameters, such as the mean and covariance matrix, the model can adapt to changing trends in the network environment, such as new business traffic and evolving access patterns, thereby improving the accuracy of subsequent traffic analysis and the model's adaptability.

[0097] In the embodiment of the present application, the method further includes:

[0098] Generate correction instructions corresponding to the abnormal monitoring results according to the abnormal monitoring results of the current traffic data;

[0099] Adjust the current flow data and the device configuration file of the current flow data according to the correction instructions corresponding to the abnormal monitoring results;

[0100] After adjusting the current traffic data and the device configuration file of the current traffic data, the step of inputting the current feature vector of the current traffic data into the Gaussian mixture model to obtain the probability distribution structure of the current feature vector output by the Gaussian mixture model is repeated until the KL divergence value corresponding to the KL divergence result is less than or equal to the preset KL divergence value.

[0101] After determining the abnormal monitoring results, correction instructions corresponding to the abnormal monitoring results can be automatically generated to help users make repairs.

[0102] For exceptions caused by configuration omissions (such as newly added IP segments not included in monitoring), you can generate corresponding ACL rule addition commands, for example: iptables-AINPUT-s 10.2.0.0 / 24-j LOG, which is used to include the newly appeared IP segments in the log collection and security monitoring scope;

[0103] To address traffic drift caused by incorrect zone mapping, a VLAN modification instruction can be generated, such as switchport access vlan 200, to ensure that device traffic is correctly divided into the corresponding security zone.

[0104] For confusion in monitoring area labels caused by parsing errors, the most likely correct area can be matched based on current traffic characteristics, and modification suggestions can be generated. For example, it is recommended to change the current area label from "Test Area" to "Online Banking Area", thereby improving the accuracy of subsequent log classification and analysis.

[0105] Through the above method, not only can anomalies be identified, but also, combined with the device configuration context, repair suggestions or instructions with practical operational value can be output, thereby improving security operation efficiency and automated handling capabilities.

[0106] After completing the configuration adjustments according to the correction instructions, the feature vectors of the current traffic data are re-extracted and re-entered into the Gaussian mixture model. Its probability distribution structure and KL divergence value are calculated to assess whether the repaired traffic behavior has returned to normal. This process is iterated until the KL divergence value is less than or equal to the preset threshold, indicating that the difference between the current traffic characteristics and the historical model has been effectively eliminated and the system status has become stable and compliant.

[0107] The traffic monitoring method provided in the embodiment of the present application realizes intelligent modeling and dynamic adaptation of network traffic by introducing technical means such as traffic feature learning, two-dimensional verification of configuration and traffic, and normalized lightweight inspection. The Gaussian mixture model is used to cluster and analyze historical traffic, and a typical behavior model of each region is constructed, which can adapt to dynamic scenarios such as IP changes in the cloud environment; the KL divergence comparison of device configuration files and real-time traffic features is combined to cross-validate from the two dimensions of configuration and traffic, thereby improving the accuracy of anomaly identification and avoiding misjudgment in a single dimension; and low-proportion sampling detection (1%-5%) is triggered by change events to achieve resource-friendly high-frequency compliance inspections, effectively reducing system overhead and improving the practicality and sustainability of security monitoring in large-scale network environments.

[0108] Figure 3 A flow chart of another flow monitoring method provided by this application, such as Figure 3 As shown, the method includes: traffic feature learning, compliance detection, abnormal alarm, optimization adjustment, normalization inspection and network change events.

[0109] in:

[0110] Traffic feature learning:

[0111] 1. Input historical security device logs and device configuration files. Historical security device logs include source / destination IP addresses, ports, protocols, and traffic volume. Device configuration files include ACL rules, VLAN mapping tables, monitoring area definitions, and IP address allocation tables.

[0112] 2. Raw traffic data is segmented by monitoring area (such as the Internet area, third-party external connection area, WAN area, and office area), and structured feature vectors are extracted from it, including IP segment distribution (CIDR aggregation), top 10 port access frequency, and protocol proportion (TCP / UDP / ICMP). A Gaussian mixture model (GMM) is used to cluster the historical traffic characteristics of each area and learn the probability density function of its traffic distribution. The number of clusters is set to 5, and the covariance type is set to full to capture the complex distribution relationships between different dimensions. After training, the model parameters (including area ID, feature mean, covariance matrix, etc.) are persistently stored in a database (such as MongoDB) for subsequent access and comparison. In actual applications, some traffic may appear in multiple monitoring areas simultaneously (for example, branch Internet traffic is transmitted through the head office entrance, resulting in records in both the head office Internet area and the WAN area). Such data requires additional labeling and attribution to avoid misjudgment and ensure the accuracy of model evaluation.

[0113] Compliance detection and anomaly alerts:

[0114] Taking real-time traffic data packets (sampling 1%-5%) and the current device configuration file as input, the system first extracts feature vectors from the real-time traffic and calculates the KL divergence between it and the historical traffic distribution based on the trained Gaussian mixture model. When the KL divergence value exceeds the set threshold (such as 1.5), it is determined to be a traffic anomaly. At the same time, the configuration rules of devices such as firewalls and switches are analyzed to verify the mapping relationship between the monitoring area and the IP segment (such as VLAN 100 should correspond to the IP segment 10.1.0.0 / 16) to confirm whether the configuration is compliant. On this basis, the anomaly type is classified based on the traffic characteristics and configuration information: if an IP segment not covered by the model is found, it is determined to be a "configuration error"; if traffic appears in an unexpected monitoring area (such as the production area IP appears in the office area), it is determined to be a "traffic drift"; if the area label naming is confusing or inconsistent with the actual traffic, it is determined to be a "parsing error", thereby achieving fine-grained identification and classification of network anomalies.

[0115] Optimization adjustments:

[0116] Taking anomaly detection reports (including error type and impact scope) as input, the system automatically generates configuration correction suggestions based on the anomaly type. For configuration omission errors (e.g., a newly added IP segment not included in monitoring), it generates corresponding ACL rule addition commands, such as iptables-AINPUT-s 10.2.0.0 / 24-j LOG. For zone mapping errors, it generates VLAN modification instructions, such as switchport access vlan 200. For zone tag parsing errors, it identifies the most likely correct zone based on traffic characteristics and generates naming modification suggestions, such as changing the current zone tag to "Online Banking Zone." The generated suggestions are pushed to administrators through a ticketing system (e.g., Jira) for review and confirmation. Upon approval, the system automatically executes the remediation action. Once the remediation is complete, the compliance check process is retriggered to continuously verify the remediation results until the KL divergence value falls below the preset threshold, ensuring that traffic behavior returns to normal.

[0117] Routine inspections and network change events:

[0118] The traffic monitoring process is triggered by scheduled tasks (such as daily or weekly scheduled execution) and network change events (such as when a device upgrade work order status is updated to "Complete") to ensure timely compliance checks after network adjustments. To reduce system resource consumption, a lightweight detection mechanism is adopted: during the data collection phase, only 1% of traffic is randomly sampled for analysis, effectively reducing CPU load; during the model processing phase, a lightweight logistic regression model is used instead of the computationally complex Gaussian mixture model, significantly improving processing speed while maintaining basic detection capabilities, achieving resource-friendly, high-frequency response automated monitoring capabilities.

[0119] Figure 4 The schematic diagram of the flow monitoring device provided in this application is as follows: Figure 4 As shown, the flow monitoring device 40 provided in this embodiment includes:

[0120] An acquisition module 401 is used to acquire current traffic data received in a target area and a current feature vector of the current traffic data;

[0121] A first obtaining module 402 is configured to obtain a KL divergence result based on a clustering result of historical traffic data in a target area according to a current feature vector of the current traffic data and a Gaussian mixture model corresponding to the target area; the Gaussian mixture model is trained based on the historical feature vectors of the historical traffic data in the target area;

[0122] The second obtaining module 403 is used to obtain the abnormal monitoring result of the current traffic data according to the KL divergence result.

[0123] In a possible implementation, the acquisition module 401 may also be specifically configured to:

[0124] Get current traffic data and device configuration files of current traffic data;

[0125] Perform feature vector extraction processing on the current traffic data to obtain the current feature vector of the current traffic data;

[0126] Determine the target area to which the current traffic data belongs based on the device configuration file of the current traffic data.

[0127] In a possible implementation, the first obtaining module 402 may also be specifically configured to:

[0128] Obtaining historical traffic data, device configuration files of the historical traffic data, and historical feature vectors of the historical traffic data;

[0129] Segment the historical traffic data based on the device configuration file of the historical traffic data to determine the segmented traffic data belonging to different monitoring areas;

[0130] The segmented traffic data is input into the preset initial Gaussian mixture model for unsupervised model training to obtain a Gaussian mixture model;

[0131] According to the characteristic mean and covariance matrix of the Gaussian mixture model, the clustering result of the Gaussian mixture model on the historical traffic data in the target area is obtained.

[0132] In a possible implementation, the flow data in the current flow data and the flow data in the historical flow data in the first obtaining module 402 both include a source IP address, a destination IP address, a source port, a destination port, and a protocol type;

[0133] In a possible implementation, the vector types in the current feature vector and the vector types in the historical feature vector in the first obtaining module 402 both include IP segment distribution, top 10 ports, and protocol proportions.

[0134] In a possible implementation, the first obtaining module 402 may also be specifically configured to:

[0135] Determine the characteristic distribution of the current flow according to the current characteristic vector of the current flow data;

[0136] The KL divergence result is obtained based on the clustering results of the historical traffic data in the target area and the current feature vector of the current traffic data according to the Gaussian mixture model corresponding to the target area.

[0137] In a possible implementation, the second obtaining module 403 may also be specifically configured to:

[0138] When the KL divergence value corresponding to the KL divergence result is greater than the preset KL divergence value, the abnormal type of the current traffic data is determined based on the device configuration file of the current traffic data. The abnormal types of the current traffic data include configuration errors that indicate that newly added IP segments are not covered, traffic drift in the wrong monitoring area, and parsing errors with incorrect monitoring area naming.

[0139] According to the abnormal type of the current traffic data, the abnormal monitoring result of the current traffic data is obtained.

[0140] In a possible implementation, the second obtaining module 403 may also be specifically configured to:

[0141] Generate correction instructions corresponding to the abnormal monitoring results according to the abnormal monitoring results of the current traffic data;

[0142] Adjust the current flow data and the device configuration file of the current flow data according to the correction instructions corresponding to the abnormal monitoring results;

[0143] After adjusting the current traffic data and the device configuration file of the current traffic data, repeatedly inputting the current feature vector of the current traffic data into the Gaussian mixture model to obtain a probability distribution structure of the current feature vector output by the Gaussian mixture model until a KL divergence value corresponding to the KL divergence result is less than or equal to a preset KL divergence value;

[0144] In a possible implementation, the second obtaining module 403 may also be specifically configured to:

[0145] When the abnormal monitoring result of the current traffic data represents that the current traffic data is normal, the Gaussian mixture model is updated according to the current traffic data and the historical traffic data.

[0146] The flow monitoring device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar, and are not described in detail in this embodiment.

[0147] Figure 5 This is a schematic diagram of the flow monitoring device provided in this application. Figure 5 As shown, the flow monitoring device 50 provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0148] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that the at least one processor 501 performs the above method.

[0149] The specific implementation process of the processor 501 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0150] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules in the processor.

[0151] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory.

[0152] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be classified as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0153] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0154] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0155] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0156] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in a device as discrete components.

[0157] The division of units is merely a logical functional division; actual implementations may employ alternative divisions, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, any direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units, either through an interface, electrical, mechanical, or other means.

[0158] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0159] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0160] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0161] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0162] Finally, it should be noted that those skilled in the art will readily identify other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The present invention is not limited to the precise structure described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present invention is limited solely by the appended claims.

Claims

1. A flow monitoring method, characterized in that: include: Acquire current traffic data received in the target area and a current feature vector of the current traffic data; Obtaining a KL divergence result based on a current feature vector of the current traffic data and a clustering result of the historical traffic data in the target area using a Gaussian mixture model corresponding to the target area; The Gaussian mixture model is trained based on historical feature vectors of historical traffic data in the target area; According to the KL divergence result, the abnormal monitoring result of the current traffic data is obtained.

2. The method according to claim 1, characterized in that The acquiring of the current traffic data received in the target area and the current feature vector of the current traffic data includes: Acquire the current traffic data and a device configuration file of the current traffic data; Performing feature vector extraction processing on the current flow data to obtain a current feature vector of the current flow data; The target area to which the current traffic data belongs is determined according to the device configuration file of the current traffic data.

3. The method according to claim 1, characterized in that The method further comprises: Acquire the historical traffic data, a device configuration file of the historical traffic data, and a historical feature vector of the historical traffic data; Segmenting the historical traffic data according to the device configuration file of the historical traffic data to determine segmented traffic data belonging to different monitoring areas; Inputting the segmented traffic data into a preset initial Gaussian mixture model for unsupervised model training to obtain the Gaussian mixture model; According to the characteristic mean and covariance matrix of the Gaussian mixture model, a clustering result of the Gaussian mixture model on the historical traffic data in the target area is obtained.

4. The method according to any one of claims 1 to 3, characterized in that The flow data in the current flow data and the flow data in the historical flow data both include a source IP address, a destination IP address, a source port, a destination port, and a protocol type; and / or, The vector type in the current feature vector and the vector type in the historical feature vector both include IP segment distribution, top 10 ports, and protocol proportions.

5. The method according to claim 1, wherein The KL divergence result is obtained based on the current feature vector of the current traffic data, the clustering result of the Gaussian mixture model corresponding to the target area on the historical traffic data in the target area, and the feature mean and covariance matrix of the Gaussian mixture model, including: determining a characteristic distribution of the current flow according to a current characteristic vector of the current flow data; A KL divergence result is obtained based on a clustering result of the historical traffic data in the target area and a current feature vector of the current traffic data using a Gaussian mixture model corresponding to the target area.

6. The method according to claim 1, characterized in that Obtaining the abnormal monitoring result of the current traffic data according to the KL divergence result includes: When the KL divergence value corresponding to the KL divergence result is greater than a preset KL divergence value, the abnormal type of the current traffic data is determined according to the device configuration file of the current traffic data; the abnormal type of the current traffic data includes a configuration error indicating that a newly added IP segment is not covered, a traffic drift in an incorrect monitoring area, and a parsing error in which the monitoring area is named incorrectly; According to the abnormality type of the current flow data, an abnormality monitoring result of the current flow data is obtained.

7. The method according to claim 6, characterized in that The method further comprises: generating, according to the abnormal monitoring result of the current flow data, a correction instruction corresponding to the abnormal monitoring result; adjusting the current flow data and a device configuration file of the current flow data according to a correction instruction corresponding to the abnormal monitoring result; After adjusting the current traffic data and the device configuration file of the current traffic data, repeatedly performing the step of inputting the current feature vector of the current traffic data into a Gaussian mixture model to obtain a probability distribution structure of the current feature vector output by the Gaussian mixture model until a KL divergence value corresponding to the KL divergence result is less than or equal to a preset KL divergence value; and / or, When the abnormal monitoring result of the current traffic data indicates that the current traffic data is normal, the Gaussian mixture model is updated according to the current traffic data and the historical traffic data.

8. A flow monitoring device, characterized in that: include: An acquisition module, configured to acquire current traffic data received in a target area and a current feature vector of the current traffic data; A first obtaining module is used to obtain a KL divergence result based on a clustering result of the historical traffic data in the target area according to a current feature vector of the current traffic data and a Gaussian mixture model corresponding to the target area; The Gaussian mixture model is trained based on historical feature vectors of historical traffic data in the target area; The second obtaining module is used to obtain the abnormal monitoring result of the current traffic data according to the KL divergence result.

9. A flow monitoring device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images