A traffic analysis method, device, storage medium and computer device

Abstract

The application provides a traffic analysis method and device, a storage medium and a computer device, wherein the method is applied to any traffic probe in a probe cluster, and comprises the following steps: in the case that the real-time hardware load of the traffic probe exceeds a preset load lower limit, judging whether each data packet collected by the traffic probe in real time matches a risk packet feature; if not, determining a first feature entropy value of a traffic sequence composed of each data packet collected in each preset time window according to traffic feature information of the traffic sequence in multiple feature dimensions; determining a first target analysis strategy from multiple risk analysis strategies according to the first feature entropy value and historical feature entropy values corresponding to historical traffic sequences in multiple historical time windows; and performing risk analysis on the traffic sequence according to the first target analysis strategy to obtain a risk analysis result.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a traffic analysis method, apparatus, storage medium, and computer device. Background Technology

[0002] With the rapid development of the Internet, the Internet of Things, big data, and artificial intelligence technologies, the problem of data leakage caused by various cyberattacks is becoming increasingly serious. Therefore, real-time monitoring and risk analysis of network traffic has become one of the important technical means to ensure network security. Traffic probes, as a common network security device for traffic capture and risk analysis, are usually deployed in bypass mode at the link nodes to be monitored, such as through port mirroring or optical splitters, to capture and analyze network traffic on the link nodes.

[0003] However, traditional techniques for risk analysis using traffic probes often fall into two categories: The first is a centralized full-volume upload and analysis scheme, where the traffic probe directly uploads all collected traffic to the analysis platform for unified detection. However, this scheme places stringent demands on the analysis platform's computing power, storage, and the transmission bandwidth between the traffic probe and the analysis platform, making it extremely unsuitable for high-traffic scenarios. The second category is a static strategy and fixed sampling scheme, where the traffic probe performs risk matching, sampling, and risk detection on network traffic based on analysis rules issued by the analysis platform. However, this method relies excessively on the rules issued by the analysis platform; if the rules are not issued in a timely manner, it can easily lead to missed or false alarms. Summary of the Invention

[0004] In view of this, this application provides a traffic analysis method, apparatus, storage medium, and computer device to solve the problems of poor applicability to high-traffic scenarios and the risk of missed or false alarms in traffic analysis.

[0005] Specifically, this application is implemented through the following technical solution: In a first aspect, embodiments of this disclosure provide a traffic analysis method, applied to any traffic probe in a probe cluster, the method comprising: When the real-time hardware load of the traffic probe exceeds a preset lower load limit, for each data packet collected in real time by the traffic probe, it is determined whether each data packet matches the risk packet characteristics; the risk packet characteristics are dynamically updated based on the collaborative early warning notifications between the traffic probes in the probe cluster. If not, then for the traffic sequence composed of each data packet collected within each preset time window, the first feature entropy value of the traffic sequence is determined based on the traffic feature information of the traffic sequence under multiple feature dimensions. Based on the first feature entropy value and the historical feature entropy values ​​corresponding to historical traffic sequences under multiple historical time windows, a first target analysis strategy is determined from multiple risk analysis strategies; the multiple risk analysis strategies include a full analysis strategy, a sampling analysis strategy, and a compression analysis strategy; the sampling ratio under the sampling analysis strategy is dynamically updated according to the collaborative early warning notification; According to the first target analysis strategy, risk analysis is performed on the traffic sequence to obtain risk analysis results.

[0006] Secondly, embodiments of this disclosure also provide a flow analysis device, applied to any flow probe in a probe cluster, the device comprising: The acquisition module is used to determine whether each data packet collected in real time by the traffic probe matches the risk packet feature information when the real-time hardware load of the traffic probe exceeds a preset load lower limit; the risk packet feature is dynamically updated based on the collaborative early warning notification between each traffic probe in the probe cluster. The calculation module, if not, is used to determine the first feature entropy value of the traffic sequence based on the traffic feature information of the traffic sequence under multiple feature dimensions, for the traffic sequence composed of each data packet collected within each preset time window. The determination module is used to determine a first target analysis strategy from multiple risk analysis strategies based on the first feature entropy value and the historical feature entropy values ​​corresponding to historical traffic sequences under multiple historical time windows; the multiple risk analysis strategies include a full analysis strategy, a sampling analysis strategy, and a compression analysis strategy; the sampling ratio under the sampling analysis strategy is dynamically updated according to the collaborative early warning notification; The analysis module is used to perform risk analysis on the traffic sequence according to the first target analysis strategy and obtain the risk analysis results.

[0007] Thirdly, an optional implementation of this disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps in the first aspect described above.

[0008] Fourthly, an optional implementation of this disclosure also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the first aspect described above.

[0009] For a description of the effects of the aforementioned flow analysis device, storage medium, and computer equipment, please refer to the description of the aforementioned flow analysis method; it will not be repeated here.

[0010] This disclosure provides a method, apparatus, storage medium, and computer device for traffic analysis. It uses the real-time hardware load of a traffic probe as a prerequisite for the traffic analysis process. Subsequent processing steps, such as risk packet matching, entropy calculation, and analysis selection, are only performed when the load exceeds a preset lower limit. This makes the hardware load status a factor in traffic analysis, effectively preventing the traffic probe from crashing due to overload and failing to perform traffic analysis. By matching the risk packet characteristics of each data packet, data packets that meet known risks can be directly identified, thereby reducing the scale of traffic entering subsequent complex calculation steps and effectively improving the overall detection and processing efficiency. For data packets that do not match risk packet characteristics, a traffic sequence is formed in units of preset time windows. The first feature entropy value of the traffic sequence is calculated based on traffic feature information of multiple feature dimensions. Based on the first feature entropy value and the historical feature entropy values ​​of multiple historical time windows, the degree of feature deviation between the current traffic sequence and the traffic sequences under historical windows can be discovered. This allows for the dynamic and accurate determination of the first target analysis strategy from full analysis, sampling analysis, and compression analysis strategies. Thus, by making real-time judgments based on entropy values ​​that reflect the intrinsic information value of traffic sequences, reliance on static platform strategies can be eliminated. This allows for extreme compression of repetitive background traffic and zero-latency full capture of sudden abnormal traffic, significantly improving the ability to detect unknown threats and zero-day attacks, as well as adaptability to large-scale traffic scenarios. Risk analysis is performed on traffic sequences according to the matched target analysis strategy, enabling differentiated and targeted detection of traffic with different levels of anomalies, while ensuring no data packet is missed, effectively avoiding false positives or false negatives. Furthermore, by using collaborative early warning among various traffic probes, the sampling ratio under the risk packet characteristics and sampling analysis strategy can be dynamically adjusted. This horizontal transmission and interaction within the cluster enables rapid deployment and consensus among different anomalies within the probe cluster, effectively curbing threat spread.

[0011] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present application of a traffic analysis method applied to any traffic probe in a probe cluster; Figure 2 This is an exemplary embodiment of the present application illustrating an architecture diagram of a probe cluster and analysis platform; Figure 3 This is a schematic diagram of an exemplary embodiment of the present application illustrating a flow analysis device applied to any flow probe in a probe cluster; Figure 4 This is a schematic diagram of the structure of a computer device shown in an exemplary embodiment of this application. Detailed Implementation

[0013] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0014] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0015] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0016] Research has found that when using traffic probes for risk analysis, traditional technical methods are often divided into the following two categories: 1. The first category is the full upload centralized analysis scheme. The traffic probe only performs simple data packet mirroring and forwarding functions, without sampling or filtering the traffic. All the original traffic data packets collected are sent to the analysis platform. The analysis platform performs unified storage and risk identification of all traffic. However, this type of method has the following problems: (1) The full upload centralized scheme requires the full upload of all collected traffic to the analysis platform, which will occupy a lot of link bandwidth resources. At the same time, the analysis platform needs to bear the storage and risk identification tasks of the full traffic, which results in strict requirements on the computing power, storage and transmission bandwidth between the traffic probe and the analysis platform. It is extremely unsuitable for high traffic scenarios, and all risk identification depends on the backend analysis platform, resulting in high detection latency. (2) A large amount of background traffic that does not need to be detected (such as broadcast packets, normal business heartbeats, data backup streams) will consume most of the computing power and storage resources of the analysis platform. The computing power used for deep threat detection is severely squeezed, and the input-output ratio of security analysis is extremely low.

[0017] 2. The second category is static strategies and fixed sampling schemes. Based on the full traffic logs reported by the traffic probe in the previous cycle, the analysis platform uses methods such as clustering, statistics, and machine learning to analyze risky traffic characteristics (such as specific source IP ranges, destination ports, and malicious payload characteristics) and secure traffic characteristics (such as whitelisted IP domains and commonly used business ports). These risky and secure traffic characteristics are then compiled into analysis rules containing risk information, security information, and a fixed sampling ratio corresponding to each secure traffic characteristic, and this rule is sent to the traffic probe. The traffic probe, based on the analysis rules issued by the analysis platform in the latest cycle, performs risk matching, sampling, and risk analysis on the collected traffic. Specifically, it matches the collected traffic with risk information, performing risk analysis on successfully matched data packets. Data packets matching security information are sampled according to the corresponding fixed sampling ratio, and risk analysis is performed on each sampled data packet. Traffic that does not match either type of information is not processed for risk analysis and is directly forwarded. After completing the detection, the traffic probe reports the full analysis and the traffic logs after sampling analysis to the analysis platform for iterative updates of the analysis rules in the next cycle. However, this type of solution has the following problems: a) The analysis rules, which include risk information, security information, and sampling ratios, are updated and distributed on a fixed schedule. For abnormal traffic corresponding to newly generated unknown attacks (such as the detection of unknown vulnerabilities or the spread of new variant viruses) that occur between two rule updates, these traffic will not enter the traffic analysis process because there are no corresponding matching rules in the currently effective old analysis rules. They will only be classified as unmatched traffic and forwarded directly, failing to cover newly emerging unknown risks in a timely manner within the rule update interval. This severely affects the lag of the analysis rules, making it difficult to cope with sudden unknown threats and easily leading to missed detections. b) The sampling ratio depends on the analysis platform's analysis of the traffic logs from the previous period, making it impossible to adaptively adjust the traffic. Furthermore, the dynamic anomalies of the content carried by the traffic cannot be detected during the sampling and analysis process. For example, the feature information corresponding to traffic A is marked as secure traffic feature information in the analysis rules. During the traffic analysis process, sampling and risk analysis only need to be performed according to a preset fixed sampling ratio. Attackers can exploit the trust placed in A within the analysis rules as cover to embed encrypted malicious payloads into A's traffic channels, thereby carrying out attacks such as data theft and encrypted remote control. This can cause A to suddenly begin transmitting high-entropy encrypted data at a certain moment. Even if A's traffic payload has undergone abnormal changes, the static strategy and fixed sampling scheme will still sample and analyze it according to the original fixed sampling ratio, which can easily miss key packets carrying malicious behavior, resulting in the loss of attack characteristics and forensic evidence, and making it impossible to identify the hidden risks hidden within the traffic corresponding to the secure traffic characteristics. c. Because the business priority of different traffic is not considered, the integrity of core business detection cannot be guaranteed in high-traffic scenarios. Since the solution uses uniform sampling and analysis rules, it does not perform hierarchical scheduling of core and non-core businesses.When traffic exceeds the computing power limit of the traffic probes, the only way to alleviate the load pressure is to reduce the sampling ratio. Core business traffic cannot be guaranteed priority detection. d. Because there is no horizontal interaction mechanism between the traffic probes, they all operate as independent nodes. On the one hand, new risk features captured by a single probe cannot be synchronized to other probes in real time, and the probe cluster cannot achieve information sharing and interaction. On the other hand, when the same attack traffic flows through multiple traffic probes, each node needs to repeatedly execute the complete feature identification process, resulting in redundant consumption of cluster computing power and low hardware resource utilization.

[0018] Based on the above research, this disclosure provides a traffic analysis method, apparatus, storage medium, and computer device. It uses the real-time hardware load of the traffic probe as a prerequisite for the traffic analysis process. Subsequent processing steps such as risk packet matching, entropy calculation, and analysis selection are only performed when the load exceeds a preset lower limit. This makes the hardware load status one of the factors considered in traffic analysis, effectively avoiding the problem of traffic probes crashing due to overload and failing to perform traffic analysis. By matching the risk packet characteristics of each data packet, data packets that meet known risks can be directly identified, thereby reducing the scale of traffic entering subsequent complex calculation steps and effectively improving the overall detection and processing efficiency. For data packets that do not match risk packet characteristics, a traffic sequence is formed in units of preset time windows. The first feature entropy value of the traffic sequence is calculated based on traffic feature information of multiple feature dimensions. Based on the first feature entropy value and the historical feature entropy values ​​of multiple historical time windows, the degree of feature deviation between the current traffic sequence and the traffic sequences under historical windows can be found. This allows for the dynamic and accurate determination of the first target analysis strategy from full analysis, sampling analysis, and compression analysis strategies. Thus, by making real-time judgments based on entropy values ​​that reflect the intrinsic information value of traffic sequences, reliance on static platform strategies can be eliminated. This allows for extreme compression of repetitive background traffic and zero-latency full capture of sudden abnormal traffic, significantly improving the ability to detect unknown threats and zero-day attacks, as well as adaptability to large-scale traffic scenarios. Risk analysis is performed on traffic sequences according to the matched target analysis strategy, enabling differentiated and targeted detection of traffic with different levels of anomalies, while ensuring no data packet is missed, effectively avoiding false positives or false negatives. Furthermore, by using collaborative early warning among various traffic probes, the sampling ratio under the risk packet characteristics and sampling analysis strategy can be dynamically adjusted. This horizontal transmission and interaction within the cluster enables rapid deployment and consensus among different anomalies within the probe cluster, effectively curbing threat spread.

[0019] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0020] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0021] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0022] To facilitate understanding of this embodiment, a traffic analysis method disclosed in this disclosure will first be described in detail. The execution subject of the traffic analysis method provided in this disclosure is generally a terminal device or other processing device with certain computing power. The terminal device can be a user equipment (UE), mobile device, terminal, personal digital assistant (PDA), handheld device, computer device, network security device (such as traffic probe), etc. In some possible implementations, the traffic analysis method can be implemented by the processor calling computer-readable instructions stored in the memory.

[0023] The following example uses any traffic probe in the probe cluster as the execution subject to illustrate the traffic analysis method provided in this embodiment.

[0024] like Figure 1 The flowchart shown is a flow chart of a traffic analysis method applied to any traffic probe in a probe cluster, provided by an embodiment of this disclosure. The method may include the following steps: S101: When the real-time hardware load of the traffic probe exceeds the preset lower limit of the load, determine whether each data packet collected by the traffic probe in real time matches the characteristics of the risk packet.

[0025] Among them, the risk package features are dynamically updated based on the collaborative early warning notifications between various traffic probes in the probe cluster.

[0026] Here, a traffic probe is a network security device deployed on network link nodes, used for capturing and analyzing network traffic. In actual deployment, the traffic probe can be connected to the network link using methods such as port mirroring, optical splitters, or cascading, depending on the network environment. Real-time hardware load reflects the current hardware operating load pressure of the traffic probe, specifically including indicators such as Central Processing Unit (CPU) utilization, the depth of the queue of pending data packets in memory, and the Direct Memory Access (DMA) frame loss count. The depth of the queue of pending data packets in memory refers to the number of data packets currently waiting for the CPU to process, and the DMA frame loss count represents the cumulative number of data packets that the network card has successfully received but discarded due to a full memory buffer.

[0027] Based on real-time hardware load, the load level of traffic probes can be divided into three levels from low to high: low load, medium load, and high load. A preset lower load limit is the boundary threshold between the low and medium load levels. If the real-time hardware load does not exceed this value, the current traffic probe is determined to be in the low load level; if the real-time hardware load exceeds the preset lower load limit, the current traffic probe is determined to be in the high or medium load level.

[0028] Data packets are the smallest unit of network data transmission and the core processing object of traffic probes. They contain a five-tuple of information—source IP address, source port number, destination IP address, destination port number, and transport layer protocol number—to uniquely identify a network session, as well as payload information to carry service content. Risk packet features are a set of known risk features stored locally on the traffic probe. Initial risk packet features can be issued by the analysis platform. Risk packet features indicate the packet characteristics that risky data packets may carry, such as risky traffic type, risky domain name, and traffic five-tuple. After obtaining the initial risk packet features, the traffic probe can dynamically update the locally stored risk packet features based on collaborative early warning notifications with other traffic probes in the probe cluster. Alternatively, the analysis platform or security analysts can enter or import new risk packet features into the traffic probe. Collaborative early warning notifications are early warning interaction information transmitted horizontally between traffic probes within the probe cluster. They can be sent by one traffic probe to other traffic probes when a risk is detected, thereby achieving risk information synchronization among the probes.

[0029] In practical implementation, for any traffic probe, during the traffic analysis process, it can use the traffic acquisition module to collect each data packet and the hardware load monitoring module to obtain real-time hardware load indicators such as CPU utilization, memory pending data packet queue depth, and network card DMA packet loss count. These indicators are used as its real-time hardware load and compared with a preset load lower limit. For example, a three-level load level can be set as follows: Low load level: CPU utilization <50% and memory queue depth <30% and DMA packet loss count is 0; Medium load level: 50% ≤ CPU utilization < 95% and memory queue depth < 80% and DMA packet loss count is 0, or, CPU utilization < 50% and 30% ≤ memory queue depth < 80% and DMA packet loss count is 0; High load level: CPU utilization ≥ 95% or memory queue depth ≥ 80% or DMA packet loss count > 0.

[0030] Specifically, when CPU utilization reaches 50%, memory queue depth reaches 30%, or DMA packet loss occurs, it is determined that the preset load threshold has been exceeded. When the real-time hardware load exceeds the preset load threshold, each data packet collected in real time by the traffic probe can be matched with the latest risk packet characteristics stored locally by the traffic probe to determine whether each data packet possesses risk packet characteristics.

[0031] S102: If not, then for each data packet collected within each preset time window, the first feature entropy value of the traffic sequence is determined based on the traffic feature information of the traffic sequence under multiple feature dimensions.

[0032] Here, the preset time window is a pre-set time window, the length of which can be set empirically. This embodiment does not impose a specific limitation; for example, the time length can be 1 second. The traffic sequence is a sequence composed of all data packets that do not match the risk packet characteristics within a single preset time window, arranged in chronological order. The traffic feature information under multiple feature dimensions are preset dimensions that can characterize information entropy. These can typically include message length distribution probability dimension, data packet arrival time interval distribution dimension, payload byte value distribution (such as the first 64 bytes), etc. The types and number of dimensions can be flexibly selected according to the business scenario. The first feature entropy value is the information entropy value calculated based on the multi-dimensional traffic feature distribution, used to quantify the dispersion of traffic feature information within the current time window.

[0033] In practice, for data packets that do not match the risk packet characteristics, the traffic probe can continuously collect these data packets into a traffic sequence with a preset time window of 1 second. Then, using the information entropy calculation engine in the traffic probe, the probability distribution of the traffic sequence under multiple feature dimensions such as message length distribution probability, data packet arrival time interval, and the value of the first 64 bytes of payload can be statistically analyzed based on the packet information of each data packet in the traffic sequence. Based on the probability distribution results of each feature dimension, the first feature entropy value H corresponding to the overall traffic sequence collected within the preset time window is determined.

[0034] For example, the entropy value of each feature dimension can be calculated using the Shannon entropy formula based on the probability distribution of each feature dimension. Then, the entropy values ​​of each feature dimension are weighted and summed according to preset weights to finally obtain the first feature entropy value corresponding to the traffic sequence.

[0035] S103: Based on the first feature entropy value and the historical feature entropy values ​​corresponding to historical flow sequences under multiple historical time windows, determine the first target analysis strategy from multiple risk analysis strategies.

[0036] Among them, various risk analysis strategies include full analysis strategy, sampling analysis strategy and compressed analysis strategy; the sampling ratio under the sampling analysis strategy is dynamically updated according to the collaborative early warning notification.

[0037] Here, multiple historical time windows refer to consecutive preset time windows preceding the current preset time window, such as five consecutive 1-second windows in the past. The historical feature entropy value is the feature entropy value of the traffic sequence corresponding to each historical time window, i.e., the first feature entropy value of each historical time window. The historical traffic sequence is the traffic sequence composed of data packets collected within the historical time window that do not match risk packet features, arranged in order of arrival time. Multiple risk analysis strategies include a full analysis strategy, a sampling analysis strategy, and a compression analysis strategy. The full analysis strategy instructs risk analysis to be performed on each data packet. The sampling analysis strategy instructs risk analysis to be performed on each data packet after sampling. The sampling ratio under the sampling analysis strategy is not a fixed preset ratio but is dynamically updated based on collaborative early warning notifications between traffic probes. The compression analysis strategy instructs risk analysis to be performed on the overall traffic sequence after feature compression.

[0038] In practice, the traffic probe can maintain a sequence of historical feature entropy values ​​for multiple historical time windows in a local cache using a sliding time window approach. This sequence contains the historical feature entropy values ​​corresponding to various historical time windows. The length of the dynamic sliding window is the total length of the multiple historical time windows, for example, 5 seconds. After obtaining the first feature entropy value for the current preset time window, the first target analysis strategy for the traffic sequence corresponding to the first feature entropy value can be determined from multiple risk analysis strategies based on the historical feature entropy values ​​maintained within the current sliding time window. For example, the deviation between the first feature entropy value and each historical feature entropy value can be compared, and the first target analysis strategy can be selected from multiple risk analysis strategies based on the magnitude of the deviation. The first target analysis strategy is the analysis strategy selected for the current preset time window; one preset time window corresponds to one first target analysis strategy.

[0039] Understandably, if the traffic probe is in the early stage of traffic identification and there is no effective historical time window or the number of effective historical windows is insufficient (e.g., 5), the preset feature entropy value pre-issued by the analysis platform can be used as the historical feature entropy value for analysis strategy selection.

[0040] S104: According to the first objective analysis strategy, perform risk analysis on the flow sequence and obtain the risk analysis results.

[0041] Here, the risk analysis results are used to indicate whether each data packet in the traffic sequence is risky, and the level of risk when it is, etc.

[0042] In practice, the traffic probe can use a risk identification engine to perform the corresponding risk analysis process on the traffic sequence according to the determined first target analysis strategy, thereby obtaining the risk analysis results of the traffic sequence.

[0043] In this way, when the hardware load of the traffic probe exceeds the preset lower limit, based on the risk packet characteristics dynamically updated through collaborative early warning notifications from the probe cluster, pre-matching is performed on each data packet. This achieves adaptive scheduling of the risk analysis process based on the hardware load status, and reduces the amount of data processed in subsequent stages through pre-filtering, thereby reducing overall processing overhead. On this basis, the first feature entropy value is calculated for the traffic sequence composed of data packets with unmatched features, and combined with historical feature entropy values, the corresponding risk analysis strategy is dynamically adapted to the traffic sequence. This enables quantitative perception of changes in traffic sequence characteristics, allowing for rapid identification of sudden abnormal traffic sequences. Simultaneously, the horizontal collaborative early warning mechanism based on the probe cluster ensures the real-time updating of risk packet characteristics and sampling ratios. By differentially matching risk analysis strategies to traffic sequences, the detection accuracy of traffic analysis can be maintained while effectively reducing overall processing overhead.

[0044] In one embodiment, S103 described above can be implemented according to the following steps A1 to A4: A1: Based on the degree of load deviation between the real-time hardware load and the preset load limit, determine the first and second weights of the entropy standard deviation for multiple historical feature entropy values.

[0045] Here, the preset load limit is the pre-set threshold separating high and medium load levels. The load deviation is the degree of deviation between the preset load limit and the real-time hardware load. The entropy standard deviation is the standard deviation of a sequence of historical feature entropy values, used to measure the volatility of historical entropy values. The first weight and second weight are weighting parameters used to adjust the entropy threshold; their specific values ​​can be flexibly configured according to business detection needs, either set to different values ​​or the same value.

[0046] In practice, the traffic probe calculates the load deviation between the preset load limit and the real-time hardware load. The load deviation is equal to the real-time hardware load minus the preset load limit. When the load deviation is greater than or equal to 0, it indicates that the real-time hardware load of the traffic probe is at a high load level, and the larger the absolute value of the load deviation, the more severe the deviation. When the load deviation is less than 0, it indicates that the real-time hardware load of the traffic probe is at a medium load level.

[0047] When the load deviation is less than 0, it means that the real-time hardware load is between the preset lower load limit and the preset upper load limit, that is, the real-time hardware load of the current traffic probe is at the medium load level. At this time, the first weight is determined. The first preset value (e.g., 1.5), the second weight The second preset value (e.g., 2.5).

[0048] When the load deviation is greater than or equal to 0, it indicates that the real-time hardware load is above the preset load limit, meaning the current traffic probe's real-time hardware load is at a high load level. In this case, the first weight is determined. The first preset value and the first offset Δ The difference, for example, when the first preset value is 1.5, Δ When =0.5, that is =1, which allows more flows with lower feature entropy values ​​to be classified as low-entropy flows. Second weight The second preset value and the second offset Δ The sum of values. For example, when the second preset value is 2.5, Δ When =0.5, =3, so that full capture is only triggered when the feature entropy value H is greater than a higher threshold. Adjusting the weights through offset allows for... and The "intermediate region" between them widens, but more importantly, it allows the coverage of the "low-entropy-compression analysis strategy" to expand downwards, while the coverage of the "high-entropy-full analysis strategy" contracts upwards. This results in more traffic being processed by the extremely compressed summary mode, while only extremely abnormal traffic triggers high-load full analysis, thus protecting the core processing capacity of the system during overload. Recovery mechanism: When the load level of the traffic probe drops from high level (i.e., high load level) to medium or low level (i.e., medium or low load level), the drift coefficient... and The system will gradually return to the preset values ​​(i.e., the first and second preset values), and the upper and lower thresholds will also return to normal logic accordingly.

[0049] A2: Determine the initial lower threshold based on the average entropy value of multiple historical feature entropy values ​​and the product of the first weight and the standard deviation of the entropy value; and determine the initial upper threshold based on the average entropy value and the product of the second weight and the standard deviation of the entropy value.

[0050] Here, the mean entropy value is the average value of the historical feature entropy value sequence, used to characterize the average level of historical traffic entropy values. The initial lower threshold is the boundary of low entropy, and the initial upper threshold is the boundary of high entropy.

[0051] In practice, the initial lower limit threshold is determined according to the following formula. : ; And determine the initial upper limit threshold according to the following formula. : .

[0052] in, μ The mean entropy value of the historical feature entropy value sequence. As the first weight, σ The standard deviation of the entropy values ​​in the historical feature entropy value sequence. It is the second weight.

[0053] A3: The initial lower limit threshold is not lower than the preset lower limit threshold and the initial upper limit threshold does not exceed the preset upper limit threshold as the first constraint condition, and the change range of the initial lower and upper limit thresholds compared with the previously determined historical thresholds does not exceed the preset range as the second constraint condition. The initial lower limit threshold and the initial upper limit threshold are adjusted respectively to obtain the current target upper limit threshold and the target lower limit threshold.

[0054] Here, the preset lower limit threshold is the pre-set minimum lower limit. It cannot be lower than this value; the preset upper limit threshold is the pre-set maximum limit. It cannot exceed this value. Historically, the lower limit threshold is the lower limit threshold used by the flow probe within the previous preset time window. and upper limit threshold This constraint is used to limit the adjustment step size of the current threshold. The preset amplitude is the maximum allowable change step size between any two adjacent preset time windows. The first constraint and the second constraint are constraints on the adjustment of the initial lower threshold and the initial upper threshold. The first constraint is a boundary restriction based on the preset lower threshold and the preset upper threshold, and the second constraint is a restriction on the threshold adjustment amplitude based on historical lower thresholds and the preset amplitude.

[0055] The target lower threshold is the low-entropy boundary that is ultimately determined by the current preset time window after adjusting for two constraints; the target upper threshold is the high-entropy boundary that is ultimately determined by the current preset time window after adjusting for two constraints.

[0056] In practice, the flow probe first uses a preset lower threshold. and preset upper limit threshold Constraints are imposed on the initial lower threshold and the initial upper threshold to ensure Not lower than the preset lower limit , Not higher than the preset upper limit That is, ensure , Always within the range { , }Inside.

[0057] If the initial lower limit threshold is lower than Then the initial lower threshold will be updated to That is, setting = If the initial upper limit threshold is higher than Then the initial upper limit threshold will be updated to That is, setting = .

[0058] Then obtain the historical upper and lower limit thresholds from the previous time period, and then apply the first constraint. , The data is compared with historical target upper and lower limits. If the change exceeds a preset range, the step size is limited according to the preset range to avoid [further issues]. , The issue of drastic changes preventing attackers from "taming" the threshold by slowly increasing the entropy value over a long period is addressed. The threshold determined after adjusting two constraints is the target lower limit threshold used within the current preset time window. and target upper limit threshold .

[0059] A4: Based on the relationship between the first feature entropy value and the target upper and lower thresholds, determine the first target analysis strategy from multiple risk analysis strategies.

[0060] In practice, the flow probe compares the first feature entropy value H with the target upper limit threshold. and target lower limit threshold By comparing the magnitudes, and based on the resulting relationships, the primary target analysis strategy is determined from among various risk analysis strategies.

[0061] when When the first characteristic entropy value of the traffic sequence is lower than the low entropy boundary, the traffic sequence as a whole exhibits high regularity and repetition, such as periodic heartbeat packets and fixed-format log synchronization traffic, and the first target analysis strategy is determined to be the compression analysis strategy.

[0062] when When the first characteristic entropy value of the traffic sequence is higher than the high entropy boundary, the traffic sequence as a whole exhibits high uncertainty and unstructured characteristics, such as encrypted probe traffic, unknown hidden tunnels, zero-day attack payloads, etc., then the first target analysis strategy is determined to be the full analysis strategy.

[0063] when If the entropy value of the current flow sequence is within the normal fluctuation range, it indicates that it does not have obvious low-value regularity characteristics, nor does it show significant high-risk randomness characteristics. Therefore, the first target analysis strategy is determined to be the sampling analysis strategy.

[0064] This feature information entropy autonomous mode switching mechanism, based on a dynamic drift threshold, dynamically adjusts the threshold according to the historical entropy baseline and standard deviation, and is protected by a rate limiter. This allows the flow probe to flexibly adjust the entropy threshold based on load conditions, narrowing the trigger range of the full analysis strategy and expanding the coverage of the compressed analysis strategy under high load scenarios, thus reducing data processing pressure. Simultaneously, by setting a first constraint through preset lower and upper thresholds, the first constraint ensures that the final output target threshold always remains within a reasonable range, guaranteeing the effectiveness of the detection capability. The second constraint, by limiting the maximum change step of the threshold within a single preset time window, improves the detection mechanism's resistance to attacks and ensures the stability of system operation and detection results. Furthermore, the flow probe can autonomously switch between the "compressed analysis strategy" and the "full analysis strategy" without waiting for external commands, based on the comparison between the calculated flow feature entropy value and the dynamically updated threshold.

[0065] In one embodiment, S104 described above can be implemented according to the following steps B1~B2: B1: When the first target analysis strategy is the sampling analysis strategy, the latest sampling ratio under the sampling analysis strategy is used to sample each data packet in the traffic sequence to obtain the sampled data packet.

[0066] Here, the latest sampling ratio is the latest sampling ratio cached locally by the traffic probe. This ratio can be dynamically updated and the most recently effective sampling ratio through collaborative early warning notifications. The sampled data packets are a portion of the data packets to be analyzed, sampled from the traffic sequence.

[0067] In practice, when the primary target analysis strategy is determined to be a sampling strategy, the traffic probe can sample each data packet in the traffic sequence according to the latest sampling ratio (e.g., 100:1) to obtain the sampled data packets. The sampling method can be probabilistic random sampling, where the required number of data packets is calculated based on the latest sampling ratio, and a corresponding number of data packets are randomly selected from the traffic sequence to form the sampled data packets. Alternatively, sequential interval sampling can be used, where data packets are sequentially extracted from the traffic sequence according to the interval step size corresponding to the sampling ratio to form the sampled data packets. The specific sampling method is not limited here.

[0068] B2: Based on the characteristics of the sampled data packets, perform risk analysis on the sampled data packets and obtain the risk analysis results.

[0069] Here, packet characteristics are feature parameters used to identify the risk attributes of packet data, which may include, but are not limited to, packet quintuples, payload entropy values, message length, and other feature parameters.

[0070] In practice, the traffic probe can send sampled data packets to its own deployed risk identification engine. The engine then performs risk analysis on the sampled data packets based on their characteristics. For example, it can use the five-tuple data packet features extracted from the sampled packets to match local risk rule bases to identify common known threats. Alternatively, it can determine the abnormal dispersion of traffic load based on load entropy values ​​to identify highly random, unknown, and abnormal traffic. The final risk analysis results can include information such as whether the sampled data packets pose an abnormal risk, the corresponding risk level, and the anomaly type.

[0071] In one embodiment, S104 can also be implemented according to the following steps C1~C3: C1: When the first target analysis strategy is a compression analysis strategy, the target sequence statistical characteristics of the traffic sequence are generated based on the data packet content and data packet collection time of each data packet in the traffic sequence.

[0072] The statistical features of the target sequence may include at least one of the following: packet length statistical features, time interval features, payload hash features, and communication protocol features.

[0073] Here, packet length statistical features are characteristics calculated based on the message length data of each data packet in the traffic sequence. These can include statistical measures such as the mean and variance of the data packet length, used to characterize the message size distribution pattern of the traffic sequence. Time interval features are obtained by calculating the time interval between adjacent data packets based on their acquisition time, used to characterize the distribution characteristics of the traffic sequence in the time dimension. Payload hash features can be obtained by calculating the hash value of each data packet separately based on its 5-tuple and payload content, and then aggregating all hash values. This feature characterizes the packet content distribution of the traffic sequence. Communication protocol features are characteristics obtained by parsing the packet header and payload protocol fields, and can include information such as network layer or transport layer protocol type and handshake distribution, used to clarify the communication attributes of the traffic.

[0074] Target sequence statistical features can be used to characterize the overall feature distribution carried by each data packet in a traffic sequence. Therefore, target sequence statistical features can also be called statistical fingerprint features of traffic sequences.

[0075] In practice, a risk identification engine can be used to calculate various features of the traffic sequence, such as packet length statistical features, time interval features, payload hash features, and communication protocol features, based on the quintuple, payload entropy value, message length distribution, and other data packet content of each data packet in the traffic sequence, as well as the collection time corresponding to each data packet. These features are then used as the target sequence statistical features of the traffic sequence.

[0076] C2: Obtain the statistical feature distribution of each historical flow sequence that does not have any risk, and determine the degree of matching between the statistical features of the target sequence and the statistical feature distribution.

[0077] Here, the statistical feature distribution can be the feature distribution of each dimension obtained by aggregating and fitting the sequence statistical features based on normal, risk-free historical traffic sequences. It is used to characterize the value range and distribution pattern of traffic under normal circumstances. The dimensions of the statistical feature distribution correspond one-to-one with the dimensions of the target sequence statistical features. For example, they can both include packet length statistical feature dimensions, time interval feature dimensions, payload hash feature dimensions, and communication protocol feature dimensions. In practice, a risk identification engine can be used to first obtain the traffic feature distribution of each historical traffic sequence that does not have any risk, and then determine the degree of matching between each feature in the target sequence statistical features of the traffic sequence and the statistical feature distribution of the corresponding dimension.

[0078] Since historical traffic sequences without risks are not stored in the initial stage of risk analysis, a pre-configured standard baseline distribution is used for the statistical feature distribution. In the continuous risk identification stage, either the statistical features of the most recent historical traffic sequence confirmed as risk-free can be selected as the reference distribution, or the statistical feature distribution of each historical traffic sequence without risks can be obtained by combining the features of multiple such sequences. The first method yields a more real-time traffic feature distribution that can adapt to short-term changes in business traffic, while the second method generates a more stable distribution. In practical applications, the choice can be made as needed, and this application does not impose any restrictions on this.

[0079] C3: If the matching degree is within the preset range, the risk analysis result of the traffic sequence is determined to be no risk.

[0080] Here, the preset range is a pre-defined matching degree interval used to define the range of characteristic deviations that normal traffic can tolerate. Its value can be flexibly configured according to the detection sensitivity and false alarm tolerance requirements of the business scenario. Narrowing the range can improve the sensitivity of anomaly detection, while widening the judgment range can reduce false judgments caused by normal traffic fluctuations.

[0081] For example, the quantification range of the matching degree is [0,1]. A higher value indicates a higher degree of fit between the statistical characteristics of the target sequence of the current traffic sequence and the statistical characteristics of normal, risk-free traffic. The preset degree range is set to [0.8,1]. When the matching degree equals 1, it means that the statistical characteristics of the current traffic sequence perfectly match the normal baseline, and the risk analysis result of the traffic sequence is determined to be risk-free. When the matching degree is in the range of [0.8,1), it means that the current traffic sequence has a reasonable deviation from the baseline, but it is still within the tolerable range. When the matching degree is less than 0.8, it means that the deviation of the current traffic sequence from the normal baseline exceeds the tolerable range, and the traffic sequence is determined to have abnormal risk.

[0082] In practice, the matching degree can be compared with a preset range. If the matching degree of the target sequence statistical features in each dimension is within the preset range, the risk analysis result of the traffic sequence is determined to be no risk. Conversely, if the matching degree of at least one dimension of the target sequence statistical features is not within the preset range, the traffic sequence is determined to be risky, and all data packets in the traffic sequence are determined to be risky. In this case, an alarm log for the traffic sequence can be generated and sent to the analysis platform.

[0083] In one embodiment, after determining that the risk analysis result of the traffic sequence is that there is no risk, this application also provides a processing flow for when the matching degree is less than the upper limit of a preset degree range, which can be implemented according to the following steps D1~D2: D1: If the matching degree is less than the upper limit of the preset degree range, a collaborative early warning notification of the risk warning type related to the traffic sequence will be generated.

[0084] Among them, the collaborative early warning notification of the risk warning type shall include at least the traffic feature identifier corresponding to the traffic sequence, the degree of deviation between the matching degree and the upper limit of the range, the confidence level corresponding to the degree of deviation, and the notification generation time.

[0085] Here, the upper limit of the range is the highest matching degree value within the preset range, specifically 1. The deviation degree is the offset between the upper limit of the preset range and the current matching degree. The confidence level corresponding to the deviation degree is the confidence coefficient. The confidence level and the deviation degree are positively correlated; that is, the larger the deviation degree, the more obvious the traffic deviates from the normal statistical distribution, the higher the possibility of potential abnormal risks, and the higher the corresponding confidence value. The traffic feature identifier is a feature identifier that can uniquely identify the traffic category. For example, the traffic feature identifier can be risk type, domain name, IP address, or traffic type.

[0086] The risk warning type in this application is used to indicate the notification type of the collaborative early warning notification, which is used to indicate that a minor risk has been discovered.

[0087] In practical implementation, when the matching degree of the target sequence statistical features in each dimension all falls within the preset range, but the statistical feature of any target sequence in any dimension fails to reach the upper limit of the range, it indicates that although the traffic is within a tolerable normal fluctuation range, its statistical features deviate from normal statistical features, and there is still a possibility of risk. At this time, the traffic probe can use the horizontal collaboration module to generate a traffic feature identifier based on the target sequence statistical features that have not reached the upper limit of the range and the packet features of each data packet in the traffic sequence. The current time is used as the notification generation time, and then a collaborative early warning notification of the risk warning type is generated based on the traffic feature identifier, the degree of deviation between the matching degree and the upper limit of the range, the confidence level determined by the degree of deviation, and the notification generation time.

[0088] D2: Send collaborative early warning notifications of the risk alert type to each other traffic probe in the probe cluster.

[0089] Other traffic probes are used to adjust the sampling ratio for data packets with traffic characteristic identifiers based on the confidence level in the collaborative early warning notification.

[0090] In practice, traffic probes that generate risk alert type collaborative warning notifications can send collaborative warning notifications to other traffic probes in the cluster through communication channels within the probe cluster (such as based on Generic Routing Encapsulation (GRE) or Virtual Extensible LAN (VXLAN)).

[0091] Upon receiving a coordinated alert notification, any traffic probe within the cluster can determine the corresponding traffic type based on the traffic feature identifier in the notification. Then, based on the confidence level carried in the alert notification, it dynamically adjusts its sampling ratio for that type of data packet to obtain its latest sampling ratio. The adjustment range of the sampling ratio is positively correlated with the confidence level; the higher the confidence level, the larger the adjustment range, thus specifically increasing the sampling density of traffic that is more likely to pose a risk. Conversely, the lower the confidence level, the smaller the adjustment range, avoiding unnecessary computational overhead.

[0092] For example, this embodiment can set two confidence thresholds, such as setting the first confidence threshold to 0.7 and the second confidence threshold to 0.3.

[0093] For example, taking the current traffic probe as the first traffic probe, after determining that the risk analysis result of the traffic sequence is that there is no risk, if the local risk identification engine is used to find that the traffic sequence has a weak abnormal trend based on the statistical characteristics and distribution of the target sequence (for example, the handshake fingerprint of the Transport Layer Security (TLS) protocol of the whitelisted domain name does not match the historical baseline (i.e., the distribution of statistical characteristics), but the abnormal intensity does not reach the threshold for triggering a global alarm on the analysis platform), the first traffic probe generates a risk warning token as a collaborative early warning notification and broadcasts it to each second traffic probe in the probe cluster through a horizontal channel. After receiving the warning token, the second traffic probe can adjust its local sampling strategy according to the confidence coefficient. For example, when the confidence is higher than the first confidence threshold (e.g., 0.7), it automatically reduces the sampling packet loss ratio for the traffic type corresponding to the feature identifier (e.g., from 1000:1 to 10:1); thereby significantly increasing the collection density of traffic that is more likely to have risks without affecting the normal processing of other traffic. When the confidence level is lower than the second confidence threshold (e.g., 0.3), it indicates that the probability of the traffic feature being risky is relatively low. Other traffic probes will only record the collaborative early warning notification and will not adjust the sampling ratio of the corresponding traffic.

[0094] In one embodiment, this application can also update the risk package characteristics based on the collaborative early warning notification of the risk warning type. Specifically, if within a preset time range, multiple collaborative early warning notifications of the risk warning type are received from other traffic probes in the probe cluster, and the traffic feature identifiers in each collaborative early warning notification are the same, then the risk package characteristics are updated according to the traffic feature identifier.

[0095] Here, the preset duration range is a pre-defined time window used to count the number of collaborative early warning notifications.

[0096] In practice, after receiving a collaborative warning notification of the same risk alert type from another traffic probe, the traffic probe can determine whether it has received any other collaborative warning notifications from other traffic probes within a preset time range. If not, it indicates that there are currently no multiple traffic probes simultaneously detecting the same type of classification, and the traffic probe can adjust its sampling ratio based on the currently received collaborative warning notifications. If yes, all collaborative warning notifications of the same risk alert type received within the preset time range can be classified according to the traffic feature identifier carried in the collaborative warning notification, and the source traffic probes of the collaborative warning notifications under each type of traffic feature identifier can be counted. If the number of different source traffic probes corresponding to the same traffic feature identifier reaches a preset number threshold, it is determined that the abnormal risk of the traffic under that type of traffic feature identifier no longer belongs to the normal fluctuation under a single probe, and then that type of traffic feature identifier is updated to the risk packet feature stored locally by all traffic probes. After the risk packet feature update is completed, when all traffic probes in the cluster perform the pre-matching stage of risk packet features, they can identify each data packet carrying that type of traffic feature identifier from the real-time collected traffic and perform risk analysis on all data packets. If the preset threshold is not reached, you can continue to wait for a new collaborative early warning notification to arrive.

[0097] For example, if multiple traffic probes report weak anomalies and send alert tokens for the same traffic feature identifier within a preset time range, a group consensus mechanism is triggered. Each traffic probe in the probe cluster can automatically update the traffic category corresponding to the traffic feature identifier to the global risk information.

[0098] In one embodiment, when the real-time hardware load of the traffic probe does not exceed a preset lower load limit, risk analysis is performed on each data packet collected by the traffic probe in real time to obtain the risk analysis results of each data packet.

[0099] In practice, if the real-time hardware load of the traffic probe does not exceed the preset lower load limit, it indicates that the traffic probe is currently under low load, with ample idle computing resources and cache space, capable of handling the full processing of traffic. In this case, risk analysis is performed on each data packet collected by the traffic probe in real time, yielding the risk analysis results for each packet. This approach avoids missing any collected data packets, maximizing detection accuracy, while also preventing excessive idleness of the traffic probe's computing resources.

[0100] Alternatively, if the data packets collected in real time by the traffic probe match the characteristics of the risk packets, risk analysis can be performed on the data packets to obtain the risk analysis results.

[0101] In practice, if the data packets collected by the traffic probe in real time match the risk packet characteristics, it indicates that the matched data packets carry known risk characteristics and need to be analyzed in detail. Therefore, the risk identification engine can be used to identify the risks of each matched data packet and obtain the corresponding risk identification results.

[0102] In one embodiment, the step of "determining the first feature entropy value of the traffic sequence based on the traffic feature information of the traffic sequence under multiple feature dimensions" in S102 above can be implemented as follows: If the real-time hardware load does not exceed the preset load limit, or if the real-time hardware load exceeds the preset load limit and the load deviation between the real-time hardware load and the preset load limit does not exceed the preset deviation level, the first feature entropy value of the traffic sequence is determined based on the traffic feature information of the traffic sequence under multiple feature dimensions for each traffic sequence composed of data packets collected within each preset time window.

[0103] Here, the high load level can further include a light high load level and a heavy high load level. The preset deviation level is a threshold for distinguishing between high and low overload levels, and can be used to further distinguish between light and heavy high load levels within a high load state. The level of the preset deviation level can be set empirically, and this application embodiment does not impose specific limitations. The load deviation level is used to characterize the magnitude of the relative difference between the real-time hardware load and the preset load upper limit.

[0104] When the real-time hardware load exceeds the preset load limit, but the load deviation does not exceed the preset deviation level, the current real-time hardware load is judged to be a slightly high load level. This situation indicates that although the flow probe is at a high load level, the extent of exceeding the preset load limit is small, and the hardware resources still have a certain processing margin and have not reached the level of resource shortage.

[0105] When the real-time hardware load exceeds the preset load limit and the load deviation exceeds the preset deviation level, the current real-time hardware load is judged to be at a level of heavy high load. This situation indicates that the current traffic probe's processing resources are under extreme strain.

[0106] In practice, since the pre-judgment has confirmed that the real-time hardware load of the current traffic probe exceeds the preset lower load limit, meaning there are only two scenarios: medium load or high load. Therefore, when the real-time hardware load does not exceed the preset upper load limit, the traffic probe can be determined to be at a medium load level. When the real-time hardware load exceeds the preset upper load limit, and the load deviation between the real-time hardware load and the preset upper load limit does not exceed a preset deviation level, the traffic probe can be determined to be in a slightly high load state. When the real-time hardware load exceeds the preset upper load limit, and the load deviation between the real-time hardware load and the preset upper load limit exceeds a preset deviation level, the traffic probe can be determined to be in a heavily high load state.

[0107] When the traffic probe is under medium or lightly high load, for the traffic sequence composed of each data packet collected by the traffic probe within each preset time window, the first feature entropy value of the traffic sequence can be determined based on the traffic feature information of the traffic sequence under multiple feature dimensions. The steps for determining the first feature entropy value are similar to the process of calculating the first feature entropy value in step S102, and will not be repeated here.

[0108] Alternatively, when the traffic probe is under light high load, for each data packet collected by the traffic probe within each preset time window, the proportion of high-priority data packets in each data packet can be determined first. If this proportion is less than a preset proportion, the data packets collected within each preset time window can be grouped into a traffic sequence, and then the first characteristic entropy value of the traffic sequence can be determined. If this proportion is not less than a preset proportion, the high-priority data packets can be filtered out and processed according to the full-volume analysis strategy, while the low-priority data packets are grouped into a traffic sequence and the first characteristic entropy value is calculated.

[0109] In one embodiment, this application also provides a flow analysis step when the load deviation exceeds a preset deviation level. Specifically, it can be implemented according to the following steps E1~E2: E1: When the load deviation exceeds the preset deviation level, determine the service priority of each data packet to which the traffic service belongs for each data packet collected in real time by the traffic probe.

[0110] Here, business priority refers to the pre-assigned importance levels of different business traffic. Higher priority business traffic requires a higher level of security protection. Business priority can be divided into multiple levels according to the actual business architecture, such as core business, important business, and general business.

[0111] In practice, if the load deviation exceeds a preset deviation level, the traffic probe is determined to be under heavy load. The traffic feature identifiers in the real-time collected data packets are matched against a locally stored service priority list to determine the service priority of each data packet. The service priority list is a pre-configured mapping rule table stored locally on the traffic probe, recording the relationship between the traffic feature identifiers of various services and their corresponding priority levels.

[0112] E2: From all data packets, select the target data packet with the highest business priority, perform risk analysis on the target data packet, and obtain the risk analysis results of the target data packet.

[0113] Here, the target data packet is the highest priority data packet determined after filtering by business priority in a heavy load scenario.

[0114] In practice, after determining the service priorities of all data packets, the traffic probe filters each data packet according to its priority, selecting the data packet with the highest service priority as the target data packet. Then, the risk identification engine can be used to perform risk analysis on each target data packet to obtain the risk analysis results.

[0115] For the remaining data packets (i.e., non-highest priority packets and unprocessed historical packets), the traffic probe can dynamically select the corresponding strategy based on the current remaining computing resources. When there are still a small amount of computing resources remaining, the remaining data packets are sampled at a very high preset sampling ratio. For example, the preset sampling ratio can be set to a sampling ratio of 10000:1, and risk analysis is performed only on a very small number of sampled data packets. When there are virtually no computing resources remaining, the remaining data packets are directly discarded without performing risk analysis.

[0116] In one embodiment, when the load deviation exceeds a preset deviation level, this application also provides a method for assisting in sharing the traffic analysis task through other traffic probes in the probe cluster. Specifically, it can be implemented according to the following steps F1~F2: F1: When the load deviation exceeds the preset deviation level, generate a collaborative early warning notification of the overload notification type based on the target data packet characteristics of the highest non-service priority data packet not processed by the traffic probe, and the assistance analysis indication related to the highest non-service priority data packet.

[0117] Here, the non-highest priority data packets refer to those data packets determined to be non-highest priority after being filtered by business priorities in heavy load scenarios. Target data packet characteristics are the characteristics of the non-high-priority data packets, such as the packet's five-tuple characteristics, traffic type, entropy information, and payload information. Overload notification type collaborative early warning notifications are collaborative interaction information sent to other traffic probes in the probe cluster when a traffic probe enters a heavy load state. Assisted analysis instructions are auxiliary information used to instruct other traffic probes to share the traffic analysis task, including at least the requested traffic sharing ratio and the current traffic probe's own processing ratio for the non-highest priority data packets. The requested traffic sharing ratio is used to request other traffic probes to take on an additional processing ratio.

[0118] In practical implementation, when the real-time hardware load of the current traffic probe is at a heavy load level, for data packets that are not the highest priority data packets after being filtered by service priority, the traffic probe can utilize the horizontal collaboration module to determine the additional processing ratio that each other traffic probe needs to undertake, based on the target data packet characteristics of the highest priority data packet itself and the proportion of the highest priority data packets it processes relative to the total number of data packets it processes. Then, based on the processing ratio and the five-tuple information related to other traffic probes in the traffic that the current traffic probe is expected to process, an assistance analysis instruction is generated for each other traffic probe. Based on the assistance analysis instruction and the current load level of the traffic probe (or real-time hardware load), an overload notification type collaborative early warning notification is generated, which can also be called an overload early warning token.

[0119] F2: Sends collaborative warning notifications of the overload notification type to each other traffic probe.

[0120] Other traffic probes are used to adjust the sampling ratio for data packets with target data packet characteristics based on the assistance analysis instructions in the collaborative early warning notification.

[0121] In practice, the current traffic probe sends overload notification-type collaborative early warning notices to other traffic probes in the probe cluster through the communication channel between them. Upon receiving the collaborative early warning notice, each of the other traffic probes can determine whether it has sufficient processing capacity for analysis based on the assistance analysis instructions in the notice and its own hardware load. If its processing capacity allows, it automatically adjusts its sampling strategy, increasing the sampling ratio of traffic types corresponding to the characteristics of the target data packet to assist in completing the traffic analysis task.

[0122] For example, there are three traffic probes in the probe cluster: Probe A, Probe B, and Probe C. Probe A is currently under heavy load. Probe A reduces the sampling ratio of non-highest priority traffic to 10000:1 and generates an overload notification type collaborative early warning. This collaborative early warning is distributed to Probe B and Probe C through the internal communication channel of the probe cluster. Upon receiving the collaborative early warning, Probe B first checks its real-time hardware load at 32%, confirming sufficient processing capacity. Then, based on the assistance analysis instructions in the collaborative early warning, it automatically adjusts its local sampling strategy, increasing the sampling ratio of traffic types matching the target data packet characteristics from 100:1 to 20:1. Upon receiving the collaborative early warning, Probe C checks its real-time load at 70%, determining that only a small amount of processing capacity remains. Therefore, it only increases the sampling ratio of the corresponding traffic from 100:1 to 50:1. Without increasing the burden on probe A, probes B and C increased their sampling ratios by different margins based on their respective processing capacity, thus sharing the task of analyzing some traffic that might have been lost due to the downgraded processing of probe A. This maintained the overall sampling quantity and detection level of this type of traffic within the cluster, achieving load balancing and collaborative defense within the cluster.

[0123] Optionally, if the traffic probes in the probe cluster store the traffic types that other traffic probes need to process, then when the current traffic probe is under heavy load, it can determine from the other traffic probes which will process the highest-priority non-service data packet in its current traffic probe, based on the target data packet characteristics of the highest-priority non-service data packet that has not been processed, and the stored traffic types that other traffic probes need to process. Then, the current traffic probe can generate a collaborative early warning notification of the overload notification type associated with each target traffic probe based on the target data packet characteristics of the highest-priority non-service data packet that has not been processed by the current traffic probe, and send it to the target traffic probe.

[0124] In one embodiment, when the load deviation exceeds a preset deviation level, this application also provides a traffic analysis step that combines priority and entropy values. Specifically, it can be implemented according to the following steps G1~G4: G1: When the load deviation exceeds the preset deviation level, determine the service priority of each data packet to which the traffic service belongs for each data packet collected in real time by the traffic probe.

[0125] In practice, when the load deviation exceeds the preset deviation level, i.e. when the current traffic probe is under heavy high load, the traffic probe obtains the preset service priority list and determines the service priority of each traffic service to which each data packet belongs for each data packet collected in real time by the traffic probe.

[0126] G2: Select the target data packet with the highest business priority from all data packets, and divide each target data packet into a target sequence according to the collection time of each target data packet.

[0127] Here, the target data packet is the data packet corresponding to the highest priority business that needs to be prioritized for ensuring detection resources in a heavy-load scenario.

[0128] In practice, the traffic probe can filter out the target data packets with the highest service priority from the data packets based on the service priority of the traffic service to which each data packet belongs. Then, according to the collection time sequence of each target data packet, according to a preset time length (usually 1 second), or according to the number of packets in a preset sequence (e.g., 10 packets per group) as the division unit, each target data packet can be divided into a target sequence.

[0129] G3: Based on the traffic characteristic information of the target sequence under multiple feature dimensions, determine the second feature entropy value of the target sequence, and based on the second feature entropy value and the historical feature entropy values ​​corresponding to multiple historical target sequences, determine the second target analysis strategy from multiple risk analysis strategies.

[0130] Here, multiple historical target sequences are traffic sequences consisting of data packets with the highest service priority within a historical time window. The second feature entropy value is the feature entropy value calculated only for the target sequence of the highest priority service.

[0131] In practice, based on the traffic characteristic information of the target sequence under multiple feature dimensions (such as message length distribution, arrival time interval distribution, payload byte distribution, etc.), the second feature entropy value of the target sequence is calculated. The traffic probe can retrieve the historical feature entropy values ​​corresponding to multiple historical target sequences. Then, based on the historical feature entropy values ​​and second feature entropy values ​​corresponding to multiple historical target sequences, a second target analysis strategy for the current target sequence can be determined. Understandably, the specific steps for determining the second target strategy from multiple risk analysis strategies are similar to the process of steps A1~A3 above, and will not be repeated here.

[0132] G4: Following the second objective analysis strategy, conduct risk analysis on the objective sequence to obtain the risk analysis results.

[0133] In practice, a risk identification engine can be used to perform a corresponding risk analysis process on the target sequence according to the second target analysis strategy, so as to obtain the risk analysis results of the detected target sequence.

[0134] By combining priority and entropy, in extreme scenarios with heavy load, the system ensures the allocation of computing power to core businesses through business priority screening, and realizes the scheduling of computing power for core businesses through dynamic strategies driven by entropy. This achieves a balance between detection coverage and risk identification accuracy for core businesses under limited computing power.

[0135] like Figure 2 The diagram shown illustrates the architecture of a probe cluster and analysis platform provided in this embodiment. This application employs a distributed deployment mode with an upper-layer analysis platform and multiple homogeneous traffic probes at the lower layer. The probe cluster includes n probes, such as traffic probe 1, traffic probe 2, ..., traffic probe n. The traffic probes within the cluster are interconnected bidirectionally through internal communication channels, forming a horizontal communication link. This supports direct horizontal information exchange between traffic probes without requiring the analysis platform as an intermediary. A single traffic probe can send warning tokens and / or overload tokens to other traffic probes in the cluster to achieve risk warning sharing or task sharing, reducing the risk of overload for a single traffic probe. Each traffic probe may include a traffic acquisition module, an information entropy calculation engine, a horizontal collaboration module, a risk identification engine, and a hardware load monitoring module. The traffic acquisition module is used to collect various data packets. The information entropy calculation engine is used to calculate the characteristic entropy values ​​of various traffic sequences or target sequences. The horizontal collaboration module can generate and send alert tokens and / or overload warning tokens, as well as receive alert tokens and / or overload warning tokens sent by other traffic probes, and update its own sampling ratio or risk packet characteristics based on the received alert tokens and / or overload warning tokens. The risk identification engine is used to detect and identify risks in traffic sequences or data packets. The hardware load monitoring module is used to monitor the real-time hardware load of the traffic probes and select analysis strategies based on the impact of real-time hardware load. That is, each traffic probe in this application has functions such as risk identification, information entropy calculation, generation and sending of collaborative warning notifications, and hardware load status detection. Traffic analysis no longer relies on the real-time intervention of the analysis platform; the probe cluster itself can independently identify and analyze the risks of the collected traffic. Each traffic probe can send the obtained risk analysis results to the analysis platform for use by operations and maintenance personnel.

[0136] The distributed deployment approach of this application enables the probe cluster to make autonomous decisions. It ensures the accuracy of traffic analysis through the independent risk identification and entropy calculation capabilities of each probe, while a horizontal collaboration mechanism facilitates rapid sharing of risk information and task distribution among the various traffic probes. This effectively balances the traffic analysis workload across the probes and reduces the overhead of real-time data transfer with the analysis platform, thus lowering the dependence on the platform's performance.

[0137] This application uses the real-time hardware load of the traffic probe as a prerequisite for the traffic analysis process. It divides the load into multiple levels based on preset load limits (lower, upper, and deviation levels) to achieve rational allocation of computing resources. When the real-time hardware load exceeds the preset lower limit, risk packet feature matching is performed on each data packet to directly identify known risk packets, reducing the volume of traffic entering subsequent complex calculations and effectively improving overall detection and processing efficiency. Data packets without matched risk packet features are grouped into traffic sequences within preset time windows. The first feature entropy value of the traffic sequence is calculated, and combined with historical feature entropy values, the first target analysis strategy is dynamically determined from full analysis, sampling analysis, and compression analysis strategies, improving resource utilization efficiency while ensuring detection accuracy. The entropy value judgment threshold is double-calibrated by the first and second constraints to ensure the stability of the detection process and results. In heavy high-load scenarios where the load deviation exceeds the preset deviation level, business priority filtering ensures the integrity of the detection of the highest priority traffic, avoiding downgrading of risk analysis for core businesses in high-traffic scenarios. Simultaneously, by using overload notification types for collaborative early warning notifications, other probes within the cluster share the analysis tasks for low-priority traffic, distributing the computational pressure on a single probe and further saving local probe hardware resources. The risk package characteristics and sampling ratios under the sampling analysis strategy are dynamically updated based on collaborative early warning notifications sent by other probes within the probe cluster, improving the overall resource utilization efficiency of the cluster and better addressing the issues of poor applicability of existing technologies in high-traffic scenarios and the problems of missed or false alarms.

[0138] In summary, the traffic analysis method provided in this application can achieve the following: 1. Content-aware, zero-latency adaptive defense: Through entropy detection with dynamic drift thresholds (i.e., dynamic upper and lower limit thresholds), traffic probes can break free from dependence on static platform policies and instead achieve real-time judgment of the intrinsic information (i.e., entropy information) value of passing traffic. This enables both extreme compression of repetitive background traffic and zero-latency full capture of sudden abnormal traffic, significantly improving the ability to capture unknown threats and zero-day attacks. 2. Distributed, low-latency threat lateral immune system: Through federated learning and alert token mechanisms among traffic probes, threat response time can be shortened from "minutes" to "milliseconds," enabling rapid deployment and collective consensus on local anomalies within the neighborhood, effectively curbing threat spread. 3. Performance-driven, highly robust elastic defense capability: By linking the sampling strategy to the hardware load status of the traffic probe itself, it can intelligently execute critical path protection under high traffic surges, preventing traffic probes from crashing due to overload, while ensuring uninterrupted security monitoring of core traffic services. 4. Possesses cross-layer collaboration capabilities. When the hardware load reaches a high level, the system automatically triggers horizontal overload warnings and adaptive threshold adjustments, achieving a dynamic balance between load balancing and security detection. 5. Reduced costs: Effectively reduces the total data traffic sent to the risk identification engine. With the same traffic volume, fewer traffic probes can be deployed, or a larger volume of traffic can be handled with the same number of probes, directly reducing hardware costs.

[0139] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0140] Based on the same inventive concept, this disclosure also provides a flow analysis device corresponding to the flow analysis method. Since the principle of the device in this disclosure for solving the problem is similar to the flow analysis method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0141] like Figure 3 The diagram shown is a schematic of a traffic analysis device applied to any traffic probe in a probe cluster, according to an embodiment of this disclosure, including: The acquisition module 301 is used to determine whether each data packet collected in real time by the traffic probe matches the risk packet feature information when the real-time hardware load of the traffic probe exceeds a preset load lower limit; the risk packet feature is dynamically updated based on the collaborative early warning notification between each traffic probe in the probe cluster. Calculation module 302, if not, is used to determine the first feature entropy value of the traffic sequence based on the traffic feature information of the traffic sequence under multiple feature dimensions, for the traffic sequence composed of each data packet collected in each preset time window. The determination module 303 is used to determine a first target analysis strategy from multiple risk analysis strategies based on the first feature entropy value and the historical feature entropy values ​​corresponding to historical traffic sequences under multiple historical time windows; the multiple risk analysis strategies include a full analysis strategy, a sampling analysis strategy, and a compression analysis strategy; the sampling ratio under the sampling analysis strategy is dynamically updated according to the collaborative early warning notification; The analysis module 304 is used to perform risk analysis on the traffic sequence according to the first target analysis strategy and obtain risk analysis results.

[0142] In one possible implementation, the determining module 303, when determining the first target analysis strategy from multiple risk analysis strategies based on the first feature entropy value and the historical feature entropy values ​​corresponding to historical traffic sequences under multiple historical time windows, is used to: Based on the degree of load deviation between the real-time hardware load and the preset load limit, determine the first weight and the second weight of the entropy standard deviation for multiple historical feature entropy values; An initial lower threshold is determined based on the product of the mean entropy value of multiple historical feature entropy values ​​and the first weight and the standard deviation of the entropy value; and an initial upper threshold is determined based on the product of the mean entropy value and the second weight and the standard deviation of the entropy value. The initial lower limit threshold is not lower than the preset lower limit threshold and the initial upper limit threshold is not greater than the preset upper limit threshold as the first constraint condition, and the change range of the initial lower and upper limit thresholds compared with the previously determined historical upper and lower limit thresholds is not greater than the preset range as the second constraint condition. The initial lower limit threshold and the initial upper limit threshold are adjusted respectively to obtain the current target upper limit threshold and target lower limit threshold. Based on the relationship between the first feature entropy value and the target upper limit threshold and the target lower limit threshold, a first target analysis strategy is determined from multiple risk analysis strategies.

[0143] In one possible implementation, the analysis module 304, when performing risk analysis on the traffic sequence according to the first target analysis strategy and obtaining the risk analysis result, is used to: When the first target analysis strategy is the sampling analysis strategy, the latest sampling ratio under the sampling analysis strategy is used to sample each data packet in the traffic sequence to obtain sampled data packets; Based on the data packet characteristics of the sampled data packets, a risk analysis is performed on the sampled data packets to obtain the risk analysis results.

[0144] In one possible implementation, the analysis module 304, when performing risk analysis on the traffic sequence according to the first target analysis strategy and obtaining the risk analysis result, is used to: When the first target analysis strategy is the compression analysis strategy, target sequence statistical features of the traffic sequence are generated based on the data packet content and data packet collection time of each data packet in the traffic sequence; the target sequence statistical features include at least one of packet length statistical features, time interval features, payload hash features, and communication protocol features; Obtain the statistical feature distribution of each historical traffic sequence that does not pose a risk, and determine the degree of matching between the statistical features of the target sequence and the statistical feature distribution; If the matching degree is within a preset range, the risk analysis result of the traffic sequence is determined to be that there is no risk.

[0145] In one possible implementation, the device further includes a first notification module 305, for: If the matching degree is less than the upper limit of the preset degree range, a collaborative early warning notification of the risk warning type related to the traffic sequence is generated; the collaborative early warning notification of the risk warning type includes at least the traffic feature identifier corresponding to the traffic sequence, the degree of deviation between the matching degree and the upper limit of the range, and the confidence level and notification generation time corresponding to the degree of deviation; The collaborative early warning notification of the risk alert type is sent to each other traffic probe in the probe cluster; the other traffic probes are used to adjust the sampling ratio for data packets with the traffic feature identifier according to the confidence level in the collaborative early warning notification.

[0146] In one possible implementation, the device further includes an update module 306, for: If, within a preset time period, multiple other traffic probes in the probe cluster send collaborative early warning notifications of the risk alert type, and the traffic feature identifiers in each collaborative early warning notification are the same, then the risk packet feature is updated based on the traffic feature identifier.

[0147] In one possible implementation, the analysis module 304 is configured to: If the real-time hardware load of the traffic probe does not exceed the preset lower limit, risk analysis is performed on each data packet collected by the traffic probe in real time to obtain the risk analysis results of each data packet. Alternatively, if the data packets collected in real time by the traffic probe match the characteristics of the risk packets, risk analysis can be performed on the data packets to obtain the risk analysis results of the data packets.

[0148] In one possible implementation, the calculation module 302, when determining the first feature entropy value of the traffic sequence composed of data packets collected within each preset time window based on the traffic feature information of the traffic sequence under multiple feature dimensions, is configured to: If the real-time hardware load does not exceed the preset load limit, or if the real-time hardware load exceeds the preset load limit and the load deviation between the real-time hardware load and the preset load limit does not exceed the preset deviation level, for the traffic sequence composed of each data packet collected in each preset time window, the first feature entropy value of the traffic sequence is determined according to the traffic feature information of the traffic sequence under multiple feature dimensions.

[0149] In one possible implementation, the computing module 302 is further configured to: If the load deviation exceeds the preset deviation level, the service priority of the traffic service to which each data packet belongs is determined for each data packet collected in real time by the traffic probe. From all the data packets, select the target data packet with the highest business priority; The analysis module 304 is further configured to: perform risk analysis on the target data packet and obtain the risk analysis result of the target data packet.

[0150] In one possible implementation, the device further includes a second notification module 307, used for: If the load deviation exceeds the preset deviation level, an overload notification type collaborative early warning notification is generated based on the target data packet characteristics of the highest non-service priority data packet not processed by the traffic probe, and the assistance analysis indication related to the highest non-service priority data packet. The overload notification type of the collaborative early warning notification is sent to each of the other traffic probes; the other traffic probes are used to adjust the sampling ratio for data packets with the characteristics of the target data packet according to the assistance analysis instruction in the collaborative early warning notification.

[0151] In one possible implementation, the computing module 302 is further configured to: If the load deviation exceeds the preset deviation level, the service priority of the traffic service to which each data packet belongs is determined for each data packet collected in real time by the traffic probe. From the data packets, the target data packets with the highest service priority are selected, and the target data packets are divided into target sequences according to the collection time of each target data packet. Based on the traffic characteristic information of the target sequence under multiple feature dimensions, determine the second feature entropy value of the target sequence; The determining module 303 is further configured to determine the second target analysis strategy from multiple risk analysis strategies based on the second feature entropy value and the historical feature entropy values ​​corresponding to multiple historical target sequences; The analysis module 304 is further configured to perform risk analysis on the target sequence according to the second target analysis strategy, and obtain risk analysis results.

[0152] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0153] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0154] Based on the same technical concept, embodiments of this application also provide a computer device. (Refer to...) Figure 4 The diagram shown is a structural schematic of a computer device provided in an embodiment of this application, comprising: The processor 401, memory 402, and bus 403 are included. Memory 402 stores machine-readable instructions that can be executed by the processor 401. The processor 401 executes the machine-readable instructions stored in memory 402. When the machine-readable instructions are executed by the processor 401, the processor 401 executes the above-described S101~S104.

[0155] The aforementioned memory 402 includes a main memory 4021 and an external memory 4022. The main memory 4021, also known as internal memory, is used to temporarily store the computational data in the processor 401, as well as the data exchanged with external memory such as a hard disk 4022. The processor 401 exchanges data with the external memory 4022 through the main memory 4021. When the computer device is running, the processor 401 and the memory 402 communicate through the bus 403, so that the processor 401 executes the execution instructions mentioned in the above method embodiments.

[0156] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the traffic analysis method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0157] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the traffic analysis method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0158] The computer program product can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0160] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0161] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0162] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0163] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the user through pop-up information or by asking the user to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0164] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A flow analysis method, characterized in that, The method, applied to any traffic probe in a probe cluster, includes: When the real-time hardware load of the traffic probe exceeds a preset lower load limit, for each data packet collected in real time by the traffic probe, it is determined whether each data packet matches the risk packet characteristics; the risk packet characteristics are dynamically updated based on the collaborative early warning notifications between the traffic probes in the probe cluster. If not, then for the traffic sequence composed of each data packet collected within each preset time window, the first feature entropy value of the traffic sequence is determined based on the traffic feature information of the traffic sequence under multiple feature dimensions. Based on the first feature entropy value and the historical feature entropy values ​​corresponding to historical traffic sequences under multiple historical time windows, a first target analysis strategy is determined from multiple risk analysis strategies; the multiple risk analysis strategies include a full analysis strategy, a sampling analysis strategy, and a compression analysis strategy; the sampling ratio under the sampling analysis strategy is dynamically updated according to the collaborative early warning notification; According to the first target analysis strategy, risk analysis is performed on the traffic sequence to obtain risk analysis results; The step of determining the first target analysis strategy from multiple risk analysis strategies based on the first feature entropy value and the historical feature entropy values ​​corresponding to historical traffic sequences under multiple historical time windows includes: Based on the degree of load deviation between the real-time hardware load and the preset load limit, determine the first weight and the second weight of the entropy standard deviation for multiple historical feature entropy values; An initial lower threshold is determined based on the product of the mean entropy value of multiple historical feature entropy values ​​and the first weight and the standard deviation of the entropy value; and an initial upper threshold is determined based on the product of the mean entropy value and the second weight and the standard deviation of the entropy value. The initial lower limit threshold is not lower than the preset lower limit threshold and the initial upper limit threshold is not greater than the preset upper limit threshold as the first constraint condition, and the change range of the initial lower and upper limit thresholds compared with the previously determined historical upper and lower limit thresholds is not greater than the preset range as the second constraint condition. The initial lower limit threshold and the initial upper limit threshold are adjusted respectively to obtain the current target upper limit threshold and target lower limit threshold. Based on the relationship between the first feature entropy value and the target upper and lower thresholds, a first target analysis strategy is determined from multiple risk analysis strategies. Specifically, when the first feature entropy value is less than or equal to the target lower threshold, it indicates that the overall flow sequence exhibits regularity and repetition, and the first target analysis strategy is determined to be a compression analysis strategy. When the first feature entropy value is greater than or equal to the target upper threshold, it indicates that the overall flow sequence exhibits uncertainty and unstructured characteristics, and the first target analysis strategy is determined to be a full-volume analysis strategy. When the first feature entropy value is greater than the target lower threshold and less than the target upper threshold, it indicates that the entropy value of the flow sequence is within the normal fluctuation range, and the first target analysis strategy is determined to be a sampling analysis strategy.

2. The method according to claim 1, characterized in that, The step of performing risk analysis on the traffic sequence according to the first target analysis strategy to obtain risk analysis results includes: When the first target analysis strategy is the sampling analysis strategy, the latest sampling ratio under the sampling analysis strategy is used to sample each data packet in the traffic sequence to obtain sampled data packets; Based on the data packet characteristics of the sampled data packets, a risk analysis is performed on the sampled data packets to obtain the risk analysis results.

3. The method according to claim 1, characterized in that, The step of performing risk analysis on the traffic sequence according to the first target analysis strategy to obtain risk analysis results includes: When the first target analysis strategy is the compression analysis strategy, target sequence statistical features of the traffic sequence are generated based on the data packet content and data packet collection time of each data packet in the traffic sequence; the target sequence statistical features include at least one of packet length statistical features, time interval features, payload hash features, and communication protocol features; Obtain the statistical feature distribution of each historical traffic sequence that does not pose a risk, and determine the degree of matching between the statistical features of the target sequence and the statistical feature distribution; If the matching degree is within a preset range, the risk analysis result of the traffic sequence is determined to be that there is no risk.

4. The method according to claim 3, characterized in that, The method further includes: If the matching degree is less than the upper limit of the preset degree range, a collaborative early warning notification of the risk warning type related to the traffic sequence is generated; the collaborative early warning notification of the risk warning type includes at least the traffic feature identifier corresponding to the traffic sequence, the degree of deviation between the matching degree and the upper limit of the range, and the confidence level and notification generation time corresponding to the degree of deviation; The collaborative early warning notification of the risk alert type is sent to each other traffic probe in the probe cluster; the other traffic probes are used to adjust the sampling ratio for data packets with the traffic feature identifier according to the confidence level in the collaborative early warning notification.

5. The method according to claim 1, characterized in that, The method further includes: If, within a preset time period, multiple other traffic probes in the probe cluster send collaborative early warning notifications of the risk alert type, and the traffic feature identifiers in each collaborative early warning notification are the same, then the risk packet feature is updated based on the traffic feature identifier.

6. The method according to claim 1, characterized in that, The method further includes: If the real-time hardware load of the traffic probe does not exceed the preset lower limit, risk analysis is performed on each data packet collected by the traffic probe in real time to obtain the risk analysis results of each data packet. Alternatively, if the data packets collected in real time by the traffic probe match the characteristics of the risk packets, risk analysis can be performed on the data packets to obtain the risk analysis results of the data packets.

7. The method according to claim 1, characterized in that, The step of determining the first feature entropy value of the traffic sequence, which is composed of data packets collected within each preset time window, based on the traffic feature information of the traffic sequence under multiple feature dimensions, includes: If the real-time hardware load does not exceed the preset load limit, or if the real-time hardware load exceeds the preset load limit and the load deviation between the real-time hardware load and the preset load limit does not exceed the preset deviation level, for the traffic sequence composed of each data packet collected within each preset time window, the first feature entropy value of the traffic sequence is determined based on the traffic feature information of the traffic sequence under multiple feature dimensions.

8. The method according to claim 7, characterized in that, The method further includes: If the load deviation exceeds the preset deviation level, the service priority of the traffic service to which each data packet belongs is determined for each data packet collected in real time by the traffic probe. From the various data packets, the target data packet with the highest business priority is selected, and a risk analysis is performed on the target data packet to obtain the risk analysis result of the target data packet.

9. The method according to claim 8, characterized in that, The method further includes: If the load deviation exceeds the preset deviation, a collaborative early warning notification of the overload notification type is generated based on the target data packet characteristics of the highest non-service priority data packet not processed by the traffic probe and the assistance analysis indication related to the highest non-service priority data packet. The overload notification type of the collaborative early warning notification is sent to each of the other traffic probes; the other traffic probes are used to adjust the sampling ratio for data packets with the characteristics of the target data packet according to the assistance analysis instruction in the collaborative early warning notification.

10. The method according to claim 7, characterized in that, The method further includes: If the load deviation exceeds the preset deviation level, the service priority of the traffic service to which each data packet belongs is determined for each data packet collected in real time by the traffic probe. From the data packets, the target data packets with the highest service priority are selected, and the target data packets are divided into target sequences according to the collection time of each target data packet. Based on the traffic characteristic information of the target sequence under multiple feature dimensions, the second feature entropy value of the target sequence is determined, and based on the second feature entropy value and the historical feature entropy values ​​corresponding to multiple historical target sequences, the second target analysis strategy is determined from multiple risk analysis strategies. According to the second target analysis strategy, risk analysis is performed on the target sequence to obtain risk analysis results.

11. A flow analysis device, characterized in that, The device is used for any traffic probe in a probe cluster, and includes: The acquisition module is used to determine whether each data packet collected in real time by the traffic probe matches the risk packet feature information when the real-time hardware load of the traffic probe exceeds a preset load lower limit; the risk packet feature is dynamically updated based on the collaborative early warning notification between each traffic probe in the probe cluster. The calculation module, if not, is used to determine the first feature entropy value of the traffic sequence based on the traffic feature information of the traffic sequence under multiple feature dimensions, for the traffic sequence composed of each data packet collected within each preset time window. The determination module is used to determine a first target analysis strategy from multiple risk analysis strategies based on the first feature entropy value and the historical feature entropy values ​​corresponding to historical traffic sequences under multiple historical time windows; the multiple risk analysis strategies include a full analysis strategy, a sampling analysis strategy, and a compression analysis strategy; the sampling ratio under the sampling analysis strategy is dynamically updated according to the collaborative early warning notification; wherein, when determining the first target analysis strategy from multiple risk analysis strategies based on the first feature entropy value and the historical feature entropy values ​​corresponding to historical traffic sequences under multiple historical time windows, the module is used to determine a first weight and a second weight for the entropy standard deviation of the multiple historical feature entropy values ​​based on the load deviation degree between the real-time hardware load and the preset load limit; An initial lower threshold is determined based on the product of the average entropy value of multiple historical feature entropy values ​​and the first weight and the standard deviation of the entropy value. An initial upper threshold is determined based on the product of the average entropy value, the second weight, and the standard deviation of the entropy value. The initial lower threshold and the initial upper threshold are adjusted according to the first constraint that the initial lower threshold is not lower than a preset lower threshold and the initial upper threshold does not exceed a preset upper threshold, and the change in the initial lower and upper thresholds compared to the previously determined historical thresholds does not exceed a preset range, to obtain the current target upper threshold and target lower threshold. The relationship between the first feature entropy value and the target upper threshold and target lower threshold is then considered. Based on the magnitude relationship, from multiple risk analysis strategies, a first target analysis strategy is determined. Specifically: when the first feature entropy value is less than or equal to the target lower threshold, it indicates that the overall flow sequence exhibits regularity and repetition, and the first target analysis strategy is determined to be a compression analysis strategy; when the first feature entropy value is greater than or equal to the target upper threshold, it indicates that the overall flow sequence exhibits uncertainty and unstructured characteristics, and the first target analysis strategy is determined to be a full-volume analysis strategy; when the first feature entropy value is greater than the target lower threshold and less than the target upper threshold, it indicates that the entropy value of the flow sequence is within the normal fluctuation range, and the first target analysis strategy is determined to be a sampling analysis strategy. The analysis module is used to perform risk analysis on the traffic sequence according to the first target analysis strategy and obtain the risk analysis results.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 10.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 10.

Images