A flow self-adaptive pressure reduction control method, device, equipment and medium

Abstract

The application discloses a flow self-adaptive decompression control method and device, equipment and medium, and relates to the technical field of computer network security and data processing. The method evaluates the pressure state by monitoring the resource occupation index of the system; when the first decompression trigger condition is met, the packet taking frequency of the upper application from the driving layer is reduced, and the flow is trimmed by using the natural overflow of the cache; when the first recovery control condition is met, the strategy is cancelled to recover the full amount of packet taking. The method of the application does not need to consume a large amount of CPU and memory resources, can accurately judge the pressure, and guarantees the system stability and core monitoring function continuity under high load.

Description

Technical Field

[0001] This invention relates to the field of computer network security and data processing technology, specifically to a flow adaptive pressure reduction control method, device, equipment, and medium. Background Technology

[0002] With the development of cloud computing and big data technologies, network traffic monitoring systems play a vital role in ensuring network security, analyzing user behavior, and maintaining service stability.

[0003] Network traffic monitoring systems need to perform deep packet inspection (DPI) on real-time traffic at network entry points, which typically includes full data collection, protocol parsing, feature matching, and storage indexing.

[0004] However, existing traffic monitoring systems face sudden, high-concurrency anomalies (such as DDoS attacks or traffic surges caused by sudden hotspot events), resulting in a rapid increase in inbound traffic. Traditional traffic control and overload protection schemes, such as the token bucket and leaky bucket algorithms, consume significant CPU cycles and memory resources under extreme traffic surges. This can cause the system to become overloaded or even paralyzed due to the handling of discarding logic before it even has a chance to discard traffic.

[0005] The memory consumption of deep packet inspection increases non-linearly with the complexity of data packets. Existing general system load balancing strategies are unable to accurately detect the "critical point" of memory accumulation. Once a sudden surge in traffic exceeds the system's processing capacity, the message processing queue will quickly accumulate, and a large number of pending data packet objects will occupy Java heap memory, which can easily trigger an OutOfMemoryError (OOM), causing the monitoring process to crash and resulting in the permanent loss of critical security audit data.

[0006] Therefore, there is an urgent need for a rate limiting technology that can reduce overhead to cope with sudden traffic surges, accurately assess pressure, and ensure uninterrupted core monitoring functions under extreme conditions. Summary of the Invention

[0007] This invention provides a flow adaptive pressure reduction control method, device, equipment, and medium to solve the problems of system overload / crash and interruption of core safety monitoring functions caused by excessive consumption of traditional application-layer flow limiting logic under extreme pressure in flow monitoring systems.

[0008] This invention is achieved through the following technical solution:

[0009] A first aspect of the present invention provides a flow adaptive pressure reduction control method, comprising:

[0010] The system monitors resource usage indicators and assesses the system's stress status based on these indicators.

[0011] When the pressure state meets the first decompression trigger condition, the first decompression strategy is executed; the first decompression strategy is to reduce the frequency at which the upper-layer application reads network data packets from the packet capture cache of the driver layer.

[0012] When the pressure state meets the first recovery control condition, the first decompression strategy is revoked, allowing the upper-layer application to read all network data packets from the packet capture cache.

[0013] Furthermore, the method for reducing the frequency at which upper-layer applications read network data packets from the packet capture cache of the driver layer is as follows:

[0014] The upper-layer application's packet retrieval thread is controlled to enter a forced sleep state after successfully reading one or multiple data packets consecutively; the sleep duration is either a fixed value or dynamically configured based on the system's current resource usage indicators.

[0015] Furthermore, it also includes:

[0016] When the pressure state meets the second decompression trigger condition, the second decompression strategy is executed; the second decompression strategy is: at the application layer traffic acquisition entry point, according to preset traffic filtering rules, network data packets identified as non-core business packets are discarded, and only core business data packets are allowed to enter the application layer memory; and,

[0017] When the pressure state meets the second recovery control condition, the second decompression strategy is revoked and the first decompression strategy is executed.

[0018] Furthermore, the first stress reduction trigger condition is: the resource occupancy index continues to increase during the first continuous monitoring period;

[0019] The second stress reduction trigger condition is: the resource occupancy index continues to increase and the growth rate accelerates during the second continuous monitoring period, or the resource occupancy index exceeds a hard threshold.

[0020] The hard thresholds include memory overflow risk thresholds and queue backlog dead limit thresholds.

[0021] Furthermore, the resource usage metrics include at least one of CPU utilization, memory utilization, and message processing queue length.

[0022] Further, the first recovery control condition is: the resource occupancy index is lower than the threshold or range required by the first decompression trigger condition minus the first hysteresis value, and is maintained for a first time length;

[0023] The second recovery control condition is: the resource occupancy index is lower than the threshold or range required by the second decompression trigger condition minus the second backlash value, and is maintained for a second duration.

[0024] Furthermore, the upper-layer application is a traffic acquisition module, which is used to sequentially read network data packets from the packet capture cache of the driver layer;

[0025] The packet capture buffer is either a network card hardware buffer or a circular buffer maintained by a kernel-mode packet capture driver.

[0026] A second aspect of the present invention provides a flow adaptive pressure reduction control device, comprising:

[0027] The traffic acquisition module is used to access the packet capture cache of the driver layer through the network port and read network data packets from the packet capture cache in a controlled manner.

[0028] The stress assessment module is used to monitor the system's resource usage indicators in real time and assess the system's stress status based on the resource usage indicators.

[0029] A pressure reduction controller is connected to the flow acquisition module and the pressure assessment module respectively, and is used to determine a pressure reduction strategy based on the pressure status, and to send a corresponding pressure reduction strategy execution command to the flow acquisition module.

[0030] The stress reduction strategy includes:

[0031] When the pressure state meets the first decompression trigger condition, the first decompression strategy is executed; the first decompression strategy is to reduce the frequency at which the upper-layer application reads network data packets from the packet capture cache of the driver layer.

[0032] When the pressure state meets the first recovery control condition, the first decompression strategy is revoked, allowing the upper-layer application to read all network data packets from the packet capture cache.

[0033] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the flow adaptive pressure reduction control method according to any one of the first aspects of the present invention.

[0034] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the flow adaptive pressure reduction control method according to any one of the first aspects of the present invention.

[0035] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0036] This invention differs from existing technologies that employ application-layer proactive packet dropping schemes that consume significant CPU resources. Instead, it monitors system resource usage metrics to determine the system's processing pressure. When this pressure triggers preset decompression conditions, it reduces the packet capture frequency of upper-layer applications, delegating complex packet drop decisions and execution to the underlying caching mechanism. Utilizing the first-in-first-out (FIFO) and overflow overwrite characteristics of the packet capture cache, it achieves traffic trimming. During peak traffic periods, it significantly reduces CPU context switching and object memory allocation overhead, greatly improving the system's survivability and stability under high load. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0038] Figure 1 This is an application system architecture diagram of a flow adaptive pressure reduction control method according to an embodiment of the present invention;

[0039] Figure 2 This is a flowchart of a flow adaptive pressure reduction control method according to an embodiment of the present invention;

[0040] Figure 3 This is a flowchart of a graded pressure reduction control method according to an embodiment of the present invention;

[0041] Figure 4 This is a block diagram of a flow adaptive pressure reduction control device according to an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0043] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims, and accompanying drawings of this invention are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to other steps or units inherent in the device.

[0044] The terminology used in the various embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to limit the various embodiments of the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. The terms (such as those defined in commonly used dictionaries) are to be interpreted as having the same meaning as in the context of the relevant technical field and are not to be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0045] The embodiments of the present invention propose a traffic adaptive pressure reduction control method, which is suitable for high-concurrency and burst network traffic scenarios, especially for scenarios that require full traffic collection and deep packet inspection. It can achieve low-overhead resistance to burst traffic impacts, accurate pressure judgment and core business protection, and effectively solve the problems of high resource consumption and poor system stability of traditional monitoring and rate limiting solutions.

[0046] The method proposed in this invention can be applied to various network traffic monitoring systems, such as enterprise-level network security audit systems, cloud service traffic monitoring platforms, and carrier network quality monitoring systems. Figure 1 The diagram shown is an architecture diagram of an application system of the method of the present invention. Its application scope mainly includes the driver layer, the application layer, and the network port connecting the two.

[0047] Driver layer: Located at the bottom layer of the system, it interacts directly with network hardware devices (such as network interface cards) and mainly includes packet capture buffers and kernel-level packet capture drivers. The packet capture buffer is a crucial area for storing network data packets to be read. Specifically, it can be a network interface card hardware buffer or a circular buffer maintained by the kernel-level packet capture driver (such as a circular buffer implemented based on packet capture drivers like Lippcap or PF_RING). The core function of the driver layer is to receive raw network data packets from network ports and temporarily store them in the packet capture buffer, providing a data source for upper-layer applications.

[0048] Application Layer: Located above the driver layer, this layer implements the core functions of traffic monitoring. It mainly includes a traffic acquisition module, an application layer memory queue, a message processing queue, and a deep packet inspection module (used for protocol parsing, feature matching, etc.). The traffic acquisition module, as a core component of the upper-layer application, accesses the driver layer's packet capture cache via a network port. It is responsible for reading network data packets from the capture cache and transmitting them to the application layer memory queue, where they are subsequently processed by the deep packet inspection module. The message processing queue is used to cache pending business tasks; its length directly reflects the system's business processing pressure.

[0049] After network packets enter the system from the network port, they are first received by the driver layer and stored in the packet capture buffer. The application layer traffic acquisition module reads packets from the packet capture buffer at a preset frequency and passes them into the application layer memory queue. The deep packet inspection module extracts packets from the application layer memory queue, performs operations such as protocol parsing and feature matching, and stores or reports the processing results. At the same time, the system's CPU provides computing resources for the operation of each module, while the memory provides storage support for the application layer memory queue and message processing queue.

[0050] Based on the above system architecture, the method of the present invention establishes an adaptive pressure reduction control mechanism between the application layer and the driver layer to achieve accurate perception of system pressure and dynamic adjustment of traffic, thereby solving many defects of the existing technology in the case of sudden traffic.

[0051] The implementation process of traditional traffic monitoring and control methods in the above system architecture is as follows: During the traffic acquisition phase, the application-layer traffic acquisition module reads all network data packets from the driver-layer packet capture cache at a fixed high rate, achieving full acquisition, and then transfers all of them to the application-layer memory queue. During the traffic monitoring phase, system resources are evaluated based on static thresholds, such as setting CPU utilization at 80%, memory usage at 90%, and message processing queue length at 10,000 as decompression thresholds. When a certain indicator instantaneously reaches or exceeds the corresponding decompression threshold, the system is determined to be overloaded. During the rate limiting control phase, once the system is determined to be overloaded, an application-layer active drop policy is activated. The deep packet inspection module or a dedicated rate limiting module judges each data packet in the memory queue, filters out some data packets according to preset rules (such as random drop or proportional drop), and performs drop operations to reduce the system processing pressure. When the system resource indicators instantaneously drop below the threshold, the rate limiting policy is revoked, the traffic acquisition module resumes full acquisition, and the system returns to normal processing mode.

[0052] In scenarios with sudden traffic surges, the above rate limiting methods require the application layer to receive massive amounts of data packets and store them in a memory queue. Subsequently, the rate limiting module must judge and discard the data packets in the queue one by one. During this process, the operations of receiving, storing, judging, and discarding data packets consume a large amount of CPU computing resources (for executing judgment logic and memory management) and memory resources (for caching data packets to be judged). This causes the system's own load to increase sharply when processing rate limiting logic, and may even lead to extreme situations where the more rate limiting is applied, the busier it becomes, and eventually, it crashes.

[0053] The flow adaptive pressure reduction control method of this invention fundamentally improves upon the shortcomings of traditional methods in terms of technical concept and implementation, such as... Figure 2 As shown, it includes the following steps.

[0054] S100 is a resource usage indicator for monitoring the system, used to assess the system's stress status.

[0055] The system in this implementation refers to a network traffic monitoring system with functions such as network traffic collection, deep packet inspection, and / or business processing, such as an enterprise-level network security audit system or a cloud service traffic monitoring platform.

[0056] Resource usage metrics can include at least one of CPU utilization, memory utilization, and message processing queue length. Single or combined metrics can be selected for monitoring based on the system's core resource bottlenecks.

[0057] Among these metrics, CPU utilization refers to the activity level of the system's CPU per unit of time, reflecting the system's computing resource load. It is obtained in real-time through the process management interface provided by the operating system. Memory usage refers to the ratio of currently used physical memory to total physical memory, reflecting the system's storage resource load. It is obtained through the operating system's memory management interface. Message processing queue length refers to the number of data packets waiting to be processed by the deep packet inspection module in the application layer, reflecting the system's business processing pressure. It is obtained by reading the real-time count of the message processing queue.

[0058] This step can achieve real-time data acquisition, storage, and preliminary processing through hardware circuits or by running the corresponding computer program on a processor.

[0059] Based on the monitoring results of resource usage indicators, the system pressure status can be divided into normal state, mild pressure state, moderate pressure state, and near-collapse state.

[0060] Because the instantaneous value of a single metric cannot accurately perceive the trend and "critical point" of memory accumulation, in deep packet inspection scenarios, the complexity of data packets varies greatly, and memory consumption grows non-linearly, potentially escalating rapidly from a low memory usage rate to a memory overflow threshold within a short period. Without monitoring the memory growth rate, rate limiting often fails to be triggered in time. By the time the metric reaches the threshold, memory is already nearly exhausted, and performing a drop operation at this point is unlikely to prevent memory overflow, leading to a crash in the monitoring process.

[0061] Furthermore, because static thresholds are determined instantaneously, they ignore the trend of resource changes. During periods of sudden traffic fluctuations, system resource metrics may fluctuate frequently around the thresholds, leading to frequent activation and deactivation of rate limiting policies. For example, CPU utilization might rise from 79% to 81% within one second (triggering rate limiting), then drop to 78% in the next second (revoking rate limiting), and then rise again to 82% (triggering rate limiting again), and so on. This frequent switching causes drastic fluctuations in the number of data packets in the application-layer memory queue, disrupts the processing rhythm of the deep packet inspection module, severely affects the stability and continuity of business processing, and reduces the accuracy of traffic monitoring.

[0062] Therefore, in a preferred embodiment, a comprehensive indicator and the changing trends of each indicator are used to determine the system stress state. The comprehensive indicator includes CPU utilization, memory usage, and message processing queue length. By monitoring CPU utilization, memory usage, and message processing queue length over a period of time, their respective changing trends and growth rates are determined, and numerical thresholds and rate-of-change thresholds for each indicator are set for each stress state. The stress state is then determined by comparison.

[0063] The system stress state is determined by the changing trends of resource usage indicators. For example, if CPU utilization shows no continuous increase within a set period and does not exceed a set threshold, the system is considered to be in a normal state. If CPU utilization shows a continuous increase within a preset monitoring period or exceeds a set threshold, the system is considered to be under stress, requiring stress reduction strategies. The stress state can be further categorized into mild stress, moderate stress, or near-collapse based on the severity of the values. The same approach is used for analyzing memory utilization and message queue length. Finally, the stress state is determined by combining the results of CPU utilization, memory utilization, and message queue length assessments. The final stress state result can be the majority of multiple stress state assessments. For example, if CPU utilization and memory utilization both point to moderate stress, while message queue length points to mild stress, then the system is determined to be under moderate stress.

[0064] In some preferred embodiments, a weighted fusion approach using multiple indicators can be employed for stress assessment to address the resource bottleneck differences across different systems. For example, for CPU-intensive traffic monitoring systems, CPU utilization is assigned the first weight, memory usage the second, and message processing queue length the third (first weight > second weight > third weight); for memory-intensive systems, memory usage is assigned the first weight, CPU utilization the second, and message processing queue length the third. Normalized values ​​for each indicator are calculated, mapping them to the 0-1 range. Then, the fused stress value is calculated according to the weights. When the stress value continuously increases during the monitoring period, the corresponding stress reduction strategy is triggered. This approach better reflects the actual resource characteristics of the system and improves the accuracy of stress assessment.

[0065] Under normal conditions, no stress reduction strategy is implemented, and the system performs the traffic collection, monitoring, and parsing process according to the established configuration. Under mild stress, the corresponding mild stress reduction strategy is implemented; under moderate stress, the corresponding moderate stress reduction strategy is implemented.

[0066] S200, when the pressure state meets the first decompression trigger condition, execute the first decompression strategy; wherein, the first decompression strategy is: reduce the frequency at which the upper layer application reads network data packets from the packet capture cache of the driver layer.

[0067] In this step, the first decompression trigger condition can correspond to the system being in a state of mild stress. In this state, the system is under continuous pressure but has not yet reached its resource limit and still has a certain business processing capacity. The judgment is based on the continuous increase of resource usage indicators (one or more of CPU utilization, memory utilization, and message processing queue length) within the first continuous monitoring period. For example, if within 5 seconds of continuous monitoring, CPU utilization continuously increases from 60% to 75%, memory utilization continuously increases from 70% to 80%, or message processing queue length increases from 5000 to 8000, the first decompression trigger condition is met, and the corresponding first decompression strategy is executed.

[0068] Unlike traditional application-layer methods that actively judge and discard data packets, this invention reduces the frequency at which upper-layer applications retrieve packets from the driver-layer packet capture cache, allowing data packets to naturally remain in the cache. When the influx rate of data packets consistently exceeds the read rate of the upper-layer application, older cached data is naturally overwritten (overflow) by new data, thus automatically reducing traffic without requiring any complex judgment or discarding logic at the application layer. This mechanism, which passively overflows data packets from the packet capture cache, significantly reduces the computational overhead and memory usage at the application layer, enabling the system to reserve more resources for core business processing during sudden traffic surges.

[0069] In some implementations, the first decompression trigger condition may correspond to the system being in a moderate-pressure state, where the system is under continuous pressure (greater than the pressure under a light-pressure state), but has not yet reached its resource limit and still possesses a certain business processing capacity. The judgment criteria may be that resource usage indicators (one or more of CPU utilization, memory utilization, and message processing queue length) continuously increase within the third continuous monitoring period (the third continuous monitoring period is shorter than the first continuous monitoring period) or continuously increase within the first continuous monitoring period, but at a greater rate. In this case, the decompression strategy of reducing the frequency at which the upper-layer application reads network data packets from the packet capture cache of the driver layer is also executed. However, compared to being judged as being in a light-pressure state, the reduced packet capture frequency can be greater, i.e., reading network data packets at a lower frequency.

[0070] In one specific implementation, reducing the frequency at which upper-layer applications read network packets from the packet capture cache of the driver layer can be achieved by controlling the packet capture thread of the traffic acquisition module to enter a forced sleep state after successfully reading a packet, which corresponds to a moderate pressure state. The sleep duration can be a fixed value, suitable for scenarios where the system pressure increases relatively gradually.

[0071] The sleep duration can be dynamically configured based on the system's current resource usage metrics. For example, when the CPU utilization growth rate is 3% / second, the sleep duration is set to 10 milliseconds; when the growth rate is 5% / second, the sleep duration is set to 20 milliseconds. The specific mapping relationship can be implemented through a preset configuration table, which contains the sleep duration corresponding to different growth rate ranges. The stress assessment module calculates the growth rate in real time and queries the configuration table, sending the corresponding sleep duration to the stress reduction controller, which then issues it to the traffic acquisition module for execution. The traffic acquisition module adjusts the working mode of its packet acquisition thread according to the instructions and executes the sleep operation.

[0072] In one specific implementation, reducing the frequency at which upper-layer applications read network data packets from the packet capture cache of the driver layer can be achieved by controlling the packet capture thread to enter a forced sleep state after continuously reading multiple data packets (e.g., 2 to 5), which can correspond to handling mild stress states.

[0073] When the packet capture frequency of the traffic acquisition module decreases, the data packets in the driver layer's packet capture buffer cannot be read in a timely manner, while new data packets from the network continue to flow in and be stored in the packet capture buffer. Since the packet capture buffer has a limited capacity, when the buffer is full, subsequent incoming data packets will overwrite older data packets according to the first-in, first-out (FIFO) principle, achieving natural overflow. This method does not require the application layer to perform any judgment or discarding operations on data packets; traffic reduction can be achieved simply by adjusting the packet capture frequency, significantly reducing CPU and memory overhead.

[0074] S300: When the stress state meets the first recovery control condition, the first stress reduction strategy being executed is canceled, allowing the upper-layer application to read all network data packets from the packet capture cache.

[0075] The system monitors resource usage metrics in real time. When the first recovery control condition is met, it sends a recovery command to the pressure reduction controller. Upon receiving the command, the pressure reduction controller sends a command to the traffic acquisition module to cancel the first pressure reduction strategy. The packet acquisition module's packet capture thread stops its sleep operation, resumes full-speed packet capture mode, and reads all network data packets from the packet capture buffer. The system then returns to normal processing status.

[0076] In some implementations, to prevent the system from oscillating frequently at the pressure critical point, a hysteresis comparison mechanism is introduced during the recovery control process. Specifically, the first recovery control condition is set as follows: the resource occupancy index is lower than the threshold or range required for the first decompression trigger condition minus the first hysteresis value, and this condition is maintained for a first time duration.

[0077] The first backlash value can be preset according to system characteristics. For example, the first backlash value for CPU utilization can be set to 10%, the first backlash value for memory usage can be set to 8%, and the first backlash value for message processing queue length can be set to 2000. If the first decompression trigger condition is that CPU utilization continuously increases to 75% within 5 seconds, then the first recovery control condition is that CPU utilization drops below 75%-10%=65% and remains there for 3 seconds. When CPU utilization drops to 65% and does not rebound for 3 seconds, the first recovery control condition is met. If the first decompression trigger condition is that the message processing queue length increases from 5000 to 8000 (the threshold range is 5000-8000), then the first recovery control condition is that the message processing queue length drops below 8000-2000=6000 and remains there for 3 seconds, which satisfies the first recovery control condition.

[0078] This implementation effectively filters out false alarms caused by instantaneous traffic spikes by introducing trend analysis and growth rate judgment within a continuous monitoring period. When the system experiences brief traffic fluctuations, the pressure reduction strategy will not be triggered because the condition for continuous growth is not met, thus avoiding the false triggering problem of traditional static threshold methods.

[0079] In the first decompression strategy, the sleep duration can be dynamically adjusted not only based on the growth rate of current resource usage indicators but also in conjunction with the packet capture cache utilization rate. The system monitors the packet capture cache utilization rate in real time. When the packet capture cache utilization rate is below a certain set threshold, the sleep duration is appropriately shortened; when the packet capture cache utilization rate is above a certain set threshold, the sleep duration is appropriately extended. This adjustment method ensures that the packet capture frequency matches the state of the packet capture cache, avoiding insufficient or excessive cache overflow, thereby optimizing the traffic reduction effect.

[0080] Furthermore, such as Figure 3As shown, the above-mentioned flow adaptive pressure reduction control method of the present invention further includes the following steps:

[0081] S400 When the pressure state meets the second decompression trigger condition, the second decompression strategy is executed. The second decompression strategy is: at the traffic collection entry point of the application layer, according to the preset traffic filtering rules, network data packets identified as non-core business are discarded, and only core business data packets are allowed to enter the application layer memory.

[0082] S500: When the pressure condition meets the second recovery control condition, the second pressure reduction strategy is canceled and the first pressure reduction strategy is executed.

[0083] In this embodiment, the second decompression trigger condition can correspond to the system being under heavy pressure, i.e., the system is close to its resource limit and faces the risk of crashing. In this state, the decompression strategy sacrifices the integrity of regular monitoring data, ensuring the survival of system processes by cutting off the occupation of non-core resources, and does not miss critical network attack alarms.

[0084] The second decompression trigger condition can be any of the following conditions:

[0085] Condition 1: The resource occupancy index continues to increase and the growth rate accelerates during the second continuous monitoring period;

[0086] Condition 2: Resource usage exceeds the hard threshold.

[0087] Condition 1 is, for example: CPU utilization is detected to increase from 75% to 90% within 6 seconds, with an increase rate of 3% / second in the first 3 seconds and an increase rate of 5% / second in the last 3 seconds; or the message processing queue length increases from 8000 to 15000 within 6 seconds, with the increase rate increasing from 1000 / second to 2000 / second.

[0088] Hard thresholds include memory overflow risk thresholds and queue backlog dead limit thresholds. For example, a memory overflow risk threshold of 95% (when memory utilization exceeds 95%, there is an extremely high risk of OOM) and a queue backlog dead limit threshold of 20,000 (when the message processing queue length exceeds 20,000, the system's processing capacity is completely saturated and it cannot process new tasks in a timely manner).

[0089] The second stress-reduction strategy involves executing a "core business retention and non-core business blocking" logic at the application layer traffic collection entry point. Pre-defined traffic filtering rules identify core and non-core business traffic, directly discarding non-core business data packets and allowing only core business data packets into application layer memory. This maximizes system resource conservation and ensures the continuity of core business operations.

[0090] Pre-defined traffic filtering rules can be implemented by storing a traffic filtering rule table in the stress reduction controller. This rule table clearly distinguishes between the characteristics of core and non-core business operations. Core business operations can include network attack signature detection, threat intelligence gathering, and traffic monitoring of critical business systems (such as financial transaction systems and core databases). Their characteristics may include specific source IP addresses, destination IP addresses, port numbers, protocol types, and packet characteristic fields. Non-core business operations can include ordinary web browsing traffic, routine health check API calls, and entertainment application traffic. Their characteristics are also defined through IP addresses, port numbers, protocol types, and field contents. After receiving the severe stress status determination result from the stress assessment module, the stress reduction controller sends a second stress reduction strategy execution command and filtering rules to the traffic acquisition module. The traffic acquisition module then performs traffic filtering and discarding operations.

[0091] During the actual filtering process, when the traffic acquisition module receives data packets from the driver layer's packet capture cache, it first extracts the characteristics of the data packets and matches them against a preset filtering rule table. If the match is a non-core business characteristic, the data packet is discarded directly at the traffic acquisition entry point, preventing it from entering the memory queue and not allocating any Java object memory. If the match is a core business characteristic, it is allowed to enter the application layer memory and is processed preferentially by the deep packet inspection module.

[0092] By directly blocking non-core business traffic, the system avoids non-core data packets from occupying application layer memory and CPU resources, enabling it to concentrate all available resources on processing core security business. This ensures that critical functions such as network attack detection and threat intelligence collection remain uninterrupted under extreme pressure, achieving the goal of "survival".

[0093] Users can add, modify, or delete feature rules for core business processes in real time through the system management interface. Simultaneously, core business processes can be prioritized; for example, attack signature detection can be set as the first priority, threat intelligence gathering as the second priority, and critical business system traffic monitoring as the third priority. When system resources are extremely strained, only the first-priority core processes can be retained, further conserving resources and ensuring the normal operation of the most critical functions.

[0094] In one specific implementation, the second recovery control condition is set as follows: the resource occupancy index is lower than the threshold or range required by the second decompression trigger condition minus the second backlash value, and is maintained for a second duration.

[0095] The second round-trip value can be set higher than the first round-trip value to ensure that system resources are fully stable before recovery, avoiding a rapid return to a state of heavy stress. For example, the second round-trip value for CPU utilization can be set to 15%, the second round-trip value for memory usage to 12%, and the second round-trip value for message processing queue length to 5000.

[0096] If the second stress-relief trigger condition is that CPU utilization accelerates to 90% within 6 seconds, then the second recovery control condition is that CPU utilization drops below 90% - 15% = 75% and remains there for 5 seconds. When CPU utilization drops to 73% and remains there for 5 seconds, the second recovery control condition is met. If the second stress-relief trigger condition is that memory utilization exceeds 95% (a hard threshold), then the second recovery control condition is that memory utilization drops below 95% - 12% = 83% and remains there for 5 seconds. When memory utilization drops to 80% and remains there for 5 seconds, the recovery condition is met.

[0097] After the system detects that the resource indicators meet the second recovery control condition, it sends a recovery command to the pressure reduction controller. Upon receiving the command, the pressure reduction controller sends a command to the traffic acquisition module to cancel the second pressure reduction strategy, and the traffic acquisition module stops performing traffic filtering and discarding operations. Simultaneously, the pressure reduction controller sends a command to the traffic acquisition module to execute the first pressure reduction strategy, and the system switches from heavy pressure reduction mode to light pressure reduction mode. If the resource indicators subsequently meet the first recovery control condition further, the first pressure reduction strategy will be canceled, and the system will return to normal mode.

[0098] Step S500 can also be that when the pressure state meets the second recovery control condition, the second decompression strategy is directly revoked, so that the upper layer application reads all network data packets from the packet capture cache.

[0099] This implementation sets hard red lines by establishing memory overflow risk thresholds and queue backlog deadlines. When the system faces a crash risk, a severe stress reduction strategy is immediately initiated, blocking non-core business traffic and retaining only core security services. This design constructs the system's last line of defense, ensuring that critical functions such as network attack detection and threat intelligence gathering do not fail under extreme attack scenarios, preventing the loss of critical security audit data due to system crashes, and safeguarding the core value of network security protection. Simultaneously, the two-tiered stress reduction strategy allows the system to take corresponding control measures based on the severity of the stress, achieving precise hierarchical stress control while balancing the integrity of business processing and system stability.

[0100] Based on the same inventive concept, embodiments of the present invention provide a flow adaptive pressure reduction control device, such as... Figure 4 As shown, it includes:

[0101] The traffic acquisition module 401 is used to access the packet capture buffer of the driver layer through the network port and read network data packets from the packet capture buffer in a controlled manner.

[0102] The stress assessment module 402 is used to monitor the system's resource usage indicators in real time and assess the system's stress status based on the resource usage indicators.

[0103] The pressure reduction controller 403 is connected to the flow acquisition module 401 and the pressure assessment module 402 respectively, and is used to determine the pressure reduction strategy according to the pressure status, and to send the corresponding pressure reduction strategy execution command to the flow acquisition module.

[0104] This stress reduction strategy includes:

[0105] When the stress state meets the first stress reduction trigger condition, the first stress reduction strategy is executed; the first stress reduction strategy is to reduce the frequency at which the upper layer application reads network data packets from the packet capture cache of the driver layer.

[0106] When the stress state meets the first recovery control condition, the first stress reduction strategy is revoked, allowing the upper-layer application to read all network data packets from the packet capture cache.

[0107] Among them, the traffic acquisition module 401, as the core component of the upper-layer application, establishes a communication connection with the packet capture cache of the driver layer through the network port, and reads the network data packets in the packet capture cache in sequence. Its reading rate is dynamically controlled by the instructions of the pressure reduction controller 403.

[0108] Furthermore, the traffic acquisition module 401 has a built-in packet retrieval thread. When it receives the first pressure reduction strategy execution instruction issued by the pressure reduction controller 403, the packet retrieval thread automatically enters a forced sleep state after successfully reading one or more consecutive data packets.

[0109] The sleep duration can be preset to a fixed value by the pressure relief controller 403, or the pressure assessment module 402 can collect the current resource usage indicators of the system in real time and transmit the indicator data to the pressure relief controller 403, which will then dynamically calculate and issue an appropriate sleep duration.

[0110] Furthermore, the monitoring scope of the stress assessment module 402 includes at least one of CPU utilization, memory usage, and message processing queue length, and collects data of each indicator in real time by calling the operating system interface.

[0111] Furthermore, the pressure reduction controller 403 stores the execution logic of the second pressure reduction strategy. When the pressure assessment module 402 determines that the system pressure state meets the second pressure reduction trigger condition, the pressure reduction controller 403 sends the second pressure reduction strategy instruction and the preset flow filtering rules to the flow acquisition module 401.

[0112] The traffic acquisition module 401 adds a feature recognition unit at the traffic acquisition entry point of the application layer. By extracting features such as IP address, port number, and protocol type of data packets, it matches them with filtering rules, discards non-core business data packets, and only allows core business data packets to enter the application layer memory.

[0113] When the pressure condition meets the second recovery control condition, the pressure reduction controller 403 issues a command to cancel the execution of the second pressure reduction strategy and the execution of the first pressure reduction strategy. The flow acquisition module 401 stops the feature recognition and filtering operation and switches to execute the first pressure reduction strategy.

[0114] The stress assessment module 402 has a built-in trigger condition determination unit, which presets a first continuous monitoring period and a second continuous monitoring period, as well as a memory overflow risk threshold and a queue backlog dead limit threshold. When the resource usage indicator is detected to be continuously increasing during the first continuous monitoring period, it is determined that the first stress reduction trigger condition is met; when the indicator is continuously increasing during the second continuous monitoring period and the growth rate accelerates, or exceeds any hard threshold, it is determined that the second stress reduction trigger condition is met, and the determination result is fed back to the stress reduction controller 403.

[0115] Furthermore, the pressure reduction controller 403 has a built-in recovery condition determination unit, which presets a first hysteresis value, a second hysteresis value, and a corresponding maintenance time length. When the resource occupancy index is lower than the first pressure reduction trigger condition threshold minus the first hysteresis value and is maintained for a first time length, it is determined that the first recovery control condition is met; when the index is lower than the second pressure reduction trigger condition threshold minus the second hysteresis value and is maintained for a second time length, it is determined that the second recovery control condition is met, thereby triggering the corresponding strategy cancellation operation.

[0116] Furthermore, the packet capture buffer can be a hardware buffer on the network interface card (NIC) or a circular buffer maintained by the kernel-mode packet capture driver. By configuring the size of the NIC / packet capture buffer, the system's tolerance for burst traffic can be precisely adjusted.

[0117] Embodiments of the present invention also provide an electronic device including a processor and a memory, wherein the number of processors may be one or more. The memory, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules. The processor executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory, thereby implementing the flow adaptive pressure reduction control method of any of the above embodiments of the present invention.

[0118] The memory may primarily comprise a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory may include high-speed random access memory (RAM) and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory may further include memory remotely located relative to the processor, which can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0119] Embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the flow adaptive pressure reduction control method of any embodiment of the present invention.

[0120] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0121] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0122] Embodiments of the present invention also provide a computer program product that, when run on a computer, causes the computer to execute the flow adaptive pressure reduction control method of any of the above embodiments of the present invention.

[0123] The above embodiments are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the appended claims.

Claims

1. A flow adaptive pressure reduction control method, characterized in that, include: The system monitors resource usage indicators and assesses the system's stress status based on these indicators. When the pressure state meets the first decompression trigger condition, the first decompression strategy is executed; the first decompression strategy is to reduce the frequency at which the upper-layer application reads network data packets from the packet capture cache of the driver layer. When the pressure state meets the first recovery control condition, the first decompression strategy is revoked, so that the upper-layer application reads all network data packets from the packet capture cache. When the pressure state meets the second decompression trigger condition, the second decompression strategy is executed; the second decompression strategy is: at the traffic collection entry point of the application layer, according to the preset traffic filtering rules, network data packets identified as non-core business are discarded, and only core business data packets are allowed to enter the application layer memory. as well as, When the pressure state meets the second recovery control condition, the second decompression strategy is revoked and the first decompression strategy is executed; The first stress reduction trigger condition is: the resource occupancy index continues to increase during the first continuous monitoring period; The second stress reduction trigger condition is: the resource occupancy index continues to increase and the growth rate accelerates during the second continuous monitoring period, or the resource occupancy index exceeds a hard threshold. The hard thresholds include memory overflow risk thresholds and queue backlog dead limit thresholds.

2. The flow adaptive pressure reduction control method according to claim 1, characterized in that, The method to reduce the frequency at which upper-layer applications read network data packets from the packet capture cache of the driver layer is as follows: The upper-layer application's packet-fetching thread is forced to enter a sleep state after successfully reading one or multiple data packets consecutively. Its sleep duration can be a fixed value or dynamically configured based on the current resource usage index of the system.

3. The flow adaptive pressure reduction control method according to claim 1 or 2, characterized in that, The resource usage metrics include at least one of CPU utilization, memory utilization, and message processing queue length.

4. The flow adaptive pressure reduction control method according to claim 1, characterized in that, The first recovery control condition is: the resource occupancy index is lower than the threshold or range required by the first decompression trigger condition minus the first hysteresis value, and is maintained for a first time length; The second recovery control condition is: the resource occupancy index is lower than the threshold or range required by the second decompression trigger condition minus the second backlash value, and is maintained for a second duration.

5. The flow adaptive pressure reduction control method according to claim 1, characterized in that, The upper-layer application is a traffic acquisition module, which is used to sequentially read network data packets from the packet capture cache of the driver layer. The packet capture buffer is either a network card hardware buffer or a circular buffer maintained by a kernel-mode packet capture driver.

6. A flow adaptive pressure reduction control device, characterized in that, include: The traffic acquisition module is used to access the packet capture cache of the driver layer through the network port and read network data packets from the packet capture cache in a controlled manner. The stress assessment module is used to monitor the system's resource usage indicators in real time and assess the system's stress status based on the resource usage indicators. A pressure reduction controller is connected to the flow acquisition module and the pressure assessment module respectively, and is used to determine a pressure reduction strategy based on the pressure status, and to send a corresponding pressure reduction strategy execution command to the flow acquisition module. The stress reduction strategy includes: When the pressure state meets the first decompression trigger condition, the first decompression strategy is executed; the first decompression strategy is to reduce the frequency at which the upper-layer application reads network data packets from the packet capture cache of the driver layer. When the pressure state meets the first recovery control condition, the first decompression strategy is revoked, so that the upper-layer application reads all network data packets from the packet capture cache. When the pressure state meets the second decompression trigger condition, the second decompression strategy is executed; the second decompression strategy is: at the application layer traffic acquisition entry point, according to preset traffic filtering rules, network data packets identified as non-core business packets are discarded, and only core business data packets are allowed to enter the application layer memory; and, When the pressure state meets the second recovery control condition, the second decompression strategy is revoked and the first decompression strategy is executed; The first stress reduction trigger condition is: the resource occupancy index continues to increase during the first continuous monitoring period; The second stress reduction trigger condition is: the resource occupancy index continues to increase and the growth rate accelerates during the second continuous monitoring period, or the resource occupancy index exceeds a hard threshold. The hard thresholds include memory overflow risk thresholds and queue backlog dead limit thresholds.

7. The flow adaptive pressure reduction control device according to claim 6, characterized in that, The traffic acquisition module has a built-in packet retrieval thread. When it receives the first pressure reduction strategy execution instruction issued by the pressure reduction controller, the packet retrieval thread automatically enters a forced sleep state after successfully reading one or more consecutive data packets. The sleep duration is preset to a fixed value by the pressure reduction controller, or dynamically configured according to the current resource usage index of the system.

8. The flow adaptive pressure reduction control device according to claim 6, characterized in that, The first recovery control condition is: the resource occupancy index is lower than the threshold or range required by the first decompression trigger condition minus the first hysteresis value, and is maintained for a first time length; The second recovery control condition is: the resource occupancy index is lower than the threshold or range required by the second decompression trigger condition minus the second backlash value, and is maintained for a second duration.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the flow adaptive pressure reduction control method according to any one of claims 1-5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the flow adaptive pressure reduction control method according to any one of claims 1-5.

Images