Large model firewall fine-grained dynamic scheduling method and system
By constructing a heterogeneous execution resource pool and performance probes, fine-grained dynamic scheduling of large-scale firewalls is achieved, solving the problems of resource pool coupling and scheduling unfairness, improving system throughput and shortening service recovery time.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU ANQUAN DIGITAL INTELLIGENCE TECH CO LTD
- Filing Date
- 2026-04-14
- Publication Date
- 2026-07-21
Smart Images
Figure CN122437680A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of information technology, specifically to a fine-grained dynamic scheduling method and system for large-scale firewalls. Background Technology
[0002] The widespread application of Large Language Models (LLMs) has spurred the development of firewall systems specifically designed to protect the input and output security of these models. These firewalls typically integrate multiple security detection modules, such as jailbreak attack detection, sensitive information identification, and harmful content filtering. Each module may involve heterogeneous computing tasks such as regular expression matching (CPU-intensive), large model inference calls (GPU-intensive), or remote policy queries (I / O-intensive). To ensure system stability under high concurrency scenarios, existing technologies generally employ a coarse-grained degradation strategy triggered by a global threshold. When the overall system CPU utilization exceeds a preset threshold, all non-critical security modules are uniformly downgraded, skipped, or delayed.
[0003] However, existing technologies suffer from the following drawbacks: resource pool coupling issues: the loads of CPU, GPU, and I / O resources interfere with each other, and overload of a certain type of resource (such as GPU memory) may cause unrelated modules (such as pure CPU computing modules) to be mistakenly downgraded, resulting in unnecessary loss of detection capabilities; lack of module granularity: existing solutions cannot distinguish between high-resource-overhead modules and low-resource-overhead modules, causing lightweight modules to "pay the price" for heavyweight modules, resulting in low effective system throughput; rigid recovery mechanisms: recovery after downgrade often relies on the overall decline of global indicators rather than independent recovery according to resource type, prolonging service degradation time; unfair scheduling: high-priority but low-resource-consumption modules may be incorrectly suppressed due to global policies, violating the service quality assurance principle. Summary of the Invention
[0004] This specification describes a fine-grained dynamic scheduling method and system for a large-model firewall through several embodiments.
[0005] Firstly, this specification provides a fine-grained dynamic scheduling method for a large-model firewall, including the following steps:
[0006] Define a unified abstract detection interface, with each security module of the large model serving as its implementation class. The static attributes of the abstract detection interface record the scheduling information of the security module.
[0007] Three types of execution resource pools are constructed, including a CPU thread pool, a GPU inference queue, and an asynchronous I / O thread pool.
[0008] The firewall's security modules are associated with at least one execution resource pool based on the scheduling information.
[0009] Deploy module performance probes to collect operational metrics after each execution of the security module and calculate the real-time load of the execution resource pool.
[0010] When the real-time load of any execution resource pool exceeds a preset upper threshold, the security module in that execution resource pool is downgraded according to the static attributes.
[0011] When the real-time load of any execution resource pool falls below the lower threshold and remains below the preset duration, the degraded security module within that execution resource pool is attempted to be restored based on the static attributes.
[0012] Secondly, embodiments of this specification provide a fine-grained dynamic scheduling system for a large-model firewall, comprising:
[0013] The interface module defines a unified abstract detection interface, with each security module of the large model serving as its implementation class. The static attributes of the abstract detection interface record the scheduling information of the security module.
[0014] The module constructs three types of execution resource pools, including a CPU thread pool, a GPU inference queue, and an asynchronous I / O thread pool.
[0015] The association module associates the firewall's security modules with at least one execution resource pool based on the scheduling information.
[0016] The probe module deploys performance probes to collect operational metrics after each execution of the security module and calculates the real-time load of the execution resource pool.
[0017] The scheduling module, when the real-time load of any execution resource pool exceeds a preset upper threshold, initiates a downgrade of the security module in that execution resource pool based on the static attributes. When the real-time load of any execution resource pool falls below the lower threshold and remains below the preset duration, it attempts to restore the downgraded security module in that execution resource pool based on the static attributes.
[0018] Thirdly, embodiments of this specification provide an electronic device, including a processor and a memory;
[0019] The processor is connected to the memory;
[0020] The memory is used to store executable program code;
[0021] The processor runs a program corresponding to the executable program code stored in the memory to perform the method described in any of the above aspects.
[0022] Fourthly, embodiments of this specification provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above aspects.
[0023] Fifthly, embodiments of this specification provide a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the above aspects.
[0024] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:
[0025] In several embodiments of this specification, a fine-grained dynamic scheduling method and system for large-model firewalls is provided. This method constructs three independent heterogeneous execution resource pools: a CPU thread pool, a GPU inference queue, and an asynchronous I / O thread pool. The real-time load of each resource pool is calculated separately, enabling precise degradation through resource pool isolation. When a certain type of resource is overloaded, degradation is only initiated for security modules within that resource pool, leaving modules in other resource pools completely unaffected. This avoids the problem of unrelated modules being mistakenly downgraded due to interference between CPU, GPU, and I / O loads in traditional global threshold strategies, maximizing the preservation of the firewall's overall detection capabilities. The system uses module performance probes to collect the execution time and resource consumption of each security module in real time, calculates a normalized resource overhead score, and constructs a comprehensive weight based on module priority. During degradation, modules are executed sequentially in descending order of weight, ensuring that high-overhead, low-priority modules are degraded first, while low-overhead, high-priority lightweight modules always receive execution guarantees. This module cost-aware scheduling mechanism effectively overcomes the unfair phenomenon of "lightweight modules paying for heavyweight modules," improving the system's effective throughput. This invention employs a gradual degradation strategy, reassessing the resource pool load after each degradation to ensure a smooth decline in service quality rather than a precipitous collapse. Simultaneously, each resource pool independently sets recovery thresholds and duration conditions. When the load of a resource pool continuously falls below the lower threshold, recovery is only attempted for the degraded modules within that pool, eliminating the need to wait for global metrics to recover and significantly shortening service degradation time. New security modules only need to implement a unified abstract detection interface and declare scheduling information to be automatically incorporated into the scheduling system via performance probes, establishing a dynamic feedback loop of "resource type—module cost—pool-level load." This provides excellent scalability and adaptability, and all security modules skipped due to degradation have their original requests and context written to the audit log, meeting the traceability and compliance requirements of strictly regulated scenarios.
[0026] Other features and advantages of various embodiments of this specification will be further revealed in the following detailed description and accompanying drawings. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 This is a schematic diagram of the fine-grained dynamic scheduling of the firewall provided in this manual.
[0029] Figure 2 This is a flowchart illustrating the fine-grained dynamic scheduling method for firewalls provided in this specification.
[0030] Figure 3 This is a schematic diagram of the firewall fine-grained dynamic scheduling system provided in this specification.
[0031] Figure 4 This is a schematic diagram of the electronic device provided in this manual. Detailed Implementation
[0032] The technical solutions of the embodiments of this specification will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of this specification and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of this specification.
[0033] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.
[0034] In the following description, terms such as “inner,” “outer,” “upper,” “lower,” “left,” and “right” are used only to facilitate the description of the embodiments and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this specification.
[0035] All data involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0036] This invention provides a fine-grained dynamic scheduling method and system for large-model firewalls. Please refer to the appendix. Figure 1 After receiving user input 11, the large model 12 generates content 13. This content 13 is not directly displayed to the user; it must first pass the checks of multiple security modules 14 before being displayed as output content 15. The solution described in this specification defines a unified abstract detection interface. Each security module of the large model firewall acts as an implementation class of this interface, and the scheduling information of each security module is recorded in the static properties of the interface. The scheduling information includes resource type, priority, whether it can be skipped, and whether it can be replaced. The resource type is CPU-intensive, GPU-intensive, or I / O-intensive. Secondly, three types of heterogeneous execution resource pools are constructed: a CPU thread pool, a GPU inference queue, and an asynchronous I / O thread pool. Specifically, resources for executing computational tasks are assigned to the CPU thread pool, resources for executing large model calls are assigned to the GPU inference queue, and resources for executing non-blocking operations are assigned to the asynchronous I / O thread pool. Then, based on the scheduling information of the security modules, they are associated with the corresponding execution resource pools: CPU-intensive modules are associated with the CPU thread pool, GPU-intensive modules with the GPU inference queue, and I / O-intensive modules with the asynchronous I / O thread pool. Next, a module performance probe is deployed to collect operational metrics after each security module is executed. These metrics include execution time, resource consumption, and number of timeouts / failures, and the real-time load of each execution resource pool is calculated. Specifically, the CPU thread pool score is calculated as: score_cpu = 0.6 × (queue length / maximum allowed queue) + 0.4 × (average latency / 200ms); the GPU inference queue score is calculated as: score_gpu = 0.7 × (memory utilization) + 0.3 × (P99 latency / 800ms); and the asynchronous I / O thread pool score is calculated as: score_io = 0.5 × (response latency / preset latency threshold) + 0.5 × error rate. When the real-time load of any execution resource pool exceeds the preset upper threshold, the security modules within that resource pool are degraded based on static attributes. During degradation, the normalized resource overhead score (cost) is first calculated based on execution time and resource consumption. Then, the comprehensive weight W = cost × (1 - priority / 5) is calculated. Security modules are arranged in descending order of W, and degradation is initiated one by one until the load falls below the upper threshold to a preset level. Degradation actions include: switching to the distillation mini-model if the module is replaceable; skipping execution if it can be skipped; otherwise, reducing priority until it is restored. When the real-time load of any execution resource pool falls below the lower threshold for a preset duration, the degradation of security modules in that resource pool is attempted to be restored based on static attributes.
[0037] This manual first provides a fine-grained dynamic scheduling method for large-model firewalls. Please refer to the appendix. Figure 2 The steps include:
[0038] Step S1) Define a unified abstract detection interface, with each security module of the large model serving as its implementation class. The static attributes of the abstract detection interface record the scheduling information of the security module.
[0039] Defining a unified abstract detection interface is the core foundation for building a pluggable and dynamically schedulable security module system. This interface specifies the detection methods that all security detection modules must implement, with each security module serving as a concrete implementation class of this interface. To achieve fine-grained elastic scheduling, this interface records the scheduling information of each security module through static attributes. The scheduling information includes resource type, priority, whether it can be skipped, and whether it can be replaced. Among them, the resource type is further subdivided into CPU-intensive, GPU-intensive, or I / O-intensive, used to accurately associate the module with the corresponding execution resource pool (CPU thread pool, GPU inference queue, or asynchronous I / O thread pool); the priority value ranges from 1 to 5, with higher values indicating higher importance of the module, and lower-priority modules will be processed first during degradation; whether it can be skipped indicates whether the module is a critical module, if so, it cannot be skipped directly and other degradation methods must be adopted; whether it can be replaced indicates whether the module supports switching to a distillation mini-model, so that the detection task can continue to be executed at a lower accuracy or faster speed when resources are scarce.
[0040] Step S2) Construct three types of execution resource pools, including a CPU thread pool, a GPU inference queue, and an asynchronous I / O thread pool.
[0041] The specific construction method is as follows: Execution resources for computational tasks are allocated to a CPU thread pool. This thread pool handles CPU-intensive computational tasks such as rule matching, regular expression scanning, and keyword filtering. Internally, it maintains a worker thread queue to support concurrent task submission and execution. Execution resources for large model calls are allocated to a GPU inference queue. This queue manages all security detection modules that require inference using large language models (such as semantically sensitive content analysis and jailbreak attack detection). Queuing requests enables efficient reuse of GPU memory and batch processing optimization, and supports timeout and circuit breaker mechanisms. Execution resources for non-blocking operations are allocated to an asynchronous I / O thread pool. This thread pool handles I / O-intensive non-blocking operations such as remote policy queries, log reporting, and external API calls. It uses coroutines or asynchronous callback models to avoid thread blocking and improve resource utilization. The three types of resource pools are physically or logically isolated from each other, each maintaining its own independent task queue, number of worker threads / coroutines, and load monitoring metrics. CPU thread pools focus on queue length and average latency, GPU inference queues focus on memory usage and P99 latency, and asynchronous I / O thread pools focus on response latency and error rate.
[0042] Step S3) Associate the firewall's security module with at least one execution resource pool according to the scheduling information.
[0043] Associating firewall security modules with at least one execution resource pool based on the scheduling information is the core step in establishing the mapping relationship between modules and heterogeneous resource pools. Specifically, each security module, when implementing the unified abstract detection interface, declares its scheduling information through static attributes, where the resource type (CPU-intensive, GPU-intensive, or I / O-intensive) determines the execution resource pool to which the module should belong. The association methods include: associating CPU-intensive modules with a dedicated CPU thread pool, which handles computational tasks such as rule matching and regular expression scanning; associating GPU-intensive modules with a dedicated GPU inference queue, which manages all detection tasks that require inference using large language models; and associating I / O-intensive modules with an asynchronous I / O thread pool, which is used for non-blocking operations such as remote policy querying and log reporting. At least one means that some composite modules may involve multiple resource types at the same time. For example, if a module first performs lightweight CPU preprocessing and then calls GPU inference, it can be associated with both the CPU thread pool and the GPU inference queue according to the resource requirements of each stage, and the scheduler can allocate it to the corresponding resource pool for execution at different stages.
[0044] Step S4) Deploy the module performance probe to collect running indicators after each execution of the security module and calculate the real-time load of the execution resource pool.
[0045] During firewall system operation, a performance probe is deployed for the execution of each security module. This probe automatically collects key operational metrics for each call after its completion. The collected metrics include execution time, resource consumption (such as CPU time slice increment and GPU memory increment), and the number of timeouts / failures. These metrics are used to calculate the normalized resource cost score for each security module and also serve as an aggregated data source to calculate the real-time load of the three types of execution resource pools. Each resource pool's load is calculated using an independent scoring formula, as described in claim 4: For the CPU thread pool, the score `score_cpu` = 0.6 × (queue length / maximum allowed queue) + 0.4 × (average latency / 200ms), where queue length reflects the current backlog of tasks, and average latency reflects task processing efficiency; for the GPU inference queue, the score `score_gpu` = 0.7 × (memory utilization) + 0.3 × (P99 latency / 800ms), where memory utilization directly reflects GPU resource saturation, and P99 latency reflects the queuing and inference time of long-tail requests; for the asynchronous I / O thread pool, the score `score_io` = 0.5 × (response latency / preset latency threshold) + 0.5 × error rate, balancing response time and stability. All scores are normalized to the range of 0 to 1, with higher scores indicating a heavier resource pool load.
[0046] Step S5) When the real-time load of any execution resource pool exceeds the preset upper threshold, the security module in the execution resource pool is downgraded according to the static attributes.
[0047] When the real-time load of any execution resource pool exceeds a preset upper threshold, a fine-grained degradation process is triggered for that resource pool. The preset upper threshold can be set to a load score greater than 0.8 (taking the normalized range of 0 to 1 as an example). At this time, only the security modules in the overloaded resource pool are degraded, while modules in other resource pools are unaffected, achieving precise intervention. The specific execution method of degradation includes the following steps:
[0048] First, based on the operational metrics (execution time and resource consumption) collected by the module performance probes, a normalized resource overhead score (cost) is calculated for each security module. The score quantifies the system resources consumed by a module in a single detection; modules with higher overhead should be prioritized for degradation.
[0049] Secondly, combining the priority attribute (ranging from 1 to 5, with higher values indicating greater importance) of the module's static attributes, the overall weight W is calculated as cost × (1 - priority / 5). Modules with higher resource overhead (cost) and lower priority have higher weights (W). The scheduler sorts all non-critical modules (or all degradable modules) in the current resource pool in descending order of weight (W), with higher-weight modules appearing earlier in the degradation sequence. Degradation actions are initiated one by one according to the sorting order. After each degradation, the real-time load of the current resource pool is reassessed until the load falls below the upper threshold to a preset level. This step-by-step degradation strategy avoids excessive sacrifice at once, allowing for a smooth decline in system service quality.
[0050] When the "Replaceable" attribute of a module is true, the module is switched to the corresponding distillation mini-model for detection. The distillation mini-model has a small number of parameters, fast computation speed, and low resource consumption, enabling it to continue providing detection capabilities at a lower accuracy cost even under resource constraints. When "Replaceable" is false but "Skipable" is true, the execution of the module is skipped, and the original request and context are written to the audit log to ensure compliance traceability. When "Skipable" is also false (i.e., the module is a critical module that cannot be skipped or replaced), the module's priority is reduced (e.g., its weight is temporarily reduced or it is placed at the end of the demotion sequence), so that its execution is delayed as much as possible in subsequent scheduling until the system load returns to normal, at which point its original priority is restored.
[0051] Step S6) When the real-time load of any execution resource pool falls below a lower threshold and remains below it for a preset duration, an attempt is made to restore the degraded security module within that execution resource pool based on the static attributes. The lower threshold can be set to a load score below 0.6, and the preset duration is, for example, 30 consecutive seconds, to ensure that the load drop is stable rather than a momentary fluctuation. The restoration process strictly adheres to the module's static attributes and specifically includes the following mechanisms:
[0052] First, all downgraded security modules are selected from the current resource pool, including modules switched to the distillation mini-model, skipped modules, and modules with suppressed priority. Candidate modules are then sorted according to descending priority and ascending normalized resource overhead. Modules with higher priority are restored first, reflecting the principle of prioritizing critical business operations. In the case of equal priority, modules with lower resource overhead are restored first to reduce the risk of load rebound after restoration.
[0053] Secondly, the system attempts to restore modules one by one in the sorted order. For modules that were previously switched to the distillation small model, the restoration action is to switch back to the original large model detection logic; for modules that were previously skipped, the restoration action is to reintegrate them into the security detection pipeline; for modules whose priority was suppressed, the restoration action is to restore their original priority. After each module restoration attempt, the system enters a short monitoring window to observe whether the load score of the current resource pool shows a significant rebound. If the load score remains stable, the restoration of that module is confirmed to be effective, and the system continues to attempt to restore the next module. If the load score rises sharply in the monitoring window, the module is immediately downgraded again, and the current restoration process is terminated, waiting for the next trigger condition to be met before attempting to restore it again.
[0054] On the other hand, this specification provides a fine-grained dynamic scheduling system for large-model firewalls; please refer to the appendix. Figure 3 ,include:
[0055] Interface module 100 defines a unified abstract detection interface, with each security module of the large model serving as its implementation class. The static attributes of this abstract detection interface record the scheduling information of each security module. The interface records the scheduling information of each module through static attributes, including resource class, priority, whether it can be skipped, and whether it can be replaced. At system startup, the interface module loads all implementation classes through classpath scanning or configuration files and registers them in the internal registry.
[0056] Module 200 constructs three types of execution resource pools: a CPU thread pool, a GPU inference queue, and an asynchronous I / O thread pool. The CPU thread pool adopts a standard work queue model, with the number of core threads, maximum number of threads, and queue capacity configured based on the number of server CPU cores and expected load. It is used to execute computationally intensive tasks such as rule matching and regular expression scanning. The GPU inference queue uses a bounded blocking queue, equipped with a dedicated scheduling thread to manage large model call requests, and integrates a memory monitoring component to collect real-time memory usage and P99 inference latency. The asynchronous I / O thread pool is based on a coroutine or non-blocking I / O model and is used to handle non-blocking operations such as remote policy queries and audit log reporting.
[0057] The association module 300 associates firewall security modules with at least one execution resource pool based on the scheduling information. The module reads the scheduling information of each security module from the interface module registry and associates the module with the corresponding execution resource pool according to resource type: CPU-intensive modules are associated with the CPU thread pool, GPU-intensive modules with the GPU inference queue, and I / O-intensive modules with the asynchronous I / O thread pool. For composite modules that need to use multiple resources simultaneously, the association module allows them to be associated with multiple resource pools, and the scheduling module dynamically delivers them in stages during execution. After association is completed, each resource pool maintains a list of modules belonging to that pool for subsequent degradation and recovery decisions.
[0058] The probe module 400 deploys performance probes to collect runtime metrics after each execution of a security module, calculating the real-time load of the execution resource pool. Before and after each security module execution, the module non-intrusively collects runtime metrics using aspect-oriented programming (AOP) techniques, including execution time, resource consumption, and the number of timeouts or failures. The collected data is asynchronously sent to a metric aggregator, which uses a sliding window to calculate a normalized resource cost score for each module. The score comprehensively reflects the average resource consumption of a single module execution. Simultaneously, the probe module periodically obtains aggregated data such as queue length, average latency, memory utilization, P99 latency, and error rate from the monitoring components of each resource pool, calculating real-time load scores for the three types of resource pools. All scores are normalized to the range of 0-1, with higher scores indicating heavier loads, and are updated in real-time to shared memory.
[0059] The scheduling module 500, when the real-time load of any execution resource pool exceeds a preset upper threshold, initiates a degradation of the security modules within that execution resource pool based on the static attributes. When the real-time load of any execution resource pool falls below the lower threshold for a preset duration, it attempts to restore the degraded security modules within that execution resource pool based on the static attributes. The module continuously monitors the real-time load score of each resource pool and has built-in degradation and recovery controllers. When the score of any resource pool exceeds the preset upper threshold, the degradation controller only performs degradation operations on non-critical modules within that pool. First, it obtains the resource overhead score of each module from the probe module, then arranges candidate modules in descending order of weight, and performs degradation actions one by one: if the module is replaceable, it switches to the distillation small model; if the module is not replaceable but can be skipped, it skips execution and writes the request context to the audit log; if the module is neither replaceable nor skippable, its priority is temporarily reduced. After each degradation, the pool load is reassessed, and degradation stops once the score falls below the security score. When the score of a resource pool remains below the lower threshold for a preset duration, the recovery controller retrieves all degraded modules from that pool, sorts them by priority in descending order and resource cost in ascending order, and attempts to recover them one by one: the distillation model switches back to the original model, skipped modules are reintroduced into the pipeline, and priority is suppressed to restore the original value. After each module is recovered, the monitoring window is activated. If the load score rebounds above the preset rebound threshold, the recovery operation is rolled back and the current recovery round is terminated.
[0060] The present invention will be described in detail below through specific application scenarios.
[0061] This embodiment uses a large-scale firewall deployed in an online customer service system as an example. This firewall integrates four security modules: jailbreak attack detection, sensitive information leakage detection, harmful content filtering, and topic deviation control. Each module has different computational characteristics: jailbreak attack detection and harmful content filtering require calling a large language model for semantic analysis, which is GPU-intensive; sensitive information leakage detection uses regular expressions and rule matching, which is CPU-intensive; and topic deviation control requires querying a remote knowledge base, which is I / O-intensive.
[0062] S101) Define a unified abstract detection interface
[0063] The system first defines a unified abstract detection interface called Detectable. Each security module implements this interface and declares its scheduling information in the interface's static properties. The scheduling information includes four items: resource type, priority, whether it can be skipped, and whether it can be replaced. Specifically, the jailbreak attack detection module declares its resource type as GPU-intensive, its priority as 5 (highest), whether it can be skipped as false (meaning critical modules cannot be skipped), and whether it can be replaced as true (meaning switching to the distillation mini-model is allowed). The sensitive information leakage detection module declares its resource type as CPU-intensive, its priority as 4, whether it can be skipped as true, and whether it can be replaced as false. The harmful content filtering module declares its resource type as GPU-intensive, its priority as 3, whether it can be skipped as false, and whether it can be replaced as true. The topic deviation control module declares its resource type as I / O-intensive, its priority as 2, whether it can be skipped as true, and whether it can be replaced as false. Through this unified interface, the scheduling information of all modules is standardized and stored for subsequent read by the scheduler.
[0064] S102) Construct three types of execution resource pools
[0065] The system constructs three isolated execution resource pools based on the hardware environment. The first is a CPU thread pool, with a core thread count set to twice the number of server CPU cores and a maximum thread count set to four times the number of cores, with a work queue capacity of 200. This thread pool is specifically used for CPU-intensive tasks such as rule matching and regular expression scanning. The second is a GPU inference queue, employing a bounded blocking queue with a maximum capacity of 64, and configured with a dedicated scheduling thread responsible for submitting inference requests in the queue to the GPU for execution. The GPU inference queue monitors memory usage and inference latency in real time. The third is an asynchronous I / O thread pool, implemented based on a coroutine model, with 32 threads, used for handling non-blocking operations such as remote knowledge base queries and audit log reporting. Each of the three resource pools maintains its own independent task queue and monitoring metrics, ensuring no interference between them.
[0066] S103) Associate the security module with the execution resource pool
[0067] The scheduler reads the resource type from the scheduling information of each module and associates the module with the corresponding resource pool. GPU-intensive jailbreak attack detection and harmful content filtering modules are associated with the GPU inference queue; CPU-intensive sensitive information leakage detection modules are associated with the CPU thread pool; and I / O-intensive topic deviation control modules are associated with the asynchronous I / O thread pool. These associations are stored in the scheduler's module registry, and each resource pool maintains its own list of modules belonging to that pool.
[0068] S104) Deployment Module Performance Probe and Load Calculation
[0069] Before and after the execution of the `detect` method in each security module, the system deploys performance probes using aspect-oriented programming. The probes collect the execution time (milliseconds), resource consumption (CPU time slice increment, GPU memory increment), and whether a timeout or failure occurred for each execution. This raw data is fed into a module performance analyzer, which uses a sliding window to calculate a normalized resource overhead score (cost) for each module. For example, the jailbreak attack detection module has a longer execution time and higher memory consumption, resulting in a cost of 0.9; the sensitive information leakage detection module has a very short execution time and low CPU consumption, resulting in a cost of 0.2.
[0070] Meanwhile, the system calculates the real-time load score for the three types of resource pools every second:
[0071] CPU thread pool score = 0.6 × (current queue length / maximum queue capacity 200) + 0.4 × (average latency of the last 10 tasks / 200ms).
[0072] GPU inference queue score = 0.7 × (current memory usage) + 0.3 × (P99 latency / 800ms).
[0073] I / O thread pool score = 0.5 × (average response latency / preset threshold 100ms) + 0.5 × (error rate in the last minute).
[0074] All scores are normalized to the [0,1] range, with higher scores indicating heavier loads. The preset upper threshold is 0.8, the lower threshold is 0.6, and the preset duration is 30 seconds.
[0075] S105) Fine-grained degradation process
[0076] Suppose that at a certain moment, the load score of the GPU inference queue reaches 0.865, exceeding the upper threshold of 0.8. The system triggers a degradation process for this resource pool, while the CPU thread pool and I / O thread pool remain unaffected. The degradation steps are as follows:
[0077] First, non-critical modules are selected from the module list of the GPU inference queue. In this embodiment, the jailbreak attack detection module is a critical module, while the harmful content filtering module is a non-critical module; therefore, the candidate modules only include the harmful content filtering module.
[0078] Next, calculate the overall weight of each candidate module. The normalized resource cost of the harmful content filtering module is 0.85, the priority is 3, and the weight W = 0.85 × (1 - 3 / 5) = 0.85 × 0.4 = 0.34. Since there is only one candidate module, the downgrade sequence is this module.
[0079] Then, a downgrade action is performed. The harmful content filtering module's replaceability is set to true, so the system switches the module to the corresponding distillation mini-model. After the switch, the module's inference time decreases from an average of 500ms to 150ms, and its memory usage decreases from 6GB to 1.5GB.
[0080] After the downgrade, the system recalculated the GPU inference queue load score. Memory utilization decreased from 85% to 70%, P99 latency decreased from 720ms to 400ms, and the new score was approximately 0.72. This score was still above the safety watermark of 0.7, but since it was below the upper threshold of 0.8 and there were no other candidate modules, the downgrade process was terminated. The system service quality declined smoothly, sacrificing only some accuracy of the harmful content filtering module, while the critical jailbreak attack detection module maintained full accuracy throughout.
[0081] S106) Independent Recovery Process
[0082] After approximately 30 seconds, due to a decrease in traffic, the GPU inference queue's memory utilization further dropped to 55%, P99 latency decreased to 300ms, and the load score dropped to 0.58. This low-load state persisted for more than 30 seconds. The system then triggered a recovery process for the GPU inference queue.
[0083] The recovery steps are as follows: All downgraded modules in the resource pool are retrieved. In this embodiment, only the harmful content filtering module is in the distillation model state. The system sorts these modules in descending priority and ascending resource cost, attempting to recover them. The recovery action involves switching the distilled small model back to the original large model. After switching back, the system enters a 5-second monitoring window to continuously observe the load score. Assuming that after recovery, the video memory usage rises to 60%, the P99 latency rises to 350ms, and the load score rises to 0.62, which does not exceed the safety score of 0.65, the recovery is confirmed to be effective. If the load score rebounds sharply to above 0.75 after recovery, the system will switch back to the distillation model and terminate this round of recovery.
[0084] As can be seen from the above embodiments, the method of the present invention achieves precise degradation of resource pool isolation, fair scheduling with module cost awareness, smooth degradation of each module, and rapid recovery of resource pools independently, effectively solving the problems of global coupling, coarse-grained degradation, and rigid recovery in the prior art.
[0085] Example 2
[0086] Please see Figure 4 The diagram shows a structural schematic of an electronic device provided in an embodiment of this specification.
[0087] like Figure 4As shown, the electronic device 1100 may include: at least one processor 1101, at least one network interface 1104, a user interface 1103, a memory 1105, and at least one communication bus 1102. The communication bus 1102 can be used to connect and communicate with the various components mentioned above. The user interface 1103 may include buttons, and optionally may include standard wired or wireless interfaces. The network interface 1104 may include, but is not limited to, a Bluetooth module, an NFC module, or a WiFi module. The processor 1101 may include one or more processing cores. The processor 1101 connects to various parts within the electronic device 1100 using various interfaces and lines, and performs various functions of the routing device and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1105, and by calling data stored in the memory 1105. Optionally, the processor 1101 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 1101 may integrate one or more combinations of CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content that the display screen needs to show; and the modem is used for wireless communication.
[0088] It is understandable that the aforementioned modem may not be integrated into the processor 1101, but may be implemented using a separate chip.
[0089] The memory 1105 may include RAM or ROM. Optionally, the memory 1105 may include a non-transitory computer-readable medium. The memory 1105 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1105 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 1105 may also be at least one storage device located remotely from the aforementioned processor 1101. As a computer storage medium, the memory 1105 may include an operating system, a network communication module, a user interface module, and application programs. The processor 1101 may be used to call the application programs stored in the memory 1105 and execute the methods in the various embodiments described above.
[0090] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform multiple steps as described in the above embodiments. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.
[0091] This specification also provides a computer program product, including a computer program that, when executed by a processor, implements the multiple steps described in the above embodiments.
[0092] Where there is no conflict, the technical features in this embodiment and implementation scheme can be combined arbitrarily.
[0093] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes multiple computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center integrating multiple available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).
[0094] When implemented through hardware or firmware, the aforementioned method flow is programmed into the hardware circuit to obtain the corresponding hardware circuit structure and achieve the corresponding function. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit, whose logic function is determined by the user programming the device. Designers can program a digital system onto a PLD themselves, eliminating the need for chip manufacturers to design and fabricate dedicated integrated circuit chips. Furthermore, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, similar to the software compiler used in program development. The original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There is not just one HDL, but many. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of the aforementioned hardware description languages and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.
[0095] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.
Claims
1. A fine-grained dynamic scheduling method for large-scale firewalls, characterized in that, Including the following steps: Define a unified abstract detection interface, with each security module of the large model serving as its implementation class. The static attributes of the abstract detection interface record the scheduling information of the security module. Three types of execution resource pools are constructed, including a CPU thread pool, a GPU inference queue, and an asynchronous I / O thread pool. The firewall's security modules are associated with at least one execution resource pool based on the scheduling information. Deploy module performance probes to collect operational metrics after each execution of the security module and calculate the real-time load of the execution resource pool. When the real-time load of any execution resource pool exceeds a preset upper threshold, the security module in that execution resource pool is downgraded according to the static attributes. When the real-time load of any execution resource pool falls below the lower threshold and remains below the preset duration, the degraded security module within that execution resource pool is attempted to be restored based on the static attributes.
2. The fine-grained dynamic scheduling method for a large-model firewall according to claim 1, characterized in that, The scheduling information of the security module includes resource type, priority, whether it can be skipped, and whether it can be replaced. The resource type is CPU-intensive, GPU-intensive, or I / O-intensive. The method of associating a firewall's security module with at least one execution resource pool based on the scheduling information includes: Security modules with resource types of CPU intensive, GPU intensive, or I / O intensive are associated with the CPU thread pool, GPU inference queue, and asynchronous I / O thread pool, respectively.
3. The fine-grained dynamic scheduling method for a large-model firewall according to claim 1, characterized in that, The methods for constructing three types of execution resource pools include: The execution resources for performing computational tasks are allocated to the CPU thread pool; The execution resources for large model calls are allocated to the GPU inference queue; Execution resources that perform non-blocking operations are assigned to the asynchronous I / O thread pool.
4. The fine-grained dynamic scheduling method for a large-model firewall according to claim 1, characterized in that, The operational metrics include execution time, resource consumption, and number of timeouts / failures. Methods for calculating the real-time load of the execution resource pool include: CPU thread pool score_cpu = 0.6 × (queue length / maximum allowed queue) + 0.4 × (average latency / 200ms); GPU inference queue score_gpu = 0.7 × (memory usage) + 0.3 × (P99 latency / 800ms); The asynchronous I / O thread pool score is calculated as follows: score_io = 0.5 × (response latency / preset latency threshold) + 0.5 × error rate.
5. The fine-grained dynamic scheduling method for a large-model firewall according to claim 4, characterized in that, The method for initiating a downgrade of the security module within the execution resource pool based on the static attributes includes: The normalized resource overhead score (cost) of the security module is calculated based on execution time and resource consumption. Calculate the weight W of each security module, W = cost × (1 - priority / 5), where priority is the priority of the security module, and the value range is [1, 5]. The security modules are arranged in descending order of weight W, and the degradation is initiated one by one in the order of arrangement until the real-time load of the execution resource pool is lower than the upper threshold and reaches the preset range.
6. The fine-grained dynamic scheduling method for a large-model firewall according to claim 2, characterized in that, Methods to initiate a downgrade include: When the condition "whether it can be replaced" is true, switch to the corresponding distillation mini-model; If the statement "Can be skipped?" is true, execution is skipped; if the statement "Can be skipped?" is false, the priority of the security module is reduced until it is attempted to be restored.
7. A fine-grained dynamic scheduling system for a large-scale firewall, characterized in that, include: The interface module defines a unified abstract detection interface, with each security module of the large model serving as its implementation class. The static attributes of the abstract detection interface record the scheduling information of the security module. The module constructs three types of execution resource pools, including a CPU thread pool, a GPU inference queue, and an asynchronous I / O thread pool. The association module associates the firewall's security modules with at least one execution resource pool based on the scheduling information. The probe module deploys performance probes to collect operational metrics after each execution of the security module and calculates the real-time load of the execution resource pool. The scheduling module, when the real-time load of any execution resource pool exceeds a preset upper threshold, initiates a downgrade of the security module in that execution resource pool based on the static attributes. When the real-time load of any execution resource pool falls below the lower threshold and remains below the preset duration, it attempts to restore the downgraded security module in that execution resource pool based on the static attributes.
8. An electronic device, characterized in that, Including the processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.