A vulnerability patching method and system

Abstract

The application provides a vulnerability repair method and system, and relates to the technical field of network security vulnerability verification and enterprise vulnerability management. The application obtains a task containing a vulnerability identifier, asset information and to-be-verified repair content, processes the task in parallel according to a pipeline split according to business execution steps, and configures a resource pool matching external dependencies for each stage; a stage fingerprint is generated when the task is transferred, a result is reused when a subtask is executed, otherwise the execution is scheduled according to vulnerability harm, asset importance and threat intelligence, and the vulnerability repair is completed according to the verification result. The application solves the problems of low throughput of the existing general pipeline, slow response of high-priority tasks, and easy starvation of low-priority tasks, and achieves the effect of collaborative optimization of throughput improvement, emergency task response acceleration and low-priority task starvation prevention without changing the original business and hardware.

Description

Technical Field

[0001] This application belongs to the field of network security vulnerability verification and enterprise vulnerability management technology, and in particular relates to a vulnerability repair method and system. Background Technology

[0002] In enterprise vulnerability governance scenarios, pipeline architectures are often used to process batch vulnerability verification tasks in parallel to improve throughput. Most existing pipeline solutions directly apply general data processing frameworks, breaking down the verification process into multiple stages for parallel execution. However, these solutions are not adapted to the inherent characteristics of vulnerability verification scenarios. The vulnerability verification process involves access steps to external dependencies such as publicly available vulnerability sources and large-scale model inference services, which are significantly more time-consuming than other local processing steps. Furthermore, these external services themselves have rate limiting or concurrency constraints. The common practice in general pipelines of simply categorizing resources by computational or I / O intensity can easily lead to insufficient resource utilization at bottleneck stages or trigger rate limiting in external services. Regarding cache reuse, conventional pipelines can only reuse results that have already been executed and written to the cache. When a large number of tasks with the same vulnerability identifier and input parameters are submitted in a concentrated manner, the cached results are often not yet generated, and the results of currently executing homogeneous tasks cannot be directly utilized, resulting in a large amount of redundant computation at bottleneck stages. Regarding priority scheduling, general scheduling strategies only raise task priority based on task waiting time or fixed weights. When handling high-priority emergency tasks, it is often necessary to wait for low-priority tasks to release resources before re-executing the time-consuming bottleneck stage, resulting in response delays for high-priority tasks and wasted computing resources. In addition, low-priority tasks are prone to not getting an execution opportunity for a long time under continuous impact from high-priority tasks. Summary of the Invention

[0003] This application provides a vulnerability remediation method and system. By splitting the vulnerability verification business steps into pipelines for parallel processing, it configures execution resource pools that match external dependencies for each stage, and combines a stage fingerprint prediction and reuse mechanism with a multi-dimensional scenario-based priority scheduling strategy to process vulnerability verification tasks. Without modifying the original business logic or adding hardware, it simultaneously achieves the overall synergistic effect of improving system throughput, accelerating emergency task response, and preventing low-priority tasks from starving.

[0004] The first aspect of this application provides a vulnerability remediation method, including: Obtain multiple vulnerability verification tasks, each of which includes a vulnerability identifier, asset information, and content to be verified and remedied. Each vulnerability verification task is processed in parallel along a pipeline. The pipeline stages are derived from the execution steps of the vulnerability verification business. The processing includes: configuring an execution resource pool for each stage that matches the external dependency constraints of the corresponding stage; when each task flows to at least one stage in the pipeline, generating a stage fingerprint that reflects the uniqueness of the input parameters of the current stage; if the stage fingerprint matches the fingerprint in the cache of completed stage results or the fingerprint of the sub-task of the same stage being executed, the matched stage output result is directly reused; otherwise, the task is placed in the execution resource pool of that stage and scheduled for execution according to the scheduling priority determined by at least one of the vulnerability severity level, asset importance, and threat intelligence to obtain the stage output result; and performing vulnerability remediation operations on the corresponding assets based on the verification results output by the final stage of the pipeline.

[0005] Optionally, the pipeline stages include multiple stages such as input parsing, baseline loading, evidence acquisition, root cause analysis, semantic alignment, verification calculation, result verification, and output rendering; the at least one stage includes at least the evidence acquisition stage and the root cause analysis stage, which have the highest time consumption.

[0006] Optionally, the generation of a stage fingerprint reflecting the uniqueness of the input parameters of the current stage, if the stage fingerprint matches the fingerprint in the cache of completed stage results or the fingerprint of the subtask of the same stage being executed, then the matched stage output result is directly reused, including: Generate a stage fingerprint based on all input parameters for the current stage; The stage fingerprint query has completed the stage result cache. If a match is found, the stage output result in the cache is read. If no match is found, check if there is a matching subtask in the list of subtasks currently being executed. If there is a matching subtask, add the current task to the waiting list of that subtask and reuse its output after it is completed.

[0007] Optionally, the stage fingerprint is a hash digest generated based on the input parameters of the current stage. Different combinations of input parameters are used to generate it for different stages, including at least some parameters from vulnerability identifier, asset identifier, patch content hash, evidence source synchronization timestamp, large model version number, rule engine version number, and verification threshold configuration.

[0008] Optionally, if a subtask with the same fingerprint is being executed, the current task is added to the waiting list of the subtask being executed, and its output is reused directly after the subtask is completed. This further includes: If the execution of a subtask fails, the current task is put back into the execution resource pool of the corresponding stage and preempted for execution with the scheduling priority of the current task.

[0009] Optionally, the scheduling execution based on a scheduling priority determined by at least one of the vulnerability severity level, asset importance, and threat intelligence includes: Static base priority is determined based on vulnerability severity level, asset importance level, and task submitter weight; Priority is dynamically adjusted based on intelligence usage markers, mission deadlines, mission waiting times, and mission types. The scheduling priority of a task is obtained by superimposing the static base priority and the dynamic adjustment priority, and then the task is scheduled in the execution resource pool according to the scheduling priority.

[0010] Optionally, scheduling in the execution resource pool according to the scheduling priority includes: For tasks with the same scheduling priority, the task with more pipeline stages completed will be scheduled first. Set the upper limit for the execution resource pool for the highest priority tasks, and reserve the minimum guaranteed resource quota for low priority tasks; When a high-priority task enters the execution resource pool, the scheduling priority of a low-priority task with the same vulnerability identifier as the high-priority task is temporarily raised to the same level as the high-priority task for bundled scheduling.

[0011] Optionally, the stages of the pipeline can pass read-only stage output results between them via shared memory, and different stage subtasks can reference these read-only results via pointers.

[0012] Optionally, the parallel processing in the pipeline also includes: For batch tasks submitted in the same batch, the outputs of multiple tasks are merged into a single repair report at the final stage of the pipeline.

[0013] A second aspect of this application provides a vulnerability remediation system, comprising: The task acquisition module is used to acquire multiple vulnerability verification tasks. Each vulnerability verification task includes a vulnerability identifier, asset information, and content to be verified and repaired. The pipeline processing module is used to process each vulnerability verification task in parallel according to the pipeline. The pipeline stages are obtained by splitting the execution steps of the vulnerability verification business. The pipeline processing module includes: a resource configuration unit, used to configure an execution resource pool for each stage that matches the external dependency constraints of the corresponding stage; a fingerprint matching unit, used to generate a stage fingerprint reflecting the uniqueness of the input parameters of the current stage when each task flows to at least one stage in the pipeline. If the stage fingerprint matches the fingerprint in the cache of the results of the completed stage or the fingerprint of the sub-task of the same stage that is being executed, the matched stage output result is directly reused; otherwise, the task is placed into the execution resource pool of that stage; and a priority scheduling unit, used to schedule and execute the tasks placed into the execution resource pool according to the scheduling priority determined by at least one of the vulnerability severity level, asset importance, and threat intelligence, to obtain the stage output result. The repair execution module is used to perform vulnerability repair operations on the corresponding assets based on the verification results output at the end of the pipeline.

[0014] This application obtains multiple vulnerability verification tasks, processes each task in parallel according to a pipeline, and the pipeline stages are derived from the execution steps of the vulnerability verification business. This approach aligns with the inherent process of vulnerability verification at the architecture level, avoiding the additional scheduling overhead caused by excessive splitting.

[0015] Specifically, by configuring execution resource pools that match external dependency constraints for each stage, the concurrency of stages that require access to external vulnerability sources, such as evidence acquisition, is limited by the rate limiting threshold of the vulnerability source. The concurrency of stages that call large model services, such as root cause analysis, is limited by the concurrency limit of the model service. Resources for other local computing or lightweight I / O stages are configured according to processor capabilities. This maximizes the resource utilization and throughput of bottleneck stages within the limits allowed by external dependencies, avoiding rate limiting errors caused by resource over-provisioning or idle waiting caused by insufficient resources.

[0016] When each task flows to at least one stage, a stage fingerprint reflecting the uniqueness of the input parameters of the current stage is generated. The fingerprint is then queried first to see if it matches a completed result cache or a subtask of the same stage that is currently being executed. If a match is found, its output result is directly reused. This allows multiple tasks with the same vulnerability identifier, the same evidence source timestamp, and the same model version to avoid independently and repeatedly executing time-consuming evidence acquisition or root cause analysis steps, regardless of whether they are submitted at the same time or whether there are currently tasks executing the same stage. This significantly reduces redundant calculations at bottleneck stages and reduces the number of external service requests.

[0017] For tasks without reusable results, scheduling is based on priorities determined by vulnerability severity, asset importance, and threat intelligence. This ensures that critical vulnerability tasks with in-the-wild exploitation or impact on core assets receive priority access to bottleneck execution resources. Furthermore, by associating high-priority tasks with executing results sharing the same fingerprint from low-priority tasks, high-priority tasks can reuse their results immediately upon completion of the low-priority, homogeneous task, without suspending the low-priority task and re-executing the entire bottleneck phase. This reduces the bottleneck phase time of high-priority tasks to hundreds of milliseconds with minimal increase in low-priority task latency, effectively shortening the average processing time for emergency tasks. Simultaneously, the task waiting time included in the priority dimension continuously raises the scheduling priority of long-backlogged low-priority tasks, preventing them from starving.

[0018] The aforementioned resource matching, predictive reuse, and scenario-based priority scheduling work together in the same pipeline process, achieving the overall effects of increased throughput, accelerated emergency task response, and prevention of starvation for low-priority tasks without modifying the original vulnerability verification business logic or adding additional hardware. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of a vulnerability remediation method in an embodiment of this application; Figure 2 This is a further flowchart of S200 of a vulnerability repair method in an embodiment of this application; Figure 3 This is another further flowchart of S200 of a vulnerability repair method in an embodiment of this application; Figure 4 This is a further flowchart of S230 of a vulnerability repair method in an embodiment of this application; Figure 5 This is a structural block diagram of a vulnerability repair system according to an embodiment of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not limiting, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without such specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] Existing pipeline solutions typically apply general data processing frameworks, breaking down execution steps and configuring resources according to general computing rules. The resource configuration of these frameworks usually only categorizes tasks as compute-intensive or I / O-intensive based on their load type, then configures the number of threads according to the number of processor cores or memory size. This configuration approach fails to consider the numerous external dependencies present in vulnerability verification scenarios. In vulnerability verification, the execution speed of many stages is not limited by the local processor or memory, but rather by the concurrency limits or rate-limiting thresholds of external services. Configuring resources according to the general framework approach can easily lead to a mismatch between the resource configuration of bottleneck stages and the processing capacity of external services. For example, configuring too many threads for stages that need to access publicly known vulnerability sources can trigger the vulnerability source's rate-limiting mechanism, causing many tasks to fail and reducing overall throughput. Conversely, configuring too few threads will result in underutilization of the processing capacity of external services, leaving system resources idle.

[0023] Furthermore, existing caching mechanisms typically only reuse results that have already been executed and written to the cache. When a large number of tasks with identical input parameters are submitted simultaneously, after the first task begins execution, subsequent tasks arrive before the first task's result has been generated and written to the cache. Therefore, subsequent tasks cannot access the cache and must independently execute the same computational steps. This situation is very common in vulnerability verification scenarios. For example, when a new N-day vulnerability is exposed (where N represents the number of days since the vulnerability was publicly disclosed and the official vendor has released a patch), the enterprise will simultaneously submit a large number of verification tasks targeting this vulnerability. These tasks share the same vulnerability identifier, the same evidence source synchronization timestamp, and the same large model version number, resulting in identical input parameters during the evidence acquisition and root cause analysis phases. According to existing caching mechanisms, these tasks will independently execute the evidence acquisition and root cause analysis steps, leading to a large amount of redundant computation and external service requests.

[0024] Furthermore, existing priority scheduling strategies typically employ preemptive scheduling when handling high-priority emergency tasks. When a high-priority task arrives, if all execution resources are occupied by low-priority tasks, the scheduler suspends a low-priority task, releasing resources for the high-priority task to execute. While this allows high-priority tasks to begin execution quickly, it wastes the computing resources already consumed by the suspended low-priority tasks. Moreover, high-priority tasks need to re-execute the entire bottleneck phase, resulting in a still lengthy processing time. In vulnerability verification scenarios, high-priority emergency tasks and low-priority batch tasks often share the same vulnerability identifiers and input parameters, requiring them to perform identical evidence acquisition and root cause analysis steps. If preemptive scheduling is used, high-priority tasks need to re-execute steps already partially executed by low-priority tasks, wasting already consumed resources and extending the processing time of high-priority tasks.

[0025] In response to the problems existing in the current technology, such as Figure 1 As shown, this application provides a vulnerability remediation method, including: S100: Obtain multiple vulnerability verification tasks, each of which includes a vulnerability identifier, asset information, and content to be verified and remedied. S200: Process each vulnerability verification task in parallel according to the pipeline. The pipeline stages are obtained by splitting the execution steps of the vulnerability verification business. The processing includes: configuring an execution resource pool for each stage that matches the external dependency constraints of the corresponding stage; when each task flows to at least one stage in the pipeline, generating a stage fingerprint that reflects the uniqueness of the input parameters of the current stage; if the stage fingerprint matches the fingerprint in the cache of the completed stage results or the fingerprint of the sub-task of the same stage that is being executed, then the matched stage output result is directly reused; otherwise, the task is put into the execution resource pool of that stage and scheduled for execution according to the scheduling priority determined by at least one of the vulnerability severity level, asset importance, and threat intelligence to obtain the stage output result. S300: Based on the verification results output at the end of the pipeline, perform vulnerability remediation operations on the corresponding assets.

[0026] In this embodiment, multiple vulnerability verification tasks are first obtained. A vulnerability verification task is a task that verifies the effectiveness of a fix for a specific vulnerability on a specific asset. Each vulnerability verification task includes a vulnerability identifier, asset information, and the fix to be verified. The vulnerability identifier uniquely identifies a vulnerability, the asset information identifies the target device or system to be verified, and the fix to be verified refers to patches or remediation measures already applied to the target asset. Methods for obtaining vulnerability verification tasks may include pulling from a task queue, receiving task requests submitted by an external system, or periodically generating batch inspection tasks. This application is not limited to these methods; any method capable of obtaining vulnerability verification tasks can be applied to this application.

[0027] In this embodiment, vulnerability verification tasks are processed in parallel using a pipeline. Pipeline parallel processing refers to breaking down a complete processing flow into multiple consecutive stages, each handled by an independent execution unit. Different stages of different tasks can be executed simultaneously. The pipeline stages are derived based on the execution steps of the vulnerability verification process. The principle of this breakdown is to strictly follow the original execution order of vulnerability verification, dividing the complete verification process into multiple independent, side-effect-free execution units. Each execution unit has clearly defined inputs and outputs; the output of the previous execution unit serves as the input of the next, forming a linear execution flow. This breakdown method does not require modification of the original business logic; only stage tagging of the existing business code is needed to complete the transformation.

[0028] In this embodiment, an execution resource pool matching the external dependency constraints of the corresponding stage is configured for each stage. The execution resource pool contains a certain number of execution threads and task queues for executing subtasks of the corresponding stage. The execution resource pools of different stages are independent and do not affect each other. External dependency constraints refer to the limitations of external services that need to be accessed during the execution of that stage, including the concurrency limit, rate limiting threshold, and response time of the external services. Configuring an execution resource pool matching the external dependency constraints for each stage ensures that the execution speed of each stage matches the processing capacity of the external services, avoiding rate limiting of external services due to resource over-provisioning or system idleness due to insufficient resources. For example, for a stage that needs to access publicly available vulnerability sources, the number of threads in its execution resource pool cannot exceed the rate limiting threshold of the publicly available vulnerability source; otherwise, the rate limiting mechanism of the vulnerability source will be triggered, causing task execution failure. For a stage that needs to call a large model service, the number of threads in its execution resource pool cannot exceed the concurrency limit of the large model service; otherwise, the large model service will be overloaded, and the response time will increase.

[0029] In this embodiment, when each task flows to at least one stage in the pipeline, a stage fingerprint reflecting the uniqueness of the input parameters of the current stage is generated. A stage fingerprint is a unique feature summary generated based on all variable input parameters of the current stage. If two tasks have identical input parameters in the same stage, their stage fingerprints will be identical, and the output of that stage will also be identical. The purpose of generating stage fingerprints is to identify reusable data between different tasks and avoid repeatedly executing the same computational steps.

[0030] In this embodiment, if a stage fingerprint matches the fingerprint in the completed stage result cache or the fingerprint of a subtask of the same stage that is currently being executed, the matched stage output result is directly reused. The completed stage result cache stores the output results of completed stage subtasks. When a new task enters a stage, the completed stage result cache is first queried. If a record with the same stage fingerprint as the task exists, the output result in the cache is read directly without executing the subtask of the current stage. If no matching record exists in the completed stage result cache, the list of subtasks currently being executed is queried to see if a subtask with the same stage fingerprint as the task is being executed. If so, the current task is added to the waiting list of the executing subtask, and its output result is reused directly after the executing subtask is completed. This mechanism can reuse not only completed results but also results that are being generated, solving the cold start problem of the general caching mechanism when submitting in the same batch of tasks.

[0031] In this embodiment, if the stage fingerprint does not match the fingerprint in the cache of completed stage results or the fingerprint of the currently executing subtask of the same stage, the task is placed in the execution resource pool of that stage and scheduled for execution based on a scheduling priority determined by at least one of the vulnerability severity level, asset importance, and threat intelligence, to obtain the stage output result. The scheduling priority is used to determine the execution order of tasks in the execution resource pool. The calculation of the scheduling priority comprehensively considers the severity of the vulnerability itself, the importance of the affected assets, and threat intelligence information such as whether there is in-the-wild exploitation. In this way, it can be ensured that vulnerability verification tasks with high severity, impact on important assets, and potential in-the-wild exploitation can obtain execution resources first, shortening processing time.

[0032] In this embodiment, vulnerability remediation operations are performed on the corresponding assets based on the verification results output at the end of the pipeline. The verification results indicate whether the remediation content effectively fixes the target vulnerability. If the verification results show that the remediation is effective, no additional operations are required. If the verification results show that the remediation is ineffective, further vulnerability remediation operations need to be performed on the corresponding assets, such as reapplying patches, updating system configurations, and deploying security policies.

[0033] In this application embodiment, addressing the resource configuration deficiencies in existing technologies, this application configures execution resource pools that match the external dependency constraints of each stage. This ensures that the execution speed of each stage precisely matches the processing capacity of external services, maximizing resource utilization and overall throughput at bottleneck stages without triggering external service rate limiting. Furthermore, addressing the cache reuse deficiencies in existing technologies, this application extends the reusable scope to currently executing subtasks. This allows subsequent submissions of the same task to directly reuse the results being generated by the first task without re-execution, significantly reducing redundant computations at bottleneck stages and decreasing the number of requests to external services. Simultaneously, addressing the priority scheduling deficiencies in existing technologies, this application employs a predictive reuse mechanism. When a high-priority task arrives, it first checks if a low-priority task with the same fingerprint is already executing. If so, the high-priority task does not need to preempt resources; it simply adds itself to a waiting list, waiting for the low-priority task to complete before directly reusing its results. This approach reduces the time spent on bottleneck stages of high-priority tasks to the time spent waiting for low-priority tasks to complete, without increasing the latency of low-priority tasks by almost nothing. This significantly shortens the average processing time of emergency tasks and avoids resource waste.

[0034] It should be noted that the vulnerability remediation method and system involved in this application are applicable to network security vulnerability verification and enterprise vulnerability governance scenarios, and can handle various task types such as batch vulnerability inspection and emergency vulnerability verification. The implementation process of this application does not require modification of the original vulnerability verification business logic; all modifications are performed at the scheduling layer and will not affect the stability of the existing system. All functions of this application can be implemented on a single node without introducing additional middleware or hardware devices, resulting in low deployment costs.

[0035] In an optional embodiment, the pipeline stages include multiple stages such as input parsing, baseline loading, evidence acquisition, root cause analysis, semantic alignment, verification computation, result verification, and output rendering; the at least one stage includes at least the evidence acquisition stage and the root cause analysis stage, which have the highest time consumption.

[0036] In this embodiment, the input parsing stage verifies the legality of the input vulnerability identifier, asset information, and content to be verified for remediation, and formats them into a unified structure. Specifically, the baseline loading stage retrieves official authoritative baseline data for the corresponding vulnerability, including root cause, impact scope, and remediation standards; the evidence acquisition stage retrieves remediation evidence for the vulnerability from public vulnerability sources or internal code repositories, including official patches, commit records, announcements, and exploit intelligence; the root cause analysis stage generates a structured root cause representation and remediation pattern labels based on the baseline and evidence; the semantic alignment stage extracts semantic features between the content to be verified for remediation and the official remediation patch, completing semantic matching; the verification calculation stage calculates indicators such as root cause coverage, dangerous path coverage, and boundary condition coverage based on the alignment results, and outputs preliminary verification conclusions; the result verification stage verifies the consistency of the verification conclusions and the reachability of referenced links, and downgrades the results if they do not meet the requirements; and the output rendering stage formats and outputs the verification report as required.

[0037] In this application's embodiments, at least one stage includes the evidence acquisition stage and the root cause analysis stage, which have the highest time consumption. In the vulnerability verification process, the evidence acquisition stage and the root cause analysis stage account for the highest proportion of the total process time. Specifically, the evidence acquisition stage requires access to external vulnerability sources, resulting in high network latency and unstable response times from these sources; the root cause analysis stage requires calling large model services or rule engines for complex calculations and inferences, leading to long processing times. These two stages are the bottlenecks of the entire pipeline, determining the overall system throughput and response time. Therefore, the predictive reuse mechanism and priority scheduling mechanism of this application primarily optimize these two stages to maximize system performance. Other stages have shorter time consumption and less impact on overall performance; therefore, simple first-in-first-out scheduling is sufficient, without the need for complex priority arbitration to avoid introducing unnecessary scheduling overhead.

[0038] It should be noted that existing pipeline stage decomposition techniques typically aim to maximize parallelism by breaking down the process into the finest possible granularity. However, excessively fine decomposition can increase switching and scheduling overhead between stages, ultimately reducing overall performance. This application's stage decomposition strictly follows the original business execution order of the vulnerability verification, dividing the process into eight stages, each representing the smallest native execution unit of the business logic. This approach ensures sufficient parallelism while avoiding the additional overhead of excessive decomposition. The business logic of each stage is completely consistent with the original serial logic, requiring no modification to any business code. Only stage tagging and flow control at the scheduling layer are needed, achieving a zero-business-intrusion transformation.

[0039] In an optional embodiment, such as Figure 2As shown, the generation of a stage fingerprint that reflects the uniqueness of the input parameters of the current stage, if the stage fingerprint matches the fingerprint in the cache of completed stage results or the fingerprint of the subtask of the same stage being executed, then the matched stage output result is directly reused, including: S201: Generate a stage fingerprint based on all input parameters of the current stage; S202: Use the stage fingerprint to query the completed stage result cache. If a match is found, read the stage output result from the cache. S203: If no match is found, check if there is an executing subtask with the same fingerprint in the list of subtasks currently being executed. If there is an executing subtask with the same fingerprint, add the current task to the waiting list of the executing subtask and reuse its output result directly after the executing subtask is completed.

[0040] In this embodiment, a stage fingerprint is generated based on all input parameters of the current stage. Variable input parameters refer to all input parameters that affect the stage output. The set of variable input parameters differs for different stages. When generating a stage fingerprint, all variable input parameters of the current stage need to be concatenated into a string in a fixed order, and then hashed to obtain the stage fingerprint. To ensure the uniqueness of the stage fingerprint, it must include all variable input parameters that may affect the stage output; none can be omitted. If a variable input parameter is omitted, two tasks with different input parameters may generate the same stage fingerprint, thus reusing incorrect results and leading to incorrect verification conclusions.

[0041] In this embodiment, the completed stage result cache is queried using the stage fingerprint. The completed stage result cache is a global key-value pair storage structure, where the key is the stage fingerprint and the value is the corresponding stage output result. The completed stage result cache can be implemented using a memory cache to improve query speed. When a new task enters a certain stage, the stage fingerprint of the task is first calculated, and then the completed stage result cache is queried using that stage fingerprint as the key. If a corresponding record is found, the stage output result in the cache is read directly, without needing to execute the subtask of the current stage. After reading the cached result, this result is used as the output of the current stage and pushed to the execution queue of the next stage.

[0042] In this embodiment, if the completed stage result cache is not found, the system queries the list of currently executing subtasks to see if there is an executing subtask with a matching fingerprint. The list of currently executing subtasks is a global list used to record information about all currently executing subtasks, including the subtask's stage fingerprint, execution status, and waiting list. When an executing subtask with a matching fingerprint is found, the current task is added to the waiting list of that executing subtask. The waiting list is a linked list structure used to store all tasks waiting for the subtask to complete. When the executing subtask completes, its output is written to the completed stage result cache, and then the waiting list is traversed, sending the result to all waiting tasks. Upon receiving the result, the waiting task uses it as the output of the current stage and pushes it to the execution queue of the next stage.

[0043] It should be noted that in existing technologies, caching is typically designed based on the principle of read-after-write, meaning that subsequent tasks can only reuse the result after a task has completed execution and written its result to the cache. This design can effectively reduce redundant computation in scenarios with long task submission intervals. However, in scenarios where tasks are submitted in the same batch, after the first task starts executing, subsequent tasks arrive before the first task's result has been generated. Therefore, subsequent tasks cannot hit the cache and must execute independently. This situation is very common in vulnerability verification scenarios. For example, when a new N-day vulnerability is exposed, enterprises may submit hundreds or even thousands of verification tasks targeting that vulnerability in a short period of time. These tasks have essentially the same input parameters, so they need to perform the exact same computational steps in the evidence acquisition and root cause analysis phases. According to existing caching mechanisms, these tasks will all execute independently, resulting in a large amount of redundant computation and external service requests, which seriously affects the system's throughput and response time. This application solves the cache cold start problem when tasks are submitted in the same batch by maintaining a list of executing subtasks and allowing subsequent tasks to be added to a waiting list to reuse the results being generated. This mechanism allows the results of the same subtask to be reused regardless of whether tasks are submitted at the same time, as long as their input parameters are the same. This significantly reduces redundant computations in the bottleneck stage and improves the overall performance of the system.

[0044] Furthermore, in existing technologies, when multiple tasks need to perform the same computational steps, task merging is typically used to combine multiple identical tasks into one task for execution, and then the result is distributed to all requesters. However, task merging requires merging judgment at task submission and necessitates maintaining the state of the merged task and the result distribution mechanism, resulting in high implementation complexity. Moreover, task merging is generally only applicable to tasks submitted simultaneously; it cannot merge tasks submitted sequentially. The predictive reuse mechanism of this application does not require merging judgment at task submission, but rather performs fingerprint matching when the task enters each stage. Regardless of whether tasks are submitted simultaneously or sequentially, as long as their stage fingerprints are the same, the result of the currently executing subtask can be reused. This approach is simple to implement and can cover all possible reuse scenarios, including tasks submitted sequentially.

[0045] In an optional embodiment, the stage fingerprint is a hash digest generated based on the input parameters of the current stage. Different combinations of input parameters are used to generate it for different stages, including at least some parameters from vulnerability identifier, asset identifier, patch content hash, evidence source synchronization timestamp, large model version number, rule engine version number, and verification threshold configuration.

[0046] In this embodiment, the stage fingerprint is a hash digest generated based on the input parameters of the current stage. A hash digest is unique and irreversible, capable of converting an input of arbitrary length into a fixed-length output. As long as the input parameters are the same, the generated hash digests will be the same. If the input parameters are different, the probability of generating the same hash digest is extremely low and can be ignored. Therefore, using a hash digest as a stage fingerprint can effectively identify reusable data between different tasks.

[0047] In this embodiment, different combinations of input parameters are used to generate stage fingerprints for different stages. Each stage's input parameter combination includes all variable input parameters that may affect the output result of that stage. For example, for the input parsing stage, its output is determined by the vulnerability identifier, asset identifier, and the content to be verified for repair; therefore, the stage fingerprint is generated by these three parameters. For the baseline loading stage, its output is determined by the vulnerability identifier and the local vulnerability database version number; therefore, the stage fingerprint is generated by these two parameters. For the evidence acquisition stage, its output is determined by the vulnerability identifier, evidence source synchronization timestamp, and pull strategy; therefore, the stage fingerprint is generated by these three parameters. For the root cause analysis stage, its output is determined by the baseline hash, evidence set hash, and analysis engine version number; therefore, the stage fingerprint is generated by these three parameters. For the semantic alignment stage, its output is determined by the root cause hash and the hash of the content to be verified for repair; therefore, the stage fingerprint is generated by these two parameters. For the verification calculation stage, its output is determined by the alignment result hash and the verification threshold configuration; therefore, the stage fingerprint is generated by these two parameters. For the result verification stage, its output is determined by the preliminary conclusion hash and the evidence set hash; therefore, the stage fingerprint is generated by these two parameters. For the output rendering stage, the output result is determined by the final conclusion hash and the output format configuration, so the stage fingerprint is generated by these two parameters.

[0048] It is important to note that the generation of stage fingerprints must include all variable input parameters that may affect the output of that stage. Omitting any variable input parameter could lead to two tasks with different input parameters generating the same stage fingerprint, thus reusing erroneous results. For example, in the root cause analysis stage, if the analysis engine version number is omitted, the new analysis engine might generate different root cause results after an update. However, since the stage fingerprint does not include the analysis engine version number, results generated using the older version of the analysis engine could be reused by tasks using the newer version, leading to incorrect verification conclusions. Therefore, this application designs a dedicated combination of input parameters for each stage to ensure that all variable input parameters that may affect the output are included in the stage fingerprint generation process.

[0049] Furthermore, to further reduce the probability of hash collisions, a secondary verification step can be added during fingerprint matching. After a stage fingerprint matches successfully, several core business fields of the two tasks are compared, such as vulnerability identifiers and asset identifiers. If the core business fields also match, the reuse result is confirmed. If the core business fields do not match, a hash collision is considered to have occurred, and reuse is not performed. The overhead of the secondary verification is extremely low, almost negligible, but it can completely eliminate the risk caused by hash collisions.

[0050] In an optional embodiment, if there is an ongoing subtask with a matching fingerprint, the current task is added to the waiting list of the ongoing subtask, and its output is reused directly after the ongoing subtask completes. This further includes: If the execution of a subtask fails, the current task is put back into the execution resource pool of the corresponding stage and preempted for execution with the scheduling priority of the current task.

[0051] In this embodiment, if a subtask fails to execute, the current task is re-placed in the execution resource pool of the corresponding stage and preempted for execution according to its current scheduling priority. Execution failure means that an error occurred during the execution of the subtask, preventing the generation of correct output. Reasons for execution failure may include network failure, unavailability of external services, incorrect data format, etc. When a subtask fails to execute, the scheduler traverses the waiting list of that subtask, re-placing all waiting tasks in the execution resource pool of the corresponding stage. These tasks will then re-participate in scheduling according to their respective scheduling priorities. Since higher-priority tasks have higher scheduling priority, they will obtain execution resources first and re-execute the subtask of that stage.

[0052] It should be noted that the predictive reuse mechanism of this application includes comprehensive fallback logic to handle situations where subtask execution fails. When a subtask fails, all tasks awaiting the result of that subtask are re-entered into the execution queue and rescheduled for execution according to their respective priorities. This approach ensures that even if a subtask fails, it will not affect the normal execution of other tasks. In particular, high-priority tasks will receive execution resources first and be re-executed, preventing excessive processing delays due to the failure of low-priority tasks.

[0053] Furthermore, in existing technologies, when a task fails, typically only the task itself is re-executed, without affecting other tasks. However, in the predictive reuse mechanism of this application, multiple tasks may be waiting for the result of the same executing subtask. If that subtask fails, all waiting tasks need to be re-executed. Therefore, this application designs a dedicated failure handling mechanism that can put all waiting tasks back into the execution queue and reschedule them according to their respective priorities. This mechanism ensures the fault tolerance and reliability of the system.

[0054] In an optional embodiment, such as Figure 3 As shown, the scheduling execution based on a scheduling priority determined by at least one of the vulnerability severity level, asset importance, and threat intelligence includes: S210: Determine the static basic priority based on the vulnerability severity level, asset importance level, and task submitter weight; S220: Dynamically adjust priorities based on in-the-field intelligence markers, mission deadlines, mission waiting times, and mission types; S230: The scheduling priority of a task is obtained by superimposing the static basic priority and the dynamic adjustment priority, and the task is scheduled in the execution resource pool according to the scheduling priority.

[0055] In this embodiment, a static basic priority is determined based on the vulnerability severity level, asset importance level, and task submitter weight. The static basic priority is the task's fundamental priority, determined at the time of task submission and remaining largely unchanged throughout the task's lifecycle. The vulnerability severity level measures the inherent danger of the vulnerability and is typically scored using a general vulnerability scoring system. The asset importance level measures the importance of the affected asset to the enterprise's business, such as core business systems, important business systems, and general business systems. The task submitter weight measures the submitter's authority and urgency; for example, tasks submitted by an incident response team are usually more urgent than those submitted by ordinary developers. The static basic priority can be calculated using a weighted summation method, adding the vulnerability severity level, asset importance level, and task submitter weight according to a certain weight ratio to obtain the static basic priority.

[0056] In this embodiment, dynamic priority adjustment is determined based on in-the-wild exploitation intelligence markers, task deadlines, task waiting times, and task type. Dynamic priority adjustment changes dynamically during task execution to adapt to different business scenarios and needs. In-the-wild exploitation intelligence markers indicate whether a publicly available exploit tool or in-the-wild exploitation behavior exists. Vulnerabilities with in-the-wild exploitation typically require priority handling to prevent exploitation by attackers. Task deadlines indicate the time by which the task needs to be completed. Tasks closer to their deadlines have higher priority. Task waiting times indicate how long the task has been waiting in the queue. Tasks with longer waiting times have higher priority to prevent low-priority tasks from being unable to obtain execution opportunities for extended periods. Task type distinguishes between real-time tasks and batch tasks. Real-time tasks typically require priority handling, while batch tasks can be processed when the system is idle. The calculation of dynamic priority adjustment can be performed using an additive method, adding corresponding scores based on each dimension.

[0057] In this embodiment, the scheduling priority of a task is obtained by superimposing a static base priority and a dynamic adjustment priority, and then the task is scheduled in the execution resource pool according to the scheduling priority. The scheduling priority is the final execution priority of the task, which determines the execution order of the task in the execution resource pool. The task with the higher the scheduling priority, the earlier it obtains execution resources. By superimposing the static base priority and the dynamic adjustment priority, the inherent attributes of the task and the dynamic changes can be comprehensively considered, achieving a more reasonable scheduling.

[0058] It should be noted that existing priority scheduling technologies typically employ only simple fixed-weight or first-in-first-out (FIFO) methods. Fixed-weight methods cannot adapt to dynamically changing business needs; for example, they cannot handle urgent vulnerabilities exploited in the wild, nor can they prevent low-priority tasks from being starved indefinitely. FIFO methods completely disregard the importance and urgency of tasks, leading to high-priority tasks being blocked by low-priority tasks, failing to meet emergency response requirements. This application's multi-dimensional priority calculation rule comprehensively considers factors such as vulnerability severity, asset importance, submitter's responsibilities, in-the-wild exploit intelligence, deadlines, waiting times, and task type, thus more comprehensively reflecting the urgency and importance of tasks. In particular, the introduction of the task waiting time dimension can continuously raise the scheduling priority of long-backlogged low-priority tasks, ensuring that all tasks eventually get an execution opportunity and avoiding the starvation problem of low-priority tasks.

[0059] Furthermore, existing multi-dimensional priority scheduling technologies typically perform complex priority arbitration across all stages, introducing significant scheduling overhead. In vulnerability verification scenarios, most stages are short-duration, resulting in high scheduling overhead and ultimately reducing overall system performance. This application implements complex priority arbitration only in the two bottleneck stages with the highest time consumption: evidence acquisition and root cause analysis. Other stages employ simple first-in-first-out scheduling. This approach ensures that high-priority tasks receive priority access to execution resources in bottleneck stages while avoiding unnecessary scheduling overhead in non-bottleneck stages, keeping single-task scheduling overhead extremely low.

[0060] In an optional embodiment, such as Figure 4 As shown, the scheduling based on the scheduling priority in the execution resource pool includes: S2301: For tasks with the same scheduling priority, prioritize scheduling the task that has completed more pipeline stages. S2302: Set the upper limit of the execution resource pool for the highest priority tasks, and reserve the minimum guaranteed resource quota for low priority tasks; S2303: When a high-priority task enters the execution resource pool, the scheduling priority of a low-priority task with the same vulnerability identifier as the high-priority task is temporarily raised to the same level as the high-priority task for bundled scheduling.

[0061] In this embodiment, for tasks with the same scheduling priority, the task that has completed more pipeline stages is given priority in scheduling. When multiple tasks with the same scheduling priority are waiting to be executed in the execution resource pool, the scheduler will prioritize the task that has completed more pipeline stages. This is because tasks that have completed more stages have already consumed certain computing resources, and if they are preempted by newly entered tasks of the same priority, the consumed resources will be wasted. By prioritizing the scheduling of tasks that have completed more stages, resource waste can be avoided and the overall efficiency of the system can be improved.

[0062] In this embodiment, an upper limit is set for the execution resource pool usage of the highest priority tasks, while a minimum guaranteed resource quota is reserved for low-priority tasks. Although the highest priority tasks require priority processing, they cannot occupy all execution resources; otherwise, low-priority tasks would be completely denied execution opportunities, resulting in starvation. Therefore, this application sets an upper limit for the execution resource pool usage of the highest priority tasks; for example, the highest priority tasks can occupy a maximum of 70% of the execution resource pool slots. Simultaneously, a minimum guaranteed resource quota is reserved for low-priority tasks; for example, at least 10% of the slots are reserved for medium- and low-priority tasks. When there are no high-priority tasks, the reserved slots can be temporarily allocated to low-priority tasks to improve resource utilization. When a high-priority task arrives, the reserved slots are reclaimed for executing the high-priority task.

[0063] In this embodiment, when a high-priority task enters the execution resource pool, the scheduling priority of low-priority tasks with the same vulnerability identifier as the high-priority task is temporarily increased to the same level as the high-priority task for bundled scheduling. When a high-priority task enters the execution resource pool, the scheduler queries whether there are any low-priority tasks with the same vulnerability identifier as the high-priority task. If so, the scheduling priority of these low-priority tasks is temporarily increased to the same level as the high-priority task, allowing them to be executed together with the high-priority task. The purpose of this is to maximize the cache hit rate and avoid repeatedly fetching evidence and root causes of the same vulnerability. Because these low-priority tasks have the same vulnerability identifier as the high-priority task, their input parameters are basically the same in the evidence acquisition and root cause analysis stages, allowing them to reuse the results of the high-priority task. Through bundled scheduling, these low-priority tasks can be executed simultaneously with the high-priority task, so that after the high-priority task completes, the low-priority task can directly reuse its results without re-execution, further reducing redundant computation.

[0064] It should be noted that existing priority scheduling technologies typically employ an absolute priority approach, where high-priority tasks can preempt all execution resources, while low-priority tasks can only execute when no high-priority tasks are available. This approach can lead to low-priority tasks being unable to secure execution opportunities for extended periods due to continuous pressure from high-priority tasks, resulting in a severe starvation problem. This application addresses this issue by setting a resource cap for the highest-priority tasks and reserving a minimum guaranteed resource quota for low-priority tasks, ensuring that low-priority tasks always have a chance to obtain execution resources and avoiding the starvation problem. Furthermore, the resource quota is dynamically adjusted; when no high-priority tasks are available, the reserved resources can be allocated to low-priority tasks, improving resource utilization.

[0065] Furthermore, existing scheduling technologies typically do not consider data relationships between different tasks. When high-priority and low-priority tasks have the same vulnerability identifier, they need to perform the exact same evidence gathering and root cause analysis steps. Under conventional priority scheduling, high-priority tasks execute first, and low-priority tasks execute after the high-priority tasks have completed. While this allows low-priority tasks to reuse the results of high-priority tasks, it also prolongs their processing time. This application addresses this by using a scheduling rule that binds vulnerability identifiers to temporarily elevate the priority of low-priority tasks with the same vulnerability identifiers as high-priority tasks, allowing them to execute concurrently. This way, once the high-priority task completes the evidence gathering and root cause analysis phases, the low-priority task can immediately reuse its results without waiting for the high-priority task to complete the entire process, thus shortening the processing time of low-priority tasks and further reducing redundant computations.

[0066] In an optional embodiment, read-only stage output results are passed between stages of the pipeline via shared memory, and different stage subtasks reference these read-only results via pointers. In this embodiment, the stage output is read-only after generation and will not be modified by any stage. Therefore, the stage output can be stored in a shared memory area, and subtasks in different stages can reference the result through pointers, eliminating the need for memory copying. This zero-copy transfer method can significantly reduce the data transfer overhead between stages and improve the overall system performance.

[0067] It should be noted that existing pipeline stages typically use message queues or memory copies to transfer data. Message queues require serializing data before sending it to the queue, and then deserializing it at the receiver, resulting in significant overhead. Memory copies require copying data from one memory region to another, which also incurs substantial overhead when dealing with large amounts of data. In vulnerability verification scenarios, the output data from each stage can be substantial; for example, evidence sets may contain numerous patch files and announcements. Using memory copies to transfer data would consume significant memory bandwidth and processor resources, impacting overall system performance. This application achieves zero-copy data transfer through shared memory and pointer references, requiring only the transfer of a pointer and eliminating the need to copy the data itself, thus reducing data transfer overhead to an extremely low level.

[0068] Furthermore, since the output of each stage is read-only and cannot be modified by any stage, there is no issue of concurrent access, and no locking protection is required. This further improves the system's concurrency performance and stability.

[0069] In an optional embodiment, the pipelined parallel processing further includes: For batch tasks submitted in the same batch, the outputs of multiple tasks are merged into a single repair report at the final stage of the pipeline.

[0070] In this embodiment, batch tasks submitted in the same batch typically have the same vulnerability identifier or the same asset group, and their verification reports will contain a lot of identical content, such as vulnerability descriptions and remediation suggestions. Generating a separate report for each task would result in a large amount of duplicate content being written, increasing I / O overhead. By merging the outputs of batch tasks in the same batch into a single remediation report, the identical content only needs to be written once, significantly reducing I / O operations and improving report generation efficiency.

[0071] It should be noted that batch rendering is an optional optimization step and will not affect the core functionality of the system. Users can choose whether to enable this feature based on their actual needs. When this feature is enabled, the scheduler tracks all tasks in the same batch, and once all tasks in the batch are completed, merges their verification results into a single report. When this feature is disabled, each task generates a separate report.

[0072] To further illustrate the specific implementation of this application, an example is taken of a company's vulnerability management platform handling a newly emerging N-day vulnerability, illustrating the complete execution flow of this application. The security operations team submitted one thousand batch inspection tasks for non-core assets, while the incident response team submitted ten verification tasks for core business systems. After receiving these tasks, the task acquisition module submits them to the pipeline processing module. The pipeline processing module first calculates the scheduling priority for each task. The batch inspection tasks have a medium-risk vulnerability, a general asset importance, are submitted by the security operations team, have no in-the-wild exploit intelligence, and have no deadline; therefore, their static base priority is low, their dynamic adjustment priority is zero, and their overall scheduling priority is low. The incident verification tasks have a severe vulnerability, a core asset importance, are submitted by the incident response team, and have in-the-wild exploit intelligence; therefore, their static base priority is high, their dynamic adjustment priority is high, and their overall scheduling priority is much higher than that of the batch inspection tasks.

[0073] The pipelined processing module processes all tasks in parallel. Batch inspection tasks first enter the input parsing phase, then the baseline loading phase, and finally the evidence acquisition phase. Since the evidence acquisition phase is the bottleneck, batch inspection tasks are queued in the evidence acquisition phase's execution resource pool. When emergency verification tasks are submitted, they first enter the input parsing and baseline loading phases, which use a first-in, first-out scheduling method, resulting in fast execution. Emergency verification tasks quickly complete the input parsing and baseline loading phases and then enter the evidence acquisition phase.

[0074] The fingerprint matching unit calculates a phase fingerprint for the evidence acquisition phase of each emergency verification task. Since the emergency verification tasks and batch inspection tasks share the same vulnerability identifier, evidence source synchronization timestamp, and retrieval strategy, their phase fingerprints are identical. The fingerprint matching unit queries the cache of completed phase results but finds no matching record. It then queries the list of subtasks currently executing in the evidence acquisition phase and discovers a batch inspection task subtask currently executing with the same phase fingerprint as the emergency verification task. Therefore, the fingerprint matching unit adds all emergency verification tasks to the waiting list of this executing subtask.

[0075] Once the subtasks of the batch inspection task are completed, the results are written to the completed stage result cache, and then all emergency verification tasks on the waiting list are notified. The emergency verification tasks directly reuse this result, skipping the evidence acquisition stage and entering the root cause analysis stage. In the root cause analysis stage, the same process occurs again. The stage fingerprints of the emergency verification tasks are the same as those of the batch inspection tasks, therefore they can reuse the results of the root cause analysis subtasks being executed by the batch inspection tasks.

[0076] Ultimately, the emergency verification task completed all stages of processing within a very short time and generated verification results. Based on the verification results, the remediation execution module performed corresponding vulnerability remediation operations on the core business system. The batch inspection task also completed processing within the normal timeframe and generated verification results. Throughout the entire process, the emergency verification task did not preempt the execution resources of the batch inspection task, nor did it waste already consumed computing resources. Simultaneously, the batch inspection task did not experience starvation issues, and all tasks were completed within a reasonable timeframe.

[0077] Based on the vulnerability remediation method described in the above embodiments, this application also provides a vulnerability remediation system, such as... Figure 5 As shown, it includes: The task acquisition module is used to acquire multiple vulnerability verification tasks. Each vulnerability verification task includes a vulnerability identifier, asset information, and content to be verified and repaired. The pipeline processing module is used to process each vulnerability verification task in parallel according to the pipeline. The pipeline stages are obtained by splitting the execution steps of the vulnerability verification business. The pipeline processing module includes: a resource configuration unit, used to configure an execution resource pool for each stage that matches the external dependency constraints of the corresponding stage; a fingerprint matching unit, used to generate a stage fingerprint reflecting the uniqueness of the input parameters of the current stage when each task flows to at least one stage in the pipeline. If the stage fingerprint matches the fingerprint in the cache of the results of the completed stage or the fingerprint of the sub-task of the same stage that is being executed, the matched stage output result is directly reused; otherwise, the task is placed into the execution resource pool of that stage; and a priority scheduling unit, used to schedule and execute the tasks placed into the execution resource pool according to the scheduling priority determined by at least one of the vulnerability severity level, asset importance, and threat intelligence, to obtain the stage output result. The repair execution module is used to perform vulnerability repair operations on the corresponding assets based on the verification results output at the end of the pipeline.

[0078] In this embodiment, the vulnerability remediation system includes a task acquisition module, a pipeline processing module, and a remediation execution module. The task acquisition module can pull tasks from a task queue, receive task requests submitted by external systems, and periodically generate batch inspection tasks.

[0079] In this embodiment, the pipeline processing module is used to process each vulnerability verification task in parallel according to the pipeline. The pipeline processing module includes a resource configuration unit, a fingerprint matching unit, and a priority scheduling unit. The resource configuration unit is used to configure an execution resource pool that matches the external dependency constraints of the corresponding stage for each stage. The resource configuration unit can dynamically adjust the parameters of the execution resource pool, such as the number of core threads and the queue length, according to the external dependency constraints of each stage. The fingerprint matching unit is used to generate a stage fingerprint that reflects the uniqueness of the input parameters of the current stage when each task flows to at least one stage in the pipeline. If the stage fingerprint matches the fingerprint in the cache of the results of the completed stage or the fingerprint of the subtask of the same stage that is being executed, the matched stage output result is directly reused; otherwise, the task is placed into the execution resource pool of that stage. The priority scheduling unit is used to schedule and execute the tasks placed into the execution resource pool according to the scheduling priority determined by at least one of the vulnerability severity level, asset importance, and threat intelligence to obtain the stage output result.

[0080] In this embodiment, the remediation execution module performs vulnerability remediation operations on the corresponding assets based on the verification results output at the end of the pipeline. If the verification results show that the remediation is effective, the remediation execution module does not need to perform any operations. If the verification results show that the remediation is ineffective, the remediation execution module will perform further remediation operations on the corresponding assets according to the vulnerability type and remediation recommendations, such as reapplying patches, updating system configurations, and deploying security policies.

[0081] It should be noted that the vulnerability remediation system in this application adopts a modular design with clear interfaces between modules, facilitating maintenance and expansion. Each module can be upgraded and replaced independently without affecting the functionality of other modules. For example, if the caching implementation needs to be changed, only the caching part in the fingerprint matching unit needs to be modified, without modifying the code of other modules.

[0082] It should be noted that the technical solution in this application is designed specifically for the problems of vulnerability verification scenarios, and is not a simple application of general technologies. General pipeline frameworks are designed to handle common data processing tasks, such as log analysis and data cleaning. These tasks are characterized by large data volumes, relatively simple processing logic, and few external dependencies. However, vulnerability verification scenarios are characterized by large fluctuations in task volume, complex processing logic, numerous external dependencies, and strict requirements on task priority. General pipeline frameworks cannot adapt to these unique characteristics, thus leading to various problems when applied to vulnerability verification scenarios. This application, through in-depth analysis of the business characteristics and load features of vulnerability verification scenarios, designs scenario-specific stage splitting rules, resource configuration methods, predictive reuse mechanisms, and priority scheduling rules, solving the problems of low throughput, slow response of high-priority tasks, starvation of low-priority tasks, and low resource utilization that exist in general pipeline frameworks in vulnerability verification scenarios.

[0083] Those skilled in the art will understand that the technical solution of this application can be modified and varied in various ways according to actual needs. For example, the stage splitting can be adjusted according to the business processes of different enterprises, the weight of priority calculation can be adjusted according to the security policies of different enterprises, and the caching implementation method can be selected according to different performance requirements. These modifications and variations all fall within the protection scope of this application and will not be elaborated upon here.

[0084] In this embodiment, the execution resource pool can be implemented in various ways. For example, it can be implemented using the thread pool built into the programming language, or a custom thread pool can be used. The parameters of the thread pool include the core thread count, maximum thread count, idle thread lifespan, and task queue type. The core thread count is the number of threads that are always maintained in the thread pool, even when these threads are idle. The maximum thread count is the maximum number of threads allowed in the thread pool. When the task queue is full, the thread pool will create new threads to process tasks until the number of threads reaches the maximum thread count. The idle thread lifespan refers to the maximum time an idle thread waits for a new task before terminating when the number of threads exceeds the core thread count. The task queue is used to store tasks waiting to be executed and can be a bounded queue or an unbounded queue. The length of a bounded queue is fixed; when the queue is full, new tasks will be rejected or other processing strategies will be triggered. The length of an unbounded queue is infinite and will not reject new tasks, but may lead to memory overflow.

[0085] In this embodiment, the parameter configuration of the execution resource pool differs for different stages. For lightly I / O-intensive or lightly computationally intensive stages such as input parsing, baseline loading, verification calculation, and output rendering, the number of core threads can be configured to twice the number of processor cores, the maximum number of threads can be the same as the number of core threads, and the task queue can be a bounded queue with a queue length of 1024. This configuration ensures that these stages have enough threads to handle tasks while avoiding the creation of too many threads that would increase system overhead. For moderately computationally intensive or moderately I / O-intensive stages such as semantic alignment and result verification, the number of core threads can be configured to 1.5 times the number of processor cores, the maximum number of threads can be the same as the number of core threads, and the task queue can be a bounded queue with a queue length of 512. For high-latency I / O-intensive stages such as evidence acquisition, the number of core threads needs to be configured according to the rate limiting threshold of the external vulnerability source, the maximum number of threads can be the same as the number of core threads, and the task queue can be a bounded queue with a queue length of 256. For computationally intensive or API-intensive phases such as root cause analysis, the number of core threads needs to be configured based on the concurrency limit of the large model service. The maximum number of threads can be the same as the number of core threads. The task queue can be a bounded queue with a queue length of 128.

[0086] It's important to note that the execution resource pool parameters are not fixed and can be dynamically adjusted based on actual operational conditions. For example, the queue backlog and thread utilization of each stage can be sampled at regular intervals. If the queue backlog in a certain stage continues to increase, it indicates insufficient processing capacity for that stage, and the number of core threads can be appropriately increased. Conversely, if the thread utilization in a certain stage remains consistently low, it indicates excessive processing capacity for that stage, and the number of core threads can be appropriately reduced. Dynamically adjusting the execution resource pool parameters allows the system to better adapt to load changes and improves resource utilization.

[0087] In this embodiment, the caching of completed stage results can be implemented in various ways. For example, a memory cache or a distributed cache can be used. Memory caches are fast, but have limited capacity, and data is lost after a system restart. Distributed caches have large capacity and data can be persisted, but access speed is relatively slow. In a single-node deployment, a memory cache is sufficient. In a multi-node deployment, a distributed cache can be used to share cached data.

[0088] The cached results of completed phases can have an expiration time set to prevent incorrect verification conclusions due to expired cached data. For example, for the results cache of the evidence acquisition phase, the expiration time can be set to the same time interval as the evidence source synchronization interval. When the evidence source is updated, the old cached data will automatically expire, and the new task will fetch the latest evidence again. For the results cache of the root cause analysis phase, the expiration time can be set to the same time as the analysis engine version update cycle. When the analysis engine version is updated, the old cached data will automatically expire, and the new task will use the new analysis engine to regenerate the root cause results.

[0089] In this embodiment, the list of executing subtasks can be implemented using a hash table. The key of the hash table is a stage fingerprint, and the value is information about the executing subtask. When a subtask begins execution, its stage fingerprint and information are added to the hash table. When a subtask completes or fails, it is removed from the hash table. This method allows for quick lookups to find executing subtasks with the same fingerprint, with a constant time complexity.

[0090] A waiting list can be implemented using a doubly linked list. Each node contains a reference to a task and a pointer to the next node. When a task needs to wait for the result of an executing subtask, it is added to the end of the waiting list. When the executing subtask completes, the waiting list is traversed, and the result is sent to all waiting tasks. Doubly linked lists offer efficient insertion and deletion operations, making them suitable for implementing waiting lists.

[0091] In the embodiments of this application, scheduling priority can be calculated in various ways. For example, it can be calculated using a weighted summation method or a priority matrix method. The weighted summation method is simple, intuitive, and easy to implement. The priority matrix method allows for more flexible configuration of the relationships between different dimensions. This application is not limited to this; any method that can comprehensively consider multiple dimensions to calculate scheduling priority can be applied to this application.

[0092] The weighting of static base priorities can be adjusted based on the company's security strategy. For example, if the company prioritizes the severity of vulnerabilities, it can increase the weight of vulnerability severity level. If the company prioritizes the importance of assets, it can increase the weight of asset importance level. If the company prioritizes the tasks of the incident response team, it can increase the weight of submitter weight. The scoring rules for dynamically adjusting priorities can also be adjusted according to the company's actual needs. For example, different scoring values ​​can be set based on different levels of in-the-wild exploitation threats, different scoring values ​​can be set based on different deadlines, and different scoring increments can be set based on different task waiting times.

[0093] In this embodiment, the scheduling rule of binding vulnerability identifiers can be implemented by querying when a task enters the execution resource pool. When a high-priority task enters the execution resource pool, the scheduler traverses all waiting tasks in the pool to find low-priority tasks with the same vulnerability identifier as the high-priority task. If found, the scheduling priority of these low-priority tasks is temporarily raised to the same level as the high-priority task. After these low-priority tasks complete their execution, their scheduling priority is restored to its original level.

[0094] It should be noted that the scheduling rule bundled with vulnerability identifiers is an optional optimization rule and will not affect the core functionality of the system. Users can choose whether to enable this rule based on their actual needs. Enabling this rule can further improve the cache hit rate and reduce redundant calculations. When this rule is disabled, the system can still operate normally, but the cache hit rate will be reduced.

[0095] In this embodiment, the implementation of zero-copy transfer requires operating system support. Modern operating systems provide shared memory mechanisms, allowing different processes or threads to access the same memory region. In Java, the `allocateDirect` method of the `ByteBuffer` class can be used to allocate direct memory. Direct memory is not managed by the Java Virtual Machine heap and can be directly accessed by native code. In C++, the `mmap` system call can be used to map files or devices into the address space of a process, thus implementing shared memory.

[0096] Once generated, the output of each stage is marked as read-only, and any attempt to modify it will be rejected. This ensures the consistency and correctness of the stage output and avoids data errors caused by concurrent modifications. Subtasks in different stages reference this read-only result through pointers, eliminating the need for memory copying and achieving zero-copy transfer.

[0097] In this embodiment, batch merging rendering is implemented by tracking all tasks in the same batch. When a batch of tasks is submitted, a unique batch identifier is assigned to that batch. All tasks belonging to that batch carry this batch identifier. The scheduler maintains a batch status table, recording the total number of tasks and the number of completed tasks in each batch. When a task completes, the scheduler updates the number of completed tasks in the corresponding batch. When all tasks in a batch are completed, the scheduler calls the report generation module to merge the verification results of all tasks in that batch into a single repair report.

[0098] The generation of merged reports can be achieved using a template engine. The report template contains both fixed and variable content. Fixed content, such as vulnerability descriptions and remediation suggestions, only needs to be generated once. Variable content, such as verification results for each asset, requires generating corresponding content for each asset. By merging the variable content of tasks in the same batch into a single report template, the generation and writing of duplicate content can be significantly reduced, improving report generation efficiency.

[0099] In this application embodiment, the deployment method of the vulnerability remediation system can be flexible and diverse. It can be deployed in a single-node configuration or in a multi-node distributed configuration. In a single-node deployment, all modules run on the same server, suitable for small and medium-sized enterprises. In a multi-node distributed deployment, different modules can run on different servers and communicate via a network, suitable for large enterprises. Distributed deployment can improve system throughput and reliability; when one node fails, other nodes can take over its work without affecting the operation of the entire system.

[0100] In a distributed deployment, the results of completed stages can be cached using a distributed cache such as Redis. The list of executing subtasks can be implemented using a distributed coordination service such as ZooKeeper. Task queues can be implemented using message queues such as Kafka. This enables cache sharing, task scheduling, and state synchronization across different nodes.

[0101] It should be noted that the technical solution of this application still maintains its advantages in a distributed deployment. The predictive reuse mechanism can achieve cross-node result reuse through distributed caching and distributed coordination services. The priority scheduling mechanism can be executed independently on each node, or it can achieve unified scheduling across nodes through a global scheduler. The resource configuration mechanism can be configured independently based on the hardware configuration and external dependency constraints of each node, or it can be configured uniformly through a global configuration center.

[0102] The technical solution presented in this application boasts excellent scalability. As business grows and the number of vulnerabilities increases, the system's throughput can be linearly scaled by adding server nodes. No modification to the core business logic is required; only additional deployment nodes need to be added. This scalability effectively meets the ever-growing vulnerability management needs of enterprises.

[0103] Furthermore, the technical solution presented in this application has low modification costs and is easy to implement. All modifications are performed at the scheduling layer, without requiring changes to the original vulnerability verification business logic. Enterprises can gradually apply the technical solution of this application to their existing systems without affecting their operation. The modification cycle is short, and the results are quick, enabling rapid improvement of an enterprise's vulnerability management capabilities.

[0104] In this embodiment, the system's monitoring and alarm functions are also crucial. By monitoring metrics such as queue backlog, thread utilization, task execution time, cache hit rate, and external service call success rate at each stage, the system's operational status can be understood in real time. When a metric exceeds a preset threshold, the system will issue an alarm to notify operations and maintenance personnel for timely handling. For example, if the queue backlog in the evidence acquisition stage continues to increase, it may indicate a fault or rate limiting issue with the external vulnerability source, requiring prompt investigation. Similarly, a low cache hit rate may indicate a problem with the stage fingerprint generation rules, requiring adjustment.

[0105] Monitoring data can be stored in a time-series database, such as Prometheus. Visualization tools, such as Grafana, can be used to display the monitoring data in chart form, making it easy for operations and maintenance personnel to view and analyze. Alerts can be sent to operations and maintenance personnel via email, SMS, instant messaging tools, etc.

[0106] In this embodiment, the system's logging function is also essential. It can record the execution process of each task, the input and output of each stage, scheduling decisions, error messages, etc. Logging helps developers and operations personnel troubleshoot problems, analyze system performance, and optimize system configuration. Logs can be divided into different levels, such as DEBUG, INFO, WARN, ERROR, etc. In a production environment, the log level can be set to INFO or WARN to record only important information. In development and testing environments, the log level can be set to DEBUG to record detailed debugging information.

[0107] Logs can be stored in local files or in a distributed logging system, such as the ELK Stack. Distributed logging systems can centrally collect, store, and analyze logs from multiple nodes, facilitating troubleshooting in distributed deployments.

[0108] In this application embodiment, system security is also a crucial factor to consider. The vulnerability verification system handles sensitive enterprise information, such as asset information, vulnerability information, and remediation information. If this information is leaked, it could pose serious security risks to the enterprise. Therefore, a series of security measures are needed to protect the system. For example, identity authentication and authorization control should be implemented for system access, allowing only authorized users to access the system. Transmitted data should be encrypted to prevent theft during transmission. Stored data should be encrypted to prevent leakage during storage. Regular security scans and vulnerability assessments should be performed on the system to promptly identify and remediate security vulnerabilities.

[0109] Furthermore, system reliability and fault tolerance are also crucial. A series of measures are needed to ensure the system's continuous and stable operation. For example, redundant deployment of critical components can be implemented to avoid single points of failure. Tasks should be persistently stored so that unfinished tasks can be recovered after a system restart. A task retry mechanism should be implemented to automatically retry tasks when they fail, until successful or the maximum number of retries is reached. An automatic system recovery mechanism should be implemented to automatically restore the system to a normal state when a failure occurs.

[0110] In this embodiment, system performance testing and stress testing are also essential. Before the system goes live, thorough performance and stress testing are required to verify whether the system's throughput, response time, resource utilization, and other indicators meet the design requirements. Performance testing can simulate normal business loads to test the system's performance under normal conditions. Stress testing can simulate conditions exceeding normal business loads to test the system's extreme performance and stability. Through performance and stress testing, performance bottlenecks and potential problems in the system can be identified, allowing for timely optimization and adjustments.

[0111] Performance and stress testing can be implemented using professional testing tools such as JMeter and LoadRunner. These tools can simulate varying numbers of concurrent users and submit different numbers of vulnerability verification tasks to test the system's performance. They can record system metrics such as throughput, average response time, maximum response time, and resource utilization, and compare these metrics with design requirements. If any metric is found to be unsatisfactory, the cause needs to be analyzed and optimization implemented.

[0112] For example, during stress testing, if the queue backlog during the evidence acquisition phase continues to increase, it indicates insufficient processing capacity at this stage. You can check if the rate limiting thresholds for external vulnerability sources are set correctly and if the core thread count of the execution resource pool is configured appropriately. If the rate limiting thresholds for external vulnerability sources have some leeway, you can appropriately increase the core thread count of the execution resource pool. If the rate limiting thresholds for external vulnerability sources have reached their limit, you can consider increasing the number of external vulnerability sources or optimizing the evidence acquisition logic to reduce the number of requests to external vulnerability sources.

[0113] For example, in stress testing, if a low cache hit rate is found, it indicates that the predictive reuse mechanism is not functioning as intended. It's possible to check if the stage fingerprint generation rules are correct and if any variable input parameters are missing. If the stage fingerprint generation rules are correct, it's possible to check if the maintenance of the list of currently executing subtasks is normal and if any subtasks are not promptly removed from the list after completion.

[0114] Through thorough performance and stress testing, we can ensure that the system can run stably and efficiently after going live, meeting the vulnerability management needs of enterprises.

[0115] In this application embodiment, system upgrades and maintenance are also important factors to consider. With the continuous development of vulnerability verification technology and the evolving business needs of enterprises, the system requires constant upgrades and maintenance. Therefore, the system design should possess good maintainability and upgradeability. Modular design makes system upgrades and maintenance more convenient. Each module can be upgraded and replaced independently without affecting the functionality of other modules. For example, if it is necessary to upgrade the interface of the large model service, only the business logic and resource configuration of the root cause analysis phase need to be modified, without modifying the code of other modules.

[0116] Furthermore, version management is crucial. Each system version needs to be recorded and managed, including version number, release date, updates, and known issues. When system problems arise, the version number can be used to quickly locate the problem, allowing for repairs and rollbacks.

[0117] In this embodiment, user training and technical support are also crucial for the successful implementation of the system. System training is required for the company's security operations personnel, emergency response personnel, and R&D personnel to ensure they understand the system's functions and operating methods. Timely and effective technical support is essential to help users resolve any issues encountered during use. Technical support can be provided through online documents, video tutorials, technical support hotlines, email, and other means.

[0118] In summary, the vulnerability remediation method and system of this application, through a scenario-specific pipeline architecture, a predictive reuse mechanism, and a multi-dimensional priority scheduling mechanism, solves the problems of low throughput, slow response of high-priority tasks, starvation of low-priority tasks, and low resource utilization existing technologies. The technical solution of this application has low modification costs and implementation difficulty, good scalability and maintainability, and can meet the needs of enterprise-level batch vulnerability inspection and emergency vulnerability verification. The technical solution of this application is not a simple application of general technologies, but rather an in-depth analysis and innovative design targeting the specific problems of vulnerability verification scenarios, possessing significant technical advantages and practical value.

Claims

1. A method of patching a vulnerability, the method comprising: include: Obtain multiple vulnerability verification tasks, each of which includes a vulnerability identifier, asset information, and content to be verified and remedied. Each vulnerability verification task is processed in parallel along a pipeline. The pipeline stages are derived from the execution steps of the vulnerability verification business. The processing includes: configuring an execution resource pool for each stage that matches the external dependency constraints of the corresponding stage; when each task flows to at least one stage in the pipeline, generating a stage fingerprint that reflects the uniqueness of the input parameters of the current stage; if the stage fingerprint matches the fingerprint in the cache of the results of the completed stage or the fingerprint of the subtask of the same stage that is being executed, the matched stage output result is directly reused; otherwise, the task is placed in the execution resource pool of that stage and scheduled for execution according to the scheduling priority determined by at least one of the vulnerability severity level, asset importance, and threat intelligence to obtain the stage output result. Based on the verification results output at the end of the pipeline, perform vulnerability remediation operations on the corresponding assets; The generation of a stage fingerprint that reflects the uniqueness of the input parameters for the current stage, if the stage fingerprint matches a fingerprint in the cache of completed stage results or a fingerprint of a subtask of the same stage being executed, then the matched stage output result is directly reused, including: Generate a stage fingerprint based on all input parameters for the current stage; The stage fingerprint query has completed the stage result cache. If a match is found, the stage output result in the cache is read. If no match is found, check if there is a matching subtask in the list of subtasks currently being executed. If there is a matching subtask, add the current task to the waiting list of that subtask and reuse its output result after the subtask is completed. The stage fingerprint is a hash digest generated based on the input parameters of the current stage. Different combinations of input parameters are used to generate it for different stages. It includes at least some parameters from the vulnerability identifier, asset identifier, patch content hash, evidence source synchronization timestamp, large model version number, rule engine version number, and verification threshold configuration. If a subtask with the same fingerprint is being executed, the current task is added to the waiting list of that subtask. Once the subtask is completed, its output is reused. This also includes: If the execution of a subtask fails, the current task is put back into the execution resource pool of the corresponding stage and preempted for execution with the scheduling priority of the current task.

2. The method of claim 1, wherein, The pipeline stages include multiple stages such as input parsing, baseline loading, evidence acquisition, root cause analysis, semantic alignment, verification calculation, result verification, and output rendering; the at least one stage includes at least the evidence acquisition stage and the root cause analysis stage, which have the highest time consumption.

3. The method of claim 1, wherein, The scheduling execution is based on a priority determined by at least one of the following: vulnerability severity level, asset importance, and threat intelligence: Static base priority is determined based on vulnerability severity level, asset importance level, and task submitter weight; Priority is dynamically adjusted based on intelligence usage markers, mission deadlines, mission waiting times, and mission types. The scheduling priority of a task is obtained by superimposing the static base priority and the dynamic adjustment priority, and then the task is scheduled in the execution resource pool according to the scheduling priority.

4. The method of claim 3, wherein, The step of scheduling in the execution resource pool according to the scheduling priority includes: For tasks with the same scheduling priority, the task with more pipeline stages completed will be scheduled first. Set the upper limit for the execution resource pool for the highest priority tasks, and reserve the minimum guaranteed resource quota for low priority tasks; When a high-priority task enters the execution resource pool, the scheduling priority of a low-priority task with the same vulnerability identifier as the high-priority task is temporarily raised to the same level as the high-priority task for bundled scheduling.

5. The method according to any one of claims 1 to 4, characterized in that, The stages of the pipeline pass read-only stage output results to each other through shared memory, and different stage subtasks reference the read-only stage output results through pointers.

6. The method according to any one of claims 1 to 4, characterized in that, The parallel processing in the pipeline also includes: For batch tasks submitted in the same batch, the outputs of multiple tasks are merged into a single repair report at the final stage of the pipeline.

7. A vulnerability remediation system, characterized by, include: The task acquisition module is used to acquire multiple vulnerability verification tasks. Each vulnerability verification task includes a vulnerability identifier, asset information, and content to be verified and repaired. The pipeline processing module is used to process each vulnerability verification task in parallel according to the pipeline. The pipeline stages are obtained by splitting the execution steps of the vulnerability verification business. The pipeline processing module includes: a resource configuration unit, used to configure an execution resource pool for each stage that matches the external dependency constraints of the corresponding stage; a fingerprint matching unit, used to generate a stage fingerprint reflecting the uniqueness of the input parameters of the current stage when each task flows to at least one stage in the pipeline. If the stage fingerprint matches the fingerprint in the cache of the results of the completed stage or the fingerprint of the sub-task of the same stage that is being executed, the matched stage output result is directly reused; otherwise, the task is placed into the execution resource pool of that stage; and a priority scheduling unit, used to schedule and execute the tasks placed into the execution resource pool according to the scheduling priority determined by at least one of the vulnerability severity level, asset importance, and threat intelligence, to obtain the stage output result. The generation of a stage fingerprint that reflects the uniqueness of the input parameters for the current stage, if the stage fingerprint matches a fingerprint in the cache of completed stage results or a fingerprint of a subtask of the same stage being executed, then the matched stage output result is directly reused, including: Generate a stage fingerprint based on all input parameters for the current stage; The stage fingerprint query has completed the stage result cache. If a match is found, the stage output result in the cache is read. If no match is found, check if there is a matching subtask in the list of subtasks currently being executed. If there is a matching subtask, add the current task to the waiting list of that subtask and reuse its output result after the subtask is completed. The stage fingerprint is a hash digest generated based on the input parameters of the current stage. Different combinations of input parameters are used to generate it for different stages. It includes at least some parameters from the vulnerability identifier, asset identifier, patch content hash, evidence source synchronization timestamp, large model version number, rule engine version number, and verification threshold configuration. If a subtask with the same fingerprint is being executed, the current task is added to the waiting list of that subtask. Once the subtask is completed, its output is reused. This also includes: If the execution of a subtask fails, the current task is put back into the execution resource pool of the corresponding stage and preempted for execution with the scheduling priority of the current task. The repair execution module is used to perform vulnerability repair operations on the corresponding assets based on the verification results output at the end of the pipeline.

Images