Vulnerability scanning task execution method and device, equipment and storage medium
By constructing a rule library to predict and manage interface conflicts between plugins, the method enhances vulnerability scan efficiency and accuracy by dynamically scheduling tasks to prevent crashes and conflicts in production environments.
Patent Information
- Application Number
- CN202510460003.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-14
- Publication Date
- 2025-07-15
AI Technical Summary
In the existing automated vulnerability scanning regression test, the plug-in execution order depends on manual configuration, and it is impossible to dynamically deal with resource conflicts in the target production environment, resulting in performance degradation and even crashes, inefficient scanning and mixed task conflicts.
By building a rule base to predict interface call conflicts, atomic task groups are generated, and high concurrency and low conflicts are automated task scheduling, including interface-level conflict prediction, atomic task combination and target state checking mechanisms.
It significantly improves scanning efficiency and accuracy, avoids service crashes and mixed task conflicts, and improves the stability and efficiency of task execution.
Smart Images
Figure CN120316009A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular, to a method, apparatus, device, and storage medium for executing vulnerability scanning tasks. Background Art
[0002] In the field of testing, automated vulnerability scanning regression testing has gradually replaced basic manual testing and become an emerging testing method. When creating a test task, multiple plugins can be selected, and each plugin corresponds to a vulnerability scanning test case. When the task is executed, these plugins are called plugin subtasks, and the plugin subtasks can be manually configured to be serial or parallel.
[0003] However, the execution order of the plugins in the current automated vulnerability scanning regression testing depends on manual configuration, and the parallel or serial strategy is assigned based on fixed rules, which cannot dynamically handle resource conflicts in the target production environment. As a result, it may lead to the problem of rigid task scheduling, where the performance of the target production environment may decline or even crash.
[0004] It can be seen that how to avoid the problems of service crashes, low scanning efficiency, and hybrid task conflicts in traditional solutions is a problem that needs to be solved by those skilled in the art. Summary of the Invention
[0005] This application provides a method, apparatus, device, and storage medium for executing vulnerability scanning tasks. Interface-level conflict prediction can be achieved through the constructed rule library, and pre-operation merging and target state checking mechanisms can be implemented through the constructed atomic task groups, enabling high-concurrency and low-conflict automated task scheduling, and significantly improving scanning efficiency and accuracy. It can at least solve the problems of service crashes, low scanning efficiency, and hybrid task conflicts in related technologies.
[0006] This application provides a method for executing a vulnerability scanning task, including:
[0007] Determine a number of plugins to be executed corresponding to the current vulnerability scanning task;
[0008] Construct a rule library for predicting interface call conflicts between the plugins to be executed; the interface call conflict is a call conflict generated when the pre-operations of the plugins to be executed make interface calls;
[0009] Classify and merge the pre-operations of the several plugins to be executed by using the interface call conflict prediction results obtained based on the current rule library, so as to generate each atomic task group;
[0010] Execute each vulnerability scanning subtask corresponding to each atomic task group to obtain a task execution result corresponding to the vulnerability scanning task.
[0011] This application also provides a device for executing a vulnerability scanning task, including:
[0012] A plugin determination module, configured to determine a number of to-be-executed plugins corresponding to the current vulnerability scanning task;
[0013] A rule library establishment unit, configured to construct a rule library for predicting interface call conflicts between the to-be-executed plugins; the interface call conflict is a call conflict generated when the pre-operations of the to-be-executed plugins make interface calls;
[0014] A task group generation unit, configured to classify and merge the pre-operations of the to-be-executed plugins by using the interface call conflict prediction results between the to-be-executed plugins obtained based on the current rule library, so as to generate each atomic task group;
[0015] A task execution unit, configured to execute each vulnerability scanning subtask corresponding to each atomic task group to obtain a task execution result corresponding to the vulnerability scanning task.
[0016] This application further provides an electronic device, including: a memory, configured to store a computer program;
[0017] A processor, configured to implement the steps of any one of the above vulnerability scanning task execution methods when executing the computer program.
[0018] This application further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the above vulnerability scanning task execution methods are implemented.
[0019] This application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of any one of the above vulnerability scanning task execution methods are implemented.
[0020] Thus, through the method of this application, the current vulnerability scanning task can be determined, and then a number of to-be-executed plugins corresponding to the current vulnerability scanning task can be determined. Since each plugin executes the corresponding vulnerability scanning subtask and may generate interface call conflicts when making interface calls, a rule library for predicting interface call conflicts between the to-be-executed plugins can be constructed, and the pre-operations of the to-be-executed plugins can be classified and merged by using the interface call conflict prediction results between the to-be-executed plugins obtained based on the current rule library, so as to generate different types of atomic task groups. Finally, by executing each vulnerability scanning subtask corresponding to each atomic task group, a task execution result corresponding to the vulnerability scanning task is obtained. In this way, through the constructed rule library to achieve interface-level conflict prediction, through the constructed atomic task groups to merge the pre-operations and the target status check mechanism, high-concurrency and low-conflict automated task scheduling can be realized, the problems of target service crash, low scanning efficiency and mixed task conflicts in the traditional solution are solved, and the scanning efficiency and accuracy are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.
[0022] Figure 1 It is a flowchart of a method for executing a vulnerability scanning task provided by an embodiment of the present application;
[0023] Figure 2 It is a flowchart of a specific method for executing a vulnerability scanning task provided by an embodiment of the present application;
[0024] Figure 3 It is a schematic diagram of an execution framework of a method for executing a vulnerability scanning task provided by an embodiment of the present application;
[0025] Figure 4 It is a schematic diagram of the structure of a device for executing a vulnerability scanning task provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0026] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present application.
[0027] It should be noted that in the description of the present application, the terms "including", "comprising" or any other variation thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such a process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence.
[0028] To enable those skilled in the art of the present technology to better understand the solutions of the present application, the following further details the present application with reference to the drawings and specific embodiments.
[0029] Combined with the specific application environment architecture or specific hardware architecture on which the execution of the vulnerability scanning task method depends, the specific application environment architecture or specific hardware architecture is described herein.
[0030] See Figure 1As shown in the figure, an embodiment of the present application provides a method for executing a vulnerability scanning task. In combination with the execution process of the vulnerability scanning task execution method, the method is described in detail. The method includes:
[0031] Step S11: Determine a plurality of to-be-executed plugins corresponding to the current vulnerability scanning task.
[0032] In this embodiment, when creating a test task for vulnerability scanning, multiple plugins can be selected. Each plugin corresponds to a vulnerability scanning use case. When the task is executed, these plugins are called plugin subtasks. Therefore, a plurality of to-be-executed plugins corresponding to the current vulnerability scanning task can be determined, so as to construct a rule library based on the plurality of to-be-executed plugins subsequently.
[0033] Step S12: Construct a rule library for predicting interface call conflicts between the to-be-executed plugins; the interface call conflict is a call conflict generated when the pre-operations of the to-be-executed plugins make interface calls.
[0034] In this embodiment, since each plugin executes the corresponding vulnerability scanning subtask and interface call conflicts may occur when calling interfaces, a rule library for predicting interface call conflicts between the to-be-executed plugins can be constructed. The specific process is as follows:
[0035] Step S121: Determine a plurality of to-be-executed plugin declaration files corresponding to the plurality of to-be-executed plugins.
[0036] Since each plugin corresponds to a to-be-executed plugin declaration file, that is, a YAML (YAML Ain't Markup Language) declaration file, and various data structures can be declared in the declaration file, including but not limited to lists, dictionaries, nested structures, etc. In this embodiment, each plugin needs to clearly mark its interface call declaration in the YAML declaration file, and each YAML declaration file contains preset core fields, such as interface path and method, operation type (read / write), request parameter definition, output product path, input dependency file, explicit label, whether exclusive interface is required, timeout threshold, interface parameters, etc.
[0037] Step S122: Parse the plurality of to-be-executed plugin declaration files by traversing the attribute information in the plurality of to-be-executed plugin declaration files to obtain metadata corresponding to each to-be-executed plugin declaration file.
[0038] After determining each plugin declaration file to be executed, a set of YAML declaration files can be obtained. Then, it is necessary to parse each YAML declaration file in the set to extract the required metadata from each YAML declaration file. Specifically, the operations field of each YAML declaration file in the set can be traversed to extract the interface path and method (such as POST / api / user), operation type (read / write), interface parameters (such as parameter name, value range, whether dynamically generated), dependency relationships (such as input_dependencies and output_resources, that is, the output of the former is the input of the latter), and explicit tags (such as sql_session indicating that operations within the same session need to be serialized) corresponding to each declaration file.
[0039] Step S123: Determine the explicit tags and interface parameters corresponding to several plugins to be executed based on the metadata.
[0040] After extracting the required metadata, it is necessary to extract the explicit tags and interface parameters from the metadata. It should be noted that the rule library is used to predict interface call conflicts between each plugin to be executed, and its construction depends on the explicit tags and interface parameters of the plugin metadata. Through the explicit tags, potential interface call conflict scenarios can be determined, and through the interface parameters, the dependency scenarios between plugins can be determined.
[0041] Step S124: Construct explicit conflict rules, implicit conflict rules, explicit dependency rules, and implicit dependency rules based on the explicit tags and interface parameters.
[0042] After determining the explicit tags and interface parameters, rules can be constructed according to the explicit tags and interface parameters.
[0043] Among them, the explicit conflict rule is that if there are several first target plugins with the same target explicit tag among several plugins to be executed, then the several first target plugins are executed serially. That is, if there are multiple plugins sharing the same display tag, the system needs to determine which plugin's interface to call. At this time, grouped management may help to distinguish different plugin instances and ensure that the correct plugin processes the request. Otherwise, the system may not be able to distinguish which plugin should be called, resulting in errors or unpredictable behavior. Therefore, the interface calls of these plugins need to be managed by grouping according to the tag and executed serially to avoid call conflicts.
[0044] Among them, the implicit conflict rule is that if there are several second target plugins among the to-be-executed plugins that call the same interface, all perform write operations, and there are mutually exclusive fields in the parameters, then the several second target plugins are executed serially. That is to say, if there are multiple plugins calling the same interface, and at this time if these interfaces all perform write operations and the parameters in the parameters are mutually exclusive, it is easy to cause chaos in the system state. For example, plugin A requires {force: true}, and plugin B requires {force: false}, then at this time plugin A attempts to force a reset, while plugin B hopes for non-forced, which can easily cause chaos in the system state. Since the essence of mutually exclusive parameters is the manifestation of business logic conflicts, it is necessary to execute these plugins serially to avoid causing unnecessary system state chaos.
[0045] Among them, the explicit dependency rule is that if there are several third target plugins among the to-be-executed plugins that define a dependency relationship, then the several third target plugins are executed serially. That is to say, if plugin B depends on plugin A, then after the action of plugin A ends, plugin B needs to perform corresponding operations. Therefore, it is necessary to execute the plugins with a dependency relationship serially.
[0046] Among them, the implicit dependency rule is that if there are two dependent plugins among the to-be-executed plugins that perform write-then-read operations on the same target interface, then the dependent plugins are executed serially. That is to say, if plugin A performs a write operation on interface X and then plugin B needs to perform a read operation on the same interface X. Then plugin A is a dependent plugin of plugin B, and it needs to be executed serially based on the sequence of the execution actions of the two plugins. And it is necessary to define a dynamic cooling period for the write-then-read operation to avoid the situation where one plugin's operation has not ended and the other plugin has already started to execute the action. The dynamic cooling period can be reasonably set according to user requirements and is set to 1.5 times the historical average response time of the interface in this embodiment.
[0047] In this way, through the rule library constructed in this embodiment, various conflicts during the execution of plugins can be effectively avoided, and the scanning efficiency and accuracy are effectively improved.
[0048] Step S13: Use the interface call conflict prediction results between the to-be-executed plugins obtained based on the current rule library to classify and merge the pre-operations of the to-be-executed plugins to generate each atomic task group.
[0049] In this embodiment, in addition to avoiding conflicts during the execution of each plugin through the constructed rule library, the execution efficiency of each vulnerability scanning subtask can also be improved by defining each atomic task group. Specifically, based on the interface call conflict prediction results between the to-be-executed plugins obtained from the current rule library, the pre-operations of the to-be-executed plugins can be classified and merged to generate each atomic task group. The specific process is as follows:
[0050] Step S131: Determine the dependency relationships among the pre-operations of several to-be-executed plugins according to the rule library.
[0051] Since the pre-operations of various to-be-executed plugins are different, it is necessary to determine the dependency relationships and operation types among the pre-operations of each to-be-executed plugin, so as to divide the atomic task groups according to the dependency relationships and operation types. Classification can be carried out according to the scanning type. For example, the pre-operations can be divided into a Web (World Wide Web) scanning group and a Host scanning group. The Web scanning group includes all plugin tasks that call Web interfaces and can be further divided into subclasses such as data injection and file upload. The Host scanning group includes all plugin tasks that call system interfaces (such as file reading and writing, process operations) and is further divided into subclasses such as privilege escalation detection, log analysis, and configuration check. Classification can also be carried out according to the operation type. For example, it can be divided into initial operation classes (such as user creation before testing), change status classes (such as changing the status of a certain switch before testing), and recovery operation classes (such as restoring the factory status before testing). Due to certain dependency relationships among various tasks and operations, more detailed classification of tasks and operations can be carried out.
[0052] Step S132: Merge the to-be-executed plugins that have no dependency relationships among several to-be-executed plugins into a first atomic task group; the first atomic task group is an atomic task group that needs to be executed in parallel.
[0053] In this embodiment, the to-be-executed plugins that have no dependency relationships among the to-be-executed plugins can be merged into a first atomic task group that is executed in parallel. This task group is a task group with strong compatibility among tasks. There is no resource overlap among the tasks in the task group, so each task can be executed in parallel.
[0054] Step S133: Split and reorganize the to-be-executed plugins that need to be partially executed serially among several to-be-executed plugins into a second atomic task group; the second atomic task group is an atomic task group that needs to be partially executed serially.
[0055] In this embodiment, the to-be-executed plugins with partial call conflicts among the execution plugins can be split and reorganized into a second atomic task group that needs to be partially executed serially. This task group is a task group with weak compatibility among tasks. Among them, there are partial call conflicts among some tasks, so the conflicting plugins need to be executed serially.
[0056] Step S134: Separate the to-be-executed plugins with mutually exclusive operations among several to-be-executed plugins into individual atomic task groups to obtain a third atomic task group; the third atomic task group is an atomic task group that needs to be executed serially.
[0057] In this embodiment, it is necessary to separate the to-be-executed plugins with mutually exclusive operations between them into individual third atomic task groups. Since the operation resources between each pair of plugins are mutually exclusive, the operations corresponding to the plugins are prohibited from being executed in parallel to avoid system problems. It is necessary to set each plugin in the third atomic task group to be executed in parallel.
[0058] Step S135: Merge the to-be-executed plugins with dependencies between several to-be-executed plugins into a fourth atomic task group; the fourth atomic task group is an atomic task group that needs to be executed serially.
[0059] In this embodiment, if there are dependencies between plugins, it is necessary to merge such to-be-executed plugins into a fourth atomic task group. Since the plugins in this task group have dependencies, they must be executed serially and must be executed in the order of the dependencies between the plugins.
[0060] Step S14: Execute each vulnerability scanning subtask corresponding to each atomic task group to obtain a task execution result corresponding to the vulnerability scanning task.
[0061] In this embodiment, it is necessary to execute each vulnerability scanning subtask corresponding to each atomic task group, and monitor in real time the task execution result corresponding to the vulnerability scanning task, and determine whether the task is successfully executed and whether the system metrics are abnormal based on the task execution result. If the task fails to be successfully executed, or the system metrics are abnormal, it is necessary to analyze the system metrics based on a preset metric abnormality judgment rule to determine the abnormality type of the abnormal metric. And the abnormality types include resource load abnormality, business response abnormality, and scanning side effects. Among them, the resource load abnormality includes abnormalities of metrics such as processor usage rate, memory occupancy, disk I / O (Input / Output), and network bandwidth; the business response abnormality includes HTTP error rate, interface average latency abnormality, and TPS (Transactions Per Second) abnormality; the scanning side effects include port status mutation, process crash, denial of service, etc.
[0062] It can be seen that in this embodiment, the current vulnerability scanning task can be determined, and then several to-be-executed plugins corresponding to the current vulnerability scanning task can be determined. Since each plugin executes corresponding vulnerability scanning subtasks and interface call conflicts may occur when calling interfaces, a rule library for predicting interface call conflicts between to-be-executed plugins can be constructed, and the pre-operations of several to-be-executed plugins can be classified and merged using the interface call conflict prediction results obtained based on the current rule library to generate different types of atomic task groups. Finally, by executing each vulnerability scanning subtask corresponding to each atomic task group, a task execution result corresponding to the vulnerability scanning task is obtained. In this way, through the constructed rule library to achieve interface-level conflict prediction, through the constructed atomic task groups to merge pre-operations and target status check mechanism, high-concurrency and low-conflict automated task scheduling can be realized, solving the problems of target service crash, low scanning efficiency and hybrid task conflict in the traditional solution, and significantly improving the scanning efficiency and accuracy.
[0063] Based on the foregoing embodiments, it can be known that exceptions may occur when executing a vulnerability scanning task, and when an exception occurs, the constructed rule library can be adjusted through exception adjustment. Therefore, this embodiment details how to adjust the rule library. Refer to Figure 2 As shown, an embodiment of the present application provides a method for executing a vulnerability scanning task, including:
[0064] Step S21, determine several to-be-executed plugins corresponding to the current vulnerability scanning task.
[0065] Step S22, construct a rule library for predicting interface call conflicts between to-be-executed plugins; the interface call conflict is a call conflict generated when the pre-operations of the to-be-executed plugins make interface calls.
[0066] Step S23, use the interface call conflict prediction results obtained based on the current rule library to classify and merge the pre-operations of several to-be-executed plugins to generate each atomic task group.
[0067] Step S24, execute each vulnerability scanning subtask corresponding to each atomic task group to obtain a task execution result corresponding to the vulnerability scanning task.
[0068] Step S25, if the task execution result indicates that there are abnormal indicators in the system indicators, determine the abnormal type corresponding to the abnormal indicator, and determine the target correction rule corresponding to the abnormal type from the preset correction rules.
[0069] In this embodiment, if there are abnormal indicators among the task execution results representing the system indicators, the abnormal type corresponding to the abnormal indicator is determined, and the corresponding rule is corrected. For example, if the target load is too high due to the still high packet sending frequency and the established test task cannot be completed, it indicates that there is a problem with the implicit dependency rule. Therefore, the implicit dependency rule can be corrected according to the target correction rule corresponding to the abnormal type. If there are problems with other rules, the same method is used for processing.
[0070] Step S26: Correct the target conflict prediction rule corresponding to the abnormal indicator in the rule library based on the target correction rule to obtain a new current rule library.
[0071] In this embodiment, it is necessary to correct the target conflict prediction rule corresponding to the abnormal indicator in the rule library according to the target correction rule. For example, if the target load is too high due to the still high packet sending frequency and the established test task cannot be completed, in order to handle the problem existing in this implicit dependency rule, the load can be reduced and the cooling period can be increased. The cooling period is corrected from the original historical average × 1.5 to historical average × 2.0 to relieve the target load situation and thus avoid target collapse. If it is detected that the concurrent call of the interface POST / api / reset causes the processor to be overloaded, a new rule can be added to the rule library: max_concurrent = 3, that is, the maximum number of processors allowed by the system is adjusted to 3, so as to limit the response of the interface and reduce the CPU load to avoid target collapse. If some rules cause the false alarm rate to be too high, for example, the false alarm rate exceeds 20%, the rules causing false alarms can be disabled. After correcting the rules, a new current rule library can be generated.
[0072] Furthermore, the atomic task group can be corrected. For example, if a certain task group causes abnormal indicators or a too high false alarm rate, the task group that frequently triggers abnormalities can be lowered in scheduling priority and split into smaller units. Since there are many mixed operations in the large task group and it is difficult to quickly locate the specific subtask causing the error, the split small task groups are executed independently, and the specific operation can be directly associated through the error log. Therefore, the large task group can be split into smaller atomic units (such as reducing from 50 concurrent requests to 10 requests) to reduce the impact of a single request on the target system. Further, if in the task of a certain scanning port, the port becomes unavailable due to too many initially set scanning parameters, that is, too many combined parameters, at this time, the parameters need to be corrected, and the compatibility threshold of this parameter can be adjusted to a value at which the port will not be exhausted of resources according to actual requirements.
[0073] Step S27: Generate a new atomic task group based on the new current rule library and re-execute the corresponding vulnerability scanning subtasks until the preset vulnerability scanning task end condition is met.
[0074] In this embodiment, it is necessary to generate a new atomic task group based on the new current rule base, wherein the vulnerability scanning subtasks in the atomic task group need to be classified, specifically into emergency tasks, parallel tasks and serial tasks, wherein the emergency task is a high-risk vulnerability scanning task, such as a task with a CVSS (Common Vulnerability Scoring System) greater than 9.0 or an emergency task manually marked, and the emergency task needs to be added to the real-time queue. The scheduling rule of the real-time queue is that if there is an emergency task, the parallel tasks and serial tasks are suspended, and dedicated execution resources are reserved for the emergency task. Therefore, when an emergency task needs to be executed, other tasks need to be unconditionally interrupted, resources are allocated to the emergency task first, and an independent resource pool needs to be reserved for the emergency task.
[0075] Furthermore, parallel tasks are tasks that need to be executed in parallel except for urgent tasks. Such tasks are parallel tasks without resource conflicts, such as read-only web interface scanning. Parallel tasks need to be added to parallel queues, and the scheduling rules of parallel queues are to dynamically adjust the number of parallel executions of parallel tasks according to the current system load, such as the number of concurrent tasks = 10 when CPU < 70%, and downgrade to 5 when ≥ 70%, and to allocate parallel tasks of the same interface to different time periods for execution, such as POST / api / reset is triggered every 5 seconds.
[0076] Furthermore, serial tasks are tasks that need to be executed serially except for emergency tasks. Such tasks are tasks that require exclusive resources and have plugin dependencies, such as file locks, port writing, etc. Serial tasks need to be added to the serial queue. The scheduling rule of the serial queue is to execute serial tasks in the order of tasks in the serial queue, and retry the task according to the preset retry interval when the serial task fails.
[0077] In this embodiment, when there are abnormal indicators in the system indicators, the rules in the rule base corresponding to the abnormality can be corrected, and then the current new rule base obtained after the correction can be used to reschedule the task, thereby avoiding the occurrence of such abnormalities again. In this way, the task queue and the rule base can be dynamically adjusted, avoiding the static execution method of allocating tasks as serial or parallel each time in the prior art, thereby avoiding the situation in the prior art that the target crashes after the task is executed once, and each task will cause the target to crash, effectively improving the stability of task execution and improving the user experience.
[0078] See also Figure 3As shown in the figure, this is the execution framework of the vulnerability scanning task execution method of this application. First, it is necessary to determine a number of plugins to be executed corresponding to the current vulnerability scanning task. Then, the interface conflict prediction module parses the interface call declarations in the plugin metadata to build an interface-level conflict prediction rule library. Next, the atomic task generation module, based on the interface conflict prediction rule library, intelligently classifies and merges the pre-operations of multiple plugins to generate atomic task groups, and the target exception monitoring module monitors the business situation and load metrics of the target system in real time, and modifies the conflict prediction rules and atomic task generation rules according to the exception check results. Finally, the task dynamic scheduling module dynamically optimizes the conflict prediction rule library and the atomic task group generation strategy, and optimizes the task execution order through multi-level queue management, elastic resource control, and dynamic scheduling strategies, maximizing the scanning efficiency while ensuring the stability of the target service. In this way, through the interface-level conflict prediction, pre-operation merging, and target status check mechanisms, high-concurrency and low-conflict automated task scheduling can be achieved, solving the problems of target service crashes, low scanning efficiency, and mixed task conflicts in traditional solutions, and significantly improving the scanning efficiency and accuracy.
[0079] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.
[0080] See Figure 4 As shown in the figure, an embodiment of this application also provides a vulnerability scanning task execution device, including:
[0081] A plugin determination module 11, configured to determine a number of plugins to be executed corresponding to the current vulnerability scanning task;
[0082] A rule library establishment module 12, configured to build a rule library for predicting interface call conflicts between the to-be-executed plugins; the interface call conflict is a call conflict generated when the pre-operations of the to-be-executed plugins make interface calls;
[0083] A task group generation module 13, configured to use the interface call conflict prediction results between the to-be-executed plugins obtained based on the current rule library to classify and merge the pre-operations of the to-be-executed plugins to generate each atomic task group;
[0084] A task execution module 14, configured to execute each vulnerability scanning subtask corresponding to each atomic task group to obtain a task execution result corresponding to the vulnerability scanning task.
[0085] It can be seen that through this embodiment, the current vulnerability scanning task can be determined, and then several to-be-executed plugins corresponding to the current vulnerability scanning task can be determined. Since each plugin executes corresponding vulnerability scanning subtasks, interface call conflicts may occur when calling interfaces. Therefore, a rule library for predicting interface call conflicts between each to-be-executed plugin can be constructed, and the pre-operations of several to-be-executed plugins can be classified and merged by using the interface call conflict prediction results between each to-be-executed plugin obtained based on the current rule library to generate different types of atomic task groups. Finally, by executing each vulnerability scanning subtask corresponding to each atomic task group, the task execution result corresponding to the vulnerability scanning task can be obtained. In this way, through the constructed rule library to achieve interface-level conflict prediction, through the constructed atomic task groups to merge pre-operations and target status check mechanism, high-concurrency and low-conflict automated task scheduling can be realized, solving the problems of target service crash, low scanning efficiency and hybrid task conflicts in the traditional solution, and significantly improving the scanning efficiency and accuracy.
[0086] In some embodiments, the rule library establishment module 12 may specifically include:
[0087] A file determination unit, configured to determine several to-be-executed plugin declaration files corresponding to several to-be-executed plugins;
[0088] A file traversal unit, configured to parse several to-be-executed plugin declaration files by traversing the attribute information in the several to-be-executed plugin declaration files to obtain metadata corresponding to each to-be-executed plugin declaration file;
[0089] A parameter determination unit, configured to determine explicit tags and interface parameters corresponding to several to-be-executed plugins based on the metadata;
[0090] A rule construction unit, configured to construct explicit conflict rules, implicit conflict rules, explicit dependency rules, and implicit dependency rules based on the explicit tags and interface parameters;
[0091] Among them, the explicit conflict rule is that if there are several first target plugins sharing the same target explicit tag among several to-be-executed plugins, then the several first target plugins are executed serially;
[0092] Among them, the implicit conflict rule is that if there are several second target plugins that call the same interface, both perform write operations, and there are mutually exclusive fields in the parameters among several to-be-executed plugins, then the several second target plugins are executed serially;
[0093] Among them, the explicit dependency rule is that if there are several third target plugins defining a dependency relationship among several to-be-executed plugins, then the several third target plugins are executed serially;
[0094] Among them, the implicit dependency rule is that if there are two dependent plugins among a number of plugins to be executed that perform write-then-read operations on the same target interface, the dependent plugins are executed serially.
[0095] In some embodiments, the task group generation module 13 may specifically include:
[0096] A dependency relationship determination unit, configured to determine the dependency relationship between the pre-operations of a number of plugins to be executed according to the rule library;
[0097] A first atomic task group construction unit, configured to merge the plugins to be executed that have no dependencies among a number of plugins to be executed into a first atomic task group; the first atomic task group is an atomic task group that needs to be executed in parallel;
[0098] A second atomic task group construction unit, configured to split and reorganize the plugins to be executed that need to be partially serially executed among a number of plugins to be executed into a second atomic task group; the second atomic task group is an atomic task group that needs to be partially serially executed;
[0099] A third atomic task group construction unit, configured to separate the plugins to be executed with mutually exclusive operations among a number of plugins to be executed into individual atomic task groups to obtain a third atomic task group; the third atomic task group is an atomic task group that needs to be serially executed;
[0100] A fourth atomic task group construction unit, configured to merge the plugins to be executed that have dependencies among a number of plugins to be executed into a fourth atomic task group; the fourth atomic task group is an atomic task group that needs to be serially executed.
[0101] In some embodiments, the task execution module 14 may specifically include:
[0102] An index collection unit, configured to execute each vulnerability scanning subtask corresponding to each atomic task group and collect system indexes in real time;
[0103] An abnormal index judgment unit, configured to analyze the system indexes based on a preset index abnormality judgment rule to determine whether there are abnormal indexes in the system indexes.
[0104] In some embodiments, the vulnerability scanning task execution device may further include:
[0105] A rule determination sub-module, configured to determine the abnormal type corresponding to the abnormal index if the task execution result indicates that there are abnormal indexes in the system indexes, and determine the target correction rule corresponding to the abnormal type from the preset correction rules;
[0106] A rule correction sub-module, configured to correct the target conflict prediction rule corresponding to the abnormal index in the rule library based on the target correction rule to obtain a new current rule library;
[0107] A task execution sub-module, configured to generate a new atomic task group based on the new current rule base, and re-execute the corresponding vulnerability scanning subtasks until a preset vulnerability scanning task end condition is met.
[0108] In some embodiments, the task execution sub-module may specifically include:
[0109] A task classification unit, configured to classify the vulnerability scanning subtasks in the atomic task group based on the corrected rules in the new current rule base to determine the emergency tasks, parallel tasks, and serial tasks in the atomic task group; parallel tasks are tasks that need to be executed in parallel except for emergency tasks, and serial tasks are tasks that need to be executed serially except for emergency tasks;
[0110] A task enqueue unit, configured to add emergency tasks to the real-time queue, add parallel tasks to the parallel queue, and add serial tasks to the serial queue;
[0111] A task execution unit, configured to execute the vulnerability scanning subtasks in the order of the real-time queue, parallel queue, and serial queue.
[0112] In some embodiments, the scheduling rule of the real-time queue is that if there are emergency tasks, the parallel tasks and serial tasks are suspended, and dedicated execution resources are reserved for the emergency tasks;
[0113] The scheduling rule of the parallel queue is to dynamically adjust the number of parallel tasks executed in parallel according to the current system load, and allocate parallel tasks with the same interface to different time periods for execution;
[0114] The scheduling rule of the serial queue is to execute the serial tasks in the order of the tasks in the serial queue, and retry the tasks according to a preset retry interval when the serial tasks fail.
[0115] For the description of the features in the embodiments corresponding to the vulnerability scanning task execution device, reference may be made to the relevant description in the embodiments corresponding to the vulnerability scanning task execution method, which will not be elaborated here one by one.
[0116] An embodiment of the present application also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the embodiments of the above vulnerability scanning task execution method.
[0117] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any one of the embodiments of the above vulnerability scanning task execution method when running.
[0118] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media capable of storing computer programs such as USB flash drives, read-only memory (ROM for short), random access memory (RAM for short), external hard drives, magnetic disks, or optical discs.
[0119] The embodiments of the present application also provide a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, the steps in any of the embodiments of the above method for executing a vulnerability scanning task are implemented.
[0120] The embodiments of the present application also provide another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the embodiments of the above method for executing a vulnerability scanning task are implemented.
[0121] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.
[0122] The above has introduced in detail a method, device, equipment, and storage medium for executing a vulnerability scanning task provided by the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present application, several improvements and modifications can still be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.
Claims
1. A method for executing a vulnerability scanning task, characterized in that, Including: Determine a number of to-be-executed plugins corresponding to the current vulnerability scanning task; Construct a rule library for predicting interface call conflicts between the to-be-executed plugins; The interface call conflict is a call conflict generated when the pre-operations of the to-be-executed plugins make interface calls; Use the interface call conflict prediction results between the to-be-executed plugins obtained based on the current rule library to classify and merge the pre-operations of the to-be-executed plugins, so as to generate each atomic task group; Execute each vulnerability scanning subtask corresponding to each atomic task group to obtain a task execution result corresponding to the vulnerability scanning task.
2. The vulnerability scanning task execution method according to claim 1, characterized in that, The constructing a rule library for predicting interface call conflicts between the to-be-executed plugins includes: Determine a number of to-be-executed plugin declaration files corresponding to the to-be-executed plugins; By traversing the attribute information in the to-be-executed plugin declaration files, parse the to-be-executed plugin declaration files to obtain the metadata corresponding to each to-be-executed plugin declaration file; Based on the metadata, determine the explicit tags and interface parameters corresponding to the to-be-executed plugins; Construct explicit conflict rules, implicit conflict rules, explicit dependency rules, and implicit dependency rules based on the explicit tags and the interface parameters; Among them, the explicit conflict rule is that if there are a number of first target plugins sharing the same target explicit tag among the to-be-executed plugins, then the first target plugins are serially executed; Among them, the implicit conflict rule is that if there are a number of second target plugins that call the same interface, both perform write operations, and there are mutually exclusive fields in the parameters among the to-be-executed plugins, then the second target plugins are serially executed; Among them, the explicit dependency rule is that if there are a number of third target plugins with defined dependency relationships among the to-be-executed plugins, then the third target plugins are serially executed; Among them, the implicit dependency rule is that if there are two dependent plugins that perform write-then-read operations on the same target interface among the to-be-executed plugins, then the dependent plugins are serially executed.
3. The vulnerability scanning task execution method according to claim 1, characterized in that, The classifying and merging the pre-operations of the to-be-executed plugins to generate each atomic task group includes: Determine the dependency relationships between the pre-operations of the to-be-executed plugins according to the rule library; Merge the to-be-executed plugins without the dependency relationship among the to-be-executed plugins into a first atomic task group; the first atomic task group is an atomic task group that needs to be executed in parallel; Split and reorganize the to-be-executed plugins that need to be partially serially executed among the to-be-executed plugins into a second atomic task group; the second atomic task group is an atomic task group that needs to be partially serially executed; Make the to-be-executed plugins with mutually exclusive operations among the to-be-executed plugins independent as individual atomic task groups to obtain a third atomic task group; the third atomic task group is an atomic task group that needs to be serially executed; Merge the to-be-executed plugins with the dependency relationship among the to-be-executed plugins into a fourth atomic task group; the fourth atomic task group is an atomic task group that needs to be serially executed.
4. The method for executing a vulnerability scanning task according to claim 1, wherein Performing each vulnerability scanning subtask corresponding to each atomic task group to obtain a task execution result corresponding to the vulnerability scanning task, including: Performing each vulnerability scanning subtask corresponding to each atomic task group and collecting system metrics in real time; Analyzing the system metrics based on a preset abnormal metric determination rule to determine whether there are abnormal metrics in the system metrics.
5. The method for executing a vulnerability scanning task according to any one of claims 1 to 4, characterized in that It further includes: If the task execution result indicates that there are abnormal metrics in the system metrics, determining the abnormal type corresponding to the abnormal metrics, and determining a target correction rule corresponding to the abnormal type from the preset correction rules; Correcting the target conflict prediction rule corresponding to the abnormal metrics in the rule library based on the target correction rule to obtain a new current rule library; Generating new atomic task groups based on the new current rule library and re-executing the corresponding vulnerability scanning subtasks until a preset vulnerability scanning task end condition is met.
6. The vulnerability scanning task execution method according to claim 5, wherein The generating new atomic task groups based on the new current rule library and re-executing the corresponding vulnerability scanning subtasks until a preset vulnerability scanning task end condition is met includes: Classifying the vulnerability scanning subtasks in the atomic task group based on the corrected rules in the new current rule library to determine the urgent tasks, parallel tasks, and serial tasks in the atomic task group; the parallel tasks are tasks that need to be executed in parallel except for the urgent tasks, and the serial tasks are tasks that need to be executed serially except for the urgent tasks; Adding the urgent tasks to the real-time queue, adding the parallel tasks to the parallel queue, and adding the serial tasks to the serial queue; Executing the vulnerability scanning subtasks in the order of the real-time queue, parallel queue, and serial queue.
7. The method for executing a vulnerability scanning task according to claim 6, wherein The scheduling rule of the urgent queue is that if there are urgent tasks, suspending the parallel tasks and the serial tasks and reserving dedicated execution resources for the urgent tasks; The scheduling rule of the real-time queue is to dynamically adjust the number of parallel executions of the parallel tasks according to the current system load and allocate the parallel tasks with the same interface to different time periods for execution; The scheduling rule of the serial queue is to execute the serial tasks in the order of the tasks in the serial queue and retry the tasks according to a preset retry interval when the serial tasks fail to execute.
8. A vulnerability scanning task execution device, characterized in that It includes: A plugin determination module for determining a number of plugins to be executed corresponding to the current vulnerability scanning task; A rule library establishment module for constructing a rule library for predicting interface call conflicts between the plugins to be executed; the interface call conflict is a call conflict generated when the pre-operations of the plugins to be executed make interface calls; A task group generation module for classifying and merging the pre-operations of the number of plugins to be executed by using the interface call conflict prediction results between the plugins to be executed obtained based on the current rule library to generate each atomic task group; A task execution module for performing each vulnerability scanning subtask corresponding to each atomic task group to obtain a task execution result corresponding to the vulnerability scanning task.
9. An electronic device, characterized in that, It includes: A memory for storing a computer program; A processor for executing the computer program to implement the steps of the vulnerability scanning task execution method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, wherein when the computer program is executed by the processor, the steps of the vulnerability scanning task execution method according to any one of claims 1 to 7 are implemented.