A scanning method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to a scanning method, device, electronic equipment and storage medium. The method comprises: when it is needed to scan a to-be-scanned vulnerability, obtaining vulnerability information and scanning environment information of the to-be-scanned vulnerability; based on the vulnerability information and the scanning environment information, predicting resource occupation information required when the to-be-scanned vulnerability is scanned; obtaining load information currently respectively corresponding to a plurality of scanning engines, and based on the load information and the resource occupation information, determining a target scanning engine from the plurality of scanning engines, and scanning the to-be-scanned vulnerability through the target scanning engine. Through the method, the resource occupation information required by the to-be-scanned vulnerability can be predicted according to the vulnerability information and the scanning environment information of the to-be-scanned vulnerability, and the to-be-scanned vulnerability is further distributed to the target scanning engine through the resource occupation information and the load information of the scanning engine, so that the balanced scheduling of the scanning engine is realized, and the scanning efficiency is improved.

Description

Technical Field

[0001] This disclosure relates to the field of cybersecurity, and in particular to a scanning method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of the Internet, network security has become increasingly important. Vulnerability scanning technology, as an important network security technology, can effectively improve network security and discover potential security problems in the network.

[0003] In related technologies, when performing vulnerability scanning, the scheduling center distributes the scanning tasks evenly to each scanning engine. Since each task requires a different load, there may be a situation where one scanning engine is overloaded, causing tasks to queue while other scanning engines are idle, resulting in low task scanning efficiency. Summary of the Invention

[0004] This disclosure provides a scanning method, apparatus, electronic device, and storage medium.

[0005] According to a first aspect of this disclosure, a scanning method is provided, the method comprising: when it is necessary to scan for a vulnerability to be scanned, acquiring vulnerability information and scanning environment information of the vulnerability to be scanned; predicting resource usage information required to scan for the vulnerability to be scanned based on the vulnerability information and scanning environment information; acquiring load information corresponding to multiple scanning engines respectively, and determining a target scanning engine from the multiple scanning engines based on the load information and resource usage information, and scanning for the vulnerability to be scanned through the target scanning engine.

[0006] According to a second aspect of this disclosure, a scanning apparatus is provided, comprising: an acquisition module for acquiring vulnerability information and scanning environment information of the vulnerability to be scanned when scanning for a vulnerability to be scanned is required; a prediction module for predicting resource usage information required to scan for the vulnerability to be scanned based on the vulnerability information and the scanning environment information; and a determination module for acquiring load information corresponding to multiple scanning engines, determining a target scanning engine from the multiple scanning engines based on the load information and resource usage information, and scanning for the vulnerability to be scanned using the target scanning engine.

[0007] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the program to implement the method described above.

[0008] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the methods described above.

[0009] According to a fifth aspect of this disclosure, a computer program product is provided, comprising: computer program code that, when run on a computer, causes the computer to perform the method described in the first aspect or any possible implementation thereof.

[0010] The scanning method, apparatus, electronic device, and storage medium provided in this disclosure acquire vulnerability information and scanning environment information of the vulnerability to be scanned when scanning is required; predict the resource consumption information required for scanning the vulnerability based on the vulnerability information and scanning environment information; acquire the current load information of multiple scanning engines; determine the target scanning engine from the multiple scanning engines based on the load information and resource consumption information; and scan the vulnerability to be scanned using the target scanning engine. It can predict the resource consumption information required for scanning the vulnerability based on the vulnerability information and scanning environment information, and further allocate the vulnerability to be scanned to the target scanning engine based on the resource consumption information and the load information of the scanning engines, thereby achieving balanced scheduling of scanning engines and improving scanning efficiency. Attached Figure Description

[0011] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0012] Figure 1 An architecture diagram of a scanning system corresponding to a scanning method is provided for an exemplary embodiment of this disclosure;

[0013] Figure 2 for Figure 1 A schematic diagram of the centralized dispatch center structure in the scanning system;

[0014] Figure 3 A schematic diagram of a scanning engine structure provided for an exemplary embodiment of this disclosure;

[0015] Figure 4 A flowchart of a scanning method provided as an exemplary embodiment of this disclosure;

[0016] Figure 5 A schematic block diagram of the functional modules of a scanning apparatus provided in an exemplary embodiment of this disclosure;

[0017] Figure 6 A structural block diagram of an electronic device provided as an exemplary embodiment of this disclosure;

[0018] Figure 7 A block diagram of a computer system provided for an exemplary embodiment of this disclosure. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0021] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0022] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0023] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0026] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation of this disclosure; other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0027] With the development of the Internet, network security has become increasingly important. Vulnerability scanning technology, as an important network security technology, can effectively improve network security and discover potential security problems in the network.

[0028] In related technologies, when performing vulnerability scanning, the scheduling center distributes the scanning tasks evenly to each scanning engine. Since each task requires a different load, there may be a situation where one scanning engine is overloaded, causing tasks to queue while other scanning engines are idle, resulting in low task scanning efficiency.

[0029] Based on this, this disclosure provides a vulnerability scanning system that trains a load balancing model using task data from historical scanning tasks, inputs the task information of the task to be scanned into the load balancing model, predicts the CPU utilization, memory usage, and estimated execution time of the task to be scanned, obtains the CPU utilization and memory usage of currently online scanning engines, determines the scanning engine that can meet the CPU utilization and memory usage of the task to be scanned, and predicts the start time of each scanning engine to start executing the task to be scanned, with the scanning engine with the fastest start time executing the task to be scanned.

[0030] The vulnerability scanning system provided in the embodiments of this disclosure will be described in detail below. Figure 1 As shown in the embodiments of this disclosure, the vulnerability scanning system includes a front-end service, a centralized scheduling center, a business processing service, a microservice framework, and a scanning engine. The front-end service carries the visualization functions of the vulnerability scanning system, mainly used for visualization configuration functions such as scan task configuration and scheduling configuration, as well as visualization dashboards for scan task statistics and vulnerability statistics. The business processing service is the core service of the vulnerability scanning system, providing corresponding back-end interfaces for the front-end and scheduling center, and other services to call, responsible for the processing and operation of various business logics of the vulnerability scanning system. The centralized scheduling center is responsible for scheduling the scanning engine to run scanning tasks and managing the scanning tasks. Technical personnel can configure scanning tasks on the front-end page and select information such as the scan start time, the list of vulnerabilities to be scanned, and the resources to be scanned. The scanning tasks are then issued to the scanning engine for execution through the centralized scheduling center. After execution, task information is collected and relevant indicators are statistically analyzed, and finally displayed on the visualization page.

[0031] like Figure 2 As shown, the centralized scheduling center is responsible for scheduling scanning engines to run scanning tasks, managing these tasks, and collecting statistics on their status, execution time, and other information. It includes an Application Programming Interface (API) module, a configuration module, a statistics module, and a scheduling module. The centralized scheduling center primarily schedules scanning engines to execute scanning tasks and manages their lifecycles. Simultaneously, it collects information reported by each scanning engine through the API module, including the CPU and memory usage of the server where the scanning engine resides, and information on currently executing scanning tasks. The configuration module is mainly used for configuring the centralized scheduling center, allowing configuration of scanning task information such as scan start time, scan resource list, scan vulnerability list, and scan vulnerability type. The statistics module is mainly used to collect statistics on the CPU, memory, and execution time consumed by each task, and to schedule tasks according to a load balancing algorithm. The scheduling module is mainly used to execute tasks, manage the lifecycle of scanning tasks, and obtain information such as task execution status and results. The following is a detailed description of each module:

[0032] API module: This module within the scheduling center is responsible for providing API interfaces to the outside world. It exposes the capabilities of the scheduling center through API interfaces, and call requests also obtain the corresponding information from the centralized scheduling center through the API module.

[0033] Configuration module: Within the scheduling center, this module is responsible for providing configuration functions, including scan task configuration, scan rate control configuration, and load balancing configuration, etc.

[0034] Statistics module: Located within the scheduling center, it is responsible for functions such as load balancing algorithm score statistics, scan task data statistics, and vulnerability data statistics.

[0035] Scheduling module: Located within the scheduling center, it is responsible for executing scanning tasks according to the scheduling configuration, monitoring the lifecycle of scanning tasks, and collecting information such as the status, execution time, and scanning results of scanning tasks.

[0036] like Figure 3As shown, the scanning engine is the component responsible for running scanning tasks, supporting batch concurrent scanning, information reporting, and scan task management. It includes a proxy module, a port scanning module, a path detection module, and a scanner module. The proxy module primarily receives and processes invocation requests, such as task execution requests, and calls different modules based on the request content; it also reports the scanning engine's execution task information and server usage information. The port scanning module primarily scans open ports on the server and can also perform service detection, probing the server's services and versions. The path detection module is mainly used to detect corresponding web directories and can detect different levels of Uniform Resource Locator (URL) paths according to settings. The scanner is mainly used to execute scanning tasks, supporting multiple tasks to execute concurrently, performing scans according to task settings, and obtaining execution results. The scanning engine is deployed on the same node as the scanners, shielding the differences between different scanners and allowing different scanners to execute scanning tasks within the same architecture.

[0037] The proxy module, located within the scanning engine, is responsible for providing API interfaces to the outside world, receiving call requests, and periodically obtaining information such as the status of the server where the scanner is located and the execution information of scanning tasks.

[0038] Port scanning module: Within the scanning engine, it is responsible for searching the network port information of the target host and determining whether it is alive, etc.

[0039] Path detection module: Within the scanning engine, it is responsible for searching the web directories of different network ports on the target host and determining whether they are accessible, etc.

[0040] Scanner: Within the scanning engine, this module is responsible for performing vulnerability scans. It supports multi-task parallel execution and can be a self-developed scanner or a commercial scanner.

[0041] Resources to be scanned: Different application software connected to the scanning engine network, application resources for which vulnerabilities are to be scanned, including host resources and web application resources, etc.

[0042] The above is an introduction to the vulnerability scanning system provided in this disclosure. The scanning method provided in this disclosure will be explained below based on the underlying implementation logic. The scanning method provided in this disclosure includes the following steps:

[0043] Step 1: Train the load balancing model.

[0044] This disclosure primarily employs a Lightweight Gradient Boosting Machine (LightGBM) model to train specific scenario parameters for vulnerability scanning. LightGBM is a framework implementing the Gradient Boosting Decision Tree (GBDT) algorithm, supporting efficient parallel training and offering advantages such as faster training speed, lower memory consumption, better accuracy, and support for distributed processing of massive amounts of data. The load balancing algorithm uses the LGBM algorithm, predicting the CPU, memory usage, and execution time of the task to be executed through iterative training on historical execution data. Since the information on the resources to be scanned and the vulnerability information are different for each task, to improve the accuracy of the trained model, as many feature terms as possible are selected. The fingerprint information of the resources to be scanned, the list of CPE names for the vulnerabilities, and the list of vulnerability types are recorded as feature terms for training the model. The resource fingerprint information is the hash value obtained by naturally concatenating the Internet Protocol (IP) information (I), server system version (V), and open port (P) information of the previously detected resources to be scanned, as shown in the following formula:

[0045] Z = Hash(IVP)

[0046] The fingerprint information list in the feature section is a hash table of all fingerprint information of the resources to be scanned, which is then sorted and naturally concatenated. The formula is as follows:

[0047]

[0048] The vulnerability list L is a hash table of the vulnerability list to be scanned, sorted by CPE name and then naturally concatenated. The formula is as follows:

[0049]

[0050] The vulnerability type list S consists of vulnerability information to be scanned, sorted by type number and then hashed using the following formula:

[0051]

[0052] Using (S, P, L) as the model feature values, the model predictions are CPU utilization C, memory utilization M, and task execution time T, which is solved by the following model function:

[0053] f(C,M,T)=lgb(S,P,L)

[0054] In some embodiments, during task execution, the agent module of the scanning engine samples CPU and memory usage every 5 minutes, calculates the CPU and memory utilization of the server where the scanning engine resides during task execution every 5 minutes, and reports it to the API module of the centralized scheduling center. Then, the statistics module of the scheduling center calculates the weighted average of the CPU and memory utilization of each sampled task as model training data, and iteratively trains and optimizes the model based on continuously completed scanning task data. When a new scanning task needs to be scheduled, the C, M, and T values ​​of this scanning task are calculated by substituting the values ​​of the relevant feature terms into the model. Then, the idle status of all currently online scanning engine nodes is obtained. First, based on CPU and memory availability, a preliminary list I of scanning engines capable of executing the task is obtained. Then, the predicted execution time for each scanning engine in I is obtained, these predicted execution times are sorted, and the scanning engine with the shortest predicted execution time from the current time is selected to execute the assigned scanning task. If no scanning engine is available to execute the current scanning task, it enters a waiting queue, and the idle status of the scanning engines is collected every 5 minutes to continuously determine whether the scanning tasks in the waiting queue can be executed, thus making full use of scanning engine resources.

[0055] In some embodiments, the training process of the load balancing model is as follows:

[0056] a. The centralized scheduling center's scheduling module triggers a scanning task and distributes the scanning task to the scanning engine for execution.

[0057] b. The scanning engine agent module receives the scanning task and then executes the scanning task according to the request information. During the execution of the task, the server load, including CPU utilization and memory utilization, is sampled every 5 minutes and reported to the centralized scheduling center API module.

[0058] c. The API module of the centralized scheduling center receives the task information and then transfers it to the statistics module for processing.

[0059] d. The statistics module calculates and stores the CPU and memory usage of the task at the time of this report.

[0060] e. After the task is completed, collect the server load and task execution time and report them to the API module of the centralized scheduling center.

[0061] f. The API module of the centralized scheduling center receives the task completion information and then transfers it to the statistics module for processing.

[0062] g. When the statistics module receives the task completion information, it first calculates the CPU usage, memory usage, and task execution time at the time of the report. Then, based on the existing 5 minutes of sampled data, it calculates the CPU and memory usage during the task using a weighted average method.

[0063] h. The list of resource fingerprints to be scanned, the list of CPE names for vulnerabilities, and the list of vulnerability types for this scanning task are used as features. The final statistical data of CPU utilization, memory utilization, and execution time are added to the training set as labels, and the model is trained using the LGBM algorithm. In some embodiments, after each scanning task is completed, the information statistically derived from the task execution can be added to the training set. Through iterative loops, the model can be continuously optimized, thereby making the model's prediction of the scanning task results more accurate and improving the efficiency of scanning task scheduling.

[0064] Step 2: Obtain the task to be scanned.

[0065] In some embodiments, technicians create scanning tasks on the front-end page, and the front-end sends the scanning tasks to the centralized dispatch center.

[0066] Step 3: Predict the workload and execution time of the task to be scanned.

[0067] In some embodiments, based on the information of the task to be scanned, the fingerprint information of the scanned resource list, the list of scanned vulnerability CPE names, and the list of scanned vulnerability types are obtained as features, and the load balancing model trained by the LGBM algorithm is called to calculate the CPU utilization, memory utilization, and estimated execution time of this task.

[0068] Step 4: Determine the scanning engine to perform the scanning task.

[0069] In some embodiments, the load information (including CPU utilization, memory utilization, etc.) of all currently online scanning engine nodes is collected in real time, and a preliminary list of nodes that can meet the CPU and memory requirements of this scanning task is selected. The information of currently executing scanning tasks in the preliminary list of nodes is statistically analyzed, and the scanning engine with the shortest predicted execution time from the current time is selected to execute the assigned scanning task.

[0070] Step 5: The scanning engine executes the scan task.

[0071] In some embodiments, the scanning engine proxy module receives a scan task execution request, first parses the request to obtain the scan object and vulnerability information, then verifies the validity of the request, and adds the scan requests to the message queue sequentially for execution. The proxy module retrieves the scan requests from the message queue and begins execution. Upon completion of the task, the scanning engine proactively notifies the centralized scheduling center API module to update the task status. The centralized scheduling center API module receives the task completion information and then transfers it to the statistics module for processing. The statistics module stores task execution information, including task execution time, completion status, and the executing scanning engine. The centralized scheduling center scheduling module periodically calls the API module to query the task execution status according to the configured task timeout period. If the timeout has occurred, it notifies the API module to terminate the task and records the execution status information.

[0072] Based on the above embodiments, this disclosure also provides a scanning method, such as... Figure 4 As shown, the method may include the following steps:

[0073] Step S100: When it is necessary to scan for a vulnerability, obtain the vulnerability information of the vulnerability to be scanned and the scanning environment information.

[0074] Step S200: Based on vulnerability information and scanning environment information, predict the resource usage information required when scanning the vulnerability to be scanned.

[0075] Step S300: Obtain the current load information corresponding to multiple scanning engines, and based on the load information and resource usage information, determine the target scanning engine from the multiple scanning engines, and scan for vulnerabilities to be scanned using the target scanning engine.

[0076] The following is a detailed explanation of each step:

[0077] In step S100, when it is necessary to scan for a vulnerability, the vulnerability information and scanning environment information of the vulnerability to be scanned are obtained.

[0078] The vulnerability information to be scanned includes the vulnerability name, vulnerability type, and the corresponding resource to be scanned. The scanning environment information includes the IP information, server information, and open port information of the resource to be scanned. The resource to be scanned can be files, code, etc., entered by technical personnel.

[0079] In step S200, based on vulnerability information and scanning environment information, the resource consumption information required to scan for the vulnerability to be scanned is predicted.

[0080] The resource usage information required for scanning vulnerabilities includes CPU usage information, memory usage information, and usage duration. The CPU usage information includes CPU utilization rate, the memory usage information includes memory utilization rate, and the usage duration is the estimated execution time for scanning vulnerabilities.

[0081] In some embodiments, vulnerability information and scanning environment information are input into a pre-trained load balancing model to obtain the resource consumption information required when scanning the vulnerability to be scanned.

[0082] In step S300, the load information corresponding to the current scan engines is obtained, and the target scan engine is determined from the multiple scan engines based on the load information and resource usage information, and the vulnerability to be scanned is scanned through the target scan engine.

[0083] The load information includes CPU utilization and memory utilization.

[0084] In one possible implementation, candidate scanning engines whose load information meets the resource usage information are selected from multiple scanning engines, and the execution time of each candidate scanning engine to scan for the vulnerability is determined. The candidate scanning engine with the earliest execution time is selected as the target scanning engine.

[0085] Through the above embodiments, the present disclosure can predict the resource consumption information required by the vulnerability to be scanned based on the vulnerability information and scanning environment information, and further allocate the vulnerability to be scanned to the target scanning engine based on the resource consumption information and the load information of the scanning engine, thereby achieving balanced scheduling of the scanning engines and improving scanning efficiency.

[0086] To more clearly illustrate the embodiments of this disclosure, step S200 above further includes the following step:

[0087] Step S210: Obtain historical vulnerability information and historical scanning environment information corresponding to historical scan vulnerabilities, and obtain historical resource usage information of historical scan vulnerabilities.

[0088] Step S220: Use historical vulnerability information and historical scanning environment information as training samples, and use historical resource usage information and historical execution duration as labels for the training samples to train the preset model and obtain the load balancing model.

[0089] Step S230: Input the vulnerability information and scanning environment information into the load balancing model to obtain the resource usage information required when scanning the vulnerability to be scanned.

[0090] The following is a detailed explanation of steps S210 to S230:

[0091] In step S210, historical vulnerability information and historical scanning environment information corresponding to historical scan vulnerabilities are obtained, and historical resource usage information of historical scan vulnerabilities is obtained.

[0092] The historical vulnerability information includes the name of the historical vulnerability, the type of the historical vulnerability, and the corresponding historical scanning resources. The historical scanning environment information includes the Internet Protocol IP information, server information, and open port information of the historical scanning resources. The historical resource usage information includes the CPU usage, memory usage, and execution time of the historical vulnerability.

[0093] Step S220: Use historical vulnerability information and historical scanning environment information as training samples, and use historical resource usage information and historical execution duration as labels for the training samples to train the preset model and obtain the load balancing model.

[0094] The preset model can be a machine learning model, such as Logistic Regression (LR), Decision Tree (DT), Stochastic Gradient Descent (SGD), Support Vector Machines (SVM), Random Forest (RF), Gradient Boosting Decision Tree (GBDT), Extreme Gradient Boosting (XGBoost), and LightGradient Boosting Machine (LGBM), etc. This disclosure does not limit the preset model.

[0095] This disclosure uses the preset model LightGBM as an example for illustration. Since the vulnerability information and scanning environment information are different for each scan, as many feature terms as possible are selected to improve the accuracy of the trained model. This disclosure uses the list of scanned vulnerability CPE names, the list of vulnerability types, and related information as feature terms for training the model. The scanning environment information is the hash value obtained by naturally concatenating the IP information (I), server system version (V), and open port information (P) of the resource to be scanned, as shown in the following formula:

[0096] Z = Hash(IVP)

[0097] Each vulnerability to be scanned may correspond to multiple resources to be scanned. The environmental information of multiple resources to be scanned is sorted to obtain an environmental information list P. After sorting all the scanning environmental information, it is naturally concatenated and hashed, as shown in the following formula:

[0098]

[0099] The vulnerability list L is a hash table of the vulnerability list to be scanned, sorted by CPE name and then naturally concatenated. The formula is as follows:

[0100]

[0101] The vulnerability type list S consists of vulnerability information to be scanned, sorted by type number and then hashed using the following formula:

[0102]

[0103] Using (S,P,L) as the model feature values, the model prediction values ​​are CPU utilization C, memory utilization M, and execution time T, which is solved by the following model function:

[0104] f(C,M,T)=lgb(S,P,L)

[0105] In some embodiments, the CPU and memory usage of the scanning engine are sampled every 5 minutes during the scanning process. The CPU and memory usage of the server where the scanning engine is located are calculated during each 5-minute vulnerability scan. The weighted average of the CPU usage and memory usage sampled for each scan is determined, and this weighted average is used as model training data. The model is iteratively trained and optimized based on the continuously completed scan data to obtain a load balancing model.

[0106] Step S230: Input the vulnerability information and scanning environment information into the load balancing model to obtain the resource usage information required when scanning the vulnerability to be scanned.

[0107] In some embodiments, vulnerability information may be a list of vulnerability names, a list of vulnerability types, and a list of resources to be scanned corresponding to the vulnerabilities. Scanning environment information may be a list of IP addresses, server versions, and open ports for each resource in the resource list.

[0108] In one possible implementation, the vulnerability information and scanning environment information are hashed to obtain hash values ​​corresponding to the vulnerability name list, vulnerability type list, and scanning environment information, respectively. The hash values ​​are then input into a load balancing model, which predicts the CPU usage, memory usage, and execution time required to scan the vulnerabilities.

[0109] To more clearly illustrate step S300 above, step S300 further includes the following steps:

[0110] Step S310: Based on load information and resource usage information, determine the candidate scanning engine from multiple scanning engines whose load information meets the resource usage information;

[0111] Step S320: Predict the start time when each of the candidate scanning engines will begin scanning for the vulnerabilities to be scanned;

[0112] Step S330: Determine the target scanning engine from the candidate scanning engines based on the start time.

[0113] Steps S310 to S330 are explained below:

[0114] In step S310, based on load information and resource usage information, candidate scanning engines whose load information meets the resource usage information are determined from multiple scanning engines.

[0115] In step S320, the start time of each scanning engine in the candidate scanning engines is predicted.

[0116] In some embodiments, the task information being scanned by the candidate scanning engine is obtained, wherein the scanning task information includes scanning vulnerability information and scanning environment information, and the start time of the candidate scanning engine is determined based on the task information.

[0117] In one possible implementation, the end time of the scanning engine's scan of the currently being scanned task can be obtained based on task information and a load balancing model, and this end time can be used as the start time for scanning the vulnerabilities to be scanned.

[0118] For example, the total scanning time required for the task being executed by the candidate scanning engine is obtained through the load balancing model, the scanning time already completed for the currently executing task is obtained, and the end time is obtained based on the already completed scanning time and the total scanning time required to execute the task.

[0119] In step S330, the target scanning engine is determined from the candidate scanning engines based on the start time.

[0120] In some embodiments, the candidate scanning engine with the shortest start time from the current time can be used as the target scanning engine, that is, the candidate scanning engine with the earliest start time can be used as the target scanning engine.

[0121] To further illustrate this disclosure more clearly, the preceding steps also include:

[0122] Step S400: Obtain the target scan duration for the vulnerability to be scanned.

[0123] Step S500: During the process of the target scanning engine scanning for the vulnerability to be scanned, obtain the scanning status of the vulnerability to be scanned.

[0124] Step S600: If the scan status is incomplete, obtain the scan duration of the target scanning engine for scanning the vulnerability to be scanned;

[0125] Step S700: If the scan duration is longer than the target scan duration, control the target scanning engine to stop scanning the vulnerability to be scanned, and re-determine the target scanning engine to scan the vulnerability to be scanned.

[0126] The following is a detailed explanation of steps S400 to S700:

[0127] In step S400, the target scan duration for the vulnerability to be scanned is obtained.

[0128] The target scan duration is the total time taken. By inputting vulnerability information and scanning environment information into the load balancing model, the predicted scan duration for the vulnerability to be scanned can be obtained.

[0129] In step S500, during the process of the target scanning engine scanning the vulnerability to be scanned, the scanning status of the vulnerability to be scanned is obtained.

[0130] The scan status can be either "completed" or "completed".

[0131] In some embodiments, the scanning status of the vulnerability to be scanned can be obtained during the scanning process based on a preset frequency, for example, the scanning status can be obtained once every 5 minutes.

[0132] In step S600, if the scanning status is incomplete, the scanning time of the target scanning engine for the vulnerability to be scanned is obtained.

[0133] In some embodiments, when the scanning status is incomplete, it is necessary to determine whether the scanning has timed out, thereby determining whether a fault has occurred in the scanning process and taking further measures.

[0134] In step S700, if the scan duration is longer than the target scan duration, the target scanning engine is controlled to stop scanning the vulnerability to be scanned, and a new target scanning engine is determined for scanning the vulnerability to be scanned.

[0135] In some embodiments, if the scan duration exceeds the target scan duration, it indicates that the target scanning engine has timed out and may be malfunctioning, making it unable to complete the scan for the vulnerability to be scanned. Therefore, it is necessary to control the target scanning engine to stop scanning and to re-determine a target scanning engine that can scan for the vulnerability to be scanned.

[0136] To more clearly illustrate the embodiments of this disclosure, the above steps also include:

[0137] Step S800: Determine whether the target scanning engine exists among multiple scanning engines based on load information and resource usage information.

[0138] In some embodiments, load information includes CPU utilization and memory utilization, and resource usage information includes CPU utilization and memory utilization. CPU idle rate and memory idle rate are obtained from CPU utilization and memory utilization. If the CPU idle rate of the scanning engine is greater than the CPU utilization required by the vulnerability to be scanned, and the memory idle rate is greater than the memory utilization required by the vulnerability to be scanned, it indicates that the load information of the scanning engine satisfies the resource usage information, and a target scanning engine exists. Conversely, if either the memory idle rate or the CPU idle rate of the scanning engine is not greater than the memory utilization or CPU utilization required by the vulnerability to be scanned, it indicates that the load information of the scanning engine does not satisfy the resource usage information, and a target scanning engine does not exist.

[0139] Step S900: If it exists, perform the step of determining the target scanning engine from multiple scanning engines; or, if it does not exist, add the vulnerability to be scanned to the target waiting queue and obtain the load information of multiple scanning engines based on a preset frequency.

[0140] In some embodiments, if there is no target scanning engine that meets the resource usage information, the vulnerability to be scanned is added to the target waiting queue, and the load information of each scanning engine is collected at a preset frequency until there is a target scanning engine whose load information meets the resource usage information.

[0141] By dividing each functional module according to its corresponding function, this disclosure provides a scanning device, which can be a server or a chip applied to a server. Figure 5 A schematic block diagram of the functional modules of a scanning apparatus provided for an exemplary embodiment of this disclosure. Figure 5 As shown, the scanning device 200 includes:

[0142] The acquisition module 201 is used to acquire vulnerability information and scanning environment information of the vulnerability to be scanned when it is necessary to scan for the vulnerability to be scanned.

[0143] Prediction module 202 is used to predict the resource consumption information required when scanning for vulnerabilities based on vulnerability information and scanning environment information;

[0144] The determination module 203 is used to obtain the current load information of multiple scanning engines, and based on the load information and resource usage information, determine the target scanning engine from the multiple scanning engines, and scan for vulnerabilities to be scanned using the target scanning engine.

[0145] In one possible implementation, the acquisition module 201 is further configured to acquire historical vulnerability information and historical scanning environment information corresponding to historical scanned vulnerabilities, and to acquire historical resource usage information of historical scanned vulnerabilities; wherein, the historical vulnerability information includes the historical scanned vulnerability name, historical scanned vulnerability type, and historical scanned resources corresponding to the historical scanned vulnerabilities; the above device further includes: a training module, configured to use the historical vulnerability information and historical scanning environment information as training samples, and use the historical resource usage information as labels for the training samples, to train a preset model to obtain a load balancing model; the determination module 203 is further configured to input the vulnerability information and scanning environment information into the load balancing model to obtain the resource usage information required when scanning the vulnerabilities to be scanned. The resource usage information includes CPU usage information, memory usage information, and usage duration, and the historical scanning environment information includes the Internet Protocol IP information, server information, and open port information of the historical scanned resources.

[0146] In one possible implementation, the training module is further configured to perform hash processing on the historical scanning environment information, the historical scanning vulnerability type list, and the historical scanning vulnerability list respectively, to obtain hash values ​​corresponding to the historical scanning environment information, the historical scanning vulnerability type list, and the historical scanning vulnerability list respectively; and use the hash values ​​as feature values ​​of training samples to train the preset model.

[0147] In one possible implementation, the determining module 203 is further configured to determine, based on load information and resource usage information, a candidate scanning engine whose load information satisfies the resource usage information from multiple scanning engines; predict the start time when each of the candidate scanning engines will begin scanning for the vulnerability to be scanned; and determine the target scanning engine from the candidate scanning engines based on the start time.

[0148] In one possible implementation, the acquisition module 201 is further configured to acquire the target scan duration of the vulnerability to be scanned; acquire the scan status of the vulnerability to be scanned during the process of the target scanning engine scanning the vulnerability to be scanned; acquire the scan duration of the target scanning engine scanning the vulnerability to be scanned when the scan status is incomplete; and the determination module 203 is further configured to control the target scanning engine to stop scanning the vulnerability to be scanned when the scan duration is longer than the target scan duration, and re-determine the target scanning engine to be scanned for scanning the vulnerability to be scanned.

[0149] In one possible implementation, the above apparatus further includes: a judgment module, used to determine whether the target scanning engine exists among the multiple scanning engines based on load information and resource usage information; if it exists, to perform the step of determining the target scanning engine from the multiple scanning engines; or, if it does not exist, to add the vulnerability to be scanned to the target waiting queue, and to obtain the load information of the multiple scanning engines based on a preset frequency.

[0150] This disclosure also provides an electronic device, including: at least one processor; a memory for storing processor-executable instructions; wherein the at least one processor is configured to execute the instructions to implement the methods disclosed in this disclosure.

[0151] Figure 6 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this disclosure. For example... Figure 6 As shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801. The processor 1801 can perform the corresponding steps in the methods disclosed in the embodiments of this disclosure.

[0152] The processor 1801 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this embodiment can be implemented by the integrated logic circuitry in the processor 1801 or by software instructions. The processor 1801 can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 1802, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 1801 reads information from the memory 1802 and, in conjunction with its hardware, completes the steps of the method described above.

[0153] Furthermore, various operations / processes according to this disclosure, implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 7 The computer system 1900 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including those described above. Figure 7 A block diagram of a computer system provided for an exemplary embodiment of this disclosure.

[0154] Computer System 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0155] like Figure 7 As shown, the computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. The RAM 1903 may also store various programs and data required for the operation of the computer system 1900. The computing unit 1901, ROM 1902, and RAM 1903 are interconnected via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0156] Multiple components in computer system 1900 are connected to I / O interface 1905, including: input unit 1906, output unit 1907, storage unit 1908, and communication unit 1909. Input unit 1906 can be any type of device capable of inputting information into computer system 1900. Input unit 1906 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1908 may include, but is not limited to, hard disks and optical disks. Communication unit 1909 allows computer system 1900 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0157] The computing unit 1901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1800 via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 can be configured to perform the methods disclosed in this disclosure by any other suitable means (e.g., by means of firmware).

[0158] This disclosure also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this disclosure.

[0159] The computer-readable storage medium in this disclosure can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include electrical connections based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0160] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0161] This disclosure also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the methods disclosed in the embodiments of this disclosure.

[0162] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0164] The modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0165] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0166] The above description is merely an embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0167] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A scanning method, characterized in that, The method includes: When it is necessary to scan for a vulnerability, the vulnerability information and scanning environment information of the vulnerability to be scanned are obtained. The vulnerability information includes the vulnerability name, vulnerability type and the resource to be scanned corresponding to the vulnerability. The scanning environment information includes the IP information, server information and open port information of the resource to be scanned. Based on the vulnerability information and the scanning environment information, predict the resource consumption information required when scanning the vulnerability to be scanned; The system obtains the current load information corresponding to multiple scanning engines, and based on the load information and the resource usage information, determines the target scanning engine from the multiple scanning engines, and scans the vulnerability to be scanned using the target scanning engine. The step of predicting the resource consumption information required to scan the vulnerability based on the vulnerability information and the scanning environment information includes: Obtain historical vulnerability information and historical scanning environment information corresponding to the historical scanned vulnerabilities, and obtain historical resource usage information of the historical scanned vulnerabilities; wherein, the historical vulnerability information includes the historical scanned vulnerability name, the historical scanned vulnerability type, and the historical scanned resources corresponding to the historical scanned vulnerability; The historical vulnerability information and historical scanning environment information are used as training samples, and the historical resource usage information is used as the label of the training samples to train the preset model and obtain the load balancing model. The vulnerability information and the scanning environment information are input into the load balancing model to obtain the resource consumption information required to scan the vulnerability to be scanned; The step of determining the target scanning engine from the plurality of scanning engines based on the load information and the resource usage information includes: Based on the load information and the resource usage information, a candidate scanning engine whose load information meets the resource usage information is determined from the plurality of scanning engines; Predict the start time when each of the candidate scanning engines will begin scanning for the vulnerabilities to be scanned; Based on the start time, a target scanning engine is determined from the candidate scanning engines, and the target scanning engine is the candidate scanning engine whose start time is closest to the current time.

2. The method according to claim 1, characterized in that, The resource usage information includes CPU usage information, memory usage information, and usage duration. The historical scanning environment information includes Internet Protocol (IP) information, server information, and open port information of the historical scanned resources.

3. The method according to claim 1, characterized in that, The historical vulnerability information includes a list of historically scanned vulnerabilities and a list of historically scanned vulnerability types; the training of the preset model includes: The historical scanning environment information, the historical scanning vulnerability type list, and the historical scanning vulnerability list are respectively hashed to obtain the hash values ​​corresponding to the historical scanning environment information, the historical scanning vulnerability type list, and the historical scanning vulnerability list. The hash value is used as the feature value of the training sample to train the preset model.

4. The method according to claim 1, characterized in that, The method further includes: Obtain the target scan duration for the vulnerability to be scanned; During the process of the target scanning engine scanning the vulnerability to be scanned, the scanning status of the vulnerability to be scanned is obtained; If the scan status is incomplete, obtain the scan duration of the target scanning engine scanning the vulnerability to be scanned; If the scan duration exceeds the target scan duration, the target scanning engine is controlled to stop scanning the vulnerability to be scanned, and a new target scanning engine is determined for scanning the vulnerability to be scanned.

5. The method according to claim 1, characterized in that, The method further includes: Based on the load information and the resource usage information, it is determined whether the target scanning engine exists among the multiple scanning engines; If it exists, perform the step of determining the target scanning engine from the plurality of scanning engines; Alternatively, if the vulnerability to be scanned does not exist, add it to the target waiting queue and obtain the load information of multiple scanning engines based on a preset frequency.

6. A scanning device, characterized in that, The device includes: The acquisition module is used to acquire vulnerability information and scanning environment information of the vulnerability to be scanned when it is necessary to scan for the vulnerability to be scanned. The vulnerability information to be scanned includes the vulnerability name, vulnerability type and the resource to be scanned corresponding to the vulnerability to be scanned. The scanning environment information includes the IP information, server information and open port information of the resource to be scanned. The prediction module is used to predict the resource consumption information required when scanning the vulnerability to be scanned, based on the vulnerability information and the scanning environment information. The determination module is used to obtain the load information corresponding to multiple scanning engines, and based on the load information and the resource usage information, determine the target scanning engine from the multiple scanning engines, and scan the vulnerability to be scanned through the target scanning engine; The step of predicting the resource consumption information required to scan the vulnerability based on the vulnerability information and the scanning environment information includes: Obtain historical vulnerability information and historical scanning environment information corresponding to the historical scanned vulnerabilities, and obtain historical resource usage information of the historical scanned vulnerabilities; wherein, the historical vulnerability information includes the historical scanned vulnerability name, the historical scanned vulnerability type, and the historical scanned resources corresponding to the historical scanned vulnerability; The historical vulnerability information and historical scanning environment information are used as training samples, and the historical resource usage information is used as the label of the training samples to train the preset model and obtain the load balancing model. The vulnerability information and the scanning environment information are input into the load balancing model to obtain the resource consumption information required to scan the vulnerability to be scanned; The step of determining the target scanning engine from the plurality of scanning engines based on the load information and the resource usage information includes: Based on the load information and the resource usage information, a candidate scanning engine whose load information meets the resource usage information is determined from the plurality of scanning engines; Predict the start time when each of the candidate scanning engines will begin scanning for the vulnerabilities to be scanned; Based on the start time, a target scanning engine is determined from the candidate scanning engines, and the target scanning engine is the candidate scanning engine whose start time is closest to the current time.

7. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method as described in any one of claims 1-5.

9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 5.

Images