Vulnerability scanning method and device, electronic equipment and computer readable storage medium

Abstract

The application discloses a vulnerability scanning method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of computer Web service applications. A plurality of idle threads are concurrently started to simultaneously start a plurality of scanning tasks and simultaneously perform vulnerability scanning on a plurality of services. When it is detected that a scanning task is not started, a task identifier of the current task is stored in a task identifier pool, and recycling of the task identifier that is not started is realized. Scanning results of the scanning tasks are obtained to obtain vulnerability information. In this way, the application concurrently operates through a plurality of threads, simultaneously performs vulnerability scanning on a plurality of services, is high in efficiency, and manages the task identifiers through the task identifier pool, recycles the task identifiers that are not started, can reduce the overhead of creating the task identifiers, and improves the resource utilization rate of the task identifiers.

Description

Technical Field

[0001] This application relates to the field of computer web service application technology, and in particular to a vulnerability scanning method, apparatus, electronic device and computer-readable storage medium. Background Technology

[0002] With the advancement of enterprise digital transformation, the number of internal web service systems has exploded. These systems commonly suffer from security vulnerabilities such as SQL injection—attackers can inject malicious SQL statements to steal sensitive database information, tamper with business data, and even control servers, posing a serious threat to enterprise information security. Therefore, enterprise security teams need to regularly scan all business systems for vulnerabilities to promptly identify and fix security risks. Current vulnerability scanning methods typically target only one target at a time, making them very inefficient. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a vulnerability scanning method, apparatus, electronic device, and computer-readable storage medium, which can perform vulnerability scanning on multiple services simultaneously through concurrent operation of multiple threads, resulting in high efficiency.

[0004] The vulnerability scanning method according to the first aspect of this application includes: Obtain the task configuration file, extract information from the task configuration file, and obtain the task configuration list; Based on the task configuration list, multiple vulnerability detection commands are constructed; Access the creation interface of the scanning engine to obtain multiple task identifiers, and store each task identifier in a pre-built task identifier pool; A task identifier is obtained from the task identifier pool, a scanning task is constructed based on the task identifier and the vulnerability detection instruction, and the task identifiers used to construct the scanning task are removed from the task identifier pool; wherein, each task identifier corresponds to one vulnerability detection instruction; Idle threads are identified from the thread pool, and a correspondence is established between the idle threads and the scanning tasks, with each idle thread corresponding to one scanning task; Concurrently start idle threads, and each idle thread calls the scanning interface of the scanning engine based on the task identifier of the corresponding scanning task and the vulnerability detection instruction to start the corresponding scanning task; The status of the scanning task corresponding to each thread is polled to obtain the status polling result; If the status polling result indicates that the current task has not started, the task identifier of the current task is stored in the task identifier pool. Obtain the scanning results of each scanning task, perform vulnerability analysis on each scanning result, and obtain vulnerability information.

[0005] The vulnerability scanning method according to the embodiments of this application has at least the following beneficial effects: The vulnerability scanning method of this application first extracts information from the task configuration file to obtain a task configuration list. The task configuration list includes relevant information used to construct vulnerability detection instructions, such as the URL information of the services to be scanned. Then, it accesses the creation interface of the scanning engine to obtain multiple task identifiers, and stores each task identifier in a pre-built task identifier pool. Task identifiers are retrieved from the task identifier pool, and scanning tasks are constructed based on the task identifiers and vulnerability detection instructions. Task identifiers already used to construct scanning tasks are removed from the task identifier pool to avoid conflicts caused by the reuse of task identifiers. Then, a correspondence is established between idle threads and scanning tasks, with each idle thread corresponding to one scanning task. Then, multiple idle threads are started concurrently to start multiple scanning tasks simultaneously and perform vulnerability scanning on multiple services at the same time, improving vulnerability scanning efficiency. During the execution of the scanning task, the status of each scanning task is periodically polled to obtain the status polling result. If the status polling result indicates that the current task has not started, it means that the task identifier of the current task has not started and can be reused. Therefore, the task identifier of the current task is stored in the task identifier pool, realizing the recycling of unstarted task identifiers. The scan results of each scanning task are obtained, and vulnerability analysis is performed on each scan result to obtain vulnerability information. Thus, this application utilizes multiple threads to perform concurrent vulnerability scans on multiple services simultaneously, resulting in high efficiency. Furthermore, by managing task identifiers through a task identifier pool and reclaiming inactive task identifiers, the overhead of creating task identifiers can be reduced, improving the resource utilization of task identifiers.

[0006] According to some embodiments of this application, after polling the status of the scanning task corresponding to each thread and obtaining the status polling result, the method further includes: If the status polling result indicates that the current task is being executed, the current task has been completed normally, or the current task has encountered an error, the vulnerability detection instruction of the current task will be marked as an executed instruction. The number of executed instructions is determined as the number of executed instructions; Based on the number of vulnerability detection commands and the number of commands executed, determine the number of commands that were not executed; If the number of unexecuted instructions is greater than the number of task identifiers in the task identifier pool, a new task identifier is created and stored in the task identifier pool. Then, the process jumps to obtaining a task identifier from the task identifier pool, constructing a scanning task based on the task identifier and the vulnerability detection instructions, and removing the task identifiers already used to construct the scanning task from the task identifier pool.

[0007] According to some embodiments of this application, after polling the status of each initiated scanning task and obtaining the status polling result, the method further includes: If the status polling result indicates that the current task has been completed normally, the status flag of the current task will be set to normal completion. If the status polling result indicates that the current task is executing incorrectly, the status flag of the current task is set to "task error".

[0008] According to some embodiments of this application, the vulnerability detection instructions include scan levels; After polling the status of each initiated scanning task and obtaining the status polling result, the process further includes: If the status polling result indicates that a current task is being executed, the current execution duration of the current task is determined. Obtain the scan level from the vulnerability detection instruction of the current task, and determine the comparison duration based on the scan level; If the execution time is longer than the comparison time, the status flag of the current task is set to timeout.

[0009] According to some embodiments of this application, after obtaining the vulnerability information, the method further includes: Based on the vulnerability information and status markers corresponding to each scanning task, a vulnerability visualization report is generated; The vulnerability visualization report was uploaded to the cloud server.

[0010] According to some embodiments of this application, after storing the task identifier of the current task in the task identifier pool, the method further includes: Mark the vulnerability detection instruction of the current task as a startup failure instruction; After obtaining the scanning results of each of the scanning tasks, performing vulnerability analysis on each of the scanning results, and obtaining vulnerability information, the method further includes: If an idle thread is detected in the thread pool, a task identifier is obtained from the task identifier pool, a scanning task is constructed based on the task identifier and the start failure instruction, and the task identifiers used to construct the scanning task are removed from the task identifier pool; then, the process jumps to determining an idle thread from the thread pool and establishing a correspondence between the idle thread and the scanning task; wherein each task identifier corresponds to one start failure instruction.

[0011] According to some embodiments of this application, before the concurrent startup of the idle thread, the following is included: The scanning engine is then tested; If the scanning interface of the scanning engine is detected to be in an inactive state, a start command is sent to the scanning engine.

[0012] A second aspect of this application provides a vulnerability scanning apparatus, comprising: The first acquisition module is used to acquire the task configuration file, extract information from the task configuration file, and obtain a task configuration list. The first construction module is used to construct multiple vulnerability detection instructions based on the task configuration list; The access module is used to access the creation interface of the scanning engine, obtain multiple task identifiers, and store each task identifier in a pre-built task identifier pool. The second construction module is used to obtain task identifiers from the task identifier pool, construct a scanning task based on the task identifiers and the vulnerability detection instructions, and remove task identifiers that have been used to construct the scanning task from the task identifier pool; wherein, each task identifier corresponds to one vulnerability detection instruction; The thread determination module is used to determine idle threads from the thread pool and establish a correspondence between idle threads and the scanning tasks, with each idle thread corresponding to one scanning task; The concurrency module is used to concurrently start idle threads. Each idle thread calls the scanning interface of the scanning engine based on the task identifier of the corresponding scanning task and the vulnerability detection instruction to start the corresponding scanning task. The polling module is used to poll the status of the scanning task corresponding to each thread and obtain the status polling result. The identifier recycling module is used to store the task identifier of the current task in the task identifier pool when the status polling result indicates that the current task has not started. The second acquisition module is used to acquire the scanning results of each of the scanning tasks, perform vulnerability analysis on each of the scanning results, and obtain vulnerability information.

[0013] A third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the vulnerability scanning method described in any one of the first aspects of the embodiment.

[0014] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vulnerability scanning method described in any one of the first aspects of the embodiment.

[0015] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0016] The present application will be further described below with reference to the accompanying drawings and embodiments, wherein: Figure 1 This is a flowchart illustrating the steps of the vulnerability scanning method according to an embodiment of this application; Figure 2 This is a flowchart illustrating the steps of a vulnerability scanning method according to another embodiment of this application; Figure 3 This is a schematic flowchart of a step following step S170 in the vulnerability scanning method of this application embodiment; Figure 4 This is a sub-process of the vulnerability scanning method in this application embodiment before step S160; Figure 5 This is a schematic diagram of the functional modules of the vulnerability scanning device according to an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0017] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0018] In the description of this application, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0019] In the description of this application, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0020] In the description of this application, unless otherwise expressly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0021] In the description of this application, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0022] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0023] The first aspect of this application provides a vulnerability scanning method. The vulnerability scanning method can be deployed and executed on a terminal, on a server, or as software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the vulnerability scanning method, but is not limited to the above forms.

[0024] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0025] Reference Figure 1 , Figure 1 This is a flowchart illustrating the steps of a vulnerability scanning method according to an embodiment of this application. The vulnerability scanning method in this embodiment may include, but is not limited to, steps S110 to S190. Step S110: Obtain the task configuration file, extract information from the task configuration file, and obtain the task configuration list; In some embodiments, the vulnerability scanning method of this application is applied to an electronic device that stores task configuration files pre-uploaded by relevant technical personnel. Therefore, the task configuration files can be obtained from the electronic device. The task configuration file can be a CSV file. Information is extracted from the CSV file using the csv.DictReader module in the Python standard library to obtain a task configuration list. The task configuration list includes multiple items, each being the URL of the service to be scanned, and also includes the request method (GET / POST), POST data, request headers, cookies, specified test parameters (param), scan level (level), risk level (risk), notes (note), Basic Auth, username (auth_user), and password (auth_pass). The task configuration list may also include other information, such as a task number, to distinguish different scanning tasks. This application does not limit this; those skilled in the art can set it according to actual circumstances.

[0026] It should be noted that CSV (Comma-Separated Values) is a common plain text file format used to store tabular data (such as data in spreadsheets or databases). It is known for its simple structure, broad compatibility, and readability, and is widely used for data exchange, backup, and programming.

[0027] Step S120: Based on the task configuration list, construct multiple vulnerability detection instructions; It is worth noting that the vulnerability detection instruction refers to the payload. In a vulnerability detection scenario, the payload is a piece of test data / instruction used to verify whether a target has a specific vulnerability. It is injected into the parameters, POST forms, request headers, etc., of the target URL, and the existence of a vulnerability is determined by observing the target's response. It should be noted that each URL corresponds to one vulnerability detection instruction. In one embodiment, a standardized vulnerability detection instruction payload is generated based on information in the task configuration list, supporting the passing of cookies, POST data, custom headers, and BasicAuth authentication information (converting the authentication information into a Base64 encoded Authorization header); it also supports specifying test parameters (testParameter) for targeted scanning. It should be noted that the vulnerability scanning method of this application is based on a scanning engine. This application does not limit the specific method for generating the vulnerability detection instruction, but the vulnerability detection instruction must conform to the standards of the scanning engine. For example, the scanning engine is SQLMap, an open-source web application SQL injection vulnerability detection tool that supports command-line mode and API service mode. Therefore, the payload in this embodiment of the application must conform to the requirements of SQLMap.

[0028] It's worth noting that in vulnerability detection commands, the risk level (risk) characterizes the risk level of the payload used by SQLMap. The higher the risk level, the more likely the payload is to negatively impact the target system (such as increasing database load or triggering data modification), but it also makes it easier to discover high-risk injection vulnerabilities (such as stacked query injection). The scan level (level) characterizes the comprehensiveness of SQLMap's scan (also called "probing depth"). The higher the level, the more types of injection points SQLMap will try, more variations of the payload, and send more requests, making it easier to discover hidden SQL injection vulnerabilities, but the scan time will also increase exponentially.

[0029] It should be noted that a URL (Uniform Resource Locator) is a string address used to identify and locate resources on the Internet, often referred to as a "web address". Since the vulnerability scanning method in this application targets web application services, the URL is the address of the web application service, and each web application service corresponds to one URL.

[0030] Step S130: Access the creation interface of the scanning engine to obtain multiple task identifiers, and store each task identifier in a pre-built task identifier pool. It is worth noting that the scanning engine is SQLMap. Therefore, task identifiers can be obtained by accessing the ` / task / new` interface of SQLMap. Each access to the ` / task / new` interface of SQLMap yields a unique task identifier, and these multiple task identifiers are then stored in a pre-built task identifier pool. It should be noted that this embodiment does not limit the number of task identifiers obtained in step S130. Those skilled in the art can set the number of task identifiers obtained in step S130 according to actual circumstances. For example, subsequent steps in this embodiment execute the scanning task by concurrently starting threads; therefore, the number of task identifiers obtained in step S130 can be set to the number of concurrent threads.

[0031] Step S140: Obtain task identifiers from the task identifier pool, construct a scanning task based on the task identifiers and vulnerability detection instructions, and remove task identifiers that have been used to construct scanning tasks from the task identifier pool; wherein, each task identifier corresponds to one vulnerability detection instruction. It is worth noting that scanning tasks are constructed by associating task identifiers with vulnerability detection instructions, with each task identifier corresponding to a specific vulnerability detection instruction. Furthermore, removing task identifiers already used to construct scanning tasks from the task identifier pool can prevent conflicts caused by the reuse of task identifiers, thus enabling dynamic management of the task identifier pool.

[0032] Step S150: Idle threads are determined from the thread pool, and a correspondence is established between idle threads and scanning tasks, with each idle thread corresponding to one scanning task. It's worth noting that a thread pool is created using `concurrent.futures.ThreadPoolExecutor`, and a concurrency threshold is set. The concurrency threshold refers to the number of threads that can run concurrently; typically, the number of threads in the pool equals the concurrency threshold. `concurrent.futures.ThreadPoolExecutor` is a class in the Python standard library's `concurrent.futures` class, used to execute multiple tasks concurrently using a thread pool, especially suitable for handling I / O-intensive tasks (such as file read / write, network requests, etc.). It significantly improves performance and resource utilization by reusing threads, reducing the overhead of creating and destroying threads.

[0033] Step S160: Concurrently start idle threads. Each idle thread calls the scanning interface of the scanning engine based on the task identifier and vulnerability detection instructions of the corresponding scanning task to start the corresponding scanning task. It's worth noting that concurrent idle threads are started. Each idle thread calls the ` / scan / {taskid} / start` interface of SQLMap via a POST request, where `taskid` represents the task identifier. This sends the corresponding vulnerability detection command to SQLMap, enabling SQLMap to scan the URLs in the vulnerability detection command for vulnerabilities. Multiple idle threads running simultaneously can scan multiple URLs at the same time, thus improving the efficiency of vulnerability scanning.

[0034] It should be noted that after an idle thread is started, it becomes a non-idle thread. When the scanning task corresponding to the non-idle thread is in a state where the current task has not started or the current task has been completed normally, the non-idle thread becomes an idle thread.

[0035] Step S170: Perform status polling on the scanning task corresponding to each thread to obtain the status polling result; Step S180: If the status polling result indicates that the current task has not started, store the task identifier of the current task in the task identifier pool. It's worth noting that for each scan task, the ` / scan / {taskid} / status` interface of SQLMap is accessed every three seconds to obtain the status polling result of the scan task, where `taskid` is the task identifier. The status polling result is one of the following: running, terminated, not running, or error. `running` indicates that the current task is executing. `terminated` indicates that the current task has completed normally. `not running` indicates that the current task has not started. `error` indicates that the current task encountered an error.

[0036] In some embodiments, network failures or issues such as the SQLMap API service becoming unresponsive due to sudden high load may cause threads to fail to start scanning tasks normally, resulting in a status polling result of "not running". Since scanning tasks with a status polling result of "not running" have not started, their task identifiers are also not used. Therefore, they are placed in the task identifier pool for use by subsequent new scanning tasks. In this way, the recycling of unstarted task identifiers can reduce the overhead of creating task identifiers and improve the resource utilization of task identifiers.

[0037] Step S190: Obtain the scanning results of each scanning task, perform vulnerability analysis on each scanning result, and obtain vulnerability information.

[0038] It is worth noting that in step S190, for each scanning task, the ` / scan / {taskid} / data` interface is called to obtain the raw scan result data. Then, the raw scan result data is parsed. If vulnerability data exists, the vulnerability information is extracted, translated from Chinese to English using a vulnerability name mapping library, and deduplicated to generate a standardized vulnerability title list. The vulnerability status of the scanning task is marked as "vuln". If no vulnerability is detected, the vulnerability status of the scanning task is marked as "safe". The task number, URL, remarks, vulnerability status, error message, scan time, reproduction command, and complete result JSON of the scanning task are appended to a JSONL format storage file as vulnerability information.

[0039] It is worth noting that the vulnerability scanning method in this embodiment of the application, through steps S110 to S190, first extracts information from the task configuration file to obtain a task configuration list. The task configuration list includes relevant information used to construct vulnerability detection instructions, such as the URL information of the services that need to be scanned. Then, it accesses the creation interface of the scanning engine to obtain multiple task identifiers, and stores each task identifier in a pre-built task identifier pool. Task identifiers are retrieved from the task identifier pool, and scanning tasks are constructed based on the task identifiers and vulnerability detection instructions. Task identifiers already used to construct scanning tasks are removed from the task identifier pool to avoid conflicts caused by repeated use of task identifiers. Then, a correspondence is established between idle threads and scanning tasks, with each idle thread corresponding to one scanning task. Then, multiple idle threads are started concurrently to simultaneously start multiple scanning tasks and perform vulnerability scanning on multiple services, improving vulnerability scanning efficiency. During the execution of scanning tasks, the status of each scanning task is periodically polled to obtain the status polling results. If the status polling result indicates that the current task is not started, it means that the task identifier of the current task is not started and can be reused. Therefore, the task identifier of the current task is stored in the task identifier pool, realizing the reclamation of inactive task identifiers. The scanning results of each scanning task are obtained, and vulnerability analysis is performed on each scanning result to obtain vulnerability information. In this way, this application performs vulnerability scanning on multiple services simultaneously through concurrent operation of multiple threads, which is highly efficient. Furthermore, by managing task identifiers through the task identifier pool and reclaiming inactive task identifiers, the overhead of creating task identifiers can be reduced, and the resource utilization of task identifiers can be improved.

[0040] In some embodiments, refer to Figure 2 , Figure 2 This is a flowchart illustrating the steps of a vulnerability scanning method according to another embodiment of this application. Figure 2 The illustrated method steps include, but are not limited to, steps S210 to S240.

[0041] Step S210: If the status polling result indicates that the current task is being executed, the current task has been completed normally, or the current task has encountered an error, mark the vulnerability detection instruction of the current task as an executed instruction. Step S220: Determine the number of executed instructions, as the number of executed instructions; Step S230: Based on multiple vulnerability detection commands and the number of commands executed, determine the number of commands that have not been executed; Step S240: If the number of unexecuted instructions is greater than the number of task identifiers in the task identifier pool, create a new task identifier and store the new task identifier in the task identifier pool. Then, proceed to obtain a task identifier from the task identifier pool, construct a scanning task based on the task identifier and the vulnerability detection instructions, and remove the task identifiers that have been used to construct the scanning task from the task identifier pool.

[0042] It is worth noting that when the status polling result is "current task is executing," "current task has completed normally," or "current task has encountered an error," it indicates that the current task has been started, meaning the vulnerability detection instruction corresponding to the current task has been executed. Therefore, the total number of vulnerability detection instructions - the number of executed instructions = the number of unexecuted instructions. If the number of unexecuted instructions is less than the number of task identifiers in the task identifier pool, there is no need to create new task identifiers. If the number of unexecuted instructions is greater than the number of task identifiers in the task identifier pool, it means that the number of task identifiers in the task identifier pool is currently insufficient. Therefore, it is necessary to create new task identifiers and jump to step S140 to start the scanning task corresponding to the unexecuted instruction based on the unexecuted instruction. This enables vulnerability scanning of the URL of each vulnerability detection instruction. By dynamically managing the task identifiers in the task identifier pool, it is possible to avoid missed scans while avoiding excessive overhead caused by creating too many task identifiers.

[0043] In some embodiments, after step S170, where the status of each initiated scanning task is polled and the status polling result is obtained, steps S171 and S172 are further included. (Refer to...) Figure 3 , Figure 3 This is a schematic flowchart of a step following step S170 in the vulnerability scanning method of this application embodiment.

[0044] Step S171: If the status polling result indicates that the current task has been completed normally, set the status flag of the current task to complete normally. Step S172: If the status polling result indicates that the current task is executing incorrectly, set the status flag of the current task to "task error".

[0045] It is worth noting that, through steps S171 and S172, when performing vulnerability scanning, not only is the information in the scanning results of the scanning task recorded, but also the status of the scanning task during execution is recorded, thus recording more comprehensive information.

[0046] In some embodiments, refer to Figure 3 After step 170, steps S173 to S175 may also be included, but are not limited to.

[0047] Step S173: If the status polling result indicates that the current task is being executed, determine the current execution duration of the current task; Step S174: Obtain the scan level in the vulnerability detection instruction of the current task, and determine the comparison duration based on the scan level; Step S175: If the execution time is longer than the comparison time, set the status flag of the current task to timeout.

[0048] It is worth noting that in step S174, the method for calculating the duration to be compared is as follows: Comparison duration = Base duration × (1 + (scanning level - 1) × weighting coefficient).

[0049] It is worth noting that the base duration and weighting coefficient are preset values, and the scanning level is a parameter in the vulnerability detection command. In this embodiment, through steps S173 to S175, the comparison duration is calculated separately for different scanning tasks. The comparison duration is different for scanning tasks of different scanning levels to determine whether different scanning tasks have timed out. Compared with related technologies that use the same duration to determine whether all scanning tasks have timed out, this application provides a more accurate determination of timeout.

[0050] In some embodiments, after step S190, there may be steps S191 and S192, among others.

[0051] Step S191: Generate a vulnerability visualization report based on the vulnerability information and status markers corresponding to each scanning task; Step S192: Upload the vulnerability visualization report to the cloud server.

[0052] It's worth noting that the vulnerability information and status markers corresponding to each scanning task are used to generate a vulnerability visualization report. This report is in HTML format. The HTML report is saved to the user-specified output directory and then uploaded to Huawei Cloud OBS to obtain a publicly accessible link. Then, using the DingTalk robot webhook interface, a Markdown-formatted notification message is generated based on the HTML report, containing information such as the number of vulnerabilities, the report link, and the scan time. This Markdown notification is sent to a designated group. After the notification is sent, the SQLMap API service process is terminated, system resources are released, and the entire scanning process ends.

[0053] In some embodiments, after storing the task identifier of the current task in the task identifier pool, the method further includes: Mark the vulnerability detection command for the current task as a startup failure command; After obtaining the scan results from each scan task, performing vulnerability analysis on each scan result, and obtaining vulnerability information, the process also includes: If an idle thread is detected in the thread pool, a task identifier is obtained from the task identifier pool. A scan task is constructed based on the task identifier and the start failure instruction, and the task identifiers used to construct the scan task are removed from the task identifier pool. The process then jumps to determine an idle thread from the thread pool and establishes a correspondence between the idle thread and the scan task. Each task identifier corresponds to a start failure instruction.

[0054] It is worth noting that when the status polling result indicates that the current task has not started, the task identifier of the current task is stored in the task identifier pool, and the vulnerability detection instruction of the current task is marked as a startup failure instruction. After step S190, the task identifier is retrieved from the task identifier pool, and a new scanning task is constructed based on the task identifier and the startup failure instruction. Then, based on the new scanning task, the process jumps to step S150 to perform vulnerability scanning on the URLs in the vulnerability detection instructions marked as startup failure instructions. This avoids the situation where URLs are missed due to task startup failure.

[0055] In some embodiments, refer to Figure 4 , Figure 4 This is a sub-process of the vulnerability scanning method in this application embodiment before step S160. Figure 4 The illustrated vulnerability scanning method may include, but is not limited to, steps S410 and S420.

[0056] Step S410: Detect the scanning engine; Step S420: If the scanning interface of the scanning engine is detected to be in an unstarted state, a start command is sent to the scanning engine.

[0057] It's worth noting that a probe request is sent via the ` / task / new` interface at the specified local or remote SQLMapAPI address (default 127.0.0.1:8775). If a 200 status code is returned with a valid taskd, the scanning engine service is considered alive. If the request times out or returns a non-200 status code, the `sqlmapapi.py` script path is automatically located, the SQLMapAPI service process is started via the `subprocess` module, and a 15-second polling wait mechanism is initiated, checking the service status every second until the service is ready. If the service is not ready within 15 seconds, the startup is considered to have failed, the program is terminated, and an error message is displayed.

[0058] A second aspect of this application provides a vulnerability scanning apparatus. (Refer to...) Figure 5 , Figure 5 This is a functional module diagram of a vulnerability scanning device according to an embodiment of this application. The vulnerability scanning device includes: The first acquisition module 510 is used to acquire the task configuration file, extract information from the task configuration file, and obtain the task configuration list. The first building module 520 is used to build multiple vulnerability detection instructions based on the task configuration list; Access module 530 is used to access the creation interface of the scanning engine, obtain multiple task identifiers, and store each task identifier in a pre-built task identifier pool. The second construction module 540 is used to obtain task identifiers from the task identifier pool, construct scanning tasks based on task identifiers and vulnerability detection instructions, and remove task identifiers that have been used to construct scanning tasks from the task identifier pool; wherein, each task identifier corresponds to a vulnerability detection instruction. The thread determination module 550 is used to determine idle threads from the thread pool and establish a correspondence between idle threads and scanning tasks, with each idle thread corresponding to one scanning task. The concurrency module 560 is used to concurrently start idle threads. Each idle thread calls the scanning interface of the scanning engine based on the task identifier and vulnerability detection instructions of the corresponding scanning task to start the corresponding scanning task. The polling module 570 is used to poll the status of the scanning task corresponding to each thread and obtain the status polling result. The identifier recycling module 580 is used to store the task identifier of the current task in the task identifier pool when the status polling result indicates that the current task has not started. The second acquisition module 590 is used to acquire the scanning results of each scanning task, perform vulnerability analysis on each scanning result, and obtain vulnerability information.

[0059] The vulnerability scanning apparatus of the second aspect of this application is used to execute the vulnerability scanning method of the first aspect of this application. When executing the method, information is first extracted from the task configuration file to obtain a task configuration list. The task configuration list includes relevant information for constructing vulnerability detection instructions, such as the URL information of the services to be scanned. Then, the creation interface of the scanning engine is accessed to obtain multiple task identifiers, which are stored in a pre-built task identifier pool. Task identifiers are retrieved from the task identifier pool, and scanning tasks are constructed based on the task identifiers and vulnerability detection instructions. Task identifiers already used to construct scanning tasks are removed from the task identifier pool to avoid conflicts caused by repeated use of task identifiers. Then, a correspondence is established between idle threads and scanning tasks, with each idle thread corresponding to one scanning task. Then, multiple idle threads are started concurrently to simultaneously start multiple scanning tasks and perform vulnerability scanning on multiple services, improving vulnerability scanning efficiency. During the execution of scanning tasks, the status of each scanning task is periodically polled to obtain the status polling results. If the status polling result indicates that the current task is not started, it means that the task identifier of the current task is not started and can be reused. Therefore, the task identifier of the current task is stored in the task identifier pool, realizing the reclamation of inactive task identifiers. The scanning results of each scanning task are obtained, and vulnerability analysis is performed on each scanning result to obtain vulnerability information. In this way, this application performs vulnerability scanning on multiple services simultaneously through concurrent operation of multiple threads, which is highly efficient. Furthermore, by managing task identifiers through the task identifier pool and reclaiming inactive task identifiers, the overhead of creating task identifiers can be reduced, and the resource utilization of task identifiers can be improved.

[0060] It should be noted that the specific implementation of this vulnerability scanning device is basically the same as the specific embodiments of the vulnerability scanning method described above, and will not be repeated here. Subject to meeting the requirements of the embodiments of this application, the vulnerability scanning device may also be equipped with other functional units to implement the vulnerability scanning method in the above embodiments.

[0061] A third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the vulnerability scanning method of any one of the first aspects of the embodiment. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0062] Reference Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device according to one embodiment. The electronic device includes: The processor 601 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 602 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 602 and is called and executed by the processor 601 using the vulnerability scanning method of the embodiments of this application. The input / output interface 603 is used to implement information input and output; The communication interface 604 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 605 transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604); The processor 601, memory 602, input / output interface 603, and communication interface 604 are connected to each other within the device via bus 605.

[0063] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vulnerability scanning method of any one of the first aspects of this application.

[0064] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0065] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0066] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0067] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0068] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0069] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0070] It should be understood that in this application, "at least one (item)" means one or more, and "more than one" means two or more. "And / or" is used to describe the mapping relationship between the mapped objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following mapped objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0071] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0072] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0073] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0074] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0075] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A vulnerability scanning method, characterized in that, include: Obtain the task configuration file, extract information from the task configuration file, and obtain the task configuration list; Based on the task configuration list, multiple vulnerability detection commands are constructed; Access the creation interface of the scanning engine to obtain multiple task identifiers, and store each task identifier in a pre-built task identifier pool; A task identifier is obtained from the task identifier pool, a scanning task is constructed based on the task identifier and the vulnerability detection instruction, and the task identifiers used to construct the scanning task are removed from the task identifier pool; wherein, each task identifier corresponds to one vulnerability detection instruction; Idle threads are identified from the thread pool, and a correspondence is established between the idle threads and the scanning tasks, with each idle thread corresponding to one scanning task; Concurrently start idle threads, and each idle thread calls the scanning interface of the scanning engine based on the task identifier of the corresponding scanning task and the vulnerability detection instruction to start the corresponding scanning task; The status of the scanning task corresponding to each thread is polled to obtain the status polling result; If the status polling result indicates that the current task has not started, the task identifier of the current task is stored in the task identifier pool. Obtain the scanning results of each scanning task, perform vulnerability analysis on each scanning result, and obtain vulnerability information.

2. The vulnerability scanning method according to claim 1, characterized in that, After polling the status of the scanning task corresponding to each thread and obtaining the status polling result, the process further includes: If the status polling result indicates that the current task is being executed, the current task has been completed normally, or the current task has encountered an error, the vulnerability detection instruction of the current task will be marked as an executed instruction. The number of executed instructions is determined as the number of executed instructions; Based on the number of vulnerability detection commands and the number of commands executed, determine the number of commands that were not executed; If the number of unexecuted instructions is greater than the number of task identifiers in the task identifier pool, a new task identifier is created and stored in the task identifier pool. Then, the process jumps to obtaining a task identifier from the task identifier pool, constructing a scanning task based on the task identifier and the vulnerability detection instructions, and removing the task identifiers already used to construct the scanning task from the task identifier pool.

3. The vulnerability scanning method according to claim 1, characterized in that, After polling the status of each initiated scanning task and obtaining the status polling result, the process further includes: If the status polling result indicates that the current task has been completed normally, the status flag of the current task will be set to normal completion. If the status polling result indicates that the current task is executing incorrectly, the status flag of the current task is set to "task error".

4. The vulnerability scanning method according to claim 3, characterized in that, The vulnerability detection command includes the scan level; After polling the status of each initiated scanning task and obtaining the status polling result, the process further includes: If the status polling result indicates that a current task is being executed, the current execution duration of the current task is determined. Obtain the scan level from the vulnerability detection instruction of the current task, and determine the comparison duration based on the scan level; If the execution time is longer than the comparison time, the status flag of the current task is set to timeout.

5. The vulnerability scanning method according to claim 4, characterized in that, After obtaining the vulnerability information, the following is also included: Based on the vulnerability information and status markers corresponding to each scanning task, a vulnerability visualization report is generated; The vulnerability visualization report was uploaded to the cloud server.

6. The vulnerability scanning method according to claim 1, characterized in that, After storing the task identifier of the current task in the task identifier pool, the method further includes: Mark the vulnerability detection instruction of the current task as a startup failure instruction; After obtaining the scanning results of each of the scanning tasks, performing vulnerability analysis on each of the scanning results, and obtaining vulnerability information, the method further includes: If an idle thread is detected in the thread pool, a task identifier is obtained from the task identifier pool, a scanning task is constructed based on the task identifier and the start failure instruction, and the task identifiers used to construct the scanning task are removed from the task identifier pool; then, the process jumps to determining an idle thread from the thread pool and establishing a correspondence between the idle thread and the scanning task; wherein each task identifier corresponds to one start failure instruction.

7. The vulnerability scanning method according to claim 1, characterized in that, Before concurrently starting the idle thread, the following is included: The scanning engine is then tested; If the scanning interface of the scanning engine is detected to be in an inactive state, a start command is sent to the scanning engine.

8. A vulnerability scanning device, characterized in that, include: The first acquisition module is used to acquire the task configuration file, extract information from the task configuration file, and obtain a task configuration list. The first construction module is used to construct multiple vulnerability detection instructions based on the task configuration list; The access module is used to access the creation interface of the scanning engine, obtain multiple task identifiers, and store each task identifier in a pre-built task identifier pool. The second construction module is used to obtain task identifiers from the task identifier pool, construct a scanning task based on the task identifiers and the vulnerability detection instructions, and remove task identifiers that have been used to construct the scanning task from the task identifier pool; wherein, each task identifier corresponds to one vulnerability detection instruction; The thread determination module is used to determine idle threads from the thread pool and establish a correspondence between idle threads and the scanning tasks, with each idle thread corresponding to one scanning task; The concurrency module is used to concurrently start idle threads. Each idle thread calls the scanning interface of the scanning engine based on the task identifier of the corresponding scanning task and the vulnerability detection instruction to start the corresponding scanning task. The polling module is used to poll the status of the scanning task corresponding to each thread and obtain the status polling result. The identifier recycling module is used to store the task identifier of the current task in the task identifier pool when the status polling result indicates that the current task has not started. The second acquisition module is used to acquire the scanning results of each of the scanning tasks, perform vulnerability analysis on each of the scanning results, and obtain vulnerability information.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the vulnerability scanning method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the vulnerability scanning method according to any one of claims 1 to 7.

Images