A lightweight asset risk scanning method and device

Abstract

The present invention discloses a lightweight asset risk scanning method and device, which relates to the field of network security technology. The method includes: evenly distributing the acquired task initial data to multiple scanning probes according to the scanning capability execution sequence table; using the scanning probe to obtain the task scanning engine in the cloud scanning engine library on demand according to the data scheduling logic and the user's preset resource restrictions; checking whether the system's free resources meet the preset resource occupation restrictions of the task scanning engine, and if so, starting the task scanning engine according to the corresponding task parameters; storing the data results produced by the task scanning engine executing the task in the data scheduling database. The present invention can quickly expand the lightweight asset risk scanning architecture, adopt a distributed on-demand call system resource solution, so that the scanning capability is modularized, easy to expand and maintain, and the results of the scanning module can be expanded in a quick way, truly realizing lightweight asset risk scanning and rapid expansion of capabilities.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a lightweight asset risk scanning method and device. Background Art

[0002] Most of the current asset risk scanning architectures have the following problems and defects:

[0003] (1) The scanning and detection engine is fixed and the scanning and detection capabilities cannot be expanded quickly;

[0004] (2) The machine performance requirements are high and it cannot be deployed on ordinary computers, such as personal PCs, resulting in high computing costs and the inability to perform asset risk scanning at low cost;

[0005] (3) Scanning node deployment is complex and time-consuming, and new scanning detection nodes cannot be quickly deployed to respond to concurrent detection needs;

[0006] (4) The network environment requirements for scanning nodes are complex, and it is impossible to simultaneously manage the private nodes of multiple customers' multi-cloud environments for scanning and detection.

[0007] Therefore, the existing technology urgently needs a lightweight asset risk scanning architecture that can be rapidly expanded, so as to make asset risk scanning easier to use, more lightweight, lower the threshold for use, and facilitate expansion and maintenance. Summary of the invention

[0008] In view of the above-mentioned defects or deficiencies in the prior art, the present invention provides a lightweight asset risk scanning method and device, which can quickly expand the lightweight asset risk scanning architecture and adopt a distributed on-demand calling system resource solution to make the scanning capability modular and easy to expand and maintain. The results of the scanning module can be expanded in a quick way, truly realizing lightweight asset risk scanning and rapid expansion of capabilities.

[0009] One aspect of the present invention provides a lightweight asset risk scanning method, including: evenly distributing the acquired task initial data to multiple scanning probes according to a scanning capability execution order table; using the scanning probe to acquire the task scanning engine in the cloud scanning engine library on demand according to the data scheduling logic and user preset resource restrictions, checking whether the system free resources meet the preset resource occupancy restrictions of the task scanning engine, and if so, starting the task scanning engine according to the corresponding task parameters; storing the data results generated by the task scanning engine executing the task in the data scheduling database.

[0010] Furthermore, the step of using the scanning probe to obtain the task scanning engine in the cloud scanning engine library on demand according to the data scheduling logic and user preset resource restrictions includes: checking through the scanning engine launcher whether the local environment contains the scanning engine required for the task, and if so, determining whether the scanning engine in the local environment is the latest version, otherwise obtaining the corresponding scanning engine from the cloud scanning engine library to the local; if the scanning engine in the local environment is the latest version, performing a system resource usage check, otherwise obtaining the latest version of the scanning engine from the cloud scanning engine library to the local, and deleting the old version of the scanning engine in the local environment.

[0011] Furthermore, the step of checking whether the system's free resources meet the resource usage limit preset by the task scanning engine includes: checking whether the system's idle CPU meets the scanning engine resource requirements, if so, checking whether the system's idle memory meets the scanning engine resource requirements, otherwise waiting for the system's idle CPU to meet the resource requirements; if the system's idle memory meets the scanning engine resource requirements, starting the scanning engine, otherwise waiting for the system's idle memory to meet the resource requirements.

[0012] Furthermore, the task parameters include a task ID, a scanning engine scheduling parameter, a privileged mode and / or a task timeout.

[0013] Furthermore, the initial task data includes the business registration name, IP address, domain name information and / or website address of the enterprise.

[0014] Another aspect of the present invention provides a lightweight asset risk scanning device, including: a data scheduling module, configured to evenly distribute the acquired task initial data to multiple scanning probes according to a scanning capability execution order table; a scanning engine startup module, configured to use the scanning probe to acquire the task scanning engine in the cloud scanning engine library on demand according to the data scheduling logic and user preset resource restrictions; check whether the system free resources meet the preset resource occupancy restrictions of the task scanning engine, and if so, start the task scanning engine according to the corresponding task parameters; a scanning data storage module, configured to store the data results generated by the task scanning engine executing the task in a data scheduling database.

[0015] Furthermore, the scanning engine startup module is configured to check whether the local environment contains the scanning engine required for the task through the scanning engine launcher. If so, it determines whether the scanning engine in the local environment is the latest version, otherwise it obtains the corresponding scanning engine from the cloud scanning engine library to the local environment; if the scanning engine in the local environment is the latest version, it performs a system resource usage check, otherwise it obtains the latest version of the scanning engine from the cloud scanning engine library to the local environment and deletes the old version of the scanning engine in the local environment.

[0016] Furthermore, the scanning engine startup module is configured to: check whether the system's idle CPU meets the scanning engine resource requirements, if so, check whether the system's idle memory meets the scanning engine resource requirements, otherwise wait for the system's idle CPU to meet the resource requirements; if the system's idle memory meets the scanning engine resource requirements, start the scanning engine, otherwise wait for the system's idle memory to meet the resource requirements.

[0017] Furthermore, the task parameters include a task ID, a scanning engine scheduling parameter, a privileged mode and / or a task timeout.

[0018] Furthermore, the initial task data includes the business registration name, IP address, domain name information and / or website address of the enterprise.

[0019] The lightweight asset risk scanning method and device provided by the present invention have the following beneficial effects:

[0020] (1) The present invention starts the scanning engine on demand, which is convenient for users to use in low-performance hardware environments, greatly reducing the environmental requirements for asset risk detection, making it lighter, easier to use and maintain;

[0021] (2) The present invention modularizes all security capabilities and stores them in a cloud-based scanning engine library. When the corresponding task is started, the latest scanning engine is pulled, which greatly improves the scalability of asset risk scanning. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0023] Figure 1 This is a schematic diagram of a lightweight asset risk scanning method provided by an embodiment of the present application. Figure 1 ;

[0024] Figure 2 This is a schematic diagram of a lightweight asset risk scanning method provided by an embodiment of the present application. Figure 2 ;

[0025] Figure 3 It is a scanning engine acquisition logic schematic diagram of a lightweight asset risk scanning method provided by an embodiment of the present application;

[0026] Figure 4 It is a structural schematic diagram of a lightweight asset risk scanning device provided by an embodiment of the present application.

[0027] Figure 5 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0030] It should be understood that although the terms first, second, third, etc. may be used to describe the acquisition modules in the embodiments of the present invention, the acquisition modules should not be limited to these terms. These terms are only used to distinguish the acquisition modules from each other.

[0031] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0032] It should be noted that the directional words such as "upper", "lower", "left", and "right" described in the embodiments of the present invention are described at the angles shown in the drawings and should not be understood as limiting the embodiments of the present invention. In addition, in the context, it should also be understood that when it is mentioned that an element is formed "on" or "under" another element, it can not only be formed directly "on" or "under" another element, but also be formed "on" or "under" another element indirectly through an intermediate element.

[0033] The present invention decouples the architecture of asset risk scanning in a data-driven manner, thereby reducing the cost of asset risk scanning, and can meet the deployment and expansion requirements in complex scenarios in a lightweight manner, making asset risk scanning easier to use, expand and maintain.

[0034] See also Figure 1 , 2 An embodiment of the present application provides a lightweight asset risk scanning method, comprising the following steps:

[0035] Step S101 : Evenly distribute the acquired initial task data to a plurality of scanning probes according to a scanning capability execution sequence table.

[0036] Specifically, the user creates a main task and transfers the initial task data of the main task to the data driving module, wherein the initial task data includes but is not limited to the business registration name, IP address, domain name information, website address, etc.

[0037] The data scheduling module evenly distributes the initial task data input by the user to each scanning probe according to the scanning capability execution sequence table. The scanning capability execution sequence table is a database table, see Table 1.

[0038] Table 1

[0039] Scanning capability type Scan data source Scan Engine Scheduling Parameters enterprise_resaon user_input none subdomain_finder user_input,system_find subdomainFinder {value} port_scan user_input,system_find portscan {value} vulscan user_input,system_find vulScan {value} brute_service user_input,system_find bruiteService {value} website_baseline user_input,system_find websiteBaseline web_content user_input,system_find webContent {value} web_app_scan user_input,system_find webAppScan {value} waf_detecter user_input,system_find wafDetecter {value} ... ... ...

[0040] Among them, enterprise_resaon means enterprise industrial and commercial registration query; subdomain_finder means subdomain enumeration; port_scan means port detection; vulscan means vulnerability risk detection; brute_service means service weak password detection; website_baseline means website security baseline check; web_content means website content security detection; web_app_scan means website application identification; waf_detecter means website protection identification.

[0041] Among them, user_input represents user input, and system_find represents the scanning engine running discovery.

[0042] Step S102, according to the data scheduling logic and the user's preset resource limit, the task scanning engine in the cloud scanning engine library is obtained on demand using the scanning probe; check whether the system free resources meet the preset resource occupancy limit of the task scanning engine, and if so, start the task scanning engine according to the corresponding task parameters.

[0043] Specifically, the scanning engine launcher pulls the scanning engine from the cloud scanning engine library on demand according to the task parameters issued by the data scheduling, and checks whether the system free resources meet the resource usage limit preset by the scanning engine.

[0044] Figure 3 The following figure shows the decision logic for obtaining the scanning engine from the cloud engine library. Figure 3The step of pulling the task scanning engine from the cloud scanning engine library on demand includes: checking whether the local environment contains the scanning engine required by the task through the scanning engine launcher, and if so, determining whether the scanning engine in the local environment is the latest version, and if not, obtaining the corresponding scanning engine from the cloud scanning engine library to the local. If the scanning engine in the local environment is the latest version, then checking the system resource usage, otherwise obtaining the latest version of the scanning engine from the cloud scanning engine library to the local, and deleting the old version of the scanning engine in the local environment.

[0045] See also Figure 3 The steps of checking whether the system free resources meet the preset resource usage limit of the scanning engine include: checking whether the system idle CPU meets the resource requirements of the scanning engine, if so, checking whether the system idle memory meets the resource requirements of the scanning engine, otherwise waiting for the system idle CPU to meet the resource requirements; if the system idle memory meets the resource requirements of the scanning engine, starting the scanning engine, otherwise waiting for the system idle memory to meet the resource requirements.

[0046] For example, the cloud engine library data table is shown in Table 2. In addition to the fields in the table below, the cloud engine library data table may also include other fields, such as: engine timeout, engine privileged mode status, engine TTY mode status, engine tag, etc.

[0047] Table 2

[0048] Scanning capability type Scan engine download address Scan Engine Scheduling Parameters CPU usage limit Memory usage limit Concurrent task limit subdomain_finder dockerimages_oss_subdomain_finder subdomainFinder {value} 30 % 30 % 1 port_scan dockerimages_oss_port_scan portscan {value} 30 % 30 % 1 vulscan dockerimages_oss_vulscan vulScan {value} 30 % 30 % 1 ... ... ... ... ... ...

[0049] When the system resources meet the minimum resource occupancy preset in the task scanning engine, the scanning engine launcher starts the scanning engine according to the corresponding task parameters, wherein the task parameters include but are not limited to: task ID, scanning engine scheduling parameters, privileged mode and / or task timeout, etc.

[0050] Step S103, storing the data results generated by the task scanning engine executing the task in the data scheduling database.

[0051] After the scanning engine task is executed, the data results are generated and stored in the data scheduling database. The data format in the data scheduling database is shown in Table 3.

[0052] Table 3

[0053] ResultSourceEngine task_id Result Subdomain detection 3298b9e3-48a5-49b3-9fc6-514792963111 Probe data results ... ... ...

[0054] A lightweight asset risk scanning method and device provided in this embodiment can quickly expand the lightweight asset risk scanning architecture and adopt a distributed on-demand calling system resource solution to modularize the scanning capability and facilitate expansion and maintenance. The results of the scanning module can be expanded in a quick way, truly realizing lightweight asset risk scanning and rapid expansion of capabilities.

[0055] See also Figure 4 Another embodiment of the present invention further provides a lightweight asset risk scanning device 200, comprising: a data scheduling module 201, a scanning engine startup module 202 and a scan data storage module 203. The lightweight asset risk scanning device 200 is used to implement the lightweight asset risk scanning method in the above method embodiment.

[0056] Specifically, the lightweight asset risk scanning device 200 includes:

[0057] The data scheduling module 201 is configured to evenly distribute the acquired task initial data to multiple scanning probes according to the scanning capability execution order table;

[0058] The scanning engine startup module 202 is configured to use the scanning probe to obtain the task scanning engine in the cloud scanning engine library on demand according to the data scheduling logic and the user's preset resource limit; check whether the system free resources meet the preset resource occupation limit of the task scanning engine, and if so, start the task scanning engine according to the corresponding task parameters;

[0059] The scan data storage module 203 is configured to store the data results generated by the task scanning engine executing the task in the data scheduling database.

[0060] Furthermore, the scanning engine startup module 202 is configured to: check whether the local environment contains the scanning engine required for the task through the scanning engine launcher, and if so, determine whether the scanning engine in the local environment is the latest version, otherwise obtain the corresponding scanning engine from the cloud scanning engine library to the local; if the scanning engine in the local environment is the latest version, perform a system resource usage check, otherwise obtain the latest version of the scanning engine from the cloud scanning engine library to the local, and delete the old version of the scanning engine in the local environment.

[0061] Furthermore, the scanning engine startup module 202 is configured to: check whether the system's idle CPU meets the scanning engine resource requirements, if so, check whether the system's idle memory meets the scanning engine resource requirements, otherwise wait for the system's idle CPU to meet the resource requirements; if the system's idle memory meets the scanning engine resource requirements, start the scanning engine, otherwise wait for the system's idle memory to meet the resource requirements.

[0062] Furthermore, the task parameters include a task ID, a scanning engine scheduling parameter, a privileged mode and / or a task timeout.

[0063] Furthermore, the initial task data includes the business registration name, IP address, domain name information and / or website address of the enterprise.

[0064] It should be noted that the lightweight asset risk scanning device 200 provided in this embodiment corresponds to a technical solution that can be used to execute various method embodiments, and its implementation principle and technical effects are similar to the method, which will not be repeated here.

[0065] Figure 5 This is a schematic diagram of the structure of an electronic device 300 provided in another embodiment of the present invention, and the electronic device is used to implement the lightweight asset risk scanning method in the method embodiment. The electronic device 300 in the embodiment of the present invention may include but is not limited to a PC, a smart phone, a tablet computer, a PDA, a server, etc. Figure 5 The electronic device 300 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0066] like Figure 5 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303 to implement the method of the embodiment of the present invention. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0067] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 5 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0068] The above description is only a preferred embodiment of the present invention. Those skilled in the art should understand that the scope of disclosure involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but also should cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

Claims

1. A lightweight asset risk scanning method, characterized in that: include: Evenly distribute the acquired task initial data to multiple scanning probes according to the scanning capability execution order table; Check whether the local environment contains the scanning engine required by the task through the scanning engine launcher. If yes, determine whether the task scanning engine in the local environment is the latest version. Otherwise, obtain the corresponding task scanning engine from the cloud scanning engine library to the local environment. If the task scanning engine in the local environment is the latest version, check the system resource usage. Otherwise, obtain the latest version of the task scanning engine from the cloud scanning engine library to the local environment and delete the old version of the task scanning engine in the local environment. Check whether the free resources of the system meet the resource occupation limit preset by the task scanning engine, and if so, start the task scanning engine according to the corresponding task parameters; The data results generated by the task scanning engine executing the task are stored in the data scheduling database.

2. A lightweight asset risk scanning method according to claim 1, characterized in that: The step of checking whether the free resources of the system meet the resource occupation limit preset by the task scanning engine includes: Check whether the system idle CPU meets the scanning engine resource requirements. If yes, check whether the system idle memory meets the scanning engine resource requirements. Otherwise, wait for the system idle CPU to meet the resource requirements. If the system free memory meets the resource requirements of the scanning engine, the scanning engine is started, otherwise it waits for the system free memory to meet the resource requirements.

3. A lightweight asset risk scanning method according to claim 2, characterized in that: The task parameters include a task ID, a scanning engine scheduling parameter, a privileged mode and / or a task timeout.

4. A lightweight asset risk scanning method according to claim 1, characterized in that: The initial task data includes the business registration name, IP address, domain name information and / or website address of the enterprise.

5. A lightweight asset risk scanning device, characterized in that: include: A data scheduling module is configured to evenly distribute the acquired task initial data to a plurality of scanning probes according to a scanning capability execution order table; The scanning engine startup module is configured to check whether the local environment contains the scanning engine required by the task through the scanning engine startup device, and if so, determine whether the task scanning engine in the local environment is the latest version, otherwise obtain the corresponding task scanning engine from the cloud scanning engine library to the local; if the task scanning engine in the local environment is the latest version, perform a system resource usage check, otherwise obtain the latest version of the task scanning engine from the cloud scanning engine library to the local, and delete the old version of the task scanning engine in the local environment; Check whether the free resources of the system meet the resource occupation limit preset by the task scanning engine, and if so, start the task scanning engine according to the corresponding task parameters; The scanning data storage module is configured to store the data results generated by the task scanning engine executing the task in the data scheduling database.

6. A lightweight asset risk scanning device according to claim 5, characterized in that: The scanning engine startup module is configured as follows: Check whether the system idle CPU meets the scanning engine resource requirements. If yes, check whether the system idle memory meets the scanning engine resource requirements. Otherwise, wait for the system idle CPU to meet the resource requirements. If the system free memory meets the resource requirements of the scanning engine, the scanning engine is started, otherwise it waits for the system free memory to meet the resource requirements.

7. A lightweight asset risk scanning device according to claim 6, characterized in that: The task parameters include a task ID, a scanning engine scheduling parameter, a privileged mode and / or a task timeout.

8. The lightweight asset risk scanning device according to claim 5, characterized in that: The initial task data includes the business registration name, IP address, domain name information and / or website address of the enterprise.

Images