A method and system for quickly conducting special scanning activities based on a rule file

Abstract

The present invention discloses a method and system for quickly conducting special scanning activities based on a rule file, which relates to the field of network security technology. The system for the special scanning activities includes a function module and a rule module; the rule module is used to define the rule file; the function module is used to execute corresponding scanning processes or behaviors according to the predefined rule file. The rule file with a standardized format provided by the present invention for special scanning realizes a system for parsing this format of rule file. By parsing the rule file, the system can collect assets in the asset mapping site or asset library according to the predefined rules. These assets are determined by the predefined rules and can be assets related to a certain type of fingerprint, a certain type of service, a certain type of port, etc. Then, according to the Poc set predefined in the rule file, special detection is carried out on the collected assets to detect the special assets of vulnerabilities that are of key concern.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and particularly to a method and system for quickly conducting special scanning activities based on rule files. Background Art

[0002] In recent years, the threats and risks of network security have been increasing day by day and have become challenges faced by mankind in the information age. During the security operation process, new vulnerabilities, hot vulnerabilities, and vulnerabilities related to systems or programs frequently used by customers need to be focused on, and it is necessary to detect in a timely manner whether there are relevant vulnerabilities in the assets exposed by customers. Especially when new vulnerabilities break out, detecting in a timely manner the assets of customers with new vulnerabilities and taking corresponding mitigation protection measures or emergency measures can intuitively let customers experience the value of security operation, prevent customer assets from suffering losses due to relevant network security vulnerabilities, or avoid being notified by network supervision units. Summary of the Invention

[0003] In view of this, according to the needs of security operation, the present invention provides a system for quickly conducting special monitoring of customer assets to cope with the special detection and monitoring of an increasing number of new vulnerabilities and hot vulnerabilities, and at the same time serving the special detection of conventional vulnerabilities.

[0004] In a first aspect, the present invention provides a system for quickly conducting special scanning activities based on rule files, including a function module and a rule module;

[0005] The rule module is used to define rule files;

[0006] The function module is used to execute corresponding scanning processes or behaviors according to predefined rule files.

[0007] Further, the function module specifically includes:

[0008] Configuring a network space asset mapping source or a self-built asset mapping source as an asset mapping source / database for the system to collect special assets;

[0009] Implementing a query API according to the configured asset mapping source / database, and at the same time defining a string keyword for each asset mapping source / database, where the keyword is used as the key value under the assets sub-element in the rule file;

[0010] Parsing the rule file and mapping the elements in the rule file to relevant structures;

[0011] Enabling or disabling the current rule special according to the value of the enable element;

[0012] Querying the query rules represented by all key-value dictionary sub-elements in the assets element list and deduplicating the queried assets;

[0013] Process the assets_record element. When the assets_record element is true, record the queried special assets to the storage medium. When the assets_record element is false, skip recording the queried assets to the storage medium and directly scan the special POC collection for the queried assets.

[0014] According to the tag element of the rule file, a fuzzy query is performed on the Poc name, Poc path, Poc tag, and vulnerability risk level in the Poc library of the functional module subsystem;

[0015] According to all the sub-elements in the scripts element list in the rule file, query the Poc path in the Poc library of the function module;

[0016] De-duplicate the POC set found based on the tag and scripts elements;

[0017] The asset collection queried by the assets element is scanned against the Poc collection queried by tags and scripts.

[0018] Furthermore, the functional module corresponds to one or more rule files and can perform multiple special tests simultaneously.

[0019] Furthermore, the rule module is a YAML rule file in a standardized format formulated according to operational requirements.

[0020] In a second aspect, the present invention provides a method for quickly performing special scanning activities based on rule files, the method comprising converting special scanning activities or behaviors into special rule files, the system identifying and translating the special rule files, and the system executing special operations according to the definition of the rule files.

[0021] Furthermore, the special scanning activities or behaviors specifically include:

[0022] Security operation analysis, the security operation analysis specifically includes:

[0023] If special security inspection actions, hot vulnerabilities, or new vulnerabilities are detected, a new special security inspection process will be implemented;

[0024] If it is necessary to continuously monitor the special project during the operation, perform periodic special project scanning and select the special project rule file.

[0025] Furthermore, the conversion into a special rule file specifically includes:

[0026] Create a new special rules file;

[0027] Define the rule name;

[0028] Define asset search rules;

[0029] Define scanning Poc set;

[0030] Save the rules file.

[0031] Furthermore, the system identifies and translates the special rule files, including:

[0032] After the selection of the special rule file or the saving of the rule file is executed, the rule file is parsed; the parsing of the rule file specifically includes mapping the rule file to the data structure in the functional module, so that the functional module translates the special scanning action expressed in the rule file into a logical action that the machine can follow.

[0033] Furthermore, the system performs special operations according to the definition of the rule file, including:

[0034] Processing the enable element, including, when the enable element is false, returning the security operations analysis; when the enable element is true, processing the assets element;

[0035] The processing of the assets element includes querying the assets element and removing duplicates from the query results;

[0036] Process the assets_record element, including, when the assets_record element is true, recording all queried assets to disk; when the assets_record element is false, proceeding to the next step;

[0037] Processing tag elements, including: the function module identifies the tag element. If there is a comma in the tag element, the tag element is split into an array by the comma, and then a fuzzy query is performed on the Poc name, Poc path, Poc label, and vulnerability risk level in the Poc library of the function module according to the value of the tag element;

[0038] Processing the scripts element, including, the function module regards all the strings in the list as Poc paths in the Poc library, and queries the Poc library;

[0039] Merge and remove duplicate special POC sets, including merging and removing duplicates of all special POCs expressed by tags and scripts elements in the special rule file;

[0040] Perform special scans, including performing POC collection scans on the queried assets;

[0041] Return to Security Operations Analysis.

[0042] In a third aspect, the present invention also provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for quickly performing a special scan activity based on a rule file as described in any one of the above are implemented.

[0043] Advantages of the present invention:

[0044] The present invention provides a rule file with a standardized format for special scans, and implements a system for parsing this format of rule file. By parsing the rule file, the system can collect assets in the asset mapping site or asset library according to predefined rules. These assets are determined by the predefined rules and can be assets related to a certain type of fingerprint, a certain type of service, a certain type of port, etc. Then, according to the set of Pocs predefined in the rule file, special detection is performed on the collected assets to detect special assets of vulnerable points that are of key concern. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Flowchart of the method for quickly performing a special scan activity based on a rule file in the present invention;

[0046] Figure 2 Specific working flowchart of the method for quickly performing a special scan activity based on a rule file in the present invention;

[0047] Figure 3 Example of rule names in an embodiment of the present invention;

[0048] Figure 4 Embodiment 1 of defining an asset search rule in an embodiment of the present invention;

[0049] Figure 5 Embodiment 2 of defining an asset search rule in an embodiment of the present invention;

[0050] Figure 6 Example of defining a set of scan Pocs in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] To make the objectives, advantages, and features of the present invention more obvious, the following detailed description further elaborates on the present invention.

[0052] Special scans are dynamically executed according to newly discovered vulnerabilities or the needs of the operation process. The present invention defines a set of special detection rule files to describe the special detection activities in the operation process, and around the special detection rule files, the system implements actions such as parsing, translation expression, and execution of scans.

[0053] Embodiment 1

[0054] Based on this system, special scanning activities can be abstracted into special rule files, and then the system parses the special rule files and performs special scanning according to the rule files. The system is divided into two subsystems, the function module subsystem and the rule module subsystem. The rule module is a rule file in a special format, and the function module executes the corresponding scanning process or behavior according to the predefined rule file.

[0055] The functional modules mainly implement the following functions:

[0056] Configure the cyberspace asset mapping source as the asset mapping source / library for the system to collect special assets, such as cyberspace asset mapping service products, Fofa, Quake, Hunter, etc., or self-built asset mapping sources, such as self-owned ElasticSearch libraries, etc.;

[0057] To implement the query API based on the configured mapping source / library, define a string keyword for each mapping source / library, which is used as the key value under the assets sub-element in the rule file;

[0058] Implement the function of parsing rule files and map the elements in the rule files to related structures;

[0059] Enable or disable the current rule based on the value of the enable element;

[0060] Implement querying of query rules represented by all key-value dictionary sub-elements in the assets element list and deduplication of the queried asset collection, where the query uses the implemented query API, the key value represents the corresponding surveying and mapping source / library, and the value value represents the query rule of the surveying and mapping source / library represented by the key value;

[0061] When the assets_record element is true, the queried special assets are recorded to the storage medium. When it is false, the queried assets are skipped and the special Poc collection is directly scanned for the queried assets.

[0062] Implement fuzzy query on Poc name, Poc path, Poc label, vulnerability risk level, etc. in the Poc library of the functional module based on the tag element of the rule file;

[0063] Implement querying of the Poc path in the Poc library of the functional module based on all the sub-elements in the scripts element list in the rule file;

[0064] De-duplicate the POC set found by the tag and scripts elements;

[0065] Implement the scanning of the asset set obtained by querying the assets element with the Poc set obtained by querying the tag and scripts;

[0066] Implement support for one or more rule files so that multiple special detections can be performed at one time.

[0067] The rule module is a series of yaml rule files with specific format specifications formulated according to operation requirements, including the element fields shown in Table 1:

[0068] Table 1

[0069]

[0070] The top-level element is a map, the key is the name of the special rule, which can be customized, and the value is also of map type, including the following elements:

[0071] desc: Special description, this is an optional field, and it can be present or absent;

[0072] category: Special classification, indicating which category the special belongs to, this is an optional field, and it can be present or absent;

[0073] parentCategory: Special parent category;

[0074] enable: Indicates whether to enable this special rule file. If set to false, this special rule file will not be used when the system performs a special scan. This is an optional field, and the default value is true;

[0075] assets: This element is one of the core elements of the rule file, indicating the rule for collecting specific special assets from the asset library / source. It is a list dictionary, where the key value represents the asset library / source, and the value represents the search rule on this asset library / source. When the system executes this rule file, it will use the search rule represented by the value to search for the asset source / library represented by the key value;

[0076] assets_record: This element is one of the core elements of the rule file, indicating whether to record the collected assets to the database or file first and then perform a special scan on these special assets during the special scan process;

[0077] tag: This element is one of the core elements of the rule file. This element defines the Poc tag, and this tag value is used to perform a fuzzy search for Pocs in the Poc library. After the searched Pocs are merged and de-duplicated with the Pocs defined in the scripts field, the assets searched at the assets location are scanned. It can be multiple values, separated by commas;

[0078] scripts: This element is one of the core elements of the rule file and is a list value, one for each item. This element is used to precisely define the Poc path. After the Poc defined in the scripts field is merged and de-duplicated with the Poc fuzzy searched at the tag, the assets searched at the assets location are scanned;

[0079] The rule file format, fields, and field description examples are as follows:

[0080] topics\rdzx\puyuan.yaml

[0081] # Special rule name

[0082] RDZX_Puyuan_KeHu:

[0083] # Special description, optional, this field can be present or absent

[0084] desc: Puyuan deserialization special item

[0085] # Special category, optional, this field can be present or absent

[0086] category: deserialize

[0087] # Special parent category, optional, this field can be present or absent

[0088] parentCategory: java

[0089] # Whether to enable this special item, optional, this field can be present or absent, default is enabled

[0090] enable: true

[0091] # This field defines the special item asset search rule, where the key field represents the asset source, and the value field represents the asset search rule on this asset source. Multiple can be defined, one for each item

[0092] assets:

[0093] # soss source, multiple values of domain / notdomain can be taken, separated by commas

[0094] - soss: notdomain=XX

[0095] # This field indicates whether to record the assets searched

[0096] assets_record: false

[0097] # This field defines the Poc tag. The value of this tag is used for fuzzy search of Pocs in the Poc library. After the searched Pocs are merged and de-duplicated with the Pocs defined in the scripts field, the assets searched at the assets location are scanned. It can be multiple values, separated by commas

[0098] tag: puyuan

[0099] # This field defines the Poc path. After the Pocs defined in the scripts field are merged and de-duplicated with the searched Pocs, the assets searched at the assets location are scanned, one for each item

[0100] scripts:

[0101] - / pocs / custom / puyuan-default-rce.yaml

[0102] - / pocs / custom / puyuan-eos-rce.yaml

[0103] - / pocs / custom / puyuan-sccp-rce.yaml。

[0104] Example 2

[0105] As Figure 1 、 Figure 2 shown, a method for quickly performing special scanning activities based on a rule file. The execution process includes

[0106] I. Security operation analysis

[0107] During the security operation analysis process, various service scenario requirements and demands will be faced. When facing special security inspection operations, special hot vulnerability projects, or newly detected vulnerabilities, a new special security monitoring process will be triggered, and at this time, the selection of the rule file will be skipped and executed in sequence; when continuous monitoring of certain special projects is required during the operation process, we will trigger periodic special scans to judge the network security situation of certain special projects, and at this time, the execution will start from the selection of the rule file and be executed in sequence.

[0108] II. Writing rule files

[0109] 1. Create a special rule file

[0110] Under the hot special project (rdzx) or other category directories in the specific directory (topics) directory, create a special rule for puyuan.yaml. You can copy an existing special file and then rename it.

[0111] 2. Define a reasonable rule name

[0112] Define reasonable rule names. The rule name, together with the rule path, needs to be unique. An example of the rule name is shown as Figure 3 as follows.

[0113] 3. Define asset search rules

[0114] Set asset search rules. The asset search rules support the following asset mapping sources / libraries:

[0115] "fofa", which supports the fofa source. Its value can be filled in according to the fofa rules;

[0116] "quake", which supports the quake source. Its value can be filled in according to the quake rules;

[0117] "hunter", which supports the hunter source. Its value can be filled in according to the hunter rules;

[0118] "starsoes", which supports the company's original self-owned asset mapping library. Its value can be filled in according to the Elastic rules. The mapping library has been deactivated;

[0119] "scanxes", which supports the Elastic asset mapping library of Starshot. Its value can be filled in according to the Elastic rules;

[0120] "scanx", same as starshot;

[0121] "starshot", which supports the Sqlite asset mapping library of Starshot. Its rule format is: assets={taskid} or assets={service} or assets={fingerextrainfo}, where taskid is the id of the scan task, service is the service name, and fingerextrainfo is the additional information of the fingerprint;

[0122] "soss", which supports the asset library of soss. Its rule format is: domain={customer name} or notdomain={customer name}, where domain means to get the assets of a certain customer, and notdomain means to get all customer assets but not the assets of a certain customer.

[0123] Specific examples are shown as Figure 4 and Figure 5 as follows.

[0124] 4. Define optional fields

[0125] Set the values of optional fields such as desc, category, parentCategory, enable, etc.

[0126] 5. Define the scanning POC set

[0127] like Figure 6 As shown in the figure, tags and scripts are used. The tags field fuzzily searches the Poc library, and can search for paths, names, labels, and threat levels. The scripts field accurately indicates the use of a certain Poc. The Pocs searched by the two fields are combined to scan the searched special assets. If there is no Poc for this vulnerability in the existing Poc rule library in the system, a new Poc for this vulnerability needs to be written.

[0128] 6. Save the rule file

[0129] Save the rule files for abstract transformation of special activities to a specific directory that can be recognized by the functional module.

[0130] 3. Select rule file

[0131] During the security operation process, when it is necessary to monitor the network security situation of certain special projects, it is necessary to trigger a periodic special scanning process. At this time, it is necessary to select special rule files in the special rule library according to the monitoring project.

[0132] 4. Parsing rule files

[0133] The rule file is mapped to a specific data structure in the functional module so that the functional module can translate the special scanning actions expressed in the rule file into specific logical actions that the machine can follow.

[0134] 5. Processing the enable element

[0135] When the function module recognizes that the value of the enable element is true, the current rule item is enabled; when it is false, the current item is disabled and the security analysis action is returned.

[0136] 6. Processing assets elements

[0137] The assets element is mapped to a list of function modules. Each value in the list is a map dictionary, where the key value represents the asset source and the value value represents the query expression of this asset source. The key value is the asset source keyword defined above. The function module uses it to find the query API corresponding to the asset source and triggers the query action with the query expression value until all assets sub-elements are processed.

[0138] All the queried results are deduplicated.

[0139] 7. Processing assets_record elements

[0140] When the function module recognizes that the value of assets_record is true, it records all queried assets to disk; otherwise, if it is false, it ignores this action and proceeds to the next step.

[0141] 8. Processing tag elements

[0142] The function module identifies the tag element. If there is a comma in the tag, the tag is split into an array by the comma. Then, according to the value of the tag element, a fuzzy query is performed on the Poc name, Poc path, Poc label, vulnerability risk level, etc. in the Poc library that comes with the function module.

[0143] 9. Processing scripts elements

[0144] The function module identifies the value of the scripts element, and its value is mapped to the list string of the function module. The function module regards all the strings in the list as Poc paths in the Poc library and queries the Poc it represents.

[0145] 10. Merge the Deduplication Special POC Set

[0146] Merge all special POCs expressed by tags and scripts elements in the special rule file and remove duplicates.

[0147] 11. Perform special scans

[0148] The functional module performs a scan of all special POCs expressed by tags and scripts for all special assets expressed by the assets element.

[0149] 12. Return to Security Operations Analysis

[0150] The results of the special scan are returned to the security operation process to evaluate the security operation effect and serve as the basis for executing the next step of the security operation process. For example, based on the evaluation of the special scan results, it is decided whether this special scan needs to be executed periodically until the expected effect of vulnerability management is achieved within the operation cycle.

[0151] Example 3

[0152] A computer-readable storage medium stores computer instructions, which implement the steps of the method in Example 1 when executed by a processor.

[0153] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, an apparatus, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0154] The present invention is described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or a plurality of flows and / or blocks

[0155] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one or more of the flows Figure 1 or a plurality of flows and / or blocks

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or a plurality of flows and / or blocks

[0157] The above embodiments have described the technical solutions of the present invention in detail. Obviously, the present invention is not limited to the described embodiments. Based on the embodiments of the present invention, those familiar with the technical field can also make various changes accordingly, but any changes equivalent or similar to the present invention belong to the scope of protection of the present invention.

[0158] The content not described in detail in this specification belongs to the prior art well-known to those skilled in the art.

Claims

1. A system for quickly performing special scanning activities based on rule files, characterized in that: Includes function modules and rule modules; The rule module is used to define a rule file; the rule module is a YAML rule file in a standardized format formulated according to operational requirements; The functional module is used to execute corresponding scanning processes or behaviors according to predefined rule files; The functional modules specifically include: Configure cyberspace asset mapping sources or self-built asset mapping sources as asset mapping sources / libraries for the system to collect special assets; Implement query API based on configured asset mapping source / library, and define string keywords for each asset mapping source / library, which are used as key values ​​under assets sub-element in rule files; Parse the rule file and map the elements in the rule file to related structures; Enable or disable the current rule specialization according to the value of the enable element; Query the query rules represented by all key-value dictionary sub-elements in the assets element list and deduplicate the queried assets; Process the assets_record element. When the assets_record element is true, record the queried special assets to the storage medium. When the assets_record element is false, skip recording the queried assets to the storage medium and directly scan the special POC collection for the queried assets. According to the tag element of the rule file, a fuzzy query is performed on the Poc name, Poc path, Poc tag, and vulnerability risk level in the Poc library of the functional module subsystem; According to all the sub-elements in the scripts element list in the rule file, query the Poc path in the Poc library of the function module; De-duplicate the POC set found based on the tag and scripts elements; The asset collection queried by the assets element is scanned against the Poc collection queried by tags and scripts.

2. A system for quickly performing special scanning activities based on rule files as claimed in claim 1, characterized in that: The functional module corresponds to one or more rule files and can perform multiple special tests at the same time.

3. A method for quickly performing special scanning activities based on rule files, characterized in that: The method includes converting the special scanning activity or behavior into a special rule file, the system identifying and translating the special rule file, and the system performing the special operation according to the definition of the rule file; The system performs special operations according to the definition of the rule file, including: Processing the enable element, including returning security operations analysis when the enable element is false and processing the assets element when the enable element is true; The processing of the assets element includes querying the assets element and removing duplicates from the query results; Process the assets_record element, including, when the assets_record element is true, recording all queried assets to disk; when the assets_record element is false, proceeding to the next step; Processing tag elements, including: the function module identifies the tag element. If there is a comma in the tag element, the tag element is split into an array by the comma, and then a fuzzy query is performed on the Poc name, Poc path, Poc label, and vulnerability risk level in the Poc library of the function module according to the value of the tag element; Processing the scripts element, including, the function module regards all the strings in the list as Poc paths in the Poc library, and queries the Poc library; Merge and remove duplicate special POC sets, including merging and removing duplicates of all special POCs expressed by tags and scripts elements in the special rule file; Perform special scans, including performing POC collection scans on the queried assets; Return to Security Operations Analysis.

4. A method for quickly performing special scanning activities based on rule files as claimed in claim 3, characterized in that: The special scanning activities or behaviors specifically include: security operation analysis; the security operation analysis specifically includes: In response to various service scenario needs and requirements, we implement new special security detection processes for special security inspection actions, hot vulnerability special projects, and new vulnerabilities detected; If it is necessary to continuously monitor the special project during the operation, perform periodic special project scanning and select the special project rule file.

5. A method for quickly performing special scanning activities based on rule files as claimed in claim 4, characterized in that: The above-mentioned actions of converting into special rules documents specifically include: Create a new special rules file; Define the rule name; Define asset search rules; Define scanning Poc set; Save the rules file.

6. A method for quickly performing special scanning activities based on rule files as claimed in claim 5, characterized in that: The system identifies and translates special rule files, including: After the selection of the special rule file or the saving of the rule file is executed, the rule file is parsed; the parsing of the rule file specifically includes mapping the rule file to the data structure in the functional module, so that the functional module translates the special scanning action expressed in the rule file into a logical action that the machine can follow.

7. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for quickly performing special scanning activities based on rule files as described in any one of claims 3 to 6 are implemented.

Images