Vulnerability detection tool evaluation method, system, device and medium

Abstract

The invention relates to an evaluation method, system and device for a vulnerability detection tool and a medium. The method comprises the steps of obtaining task information; matching at least two vulnerability detection tools based on the task information; according to the matched vulnerability detection tool, performing format conversion on the task information to adapt to the vulnerability detection tool; detecting the task information after the corresponding format conversion through a vulnerability detection tool, and generating a detection result; converting each detection result into a standard detection result; wherein the standard detection result comprises each vulnerability identified by the vulnerability detection tool and a corresponding risk level; and determining the credibility of the corresponding vulnerability detection tool according to the number of vulnerabilities of each risk level in all the standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result. According to the invention, the dynamic evaluation of the detection credibility of each vulnerability detection tool is realized.

Description

Technical Field

[0001] The present invention relates to the field of product safety technology, and in particular to an evaluation method, system, device and medium for a vulnerability detection tool. Background Art

[0002] Currently, in the field of software security testing, many vulnerability detection tool manufacturers provide a variety of detection technologies and solutions, mainly including static application security testing (SAST), dynamic application security testing (DAST), interactive application security testing (IAST), mobile application security testing (MAST), etc. These vulnerability detection tools play an important role in their respective fields, but there are significant differences in the standards, detection methods and result formats of various manufacturers. Due to the inconsistency of the standard detection results of vulnerability detection tools, it is difficult for users to directly compare and integrate the performance of these vulnerability detection tools. Therefore, it is necessary to provide an evaluation method, system, equipment and medium for vulnerability detection tools. Summary of the invention

[0003] In view of the above-mentioned shortcomings of the prior art, an object of the present invention is to provide an evaluation method, system, device and medium for a vulnerability detection tool, which improves the problem that the credibility of the vulnerability detection tools in the prior art is difficult to quantify.

[0004] To achieve the above-mentioned purpose and other related purposes, the present invention provides an evaluation method for a vulnerability detection tool, comprising: obtaining task information; matching at least two vulnerability detection tools based on the task information; converting the format of the task information according to the matched vulnerability detection tool to adapt to the vulnerability detection tool; detecting the task information after the corresponding format conversion by the vulnerability detection tool to generate a detection result; converting each detection result into a standard detection result; wherein the standard detection result includes each vulnerability identified by the vulnerability detection tool and the corresponding risk level; determining the credibility of the corresponding vulnerability detection tool according to the number of vulnerabilities of each risk level in all standard detection results, and the number of vulnerabilities of the corresponding risk level in each standard detection result.

[0005] In one embodiment of the present invention, for each vulnerability detection tool: the detection result is converted into a standard detection result, including: based on the identification of the vulnerability detection tool in the detection result, determining the standard field that matches each field in the detection result; and generating a corresponding standard detection result based on the standard field.

[0006] In one embodiment of the present invention, the credibility of the corresponding vulnerability detection tool is determined based on the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result, including: counting the number of vulnerabilities of each risk level in all standard detection results; for each standard detection result: based on the number of vulnerabilities of each risk level in the standard detection result and the number of vulnerabilities of the corresponding risk level in all standard detection results, obtaining the contribution of the corresponding vulnerability detection tool at each risk level; weighting the contribution of the vulnerability detection tool at each risk level to obtain the credibility of the vulnerability detection tool.

[0007] In one embodiment of the present invention, the contribution of the corresponding vulnerability detection tool at each risk level is obtained based on the number of vulnerabilities at each risk level in the standard detection results and the number of vulnerabilities at the corresponding risk level in all standard detection results, including: calculating the ratio of the number of vulnerabilities at each risk level in the standard detection results to the number of vulnerabilities at the corresponding risk level in all standard detection results, to obtain the level coverage of the vulnerability detection tool at each risk level; for the level coverage of the vulnerability detection tool at each risk level: based on a preset partitioning rule, determining the coverage interval in which the level coverage is located, and using the preset contribution value corresponding to the coverage interval as the contribution of the vulnerability detection tool at the risk level.

[0008] In one embodiment of the present invention, the contribution of the corresponding vulnerability detection tool at each risk level is obtained based on the number of vulnerabilities at each risk level in the standard detection results and the number of vulnerabilities at the corresponding risk level in all standard detection results, including: calculating the ratio of the number of vulnerabilities at each risk level in the standard detection results to the number of vulnerabilities at the corresponding risk level in all standard detection results, and obtaining the level coverage of the vulnerability detection tool at each risk level; for the level coverage of the vulnerability detection tool at each risk level: based on the preset partitioning rules, determining the coverage interval in which the level coverage is located, and obtaining the preset contribution value corresponding to the coverage interval; counting the false alarm rate of the vulnerability detection tool at each risk level, and weightedly summing the false alarm rates of various risk levels and the corresponding preset contribution values ​​as the contribution of the vulnerability detection tool at the risk level.

[0009] In one embodiment of the present invention, after determining the credibility of the corresponding vulnerability detection tool based on the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result, it also includes: generating a comprehensive standard detection result of all vulnerability detection tools based on the credibility of each vulnerability detection tool and the corresponding standard detection results.

[0010] In one embodiment of the present invention, after determining the credibility of the corresponding vulnerability detection tool based on the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result, it also includes: sorting the vulnerability detection tools of the same type according to the corresponding credibility, and giving priority to displaying vulnerability detection tools with high credibility.

[0011] In one embodiment of the present invention, a vulnerability detection tool evaluation system is also provided, the system comprising: a task information acquisition module for acquiring task information; a tool allocation module for determining at least two matching vulnerability detection tools based on the task information; a vulnerability detection module for converting the format of the task information according to the matching vulnerability detection tool to adapt to the vulnerability detection tool; a result generation module for detecting the task information after the corresponding format conversion through the vulnerability detection tool to generate a detection result; a result conversion module for converting each detection result into a standard detection result; wherein the standard detection result includes each vulnerability identified by the vulnerability detection tool and the corresponding risk level; an evaluation module for determining the credibility of the corresponding vulnerability detection tool according to the number of vulnerabilities of each risk level in all standard detection results, and the number of vulnerabilities of the corresponding risk level in each standard detection result.

[0012] In one embodiment of the present invention, an electronic device is also provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the evaluation method of the vulnerability detection tool described in any one of the above items.

[0013] In one embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes any of the above-mentioned vulnerability detection tool evaluation methods.

[0014] As described above, the evaluation method, system, device and medium of a vulnerability detection tool of the present invention have the following beneficial effects: after obtaining the task information, the corresponding format conversion is performed according to the matching vulnerability detection tool, thereby avoiding the problem of task failure due to format mismatch and greatly improving the compatibility of the detection task. In addition, by using different detection tools, different vulnerability types and risk scenarios can be more comprehensively covered to achieve multi-dimensional vulnerability detection. The detection results of each tool are converted into standard detection results, and by calculating the detection contribution ratio of each tool in different risk levels, and combining the vulnerability distribution of each risk level in all standard detection results, a dynamic evaluation of the detection credibility of each vulnerability detection tool is achieved, so that the credibility is quantified. Not only can the vulnerability situation be detected more comprehensively, but also more stable and reliable detection results are provided to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of a flow chart of an evaluation method for a vulnerability detection tool provided by an embodiment of the present invention;

[0016] Figure 2 Shown is a structural block diagram of an evaluation system for a vulnerability detection tool provided by an embodiment of the present invention;

[0017] Figure 3 Shown is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0019] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0020] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0021] The inventors have discovered that the security detection tools currently on the market come from different manufacturers. Due to inconsistencies in algorithms, evaluation standards, vulnerability classification methods, and report formats among different manufacturers, it is difficult to directly compare and comprehensively utilize the detection results. This difference makes it difficult for users to achieve complementary advantages between different tools in actual applications. In addition, since the vulnerability detection tools of various manufacturers use different technical means and models, there are large differences in the accuracy, coverage, and efficiency of their detection results. Therefore, although the existing tools have certain technical advantages in their respective detection fields, they often cannot maximize their effectiveness when used alone, and it is difficult to achieve comprehensive and accurate vulnerability detection. This greatly limits the effectiveness of security detection by relying solely on a single vulnerability detection tool.

[0022] The security detection tools currently on the market are usually provided by a single vendor and focus on a specific detection method, such as static application security testing (SAST), dynamic application security testing (DAST), software composition analysis (SCA), interactive application security testing (IAST) or mobile application security testing (MAST). These methods each target different application scenarios and attack vectors, but due to differences in focus, they can often only cover some security vulnerabilities of the application. For example, SAST is suitable for code-level vulnerability detection, while DAST focuses on runtime vulnerability detection. The limitations of a single detection method make it difficult to fully identify all potential security risks, which may lead to false positives or false negatives.

[0023] In addition, the security detection tools provided by different manufacturers also have significant differences in detection dimensions, detection methods and interface standards, especially in terms of interface design, data format and report output structure. When users use tools from multiple manufacturers, they often need to perform interface docking and customized development for each tool separately, which greatly increases the complexity and cost of system integration. In addition, this inconsistency may also cause data compatibility issues, making cross-vendor integration and information sharing extremely difficult.

[0024] Currently, most of the existing security detection tools are independent point solutions, lacking a unified, cross-vendor integration platform. Security tools from different vendors can usually only run independently and generate independent reports. Users need to manually compare these reports to comprehensively evaluate the security of the application. Due to the lack of intelligent collaboration and automated integration, the information flow and collaborative work between multiple tools are very limited, which not only makes the security detection process cumbersome and inefficient, but also increases the risk of manual operation errors. Especially in the face of complex enterprise environments and diverse application scenarios, the existing detection model cannot provide real-time monitoring and comprehensive risk assessment, nor can it respond to new security threats in a timely manner.

[0025] The present invention provides an evaluation method for a vulnerability detection tool. After obtaining task information, the corresponding format conversion is performed according to the matching vulnerability detection tool, thereby avoiding the problem of task failure caused by format mismatch and greatly improving the compatibility of the detection task. In addition, by using different detection tools, different vulnerability types and risk scenarios can be more comprehensively covered to achieve multi-dimensional vulnerability detection. The detection results of each tool are converted into a standard detection result, and by calculating the detection contribution ratio of each tool in different risk levels and combining the vulnerability distribution of each risk level in all standard detection results, a dynamic evaluation of the detection credibility of each vulnerability detection tool is achieved, so that the credibility is quantified. Not only can the vulnerability situation be detected more comprehensively, but also a more stable and reliable detection result is provided to the user. The present invention can effectively aggregate the advantages of tools from different manufacturers. By integrating similar tools and extracting differentiated advantages in combination with advanced algorithms, complementarity can be achieved between different detection tools, thereby improving the comprehensiveness and accuracy of vulnerability detection. Finally, through a unified standardized detection report format, unified presentation and comparison of the detection results of different tools can be achieved, thereby improving the overall detection capability and security protection level.

[0026] See also Figure 1 ,The evaluation method of vulnerability detection tools includes the following steps:

[0027] S1. Obtain task information.

[0028] This solution is applied to a vulnerability detection platform, which can correspond to multiple different vulnerability detection tools or integrate multiple vulnerability detection tools. The user enters the task information to be detected through the front-end interface of the platform, and the task information includes but is not limited to the task identifier, detection target and detection range. Furthermore, in order to ensure the accuracy of the task information, after obtaining the task information, the task information is also subjected to the legality and integrity verification. Among them, the legality verification refers to checking whether the fields in the task information meet the preset format requirements and whether the relevant values ​​are within the preset numerical range. If the fields in the task information do not meet the legality requirements, the corresponding preset processing measures can be determined based on the identification of the corresponding fields, wherein the processing measures include but are not limited to automatically changing the information in the field to meet the legality requirements, or generating an error prompt and pushing it to the user to prompt the user to modify the input content. The integrity check refers to checking whether the required fields in the task information are filled in, and generating an error prompt or filling it with the preset default information when there are missing fields.

[0029] S2. Match at least two vulnerability detection tools based on the task information.

[0030] Since different vulnerability detection tools are suitable for different detection types, the detection types include but are not limited to dynamic application security testing, static application security testing, interactive application security testing and mobile application security testing. Therefore, the adapted vulnerability detection tool can be matched according to the task information. Specifically, the task information is parsed and processed to extract key fields such as the detection range, operating environment or target language, and the key fields are used to characterize the characteristics of the current task information. By mapping and matching the key fields with the mapping rule library, at least two vulnerability detection tools that meet the current task information requirements are determined. Among them, the mapping rule library records the characteristic information of each manufacturer corresponding to the platform and each vulnerability detection tool of each manufacturer, and the content recorded in the mapping rule library includes but is not limited to the detection type, operating environment, applicable target language and supported file type supported by each vulnerability detection tool. The final matching tool can be screened out by layer-by-layer matching. Exemplarily, the target language and file range of the task can be used to screen out the tools that support the corresponding language and file requirements, and the tools that finally meet the requirements can be screened out by the operating environment as the vulnerability detection tool corresponding to the task requirements. It is understandable that for a detection task, the platform supports users to flexibly configure different detection types, select tools from different manufacturers, and freely combine tools to meet specific needs. Therefore, for the same task requirements, a combination of multiple detection tools can be used to achieve coverage and diversity of vulnerability detection through tool collaboration, so as to achieve more comprehensive vulnerability detection, and improve the problem that the existing technology cannot achieve comprehensive and accurate vulnerability detection. For example, for the same detection task, SAST can be used for detection, LAST can be used for detection, and both methods can be used for detection at the same time. Each detection method can be detected with multiple vulnerability detection tools. Finally, after each detection tool generates a detection result, the various detection results can be summarized into a complete and comprehensive detection report.

[0031] S3. According to the matched vulnerability detection tool, the task information is formatted to adapt to the vulnerability detection tool.

[0032] Considering that vulnerability detection tools from different manufacturers may have their own specific format requirements for input task information, it is necessary to convert the task information on the platform to adapt to the corresponding vulnerability detection tool. Specifically, the platform first parses the task information and extracts the key parameters associated with the current task, wherein the key parameters include but are not limited to the detection scope, target language or operating environment, etc. By querying the mapping rule library, the conversion relationship between these key parameters and the format specifications of the corresponding vulnerability detection tool can be obtained, wherein the mapping rule library records the format requirements of each vulnerability detection tool of each manufacturer for input information, which includes but is not limited to the path format, language identification method or environment configuration, etc. The parameters are formatted according to the conversion relationship found to convert the parameters into a format adapted by the corresponding vulnerability detection tool. Exemplarily, for static analysis tools, the file path needs to be converted into an absolute path list and limited to the file types supported by the tool; for dynamic analysis tools, the target needs to be converted into a standardized URL or execution script. Through this format conversion process, the task information can be accurately adjusted to a form that meets the input requirements of the target vulnerability detection tool, ensuring that the tool can correctly receive and execute the detection task, thereby ensuring the efficiency and accuracy of the detection process.

[0033] S4. Detect the task information after the corresponding format conversion through the vulnerability detection tool and generate the detection result.

[0034] The platform sends the converted task information to the matching vulnerability detection tool through the corresponding interface, so that the vulnerability detection tool can scan the specific area according to the task information and generate detection results. The detection results include but are not limited to the detected vulnerability type, risk level, vulnerability location, and repair suggestions. After the vulnerability detection tool completes the detection, it will automatically send the detection results to the platform in the form of a detection report.

[0035] S5. Convert each detection result into a standard detection result; wherein the standard detection result includes each vulnerability identified by the vulnerability detection tool and the corresponding risk level.

[0036] After the platform receives the original detection results from different vulnerability detection tools, it parses the original detection results and extracts the contents of the preset fields. Since the risk classification standards used by different tools may be different, for example, some tools use low, medium and high risks for assessment, while other tools use numerical risk assessment. After the platform receives the original detection results, it will unify the risk levels obtained by each vulnerability detection tool into the standard level supported by the platform according to the preset standard risk level mapping rules. Among them, the standard level can be adaptively set based on actual needs. In one embodiment, the standard level includes low risk, medium risk, high risk and super risk with increasing risk levels. After the platform converts the original detection results, it obtains the standard detection results corresponding to the original detection results, wherein the information in the standard detection results includes but is not limited to the vulnerability name, vulnerability type, detection location, standardized vulnerability risk level and the identification information of the corresponding vulnerability detection tool. By standardizing the original detection results generated by each vulnerability detection tool itself, the result differences caused by the format or classification rules of different tools can be effectively eliminated.

[0037] In one embodiment of the present invention, for each vulnerability detection tool: converting the detection result into a standard detection result includes:

[0038] Based on the identification of the vulnerability detection tool in the detection result, determine the standard fields that match the fields in the detection result;

[0039] Generate corresponding standard test results based on standard fields.

[0040] For each vulnerability detection tool, the following processing is performed: according to the identification of the vulnerability detection tool in the detection result, the mapping relationship between the output result of the tool corresponding to the identification and the platform standard field is searched in the mapping rule library, and the original field generated by the detection tool is converted into the standard field of the platform according to the mapping relationship. Exemplarily, a vulnerability detection tool uses "severity" to represent the vulnerability risk level, and the platform standard field is represented by "risk level". It is necessary to map "severity" to the corresponding "risk level" field. After the matching of each preset field in the original detection result is completed, the platform further analyzes the content in the detection result, extracts various vulnerability information, and re-integrates these vulnerability information according to the preset criteria, thereby forming a standard detection result corresponding to the original detection result. Through this field standardization process, different detection results can be unified into standard detection results for subsequent analysis and integration. For the same type of security detection, the unified detection field standard proposed by the present invention ensures that the detection results of each manufacturer are summarized and compared under the same standard. This unified field can cover as comprehensive detection content as possible, so that the detection information provided by different manufacturers can be matched and integrated under the same dimension, improving the problem of inconsistent detection content between different manufacturers.

[0041] S6. Determine the credibility of the corresponding vulnerability detection tool based on the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result.

[0042] The platform collects statistics on the standard test results of all vulnerability detection tools corresponding to the current detection requirements, and counts the total number of vulnerabilities of each risk level in all standard test results. For each vulnerability detection tool, the number of vulnerabilities detected by it at each risk level is counted, and the contribution ratio of the vulnerability detection tool in each risk level is calculated to obtain the credibility of the vulnerability detection tool.

[0043] In one embodiment of the present invention, determining the credibility of the corresponding vulnerability detection tool according to the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result includes:

[0044] Count the number of vulnerabilities of each risk level in all standard detection results;

[0045] For each standard test result:

[0046] Based on the number of vulnerabilities of each risk level in the standard detection results, and the number of vulnerabilities of the corresponding risk level in all standard detection results, the contribution of the corresponding vulnerability detection tool at each risk level is obtained;

[0047] The contribution of the vulnerability detection tool at each risk level is weighted to obtain the credibility of the vulnerability detection tool.

[0048] After the platform receives all the test results and performs standardized conversion, it counts the total number of vulnerabilities of each risk level that appear in all standard test results, which reflects the overall distribution of vulnerabilities of each risk level in the current task. For each standard test result corresponding to a vulnerability detection tool, the number of vulnerabilities detected by the tool at each risk level is counted separately. For example, the number of vulnerabilities detected by the tool at risk levels such as "low risk", "medium risk", "high risk" and "super risk" can be counted. Calculate the proportion of the number of vulnerabilities detected by the tool at each risk level to the total number of vulnerabilities at the corresponding risk level, and use it as the contribution of the tool at the corresponding risk level. Each risk level is preset with a weight. For each vulnerability detection tool: the contribution of the tool at each risk level and the corresponding weight are weighted to obtain the credibility of the tool.

[0049] In one embodiment of the present invention, the number of vulnerabilities at each risk level in the standard detection results and the number of vulnerabilities at the corresponding risk level in all standard detection results are used to obtain the contribution of the corresponding vulnerability detection tool at each risk level, including:

[0050] Calculate the ratio of the number of vulnerabilities of each risk level in the standard test results to the number of vulnerabilities of the corresponding risk level in all standard test results to obtain the level coverage of the vulnerability detection tool at each risk level;

[0051] For the level coverage of the vulnerability detection tool at each risk level: based on the preset partitioning rules, determine the coverage interval of the level coverage, and use the preset contribution value corresponding to the coverage interval as the contribution of the vulnerability detection tool at the risk level.

[0052] For each vulnerability detection tool, count the number of vulnerabilities it detects at each risk level, and calculate the level coverage of the tool at each risk level according to formula (1):

[0053]

[0054] Among them, C j,k is the level coverage of vulnerability detection tool j in risk level k, R k is the total number of vulnerabilities detected by all vulnerability detection tools at risk level k, r j,kis the number of vulnerabilities detected by vulnerability detection tool j in risk level k. The level coverage can effectively quantify the contribution of the tool to the corresponding risk level. In order to convert the level coverage into a specific and quantifiable contribution value, based on the partition principle, the coverage is divided into several intervals, and a preset contribution value is assigned to each interval, as shown in formula (2):

[0055] Con j,k =Pre(C j,k ) (2)

[0056] Among them, Con j,k is the contribution of vulnerability detection tool j in risk level k, Pre is the contribution value mapping rule corresponding to coverage, Pre=v i , b i-1 ≤C j,k ≤ b i , v i is the contribution corresponding to the partition, b i-1 and b i These are the partition boundaries of the coverage. This partition setting method can not only map the continuous coverage to discrete contribution value intervals to quantify the performance of the tool in different risks, but also realize the standardization of data, which is convenient for subsequent comparison of various detection tools.

[0057] For example, the boundaries are {0, 0.2, 0.4, 0.6}, and the corresponding preset contribution values ​​are {0.2, 0.4, 0.6}. Then the mapping rule is as shown in formula (3):

[0058]

[0059] This partition mapping can not only effectively quantify the contribution of the tool at each risk level, but also set the contribution value by custom setting, and set a smaller contribution value for the partition with a smaller contribution rate, thereby effectively avoiding the low coverage tool from having a greater impact on the overall assessment results. In addition, through this unified mapping mechanism, the coverage of each tool can be converted into a standard contribution value to eliminate the differences between different vulnerability detection tools and facilitate subsequent credibility calculations. It can be understood that the specific number of partitions and the corresponding boundaries and contribution values ​​can be adaptively set by technical personnel in this field based on actual needs, and are not limited here. It should be further explained that in order to ensure more accurate results, after partitioning, a corresponding weight can be preset for each partition, and for each vulnerability detection tool: the contribution of the vulnerability detection tool at each risk level and the preset weight of the corresponding partition are weighted to obtain the credibility of the vulnerability detection tool.

[0060] In another embodiment of the present invention, the number of vulnerabilities at each risk level in the standard detection results and the number of vulnerabilities at the corresponding risk level in all standard detection results are used to obtain the contribution of the corresponding vulnerability detection tool at each risk level, including:

[0061] Calculate the ratio of the number of vulnerabilities of each risk level in the standard test results to the number of vulnerabilities of the corresponding risk level in all standard test results to obtain the level coverage of the vulnerability detection tool at each risk level;

[0062] For the level coverage of vulnerability detection tools at each risk level: based on the preset partitioning rules, determine the coverage interval where the level coverage is located, and obtain the preset contribution value corresponding to the coverage interval;

[0063] The false alarm rate of the vulnerability detection tool at each risk level is counted, and the false alarm rate of various risk levels and the corresponding preset contribution value are weighted and summed as the contribution of the vulnerability detection tool at the risk level.

[0064] For each vulnerability detection tool, the level coverage of the vulnerability detection tool at each risk level is obtained by calculating the proportion of the number of vulnerabilities detected by the vulnerability detection tool at each risk level to the total number of vulnerabilities detected by all vulnerability detection tools at the corresponding risk level in the detection task. The coverage is mapped to a discrete contribution value through the partition mapping rule. In order to further improve the accuracy of the detection result evaluation, the false alarm rate of the vulnerability detection tool at the corresponding risk level is also calculated. Each risk level is preset with a false alarm rate weight and a contribution value weight. The false alarm rate and the contribution value of the corresponding risk level are weighted and summed according to the corresponding weights to obtain the contribution of the tool at the corresponding risk level. Among them, the false alarm rate is the proportion of the number of false alarms detected by the vulnerability detection tool at the corresponding risk level to the total number of vulnerabilities detected by the tool at the risk level. The present invention not only reflects the coverage capability of the tool, but also pays attention to the detection quality of the vulnerability detection tool by combining the partition mapping contribution value and the false alarm rate of the tool, so as to provide more comprehensive technical support for subsequent tool performance evaluation and credibility scoring.

[0065] In one embodiment of the present invention, after determining the credibility of the corresponding vulnerability detection tool according to the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result, it also includes: generating a comprehensive standard detection result of all vulnerability detection tools based on the credibility of each vulnerability detection tool and the corresponding standard detection result. In order to provide users with a more comprehensive and accurate detection report, the platform determines the credibility of all vulnerability detection tools corresponding to the current task, and then combines the credibility with the corresponding standard detection results to generate a comprehensive and comprehensive comprehensive standard detection result. Furthermore, the standard detection results of each tool can be weighted and fused according to its credibility, and the detection results of high-credibility tools are given a higher weight to ensure that the comprehensive results are more accurate and comprehensive. This dynamic weight adjustment mechanism can effectively automatically optimize the report generation process according to factors such as the manufacturer's detection capabilities, coverage, and accuracy, thereby ensuring that the detection results are more objective and accurate.

[0066] Furthermore, during a security detection process, the present invention also supports the configuration of security detection tools from multiple manufacturers. The detection results of each manufacturer will be summarized in the same report, and the confidence of each security vulnerability will be assessed based on the degree of repetition of the detection results (i.e., whether multiple manufacturers have detected the same security issues). The higher the repetition, the greater the confidence of the vulnerability, and therefore a more credible vulnerability will be assigned. This method can effectively improve the accuracy and reliability of vulnerability identification and ensure the comprehensiveness and authority of the report.

[0067] In one embodiment of the present invention, after determining the credibility of the corresponding vulnerability detection tool according to the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result, it also includes: sorting the vulnerability detection tools of the same type according to the corresponding credibility, and giving priority to displaying vulnerability detection tools with high credibility. Since the credibility of different detection tools is different, for vulnerability detection tools with the same function, the platform will give priority to recommending tools with high credibility to users, so as to provide users with more accurate detection results.

[0068] See also Figure 2The evaluation system 100 of the vulnerability detection tool includes: a task information acquisition module 110, a tool allocation module 120, a vulnerability detection module 130, a result generation module 140, a result conversion module 150 and an evaluation module 160. The task information acquisition module 110 is used to acquire task information. The tool allocation module 120 is used to determine at least two matching vulnerability detection tools based on the task information. The vulnerability detection module 130 is used to convert the task information into a format that is suitable for the vulnerability detection tool according to the matching vulnerability detection tool. The result generation module 140 is used to detect the task information after the corresponding format conversion through the vulnerability detection tool and generate a detection result. The result conversion module 150 is used to convert each detection result into a standard detection result; wherein the standard detection result includes each vulnerability identified by the vulnerability detection tool and the corresponding risk level. The evaluation module 160 is used to determine the credibility of the corresponding vulnerability detection tool according to the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result.

[0069] For the specific definition of the evaluation system of the vulnerability detection tool, please refer to the definition of the evaluation method for the vulnerability detection tool in the above text, which will not be repeated here. Each module in the evaluation system of the above vulnerability detection tool can be implemented in whole or in part by software, hardware and their combination. The above modules can be embedded in or independent of the processor in the computer device in hardware format, or stored in the memory of the computer device in software format, so that the processor can call the operations corresponding to the above modules.

[0070] It should be noted that, in order to highlight the innovative part of the present invention, the present embodiment does not introduce modules that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other modules in the present embodiment.

[0071] See also Figure 3 The electronic device 1 may include a memory 12, a processor 13 and a bus, and may also include a computer program stored in the memory 12 and executable on the processor 13, such as an evaluation program of a vulnerability detection tool.

[0072] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 12 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 12 can not only be used to store application software and various types of data installed in the electronic device 1, such as the evaluation code of the vulnerability detection tool, etc., but also can be used to temporarily store data that has been output or is to be output.

[0073] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 1, and uses various interfaces and lines to connect various components of the entire electronic device 1, and executes or executes programs or modules (such as evaluation programs of vulnerability detection tools, etc.) stored in the memory 12, and calls data stored in the memory 12 to execute various functions of the electronic device 1 and process data.

[0074] The processor 13 executes the operating system and various installed application programs of the electronic device 1. The processor 13 executes the application programs to implement the steps in the evaluation method of the vulnerability detection tool.

[0075] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a task information acquisition module 110, a tool allocation module 120, a vulnerability detection module 130, a result generation module 140, a result conversion module 150, and an evaluation module 160.

[0076] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium, and the computer-readable storage medium can be non-volatile or volatile. The above-mentioned software function module is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to perform part of the functions of the evaluation method of the vulnerability detection tool described in each embodiment of the present application.

[0077] In summary, the present invention discloses an evaluation method, system, device and medium for a vulnerability detection tool. After obtaining the task information, the corresponding format conversion is performed according to the matching vulnerability detection tool, thereby avoiding the problem of task failure due to format mismatch and greatly improving the compatibility of the detection task. In addition, by using different detection tools, different vulnerability types and risk scenarios can be more comprehensively covered to achieve multi-dimensional vulnerability detection. The detection results of each tool are converted into a standard detection result, and by calculating the detection contribution ratio of each tool in different risk levels, and combining the vulnerability distribution of each risk level in all standard detection results, a dynamic evaluation of the detection credibility of each vulnerability detection tool is achieved, so that the credibility is quantified. Not only can the vulnerability situation be detected more comprehensively, but also a more stable and reliable detection result is provided to the user. Therefore, the present invention effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0078] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A vulnerability detection tool evaluation method, characterized in that: The method comprises: Get task information; matching at least two vulnerability detection tools based on the task information; According to the matching vulnerability detection tool, the task information is formatted to adapt to the vulnerability detection tool; Use vulnerability detection tools to detect the task information after the corresponding format conversion and generate detection results; Convert each test result into a standard test result; wherein the standard test result includes each vulnerability identified by the vulnerability detection tool and the corresponding risk level; The credibility of the corresponding vulnerability detection tool is determined based on the number of vulnerabilities of each risk level in all standard detection results, as well as the number of vulnerabilities of the corresponding risk level in each standard detection result.

2. The evaluation method of the vulnerability detection tool according to claim 1, characterized in that: For each vulnerability detection tool: Convert the detection results to standard detection results, including: Based on the identification of the vulnerability detection tool in the detection result, determine the standard fields that match the fields in the detection result; Generate corresponding standard test results based on standard fields.

3. The evaluation method of the vulnerability detection tool according to claim 1, characterized in that: Determining the credibility of the corresponding vulnerability detection tool according to the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result includes: Count the number of vulnerabilities of each risk level in all standard detection results; For each standard test result: Based on the number of vulnerabilities of each risk level in the standard detection results, and the number of vulnerabilities of the corresponding risk level in all standard detection results, the contribution of the corresponding vulnerability detection tool at each risk level is obtained; The contribution of the vulnerability detection tool at each risk level is weighted to obtain the credibility of the vulnerability detection tool.

4. The evaluation method of the vulnerability detection tool according to claim 3, characterized in that: The number of vulnerabilities of each risk level in the standard detection results and the number of vulnerabilities of the corresponding risk level in all standard detection results are used to obtain the contribution of the corresponding vulnerability detection tool at each risk level, including: Calculate the ratio of the number of vulnerabilities of each risk level in the standard test results to the number of vulnerabilities of the corresponding risk level in all standard test results to obtain the level coverage of the vulnerability detection tool at each risk level; For the level coverage of the vulnerability detection tool at each risk level: based on the preset partitioning rules, determine the coverage interval of the level coverage, and use the preset contribution value corresponding to the coverage interval as the contribution of the vulnerability detection tool at the risk level.

5. The evaluation method of the vulnerability detection tool according to claim 3, characterized in that: The number of vulnerabilities of each risk level in the standard detection results and the number of vulnerabilities of the corresponding risk level in all standard detection results are used to obtain the contribution of the corresponding vulnerability detection tool at each risk level, including: Calculate the ratio of the number of vulnerabilities of each risk level in the standard test results to the number of vulnerabilities of the corresponding risk level in all standard test results to obtain the level coverage of the vulnerability detection tool at each risk level; For the level coverage of vulnerability detection tools at each risk level: based on the preset partitioning rules, determine the coverage interval where the level coverage is located, and obtain the preset contribution value corresponding to the coverage interval; The false alarm rate of the vulnerability detection tool at each risk level is counted, and the false alarm rate of various risk levels and the corresponding preset contribution value are weighted and summed as the contribution of the vulnerability detection tool at the risk level.

6. The evaluation method of the vulnerability detection tool according to claim 1, characterized in that: After determining the credibility of the corresponding vulnerability detection tool based on the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result, it also includes: generating a comprehensive standard detection result of all vulnerability detection tools based on the credibility of each vulnerability detection tool and the corresponding standard detection results.

7. The evaluation method of a vulnerability detection tool according to claim 1, characterized in that: After determining the credibility of the corresponding vulnerability detection tool based on the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result, it also includes: sorting the vulnerability detection tools of the same category according to the corresponding credibility, and giving priority to displaying vulnerability detection tools with high credibility.

8. A vulnerability detection tool evaluation system, characterized in that: The system comprises: Task information acquisition module, used to obtain task information; A tool allocation module, configured to determine at least two matching vulnerability detection tools based on the task information; A vulnerability detection module, used for converting the format of the task information according to the matching vulnerability detection tool to adapt to the vulnerability detection tool; The result generation module is used to detect the task information after the corresponding format conversion through the vulnerability detection tool and generate the detection result; A result conversion module, used to convert each test result into a standard test result; wherein the standard test result includes each vulnerability identified by the vulnerability detection tool and the corresponding risk level; The evaluation module is used to determine the credibility of the corresponding vulnerability detection tool based on the number of vulnerabilities of each risk level in all standard detection results and the number of vulnerabilities of the corresponding risk level in each standard detection result.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the evaluation method of the vulnerability detection tool as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the evaluation method of the vulnerability detection tool according to any one of claims 1 to 7.

Images