Method, device and electronic equipment for evaluating the effect of a vulnerability scan

Abstract

The present application relates to a kind of effect evaluation method, device and electronic equipment of vulnerability scanning, belong to vulnerability scanning technical field.The present application is by the vulnerability data generated when each target network environment is scanned by vulnerability scanning program, and the index value of the basic index of each target network environment is obtained by pre-processing, and then the vulnerability scanning effect evaluation value of scanning program to each target network environment is calculated, finally based on the vulnerability scanning effect evaluation value corresponding to each target network environment, the effect of this vulnerability scanning is finally determined.In this way, by the scanning effect of vulnerability scanning program for the system vulnerability, service vulnerability, Web vulnerability of each target network environment, a comprehensive evaluation of vulnerability scanning threat is realized once, so that the evaluation result is more comprehensive and more accurate, so that a set of reasonable vulnerability scanning and vulnerability threat evaluation scheme can be provided for vulnerability exploitation researchers or security defense developers.

Description

Technical Field

[0001] This invention belongs to the field of vulnerability scanning technology, and specifically relates to a vulnerability scanning effectiveness evaluation method, device, and electronic device. Background Technology

[0002] Computer networks possess a high degree of openness and freedom, prompting consideration of how to ensure information is not stolen or damaged when transmitting or storing information, and how to safeguard personal or commercial interests. The primary purpose of computer network security is to ensure the integrity, authenticity, and confidentiality of all content on the network, the continuity and stability of network services, and network security. However, in practical applications of computer networks, many uncontrollable factors exist. These factors limit each network security defense mechanism to a finite scope and heavily rely on human factors. Furthermore, regardless of the application or network used, vulnerabilities are inevitable; once discovered and exploited, they can cause various losses. Vulnerability scanning technology is a protective technique that scans for and identifies vulnerabilities. To address this, vulnerability scanning technology must be constantly updated to evolve with the times, continuously enhancing its functionality and effectiveness as network traffic and the number of devices increase. Summary of the Invention

[0003] (a) Technical problems to be solved

[0004] The technical problem to be solved by the present invention is to design a vulnerability scanning effect evaluation method, device, and electronic device to realize the reasonable evaluation of vulnerability scanning effect in network attack and defense.

[0005] (II) Technical Solution

[0006] To address the aforementioned technical problems, this invention provides a vulnerability scanning effectiveness evaluation method. This method is implemented based on a data source layer, an evaluation indicator layer, a capability evaluation layer, and an evaluation system layer. By using data from the data source layer, the various indicators of the evaluation indicator layer are determined. Then, based on the evaluation indicator layer, the effectiveness evaluation data of the capability evaluation layer is obtained. Finally, based on the effectiveness evaluation data of the capability evaluation layer, the evaluation value of the evaluation system layer is formed, thereby achieving the effectiveness evaluation of the entire vulnerability scanning process.

[0007] The method includes the following steps:

[0008] S201: Obtain vulnerability data generated by the vulnerability scanner during this vulnerability scan of each target network environment, and use it as data from the data source layer;

[0009] Vulnerability data refers to the data obtained by a vulnerability scanning program when scanning a target host in a target network environment, including process data generated during the vulnerability scanning process and result data after the vulnerability scanning.

[0010] The data source layer specifies the target network environments that need to be analyzed to evaluate the vulnerability scanning effectiveness of the vulnerability scanner. The more target network environments set, the more reasonable and reliable the evaluation of the vulnerability scanner's scanning effectiveness will be. The target network environments include system vulnerability environments, service vulnerability environments, and web vulnerability environments, which are used to evaluate the vulnerability scanning effectiveness of the vulnerability scanner from three different aspects: system, web, and service.

[0011] S202: Based on the vulnerability data of each target network environment, calculate the basic index values ​​of the target network environment in the evaluation index layer;

[0012] The evaluation metric layer specifies the basic metrics for each target network environment. These basic metric values ​​are derived directly from preprocessed vulnerability data from the data source layer. For system vulnerabilities, the basic metrics include the time required for vulnerability scanning and the number of vulnerabilities scanned correctly. For web vulnerabilities, the corresponding basic metrics include the time required for vulnerability scanning and the number of vulnerabilities scanned correctly. The time required for system vulnerability scanning and the number of vulnerabilities scanned correctly are calculated from system-related data generated when a system vulnerability scanning program scans for system vulnerabilities. Similarly, the time required for web vulnerability scanning and the number of vulnerabilities scanned correctly are calculated from web vulnerability-related data generated when a web vulnerability scanning program scans for web vulnerabilities. Finally, the time required for service vulnerability scanning and the number of vulnerabilities scanned correctly are calculated from service vulnerability-related data generated when a service vulnerability scanning program scans for service vulnerabilities.

[0013] S203: Based on the basic index values ​​in the evaluation index layer, calculate the vulnerability scanning effect evaluation value of the vulnerability scanning program on each target network environment, and use it as the evaluation value data of the capability evaluation layer.

[0014] In step S203, the evaluated capabilities include system vulnerability scanning capabilities, service vulnerability scanning capabilities, and Web vulnerability scanning capabilities. The corresponding capability values ​​are calculated by weighting one or more of the following at the evaluation indicator layer: vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness.

[0015] S204: Based on the vulnerability scanning performance evaluation value corresponding to each target network environment, the final effect of this vulnerability scan is determined by weight calculation.

[0016] The present invention also provides an apparatus for evaluating the effectiveness of vulnerability scanning for implementing the method.

[0017] The present invention also provides an electronic device for implementing the method.

[0018] (III) Beneficial Effects

[0019] This invention preprocesses vulnerability data generated during vulnerability scanning of various target network environments by a vulnerability scanning program to obtain the basic indicator values ​​of each target network environment. Then, it calculates the vulnerability scanning effectiveness evaluation value for each target network environment. Finally, based on the vulnerability scanning effectiveness evaluation value corresponding to each target network environment, the final effectiveness of the vulnerability scan is determined. In this way, by comprehensively evaluating the scanning effectiveness of the vulnerability scanning program for system vulnerabilities, service vulnerabilities, and web vulnerabilities in various target network environments, a comprehensive assessment of vulnerability threats is achieved, making the assessment results more comprehensive and accurate. This provides vulnerability exploitation researchers or security defense developers with a reasonable vulnerability scanning and threat assessment scheme. Attached Figure Description

[0020] Figure 1 A schematic diagram of an evaluation system provided in an embodiment of the present invention;

[0021] Figure 2 A flowchart illustrating a vulnerability scanning effectiveness evaluation method provided in an embodiment of the present invention;

[0022] Figure 3 A schematic diagram illustrating a specific vulnerability scanning and assessment system provided in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the structure of a vulnerability scanning effectiveness evaluation device provided in an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0026] Example 1:

[0027] To achieve accurate evaluation of vulnerability scanning effectiveness, this application provides a vulnerability scanning effectiveness evaluation method.

[0028] To evaluate the effectiveness of vulnerability scanning, embodiments of this invention provide a hierarchical architecture for the evaluation system, which can be found in [reference needed]. Figure 1As shown, it includes a data source layer, an evaluation indicator layer, a capability assessment layer, and an evaluation system layer. Data from the data source layer is used to determine the various indicators in the evaluation indicator layer. Based on these indicators, the effectiveness of the capability assessment layer is evaluated. Finally, the evaluation value of the evaluation system layer is formed based on the effectiveness of the capability assessment layer, ultimately achieving an evaluation of the overall vulnerability scanning process.

[0029] exist Figure 1 Based on the evaluation system shown, you can refer to Figure 2 As shown, Figure 2 This is a flowchart illustrating the vulnerability scanning effectiveness evaluation method provided in the embodiments of this application, including:

[0030] S201: Data source layer: Obtain vulnerability data generated by the vulnerability scanner during this vulnerability scan of each target network environment.

[0031] Vulnerability data refers to the data obtained by a vulnerability scanning program when scanning a target host in a target network environment. It can include process data generated during the vulnerability scanning process and result data after the vulnerability scanning.

[0032] The vulnerability data acquired should be the data specified in the data source layer. The data source layer defines the target network environments that need to be analyzed to evaluate the vulnerability scanning effectiveness of a vulnerability scanner. There can be only one target network environment, or there can be multiple target network environments. Theoretically, the more target network environments specified, the more reasonable and reliable the evaluation of the vulnerability scanner's scanning effectiveness will be.

[0033] The target network environment can include at least one of the following: a system vulnerability environment, a service vulnerability environment, and a web vulnerability environment. To more comprehensively evaluate the scanning effectiveness of a vulnerability scanner, the target network environment can be configured to include system vulnerabilities, web vulnerabilities, and service vulnerabilities, thereby evaluating the vulnerability scanning effectiveness from three different perspectives: system, web, and service.

[0034] S202: Evaluation Indicator Layer: Based on the vulnerability data of each target network environment, the basic indicator values ​​of the target network environment in the evaluation indicator layer are calculated respectively.

[0035] In this embodiment, the evaluation index layer specifies basic indicators corresponding to each target network environment. These values ​​are obtained directly from the vulnerability data in the data source layer through the aforementioned preprocessing. The basic indicators corresponding to different target network environments can be the same or different. For example, for system vulnerabilities, the basic indicators may include the time required for vulnerability scanning and the number of vulnerabilities scanned correctly. For web vulnerabilities, the corresponding basic indicators may also include the time required for vulnerability scanning and the number of vulnerabilities scanned correctly. However, there are slight differences. For example, the time required for vulnerability scanning and the number of vulnerabilities scanned correctly for system vulnerabilities are calculated using system vulnerability scanning programs to scan system vulnerabilities. Similarly, the time required for vulnerability scanning and the number of vulnerabilities scanned correctly for web vulnerabilities are calculated using web vulnerability scanning programs to scan web vulnerabilities. And the time required for vulnerability scanning and the number of vulnerabilities scanned correctly for service vulnerabilities are calculated using service vulnerability scanning programs to scan service vulnerabilities to scan service vulnerabilities.

[0036] The vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness are calculated as follows: Let the time difference between the start and end time of the vulnerability scan be t, the actual number of vulnerabilities be m (known), the number of vulnerabilities submitted after the vulnerability scan be n (obtained from the scan), and the number of vulnerabilities scanned correctly be o (where o≤n, o≤m, and o is the intersection of m and n), where m, n, and o are all positive integers. The vulnerability scanning speed is calculated as o / t, the vulnerability scanning accuracy is o / n, and the vulnerability scanning comprehensiveness is o / m. For a specific vulnerability, if there is a relatively standard scanning time t1, the vulnerability scanning speed can also be calculated using |t-t1| / t1, where |x| represents the absolute value of x.

[0037] S203: Capability Assessment Layer: Based on the basic indicator values ​​in the assessment indicator layer, calculate the evaluation value of the vulnerability scanning effect of the vulnerability scanning program on each target network environment.

[0038] In this embodiment, the capabilities evaluated by the capability assessment layer include system vulnerability scanning capability, service vulnerability scanning capability, and Web vulnerability scanning capability. The corresponding capability values ​​are obtained by weighted calculation of one or more of the vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness of the assessment indicator layer.

[0039] The basic metrics in the evaluation metric layer can include multi-level metrics. For example, for system vulnerabilities, the primary metrics include vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness; the secondary metrics include vulnerability scanning time, the number of correctly scanned vulnerabilities, the number of vulnerabilities found, and the number of actual vulnerabilities. For web vulnerabilities, the basic metrics include web vulnerability scanning speed, web vulnerability scanning accuracy, and web vulnerability scanning comprehensiveness; the secondary metrics include vulnerability scanning time, the number of correctly scanned vulnerabilities, the number of vulnerabilities found, and the number of actual vulnerabilities. For service vulnerabilities, the basic metrics can include service vulnerability scanning speed, service vulnerability scanning accuracy, and service vulnerability scanning comprehensiveness; the secondary metrics include vulnerability scanning time, the number of correctly scanned vulnerabilities, the number of vulnerabilities found, and the number of actual vulnerabilities.

[0040] However, the primary metrics can differ for different target network environments. For example, for system vulnerabilities, the primary metrics could be the vulnerability scanning speed and the vulnerability scanning accuracy; for web vulnerabilities, the primary metrics could be the vulnerability scanning speed and the vulnerability scanning comprehensiveness; and for service vulnerabilities, the primary metrics could be the vulnerability scanning accuracy and the service vulnerability scanning comprehensiveness.

[0041] The performance indicators for the capability assessment layer, namely the vulnerability scanning effectiveness evaluation value, including the system vulnerability scanning capability evaluation value, service vulnerability scanning capability evaluation value, and Web vulnerability scanning capability evaluation value, can be calculated by weighting the vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness.

[0042] S204: Based on the vulnerability scanning performance evaluation value corresponding to each target network environment, the final effect of this vulnerability scan is determined by weight calculation.

[0043] It should be understood that different evaluation tasks may have different emphases on evaluation capabilities such as system vulnerability scanning, service vulnerability scanning, and web vulnerability scanning, meaning the weighting coefficients vary. For example, in tasks assessing vulnerabilities at the operating system level (Windows and Linux), the weighting coefficient for system vulnerability scanning is greater than that for service and web vulnerability scanning; in tasks assessing database and SSH services, the weighting coefficient for service vulnerability scanning is greater than that for system and web vulnerability scanning; and in tasks assessing web page security, the weighting coefficient for web vulnerability scanning is greater than that for system and service vulnerability scanning. Therefore, the final evaluation value of this vulnerability scan is obtained by weighting the vulnerability scanning effectiveness evaluation values ​​for each target network environment.

[0044] In this embodiment, various existing weight value generation methods (such as expert scoring methods) can be used to allocate weight values ​​for each item in the evaluation index layer (first-level index, second-level index, multi-level index) and the capability evaluation layer.

[0045] In the above implementation process, the vulnerability data generated during vulnerability scanning of each target network environment by the vulnerability scanning program is preprocessed, including missing value imputation, outlier handling, and feature engineering, to obtain the basic indicator values ​​of each target network environment. Then, the vulnerability scanning effectiveness evaluation value of the scanning program for each target network environment is calculated. Finally, based on the vulnerability scanning effectiveness evaluation value corresponding to each target network environment, the final effectiveness of this vulnerability scan is determined. In this way, by comprehensively evaluating the scanning effectiveness of the vulnerability scanning program for system vulnerabilities, service vulnerabilities, and web vulnerabilities in each target network environment, a comprehensive assessment of vulnerability scanning threats is achieved, making the assessment results more comprehensive and accurate. This provides vulnerability exploitation researchers or security defense developers with a reasonable vulnerability scanning and vulnerability threat assessment scheme.

[0046] In the above implementation process, the risk assessment of vulnerability scanning programs is achieved by evaluating system vulnerabilities, service vulnerabilities, web vulnerabilities, etc., and by using real and effective vulnerability scanning scenarios.

[0047] In the above implementation process, by constructing basic evaluation index values ​​of different dimensions and classifying them according to dimensions, the weighted average method is used to form higher-level evaluation index values. After being aggregated layer by layer, a tree-shaped evaluation system is finally formed, which can clearly show the overall context of the evaluation process.

[0048] In the above implementation process, the vulnerability scanning effect of the target network environment is calculated from at least one of the following dimensions: vulnerability scanning rate, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness. The evaluation is carried out from three perspectives: scanning rate, scanning coverage, and scanning accuracy, which is more in line with the scanning steps involved in the actual vulnerability scanning process.

[0049] In the above implementation process, by using the number of correct vulnerabilities scanned in the target network environment and the time taken for scanning, the completion rate of this scanning behavior can be calculated, and finally the effect evaluation index value of the vulnerability scanning behavior in the time dimension can be obtained.

[0050] In the above implementation process, the number of vulnerabilities detected and the number of correct vulnerability scans are calculated using the actual list of vulnerabilities and the list of vulnerabilities scanned. Finally, the accuracy rate of the scanning results for the system, service and web aspects in the scanning behavior is calculated, so as to obtain the scanning effect evaluation index value of a certain dimension in the vulnerability scanning behavior.

[0051] In the above implementation process, by utilizing the number of vulnerabilities submitted by personnel and the correct array of vulnerabilities submitted, and calculating with the actual number of vulnerabilities, the value of vulnerability scan comprehensiveness can be reasonably obtained, making the calculated vulnerability scanning program's evaluation results of vulnerability scanning effects on each target network environment more comprehensive and reliable.

[0052] Example 2:

[0053] Within the framework of Embodiment 1, a more detailed description is used to illustrate the vulnerability scanning and assessment process, providing further illustrative examples for this application.

[0054] The vulnerability scanning and assessment system in this embodiment can be found in [reference needed]. Figure 3 As shown.

[0055] When assessing vulnerability capabilities, first follow Figure 3 The data source layer shown indicates the data used to obtain vulnerability data, including the vulnerability scan start time, end time, submitted vulnerability set, correctly scanned vulnerability set, and actual vulnerability set.

[0056] The system vulnerability scan duration, the number of actual vulnerabilities in the system, the number of vulnerabilities submitted during the system vulnerability scan, and the number of correctly identified vulnerabilities in the system vulnerability scan (these are all basic indicator values ​​of the target network environment in the evaluation indicator layer to be calculated in step 202) are determined using the following method, where Count() is the counting function and ∩ is the intersection calculation formula:

[0057] System vulnerability scan duration = System vulnerability scan end time - System vulnerability scan start time.

[0058] The set of correct vulnerabilities in a system vulnerability scan = the set of vulnerabilities submitted in a system vulnerability scan ∩ the set of vulnerabilities that actually exist in the system.

[0059] The actual number of vulnerabilities in the system = Count(the actual set of vulnerabilities in the system).

[0060] The number of vulnerabilities submitted during system vulnerability scanning = Count(the set of vulnerabilities submitted during system vulnerability scanning).

[0061] The number of correct vulnerabilities submitted in the system vulnerability scan = Count(set of correct vulnerabilities in the system vulnerability scan).

[0062] The following methods are used to determine the system vulnerability scanning speed, system vulnerability scanning accuracy, and system vulnerability scanning comprehensiveness (which are also the basic indicator values ​​calculated in step S202):

[0063] System vulnerability scanning speed = number of correct vulnerabilities submitted during system vulnerability scanning / system vulnerability scanning time.

[0064] System vulnerability scan accuracy = Number of correct vulnerabilities submitted during system vulnerability scan / Number of vulnerabilities submitted during system vulnerability scan.

[0065] System vulnerability scan comprehensiveness = number of correct vulnerabilities submitted during system vulnerability scan / number of vulnerabilities actually existing in the system.

[0066] The following methods are used to determine the service vulnerability scan duration, the number of actual vulnerabilities in the service, the number of vulnerabilities submitted during the service vulnerability scan, and the number of correctly identified vulnerabilities in the service vulnerability scan (which are also basic indicator values ​​calculated in step S202), where Count() is the counting function and ∩ is the intersection calculation formula:

[0067] Service vulnerability scan duration = Service vulnerability scan end time - Service vulnerability scan start time.

[0068] The set of correct vulnerabilities in a service vulnerability scan = the set of vulnerabilities submitted in a service vulnerability scan ∩ the set of vulnerabilities that actually exist in the service.

[0069] The number of vulnerabilities that actually exist in the service = Count(the set of vulnerabilities that actually exist in the service).

[0070] The number of vulnerabilities submitted during a service vulnerability scan = Count(the set of vulnerabilities submitted during a service vulnerability scan).

[0071] The number of correct vulnerabilities submitted in the service vulnerability scan = Count(set of correct vulnerabilities in the service vulnerability scan).

[0072] The following methods are used to determine the service vulnerability scanning speed, service vulnerability scanning accuracy, and service vulnerability scanning comprehensiveness (which are also the basic indicator values ​​calculated in step S202):

[0073] Service vulnerability scanning speed = Number of correct vulnerabilities submitted for service vulnerability scanning / Service vulnerability scanning time.

[0074] Service vulnerability scan accuracy = Number of correct vulnerabilities submitted during service vulnerability scan / Number of vulnerabilities submitted during service vulnerability scan.

[0075] Service vulnerability scan comprehensiveness = number of correct vulnerabilities submitted during service vulnerability scan / number of vulnerabilities actually existing in the service.

[0076] The following methods are used to determine the Web vulnerability scan duration, the number of actual existing vulnerabilities on the Web, the number of vulnerabilities submitted during the Web vulnerability scan, and the number of correctly identified vulnerabilities in the Web vulnerability scan (which are also basic indicator values ​​calculated in step S202), where Count() is the counting function and ∩ is the intersection calculation formula:

[0077] Web vulnerability scan duration = Web vulnerability scan end time - Web vulnerability scan start time.

[0078] The set of correctly identified vulnerabilities in a web vulnerability scan = the set of vulnerabilities submitted for a web vulnerability scan ∩ the set of vulnerabilities that actually exist on the web.

[0079] The number of actual vulnerabilities on the web = Count(the set of actual vulnerabilities on the web).

[0080] The number of vulnerabilities submitted in a web vulnerability scan = Count(the set of vulnerabilities submitted in a web vulnerability scan).

[0081] The number of correct vulnerabilities submitted in a web vulnerability scan = Count(the set of correct vulnerabilities in a web vulnerability scan).

[0082] Determine the Web vulnerability scanning speed, Web vulnerability scanning accuracy, and Web vulnerability scanning comprehensiveness (which are also the basic indicator values ​​calculated in step S202) using the following methods:

[0083] Web vulnerability scanning speed = Number of correct vulnerabilities submitted for web vulnerability scanning / Web vulnerability scanning time.

[0084] Web vulnerability scan accuracy = Number of correct vulnerabilities submitted during web vulnerability scan / Number of vulnerabilities submitted during web vulnerability scan.

[0085] Web vulnerability scan comprehensiveness = number of correct vulnerabilities submitted in the web vulnerability scan / number of actual vulnerabilities on the web.

[0086] Then, the preset weight w for system vulnerability scanning speed is invoked. 11 The weight w of the system vulnerability scanning accuracy 12 Weight of the comprehensiveness of system vulnerability scanning w 13 The system vulnerability scanning capability of the vulnerability scanner is calculated by weighted summation; the preset service vulnerability scanning speed weight w is invoked. 21 Weight w of service vulnerability scanning accuracy 22 Weight of the comprehensiveness of service vulnerability scanning w 23 The service vulnerability scanning capability of the vulnerability scanner is calculated by weighted summation; the preset weight w for Web vulnerability scanning speed is invoked. 31 The weight of the accuracy of web vulnerability scanning w 32 Weight of the comprehensiveness of web vulnerability scanning w 33 The web vulnerability scanning capability of this vulnerability scanner is calculated by weighted summation, as shown in the following formula:

[0087] System vulnerability scanning capability = w 11 *System vulnerability scanning speed +w 12 *System vulnerability scan accuracy +w 13 *Comprehensiveness of system vulnerability scanning.

[0088] Service vulnerability scanning capability = w 21 *Service vulnerability scanning speed +w 22 *Service vulnerability scanning accuracy +w 23 *Comprehensiveness of service vulnerability scanning.

[0089] Web vulnerability scanning capability = w 31 *Web vulnerability scanning speed +w 32 *Web vulnerability scanning accuracy +w 33 *Comprehensiveness of Web vulnerability scanning.

[0090] Next, the preset weights W1 for system vulnerability scanning capability, W2 for service vulnerability scanning capability, and W3 for web vulnerability scanning capability are called, and the final vulnerability scanning capability evaluation value Score of the vulnerability scanning program is calculated by weighted summation (which is the final result to be obtained in step S204), as shown in the following formula:

[0091] Score = W1 * System vulnerability scanning capability + W2 * Service vulnerability scanning capability + W3 * Web vulnerability scanning capability.

[0092] This case study evaluates vulnerability scanning capabilities from three perspectives: system, service, and web. The comprehensive evaluation provides theoretical support for engineering implementation and task assessment.

[0093] Example 3:

[0094] This embodiment also provides a vulnerability scanning effectiveness evaluation device 100. Please refer to [link / reference]. Figure 4 As shown, the evaluation device is for Figure 2 The device is encapsulated as a whole, and internally includes a data acquisition module, a processing module, and an effect evaluation module. The device 100 runs in software form on an electronic device or as a standalone hardware unit. Details are as follows:

[0095] See Figure 4 As shown, the device 100 includes: a data acquisition module 101, a processing module 102, and an effect evaluation module 103. Wherein:

[0096] The acquisition module 101 is a basic support component of the evaluation device, used to collect and acquire basic data generated during vulnerability scanning of each target network.

[0097] The processing module 102 is the core computing component of the evaluation device. It is used to process and calculate the basic data obtained by the acquisition module to obtain the basic index values ​​of the evaluation index layer, and calculate the vulnerability scanning effect evaluation value of the capability evaluation layer for each target network environment based on the basic index values ​​of the evaluation index layer.

[0098] The effect evaluation module 103 is the final evaluation component of the evaluation device. Based on the vulnerability scanning effect evaluation value corresponding to each target network environment, it determines the effect of this vulnerability scan at the evaluation system layer by means of weighted average method or other numerical calculation method.

[0099] In this embodiment, regarding the system vulnerability scanning capability, the data acquisition module 101 mainly collects data such as the start time of system vulnerability scanning, the end time of system vulnerability scanning, the set of vulnerabilities submitted for system vulnerability scanning, the set of vulnerabilities correctly scanned for system vulnerability scanning, and the set of vulnerabilities actually existing in the system at the data source layer; the processing module 102 mainly performs calculations at the evaluation index layer and the capability evaluation layer. The evaluation index layer includes the calculation of primary and secondary indicators. The primary indicators include the system vulnerability scanning duration, the number of vulnerabilities submitted for system vulnerability scanning, the number of vulnerabilities correctly submitted for system vulnerability scanning, and the number of actual system vulnerabilities. The secondary indicators include the system vulnerability scanning rate, the system vulnerability scanning accuracy rate, and the system vulnerability scanning comprehensiveness; the capability evaluation layer mainly calculates the system vulnerability scanning capability.

[0100] Regarding service vulnerability scanning capabilities, the data collection module 101 primarily collects data at the data source layer, including service vulnerability scan start time, service vulnerability scan end time, service vulnerability scan submitted vulnerability set, service vulnerability scan correctly identified vulnerability set, and service vulnerability set that actually exists. The processing module 102 primarily performs calculations at the evaluation indicator layer and capability evaluation layer. The evaluation indicator layer includes calculations of primary and secondary indicators. Primary indicators include service vulnerability scan duration, number of vulnerabilities submitted during service vulnerability scan, number of correctly submitted vulnerabilities during service vulnerability scan, and number of actual existing service vulnerabilities. Secondary indicators include service vulnerability scan rate, service vulnerability scan accuracy, and service vulnerability scan comprehensiveness. The capability evaluation layer primarily calculates service vulnerability scanning capabilities.

[0101] Regarding Web vulnerability scanning capabilities, the data collection module 101 primarily collects data at the data source layer, including Web vulnerability scan start time, Web vulnerability scan end time, Web vulnerability scan submitted vulnerability set, Web vulnerability scan correctly identified vulnerability set, and Web vulnerability set that actually exists. The processing module 102 primarily performs calculations at the evaluation indicator layer and capability evaluation layer. The evaluation indicator layer includes calculations of primary and secondary indicators. Primary indicators include Web vulnerability scan duration, number of Web vulnerability scan submitted vulnerabilities, number of correctly submitted Web vulnerability scan vulnerabilities, and number of actual existing Web vulnerabilities. Secondary indicators include Web vulnerability scanning rate, Web vulnerability scan accuracy, and Web vulnerability scan comprehensiveness. The capability evaluation layer primarily calculates Web vulnerability scanning capabilities.

[0102] The final vulnerability assessment result is obtained through weighted calculation or other numerical calculation methods by the effectiveness assessment module 103.

[0103] Example 4:

[0104] This embodiment provides an electronic device, see [link / reference] Figure 5 As shown, it includes a calculator 501, a memory 502, and a communication link 503. Wherein:

[0105] The calculator 501 has parallel computing capabilities and is used to execute one or more programs stored in the memory 502 to implement the vulnerability scanning effect evaluation method in the first embodiment above.

[0106] The memory 502 is used to store the evaluation calculation program and the fixed evaluation algorithm.

[0107] Communication link 503 is used to realize the connection communication and data exchange between calculator 501 and memory 502.

[0108] Understandable. Figure 5 The electronic device is only a schematic diagram of logic processing; its internal structure and units of calculator and memory may use other components.

[0109] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for evaluating the effectiveness of vulnerability scanning, characterized in that, This method is based on a data source layer, an evaluation index layer, a capability evaluation layer, and an evaluation system layer. By using data from the data source layer, the various indicators of the evaluation index layer are determined. Then, the effectiveness evaluation data of the capability evaluation layer is obtained based on the evaluation index layer. Finally, the evaluation value of the evaluation system layer is formed based on the effectiveness evaluation data of the capability evaluation layer, thus achieving the effectiveness evaluation of the entire vulnerability scanning process. The method includes the following steps: S201: Obtain vulnerability data generated by the vulnerability scanner during this vulnerability scan of each target network environment, and use it as data from the data source layer; Vulnerability data refers to the data obtained by a vulnerability scanning program when scanning a target host in a target network environment, including process data generated during the vulnerability scanning process and result data after the vulnerability scanning. The data source layer specifies the target network environments that need to be analyzed to evaluate the vulnerability scanning effectiveness of the vulnerability scanner. The more target network environments set, the more reasonable and reliable the evaluation of the vulnerability scanner's scanning effectiveness will be. The target network environments include system vulnerability environments, service vulnerability environments, and web vulnerability environments, which are used to evaluate the vulnerability scanning effectiveness of the vulnerability scanner from three different aspects: system, web, and service. S202: Based on the vulnerability data of each target network environment, calculate the basic index values ​​of the target network environment in the evaluation index layer; The evaluation metric layer specifies the basic metrics for each target network environment. These basic metric values ​​are derived directly from preprocessed vulnerability data from the data source layer. For system vulnerabilities, the basic metrics include the time required for vulnerability scanning and the number of vulnerabilities scanned correctly. For web vulnerabilities, the corresponding basic metrics include the time required for vulnerability scanning and the number of vulnerabilities scanned correctly. The time required for system vulnerability scanning and the number of vulnerabilities scanned correctly are calculated from system-related data generated when a system vulnerability scanning program scans for system vulnerabilities. Similarly, the time required for web vulnerability scanning and the number of vulnerabilities scanned correctly are calculated from web vulnerability-related data generated when a web vulnerability scanning program scans for web vulnerabilities. Finally, the time required for service vulnerability scanning and the number of vulnerabilities scanned correctly are calculated from service vulnerability-related data generated when a service vulnerability scanning program scans for service vulnerabilities. S203: Based on the basic index values ​​in the evaluation index layer, calculate the vulnerability scanning effect evaluation value of the vulnerability scanning program on each target network environment, and use it as the evaluation value data of the capability evaluation layer. In step S203, the evaluated capabilities include system vulnerability scanning capabilities, service vulnerability scanning capabilities, and Web vulnerability scanning capabilities. The corresponding capability values ​​are calculated by weighting one or more of the following at the evaluation indicator layer: vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness. S204: Based on the vulnerability scanning performance evaluation value corresponding to each target network environment, the final effect of this vulnerability scan is determined by weight calculation.

2. The method as described in claim 1, characterized in that, In S203, the vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness are calculated as follows: Let the time difference between the end time and the start time of the vulnerability scan be t, the actual number of vulnerabilities be m, the number of vulnerabilities submitted after the vulnerability scan be n, and the number of vulnerabilities scanned correctly be o, where o≤n, o≤m, o is the intersection of m and n, and m, n, and o are all positive integers. The vulnerability scanning speed is calculated as o / t, the vulnerability scanning accuracy is o / n, and the vulnerability scanning comprehensiveness is o / m. For a specific vulnerability, if there is a relatively standard scanning time t1, the vulnerability scanning speed is calculated as |t-t1| / t1, where |x| represents the absolute value of the value x.

3. The method as described in claim 2, characterized in that, The basic metrics in the evaluation metric layer include multi-level metrics. For system vulnerabilities, the primary metrics include vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness. The secondary metrics include vulnerability scanning time, the number of correctly scanned vulnerabilities, the number of vulnerabilities found, and the number of actual vulnerabilities. For web vulnerabilities, the basic metrics include web vulnerability scanning speed, web vulnerability scanning accuracy, and web vulnerability scanning comprehensiveness. The secondary metrics include vulnerability scanning time, the number of correctly scanned vulnerabilities, the number of vulnerabilities found, and the number of actual vulnerabilities. For service vulnerabilities, the basic metrics include service vulnerability scanning speed, service vulnerability scanning accuracy, and service vulnerability scanning comprehensiveness. The secondary metrics include vulnerability scanning time, the number of correctly scanned vulnerabilities, the number of vulnerabilities found, and the number of actual vulnerabilities.

4. The method as described in claim 1, characterized in that, When the primary metrics differ for each target network environment, for system vulnerabilities, the primary metrics are the vulnerability scanning speed and the vulnerability scanning accuracy; for web vulnerabilities, the primary metrics are the vulnerability scanning speed and the vulnerability scanning comprehensiveness; and for service vulnerabilities, the primary metrics are the vulnerability scanning accuracy and the vulnerability scanning comprehensiveness.

5. The method as described in claim 3, characterized in that, The performance indicators for the capability assessment layer, namely the vulnerability scanning effectiveness evaluation value, include the system vulnerability scanning capability evaluation value, service vulnerability scanning capability evaluation value, and Web vulnerability scanning capability evaluation value. These are calculated by weighting vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness.

6. The method as described in claim 5, characterized in that, In different evaluation tasks, the evaluation preferences for system vulnerability scanning capabilities, service vulnerability scanning capabilities, and web vulnerability scanning capabilities vary, i.e., the weighting coefficients differ. For example, in the operating system-level vulnerability assessment task, the weighting coefficient for system vulnerability scanning capabilities is greater than that for service vulnerability scanning capabilities and web vulnerability scanning capabilities; in the database service and SSH service assessment tasks, the weighting coefficient for service vulnerability scanning capabilities is greater than that for system vulnerability scanning capabilities and web vulnerability scanning capabilities; and in the web page security assessment task, the weighting coefficient for web vulnerability scanning capabilities is greater than that for system vulnerability scanning capabilities and service vulnerability scanning capabilities. Therefore, the final evaluation value of this vulnerability scan is obtained by weighting the vulnerability scanning effectiveness evaluation values ​​corresponding to each target network environment.

7. The method as described in claim 6, characterized in that, The vulnerability data includes the vulnerability scan start time, end time, submitted vulnerability set, scanned correct vulnerability set, and actual vulnerability set. The following methods are used to determine the system vulnerability scan duration, the number of actual vulnerabilities in the system, the number of vulnerabilities submitted during the system vulnerability scan, and the number of correctly detected vulnerabilities in the system vulnerability scan, where Count() is the counting function and ∩ is the intersection calculation formula: System vulnerability scan duration = System vulnerability scan end time - System vulnerability scan start time; The set of correct vulnerabilities identified in a system vulnerability scan = the set of vulnerabilities submitted during a system vulnerability scan ∩ the set of vulnerabilities that actually exist in the system; The number of vulnerabilities actually existing in the system = Count (the set of vulnerabilities actually existing in the system); the number of vulnerabilities submitted during system vulnerability scans = Count (the set of vulnerabilities submitted during system vulnerability scans). The number of correct vulnerabilities submitted in the system vulnerability scan = Count (the set of correct vulnerabilities in the system vulnerability scan). Determine the system vulnerability scanning speed, system vulnerability scanning accuracy, and system vulnerability scanning comprehensiveness using the following methods: System vulnerability scanning speed = number of correct vulnerabilities submitted during system vulnerability scanning / system vulnerability scanning time; System vulnerability scan accuracy = Number of correct vulnerabilities submitted during system vulnerability scan / Number of vulnerabilities submitted during system vulnerability scan; System vulnerability scan comprehensiveness = number of correct vulnerabilities submitted during system vulnerability scan / number of vulnerabilities actually existing in the system; Determine the service vulnerability scan duration, the number of actual vulnerabilities in the service, the number of vulnerabilities submitted during the service vulnerability scan, and the number of correct vulnerabilities detected in the service vulnerability scan using the following methods: Service vulnerability scan duration = Service vulnerability scan end time - Service vulnerability scan start time; The set of correct vulnerabilities in a service vulnerability scan = the set of vulnerabilities submitted for a service vulnerability scan ∩ the set of vulnerabilities that actually exist in the service. The number of vulnerabilities that actually exist in the service = Count (the set of vulnerabilities that actually exist in the service); The number of vulnerabilities submitted during a service vulnerability scan = Count (the set of vulnerabilities submitted during a service vulnerability scan). The number of correct vulnerabilities submitted in the service vulnerability scan = Count (the set of correct vulnerabilities in the service vulnerability scan). Determine service vulnerability scanning speed, service vulnerability scanning accuracy, and service vulnerability scanning comprehensiveness using the following methods: Service vulnerability scanning speed = number of correct vulnerabilities submitted for service vulnerability scanning / service vulnerability scanning time; Service vulnerability scan accuracy = number of correct vulnerabilities submitted during service vulnerability scan / number of vulnerabilities submitted during service vulnerability scan; Comprehensiveness of service vulnerability scan = Number of correct vulnerabilities submitted during service vulnerability scan / Number of vulnerabilities actually existing in the service; Determine the web vulnerability scan duration, the number of actual web vulnerabilities, the number of vulnerabilities submitted during the web vulnerability scan, and the number of correctly identified vulnerabilities during the web vulnerability scan using the following methods: Web vulnerability scan duration = Web vulnerability scan end time - Web vulnerability scan start time; The set of correctly identified vulnerabilities in a web vulnerability scan = the set of vulnerabilities submitted for a web vulnerability scan ∩ the set of vulnerabilities that actually exist on the web; The number of actual vulnerabilities on the web = Count (the set of actual vulnerabilities on the web); The number of vulnerabilities submitted during a web vulnerability scan = Count (the set of vulnerabilities submitted during a web vulnerability scan); The number of correct vulnerabilities submitted in a web vulnerability scan = Count (the set of correct vulnerabilities in a web vulnerability scan). Determine the speed, accuracy, and comprehensiveness of web vulnerability scanning using the following methods: Web vulnerability scanning speed = number of correct vulnerabilities submitted for web vulnerability scanning / web vulnerability scanning time; Web vulnerability scanning accuracy = Number of correct vulnerabilities submitted during web vulnerability scanning / Number of vulnerabilities submitted during web vulnerability scanning; Web vulnerability scan comprehensiveness = number of correct vulnerabilities submitted in the web vulnerability scan / number of actual vulnerabilities in the web; Then, the preset weight w for system vulnerability scanning speed is invoked. 11 The weight w of the system vulnerability scanning accuracy 12 Weight of the comprehensiveness of system vulnerability scanning w 13 The system vulnerability scanning capability of the vulnerability scanner is calculated by weighted summation; the preset service vulnerability scanning speed weight w is invoked. 21 Weight w of service vulnerability scanning accuracy 22 Weight of the comprehensiveness of service vulnerability scanning w 23 The service vulnerability scanning capability of the vulnerability scanner is calculated by weighted summation; the preset weight w for Web vulnerability scanning speed is invoked. 31 The weight of the accuracy of web vulnerability scanning w 32 Weight of the comprehensiveness of web vulnerability scanning w 33 The web vulnerability scanning capability of this vulnerability scanner is calculated using a weighted summation method, as shown in the following formula: System vulnerability scanning capability = System vulnerability scanning speed + System vulnerability scan accuracy + Comprehensiveness of system vulnerability scanning; Service vulnerability scanning capability = Service vulnerability scanning speed + Service vulnerability scanning accuracy + Comprehensiveness of service vulnerability scanning; Web vulnerability scanning capability = Web vulnerability scanning speed + Web vulnerability scanning accuracy + Comprehensiveness of web vulnerability scanning; Next, the preset weights W1 for system vulnerability scanning capability, W2 for service vulnerability scanning capability, and W3 for web vulnerability scanning capability are called, and the final vulnerability scanning capability evaluation value Score of the vulnerability scanning program is calculated by weighted summation. This is the final result to be obtained in step S204, as shown in the following formula: Score= System vulnerability scanning capabilities+ Service vulnerability scanning capabilities+ Web vulnerability scanning capabilities.

8. An apparatus for evaluating the effectiveness of vulnerability scanning in implementing the method as described in any one of claims 1 to 7, characterized in that, The device includes a data acquisition module, a processing module, and an effect evaluation module, wherein: The acquisition module is a basic support component of the evaluation device, used to collect and acquire basic data generated during vulnerability scanning of each target network. The processing module is the computing component of the evaluation device, which is used to process and calculate the basic data obtained by the acquisition module to obtain the basic index values ​​of the evaluation index layer, and calculate the vulnerability scanning effect evaluation value of the capability evaluation layer for each target network environment based on the basic index values ​​of the evaluation index layer. The effect evaluation module is the final evaluation component of the evaluation device. Based on the vulnerability scanning effect evaluation value corresponding to each target network environment, it determines the effect of this vulnerability scan at the evaluation system layer through the weighted average method or other numerical calculation methods.

9. The apparatus as claimed in claim 8, characterized in that, Regarding system vulnerability scanning capabilities, the data acquisition module collects data at the data source layer, including system vulnerability scan start time, system vulnerability scan end time, submitted vulnerability set, correctly scanned vulnerability set, and actual existing vulnerability set. The processing module performs data calculations at the evaluation metric layer and capability evaluation layer. The evaluation metric layer calculations include primary and secondary metric calculations. Primary metrics include system vulnerability scan duration, number of submitted vulnerabilities, number of correctly submitted vulnerabilities, and number of actual existing system vulnerabilities. Secondary metrics include system vulnerability scan rate, system vulnerability scan accuracy, and system vulnerability scan comprehensiveness. The capability evaluation layer calculations include system vulnerability scanning capability calculations. Regarding service vulnerability scanning capabilities, the data collection module is used to collect data on the start time, end time, submitted vulnerability set, correctly scanned vulnerability set, and actual existing vulnerability set at the data source layer. The processing module is used to perform data calculations at the evaluation metric layer and capability evaluation layer. The evaluation metric layer includes primary and secondary metric calculations. Primary metrics include service vulnerability scanning duration, number of submitted vulnerabilities, number of correctly submitted vulnerabilities, and number of actual existing vulnerabilities. Secondary metrics include service vulnerability scanning rate, service vulnerability scanning accuracy, and service vulnerability scanning comprehensiveness. The capability evaluation layer includes calculations of service vulnerability scanning capabilities. Regarding web vulnerability scanning capabilities, the data collection module is used to collect data on the start time, end time, submitted vulnerability set, correctly scanned vulnerability set, and actual existing web vulnerability set at the data source layer. The processing module is used to perform data calculations at the evaluation metric layer and capability evaluation layer. The evaluation metric layer includes the calculation of primary and secondary metrics. Primary metrics include web vulnerability scanning duration, number of submitted vulnerabilities, number of correctly submitted vulnerabilities, and number of actual existing web vulnerabilities. Secondary metrics include web vulnerability scanning rate, web vulnerability scanning accuracy, and web vulnerability scanning comprehensiveness. The capability evaluation layer includes the calculation of web vulnerability scanning capabilities.

10. An electronic device for implementing the method according to any one of claims 1 to 7, characterized in that, Includes a calculator, memory, and communication links, among which: The calculator has parallel computing capabilities for executing one or more programs stored in memory to implement the method as described in any one of claims 1 to 7; The memory is used to store the evaluation calculation program and the fixed evaluation algorithm; The communication link is used to enable communication and data exchange between the calculator and the memory.

Images