Vulnerability scanning effect evaluation method and device and electronic equipment

Abstract

The invention relates to a vulnerability scanning effect evaluation method and device and electronic equipment, and belongs to the technical field of vulnerability scanning. According to the method, the vulnerability data generated when vulnerability scanning is carried out on each target network environment is preprocessed through the vulnerability scanning program, the index value of the basic index of each target network environment is obtained, and then the vulnerability scanning effect evaluation value of the scanning program on each target network environment is calculated; and finally, based on the vulnerability scanning effect evaluation value corresponding to each target network environment, determining the vulnerability scanning effect. Therefore, by integrating the scanning effects of the vulnerability scanning program on the system vulnerability, the service vulnerability and the Web vulnerability of each target network environment, the comprehensive assessment on the vulnerability scanning threat is realized at one time, so that the assessment result is more comprehensive and more accurate; therefore, a set of reasonable vulnerability scanning and vulnerability threat evaluation scheme can be provided for vulnerability utilization researchers or security defense developers.

Description

Technical Field

[0001] The present invention belongs to the technical field of vulnerability scanning, and in particular relates to a vulnerability scanning effect evaluation method, device and electronic equipment. Background Art

[0002] Computer networks are highly open and free, which makes people consider how to ensure that information is not stolen or destroyed when using computer networks to transmit or store information, and how to ensure personal or commercial interests. The main purpose of computer network security is to ensure the integrity, authenticity and confidentiality of all content on the network, to ensure the continuity and stability of network services, and to ensure network security. However, in the actual application of computer networks, there are many uncontrollable factors. These factors limit each network security defense mechanism to a limited range and are highly dependent on human factors. At the same time, no matter what application or network is used, there are inevitably vulnerabilities. Once someone discovers and exploits these vulnerabilities, various losses will be caused. Vulnerability scanning technology is a protection technology that scans for various vulnerabilities to find out the vulnerabilities. In response to this phenomenon, vulnerability scanning technology must have an update function to keep pace with the times and continuously strengthen its own functions and roles as network traffic and the number of devices increase. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] The technical problem to be solved by the present invention is to design a vulnerability scanning effect evaluation method, device, and electronic equipment to achieve a rationality evaluation of the vulnerability scanning effect in network attack and defense.

[0005] (II) Technical solution

[0006] In order to solve the above technical problems, the present invention provides a vulnerability scanning effect evaluation method, which is implemented based on a data source layer, an evaluation index layer, a capability evaluation layer and an evaluation system layer. Through the data of the data source layer, the determination of each indicator of the evaluation index layer is realized, and then the effect evaluation data of the capability evaluation layer is obtained based on the evaluation index layer. Finally, the evaluation value of the evaluation system layer is formed based on the effect evaluation data of the capability evaluation layer, and finally the effect evaluation of the entire vulnerability scanning process is realized;

[0007] The method comprises the following steps:

[0008] S201: Acquire vulnerability data generated by the vulnerability scanning program when performing vulnerability scanning on each target network environment this time, as data of the data source layer;

[0009] Vulnerability data refers to the data obtained when the vulnerability scanner scans the target host in the target network environment, including the process data generated during the vulnerability scanning process and the result data after the vulnerability scanning;

[0010] The data source layer specifies the target network environments that need to be analyzed to evaluate the vulnerability scanning effect of the vulnerability scanner. The more target network environments are set, the more reasonable and reliable the evaluation of the vulnerability scanner's scanning effect will be. The target network environments include system vulnerability environment, service vulnerability environment and Web vulnerability environment, which are used to evaluate the vulnerability scanning effect of the vulnerability scanner from three different aspects: system, Web and service.

[0011] S202: According to the vulnerability data of each target network environment, basic indicator values ​​of the target network environment in the evaluation indicator layer are calculated respectively;

[0012] The evaluation index layer specifies the basic indexes corresponding to each target network environment, and the basic index values ​​are directly obtained by preprocessing the vulnerability data of the data source layer; for system vulnerabilities, the basic indexes include the time required for vulnerability scanning and the number of correctly scanned vulnerabilities, while for Web vulnerabilities, the corresponding basic indexes include the time required for vulnerability scanning and the number of correctly scanned vulnerabilities; the time required for vulnerability scanning of system vulnerabilities and the number of correctly scanned vulnerabilities are calculated from the system-related data generated when the system vulnerability scanning program is used to scan the system vulnerabilities; the time required for vulnerability scanning of Web vulnerabilities and the number of correctly scanned vulnerabilities are calculated from the data related to Web vulnerabilities generated when the Web vulnerability scanning program is used to scan the Web vulnerabilities; the time required for vulnerability scanning of service vulnerabilities and the number of correctly scanned vulnerabilities are calculated from the data related to service vulnerabilities generated when the service vulnerability scanning program is used to scan the service vulnerabilities;

[0013] S203: Calculate the vulnerability scanning effect evaluation value of the vulnerability scanning program on each target network environment according to the basic indicator value in the evaluation indicator layer, as the evaluation value data of the capability evaluation layer;

[0014] In step S203, the evaluated capabilities include system vulnerability scanning capability, service vulnerability scanning capability, and Web vulnerability scanning capability, and the corresponding capability values ​​are obtained by weighted calculation of one or more of the vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness of the evaluation index layer;

[0015] S204: Determine the final effect of this vulnerability scan based on the vulnerability scan effect evaluation value corresponding to each target network environment and in combination with weight calculation.

[0016] The present invention also provides a vulnerability scanning effect evaluation device for implementing the method.

[0017] The invention also provides an electronic device for implementing the method.

[0018] (III) Beneficial effects

[0019] The present invention pre-processes the vulnerability data generated when the vulnerability scanning program performs vulnerability scanning on each target network environment, obtains the index value of the basic index of each target network environment, and then calculates the vulnerability scanning effect evaluation value of the scanning program on each target network environment, and finally determines the effect of this vulnerability scanning based on the vulnerability scanning effect evaluation value corresponding to each target network environment. In this way, by comprehensively analyzing the scanning effects of the vulnerability scanning program on the system vulnerabilities, service vulnerabilities, and Web vulnerabilities of each target network environment, a comprehensive evaluation of vulnerability scanning threats is achieved, making the evaluation results more comprehensive and accurate, thereby providing a set of reasonable vulnerability scanning and vulnerability threat evaluation solutions for vulnerability exploitation researchers or security defense developers. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of an evaluation system provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of a flow chart of a vulnerability scanning effect evaluation method provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of a specific vulnerability scanning and evaluation system provided by an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of the structure of a vulnerability scanning effect evaluation device provided by an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0026] Embodiment 1:

[0027] In order to achieve accurate evaluation of vulnerability scanning effects, a vulnerability scanning effect evaluation method is provided in an embodiment of the present application.

[0028] In order to implement the evaluation of the effect of vulnerability scanning, the embodiment of the present invention provides an evaluation system hierarchical architecture, which can be seen in Figure 1As shown, it includes a data source layer, an evaluation index layer, a capability evaluation layer, and an evaluation system layer. Through the data of the data source layer, the indicators of the evaluation index layer are determined, and then the effect evaluation of the capability evaluation layer is realized based on the evaluation index layer. Finally, the evaluation value of the evaluation system layer is formed based on the effect evaluation of the capability evaluation layer, and finally the effect evaluation of the entire vulnerability scanning process is realized.

[0029] exist Figure 1 Based on the evaluation system shown, you can refer to Figure 2 As shown, Figure 2 A schematic diagram of a process flow of a vulnerability scanning effect evaluation method provided in an embodiment of the present application, including:

[0030] S201: Data source layer: obtaining vulnerability data generated by the vulnerability scanning program when performing vulnerability scanning on each target network environment.

[0031] Vulnerability data refers to the data obtained when the vulnerability scanner scans the target host in the target network environment. It can include process data generated during the vulnerability scanning process and result data after the vulnerability scanning.

[0032] The vulnerability data obtained should be the data specified in the data source layer. The data source layer specifies the target network environments that need to be analyzed to evaluate the vulnerability scanning effect of the vulnerability scanner. There can be only one target network environment, but there can also be multiple target network environments. In theory, the more target network environments are set, the more reasonable and reliable the evaluation of the scanning effect of the vulnerability scanner will be.

[0033] The target network environment may include at least one of a system vulnerability environment, a service vulnerability environment, and a Web vulnerability environment. In order to more comprehensively implement the scanning effect evaluation of the vulnerability scanner, the target network environment may be set to include system vulnerabilities, Web vulnerabilities, and service vulnerabilities, thereby evaluating the vulnerability scanning effect of the vulnerability scanner from three different aspects: system, Web, and service.

[0034] S202: Evaluation index layer: According to the vulnerability data of each target network environment, basic index values ​​of the target network environment in the evaluation index layer are calculated respectively.

[0035] In this embodiment, the basic indicators corresponding to each target network environment are specified in the evaluation indicator layer, and the value is obtained by directly performing the preprocessing on the vulnerability data of the data source layer. The basic indicators corresponding to different target network environments can be the same or different. For example, for system vulnerabilities, the basic indicators can include the time required for vulnerability scanning and the number of correctly scanned vulnerabilities, while for Web vulnerabilities, the corresponding basic indicators can also include the time required for vulnerability scanning and the number of correctly scanned vulnerabilities. However, there are slight differences. For example, for the time required for vulnerability scanning of system vulnerabilities and the number of correctly scanned vulnerabilities, the system-related data generated when the system vulnerability scanning program is used to scan the system vulnerabilities is calculated; for the time required for vulnerability scanning of Web vulnerabilities and the number of correctly scanned vulnerabilities, the data related to the Web vulnerabilities generated when the Web vulnerability scanning program is used to scan the Web vulnerabilities is calculated; and for the time required for vulnerability scanning of service vulnerabilities and the number of correctly scanned vulnerabilities, the service vulnerability scanning program is used to scan the service vulnerabilities, and the data related to the service vulnerabilities is calculated.

[0036] Among them, the vulnerability scanning speed, vulnerability scanning accuracy and vulnerability scanning comprehensiveness are calculated as follows: suppose the time difference between the end time and the start time of the vulnerability scanning is t, the actual number of vulnerabilities is m (known), the number of vulnerabilities submitted after the vulnerability scanning is n (obtained after scanning), the number of correctly scanned vulnerabilities is o (where o≤n, o≤m, o is the intersection of m and n), m, n, o are all positive integers, and the vulnerability scanning speed is calculated to be o / t, the vulnerability scanning accuracy is o / n, and the vulnerability scanning comprehensiveness is o / m. For a specific vulnerability, if there is a relatively standard scanning time t1, the vulnerability scanning speed can also be calculated using |t-t1| / t1, where |x| represents the absolute value of the value x.

[0037] S203: Capability evaluation layer: Calculate the vulnerability scanning effect evaluation value of the vulnerability scanning program on each target network environment according to the basic indicator value in the evaluation indicator layer.

[0038] In this embodiment, the capabilities evaluated by the capability assessment layer include system vulnerability scanning capability, service vulnerability scanning capability, and Web vulnerability scanning capability. The corresponding capability values ​​are obtained by weighted calculation of one or more of the vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness of the evaluation indicator layer.

[0039] The basic indicator values ​​in the evaluation indicator layer may include multi-level indicators. For example, for system vulnerabilities, the first-level indicators include the vulnerability scanning speed of system vulnerabilities, the vulnerability scanning accuracy of system vulnerabilities, and the vulnerability scanning comprehensiveness of system vulnerabilities; the second-level indicators include the time consumed for vulnerability scanning, the number of correctly scanned vulnerabilities, the number of scanned vulnerabilities, and the number of real vulnerabilities; for Web vulnerabilities, the basic indicators include the vulnerability scanning speed of Web vulnerabilities, the vulnerability scanning accuracy of Web vulnerabilities, and the vulnerability scanning comprehensiveness of Web vulnerabilities; the second-level indicators include the time consumed for vulnerability scanning, the number of correctly scanned vulnerabilities, the number of scanned vulnerabilities, and the number of real vulnerabilities; for service vulnerabilities, the basic indicators may include the scanning speed of service vulnerabilities, the service vulnerability scanning accuracy, and the vulnerability scanning comprehensiveness of service vulnerabilities; the second-level indicators include the time consumed for vulnerability scanning, the number of correctly scanned vulnerabilities, the number of scanned vulnerabilities, and the number of real vulnerabilities.

[0040] However, the first-level indicators of each target network environment may also be different. For example, for system vulnerabilities, the first-level indicators may be the vulnerability scanning speed of system vulnerabilities and the vulnerability scanning accuracy of system vulnerabilities; for Web vulnerabilities, the first-level indicators may be the vulnerability scanning speed of Web vulnerabilities and the vulnerability scanning comprehensiveness of Web vulnerabilities; for service vulnerabilities, the first-level indicators may be the vulnerability scanning accuracy of service vulnerabilities and the vulnerability scanning comprehensiveness of service vulnerabilities.

[0041] The indicators for capability assessment layer evaluation, namely vulnerability scanning effect evaluation value, include system vulnerability scanning capability evaluation value, service vulnerability scanning capability evaluation value, Web vulnerability scanning capability evaluation value and other values, which can be calculated by weighted calculation of vulnerability scanning speed, vulnerability scanning accuracy and vulnerability scanning comprehensiveness.

[0042] S204: Determine the final effect of this vulnerability scan based on the vulnerability scan effect evaluation value corresponding to each target network environment and in combination with weight calculation.

[0043] It should be understood that in different evaluation tasks, the evaluation preferences for system vulnerability scanning capability, service vulnerability scanning capability, Web vulnerability scanning capability, etc. are different, that is, the weight coefficients are different. For example, in the vulnerability evaluation tasks at the operating system level such as Windows and Linux, the weight coefficient of the system vulnerability scanning capability is greater than the service vulnerability scanning capability and the Web vulnerability scanning capability; in the evaluation tasks for database services, SSH services, etc., the weight coefficient of the service vulnerability scanning capability is greater than the system vulnerability scanning capability and the Web vulnerability scanning capability; in the Web page security evaluation task, the weight coefficient of the Web vulnerability scanning capability is greater than the system vulnerability scanning capability and the service vulnerability scanning capability. Therefore, the final effect evaluation value of this vulnerability scan is obtained by weighted calculation of the vulnerability scanning effect evaluation values ​​corresponding to each target network environment.

[0044] In this embodiment, various existing weight value generation methods (such as expert scoring methods, etc.) can be used to implement the allocation of weight values ​​for each sub-item of the evaluation indicator layer (primary indicator, secondary indicator, multi-level indicator) and the capability evaluation layer.

[0045] In the above implementation process, the vulnerability data generated when the vulnerability scanning program is used to scan the vulnerabilities of each target network environment is preprocessed, including missing value filling, outlier processing, feature engineering, etc., to obtain the index values ​​of the basic indicators of each target network environment, and then calculate the vulnerability scanning effect evaluation value of the scanning program on each target network environment. Finally, based on the vulnerability scanning effect evaluation value corresponding to each target network environment, the effect of this vulnerability scanning is finally determined. In this way, by comprehensively analyzing the scanning effects of the vulnerability scanning program on system vulnerabilities, service vulnerabilities, and Web vulnerabilities of each target network environment, a comprehensive evaluation of vulnerability scanning threats is achieved, making the evaluation results more comprehensive and accurate, thereby providing a set of reasonable vulnerability scanning and vulnerability threat evaluation solutions for vulnerability exploitation researchers or security defense developers.

[0046] In the above implementation process, by evaluating system vulnerabilities, service vulnerabilities, Web vulnerabilities, etc., and through real and effective vulnerability scanning scenarios, the risk assessment of the vulnerability scanning program is achieved.

[0047] In the above implementation process, by constructing basic evaluation index values ​​of different dimensions and classifying them according to the dimensions, the weighted average method is used to form the superior evaluation index value. After layers of convergence, a tree-shaped evaluation system is finally formed, which can clearly show the overall context of the evaluation process.

[0048] In the above implementation process, the vulnerability scanning effect of the target network environment is calculated from at least one dimension of vulnerability scanning rate, vulnerability scanning accuracy and vulnerability scanning comprehensiveness, and is evaluated from three perspectives of scanning rate, scanning coverage and scanning accuracy, which is more in line with the scanning steps involved in the actual vulnerability scanning process.

[0049] In the above implementation process, the number of correct vulnerabilities scanned in the target network environment and the time used for the scan can be used to calculate the completion rate of this scanning behavior, and finally the effect evaluation index value of the vulnerability scanning behavior in the time dimension can be obtained.

[0050] In the above implementation process, the actual vulnerability list and the scanned vulnerability list are used to calculate the number of vulnerabilities scanned and the number of correct vulnerability scans submitted, and finally the accuracy of the scanning results for the system, service and Web in the scanning behavior is calculated, so as to obtain the scanning effect evaluation index value of a certain dimension in the vulnerability scanning behavior.

[0051] In the above implementation process, by using the number of vulnerabilities submitted by the personnel and the correct vulnerability array submitted, and calculating with the actual number of existing vulnerabilities, the value of the vulnerability scan comprehensiveness can be reasonably calculated, so that the vulnerability scanning effect evaluation results of the calculated vulnerability scanning program for each target network environment are more comprehensive and reliable.

[0052] Embodiment 2:

[0053] Within the framework of Example 1, a more subtle expression is used to present the vulnerability scanning assessment process, which is further illustrated for this application.

[0054] The vulnerability scanning assessment system in this embodiment can be found in Figure 3 shown.

[0055] When evaluating vulnerability capabilities, first follow Figure 3 The data specified in the data source layer shown is used to obtain vulnerability data, including vulnerability scanning start time, end time, submitted vulnerability set, scanned correct vulnerability set and real vulnerability set.

[0056] According to the following method, determine the system vulnerability scanning time, the number of vulnerabilities that actually exist in the system, the number of vulnerabilities submitted by the system vulnerability scanning, and the number of correct vulnerabilities in the system vulnerability scanning (these are all basic indicator values ​​of the target network environment in the evaluation indicator layer that need to be calculated in step 202), where Count() is a counting function and ∩ is an intersection calculation formula:

[0057] System vulnerability scan duration = system vulnerability scan end time - system vulnerability scan start time.

[0058] The correct vulnerability set of system vulnerability scanning = the vulnerability set submitted by system vulnerability scanning ∩ the vulnerability set that actually exists in the system.

[0059] The number of vulnerabilities that actually exist in the system = Count (the set of vulnerabilities that actually exist in the system).

[0060] The number of vulnerabilities submitted by the system vulnerability scan = Count (the set of vulnerabilities submitted by the system vulnerability scan).

[0061] The number of correct vulnerabilities submitted by the system vulnerability scan = Count (the set of correct vulnerabilities submitted by the system vulnerability scan).

[0062] The system vulnerability scanning speed, the system vulnerability scanning accuracy, and the system vulnerability scanning comprehensiveness (also belonging to the basic indicator values ​​calculated in step S202) are determined according to the following method:

[0063] System vulnerability scanning speed = number of correct vulnerabilities submitted by the system vulnerability scanning / system vulnerability scanning duration.

[0064] System vulnerability scan accuracy rate = number of correct vulnerabilities submitted by system vulnerability scan / number of vulnerabilities submitted by system vulnerability scan.

[0065] System vulnerability scan comprehensiveness = number of correct vulnerabilities submitted by the system vulnerability scan / number of actual vulnerabilities in the system.

[0066] The service vulnerability scanning duration, the number of real vulnerabilities of the service, the number of vulnerabilities submitted by the service vulnerability scanning, and the number of correct vulnerabilities of the service vulnerability scanning (also belonging to the basic indicator value calculated in step S202) are determined according to the following method, where Count() is a counting function and ∩ is an intersection calculation formula:

[0067] Service vulnerability scan duration = service vulnerability scan end time - service vulnerability scan start time.

[0068] The correct vulnerability set of the service vulnerability scan = the vulnerability set submitted by the service vulnerability scan ∩ the actual vulnerability set of the service.

[0069] The number of vulnerabilities that actually exist in the service = Count (the set of vulnerabilities that actually exist in the service).

[0070] The number of vulnerabilities submitted by the service vulnerability scan = Count (the set of vulnerabilities submitted by the service vulnerability scan).

[0071] The number of correct vulnerabilities submitted by the service vulnerability scan = Count (the set of correct vulnerabilities in the service vulnerability scan).

[0072] The service vulnerability scanning speed, service vulnerability scanning accuracy, and service vulnerability scanning comprehensiveness (also basic indicator values ​​calculated in step S202) are determined according to the following method:

[0073] Service vulnerability scanning speed = number of correct vulnerabilities submitted by the service vulnerability scanning / service vulnerability scanning duration.

[0074] Service vulnerability scan accuracy rate = number of correct vulnerabilities submitted by the service vulnerability scan / number of vulnerabilities submitted by the service vulnerability scan.

[0075] Comprehensiveness of service vulnerability scan = number of correct vulnerabilities submitted by the service vulnerability scan / number of actual vulnerabilities in the service.

[0076] The duration of the Web vulnerability scan, the number of real Web vulnerabilities, the number of vulnerabilities submitted by the Web vulnerability scan, and the number of correct Web vulnerability scans (also a basic indicator value calculated in step S202) are determined according to the following method, where Count() is a counting function and ∩ is an intersection calculation formula:

[0077] Web vulnerability scanning duration = Web vulnerability scanning end time - Web vulnerability scanning start time.

[0078] The correct vulnerability set of Web vulnerability scanning = the vulnerability set submitted by Web vulnerability scanning ∩ the real vulnerability set of Web.

[0079] The number of real vulnerabilities on the Web = Count (the set of real vulnerabilities on the Web).

[0080] The number of vulnerabilities submitted by the Web vulnerability scan = Count (the set of vulnerabilities submitted by the Web vulnerability scan).

[0081] The number of correct vulnerabilities submitted by the Web vulnerability scan = Count (the set of correct vulnerabilities in the Web vulnerability scan).

[0082] The web vulnerability scanning speed, web vulnerability scanning accuracy, and web vulnerability scanning comprehensiveness (also basic indicator values ​​calculated in step S202) are determined according to the following method:

[0083] Web vulnerability scanning speed = number of correct vulnerabilities submitted by the Web vulnerability scanning / duration of the Web vulnerability scanning.

[0084] Web vulnerability scanning accuracy rate = number of correct vulnerabilities submitted by Web vulnerability scanning / number of vulnerabilities submitted by Web vulnerability scanning.

[0085] Comprehensiveness of web vulnerability scanning = number of correct vulnerabilities submitted by web vulnerability scanning / number of real vulnerabilities on the web.

[0086] Then, call the preset system vulnerability scanning speed weight w 11 , the weight of the system vulnerability scanning accuracy w 12 , the weight of the comprehensiveness of system vulnerability scanning 13 , calculate the system vulnerability scanning capability of the vulnerability scanner by weighted summation; call the preset service vulnerability scanning speed weight w 21 , the weight of the service vulnerability scanning accuracy w 22 , the weight of the comprehensiveness of the service vulnerability scan w 23 , calculate the service vulnerability scanning capability of the vulnerability scanner by weighted summation; call the preset weight w of the Web vulnerability scanning speed 31 , the weight of the accuracy of Web vulnerability scanning w 32 , the weight of the comprehensiveness of Web vulnerability scanning 33 , the Web vulnerability scanning capability of the vulnerability scanner is calculated by weighted summation, as shown in the following formula:

[0087] System vulnerability scanning capability = w 11 *System vulnerability scanning speed +w 12 *System vulnerability scanning accuracy +w 13 * Comprehensiveness of system vulnerability scanning.

[0088] Service vulnerability scanning capability = w 21 *Service vulnerability scanning speed +w 22 *Service vulnerability scanning accuracy +w 23 * Comprehensiveness of service vulnerability scanning.

[0089] Web vulnerability scanning capability = w 31 *Web vulnerability scanning speed +w 32 *Web vulnerability scanning accuracy +w 33 * Comprehensiveness of web vulnerability scanning.

[0090] Next, the preset system vulnerability scanning capability weight W1, service vulnerability scanning capability weight W2 and Web vulnerability scanning capability weight W3 are called, and the final vulnerability scanning capability evaluation value Score of the vulnerability scanning program (which is the final effect to be obtained in step S204) is calculated by weighted summation, as shown in the following formula:

[0091] Score = W1*system vulnerability scanning capability + W2*service vulnerability scanning capability + W3*Web vulnerability scanning capability.

[0092] In this case, the vulnerability scanning capability is evaluated from three perspectives: system, service, and Web. The evaluation is comprehensive and can provide theoretical support for engineering implementation and task evaluation.

[0093] Embodiment three:

[0094] This embodiment also provides a vulnerability scanning effect evaluation device 100. Figure 4 As shown, the evaluation device is Figure 2 The device 100 is an overall package, and includes a collection module, a processing module, and an effect evaluation module. The device 100 runs on an electronic device in the form of software or in the form of a separate hardware. The details are as follows:

[0095] See also Figure 4 As shown, the device 100 includes: a collection module 101, a processing module 102 and an effect evaluation module 103. Among them:

[0096] The acquisition module 101 is a basic supporting component of the evaluation device, and is used to collect and obtain basic data generated when performing vulnerability scanning on each target network during the vulnerability scanning process;

[0097] The processing module 102 is the core computing component of the evaluation device, which is used to process and calculate the basic data obtained by the acquisition module, obtain the basic indicator value of the evaluation indicator layer, and calculate the vulnerability scanning effect evaluation value of the capability evaluation layer for each target network environment according to the basic indicator value of the evaluation indicator layer;

[0098] The effect evaluation module 103 is the final evaluation component of the evaluation device, and determines the effect of the vulnerability scan at the evaluation system layer through a weighted average method or other numerical calculation methods based on the vulnerability scan effect evaluation values ​​corresponding to each target network environment.

[0099] In the embodiment of the present application, with respect to the system vulnerability scanning capability, the collection module 101 mainly collects data such as the system vulnerability scanning start time, the system vulnerability scanning end time, the system vulnerability scanning submitted vulnerability set, the system vulnerability scanning correct vulnerability set, and the system real vulnerability set at the data source layer; the processing module 102 mainly performs calculations at the evaluation index layer and the capability evaluation layer. The evaluation index layer includes calculations of primary and secondary indicators. The primary indicators include the system vulnerability scanning duration, the number of vulnerabilities submitted by the system vulnerability scanning, the number of vulnerabilities correctly submitted by the system vulnerability scanning, and the number of real system vulnerabilities. The secondary indicators include the system vulnerability scanning rate, the system vulnerability scanning accuracy, and the system vulnerability scanning comprehensiveness. The capability evaluation layer mainly calculates the system vulnerability scanning capability.

[0100] Regarding the service vulnerability scanning capability, the collection module 101 mainly collects data such as the service vulnerability scanning start time, the service vulnerability scanning end time, the service vulnerability scanning submitted vulnerability set, the service vulnerability scanning correct vulnerability set, and the service real vulnerability set at the data source layer; the processing module 102 mainly performs calculations at the evaluation index layer and the capability evaluation layer. The evaluation index layer includes the calculation of the first-level and second-level indicators. The first-level indicators include the service vulnerability scanning time, the number of vulnerabilities submitted by the service vulnerability scanning, the number of vulnerabilities submitted correctly by the service vulnerability scanning, and the number of real service vulnerabilities. The second-level indicators include the service vulnerability scanning rate, the service vulnerability scanning accuracy rate, and the service vulnerability scanning comprehensiveness. The capability evaluation layer mainly calculates the service vulnerability scanning capability.

[0101] Regarding the Web vulnerability scanning capability, the collection module 101 mainly collects data such as the Web vulnerability scanning start time, the Web vulnerability scanning end time, the Web vulnerability scanning submitted vulnerability set, the Web vulnerability scanning correct vulnerability set, and the Web real vulnerability set at the data source layer; the processing module 102 mainly performs calculations at the evaluation index layer and the capability evaluation layer. The evaluation index layer includes the calculation of the first-level and second-level indicators. The first-level indicators include the Web vulnerability scanning time, the number of vulnerabilities submitted by the Web vulnerability scanning, the number of vulnerabilities submitted correctly by the Web vulnerability scanning, and the number of real Web vulnerabilities. The second-level indicators include the Web vulnerability scanning rate, the Web vulnerability scanning accuracy rate, and the Web vulnerability scanning comprehensiveness; the capability evaluation layer mainly calculates the Web vulnerability scanning capability.

[0102] The final vulnerability capability assessment result is obtained by the effect assessment module 103 through weighted calculation or other numerical calculation methods.

[0103] Embodiment 4:

[0104] This embodiment provides an electronic device, see Figure 5 As shown, it includes a calculator 501, a memory 502 and a communication link 503. Among them:

[0105] The calculator 501 has parallel computing capabilities and is used to execute one or more programs stored in the memory 502 to implement the vulnerability scanning effect evaluation method in the above-mentioned embodiment 1.

[0106] The memory 502 is used to store the evaluation calculation program and the fixed evaluation algorithm.

[0107] The communication link 503 is used to realize connection communication and data exchange between the calculator 501 and the memory 502 .

[0108] Understandably, Figure 5 The electronic device is only a schematic diagram of logic processing, and its internal structure and units of calculators and memories may use other components.

[0109] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A vulnerability scanning effect evaluation method, characterized in that: The method is implemented based on the data source layer, the evaluation index layer, the capability evaluation layer and the evaluation system layer. Through the data of the data source layer, the indicators of the evaluation index layer are determined, and then the effect evaluation data of the capability evaluation layer is obtained based on the evaluation index layer. Finally, the evaluation value of the evaluation system layer is formed based on the effect evaluation data of the capability evaluation layer, and finally the effect evaluation of the entire vulnerability scanning process is realized. The method comprises the following steps: S201: Acquire vulnerability data generated by the vulnerability scanning program when performing vulnerability scanning on each target network environment this time, as data of the data source layer; Vulnerability data refers to the data obtained when the vulnerability scanner scans the target host in the target network environment, including the process data generated during the vulnerability scanning process and the result data after the vulnerability scanning; The data source layer specifies the target network environments that need to be analyzed to evaluate the vulnerability scanning effect of the vulnerability scanner. The more target network environments are set, the more reasonable and reliable the evaluation of the vulnerability scanner's scanning effect will be. The target network environments include system vulnerability environment, service vulnerability environment and Web vulnerability environment, which are used to evaluate the vulnerability scanning effect of the vulnerability scanner from three different aspects: system, Web and service. S202: According to the vulnerability data of each target network environment, basic indicator values ​​of the target network environment in the evaluation indicator layer are calculated respectively; The evaluation index layer specifies the basic indexes corresponding to each target network environment, and the basic index values ​​are directly obtained by preprocessing the vulnerability data of the data source layer; for system vulnerabilities, the basic indexes include the time required for vulnerability scanning and the number of correctly scanned vulnerabilities, while for Web vulnerabilities, the corresponding basic indexes include the time required for vulnerability scanning and the number of correctly scanned vulnerabilities; the time required for vulnerability scanning of system vulnerabilities and the number of correctly scanned vulnerabilities are calculated from the system-related data generated when the system vulnerability scanning program is used to scan the system vulnerabilities; the time required for vulnerability scanning of Web vulnerabilities and the number of correctly scanned vulnerabilities are calculated from the data related to Web vulnerabilities generated when the Web vulnerability scanning program is used to scan the Web vulnerabilities; the time required for vulnerability scanning of service vulnerabilities and the number of correctly scanned vulnerabilities are calculated from the data related to service vulnerabilities generated when the service vulnerability scanning program is used to scan the service vulnerabilities; S203: Calculate the vulnerability scanning effect evaluation value of the vulnerability scanning program on each target network environment according to the basic indicator value in the evaluation indicator layer, as the evaluation value data of the capability evaluation layer; In step S203, the evaluated capabilities include system vulnerability scanning capability, service vulnerability scanning capability, and Web vulnerability scanning capability, and the corresponding capability values ​​are obtained by weighted calculation of one or more of the vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness of the evaluation index layer; S204: Determine the final effect of this vulnerability scan based on the vulnerability scan effect evaluation value corresponding to each target network environment and in combination with weight calculation.

2. The method according to claim 1, characterized in that In S202, the vulnerability scanning speed, vulnerability scanning accuracy and vulnerability scanning comprehensiveness are calculated as follows: suppose the time difference between the end time and the start time of the vulnerability scanning is t, the number of actual vulnerabilities is m, the number of vulnerabilities submitted after the vulnerability scanning is n, and the number of correctly scanned vulnerabilities is o, where o≤n, o≤m, o is the intersection of m and n, m, n, o are all positive integers, and after calculation, the vulnerability scanning speed is o / t, the vulnerability scanning accuracy is o / n, and the vulnerability scanning comprehensiveness is o / m. For a specific vulnerability, if there is a relatively standard scanning time t1, the vulnerability scanning speed is calculated using |t-t1| / t1, where |x| represents the absolute value of the value x.

3. The method according to claim 2, characterized in that The basic indicator values ​​in the evaluation indicator layer include multi-level indicators. For system vulnerabilities, the first-level indicators include the vulnerability scanning speed of system vulnerabilities, the vulnerability scanning accuracy of system vulnerabilities and the vulnerability scanning comprehensiveness of system vulnerabilities. The second-level indicators include the time consumed for vulnerability scanning, the number of correctly scanned vulnerabilities, the number of scanned vulnerabilities and the number of real vulnerabilities. For Web vulnerabilities, the basic indicators include the vulnerability scanning speed of Web vulnerabilities, the vulnerability scanning accuracy of Web vulnerabilities and the vulnerability scanning comprehensiveness of Web vulnerabilities. The second-level indicators include the time consumed for vulnerability scanning, the number of correctly scanned vulnerabilities, the number of scanned vulnerabilities and the number of real vulnerabilities. For service vulnerabilities, the basic indicators include the scanning speed of service vulnerabilities, the service vulnerability scanning accuracy and the vulnerability scanning comprehensiveness of service vulnerabilities. The second-level indicators include the time consumed for vulnerability scanning, the number of correctly scanned vulnerabilities, the number of scanned vulnerabilities and the number of real vulnerabilities.

4. The method according to claim 1, characterized in that When the first-level indicators of each target network environment are different, for system vulnerabilities, the first-level indicators are the vulnerability scanning speed of system vulnerabilities and the vulnerability scanning accuracy of system vulnerabilities; for Web vulnerabilities, the first-level indicators are the vulnerability scanning speed of Web vulnerabilities and the vulnerability scanning comprehensiveness of Web vulnerabilities; for service vulnerabilities, the first-level indicators are the vulnerability scanning accuracy of service vulnerabilities and the vulnerability scanning comprehensiveness of service vulnerabilities.

5. The method according to claim 3, characterized in that The indicators evaluated at the capability assessment layer, namely the vulnerability scanning effect evaluation value, include the system vulnerability scanning capability evaluation value, the service vulnerability scanning capability evaluation value, and the Web vulnerability scanning capability evaluation value. They are calculated by weighted calculation of the vulnerability scanning speed, vulnerability scanning accuracy, and vulnerability scanning comprehensiveness.

6. The method according to claim 5, characterized in that In different evaluation tasks, the evaluation preferences for system vulnerability scanning capability, service vulnerability scanning capability, and Web vulnerability scanning capability have different emphases, that is, the weight coefficients are different; for operating system-level vulnerability assessment tasks, the weight coefficient of system vulnerability scanning capability is greater than that of service vulnerability scanning capability and Web vulnerability scanning capability; for database service and SSH service assessment tasks, the weight coefficient of service vulnerability scanning capability is greater than that of system vulnerability scanning capability and Web vulnerability scanning capability; for Web page security assessment tasks, the weight coefficient of Web vulnerability scanning capability is greater than that of system vulnerability scanning capability and service vulnerability scanning capability; thus, the final effect evaluation value of this vulnerability scan is obtained by weighted calculation of the vulnerability scanning effect evaluation values ​​corresponding to each target network environment.

7. The method according to claim 6, characterized in that The vulnerability data includes vulnerability scanning start time, end time, submitted vulnerability set, scanned correct vulnerability set and real vulnerability set; According to the following method, determine the system vulnerability scanning time, the number of real vulnerabilities in the system, the number of vulnerabilities submitted by the system vulnerability scanning, and the number of correct vulnerabilities in the system vulnerability scanning, where Count() is the counting function and ∩ is the intersection calculation formula: System vulnerability scanning duration = system vulnerability scanning end time - system vulnerability scanning start time; Correct vulnerability set of system vulnerability scanning = vulnerability set submitted by system vulnerability scanning ∩ vulnerability set actually existing in the system; The number of vulnerabilities that actually exist in the system = Count (the set of vulnerabilities that actually exist in the system); the number of vulnerabilities submitted by the system vulnerability scan = Count (the set of vulnerabilities submitted by the system vulnerability scan); The number of correct vulnerabilities submitted by the system vulnerability scan = Count (the set of correct vulnerabilities submitted by the system vulnerability scan); Use the following method to determine the system vulnerability scanning speed, system vulnerability scanning accuracy, and system vulnerability scanning comprehensiveness: System vulnerability scanning speed = number of correct vulnerabilities submitted by system vulnerability scanning / system vulnerability scanning duration; System vulnerability scanning accuracy rate = number of correct vulnerabilities submitted by system vulnerability scanning / number of vulnerabilities submitted by system vulnerability scanning; System vulnerability scanning comprehensiveness = number of correct vulnerabilities submitted by system vulnerability scanning / number of vulnerabilities actually existing in the system; Use the following method to determine the duration of the service vulnerability scan, the number of vulnerabilities that actually exist in the service, the number of vulnerabilities submitted by the service vulnerability scan, and the number of correct vulnerabilities scanned by the service: Service vulnerability scanning duration = service vulnerability scanning end time - service vulnerability scanning start time; Correct vulnerability set of service vulnerability scanning = vulnerability set submitted by service vulnerability scanning ∩ actual vulnerability set of service; The number of vulnerabilities that actually exist in the service = Count (the set of vulnerabilities that actually exist in the service); The number of vulnerabilities submitted by the service vulnerability scan = Count (the set of vulnerabilities submitted by the service vulnerability scan); The number of correct vulnerabilities submitted by the service vulnerability scan = Count (the set of correct vulnerabilities in the service vulnerability scan); Determine the service vulnerability scanning speed, service vulnerability scanning accuracy, and service vulnerability scanning comprehensiveness according to the following methods: Service vulnerability scanning speed = number of correct vulnerabilities submitted by service vulnerability scanning / service vulnerability scanning duration; Service vulnerability scanning accuracy rate = number of correct vulnerabilities submitted by service vulnerability scanning / number of vulnerabilities submitted by service vulnerability scanning; Comprehensiveness of service vulnerability scanning = number of correct vulnerabilities submitted by service vulnerability scanning / number of actual vulnerabilities in the service; Use the following method to determine the duration of the Web vulnerability scan, the number of real Web vulnerabilities, the number of vulnerabilities submitted by the Web vulnerability scan, and the number of correct vulnerabilities in the Web vulnerability scan: Web vulnerability scanning duration = Web vulnerability scanning end time - Web vulnerability scanning start time; Web vulnerability scanning correct vulnerability set = Web vulnerability scanning submitted vulnerability set ∩ Web real vulnerability set; The number of real vulnerabilities on the Web = Count (the set of real vulnerabilities on the Web); Number of vulnerabilities submitted by Web vulnerability scanning = Count (set of vulnerabilities submitted by Web vulnerability scanning); The number of correct vulnerabilities submitted by the Web vulnerability scan = Count (the set of correct vulnerabilities in the Web vulnerability scan); Use the following methods to determine the web vulnerability scanning speed, web vulnerability scanning accuracy, and web vulnerability scanning comprehensiveness: Web vulnerability scanning speed = number of correct vulnerabilities submitted by Web vulnerability scanning / duration of Web vulnerability scanning; Web vulnerability scanning accuracy rate = number of correct vulnerabilities submitted by Web vulnerability scanning / number of vulnerabilities submitted by Web vulnerability scanning; Comprehensiveness of web vulnerability scanning = number of correct vulnerabilities submitted by web vulnerability scanning / number of real vulnerabilities on the web; Then, call the preset system vulnerability scanning speed weight w 11 , the weight of the system vulnerability scanning accuracy w 12 , the weight of the comprehensiveness of system vulnerability scanning 13 , calculate the system vulnerability scanning capability of the vulnerability scanner by weighted summation; call the preset service vulnerability scanning speed weight w 21 , the weight of the service vulnerability scanning accuracy w 22 , the weight of the comprehensiveness of the service vulnerability scan 23 , calculate the service vulnerability scanning capability of the vulnerability scanner by weighted summation; call the preset weight w of the Web vulnerability scanning speed 31 , the weight of the accuracy of Web vulnerability scanning w 32 , the weight of the comprehensiveness of Web vulnerability scanning 33 , the Web vulnerability scanning capability of the vulnerability scanner is calculated by weighted summation, that is, the following formula: System vulnerability scanning capability = w 11 *System vulnerability scanning speed +w 12 *System vulnerability scanning accuracy +w 13 * Comprehensiveness of system vulnerability scanning; Service vulnerability scanning capability = w 21 *Service vulnerability scanning speed +w 22 *Service vulnerability scanning accuracy +w 23 * Comprehensiveness of service vulnerability scanning; Web vulnerability scanning capability = w 31 *Web vulnerability scanning speed +w 32 *Web vulnerability scanning accuracy +w 33 * Comprehensiveness of web vulnerability scanning; Next, the preset system vulnerability scanning capability weight W1, service vulnerability scanning capability weight W2 and Web vulnerability scanning capability weight W3 are called, and the final vulnerability scanning capability evaluation value Score of the vulnerability scanning program is calculated by weighted summation as the final effect to be obtained in step S204, that is, the following formula: Score = W1*system vulnerability scanning capability + W2*service vulnerability scanning capability + W3*Web vulnerability scanning capability.

8. A device for evaluating the effect of vulnerability scanning for implementing the method according to any one of claims 1 to 7, characterized in that: The device comprises a collection module, a processing module and an effect evaluation module, wherein: The acquisition module is a basic supporting component of the evaluation device, and is used to collect and obtain basic data generated when performing vulnerability scanning on each target network during the vulnerability scanning process; The processing module is a computing component of the evaluation device, which is used to process and calculate the basic data obtained by the acquisition module, respectively, to obtain the basic indicator value of the evaluation indicator layer, and calculate the vulnerability scanning effect evaluation value of the capability evaluation layer for each of the target network environments according to the basic indicator value of the evaluation indicator layer; The effect evaluation module is the final evaluation component of the evaluation device. Based on the vulnerability scanning effect evaluation values ​​corresponding to each target network environment, the effect of the vulnerability scanning at the evaluation system layer is determined by weighted average method or other numerical calculation methods.

9. The device according to claim 8, characterized in that Regarding the system vulnerability scanning capability, the collection module is used to collect data on the system vulnerability scanning start time, system vulnerability scanning end time, system vulnerability scanning submitted vulnerability set, system vulnerability scanning correct vulnerability set, and system real vulnerability set at the data source layer; the processing module is used to calculate data at the evaluation index layer and the capability evaluation layer. The data calculation at the evaluation index layer includes the calculation of primary and secondary indicators. The primary indicator includes the system vulnerability scanning time, the number of vulnerabilities submitted by the system vulnerability scanning, the number of vulnerabilities submitted correctly by the system vulnerability scanning, and the number of real system vulnerabilities. The secondary indicator includes the system vulnerability scanning rate, the system vulnerability scanning accuracy, and the system vulnerability scanning comprehensiveness; the data calculation at the capability evaluation layer includes the calculation of the system vulnerability scanning capability; Regarding the service vulnerability scanning capability, the collection module is used to collect the service vulnerability scanning start time, service vulnerability scanning end time, service vulnerability scanning submitted vulnerability set, service vulnerability scanning correct vulnerability set, and service real vulnerability set data at the data source layer; the processing module is used to calculate the data at the evaluation index layer and the capability evaluation layer. The data calculation at the evaluation index layer includes the calculation of the first-level and second-level indicators. The first-level indicators include the service vulnerability scanning time, the number of vulnerabilities submitted by the service vulnerability scanning, the number of vulnerabilities submitted correctly by the service vulnerability scanning, and the number of real service vulnerabilities. The second-level indicators include the service vulnerability scanning rate, the service vulnerability scanning accuracy rate, and the service vulnerability scanning comprehensiveness; the data calculation at the capability evaluation layer includes the calculation of the service vulnerability scanning capability; Regarding Web vulnerability scanning capabilities, the collection module is used to collect data on the Web vulnerability scanning start time, Web vulnerability scanning end time, Web vulnerability scanning submitted vulnerability set, Web vulnerability scanning correct vulnerability set, and Web real vulnerability set at the data source layer; the processing module is used to calculate data at the evaluation index layer and the capability evaluation layer. The data calculation at the evaluation index layer includes primary and secondary indicator calculations. The primary indicators include Web vulnerability scanning duration, the number of vulnerabilities submitted by Web vulnerability scanning, the number of vulnerabilities submitted correctly by Web vulnerability scanning, and the number of real Web vulnerabilities. The secondary indicators include Web vulnerability scanning rate, Web vulnerability scanning accuracy, and Web vulnerability scanning comprehensiveness. The data calculation at the capability evaluation layer includes Web vulnerability scanning capability calculations.

10. An electronic device for implementing the method according to any one of claims 1 to 7, characterized in that: It includes a calculator, a memory and a communication link, wherein: The computer has parallel computing capability and is used to execute one or more programs stored in the memory to implement the method according to any one of claims 1 to 7; The memory is used to store the evaluation calculation program and the solidified evaluation algorithm; The communication link is used to realize the connection communication and data exchange between the calculator and the memory.

Images