Method, device, equipment and product for evaluating the capabilities of defense personnel in a network range

Abstract

The present invention discloses a method, device, equipment and product for evaluating the capabilities of defense personnel in a network range. The method includes: after the drill, obtaining the identification results of exploitable vulnerability conditions and the identification results of expected hazards of vulnerability exploitation of defense drill participants in the defense competition drill scenario in the network range; calculating the vulnerability identification ability score value; obtaining the execution data recorded by the vulnerability exploitation verification program of the network nodes in the drill scenario during the drill when calling the corresponding EXP attack script; calculating the vulnerability repair and reinforcement ability score value of the defense drill participants; and calculating the ability evaluation score value of the defense drill participants from the above respective score values. The present invention can accurately evaluate the vulnerability repair and reinforcement capabilities of defense personnel in a network range, avoid the problem that the evaluation of defense personnel's capabilities overly relies on the capabilities of offensive personnel, and avoid the problem of single evaluation indicators, improving the accuracy and rationality of security drill evaluation.

Description

Technical Field

[0001] The present invention relates to the field of IT application technology, and in particular to a method, device, terminal equipment and computer program product for evaluating the capabilities of defense personnel in a network shooting range. Background Art

[0002] When conducting capability assessments, existing network security training platforms or exercise systems either only quantitatively evaluate the defense capabilities of the range itself, but lack an assessment of the defense capabilities of the range personnel; or they only focus on "single vulnerability" exploitation and repair techniques, without the concept of scenarios. The capability assessment of defense personnel separated from the network range environment may be limited by the number of network nodes and the number of vulnerability types in the network area. The evaluation methods and evaluation indicators for trainees are relatively simple, and cannot accurately evaluate the true capability level of the personnel. Moreover, during security emergency drills, it is very likely to affect the availability and stability of real network services.

[0003] Furthermore, existing cyber range and cybersecurity drill personnel capability assessments focus on behavioral log monitoring in attack-defense scenarios. This approach provides a passive defense assessment, meaning the initiation of defensive actions may depend on attacking actions, and the defensive personnel capability assessment relies heavily on the capabilities of the red team. As the attacker's capabilities decline, the defender's evaluation dimensions decrease, eventually approaching zero, resulting in a lack of rationality in the defensive personnel evaluation results. Summary of the Invention

[0004] The main purpose of the present invention is to provide a method, device, terminal device and computer program product for evaluating the capabilities of defenders in a cyber shooting range, aiming to solve the problems of existing defender capability evaluation being overly dependent on the capabilities of red team personnel, having a single evaluation index and low evaluation accuracy, and to improve the accuracy and rationality of security drill evaluation.

[0005] To achieve the above objectives, an embodiment of the present invention provides a method for evaluating the capabilities of defense personnel in a network range, the method comprising the following steps:

[0006] After the drill is completed, the defense drill participants are provided with identification results of vulnerability exploitability conditions and expected damages of vulnerability exploitation in a defense competition drill scenario at a cyber range. During the drill, the defense drill participants conduct vulnerability remediation drills based on pre-designed drill scenarios.

[0007] Calculate the vulnerability identification ability score of the defense drill participant based on the identification result of the vulnerability exploitability condition and the identification result of the expected harm of vulnerability exploitation;

[0008] During the exercise, obtain the execution data recorded by the vulnerability exploitation verification program of the network nodes in the exercise scenario when calling the corresponding EXP attack script;

[0009] According to the execution data, calculate the vulnerability repair and reinforcement ability score value of the defense exercise participant;

[0010] According to the vulnerability identification ability score value of the defense exercise participant and the vulnerability repair and reinforcement ability score value, calculate the ability evaluation score value of the defense exercise participant.

[0011] Optionally, before the step of obtaining the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result of the defense exercise participant in the defense competition exercise scenario of the network range after the exercise, it further includes:

[0012] Design the exercise scenario, where the exercise scenario includes: an exercise network topology, multiple network nodes that may have vulnerabilities or cover multiple vulnerability types, a vulnerability exploitation route, and an EXP attack script for each network node with vulnerabilities.

[0013] Optionally, before the step of obtaining the vulnerability exploitable condition identification result of the defense exercise participant in the defense competition exercise scenario of the network range after the exercise, it further includes:

[0014] During the exercise, execute the vulnerability exploitation verification program of the network nodes in the exercise scenario, and call the corresponding EXP attack script at a preset time interval;

[0015] If the execution of the EXP attack script fails, switch to the EXP attack script variant and execute again until all EXP attack script variants fail to execute;

[0016] Through the vulnerability exploitation verification program, record the execution time, the number of successful executions, the number of failed executions, and the total number of executions of the EXP attack script and the EXP attack script variant, and record the sequence of the first failure event of a single EXP attack script corresponding to each defense exercise participant and arranged in chronological order, to obtain the vulnerability repair order and relevant parameters of each defense exercise participant as the execution data, and store it in the exercise database.

[0017] Optionally, the step of obtaining the vulnerability exploitable condition identification result of the defense exercise participant in the defense competition exercise scenario of the network range after the exercise includes:

[0018] After the drill, collect the number of vulnerabilities correctly identified by the defense drill participants under the drill scenario, where the exploitable conditions of the vulnerabilities are obtained by the defense drill participants evaluating the exploitable conditions of the vulnerabilities in the drill scenario based on the retrieval results of the vulnerability information database after discovering the vulnerabilities;

[0019] Based on the number of vulnerabilities correctly identified by the defense drill participants according to the exploitable conditions of the vulnerabilities, and the total number of actual vulnerabilities in the drill scenario, calculate the identification result of the exploitable conditions of the vulnerabilities.

[0020] Optionally, the step of obtaining the identification result of the expected harm of vulnerability exploitation by the defense drill participants in the defense competition drill scenario of the network range after the drill includes:

[0021] After the drill, collect the number of vulnerabilities correctly identified by the defense drill participants regarding the expected results of vulnerability exploitation under the drill scenario, where the expected results of vulnerability exploitation are obtained by the defense drill participants evaluating the actual possible harm of the exploitable vulnerabilities in the drill scenario based on the retrieval results of the vulnerability information database after identifying the exploitable vulnerabilities;

[0022] Based on the number of vulnerabilities correctly identified by the defense drill participants according to the expected results of vulnerability exploitation, and the total number of vulnerabilities preset in the drill scenario, calculate the identification result of the expected harm of vulnerability exploitation.

[0023] Optionally, the step of calculating the vulnerability repair and reinforcement ability score value of the defense drill participants according to the execution data includes:

[0024] Obtain the vulnerability repair order of the defense drill participants in the execution data;

[0025] Based on the vulnerability repair order of the defense drill participants, and the optimal vulnerability repair order and the total score of the repair strategy preset for the drill scenario, calculate the strategy score for the order of vulnerability repair of the defense drill participants;

[0026] Based on the execution data, obtain the total number of vulnerabilities repaired by the defense drill participants, the number of vulnerabilities of specific types repaired, and the number of EXP attack script variants with execution failures for a single vulnerability;

[0027] Based on the total number of vulnerabilities repaired by the defense drill participants, the number of vulnerabilities of specific types repaired, and the number of EXP attack script variants with execution failures for a single vulnerability, calculate the vulnerability repair and reinforcement score of the defense drill participants;

[0028] Based on the vulnerability repair sequence strategy score and the vulnerability repair and reinforcement score, the vulnerability repair and reinforcement ability score value of the defense exercise participant is calculated.

[0029] Optionally, the steps of calculating the vulnerability repair and reinforcement score of the defense exercise participant based on the total number of vulnerability repairs of the defense exercise participant, the number of vulnerability repairs of a specific type, and the number of EXP attack script variants with execution failures for a single vulnerability include:

[0030] Based on the total number of vulnerability repairs of the defense exercise participant, the number of EXP attack script variants with execution failures for a single vulnerability, and the total number of vulnerabilities in the preset exercise scenario, the overall average weighted vulnerability repair failure rate AFFR of the defense exercise participant is calculated;

[0031] Based on the number of vulnerability repairs of a specific type of the defense exercise participant, the number of EXP attack script variants with execution failures for a single vulnerability, and the total number of vulnerabilities of a specific type in the preset exercise scenario, the average weighted failure rate SFFR of vulnerability repair of a specific type of the defense exercise participant is calculated;

[0032] Based on the overall average weighted vulnerability repair failure rate AFFR and the average weighted failure rate SFFR of vulnerability repair of a specific type, the vulnerability repair and reinforcement score of the defense exercise participant is calculated.

[0033] In addition, an embodiment of the present invention further provides a device for evaluating the ability of defense personnel in a network range, and the device for evaluating the ability of defense personnel in the network range includes:

[0034] An identification result acquisition module, configured to, after the exercise ends, acquire the identification result of the exploitable condition of vulnerabilities and the identification result of the expected harm of vulnerability exploitation of the defense exercise participant in the defense competition exercise scenario of the network range, wherein during the exercise, the defense exercise participant conducts vulnerability repair exercises based on a pre-designed exercise scenario;

[0035] An identification ability calculation module, configured to calculate the vulnerability identification ability score value of the defense exercise participant according to the identification result of the exploitable condition of vulnerabilities and the identification result of the expected harm of vulnerability exploitation;

[0036] An execution record module, configured to acquire the execution data recorded when the vulnerability exploitation verification program of the network node in the exercise scenario calls the corresponding EXP attack script during the exercise;

[0037] A repair and reinforcement ability calculation module, configured to calculate the vulnerability repair and reinforcement ability score value of the defense exercise participant according to the execution data;

[0038] An evaluation score calculation module, configured to calculate an ability evaluation score value of the defense exercise participant according to the vulnerability identification ability score value and the vulnerability repair and reinforcement ability score value of the defense exercise participant.

[0039] In addition, an embodiment of the present invention further provides a terminal device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the method for evaluating the ability of a defense personnel in the network range as described above is implemented.

[0040] In addition, an embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method for evaluating the ability of a defense personnel in the network range as described above is implemented.

[0041] The method, device, terminal device, and computer program product for evaluating the ability of a defense personnel in the network range proposed in the embodiments of the present invention, after the exercise ends, obtain the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result of the defense exercise participant in the defense competition exercise scenario of the network range. Among them, during the exercise, the defense exercise participant performs a vulnerability repair exercise based on a pre-designed exercise scenario; calculates the vulnerability identification ability score value of the defense exercise participant according to the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result; obtains the execution data recorded by invoking the corresponding EXP attack script of the vulnerability exploitation verification program of the network node in the exercise scenario during the exercise; calculates the vulnerability repair and reinforcement ability score value of the defense exercise participant according to the execution data; calculates the ability evaluation score value of the defense exercise participant according to the vulnerability identification ability score value and the vulnerability repair and reinforcement ability score value of the defense exercise participant. The solution of the embodiment of the present invention, due to adopting a preset exercise scenario, can design a network topology structure, network nodes covering various vulnerability types, and vulnerability exploitation routes as needed in the exercise scenario to better evaluate the ability of the defense exercise participant, especially to evaluate the ability of the defense personnel from the perspectives of vulnerability identification, vulnerability repair, and reinforcement, and to examine the vulnerability exploitable condition identification ability, actual vulnerability harm assessment ability, vulnerability repair strategy, and the effectiveness of vulnerability repair and reinforcement of the defense exercise participant as a defense personnel in the network range scenario. Therefore, based on the solution of the embodiment of the present invention, it is possible to accurately evaluate the vulnerability repair and reinforcement ability of the defense personnel in the network range, not only avoiding the problem that the evaluation of the defense personnel's ability overly relies on the ability of the attacking personnel, but also solving the problem of single evaluation indicators, and improving the accuracy and rationality of the security exercise evaluation. Description of the Drawings

[0042] Figure 1It is a schematic diagram of the functional modules of the terminal device to which the defense personnel ability evaluation device in the network range of the present invention belongs;

[0043] Figure 2 It is a schematic flowchart of an exemplary embodiment of the method for evaluating the ability of defense personnel in the network range of the present invention;

[0044] Figure 3 It is a schematic flowchart of the Hamming distance algorithm in the embodiment of the method for evaluating the ability of defense personnel in the network range of the present invention;

[0045] Figure 4 It is a schematic flowchart of another exemplary embodiment of the method for evaluating the ability of defense personnel in the network range of the present invention.

[0046] The realization, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. Specific Embodiments

[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] The main solution of the embodiment of the present invention is: after the drill, obtain the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result of the defense drill participants in the defense competition drill scenario of the network range. Among them, during the drill, the defense drill participants conduct vulnerability repair drills based on the pre-designed drill scenario; calculate the vulnerability identification ability score value of the defense drill participants according to the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result; obtain the execution data recorded by the vulnerability exploitation verification program of the network nodes in the drill scenario calling the corresponding EXP attack script during the drill; calculate the vulnerability repair and reinforcement ability score value of the defense drill participants according to the execution data; calculate the ability evaluation score value of the defense drill participants according to the vulnerability identification ability score value and the vulnerability repair and reinforcement ability score value of the defense drill participants. The solution of the embodiment of the present invention, due to adopting a preset drill scenario, can design the network topology structure, network nodes covering various vulnerability types and vulnerability exploitation routes as needed in the drill scenario, so as to better evaluate the capabilities of the defense drill participants, especially evaluate the capabilities of the defense personnel from the perspectives of vulnerability identification, vulnerability repair and reinforcement, and examine the vulnerability exploitable condition identification ability, actual harm assessment ability of vulnerabilities, vulnerability repair strategies and the effectiveness of vulnerability repair and reinforcement of the defense drill participants as defense personnel in the network range scenario. Therefore, based on the solution of the embodiment of the present invention, it is possible to accurately evaluate the vulnerability repair and reinforcement capabilities of the defense personnel in the network range, not only avoid the problem that the evaluation of the defense personnel's capabilities overly relies on the capabilities of the attacking personnel, but also solve the problem of single evaluation indicators, and improve the accuracy and rationality of the security drill evaluation.

[0049] Technical terms involved in the embodiment of the present invention:

[0050] Range: The network range is an important infrastructure for network attack and defense drills and network new technology evaluations, used to improve the stability, security and performance of networks and information systems. Its main functions include: (1) Evaluation and verification of network attack and defense weapons; (2) Support for personnel training and competitions; (3) Scientific experiments and new technology verification.

[0051] EXP (exploit): Generally refers to a vulnerability exploitation program, which is used to achieve the attacker's purpose by exploiting vulnerabilities in software and is malicious.

[0052] Payload: The "effective payload" of an attack. It refers to the code or instructions that are actually executed in the target system after the vulnerability exploitation is successful.

[0053] Blue team repair and reinforcement drill: Refers to a drill mode in which only defense personnel (blue team) participate and there is no attack and defense confrontation. The defense personnel need to repair multiple known vulnerabilities within a specified time.

[0054] Red - side ability dependence disorder: It refers to the situation where the evaluation of the ability of the defensive personnel (blue - side) is inaccurate due to being limited by the level of the ability of the attacking personnel (red - side). For example, if the red - side fails to implement an attack behavior against a certain vulnerability, the repair effect of this vulnerability by the blue - side cannot be evaluated.

[0055] Vulnerability exploitable conditions: It refers to the constraint conditions that the defensive personnel need to overcome to exploit the vulnerabilities in the network range scenario. For example, access permissions of target network nodes, network connectivity constraints, request frequency limits, etc.

[0056] Expected effects of vulnerability exploitation: It refers to the expected effects that the defensive personnel can achieve after completing the exploitation of vulnerabilities in the network range scenario. For example, obtaining access addresses of other vulnerable network nodes in the scenario, valid user passwords, etc.

[0057] In the embodiments of the present invention, it is considered that in existing related solutions, for some network security competitions, such as CTF competitions, etc., single - vulnerability exploitation and repair techniques are emphasized, without the concept of scenarios, and the evaluation means and evaluation indicators are relatively single, unable to accurately evaluate the true ability level of personnel. Moreover, existing network range personnel ability evaluations often focus on evaluating the attacking - side personnel, and the evaluation of the defensive - side personnel is almost zero, that is, the attacking - side personnel get scores, and the defensive - side personnel lose corresponding scores. The defensive - side personnel are too passive under this evaluation scheme. Therefore, how to effectively evaluate the ability of defensive personnel in the network range environment is a difficult and significant topic.

[0058] Some existing network range defense ability evaluation schemes quantify the defense ability of the range itself, starting from two aspects, namely the severity of potential attack risks and the response of devices with defense functions during attack defense, and realizing the evaluation for design defects of the defense system and problems existing during the actual operation of the devices, quantifying the defense effect, and achieving an objective evaluation of the defense efficiency. However, it is impossible to evaluate the defense ability of the range personnel.

[0059] In addition, existing related network attack - defense analysis schemes propose game - theory methods. Through an attack - defense dynamic perception model based on game theory, the network security situation is analyzed in real - time, and the Nash equilibrium degree is used to maximize the benefits of both the attack - and defense - sides, so as to make a real - time and accurate evaluation of the network security situation, providing better reference for the network security defense decisions of the staff. This scheme focuses on calculating the maximum benefits of both the attack - and defense - sides. It is jointly participated by both the attack - and defense - sides, and uses game - theory methods to alleviate the problem that the defensive - side personnel are in a passive position in the evaluation of attack - defense actual combat ability, without calculating the vulnerability repair and reinforcement ability of the defensive personnel.

[0060] Some existing network range and network security drill personnel ability evaluation schemes focus on the behavior log monitoring in the scenario of human-to-human confrontation. Monitor the attack and defense behaviors on the virtual machines of the attacker and the defender respectively to form attack and defense behavior logs; obtain the attack and defense behavior logs from the virtual machines of the attacker and the defender; extract the key information of the attack and defense behaviors from the attack and defense behavior logs; match the key information of the attack and defense behaviors according to the pre-set scoring rules to determine the attack and defense experiment scores corresponding to the attacker and the defender, and will not actively sense the vulnerability repair and reinforcement behaviors of the defense personnel.

[0061] The existing related vulnerability risk basic evaluation scheme is based on the CVSS score. On the basis of the CVSS evaluation, it redesigned the weight allocation method of the basic evaluation indicators, optimized the weight allocation according to the relative importance of the basic evaluation indicators, and combined with the grey relational degree index weight solution method. Although it makes the evaluation results more objective, improves the diversity of the evaluation results, and is convenient to intuitively distinguish the threat level of vulnerabilities, this scheme is to optimize the weight allocation method of the basic evaluation indicators based on the CVSS score, does not involve the concept of scenario, and is a threat assessment of a single vulnerability. The expected effect of vulnerability exploitation proposed in this scheme refers to the actual threat level of the vulnerability and the actual harm that can be caused due to the permutation and combination differences of the vulnerability exploitation dependency relationships in the network range scenario.

[0062] In addition, the existing related network security emergency ability determination method, when determining that a network area is under a range of network vulnerability attacks, periodically sends vulnerability detection packets to the network devices in the network area, and finally determines the time required when the proportion of network devices with vulnerabilities in the network area is reduced to a preset proportion, and then determines the network security emergency ability of the network area based on the time. However, this scheme is separated from the network range scenario and judges the network security emergency ability from the proportion of vulnerability repair, and the evaluation dimension is relatively single. This scheme is applied to the network range to examine the vulnerability identification ability, vulnerability repair strategy formulation ability, and vulnerability repair and reinforcement ability of the defense personnel.

[0063] In fact, the existing network range and network security drill personnel ability evaluation schemes focus on the behavior log monitoring in the attack and defense confrontation scenario. However, this scheme cannot objectively and accurately evaluate the vulnerability repair and reinforcement ability of the defense personnel, and its defects are reflected in:

[0064] (1) This solution scores according to rules through the monitoring of offense and defense behavior logs. The offense and defense behaviors monitored by the offense and defense behavior monitoring unit include attack behaviors and defense behaviors. The attack behaviors include SQL injection behaviors, arbitrary file upload behaviors, cross-site scripting attack behaviors, and privilege escalation access behaviors. The defense behaviors include backdoor file removal behaviors, attacker account creation deletion behaviors, and blocking attacker access connection behaviors. Although this technical solution can achieve the matching and scoring of defense behavior events and can evaluate the attacker response ability of the defender, it does not examine the vulnerability exploitable condition identification and vulnerability actual harm assessment ability from the perspective of the defender.

[0065] (2) The defense evaluation of this solution is a non-active evaluation, that is, the initiation of defense behaviors may rely on attack behaviors, and the evaluation of the defender's ability is overly dependent on the ability of the red team personnel. For example, if the attacker personnel with limited ability fail to find exploitable vulnerabilities and backdoor establishment points for the attack target, that is, they ultimately fail to establish a backdoor, the backdoor removal behavior or "backdoor removal rate" of the defender becomes meaningless. Therefore, as the ability of the attacker personnel decreases, the evaluation dimensions of the defender will become fewer and fewer until they tend to zero, and the evaluation results of the defender personnel lack rationality.

[0066] In addition, an existing method for determining network security emergency response capabilities is to determine the network security emergency response capabilities based on the time required for the proportion of network devices with vulnerabilities in the network area to decrease to a preset proportion. However, this solution cannot well evaluate the vulnerability identification ability and vulnerability repair and reinforcement ability of the defender, and its defects are reflected in:

[0067] (1) This solution is separated from the network range environment. The network range environment, as a training ground for network security personnel and a test field for offense and defense weapons, can provide scenarios containing various types of vulnerabilities, simulate various network security risk events, and can comprehensively evaluate the defender's vulnerability repair and reinforcement ability. However, the evaluation of the defender's ability separated from the network range environment may be limited by the number of network nodes and the number of vulnerability types in the network area, resulting in inaccurate ability evaluation, and during the security emergency drill process, it is very likely to affect the availability and stability of real network services.

[0068] (2) This solution only calculates the proportion of vulnerable devices based on the response results of vulnerability detection packets, and determines the network security emergency response capabilities based on the time required for the proportion of network devices with vulnerabilities to decrease to a preset proportion. The implementation method of this solution is relatively crude, lacking the calculation of the defender's vulnerability repair strategy and vulnerability repair effect, and there is a problem of poor accuracy of the evaluation results of the defender's ability.

[0069] Based on the above analysis, an embodiment of the present invention proposes a method for evaluating the vulnerability repair and strengthening ability of the defense side in a network range. The drill scenario designer designs the drill network topology, network nodes covering various vulnerability types, and vulnerability exploitation routes, and evaluates the ability of the defense side personnel from the perspectives of vulnerability identification and vulnerability repair and strengthening. It examines the defense personnel's ability to identify exploitable conditions of vulnerabilities, evaluate the actual harm of vulnerabilities, vulnerability repair strategies, and the effectiveness of vulnerability repair and strengthening in the network range scenario. This method can solve the problems of over-reliance on the ability of the red side personnel in the evaluation of the defense personnel's ability and the single evaluation index.

[0070] Specifically, referring to Figure 1 , Figure 1 is a schematic diagram of the function modules of the terminal device to which the defense personnel ability evaluation device in the network range of the present invention belongs. The defense personnel ability evaluation device in the network range can be a device independent of the terminal device and capable of data processing, and it can be carried on the terminal device in the form of hardware or software. The terminal device can be an intelligent mobile terminal such as a mobile phone or a tablet computer, or a network device such as a server.

[0071] In this embodiment, the terminal device to which the defense personnel ability evaluation device in the network range belongs at least includes an output module 110, a processor 120, a memory 130, and a communication module 140.

[0072] The memory 130 stores an operating system and a defense personnel ability evaluation program in the network range; the output module 110 can be a display screen, a speaker, etc. The communication module 140 can include a WIFI module, a mobile communication module, a Bluetooth module, etc., and communicates with external devices or servers through the communication module 140.

[0073] Among them, as an embodiment, when the defense personnel ability evaluation program in the memory 130 is executed by the processor, the following steps are implemented:

[0074] After the drill, obtain the identification results of exploitable conditions of vulnerabilities and the identification results of expected harms of vulnerability exploitation of the defense drill participants in the defense competition drill scenario in the network range. Among them, during the drill, the defense drill participants conduct vulnerability repair drills based on the pre-designed drill scenario;

[0075] According to the identification results of exploitable conditions of vulnerabilities and the identification results of expected harms of vulnerability exploitation, calculate the vulnerability identification ability score value of the defense drill participants;

[0076] Obtain the execution data recorded by calling the corresponding EXP attack script of the vulnerability exploitation verification program of the network nodes in the drill scenario during the drill;

[0077] Calculate the vulnerability repair and reinforcement ability score value of the defense drill participant according to the execution data;

[0078] Calculate the ability evaluation score value of the defense drill participant according to the vulnerability identification ability score value of the defense drill participant and the vulnerability repair and reinforcement ability score value.

[0079] In this embodiment, through the above solution, after the drill, obtain the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result of the defense drill participant in the defense competition drill scenario of the network range. Among them, during the drill, the defense drill participant conducts a vulnerability repair drill based on a pre-designed drill scenario; calculate the vulnerability identification ability score value of the defense drill participant according to the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result; obtain the execution data recorded by the vulnerability exploitation verification program of the network node in the drill scenario calling the corresponding EXP attack script during the drill; calculate the vulnerability repair and reinforcement ability score value of the defense drill participant according to the execution data; calculate the ability evaluation score value of the defense drill participant according to the vulnerability identification ability score value of the defense drill participant and the vulnerability repair and reinforcement ability score value. In the solution of the embodiment of the present invention, since a preset drill scenario is adopted, in the drill scenario, the network topology structure, network nodes covering various vulnerability types, and vulnerability exploitation routes can be designed as needed to better evaluate the ability of the defense drill participant, especially to evaluate the ability of the defense personnel from the perspectives of vulnerability identification, vulnerability repair, and reinforcement, and to examine the vulnerability exploitable condition identification ability, actual vulnerability harm assessment ability, vulnerability repair strategy, and the effectiveness of vulnerability repair and reinforcement of the defense drill participant as a defense personnel in the network range scenario. Therefore, based on the solution of the embodiment of the present invention, it is possible to accurately evaluate the vulnerability repair and reinforcement ability of the defense personnel in the network range, not only avoiding the problem that the evaluation of the defense personnel's ability overly relies on the ability of the attacking personnel, but also solving the problem of single evaluation index, and improving the accuracy and rationality of the security drill evaluation.

[0080] Based on the above terminal device architecture but not limited to the above architecture, the method embodiment of the present invention is proposed.

[0081] Refer to Figure 2 , Figure 2 is a schematic flowchart of an exemplary embodiment of the method for evaluating the ability of defense personnel in the network range of the present invention. The method for evaluating the ability of defense personnel in the network range includes:

[0082] Step S101: After the drill, obtain the identification results of exploitable conditions of vulnerabilities and the identification results of expected hazards of vulnerability exploitation in the defense competition drill scenario of the network range by the defense drill participants. During the drill, the defense drill participants conduct vulnerability repair drills based on the pre-designed drill scenario.

[0083] Specifically, the solution of this embodiment proposes a new network range defense competition drill mode, namely the blue team repair and reinforcement drill. During the blue team repair and reinforcement drill, only defense personnel participate, and there is no attack and defense confrontation link. The defense personnel need to repair multiple known vulnerabilities within a specified time. Based on this mode, accurately evaluate the vulnerability repair and reinforcement capabilities of the defense personnel in the network range, and examine the vulnerability identification capabilities and vulnerability repair and reinforcement capabilities based on the drill scenario.

[0084] The above blue team repair and reinforcement drill life cycle is divided into three stages: drill design, drill data collection, and drill evaluation.

[0085] In this embodiment, a drill scenario is pre-designed in the drill design stage, and relevant drill data is pre-collected in the drill data collection stage.

[0086] Among them, during the drill, the defense drill participants, as defense personnel, conduct vulnerability repair drills based on the pre-designed drill scenario.

[0087] Among them, the drill scenario includes: a drill network topology structure, multiple network nodes that may have vulnerabilities or cover various vulnerability types, vulnerability exploitation routes, and EXP attack scripts for each network node with vulnerabilities.

[0088] Specifically, in the drill design stage, the drill scenario designer needs to prepare the network topology structure of the drill scenario, multiple network nodes that may have vulnerabilities or cover various vulnerability types, vulnerability exploitation routes, and EXP attack scripts for each network node with vulnerabilities.

[0089] Among them, the network topology structure should include multi-level networks. Multiple network nodes in the network are assigned to different internal networks. There are hierarchical relationships and flexibly configurable network connectivity between internal networks to simulate a complex real network. The network hierarchical relationships and network connectivity are designed by the drill scenario designer.

[0090] The network node is the host in the above network topology and contains a certain number of vulnerabilities. Therefore, the drill scenario designer needs to prepare several different types of vulnerability application environments based on different operating system platforms and different application development languages, such as Windows system vulnerabilities / Microsoft Office suite vulnerabilities, PHP software vulnerabilities, Java software vulnerabilities, JavaScript software vulnerabilities, etc., so as to subdivide the multi-dimensional capabilities of the personnel in the defense range. Each vulnerability should have a scenario dependency relationship. For example, through vulnerability exploitation of the previous network node, certain information (domain names or IP addresses of other internal services, valid account passwords of other internal services, etc.) can be obtained. Due to the information exposure caused by the vulnerability of the previous network node, the originally non-exploitable vulnerability of the subsequent network node becomes an exploitable vulnerability.

[0091] The vulnerability exploitation route is a combination of the vulnerability exploitation effects of network nodes. When designing the vulnerability exploitation route, the drill scenario designer should pay attention to the rationality of the route, ensure the vulnerability exploitation feasibility of all vulnerability nodes in the vulnerability exploitation route, and finally achieve the maximum harm effect, and provide the optimal repair strategy and the total score of the repair strategy for the scenario. The drill scenario designer also needs to define a set of key network nodes. Any of the following two conditions can be used to determine a key network node: (1) It has a function of connecting the previous and the next. For example, it is a necessary node for an attacker to penetrate into the next intranet during network penetration; (2) It is a network node with the effect of exploiting high-risk vulnerabilities. At the same time, the full score for the repair of each vulnerability is set comprehensively according to whether it is a key network node and the harm degree of vulnerability exploitation. The distribution of the full score values for vulnerability repair in the scenario conforms to the normal distribution.

[0092] The EXP attack script for each network node with vulnerabilities is called by the vulnerability exploitation verification program, and it can be determined whether the vulnerability of the target environment has been repaired according to whether the EXP attack script execution fails. The EXP attack script should have variants, such as changing the exploitation method, changing the injection point, changing the payload (variants that can bypass the WAF and redundant string variants), etc.

[0093] Relevant drill data is collected in advance during the drill data collection phase.

[0094] During the drill process, the vulnerability exploitation verification program of the network node runs continuously, executes the EXP attack script at a fixed time interval. If the EXP attack script execution fails, it automatically switches to the variant of the EXP attack script and executes again until all variants have failed to execute. The vulnerability exploitation verification program completely records the attack time, the number of successful attacks, the number of failed attacks, and the total number of attacks of the EXP attack script and its variants.

[0095] After the drill, in the form of a questionnaire survey, collect the vulnerability exploitable condition identification results and vulnerability exploitation expected harm identification results of the drill participants, and calculate the vulnerability identification ability score value based on the drill scenario. At the same time, according to the data recorded by the vulnerability exploitation verification program during the drill, calculate the vulnerability repair and reinforcement ability score value of the drill participants, and finally comprehensively calculate the final ability evaluation score value of the drill participants.

[0096] Specifically, after the drill, first obtain the vulnerability exploitable condition identification results and vulnerability exploitation expected harm identification results of the defense drill participants as defenders in the defense competition drill scenario of the network range.

[0097] Specifically, as an implementation method, after the drill, to obtain the vulnerability exploitable condition identification results of the defense drill participants in the defense competition drill scenario of the network range, the following scheme can be adopted:

[0098] After the drill, collect the number of vulnerabilities correctly identified for the exploitable conditions of the vulnerabilities identified by the defense drill participants in the drill scenario, where the exploitable conditions of the vulnerabilities are obtained by the defense drill participants based on the retrieval results of the vulnerability information database to evaluate the exploitable conditions of the vulnerabilities in the drill scenario after discovering the vulnerabilities;

[0099] Based on the number of vulnerabilities correctly identified for the exploitable conditions of the defense drill participants and the total number of actual vulnerabilities in the drill scenario, calculate the vulnerability exploitable condition identification results.

[0100] Specifically, as an implementation method, after the drill, to obtain the vulnerability exploitation expected harm identification results of the defense drill participants in the defense competition drill scenario of the network range, the following scheme can be adopted:

[0101] After the drill, collect the number of vulnerabilities correctly identified for the expected results of vulnerability exploitation identified by the defense drill participants in the drill scenario, where the expected results of vulnerability exploitation are obtained by the defense drill participants based on the retrieval results of the vulnerability information database to evaluate the actual possible harm of the exploitable vulnerabilities in the drill scenario after identifying the exploitable vulnerabilities;

[0102] Based on the number of vulnerabilities correctly identified for the expected results of vulnerability exploitation of the defense drill participants and the total number of vulnerabilities preset in the drill scenario, calculate the vulnerability exploitation expected harm identification results.

[0103] The following elaborates in detail the calculation processes of the vulnerability exploitable condition identification results and the vulnerability exploitation expected harm identification results:

[0104] In this embodiment, the evaluation value of the vulnerability identification ability based on the drill scenario is comprehensively calculated by two evaluation indicators. One of the indicators is the exploit condition recognition rate index ECRR, corresponding to the above-mentioned exploit condition recognition result of the vulnerability; the other indicator is the expected result recognition rate index of vulnerability exploitation ERRR, corresponding to the above-mentioned recognition result of the expected harm of vulnerability exploitation.

[0105] Among them, for the exploit condition recognition rate index (ECRR, Exploit Condition RecognitionRate), the exploit conditions in the vulnerability information database often deviate from the real network scenario. This exploit condition recognition rate index focuses on examining the ability of the defense personnel to judge the exploit conditions of vulnerabilities in the real network environment. The defense personnel can use tools such as vulnerability scanners to initiate vulnerability scans on known network nodes, or identify possible vulnerabilities in known network nodes through fuzz testing. Once the defense personnel discover a vulnerability, they need to continue to evaluate the exploit conditions of the vulnerability in the drill scenario based on the retrieval results of the vulnerability information database. For example, the defense personnel discover that there is a remote code execution vulnerability in the target network node. Based on the retrieval results of the vulnerability information database, it is known that there are no additional permission constraints for the vulnerability itself. However, in the network range scenario, there may be some exploit condition constraints for the vulnerability, such as: access permissions of the target network node, network connectivity constraints, request frequency constraints, etc. The defense personnel need to enumerate all the existing exploit conditions of the vulnerability based on the current drill range environment. After the drill ends, the drill scenario designer can initiate a questionnaire survey to collect the exploit condition recognition results of the defense drill participants who are defense personnel. The corresponding exploit condition recognition rate index ECRR (denoted as R ecr ) is calculated by the formula:

[0106]

[0107] Among them, C is the number of vulnerabilities with correctly recognized exploit conditions, and N is the total number of vulnerabilities.

[0108] For the exploit result recognition rate (ERRR) metric, the exploit effects in the vulnerability information database are often limited to the application or system where the vulnerability lies. In a real network scenario, the impact of a single exploit can often spread and actually affect other network nodes. This exploit result recognition rate metric focuses on examining the defensive personnel's ability to judge the expected results of exploits in a real network environment. Based on the identified exploitable vulnerabilities and the retrieval results from the vulnerability information database, the defensive personnel continue to evaluate the actual possible harm of the vulnerability in the drill scenario. For example, when the defensive personnel discover an XSS vulnerability in the target node and learn from the retrieval results of the vulnerability information database that this vulnerability has the effect of stealing specific user cookies, the actual network harm it can cause is limited and cannot achieve the effect of obtaining more server permissions. However, in the drill scenario design, after stealing the specific user cookies, the attacker can impersonate the user to log in and find the IP address and port of another vulnerable network node in the user's information list. In a situation like this, the defensive personnel need to give the actual possible harm of the vulnerability and the expected results of the exploit. After the drill, the drill scenario designer can initiate a questionnaire survey to collect the results of the defensive drill participants' recognition of the expected harm of exploits. The corresponding exploit result recognition rate metric, ERRR (denoted as R err ) is calculated using the following formula:

[0109]

[0110] where E is the number of correctly identified expected exploit results, and N is the total number of vulnerabilities.

[0111] In specific implementation, to calculate the above two recognition rate metrics, after the drill, a questionnaire survey can be used to collect the number of vulnerabilities with correctly identified exploitable conditions and the number of correctly identified expected exploit results from the defensive drill participants.

[0112] Among them, the questionnaire survey is in the form of multiple-choice. After the drill, the questionnaire is automatically distributed to the defensive drill participants, who need to answer and submit it within a limited time. After the questionnaire survey, a score calculation program processes the submitted results (where multiple selections or fewer selections of the exploitable conditions or expected exploit result options of the vulnerability are treated as incorrect answers), counts the number of vulnerabilities with correctly identified exploitable conditions and the number of vulnerabilities with correctly identified expected exploit results, and calculates the two metric values of ECRR and ERRR according to the above formula and adds them together as the evaluation value of the vulnerability recognition ability of the defensive drill participants based on the drill scenario.

[0113] Step S102: Calculate the vulnerability identification ability score value of the defense exercise participant based on the vulnerability exploitable condition identification result and the expected harm identification result of vulnerability exploitation.

[0114] As mentioned above, add the indicators corresponding to the vulnerability exploitable condition identification result and the expected harm identification result of vulnerability exploitation, and calculate the vulnerability identification ability score value of the defense exercise participant.

[0115] Step S103: Obtain the execution data recorded by the EXP attack script called by the vulnerability exploitation verification program of the network node in the exercise scenario during the exercise process.

[0116] As mentioned above, relevant exercise data was collected in advance during the exercise data collection phase.

[0117] Specifically, during the exercise, execute the vulnerability exploitation verification program of the network node in the exercise scenario, and call the corresponding EXP attack script at a preset time interval; if the execution of the EXP attack script fails, switch to the EXP attack script variant and execute again until all EXP attack script variants fail to execute; record the execution time, the number of successful executions, the number of failed executions, and the total number of executions of the EXP attack script and the EXP attack script variant by the vulnerability exploitation verification program, and record the sequence of the first failure event of a single EXP attack script corresponding to each defense exercise participant, arranged in chronological order, to obtain the vulnerability repair order and relevant parameters of each defense exercise participant as the execution data, and store it in the exercise database.

[0118] Step S104: Calculate the vulnerability repair and reinforcement ability score value of the defense exercise participant based on the execution data.

[0119] Specifically, as an implementation method, first, obtain the vulnerability repair order of the defense exercise participant in the execution data, and the vulnerability repair order of the defense exercise participant can be obtained according to the sequence of the first failure event of the single EXP attack script corresponding to the defense exercise participant in the execution data;

[0120] Then, based on the vulnerability repair order of the defense exercise participant, and the optimal vulnerability repair order and the total score of the repair strategy of the preset exercise scenario, calculate the strategy score of the vulnerability repair sequence of the defense exercise participant.

[0121] Based on the execution data, obtain the total number of vulnerability repairs, the number of vulnerability repairs of a specific type, and the number of EXP attack script variants that failed to execute for a single vulnerability of the defense exercise participant.

[0122] Based on the total number of vulnerability repairs, the number of repairs for specific types of vulnerabilities, and the number of mutant EXP attack scripts with execution failures for individual vulnerabilities of the defense exercise participants, the vulnerability repair and strengthening score of the defense exercise participants is calculated;

[0123] The specific implementation is as follows:

[0124] Based on the total number of vulnerability repairs of the defense exercise participants, the number of mutant EXP attack scripts with execution failures for individual vulnerabilities, and the total number of vulnerabilities in the preset exercise scenario, the overall average weighted vulnerability repair failure rate AFFR of the defense exercise participants is calculated;

[0125] Based on the number of repairs for specific types of vulnerabilities of the defense exercise participants, the number of mutant EXP attack scripts with execution failures for individual vulnerabilities, and the total number of specific types of vulnerabilities in the preset exercise scenario, the average weighted SFFR of the repair failures for specific types of vulnerabilities of the defense exercise participants is calculated;

[0126] Based on the overall average weighted vulnerability repair failure rate AFFR and the average weighted SFFR of the repair failures for specific types of vulnerabilities, the vulnerability repair and strengthening score of the defense exercise participants is calculated.

[0127] Finally, based on the vulnerability repair sequence strategy score and the vulnerability repair and strengthening score, the vulnerability repair and strengthening ability score value of the defense exercise participants is calculated.

[0128] The following elaborates in detail the calculation process of the vulnerability repair and strengthening ability score value of the defense exercise participants:

[0129] In this embodiment, the vulnerability repair and strengthening ability score value is comprehensively calculated from two evaluation indicators. One is the vulnerability repair sequence strategy score FSS, and the other is the vulnerability repair and strengthening score FES.

[0130] Among them, for the vulnerability repair sequence strategy score (FSS, Fix Strategy Score), in a real network security attack and defense scenario, the time left for defenders to repair vulnerabilities is often very limited. Therefore, the ability to quickly locate key vulnerability nodes and preferentially repair is very important. This vulnerability repair sequence strategy score index focuses on examining the vulnerability repair strategy of defenders, and pays more attention to the benefits brought by the overall repair effect. The vulnerability repair and strengthening ability is not equal to the sum of the scores of individual vulnerability repairs, and the vulnerability repair sequence will affect the final evaluation of the vulnerability repair ability.

[0131] The exploit verification program records the sequence of the first execution failure events of a single vulnerability exploit in the form of a timeline. For example: Vulnerability D (00:01:12) --> Vulnerability B (00:41:50) --> Vulnerability A (01:30:00). Therefore, the vulnerability repair order of the defenders can be obtained, that is, first repaired Vulnerability D, then repaired Vulnerability B, and finally repaired Vulnerability A. To measure the difference between the actual repair order and the optimal repair order, a customized Hamming distance algorithm can be used to calculate this scoring value. The process of this Hamming distance algorithm can refer to Figure 3 as shown

[0132] Among them, the scoring formula for the vulnerability repair sequence strategy score FSS (denoted as S fs ) is:

[0133]

[0134] Among them, d is the Hamming distance, l is the length of the optimal repair sequence, and m is the total score of the repair strategy

[0135] For example, assume that there are four vulnerabilities A, B, C, and D in total, and the optimal repair sequence is C -> B -> A -> D, and the total score of the repair strategy is 100. According to the above vulnerability repair order D -> B -> A, the length of the actual repair sequence is less than that of the optimal repair sequence. Use the invalid value X to fill in the remaining bits of the actual repair sequence. Therefore, the Hamming distance between CBAD and DBAX is 2, and the FSS value is 50

[0136] For the vulnerability repair and reinforcement score (FES, Fix Effect Score), calculating this vulnerability repair and reinforcement scoring index aims to get rid of the dependence on the red team's capabilities, that is, limited by the different capabilities of the attackers, it cannot accurately and effectively verify whether the vulnerability has been completely repaired. In this embodiment, after the EXP attack script execution fails in the exploit verification program, the next execution will automatically switch to another EXP script, that is, the EXP attack script variant and execute again until all variants fail to execute. This vulnerability repair and reinforcement scoring index takes into account the objective fact that there are generally strong and weak aspects of the vulnerability repair capabilities of the defenders themselves. Therefore, according to the different types of vulnerabilities, the overall average weighted vulnerability repair failure rate (AFFR, Average Failed Fix Rate) and the average weighted specific type vulnerability repair failure rate (SFFR, Specific Failed Fix Rate) are proposed, which can more accurately evaluate the vulnerability repair and reinforcement capabilities of the defenders

[0137] Among them, the score of the defender for repairing vulnerability k (denoted as F k ) is calculated as:

[0138]

[0139] Among them, s k is the full score of the repair of vulnerability k, and C fk is the number of variants of the failed EXP attack of vulnerability k, and C k is the total number of variants of the EXP of vulnerability k.

[0140] The average weighted failure rate of overall vulnerability repair AFFR (denoted as R af ) is calculated as follows:

[0141]

[0142] Among them, N F is the number of vulnerabilities repaired, and N is the total number of vulnerabilities.

[0143] The average weighted failure of specific type vulnerability repair, that is, the SFFR of specific vulnerability type t (denoted as R sft ) is calculated as follows:

[0144]

[0145] Among them, N Ft is the number of vulnerabilities of type t repaired, and N t is the total number of vulnerabilities of type t.

[0146] The vulnerability repair reinforcement score FES (denoted as S fe ) is calculated as follows:

[0147]

[0148] Among them, N type is the number of vulnerability types.

[0149] During the drill, the vulnerability exploitation verification program runs continuously, collects the sequence of first execution failures of each vulnerability EXP attack script, as well as the execution time, the number of successful executions, the number of failed executions, and the total number of executions of each EXP attack script and different variant scripts, and stores them in the drill database.

[0150] After the drill, through the score calculation program, read each data value in the drill database, calculate the vulnerability repair order, the total number of vulnerabilities repaired, the number of specific type vulnerabilities repaired, and the number of failed variants of the EXP attack script of a single vulnerability respectively, and calculate the two index values of FSS and FES according to the above formula and add them up as the evaluation value of the vulnerability repair and reinforcement capabilities.

[0151] Step S105, calculate the ability evaluation score value of the defense drill participant according to the vulnerability identification ability score value of the defense drill participant and the vulnerability repair and reinforcement ability score value.

[0152] The accuracy of the vulnerability exploitable condition identification result and the vulnerability exploitation harmfulness identification result of the defense personnel in the network range scenario can verify the score of their vulnerability repair and reinforcement ability. Therefore, the final evaluation value S of the defense personnel's vulnerability repair and reinforcement ability is calculated and output by the score calculation program after the drill, and the following calculation formula is used:

[0153] S = (R err + R ecr ) × (S fs + S fe ).

[0154] Among them, R ecr represents the vulnerability exploitable condition identification result; R err represents the expected harm identification result of vulnerability exploitation; S fe represents the vulnerability repair and reinforcement score; S fe represents the score of the vulnerability repair sequence strategy.

[0155] Through the above solution in this embodiment, after the drill, the vulnerability exploitable condition identification result and the expected harm identification result of vulnerability exploitation of the defense drill participants in the defense competition drill scenario of the network range are obtained. Among them, during the drill, the defense drill participants conduct vulnerability repair drills based on the pre-designed drill scenario; according to the vulnerability exploitable condition identification result and the expected harm identification result of vulnerability exploitation, calculate the vulnerability identification ability score value of the defense drill participants; obtain the execution data recorded by the vulnerability exploitation verification program of the network nodes in the drill scenario calling the corresponding EXP attack script during the drill; according to the execution data, calculate the vulnerability repair and reinforcement ability score value of the defense drill participants; according to the vulnerability identification ability score value of the defense drill participants and the vulnerability repair and reinforcement ability score value, calculate the ability evaluation score value of the defense drill participants. The solution of the embodiment of the present invention, due to adopting a preset drill scenario, can design the network topology structure, network nodes covering various vulnerability types and vulnerability exploitation routes as needed in the drill scenario to better evaluate the ability of the defense drill participants, especially to evaluate the ability of the defense personnel from the perspectives of vulnerability identification, vulnerability repair and reinforcement, and examine the vulnerability exploitable condition identification ability, actual harm assessment ability of vulnerability exploitation, vulnerability repair strategy and the effectiveness of vulnerability repair and reinforcement of the defense drill participants as defense personnel in the network range scenario. Therefore, based on the solution of the embodiment of the present invention, it is possible to accurately evaluate the vulnerability repair and reinforcement ability of the defense personnel in the network range, not only avoiding the problem that the evaluation of the defense personnel's ability overly relies on the ability of the attacking personnel, but also solving the problem of single evaluation index, and improving the accuracy and rationality of the security drill evaluation.

[0156] Refer toFigure 4 , Figure 4 is a schematic flowchart of another exemplary embodiment of the method for evaluating the capabilities of defenders in the network range of the present invention. Based on the embodiment shown above Figure 2 , before obtaining the vulnerability exploitable condition identification result and the expected harm identification result of vulnerability exploitation of the defense exercise participants in the defense competition exercise scenario of the network range after the exercise in step S101 above, it further includes:

[0157] Step S1001, during the exercise, execute the vulnerability exploitation verification program of the network nodes in the exercise scenario, and call the corresponding EXP attack script at a preset time interval;

[0158] Step S1002, if the execution of the EXP attack script fails, switch to the EXP attack script variant and execute again until all EXP attack script variants fail to execute;

[0159] Step S1003, record the execution time, the number of successful executions, the number of failed executions, and the total number of executions of the EXP attack script and the EXP attack script variants through the vulnerability exploitation verification program, and record the sequence of the first failure events of a single EXP attack script corresponding to each defense exercise participant, arranged in chronological order, to obtain the vulnerability repair order and relevant parameters of each defense exercise participant as the execution data, and store them in the exercise database.

[0160] Compared with the embodiment shown above Figure 2 , this embodiment further includes a solution for recording the vulnerability repair order and relevant parameters of each defense exercise participant through the vulnerability exploitation verification program of the network nodes.

[0161] Specifically, the solution of this embodiment aims to get rid of the dependence on the capabilities of the red team, that is, limited by the different capabilities of the attacking personnel, it cannot accurately and effectively verify whether the vulnerability has been completely repaired. In this embodiment, after the execution of the EXP attack script fails, the vulnerability exploitation verification program will automatically switch to another EXP script, that is, the EXP attack script variant and execute again until all variants fail to execute.

[0162] Specifically, during the drill process, execute the vulnerability exploitation verification program for the network nodes in the drill scenario, and call the corresponding EXP attack script at a preset time interval; if the execution of the EXP attack script fails, switch to the EXP attack script variant and execute it again until all EXP attack script variants fail to execute; record the execution time, the number of successful executions, the number of failed executions, and the total number of executions of the EXP attack script and the EXP attack script variant through the vulnerability exploitation verification program, and record the sequence of the first failure events of a single EXP attack script corresponding to each defense drill participant, arranged in chronological order, to obtain the vulnerability repair order and relevant parameters of each defense drill participant, as the execution data, and store it in the drill database.

[0163] In the solution of the embodiment of the present invention, by adopting a preset drill scenario, in the drill scenario, the network topology structure, network nodes covering various vulnerability types, and vulnerability exploitation routes can be designed as needed to better evaluate the capabilities of defense drill participants. Especially from the perspectives of vulnerability identification, vulnerability repair and reinforcement, evaluate the capabilities of the defense personnel, and examine the vulnerability exploitable condition recognition ability, actual vulnerability hazard assessment ability, vulnerability repair strategy, and the effectiveness of vulnerability repair and reinforcement of the defense drill participants as defense personnel in the network range scenario. Therefore, based on the solution of the embodiment of the present invention, it is possible to accurately evaluate the vulnerability repair and reinforcement capabilities of defense personnel in the network range, not only avoiding the problem that the evaluation of defense personnel's capabilities overly relies on the capabilities of the attacking party personnel, but also solving the problem of single evaluation indicators, and improving the accuracy and rationality of the security drill evaluation.

[0164] Compared with the existing behavior log monitoring scheme in the live confrontation scenario, this embodiment has the following advantages:

[0165] (1) In the network range environment, it is possible to calculate the repair and reinforcement capabilities of defense personnel more accurately. Fully considering the differences in vulnerability exploitation conditions and actual vulnerability exploitation hazards of bare machine targets in the actual range environment, and the differences in the actual repair priorities of individual vulnerabilities due to the permutation and combination differences of vulnerability exploitation dependencies, this solution calculates two indicators: the exploit condition recognition rate (ECRR) and the exploit result recognition rate (ERRR). The final ability score of the defense personnel will be affected by these two indicators.

[0166] (2) To get rid of the dependence on the Red Team's capabilities, this solution is a method for evaluating the vulnerability repair and reinforcement capabilities of the Blue Team in a network live-fire exercise competition mode that focuses on evaluating the Blue Team's capabilities. Only the Blue Team participates, and there is no attack-defense confrontation. Two indicators are proposed: the Average Failed Fix Rate (AFFR) and the Specific Failed Fix Rate (SFFR). This can solve the problem of the defender's capabilities depending on the attacker's capabilities. At the same time, it subdivides various types of vulnerabilities and calculates the SFFR separately, fully considering the objective fact that defenders generally have strong and weak points in vulnerability repair capabilities, making the evaluation of defenders' capabilities more comprehensive and accurate.

[0167] Compared with the existing solution for determining network security emergency capabilities based on the time required for the proportion of network devices with vulnerabilities in a network area to drop to a preset proportion, the solution of this embodiment has the following advantages:

[0168] (1) The exercise mode is based on a network live-fire exercise environment. The exercise scenario designer can flexibly formulate inspection points for vulnerability repair technologies, combine multiple network nodes containing different types of vulnerabilities, simulate multiple network security risk events, and form a complex heterogeneous defense exercise scenario, which can comprehensively evaluate the defender's vulnerability repair and reinforcement capabilities. It can not only evaluate the defender's vulnerability repair and reinforcement capabilities but also evaluate the defender's vulnerability identification capabilities.

[0169] (2) The vulnerability exploitation verification program runs continuously, collecting the first execution failure event sequence of each vulnerability EXP attack script, as well as the execution time, the number of successful executions, the number of failed executions, and the total number of executions of each EXP attack script and its different variant scripts. The defender's vulnerability repair strategy can be deduced. Replacing multiple EXPs and their variants and continuously conducting vulnerability exploitation verification can test the effectiveness of vulnerability repair and the strength of repair and reinforcement. By comprehensively calculating the vulnerability repair and reinforcement capability score value from two aspects, the defender's vulnerability repair and reinforcement capabilities can be comprehensively and accurately evaluated.

[0170] In addition, the embodiment of the present invention also proposes a device for evaluating the capabilities of defenders in a network live-fire exercise, characterized in that the device for evaluating the capabilities of defenders in a network live-fire exercise includes:

[0171] An identification result acquisition module, configured to, after the exercise ends, acquire the vulnerability exploitable condition identification result and the expected harm identification result of vulnerability exploitation of the defense exercise participants in the defense competition exercise scenario of the network live-fire exercise, wherein during the exercise, the defense exercise participants conduct vulnerability repair exercises based on a pre-designed exercise scenario;

[0172] An identification ability calculation module, configured to calculate the vulnerability identification ability score value of the defense exercise participant according to the vulnerability exploitable condition identification result and the expected harm identification result of vulnerability exploitation;

[0173] An execution record module, configured to obtain the execution data recorded by the vulnerability exploitation verification program of the network node in the exercise scenario calling the corresponding EXP attack script during the exercise;

[0174] A repair and reinforcement ability calculation module, configured to calculate the vulnerability repair and reinforcement ability score value of the defense exercise participant according to the execution data;

[0175] An evaluation score calculation module, configured to calculate the ability evaluation score value of the defense exercise participant according to the vulnerability identification ability score value of the defense exercise participant and the vulnerability repair and reinforcement ability score value.

[0176] Further, the defense personnel ability evaluation device in the network range also includes:

[0177] A design module, configured to design the exercise scenario, where the exercise scenario includes: an exercise network topology, multiple network nodes that may have vulnerabilities or cover multiple vulnerability types, a vulnerability exploitation route, and an EXP attack script for each network node with vulnerabilities.

[0178] Further, the execution record module is further configured to execute the vulnerability exploitation verification program of the network node in the exercise scenario during the exercise, and call the corresponding EXP attack script at a preset time interval;

[0179] If the execution of the EXP attack script fails, switch to the EXP attack script variant and execute again until all EXP attack script variants fail to execute;

[0180] Record the execution time, the number of successful executions, the number of failed executions, and the total number of executions of the EXP attack script and the EXP attack script variant through the vulnerability exploitation verification program, and record the sequence of the first failed execution events of a single EXP attack script corresponding to each defense exercise participant, arranged in chronological order, to obtain the vulnerability repair order and relevant parameters of each defense exercise participant as the execution data, and store them in the exercise database.

[0181] Further, the identification result acquisition module is further configured to collect the number of vulnerabilities correctly identified by the defense exercise participant in the exercise scenario after the exercise, where the vulnerability exploitable condition is obtained by the defense exercise participant evaluating the exploitable condition of the vulnerability in the exercise scenario based on the retrieval result of the vulnerability information database after discovering the vulnerability;

[0182] Based on the number of vulnerabilities correctly identified according to the exploitable conditions of the vulnerabilities of the defense drill participants, and the total number of actual vulnerabilities in the drill scenario, the identification result of the exploitable conditions of the vulnerabilities is calculated.

[0183] Furthermore, the identification result acquisition module is further configured to, after the drill ends, collect the number of vulnerabilities correctly identified in the drill scenario by the defense drill participants for the expected exploitation results of the vulnerabilities, where the expected exploitation results of the vulnerabilities are obtained by the defense drill participants based on the retrieval results of the vulnerability information database to evaluate the actual possible harm of the exploitable vulnerabilities in the drill scenario after identifying the exploitable vulnerabilities;

[0184] Based on the number of vulnerabilities correctly identified for the expected exploitation results of the vulnerabilities of the defense drill participants, and the total number of vulnerabilities preset in the drill scenario, the identification result of the expected harm of the exploitation of the vulnerabilities is calculated.

[0185] Furthermore, the repair and reinforcement ability calculation module is further configured to obtain the vulnerability repair order of the defense drill participants in the execution data;

[0186] Based on the vulnerability repair order of the defense drill participants, and the optimal vulnerability repair order and the total score of the repair strategy preset for the drill scenario, the strategy score for the order of vulnerability repair of the defense drill participants is calculated;

[0187] Based on the execution data, the total number of vulnerabilities repaired by the defense drill participants, the number of vulnerabilities of a specific type repaired, and the number of EXP attack script variants with execution failures for a single vulnerability are obtained;

[0188] Based on the total number of vulnerabilities repaired by the defense drill participants, the number of vulnerabilities of a specific type repaired, and the number of EXP attack script variants with execution failures for a single vulnerability, the repair and reinforcement score of the vulnerabilities of the defense drill participants is calculated;

[0189] Based on the strategy score for the order of vulnerability repair and the repair and reinforcement score of the vulnerabilities, the score value of the vulnerability repair and reinforcement ability of the defense drill participants is calculated.

[0190] Furthermore, the repair and reinforcement ability calculation module is further configured to calculate the overall average weighted AFFR of vulnerability repair failures of the defense drill participants based on the total number of vulnerabilities repaired by the defense drill participants, the number of EXP attack script variants with execution failures for a single vulnerability, and the total number of vulnerabilities preset for the drill scenario;

[0191] Based on the number of fixed vulnerabilities of a specific type of the defense exercise participant, the number of variant EXP attack scripts with execution failures for a single vulnerability, and the total number of vulnerabilities of the specific type in the preset exercise scenario, the average weighted SFFR of fixed vulnerability failures of the specific type of the defense exercise participant is calculated;

[0192] Based on the average weighted AFFR of the overall fixed vulnerability failure rate and the average weighted SFFR of fixed vulnerability failures of the specific type, the fixed vulnerability reinforcement score of the defense exercise participant is calculated.

[0193] For the principle and implementation process of the defense personnel capability assessment in the network range of this embodiment, please refer to the above embodiments and will not be elaborated here.

[0194] The embodiment of the present invention also provides a terminal device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, it implements the method for assessing the capabilities of defense personnel in the network range as described above.

[0195] Since when the defense personnel capability assessment program in this network range is executed by the processor, all the technical solutions of all the foregoing embodiments are adopted, it thus has at least all the beneficial effects brought by all the technical solutions of all the foregoing embodiments, which will not be elaborated one by one here.

[0196] The embodiment of the present invention also provides a computer program product, characterized in that the computer program product includes a computer program, and when the computer program is executed by the processor, it implements the method for assessing the capabilities of defense personnel in the network range as described above.

[0197] Since when the defense personnel capability assessment program in this network range is executed by the processor, all the technical solutions of all the foregoing embodiments are adopted, it thus has at least all the beneficial effects brought by all the technical solutions of all the foregoing embodiments, which will not be elaborated one by one here.

[0198] Compared with the prior art, the method, device, terminal device and computer program product for evaluating the capabilities of defense personnel in a network range according to the embodiments of the present invention, after the drill, obtain the results of identifying exploitable conditions of vulnerabilities and the results of identifying expected hazards of vulnerability exploitation of defense drill participants in the defense competition drill scenario of the network range, wherein during the drill, the defense drill participants perform vulnerability repair drills based on a pre-designed drill scenario; calculate the vulnerability identification ability score value of the defense drill participants according to the results of identifying exploitable conditions of vulnerabilities and the results of identifying expected hazards of vulnerability exploitation; obtain the execution data recorded by the vulnerability exploitation verification program of the network nodes in the drill scenario calling the corresponding EXP attack script during the drill; calculate the vulnerability repair and reinforcement ability score value of the defense drill participants according to the execution data; calculate the ability evaluation score value of the defense drill participants according to the vulnerability identification ability score value and the vulnerability repair and reinforcement ability score value of the defense drill participants. The solution of the embodiments of the present invention, because a preset drill scenario is adopted, can design the network topology structure, network nodes covering various vulnerability types and vulnerability exploitation routes as needed in the drill scenario, so as to better evaluate the capabilities of defense drill participants, especially evaluate the capabilities of defense personnel from the perspectives of vulnerability identification, vulnerability repair and reinforcement, and examine the exploitable condition identification ability, actual hazard assessment ability of vulnerability, vulnerability repair strategy and the effectiveness of vulnerability repair and reinforcement of defense drill participants as defense personnel in the network range scenario. Therefore, based on the solution of the embodiments of the present invention, it is possible to accurately evaluate the vulnerability repair and reinforcement capabilities of defense personnel in the network range, not only avoid the problem that the evaluation of defense personnel's capabilities overly relies on the capabilities of offensive personnel, but also solve the problem of single evaluation indicators, and improve the accuracy and rationality of security drill evaluation.

[0199] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the phrase "including a..." does not exclude the existence of additional identical elements in the process, method, article or system including the element.

[0200] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0201] Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium as described above (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the methods of each embodiment of the present invention.

[0202] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A method for evaluating the capabilities of defenders in a network range, characterized in that, The method includes the following steps: After the drill, obtain the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result of the defense drill participants in the defense competition drill scenario of the network range. During the drill, the defense drill participants conduct vulnerability repair drills based on a pre-designed drill scenario. Calculate the vulnerability identification ability score value of the defense drill participants according to the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result. During the drill, obtain the execution data recorded by the vulnerability exploitation verification program of the network nodes in the drill scenario when calling the corresponding EXP attack script. Calculate the vulnerability repair and strengthening ability score value of the defense drill participants according to the execution data. Calculate the ability evaluation score value of the defense drill participants according to the vulnerability identification ability score value and the vulnerability repair and strengthening ability score value of the defense drill participants. The step of obtaining the vulnerability exploitable condition identification result of the defense drill participants in the defense competition drill scenario of the network range after the drill includes: After the drill, collect the number of vulnerabilities with correctly identified exploitable conditions identified by the defense drill participants in the drill scenario. The exploitable conditions of the vulnerabilities are obtained by the defense drill participants evaluating the exploitable conditions of the vulnerabilities in the drill scenario based on the retrieval results of the vulnerability information database after discovering the vulnerabilities. The exploitable conditions of the vulnerabilities are the constraint conditions that the defense personnel need to overcome when exploiting the vulnerabilities in the network range scenario. Calculate the vulnerability exploitable condition identification result based on the number of vulnerabilities with correctly identified exploitable conditions by the defense drill participants and the total number of actual vulnerabilities in the drill scenario.

2. The method for evaluating the capabilities of defenders in a network range according to claim 1, wherein Before the step of obtaining the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result of the defense drill participants in the defense competition drill scenario of the network range after the drill, it further includes: Design the drill scenario, which includes: a drill network topology, multiple network nodes that may have vulnerabilities or cover multiple vulnerability types, vulnerability exploitation routes, and EXP attack scripts for each network node with vulnerabilities.

3. The method for evaluating the capabilities of defenders in a network range according to claim 1, wherein Before the step of obtaining the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result of the defense drill participants in the defense competition drill scenario of the network range after the drill, it further includes: During the drill, execute the vulnerability exploitation verification program of the network nodes in the drill scenario and call the corresponding EXP attack script at a preset time interval. If the execution of the EXP attack script fails, switch to the EXP attack script variant and execute again until all EXP attack script variants fail to execute. The vulnerability exploitation verification program records the execution time, the number of successful executions, the number of failed executions, and the total number of executions of the EXP attack script and EXP attack script variants, and records the sequence of the first failed execution events of a single EXP attack script corresponding to each defense drill participant, arranged in chronological order, to obtain the vulnerability repair order and relevant parameters of each defense drill participant as the execution data, and stores them in the drill database.

4. The method for evaluating the capabilities of defenders in a network range according to claim 1, wherein The steps of obtaining the vulnerability exploitation expected harm identification result of the defense drill participant in the defense competition drill scenario of the network range after the drill include: After the drill, collect the number of vulnerabilities correctly identified in the vulnerability exploitation expected results identified by the defense drill participant in the drill scenario, where the vulnerability exploitation expected result is obtained by the defense drill participant evaluating the actual possible harm of the exploitable vulnerability in the drill scenario based on the retrieval result of the vulnerability information database after identifying the exploitable vulnerability; Based on the number of vulnerabilities correctly identified in the vulnerability exploitation expected result of the defense drill participant and the total number of vulnerabilities in the preset drill scenario, calculate the vulnerability exploitation expected harm identification result.

5. The method for evaluating the capabilities of defense personnel in a network range according to claim 3, wherein The steps of calculating the vulnerability repair and strengthening ability score value of the defense drill participant according to the execution data include: Obtain the vulnerability repair order of the defense drill participant in the execution data; Based on the vulnerability repair order of the defense drill participant, and the optimal vulnerability repair order and the total score of the repair strategy of the preset drill scenario, calculate the vulnerability repair sequence strategy score of the defense drill participant; Based on the execution data, obtain the total number of vulnerabilities repaired by the defense drill participant, the number of vulnerabilities of a specific type repaired, and the number of EXP attack script variants with failed executions for a single vulnerability; Based on the total number of vulnerabilities repaired by the defense drill participant, the number of vulnerabilities of a specific type repaired, and the number of EXP attack script variants with failed executions for a single vulnerability, calculate the vulnerability repair and strengthening score of the defense drill participant; Based on the vulnerability repair sequence strategy score and the vulnerability repair and strengthening score, calculate the vulnerability repair and strengthening ability score value of the defense drill participant.

6. The method for evaluating the capabilities of defenders in a network range according to claim 5, wherein The steps of calculating the vulnerability repair and strengthening score of the defense drill participant based on the total number of vulnerabilities repaired by the defense drill participant, the number of vulnerabilities of a specific type repaired, and the number of EXP attack script variants with failed executions for a single vulnerability include: Based on the total number of vulnerabilities repaired by the defense drill participant, the number of EXP attack script variants with failed executions for a single vulnerability, and the total number of vulnerabilities in the preset drill scenario, calculate the overall average weighted failure rate of vulnerability repair AFFR of the defense drill participant; Based on the number of fixed vulnerabilities of a specific type of the defense exercise participant, the number of variants of the EXP attack script with execution failure for a single vulnerability, and the total number of vulnerabilities of the specific type in the preset exercise scenario, the average weighted SFFR of the defense exercise participant for the fixed vulnerabilities of the specific type is calculated; Based on the average weighted AFFR of the overall vulnerability fixing failure rate and the average weighted SFFR of the fixed vulnerabilities of the specific type, the vulnerability fixing and strengthening score of the defense exercise participant is calculated.

7. An evaluation device for the capabilities of defense personnel in a network range, characterized in that, The defense personnel ability evaluation device in the network range includes: An identification result acquisition module, configured to, after the exercise ends, acquire the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result of the defense exercise participant in the defense competition exercise scenario in the network range. During the exercise, the defense exercise participant conducts vulnerability fixing exercises based on a pre-designed exercise scenario; the identification result acquisition module is further configured to, after the exercise ends, the step of acquiring the vulnerability exploitable condition identification result of the defense exercise participant in the defense competition exercise scenario in the network range includes: after the exercise ends, collecting the number of vulnerabilities with correct identification of the exploitable conditions identified by the defense exercise participant in the exercise scenario, where the exploitable conditions are obtained by the defense exercise participant evaluating the exploitable conditions of the vulnerability in the exercise scenario based on the retrieval result of the vulnerability information database after discovering the vulnerability; the exploitable conditions are the constraint conditions that the defense personnel need to overcome to exploit the vulnerabilities in the network range scenario; based on the number of vulnerabilities with correct identification of the exploitable conditions of the defense exercise participant and the total number of actual vulnerabilities in the exercise scenario, the vulnerability exploitable condition identification result is calculated; An identification ability calculation module, configured to calculate the vulnerability identification ability score value of the defense exercise participant according to the vulnerability exploitable condition identification result and the vulnerability exploitation expected harm identification result; An execution record module, configured to acquire the execution data recorded when the vulnerability exploitation verification program of the network node in the exercise scenario calls the corresponding EXP attack script during the exercise; A repair and strengthening ability calculation module, configured to calculate the vulnerability repair and strengthening ability score value of the defense exercise participant according to the execution data; An evaluation score calculation module, configured to calculate the ability evaluation score value of the defense exercise participant according to the vulnerability identification ability score value and the vulnerability repair and strengthening ability score value of the defense exercise participant.

8. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, it implements the method for evaluating the ability of defense personnel in the network range according to any one of claims 1-6.

9. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the method for evaluating the ability of defense personnel in the network range according to any one of claims 1-6.

Images