Method, device and electronic equipment for evaluating defense operation effectiveness based on network range exercises

Abstract

The embodiment of the present invention discloses a method, device, electronic device and storage medium for evaluating the effectiveness of defense operations based on network range exercises, which relates to the field of network security technology. The method includes the steps of: obtaining predetermined operation information of a user on a node in the network range; the predetermined operation information includes: operation behavior and / or the result of the operation behavior; determining the score value corresponding to the predetermined operation information according to a preset defense evaluation framework and the predetermined operation information; wherein the preset defense evaluation framework includes at least: predetermined operation information and its corresponding score value; and determining the defense operation effectiveness of the user according to the score value evaluation. The present invention facilitates the effective evaluation of the defense effectiveness of the defense personnel through the above-mentioned method steps. The present invention is applicable to attack and defense training scenarios based on network ranges.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a method, device and electronic equipment for evaluating defense operation effectiveness based on network range exercises. Background Art

[0002] CyberRange is a technology or product based on virtualization technology that simulates and reproduces the operating status and operating environment of network architecture, system equipment, and business processes in real cyberspace, so as to more effectively realize learning, research, testing, competition, exercises and other behaviors related to network security, thereby improving the network security confrontation level of practitioners and institutions.

[0003] Existing cyber range scenario training mission designs generally fall into two categories. One focuses on open, general-purpose tasks, such as focusing on target attack and defense and completing typical tasks. These tasks are easy to set, and trainees are guided by general-purpose tasks and evaluation indicators, ultimately transforming the training into completing some standard technical movements. The other type, based on the characteristics of the scenario, develops a series of detailed tasks and evaluation indicators, guiding trainees to complete more comprehensive and refined operational training. However, these tasks rely heavily on the director (third-party referee)'s manual design of tasks and indicators for specific scenarios. Furthermore, a significant amount of manual work is required in task design, evaluation indicators, and adjudication, making it difficult to form a framework for systematic training missions.

[0004] Although the above two methods have certain effects in network range scenario training, there are still shortcomings in the evaluation of the defense effectiveness of the defender's operators. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, apparatus, and electronic device for evaluating the effectiveness of defense operations based on a network range exercise, which facilitates effective evaluation of the defense effectiveness of the defender's operators.

[0006] In a first aspect, an embodiment of the present invention provides a method for evaluating the effectiveness of defense operations based on a network range exercise, comprising the steps of:

[0007] Obtaining predetermined operation information of a user on a node within the network range; the predetermined operation information includes: an operation behavior and / or a result generated by the operation behavior; determining a score value corresponding to the predetermined operation information based on a preset defense evaluation framework and the predetermined operation information; wherein the preset defense evaluation framework at least includes: predetermined operation information and its corresponding score value; and determining the user's defense operation effectiveness based on the score value evaluation.

[0008] Optionally, the preset defense evaluation framework further includes: defense effectiveness evaluation index items, and one or more predetermined operation information corresponding to the defense effectiveness evaluation index items; determining the scoring value corresponding to the predetermined operation information based on the preset defense evaluation framework and the predetermined operation information includes: classifying the predetermined operation information into corresponding defense effectiveness evaluation index item entries based on the defense effectiveness evaluation index items within the preset defense evaluation framework; and determining the scoring value corresponding to the predetermined operation information in the corresponding defense effectiveness evaluation index item entry.

[0009] Optionally, the preset defense evaluation framework further includes: defense effectiveness evaluation index items, the defense effectiveness evaluation index items include: defense mission indicators and defense capability indicators, the defense mission indicators and defense capability indicators respectively include multiple types of first-level evaluation indicators, each type of first-level evaluation indicators includes multiple first-level evaluation index items, each first-level evaluation indicator has multiple second-level evaluation indicators, the second-level evaluation indicators are indicators corresponding to the predetermined operation information and with quantifiable evaluation points, and each second-level evaluation indicator has a corresponding scoring value; determining the scoring value corresponding to the predetermined operation information according to the preset defense evaluation framework and the predetermined operation information includes: classifying the predetermined operation information into corresponding second-level evaluation index items under the defense mission indicators and defense capability indicators according to the type of the second-level evaluation indicators; and determining the scoring value corresponding to the predetermined operation information according to the scoring values ​​corresponding to the second-level evaluation indicators for the defense mission indicators and defense capability indicators.

[0010] Optionally, the method of determining the user's defense operation effectiveness based on the scoring value evaluation includes: for the defense task indicator, determining the weight value of the predetermined operation information according to the type corresponding to the secondary evaluation indicator; based on the weight value, performing weighted sum calculation on the scoring values ​​corresponding to all predetermined operation information to obtain the scoring value of the first-level evaluation indicator belonging to the corresponding category; performing weighted sum calculation on the scoring value of the first-level evaluation indicator and the preset weight value to obtain the first-category evaluation total score corresponding to the defense task indicator; and, for the defense capability indicator, determining the weight value of the predetermined operation information according to the type corresponding to the secondary evaluation indicator; based on the weight value, performing weighted sum calculation on the scoring values ​​corresponding to all predetermined operation information to obtain the first-category evaluation total score corresponding to the defense task indicator. The scores are weighted and summed to obtain the score value of the first-level evaluation indicator belonging to the corresponding category; a weighted sum is performed based on the score value of the first-level evaluation indicator and the preset weight value to obtain the second-category evaluation total score corresponding to the defense capability indicator; based on the first-category evaluation total score and the second-category evaluation total score, the user's defense operation effectiveness is evaluated and determined; or, based on the sum or weighted sum of the first-category evaluation total score and the second-category evaluation total score, the defense effectiveness evaluation total score is obtained; based on the defense effectiveness evaluation total score, the user's defense operation effectiveness is evaluated and determined; or, based on the first-category evaluation total score, the second-category evaluation total score and the defense effectiveness evaluation total score, the user's defense operation effectiveness is comprehensively evaluated and determined.

[0011] Optionally, before, simultaneously with, or after determining the user's defense operation effectiveness, a defense operation effectiveness graph of the user is generated and displayed based on the obtained score value.

[0012] Optionally, the types of the first-level evaluation indicators include: identification, shaping, protection, detection and response types, and the first-level evaluation indicators of the identification type include: system environment identification, network environment identification, business identification, user identification, configuration identification, exposure / vulnerability identification and / or activity identification; the first-level evaluation indicators of the shaping type include: system environment policy shaping, network management and control policy shaping, configuration reinforcement, encryption environment construction and / or deception environment construction; the first-level evaluation indicators of the protection type include: connection denial, creation denial, write denial, execution denial, loading denial, behavior denial, transmission denial, identity denial and / or content denial; The first-level evaluation indicators of the detection type include: system environment detection, traffic environment detection, application environment detection, data body detection and / or user behavior detection; the first-level evaluation indicators of the response type include: mitigation, evidence consolidation, host environment disposal, network side disposal, environment and data recovery and / or policy adjustment; and / or, obtaining the user's scheduled operation information on the nodes in the network target range, including: displaying the defense effectiveness evaluation indicator items and the scoring criteria corresponding to the defense effectiveness evaluation indicator items, so that the user performs scheduled operations related to the defense operation effectiveness evaluation on the nodes in the network target range; obtaining the scheduled operation information from the nodes.

[0013] Optionally, before obtaining the user's predetermined operation information on the node in the network target range, the method also includes: assigning a role identifier to the user; the role identifier is used to identify the specific responsibilities of the defense personnel participating in the defense effectiveness evaluation in network defense; and according to the role identifier, configuring the corresponding information in the defense evaluation framework for the user.

[0014] Optionally, the information in the defense assessment framework includes: defense task index items, defense capability index items, first-level assessment index types under defense task index items and defense capability index items, first-level assessment index items contained in each type of first-level assessment indicators, second-level assessment indicators under each first-level assessment indicator, scoring values ​​corresponding to the second-level assessment indicators, and weight values ​​of each type of first-level assessment indicators and second-level assessment indicators.

[0015] In a second aspect, an embodiment of the present invention provides a device for evaluating the effectiveness of defense operations based on network range exercises, comprising: an acquisition program module for acquiring predetermined operation information of a user on a node within the network range; the predetermined operation information includes: operation behavior and / or the result of the operation behavior; a determination program module for determining a score value corresponding to the predetermined operation information based on a preset defense evaluation framework and the predetermined operation information; wherein the preset defense evaluation framework includes at least: predetermined operation information and its corresponding score value; an evaluation program module for evaluating and determining the user's defense operation effectiveness based on the score value.

[0016] In a third aspect, an electronic device provided by an embodiment of the present invention includes: a housing, a processor, a memory, a circuit board and a power supply circuit, wherein the circuit board is placed inside the space enclosed by the housing, and the processor and the memory are arranged on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory is used to store executable program code; the processor runs the program corresponding to the executable program code by reading the executable program code stored in the memory, and is used to execute the method for evaluating the effectiveness of defense operations based on network target range exercises described in any embodiment of the first aspect.

[0017] The method, device and electronic device for evaluating the effectiveness of defense operations based on network range exercises provided by the embodiments of the present invention obtain the user's predetermined operation information on the nodes in the network range; the predetermined operation information includes: the operation behavior and / or the result of the operation behavior; according to a preset defense evaluation framework and the predetermined operation information, the score value corresponding to the predetermined operation information is determined; wherein, the preset defense evaluation framework at least includes: the predetermined operation information and its corresponding score value; the user's defense operation effectiveness is determined based on the evaluation of the score value, so as to facilitate the effective evaluation of the defense effectiveness of the defense personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A schematic diagram of an embodiment of a method for evaluating defense operation effectiveness based on a network range exercise according to the present invention;

[0020] Figure 2 This is a flow chart of an embodiment of a method for evaluating defense operation effectiveness based on a network range exercise according to the present invention;

[0021] Figure 3 A schematic diagram of the structure of a device for evaluating the effectiveness of defense operations based on network range exercises according to the present invention;

[0022] Figure 4 The figure is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0024] It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0025] At least in response to the shortcomings of existing network ranges in supporting defense effectiveness evaluation, the present invention proposes a method for evaluating defense effectiveness operations based on a defense framework (defense evaluation framework), and further establishes an evaluation index system for the defender's operational capabilities that combines a set of scenario tasks and defense capability indicators. During the entire training cycle, it can guide the director to improve the training content and subjects, and guide the trainees to perform defense operations around the mission objectives and capability improvement goals, thereby forming a truly valuable defense operation drill and effectiveness evaluation system, and improving the effectiveness of attack and defense drills conducted based on the network range.

[0026] Example 1

[0027] Figure 1 This is a flow chart of an embodiment of the method for evaluating the effectiveness of defense operations based on network range exercises of the present invention. Figure 1 As shown, the method for evaluating the effectiveness of defense operations based on network range exercises provided by an embodiment of the present invention can be applied to attack and defense training scenarios based on network ranges.

[0028] See Figure 1 As shown, the method for evaluating the effectiveness of defense operations based on a network range exercise may include the following steps:

[0029] S110. Obtaining predetermined operation information of a user on a node in the network range; the predetermined operation information includes: an operation behavior and / or a result generated by the operation behavior.

[0030] The Cyber ​​Range is a technology or product that uses virtualization technology to simulate and replicate the operational status and operating environment of network architecture, system equipment, and business processes in real cyberspace. This allows for more effective learning, research, testing, competitions, and exercises related to cybersecurity, thereby improving the cybersecurity capabilities of individuals and organizations. It is used to simulate a highly simulated cyberspace environment. Specifically, the predetermined operational information refers to the defensive operations performed by users within the Cyber ​​Range based on mission objectives and evaluation criteria, as well as the results of completed defensive operations, such as operational files, screenshots, text, and other evidence files from the time the operations were completed.

[0031] S120. Determine a score value corresponding to the predetermined operation information based on a preset defense evaluation framework and the predetermined operation information; wherein the preset defense evaluation framework at least includes: predetermined operation information and its corresponding score value.

[0032] Specifically, the preset defense evaluation framework is mainly used to formulate a scoring standard to evaluate the user's defense operation effectiveness; the preset defense evaluation framework at least contains predetermined operation information and its corresponding scoring value. According to the user's predetermined operation information in the network target range, the preset defense evaluation framework is used to perform corresponding scoring to determine the user's scoring value.

[0033] S130: Determine the user's defense operation effectiveness based on the evaluation score.

[0034] The user's defense operation effectiveness is determined based on the final evaluation score obtained from a series of defense operation operations performed by the user in the network target range.

[0035] In some embodiments, the preset defense evaluation framework further includes: defense effectiveness evaluation index items, and one or more predetermined operation information corresponding to the defense effectiveness evaluation index items; determining the score value corresponding to the predetermined operation information based on the preset defense evaluation framework and the predetermined operation information includes: classifying the predetermined operation information into corresponding defense effectiveness evaluation index item entries based on the defense effectiveness evaluation index items within the preset defense evaluation framework; and determining the score value corresponding to the predetermined operation information in the corresponding defense effectiveness evaluation index item entry.

[0036] In some embodiments, the preset defense evaluation framework further includes: defense effectiveness evaluation index items, the defense effectiveness evaluation index items include: defense mission indicators and defense capability indicators, the defense mission indicators and defense capability indicators respectively include multiple types of first-level evaluation indicators, each type of first-level evaluation indicators includes multiple first-level evaluation index items, each first-level evaluation indicator has multiple second-level evaluation indicators, the second-level evaluation indicators are indicators corresponding to the predetermined operation information and with quantifiable evaluation points, and each second-level evaluation indicator has a corresponding scoring value; determining the scoring value corresponding to the predetermined operation information according to the preset defense evaluation framework and the predetermined operation information includes: classifying the predetermined operation information into corresponding second-level evaluation index items under the defense mission indicators and defense capability indicators according to the type of the second-level evaluation indicators; for the defense mission indicators and defense capability indicators, determining the scoring value corresponding to the predetermined operation information according to the scoring values ​​corresponding to the second-level evaluation indicators respectively.

[0037] Among them, the defense effectiveness evaluation indicators in the preset defense evaluation framework in this embodiment include: defense task indicators and defense capability indicators. The defense task indicators are an assessment of the defender's completion of the established defense tasks during the exercise, focusing on the achievement of the results of a single task goal; while the defense capability indicators evaluate the various defense means and operations carried out by the defender during the exercise, focusing on the effectiveness of various related defense actions in the process of completing the task goals.

[0038] It can be understood that based on the multi-dimensional, multi-level evaluation index system provided by the embodiment of the present invention, the index items can also be divided into more fine-grained levels, and the evaluation index items can be divided into three evaluation index levels, or even more. Due to the same basic technical concept and the requirement of brief description, the finer-grained evaluation index item division levels will not be elaborated.

[0039] Specifically, the defense task indicators and defense capability indicators respectively include multiple types of first-level evaluation indicators, each type of first-level evaluation indicators includes multiple first-level evaluation indicator items, and each first-level evaluation indicator has multiple second-level evaluation indicators; therefore, according to the complexity or difficulty of the defense task indicators and defense capability indicators, they can be further divided into three-level evaluation indicators or four-level evaluation indicators. In this embodiment, only second-level evaluation indicators are divided; according to the defense effectiveness evaluation indicator items within the preset defense evaluation framework, the user's scheduled operation information is classified into the corresponding defense effectiveness evaluation indicator item entries. For the defense task indicators and defense capability indicators, the scoring values ​​corresponding to the user's scheduled operation information are determined according to the scoring values ​​corresponding to their first-level evaluation indicators and second-level evaluation indicators.

[0040] In some embodiments, the types of the first-level evaluation indicators include: identification, shaping, protection, detection and response types.

[0041] Among them, identification is the defender's understanding of the target environment and the defensive cyberspace terrain, and through collection and detection, it forms an understanding of cyberspace terrain information such as assets, services, users, exposure surfaces / vulnerabilities.

[0042] The first-level evaluation indicators of the identification type include: system environment identification, network environment identification, business identification, user identification, configuration identification, exposure / vulnerability identification and / or activity identification.

[0043] Specifically, each first-level evaluation indicator has multiple second-level evaluation indicators, such as the indicator item "system environment identification", which includes hardware identification, firmware identification, operating system identification, middleware identification, application software identification and other second-level evaluation indicators; the indicator item "network environment identification", the network environment refers to a system that physically interconnects multiple multimedia computers distributed in different locations, communicates with each other according to a certain protocol, and realizes the sharing of software, hardware and their network culture, including network facility identification, network resource identification, network platform identification, network communication identification, network tool identification and other second-level evaluation indicators; the indicator item "business identification" can be achieved through in-depth inspection and analysis of business traffic from the data link layer to the application layer according to the protocol The system uses parameters such as type, port number, characteristic string and traffic behavior characteristics to obtain information, and conducts classification statistics and storage. It generally includes multiple secondary evaluation indicators such as service identification, business protocol identification, and data identification; the indicator item "user identification" includes secondary evaluation indicators such as the defender identifying the user who sends the attack request and other behaviors; the indicator item "configuration identification" includes secondary evaluation indicators such as identifying the configuration of the user's computer or the configuration of other devices used by the user; the indicator item "exposure / vulnerability identification" includes secondary evaluation indicators such as network port identification and network vulnerability identification; the indicator item "activity identification" includes secondary evaluation indicators such as identifying the process currently running by the user or the process of a certain node.

[0044] Among them, shaping is the construction, reconstruction and adjustment of the defense target environment. Through the intervention of topology and scenarios, the environment, topological path, exposure surface, configuration and strategy are adjusted and optimized, and deception deployment is combined to gain relative advantages.

[0045] The first-level evaluation indicators of the shaping type include: system environment policy shaping, network control policy shaping, configuration reinforcement, encryption environment construction and / or deception environment construction.

[0046] Specifically, the first-level evaluation indicator system environment strategy shaping includes the defender's construction, reconstruction and adjustment of different operating system environment strategies; network control strategy shaping includes the defender's control, adjustment and optimization of the network attacked by the attacker; configuration reinforcement includes the defender's configuration reinforcement and optimization of the equipment it uses; encryption environment construction includes the defender's encryption of its defense target environment; deception environment construction includes the defender's construction of deception deployment to confuse the attacker's behavior.

[0047] Among them, protection is a behavioral response to threats, preventing threatening behaviors from achieving expected consequences, and denying threats and violations, including security equipment and security policies blocking and alerting attackers' behaviors.

[0048] The first-level evaluation indicators of the protection type include: connection denial, creation denial, write denial, execution denial, load denial, behavior denial, transmission denial, identity denial and / or content denial.

[0049] Specifically, the first-level evaluation indicator connection denial includes the behavior of the defender denying and blocking the connection request sent by the attacker; creation denial includes the behavior of the defender denying and blocking the creation request sent by the attacker; write denial includes the behavior of the defender denying and blocking the write request sent by the attacker; execution denial includes the behavior of the defender blocking the attacker's request to execute the process on the defender's device; load denial and transfer denial include the behavior of the defender blocking the loading behavior and transfer behavior sent by the attacker respectively; behavior denial includes the behavior of the defender blocking the threats and violations sent by the attacker; identity denial includes the behavior of the defender blocking the identified attacker's identity; content denial includes the behavior of the defender blocking the threats and violations sent by the attacker.

[0050] Among them, detection is a general term for methods to discover, locate and characterize network security threats. It is the process of discovering, calibrating and quantifying risk entities and risk activities.

[0051] The first-level evaluation indicators of the detection type include: system environment detection, traffic environment detection, application environment detection, data body detection and / or user behavior detection.

[0052] Specifically, the first-level evaluation indicator system environment detection includes the process of the defender discovering, calibrating and quantifying the system environment; traffic environment detection includes the defender's detection of the current traffic environment; application environment detection includes the defender's detection of the application environment of the current process; data body detection includes the defender's network security threat detection of the discovered data body; user behavior detection includes the defender's detection of the user behavior of sending attack requests.

[0053] Among them, response is the process of handling and managing risks and threat events, taking disposal measures for detected network security incidents and eliminating the impact.

[0054] The first-level evaluation indicators of the response type include: mitigation, evidence consolidation, host environment disposal, network-side disposal, environment and data recovery and / or policy adjustment.

[0055] Specifically, the first-level assessment indicator mitigation includes the defender's mitigation process for handling and managing risks and threat events; evidence consolidation includes the defender's process for consolidating detected network security events; host environment disposal includes the defender's disposal measures for host environments with risks and threats; network-side disposal includes the defender's disposal measures for risky networks; environment and data recovery includes the defender's recovery of environments and data that have been attacked by the attacker; and strategy adjustment includes the defender's adjustment of the defense strategy made to the attacker.

[0056] The secondary indicators in the above embodiment are indicators that can clearly submit corresponding evidence and can be evaluated and judged.

[0057] The obtaining of the user's predetermined operation information on the nodes in the network range includes: displaying the defense effectiveness evaluation index items and the scoring criteria corresponding to the defense effectiveness evaluation index items, so that the user performs predetermined operations related to the defense operation effectiveness evaluation on the nodes in the network range; and obtaining the predetermined operation information from the nodes.

[0058] Specifically, after the defense mission indicators and defense capability indicators are set, the director can adjust the score weights of these two types of indicators. By adjusting the score weights, the exercise mission can be set to focus on mission objective assessment, defense capability assessment, or a balance between the two aspects. Before the exercise begins, both the offensive and defensive sides can view the mission details, and the defensive side can view the two types of evaluation indicators for this exercise to make action plans and preparations.

[0059] In some embodiments, determining the user's defense operation effectiveness based on the evaluation score includes: determining a weight value of the predetermined operation information based on the type of the secondary evaluation indicator for the defense task indicator; performing a weighted sum calculation on the score values ​​corresponding to all predetermined operation information based on the weight value to obtain a score value of the primary evaluation indicator belonging to the corresponding category; performing a weighted sum calculation based on the score value of the primary evaluation indicator and a preset weight value to obtain a first-category evaluation total score corresponding to the defense task indicator;

[0060] Specifically, in this embodiment, for defense task indicators, the user, that is, the defender, collects information based on various operations of the defender when performing the defense task, such as vulnerabilities, weak configurations, malicious files, shadow assets, compromised assets, target files, etc. The defender selects the defense task indicator item corresponding to the evidence and uploads screenshots, files, characters and other evidence. After uploading, the director can view the evidence items and make a judgment and evaluation on each indicator based on the score weights adjusted in advance for the divided secondary evaluation indicators. The obtained score values ​​are weighted and summed to obtain the score values ​​of the first-level evaluation indicators belonging to the corresponding category, and the obtained score values ​​of the first-level evaluation indicators are also weighted and summed according to the preset weight values, and finally the total score of the first category evaluation corresponding to the defense task indicator is obtained.

[0061] Furthermore, for the defense capability indicator, the weight value of the predetermined operation information is determined according to the type corresponding to the secondary evaluation indicator; based on the weight value, the score values ​​corresponding to all predetermined operation information are weighted and summed to obtain the score value of the first-level evaluation indicator belonging to the corresponding category; and the weighted sum is performed based on the score value of the first-level evaluation indicator and the preset weight value to obtain the total score of the second category evaluation corresponding to the defense capability indicator.

[0062] Specifically, in this embodiment, for the defense capability index, the defender collects information during various operations of the defense task, such as vulnerabilities, weak configurations, malicious files, shadow assets, compromised assets, target files, etc. The defender selects the defense capability index item corresponding to the evidence and uploads screenshots, files, characters and other evidence. After uploading, the director can view the evidence item and perform weighted summation on the score values ​​corresponding to the weight values ​​of the secondary evaluation indicators divided according to the defense capability of the defender to obtain the score value of the first-level evaluation indicator of the defense capability indicator belonging to the corresponding category. The score value of the obtained first-level evaluation indicator is also weighted summed according to the preset weight value, and finally the total score of the second category evaluation corresponding to the defense capability indicator is obtained.

[0063] The user's defense operation effectiveness is evaluated and determined based on the total score of the first category evaluation and the total score of the second category evaluation; or, the total defense effectiveness evaluation score is obtained by summing or weighted summing the total score of the first category evaluation and the total score of the second category evaluation; the user's defense operation effectiveness is evaluated and determined based on the total defense effectiveness evaluation score; or, the user's defense operation effectiveness is determined through a comprehensive evaluation based on the total score of the first category evaluation, the total score of the second category evaluation, and the total defense effectiveness evaluation score.

[0064] Specifically, when finally scoring the defense operation effectiveness, the user, i.e., the defender, can be determined based on the total score of the first and second categories of evaluations. The user's defense operation effectiveness can also be determined by summing or weightedly summing the total score of the first and second categories of evaluations to obtain the total defense effectiveness evaluation score. The user's defense operation effectiveness can also be determined through a comprehensive evaluation based on the total score of the first, second, and defense effectiveness evaluations.

[0065] In some embodiments, before obtaining the user's predetermined operation information on the node in the network target range, the method also includes: assigning a role identifier to the user; the role identifier is used to identify the specific responsibilities of the defense personnel participating in the defense effectiveness evaluation in network defense; and according to the role identifier, configuring the corresponding information in the defense evaluation framework for the user.

[0066] For example, before obtaining the user's predetermined operation information on the node in the network target range, the director can assign role identifiers to the defense operation personnel participating in the defense effectiveness evaluation. For example, the roles of trainees in defensive operations are divided into two roles: the security operation and maintenance team and the security defense team. Different roles have different focuses on the effectiveness evaluation. The security operation and maintenance team focuses on the indicators of identification, shaping, and protection, and the security defense team focuses on the indicators of detection and response. When setting the scene, the director can assign the default indicator set of the corresponding role to trainees of different roles, and can adjust the personal defense capability indicators of the personnel based on the role indicators.

[0067] In some embodiments, the information in the defense assessment framework includes: defense task index items, defense capability index items, first-level assessment index types under defense task index items and defense capability index items, first-level assessment index items contained in each type of first-level assessment indicators, second-level assessment indicators under each first-level assessment indicator, score values ​​corresponding to the second-level assessment indicators, and weight values ​​of each type of first-level assessment indicators and second-level assessment indicators.

[0068] Specifically, when setting up the scene, the director can assign default indicator sets for different roles to trainees, and can configure corresponding defense task indicators, defense capability indicators, defense task indicators and first-level evaluation indicator types under defense capability indicators based on the role indicators, first-level evaluation indicator items contained in each type of first-level evaluation indicators, second-level evaluation indicators under each first-level evaluation indicator, scoring values ​​corresponding to second-level evaluation indicators, and weight values ​​of first-level evaluation indicators and second-level evaluation indicators of each type; after the defense task indicators and defense capability indicators are set, the director can adjust the score weights of these two types of indicators. By adjusting the score weights, the director can set whether the exercise task is mainly based on task goal assessment, defense capability assessment, or a balance between the two aspects.

[0069] In some embodiments, before, simultaneously with, or after determining the defense operation effectiveness of the user, a defense operation effectiveness graph of the user is generated and displayed based on the obtained score value.

[0070] Specifically, before, during, or after the defender completes the defense operation, the director can use radar charts, heat maps, rectangular tree diagrams and other visual statistical charts to display the scores of the first and second level indicators of each level and dimension obtained by the defender, which are used to evaluate the overall task completion, ability focus, and ability qualification of personnel, roles or teams.

[0071] Please see Figure 2 To help understand the technical solutions and technical effects provided by the embodiments of the present invention, the main processes of the embodiments of the present invention are described as follows in conjunction with an attack and defense training scenario (also known as an attack and defense drill) conducted based on a network range:

[0072] When setting up scenario tasks, the director selects the required indicator items from the secondary indicators of the defense framework based on the characteristics of the scenario and the focus of the assessment capabilities to form an indicator set. This secondary indicator set constitutes the mission objectives and defense task indicators for the scenario. In different cyberspace terrains, defense scenarios, and task settings, the director has different and focused capabilities for teaching, training, and assessing trainees, which are set through defense task indicators. At the same time, the director divides the defenders into roles, such as the security operations group and the security response group, selects a secondary indicator set from the defense framework, and assigns scores to the secondary indicator items as the defense capability indicator evaluation criteria for each role.

[0073] Before starting a mission, the director can adjust the score weights of the defense mission indicators and defense capability indicators; during the mission preparation phase, the defender can view the mission target indicators and defense capability indicators and plan the action plan.

[0074] During the exercise, the defender collects information through various operations. The defender selects the defense task indicators and defense capability indicators corresponding to the evidence and uploads screenshots, files, characters, and other evidence. After uploading, the director can view the evidence items, make judgments and evaluations on each indicator, and determine the score. For data that can be obtained through the data collection system, such as attacker behavior denial, attack behavior, abnormal behavior, abnormal assets, abnormal files, abnormal users, abnormal domain names, etc. discovered by security equipment, the collected data will be normalized and aggregated into indicators and provided to the director as a reference for judgment. The director can make a judgment based on the collected data and the submitted evidence. For indicators that can be partially automatically judged in this exercise scenario, the director can choose to set an automatic judgment method, and the system will automatically judge and score the collected evidence.

[0075] After the scoring is complete, the scores for the secondary indicators are used to generate the primary indicator scores for the defense mission indicators, the total score for the first category assessment, and the primary indicator scores for the defense capability indicators, the total score for the second category assessment. The overall score is calculated by weighting the total defense capability score and the total defense mission score, according to the weights set by the director. Role and team scores are calculated by summing each member's individual scores. The scores for the primary and secondary indicators at each level and dimension can be displayed using visual statistical charts such as radar charts, heat maps, and rectangular treemaps. These can be used to assess the overall task completion, capability focus, and competency qualification of individuals, roles, or teams, facilitating the effective evaluation of the defensive effectiveness of defenders.

[0076] Therefore, the method for evaluating the effectiveness of defense operations based on network range exercises provided by the embodiment of the present invention establishes a defense effectiveness evaluation system based on a defense evaluation framework, which facilitates the effective evaluation of the defense effectiveness of the defense personnel.

[0077] Furthermore, by constructing a defense assessment framework with multi-dimensional and hierarchical defense effectiveness evaluation indicators, the defense effectiveness of the defender's operators can be evaluated more effectively. The defense assessment framework with systematic defense objectives and defense indicators thus formed can improve the effectiveness of defense exercises and the effectiveness of evaluations.

[0078] Example 2

[0079] Figure 3 A device for evaluating the effectiveness of defensive operations based on cyber range exercises, including:

[0080] The acquisition program module 31 is used to obtain predetermined operation information of the user on the nodes in the network range; the predetermined operation information includes: operation behavior and / or the result of the operation behavior;

[0081] A determination program module 32 is configured to determine a score value corresponding to the predetermined operation information based on a preset defense evaluation framework and the predetermined operation information; wherein the preset defense evaluation framework at least includes: predetermined operation information and its corresponding score value;

[0082] The evaluation program module 33 is used to evaluate and determine the user's defense operation effectiveness based on the scoring value.

[0083] The device of this embodiment can be used to execute the technical solution of the method embodiment shown in Example 1. Its implementation principle and technical effects are similar and will not be repeated here.

[0084] In addition, the device of this embodiment can also be used to execute other embodiments of the method for evaluating the effectiveness of defense operations based on network range exercises corresponding to the aforementioned embodiment 1. For details not described in detail, please refer to each other and will not be repeated here.

[0085] Example 3

[0086] Figure 4 FIG. 1 is a structural diagram of an embodiment of an electronic device of the present invention. Based on the method provided in the first embodiment and the apparatus provided in the second embodiment, the embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, the step flow of any embodiment described in the first embodiment of the present invention can be implemented.

[0087] The above-mentioned electronic device may include: a shell 41, a processor 42, a memory 43, a circuit board 44 and a power supply circuit 45, wherein the circuit board 44 is placed inside the space enclosed by the shell 41, and the processor 42 and the memory 43 are arranged on the circuit board 44; the power supply circuit 45 is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory 43 is used to store executable program code; the processor 42 runs the program corresponding to the executable program code by reading the executable program code stored in the memory 43, so as to execute the method for evaluating the effectiveness of defense operations based on network target range exercises described in any of the above-mentioned embodiments.

[0088] The specific execution process of the above steps by the processor 42 and the steps further executed by the processor 42 by running the executable program code can be found in the description of the first embodiment of the present invention, and will not be repeated here.

[0089] This electronic device exists in many forms, including but not limited to:

[0090] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and are primarily designed to provide voice and data communications. These terminals include smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones.

[0091] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0092] (3) Portable entertainment devices: These devices can display and play multimedia content. These devices include audio and video players (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0093] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0094] (5) Other electronic devices with data interaction functions.

[0095] The present invention also provides an embodiment of a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the method for evaluating the effectiveness of defense operations based on network range exercises as described in any one of the embodiments of the preceding claims, thereby also achieving the corresponding technical effects, which have been described in detail above and will not be repeated here.

[0096] In summary, compared with the existing method for evaluating the skills of range personnel, the method and device for evaluating the effectiveness of defense operations based on network range exercises provided by the embodiments of the present invention establish a set of evaluation index systems for the operational capabilities of the defender that combines scenario defense tasks and defense capability indicators through task setting, a defense effectiveness indicator system, an indicator weight adjustment method, a judgment evaluation method, and data sources based on a defense evaluation framework. During the entire training cycle, the director is guided to improve the training content and subjects, and the defender is guided to perform defense operations around the task objectives and capability improvement. When the defender completes the defense operation, the scores of the first and second level indicators of each level and dimension are calculated and displayed in charts, thereby facilitating an effective evaluation of the defense effectiveness of the defender's operating personnel.

[0097] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0098] Each embodiment in this specification is described in a related manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment. For the convenience of description, the above device is described by dividing it into various units / modules according to their functions. Of course, when implementing the present invention, the functions of each unit / module can be implemented in the same or multiple software and / or hardware.

[0099] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0100] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for evaluating the effectiveness of defense operations based on a network range exercise, characterized in that: The method comprises the following steps: obtaining predetermined operation information of a user on a node in the network range; the predetermined operation information comprises: an operation behavior and / or a result generated by the operation behavior; Determining a score value corresponding to the predetermined operation information based on a preset defense evaluation framework and the predetermined operation information; wherein the preset defense evaluation framework at least includes: predetermined operation information and its corresponding score value; Determining the user's defense operation effectiveness based on the scoring evaluation; The preset defense assessment framework further includes: defense effectiveness assessment index items, the defense effectiveness assessment index items include: defense mission indicators and defense capability indicators, the defense mission indicators and defense capability indicators respectively include multiple types of first-level assessment indicators, each type of first-level assessment indicator includes multiple first-level assessment indicator items, each first-level assessment indicator is divided into multiple second-level assessment indicators, the second-level assessment indicators are indicators corresponding to the predetermined operation information and can be quantified, and each second-level assessment indicator is provided with a corresponding score value; The step of determining a score value corresponding to the predetermined operation information based on a preset defense evaluation framework and the predetermined operation information includes: Classifying the predetermined operation information into corresponding secondary evaluation indicator items under the defense mission indicator and the defense capability indicator according to the type of the secondary evaluation indicator; For the defense task index and the defense capability index, the score value corresponding to the predetermined operation information is determined according to the score value corresponding to the secondary evaluation index.

2. The method for evaluating the effectiveness of defense operations based on network range exercises according to claim 1 is characterized in that: Determining the user's defense operation effectiveness based on the evaluation of the score value includes: For the defense task indicator, determining a weight value of the predetermined operation information according to the type corresponding to the secondary evaluation indicator; Based on the weight value, a weighted sum calculation is performed on the score values ​​corresponding to all predetermined operation information to obtain the score value of the first-level evaluation indicator belonging to the corresponding category; Performing a weighted summation based on the score value of the first-level evaluation indicator and the preset weight value to obtain the first-category evaluation total score corresponding to the defense task indicator; and For the defense capability indicator, determining a weight value of the predetermined operation information according to the type corresponding to the secondary evaluation indicator; Based on the weight value, a weighted sum calculation is performed on the score values ​​corresponding to all predetermined operation information to obtain the score value of the first-level evaluation indicator belonging to the corresponding category; Perform weighted summation based on the score value of the first-level evaluation indicator and the preset weight value to obtain the second-category evaluation total score corresponding to the defense capability indicator; Evaluate and determine the user's defense operation effectiveness based on the first category evaluation total score and the second category evaluation total score; or Obtaining a total defense effectiveness evaluation score by summing or weighted summing the total score of the first category evaluation and the total score of the second category evaluation; Evaluate and determine the user's defense operation effectiveness based on the total defense effectiveness evaluation score; or The defense operation effectiveness of the user is determined through a comprehensive evaluation based on the first-category evaluation total score, the second-category evaluation total score, and the defense effectiveness evaluation total score.

3. The method for evaluating the effectiveness of defense operations based on network range exercises according to claim 1 is characterized in that: Before, during, or after determining the user's defense operation effectiveness, a defense operation effectiveness graph of the user is generated and displayed based on the obtained score value.

4. The method for evaluating the effectiveness of defense operations based on network range exercises according to claim 1 is characterized in that: The types of the first-level evaluation indicators include: identification, shaping, protection, detection and response types. The first-level evaluation indicators of the identification type include: system environment identification, network environment identification, business identification, user identification, configuration identification, exposure / vulnerability identification and / or activity identification; The first-level evaluation indicators of the shaping type include: system environment strategy shaping, network control strategy shaping, configuration reinforcement, encryption environment construction and / or deception environment construction; The first-level evaluation indicators of the protection type include: connection denial, creation denial, write denial, execution denial, load denial, behavior denial, transmission denial, identity denial and / or content denial; The first-level evaluation indicators of the detection type include: system environment detection, traffic environment detection, application environment detection, data body detection and / or user behavior detection; The first-level evaluation indicators of the response type include: mitigation, evidence consolidation, host environment disposal, network-side disposal, environment and data recovery and / or policy adjustment; and / or, The obtaining of predetermined operation information of the user on a node in the network range includes: Displaying the defense effectiveness evaluation index items and the scoring criteria corresponding to the defense effectiveness evaluation index items, so that the user can perform predetermined operations related to the defense operation effectiveness evaluation on the nodes in the network range; Obtain predetermined operation information from the node.

5. The method for evaluating the effectiveness of defense operations based on network range exercises according to claim 1 is characterized in that: Before obtaining predetermined operation information of the user on the node in the network range, the method further includes: assigning a role identifier to the user; the role identifier is used to identify the specific responsibilities of the defense operator participating in the defense effectiveness evaluation in network defense; According to the role identifier, information in a corresponding defense assessment framework is configured for the user.

6. The method for evaluating defense operation effectiveness based on network range exercises according to claim 5 is characterized in that: The information in the defense assessment framework includes: defense task index items, defense capability index items, first-level assessment index types under defense task index items and defense capability index items, first-level assessment index items contained in each type of first-level assessment indicator, second-level assessment indicators under each first-level assessment indicator, score values ​​corresponding to the second-level assessment indicators, and weight values ​​of each type of first-level assessment indicators and second-level assessment indicators.

7. A device for evaluating the effectiveness of defense operations based on network range exercises, characterized in that: include: An acquisition program module is used to obtain predetermined operation information of a user on a node in the network range; The predetermined operation information includes: operation behavior and / or the result of the operation behavior; A determination program module is configured to determine a score value corresponding to the predetermined operation information based on a preset defense evaluation framework and the predetermined operation information; wherein the preset defense evaluation framework at least includes: predetermined operation information and its corresponding score value; An evaluation program module, configured to evaluate and determine the user's defense operation effectiveness based on the scoring value; The preset defense assessment framework further includes: defense effectiveness assessment index items, the defense effectiveness assessment index items include: defense mission indicators and defense capability indicators, the defense mission indicators and defense capability indicators respectively include multiple types of first-level assessment indicators, each type of first-level assessment indicator includes multiple first-level assessment indicator items, each first-level assessment indicator is divided into multiple second-level assessment indicators, the second-level assessment indicators are indicators corresponding to the predetermined operation information and can be quantified, and each second-level assessment indicator is provided with a corresponding score value; The determination program module is specifically configured to classify the predetermined operation information into corresponding secondary evaluation indicator items under the defense mission indicator and the defense capability indicator according to the type of the secondary evaluation indicator; For the defense task index and the defense capability index, the score value corresponding to the predetermined operation information is determined according to the score value corresponding to the secondary evaluation index.

8. An electronic device, characterized in that: The electronic device includes: a housing, a processor, a memory, a circuit board and a power supply circuit, wherein the circuit board is placed inside the space enclosed by the housing, and the processor and the memory are arranged on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory is used to store executable program code; the processor runs the program corresponding to the executable program code by reading the executable program code stored in the memory, and is used to execute the method for evaluating the effectiveness of defense operations based on network range exercises as described in any one of the aforementioned claims 1 to 6.

Images