Defense system evaluation method, device and equipment

Abstract

The invention discloses a defense system evaluation method, device and equipment, and relates to the field of network security attack and defense confrontation. An attack instruction is sent to the attack node, the attack instruction is used for indicating the attack node to carry out data attack on the data node, the data node comprises a defense detection tool, and the defense detection tool is used for carrying out defense detection on the attack received by the data node and generating a defense result report; receiving an attack result report sent by the attack node; acquiring attack technical data, wherein the attack technical data is obtained by converting an attack behavior executed when the attack node performs data attack on the data node into technical data; receiving a defense result report sent by the data node; and generating an evaluation result based on the attack result report, the attack technology data and the defense result report. And the security and the defense effect of the defense system can be accurately evaluated.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of network security attack and defense confrontation, and in particular, to an evaluation method, device, and equipment for a defense system. Background Art

[0002] Enterprises set up network defense systems to deal with external network attacks. In order to improve the defense effectiveness, it is necessary to accurately evaluate the security of the defense system.

[0003] In the related art, the defense effect of the defense system is determined by simulating the process of network attack and defense confrontation. The attacker executes a pre-planned threat scenario and attacks different component nodes in the defense system. Among them, the threat scenario refers to the attack target of the attacker's attack on the defense system, such as: target location, target effect. The component nodes in the defense system act as defenders and perform corresponding defense activities after detecting the attacker's activities. The security of the defense system is evaluated based on the defense effect of the defender.

[0004] However, there are many different attack techniques that may be used to form the same attack behavior. The above process can only determine whether the defense system can defend against the currently received attack behavior, but cannot determine the specific attack techniques that the defense system can defend against. It is also impossible to standardize the security assessment process of the defense system, resulting in low accuracy of the assessment results. Summary of the invention

[0005] The embodiment of the present application provides a method, device, and apparatus for evaluating a defense system, which can accurately evaluate the security and defense effect of the defense system. The technical solution is as follows:

[0006] In one aspect, a method for evaluating a defense system is provided, the method comprising:

[0007] Sending an attack instruction to an attack node, wherein the attack instruction is used to instruct the attack node to perform a data attack on a data node, wherein the data node includes a defense detection tool, and the defense detection tool is used to perform a defense detection on the attack received by the data node and generate a defense result report;

[0008] receiving an attack result report sent by the attack node, wherein the attack result report includes attack behavior data when the attack node performs a data attack on the data node;

[0009] Acquiring attack technical data, wherein the attack technical data is obtained by converting the attack behavior performed by the attack node when performing a data attack on the data node into technical data;

[0010] receiving a defense result report sent by the data node, wherein the defense result report includes a result of a defense detection tool in the data node defending against the attack behavior;

[0011] An evaluation result is generated based on the attack result report, the attack technology data and the defense result report, and the evaluation result is used to evaluate the defense effect of the defense detection tool in data attack defense.

[0012] In another aspect, a defense system evaluation device is provided, the device comprising:

[0013] A sending module, used to send an attack instruction to an attack node, wherein the attack instruction is used to instruct the attack node to perform a data attack on a data node, wherein the data node includes a defense detection tool, and the defense detection tool is used to perform a defense detection on the attack received by the data node and generate a defense result report;

[0014] A receiving module, used to receive an attack result report sent by the attack node, wherein the attack result report includes attack behavior data when the attack node performs a data attack on the data node;

[0015] An acquisition module, used to acquire attack technical data, wherein the attack technical data is obtained by converting the attack behavior performed by the attack node when performing a data attack on the data node into technical data;

[0016] The receiving module is further used to receive a defense result report sent by the data node, wherein the defense result report includes a result of the defense detection tool in the data node defending against the attack behavior;

[0017] A generation module is used to generate an evaluation result based on the attack result report, the attack technology data and the defense result report, and the evaluation result is used to evaluate the defense effect of the defense detection tool in data attack defense.

[0018] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the defense system evaluation method as described in any of the above-mentioned embodiments of the present application.

[0019] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, a code set or an instruction set is stored in the storage medium, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the defense system evaluation method as described in any of the above-mentioned embodiments of the present application.

[0020] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the evaluation method of the defense system described in any of the above embodiments.

[0021] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0022] By simulating the network attack and defense confrontation process, the effectiveness of the defense system is evaluated, and the security loopholes of the defense system can be discovered in time. Sending attack instructions to the attack node, so that the attack node attacks the data node and receives the attack result report sent by the attack node, the attack effect of the attack node can be determined; when the attack node attacks the data, the attack behavior it performs is converted into specific attack technology data, and the defense result report generated by the data node when defending against the attack behavior from the attack node is received. Based on the attack technology data, attack result report and defense result report, the specific situation of the attack technology that the defense system can defend can be determined, and the accuracy of evaluating the defense effect of the defense system can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0025] Figure 2 is a flow chart of a method for evaluating a defense system provided by an exemplary embodiment of the present application;

[0026] Figure 3 is a flow chart of a method for evaluating a defense system provided by another exemplary embodiment of the present application;

[0027] Figure 4is a flow chart of a method for generating evaluation results based on a matrix framework provided by an exemplary embodiment of the present application;

[0028] Figure 5 is a schematic diagram of a matrix framework coverage table provided by an exemplary embodiment of the present application;

[0029] Figure 6 is a schematic diagram of an attack and defense confrontation simulation process provided by an exemplary embodiment of the present application;

[0030] Figure 7 is a structural block diagram of an evaluation device for a defense system provided by an exemplary embodiment of the present application;

[0031] Figure 8 is a structural block diagram of an evaluation device for a defense system provided by another exemplary embodiment of the present application;

[0032] Fig. 9 It is a structural block diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0034] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0035] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0036] It should be noted that the information involved in this application (including but not limited to attack instructions, attack node information, data node information, attack result reports, defense result reports, evaluation results, description information of various attack technologies contained in the matrix framework, etc.) and data (including but not limited to attack technology data, first data collected by sensors, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0037] It should be understood that, although the terms first, second, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first parameter may also be referred to as the second parameter, and similarly, the second parameter may also be referred to as the first parameter. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0038] First, a brief introduction is given to the terms involved in the embodiments of this application:

[0039] ATT&CK (Adversarial Tactics, Techniques, and Common Knowledge): is an attack model framework that describes the techniques used in each stage of an attack from the attacker's perspective; a public adversarial tactics and technical knowledge base based on attack vectors observed in the real world. The goal of ATT&CK is to create an exhaustive list of known adversarial tactics and techniques used in cyber attacks. In the ATT&CK technology matrix, there are specific scenario examples for each technology, illustrating how an attacker can exploit the technology through a certain malware or action plan.

[0040] The process of simulating network attack and defense confrontation can determine the defense effect and security of the enterprise (or related institution) defense system. In order to improve the defense effect of the defense system and promptly repair the security loopholes of the defense system, the accuracy of the evaluation results is required to be high.

[0041] When simulating attack and defense confrontation, the attacker executes a pre-planned threat scenario to attack the data nodes in the defense system. The threat scenario refers to the attack target of the attacker on the defense system, such as the target location and target effect. The data nodes in the defense system act as defenders and perform corresponding defense activities after detecting the attacker's activities. The security of the defense system is evaluated based on the defender's defense effect.

[0042] However, there are many different attack techniques that may be used to form the same attack behavior. The above process can only determine whether the defense system can defend against the currently received attack behavior, but cannot determine the specific attack techniques that the defense system can defend against. It is also impossible to standardize the security assessment process of the defense system, resulting in low accuracy of the assessment results.

[0043] This application proposes a defense system evaluation method. In the process of simulating network attack and defense confrontation, the ATT&CK technical framework is used to detect adversarial behavior by studying adversarial behavior, building sensors to obtain data, and analyzing data. The attack and defense confrontation and analysis process includes three important roles: white team, red team, and blue team.

[0044] The White Team is a neutral referee node that develops threat scenarios for testing defenses. The White Team works with the Red Team and the Blue Team to resolve issues that arise during the attack and defense confrontation and ensure that the test objectives are met. The White Team interfaces with network administrators to ensure that network assets are maintained.

[0045] The Red Team refers to the attack node, acting as the attacker in the attack and defense confrontation. The Red Team executes the planned threat scenarios, simulates the adversarial behavior, and connects with the White Team as needed, reporting any system or network vulnerability issues that arise in the attack and defense confrontation to the White Team.

[0046] The blue team refers to the data node, which plays the role of the defender in the attack and defense confrontation, and detects the activities (attack behaviors) of the red team through data analysis. Among them, the blue team is the node in the defense system.

[0047] During the confrontation between the red team and the blue team, each team generates a report describing the confrontation process and the confrontation effect in real time and sends it to the white team. When the red team attacks, the attack behavior of the red team is analyzed based on the ATT&CK technical framework to determine the types of attack techniques used by the red team.

[0048] Based on the types of attack techniques and report results, the evaluation results used to evaluate the defense system are generated, which can determine the types of attack techniques that the defense system can defend against and accurately evaluate the defense effect and security of the defense system.

[0049] Next, the implementation environment involved in the embodiments of the present application is described. For illustration, please refer to Figure 1The implementation environment involves a first client 110, a second client 120, a third client 130, and a server 140. The first client 110, the second client 120, the third client 130, and the server 140 are connected via a communication network 150. A defense system is deployed on multiple second clients 120, that is, each second client 120 is a component node in the defense system. Sensors are deployed on the server 140, and the sensors are used to collect various data (such as data during the operation of the server itself, and terminal data corresponding to the terminal connected to the server). The first client 110, the second client 120, the third client 130, and the server 140 all correspond to the network attack and defense simulation system.

[0050] The network attack and defense simulation process is implemented based on the network attack and defense simulation system, with the first client 110 acting as the attacker and the second client 120 acting as the defender. The third client 130 is a neutral node, and the server 140 collects various first data for planning the simulation process through sensors deployed by itself, and sends the data to the third client 130. The third client 130 determines the attack target of the first client 110 and the defense range of the second client 120 based on the first data.

[0051] Exemplarily, the third client 130 sends an attack instruction to the first client 110, the attack instruction includes an attack target, and after receiving the instruction, the first client 110 selects an attack method according to the attack target and launches an attack on the second client 120. The third client 130 sends a defense instruction to the second client 120, the defense instruction includes a detection range of the second client 120, and after receiving the instruction, the second client 120 detects the activities of the first client 110 in real time according to the detection range and performs defense in time.

[0052] During the confrontation process, the server 140 analyzes the attack behavior of the first client 110 based on the ATT&CK technical framework, converts it into corresponding attack technical data, and sends the attack technical data to the third client 130. The first client 110 generates an attack result report, and the second client 120 generates a defense result report, and sends the reports to the third client 130 respectively.

[0053] The third client 130 evaluates the defense effect and security of the defense system based on the above-mentioned attack technology data, attack result report and defense result report, and generates an evaluation result.

[0054] In some embodiments, the third client 130 may also analyze the attack behavior of the first client 110 based on the ATT&CK technical framework and convert it into corresponding attack technical data, which is not limited in this embodiment.

[0055] The terminals corresponding to the first client 110, the second client 120, and the third client 130 may be various forms of terminal devices such as mobile phones, tablet computers, desktop computers, portable laptops, smart TVs, vehicle-mounted terminals, smart home devices, etc., and the embodiments of the present application are not limited to this.

[0056] It is worth noting that the above-mentioned servers can be independent physical servers, or they can be server clusters or distributed systems composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0057] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to realize data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0058] In some embodiments, the above server can also be implemented as a node in a blockchain system.

[0059] In combination with the above-mentioned noun introduction and application scenarios, the evaluation method of the defense system provided by the present application is described. The method can be executed by a server or a terminal, or can be executed by a server and a terminal together. In the embodiment of the present application, the method is executed by a terminal as an example. Figure 2 As shown, Figure 2 FIG. 1 is a flow chart of a method for evaluating a defense system provided by an exemplary embodiment of the present application. The method includes the following steps.

[0060] Step 210: Send an attack instruction to the attacking node.

[0061] The attack instruction is used to instruct the attack node to launch a data attack on the data node. The data node includes a defense detection tool. The defense detection tool is used to perform defense detection on the attack received by the data node and generate a defense result report.

[0062] Schematically, the attack instruction includes an attack target when instructing the attack node to attack the data node. The attack target is used to indicate the expected attack effect achieved by the attack node when performing the data attack.

[0063] Optionally, the attack targets include but are not limited to the following types: (1) the attack node obtains the target data in the data node by attacking the data node; (2) the attack node bypasses the first data node and obtains the target data in the second data node, wherein there is a correlation between the first data node and the second data node; (3) the attack node destroys the defense system corresponding to the data node by attacking the data node; (4) the attack node clears or deletes the target data in the data node by attacking the data node; (5) the attack node tamperes with the target data in the data node by attacking the data node, etc.

[0064] There is at least one data node, and each data node is a component of the defense system. When the attack node receives an attack instruction, it will select the target data node and attack behavior to be attacked according to the attack target indicated in the attack instruction.

[0065] When fighting against attacking nodes, data nodes will continuously detect the data within their own detection range, detect the attack behavior of attacking nodes by analyzing the changes in data, and take corresponding defenses based on the detected attack behavior.

[0066] When the attack node attacks the data node, it will record the attack behavior and attack effect in real time, generate a corresponding attack result report, and send the attack result report to the terminal.

[0067] While the data node is defending against the attacking behavior of the attacking node, the data node will also record the detection, defense behavior and defense effect in real time, generate a corresponding defense result report, and send the defense result report to the terminal.

[0068] Step 220: Receive the attack result report sent by the attacking node.

[0069] The attack result report includes the attack behavior data when the attack node attacks the data node.

[0070] Optionally, the attack behavior data includes (1) the type of attack behavior used by the attack node to attack the data node; (2) each step in the process of executing the attack behavior; (3) the start and end time of executing the attack behavior; (4) the effect of the attack after execution; (5) the changes in relevant data in the defense system where the data node is located before and after the attack node executes the attack behavior, such as: the change in the number of account logins / logouts in the defense system, etc.

[0071] Schematically, the attack goal is to obtain the target data in the data node. The attack node can achieve the attack goal by executing the following attack behaviors: (1) the first attack behavior; (2) the second attack behavior; (3) the third attack behavior.

[0072] Select any one of the three attack behaviors listed above to attack the data node. Each attack behavior includes at least one attack step, such as: 1. Send a preset software installation package or application installation package to the data node; 2. After the software or application is mistakenly installed on the data node, obtain the target data in the data node through the software or application.

[0073] The attack effect achieved by executing the attack behavior is evaluated according to the attack target. For example, when the attack target is to obtain the target data in the data node, the attack effect can be determined by recording the following data in the report: (1) whether the complete target data is successfully obtained; (2) if the complete target data is not obtained, the proportion of the obtained data in the target data; (3) the time spent on obtaining the target data, etc.

[0074] It is worth noting that the data type and quantity contained in the above-mentioned attack behavior data are arbitrary, the type of attack target, the type and quantity of attack behaviors used by the attack node when attacking the data node can be arbitrary, the type, quantity and sequence of steps included in each attack behavior can be arbitrary, and the content included in the attack result report can be arbitrary, and this embodiment does not limit this.

[0075] Step 230, obtaining attack technology data.

[0076] The attack technical data is obtained by converting the attack behavior performed by the attack node when launching a data attack on the data node into technical data.

[0077] The above conversion process can be performed by the terminal or by the server.

[0078] Optionally, when the above conversion process is executed by a terminal, the terminal obtains the attack behavior data in the attack result report, performs technical conversion on the attack behavior data, and obtains attack technical data.

[0079] Since each attack behavior causes different data changes, when performing technical transformation, the terminal converts the known attack behavior into ATT&CK based on the 12 tactical objectives and the classification and definition of 244 attack techniques in the ATT&CK technical framework, and maps it to specific tactical objectives and attack techniques.

[0080] ATT&CK is logically distributed in multiple matrices, starting with "initial access tactics", through "execution", "persistence", "privilege escalation", "defense bypass", "credential access", "discovery", "lateral movement", "collection", "command and control", "data leakage", "impact" and other 12 tactics. Each tactic corresponds to multiple attack techniques, totaling 244 attack techniques.

[0081] Exemplarily, the type of attack behavior performed by the attack node on the data node is the first attack behavior, and the attack techniques that can be used include the first attack technique, the second attack technique, and the third attack technique. The terminal analyzes the attack behavior performed by the data node based on the ATT&CK technical framework and converts it into the corresponding attack technique, and finds that the attack technique used when the attack is performed is the first attack technique. The attack technique data includes but is not limited to the following: (1) Type of attack technique: the first attack technique; (2) Definition of the attack technique; (3) Knowledge points for defending against the attack technique, such as: the first attack technique can be defended against by the first defense technique or the second defense technique, etc.

[0082] Optionally, when the above conversion process is executed by a server, the terminal receives the attack technology data sent by the server, and the server is used to receive the attack result report sent by the attack node, and convert the attack behavior data in the attack result report into attack technology data.

[0083] First, the terminal sends the attack result report to the server, or the terminal sends the attack behavior data in the attack result report to the server; the server obtains the attack behavior data in the attack result report, performs technical transformation on the attack behavior data, and obtains attack technical data.

[0084] Among them, the server also analyzes the attack behaviors executed by the data nodes based on the ATT&CK technical framework and converts them into corresponding attack techniques, obtains attack technology data, and then sends the attack technology data to the terminal.

[0085] Step 240: Receive a defense result report sent by the data node.

[0086] The defense result report includes the results of the defense detection tool in the data node against the attack behavior.

[0087] Optionally, the defense result includes (1) the type of defense behavior performed by the data node when using the defense detection tool to defend the attacking node; (2) the detection range loaded by the data node when detecting the activities of the attacking node, or the authority range of the data node. The data node can obtain all data within the authority range and perform real-time detection on all nodes within the authority range; (3) the time when the data node discovers that the attacking node is performing an attacking behavior; (4) the duration of executing the defense behavior; (5) the defense effect after execution; (6) the changes in relevant data within the authority range of the data node before and after the data node executes the defense behavior, such as: whether the target data stored in the data node has been tampered with, deleted or lost.

[0088] In schematic form, the defense goal of a data node is to ensure that the target data in the data node will not be acquired, deleted, tampered with, or lost by the attacking node. The data node can implement the following defense behaviors through the defense detection tool: (1) the first defense behavior; (2) the second defense behavior; (3) the third defense behavior.

[0089] Select one of the three defense behaviors listed above to defend the attacking node. Each defense behavior includes at least one defense step, such as: 1. Detecting foreign software installation packages or application installation packages; 2. After detecting the existence of foreign software installation packages or application installation packages, automatically clearing and disinfecting them through defense detection tools.

[0090] The defensive effect achieved by executing the defensive behavior is evaluated according to the defensive goal. For example, when the defensive goal is to ensure that the target data in the data node will not be acquired, deleted, tampered with, or lost by the attacking node, the defensive effect can be determined by recording the following data in the report:

[0091] (1) Whether the attack behavior of the attacking node is successfully detected; (2) If the attack behavior is detected, after the defense behavior is executed, whether the target data is obtained, deleted, tampered with, or lost by the attacking node;

[0092] (3) The time taken to obtain the target data, etc.

[0093] It is worth noting that the types of the above-mentioned defense targets, the types and quantities of defense detection tools used by data nodes when defending against the attack behaviors of attack nodes can be arbitrary, the types and quantities of defense behaviors performed can be arbitrary, the types, quantities, and sequence of steps included in each defense behavior can be arbitrary, and the content included in the defense result report can be arbitrary, and this embodiment does not limit this.

[0094] Step 250, generating an evaluation result based on the attack result report, the attack technical data and the defense result report.

[0095] Among them, the evaluation results are used to evaluate the defensive effectiveness of defense detection tools in data attack defense.

[0096] Exemplarily, the evaluation results include the types of attack technologies that the defense system corresponding to the data node can completely defend against, the defense levels for different attack technologies (e.g., only part of the target data can be protected, and another part of the target data will still be lost), etc.

[0097] In summary, the method provided by the present application can evaluate the effectiveness of the defense system by simulating the network attack and defense confrontation process, and can timely discover the security loopholes of the defense system. Sending an attack instruction to the attack node, so that the attack node attacks the data node and receives the attack result report sent by the attack node, can determine the attack effect of the attack node; when the attack node attacks the data, the attack behavior it performs is converted into specific attack technology data, and the defense result report generated by the data node when defending against the attack behavior from the attack node is received. Based on the attack technology data, the attack result report and the defense result report, the specific situation of the attack technology that the defense system can defend can be determined, and the accuracy can be improved when evaluating the defense effect of the defense system.

[0098] The method provided in this embodiment obtains the attack behavior data in the attack result report, performs technical transformation on the attack technical behavior based on the ATT&CK technical framework, and obtains the attack technical data. This can further determine the specific attack technology that the defense system (data node) can defend against, provide more detailed data for evaluating the defense system, and improve the accuracy of the evaluation results.

[0099] Figure 3 FIG. 1 is a flowchart of a method for evaluating a defense system provided by another exemplary embodiment of the present application, which is jointly executed by an attack node, a data node, a terminal generating an evaluation result, and a server. Figure 3 As shown, the method includes the following steps.

[0100] Step 310: The terminal determines the defense target of the data node.

[0101] The defense target includes the following: (1) the target attack behavior that the data node needs to detect; (2) the scope and authority that the data node needs to detect when detecting the target attack behavior, wherein the data within the authority scope can be obtained and called by the data node; (3) when multiple attack behaviors are detected, the priority of the data node to defend against them, such as: the priority of the first attack behavior is higher than the second attack behavior. When the first attack behavior and the second attack behavior are detected at the same time, the data node first defends against the first attack behavior and then defends against the second attack behavior. In some embodiments, the data node can also defend against the first attack behavior and the second attack behavior at the same time, which is not limited in this embodiment.

[0102] Step 320: The terminal sends a first data acquisition request to the server.

[0103] A first data acquisition request is sent to the server, where the first data acquisition request is used to request the acquisition of first data, where the first data is the data used in program formulation during the attack and defense simulation process, and the server is used to collect the first data through a sensor, where the sensor is deployed inside the server.

[0104] The first data is the data required by the terminal to formulate a countermeasure simulation plan and analyze the plan.

[0105] The server is a server corresponding to the defense system, each data node is a component of the defense system, and the server collects its own data and data involved in the working process of the data node in the defense system through sensors to obtain the first data. It is worth noting that the type and amount of the first data required in each confrontation simulation process can be adjusted accordingly according to different simulation schemes.

[0106] The server sends the first data to the terminal based on the first data acquisition request, and the terminal receives the first data fed back by the server.

[0107] Step 330: The terminal formulates a simulation plan based on the first data.

[0108] Among them, the simulation scheme includes an attack simulation scheme and a defense simulation scheme, which are respectively used to indicate the actions and behaviors that the attack node and the data node need to perform in the adversarial simulation.

[0109] An attack simulation plan is formulated based on the first data, and an attack instruction is generated based on the attack simulation plan, wherein the attack instruction is used to instruct the attack node to perform a data attack on the data node.

[0110] A defense simulation plan is formulated based on the first data, and a detection instruction is generated based on the defense simulation plan, wherein the detection instruction is used to instruct the data node to detect and defend against the attack behavior executed by the attack node.

[0111] That is, based on the first data, a defense target of the defense detection tool is determined, the defense target is used to represent the expected defense effect achieved by the defense detection tool, and the defense target includes the detection range of the defense detection tool in the data node. The terminal generates a detection instruction based on the defense target and sends the detection instruction to the data node.

[0112] Step 340: The terminal determines the attack target of the attack node and sends the attack target to the attack node.

[0113] The terminal determines an attack target of the attack node based on the first data, where the attack target is used to represent an expected attack effect achieved by the attack node, and generates an attack instruction based on the attack target.

[0114] The terminal sends the attack target to the attack node.

[0115] Step 350: the attack node performs a data attack on the data node based on the attack target, and sends an attack result report to the terminal.

[0116] The attack node decides the attack behavior and attack technology to be used when attacking the data node based on the attack target. When attacking the data node, it generates an attack result report in real time.

[0117] Step 360: The data node defends against the attack behavior of the attack node based on the defense target, and sends a defense result report to the terminal.

[0118] The data node detects the attack behavior of the attack node based on the data within the defense range of the defense target, and takes corresponding defense actions in time when the attack behavior is detected. During the detection process of the data node and the execution of the defense behavior, a defense result report is generated in real time.

[0119] In step 370, the terminal sends the attack behavior data in the attack result report to the server, and the server converts the attack behavior data into attack technical data.

[0120] Among them, the server analyzes the attack behavior data based on the ATT&CK technical framework and converts it into attack technology data.

[0121] In some embodiments, the terminal may also convert the attack behavior data into attack technology data without sending the data to the server. The server may perform the conversion process, which is not limited in this embodiment.

[0122] Step 380: The terminal receives the attack technology data sent by the server, and generates an evaluation result based on the attack technology data, the attack result report, and the defense result report.

[0123] Among them, the evaluation results are used to evaluate the defensive effectiveness of defense detection tools in data attack defense.

[0124] Exemplarily, the evaluation results include the types of attack technologies that the defense system corresponding to the data node can completely defend against, the defense levels for different attack technologies (e.g., only part of the target data can be protected, and another part of the target data will still be lost), etc.

[0125] In summary, the method provided by the present application can evaluate the effectiveness of the defense system by simulating the network attack and defense confrontation process, and can timely discover the security loopholes of the defense system. Sending an attack instruction to the attack node, so that the attack node attacks the data node and receives the attack result report sent by the attack node, can determine the attack effect of the attack node; when the attack node attacks the data, the attack behavior it performs is converted into specific attack technology data, and the defense result report generated by the data node when defending against the attack behavior from the attack node is received. Based on the attack technology data, the attack result report and the defense result report, the specific situation of the attack technology that the defense system can defend can be determined, and the accuracy can be improved when evaluating the defense effect of the defense system.

[0126] The method provided in this embodiment sends a first data acquisition request to the server through the terminal, requesting the server to collect the first data required for formulating a countermeasure simulation plan based on the internally deployed sensors, and can further generate an attack simulation plan based on the first data, thereby improving the efficiency of plan formulation.

[0127] The method provided in this embodiment determines the attack target of the attack node based on the first data, generates an attack instruction based on the attack target, and can send the expected attack effect that the terminal expects the attack node to achieve to the attack node, so that the attack node can attack the data node according to the attack target, thereby realizing the attack link in the confrontation simulation process.

[0128] The method provided in this embodiment sends a detection instruction to the data node, which can instruct the data node to detect and defend against the attack behavior executed by the attack node, thereby realizing the defense link in the confrontation simulation process.

[0129] The method provided in this embodiment formulates a defense simulation plan based on the first data, and generates a detection instruction based on the defense simulation plan, which can make the formulated simulation plan more in line with the actual situation and improve the simulation effect.

[0130] The method provided in this embodiment sets the detection range of the defense detection tool in the data node in the defense target, which enables the data node to determine the ability to detect attack behaviors during the defense process, thereby facilitating further evaluation of the security of the defense system.

[0131] Figure 4 It is a flow chart of a method for generating evaluation results based on a matrix framework, which is executed by a terminal and includes the following steps.

[0132] Step 410, obtaining a preset matrix framework.

[0133] The matrix framework refers to the ATT&CK technical framework, which is expressed in the form of a matrix and contains descriptive information of multiple attack technologies.

[0134] Optionally, the matrix framework includes 12 tactical objectives and description information of a total of 244 attack techniques corresponding to the 12 tactical objectives.

[0135] ATT&CK is logically distributed in multiple matrices, starting with "initial access tactics", through "execution", "persistence", "privilege escalation", "defense bypass", "credential access", "discovery", "lateral movement", "collection", "command and control", "data leakage", "impact" and other 12 tactics. Each tactic corresponds to multiple attack techniques, totaling 244 attack techniques.

[0136] Step 420, generating a matrix framework coverage table based on the attack technology data and the matrix framework.

[0137] Among them, the matrix framework coverage table is used to count the attack technologies that the defense system successfully defends against.

[0138] The attack technology data includes the types of attack technologies used by the attack node when attacking the data node and the definition of the attack technology. Based on the attack technology data, the corresponding target attack technology is found from the matrix framework, and the target attack technology is marked based on its position in the matrix framework to obtain the matrix framework coverage table.

[0139] Step 430, determining a first technology category of the attack technology successfully defended by the defense system based on the attack result report and the defense result report, and recording the attack technology data corresponding to the first technology category in the matrix framework coverage table.

[0140] When generating the matrix framework coverage table, it is also necessary to further determine the first technology category corresponding to the attack technology that the data node (i.e., the defense system) can successfully defend against in the target attack technology based on the attack result report and the defense result report. And record the attack technology data corresponding to the first data category in the matrix framework coverage table.

[0141] Indicatively, Figure 5 As shown, Figure 5 It is a schematic diagram of a matrix framework coverage table. The matrix framework coverage table 500 includes tactical objectives 510 and attack techniques 520. Among them, the tactical objectives 510 include 12 types in total.

[0142] In the matrix framework coverage table 500, the first technology category of the attack technology that the defense system can defend against is as follows.

[0143] 1. The first technology category corresponding to "initial access" in tactical purpose 510 is: drive-by attack, external remote service, hardware addition, spear phishing attack attachment, supply chain attack;

[0144] 2. The first technology category corresponding to "execution" in the tactical objective 510 is: command line interface, dynamic data exchange;

[0145] 3. The first technology category corresponding to "persistence" in tactical objective 510 is: auxiliary functions and account manipulation;

[0146] 4. The first technology category corresponding to "privilege escalation" in tactical objective 510 is: application compatibility conversion and privilege escalation;

[0147] 5. The first technology category corresponding to "defense against evasion" in tactical objective 510 is: binary padding;

[0148] 6. The first technology category corresponding to "credential access" in tactical objective 510 is: account manipulation and brute force cracking;

[0149] 7. The first technology category corresponding to "discovery" in tactical purpose 510 is: account discovery, application window discovery, cloud service discovery, network service scanning, network sharing discovery, network sniffing, and password policy discovery;

[0150] 8. The first technology category corresponding to "lateral movement" in tactical objective 510 is: application access tokens, application deployment software, login scripts, hash pass, ticket pass, and remote file copy;

[0151] 9. The first technology category corresponding to "collection" in tactical purpose 510 is: audio capture, automated collection, clipboard data, cloud storage object data, information library data, local system data;

[0152] 10. The first technology category corresponding to "command and control" in tactical purpose 510 is: universal port, connection proxy, multi-hop proxy, multi-segment channel, multi-layer encryption, port probing;

[0153] 11. The first technology category corresponding to "data exfiltration" in tactical purpose 510 is: data compression, data encryption, and timed transmission;

[0154] 12. The first technical category corresponding to “adverse impact” in tactical objective 510 is: data destruction, firmware damage, resource hijacking, and service suspension.

[0155] That is, the above-mentioned multiple attack techniques are attack techniques that can be defended by the defense system.

[0156] In some embodiments, in addition to the above-mentioned first technology category, the matrix framework coverage table may also include the degree to which the defense system defends other technology categories other than the first technology category. For example, although the defense system cannot completely defend against the first attack technology, it can defend against the first attack technology to an extent of 80%; the matrix framework coverage table may also include the direction of improving the defense capability of the defense system when defending based on the above-mentioned first technology category and other technology categories, which is not limited in this embodiment.

[0157] Indicatively, Figure 6 As shown, Figure 6 This is a schematic diagram of the attack and defense simulation process.

[0158] In the attack and defense simulation process diagram 600, the attack node 610 attacks the data node 620 based on the automated attack tool. The known attack behaviors of the attack node 610 include but are not limited to: water-resistant attacks, remote file copying, bypassing user account control, collecting local system data, etc. During the attack process of the attack node 610, the server converts the attack behavior of the attack node 610, maps the attack behavior of the attack node 610 to the ATT&CK framework, and obtains the matrix framework coverage table.

[0159] The data node 620 detects the attack behavior of the attack node 610 based on an automated detection tool (ie, a defense detection tool).

[0160] During the attack and defense confrontation process, the terminal determines the confrontation simulation target, that is, the effect that the attack node 610 and the data node 620 are expected to achieve. After the terminal determines the target, it collects data for building a confrontation simulation plan. The terminal designs a specific confrontation simulation plan based on the collected data. The terminal constructs a threat scenario, that is, sets an attack target for the attack node 610, and allows the attack node 610 to test the defense effect of the defense system based on known attack behaviors and attack technologies. The attack node 610 simulates the threat and attacks the attack node 610. When the attack node 610 launches an attack, the data node 620 investigates the attack behavior so as to detect the attack behavior in time.

[0161] When the tasks of the attack node 610 and the data node 620 are completed during the attack and defense simulation, the activities of the attack node 610 and the data node 620 are comprehensively compared to obtain an evaluation performance report. This allows the data node 620 to understand the extent to which it has been successful in discovering the attack behavior of the attack node 610, thereby evaluating and verifying the accuracy of the defense system corresponding to the data node 620. The evaluation performance report can also be used to improve the defense system and determine which attack behaviors require the development or installation of new sensors, the collection of new data sets, or the formulation of new analysis plans.

[0162] In summary, the method provided by the present application can evaluate the effectiveness of the defense system by simulating the network attack and defense confrontation process, and can timely discover the security loopholes of the defense system. Sending an attack instruction to the attack node, so that the attack node attacks the data node and receives the attack result report sent by the attack node, can determine the attack effect of the attack node; when the attack node attacks the data, the attack behavior it performs is converted into specific attack technology data, and the defense result report generated by the data node when defending against the attack behavior from the attack node is received. Based on the attack technology data, the attack result report and the defense result report, the specific situation of the attack technology that the defense system can defend can be determined, and the accuracy can be improved when evaluating the defense effect of the defense system.

[0163] The method provided in this embodiment generates a matrix framework coverage table based on attack technology data and the matrix framework through the ATT&CK matrix framework, which can count the attack technologies successfully defended by the defense system and improve the accuracy when evaluating the defense effect of the defense system.

[0164] Figure 7 is a structural block diagram of an evaluation device for a defense system provided by an exemplary embodiment of the present application, such as Figure 7 As shown, the device includes the following parts.

[0165] A sending module 710 is used to send an attack instruction to an attack node, wherein the attack instruction is used to instruct the attack node to perform a data attack on a data node, wherein the data node includes a defense detection tool, and the defense detection tool is used to perform a defense detection on the attack received by the data node and generate a defense result report;

[0166] A receiving module 720 is configured to receive an attack result report sent by the attack node, wherein the attack result report includes attack behavior data when the attack node performs a data attack on the data node;

[0167] An acquisition module 730 is used to acquire attack technical data, where the attack technical data is obtained by converting the attack behavior performed by the attack node when performing a data attack on the data node into technical data;

[0168] The receiving module 720 is further configured to receive a defense result report sent by the data node, wherein the defense result report includes a result of the defense detection tool in the data node defending against the attack behavior;

[0169] The generating module 740 is used to generate an evaluation result based on the attack result report, the attack technical data and the defense result report, wherein the evaluation result is used to evaluate the defense effect of the defense detection tool in data attack defense.

[0170] In an optional embodiment, the acquisition module 730 is also used to obtain the attack behavior data in the attack result report; perform technical conversion on the attack behavior data to obtain the attack technical data; or, receive the attack technical data sent by a server, the server is used to receive the attack result report sent by the attack node, and convert the attack behavior data in the attack result report into the attack technical data.

[0171] In an optional embodiment, if Figure 8 As shown, the device also includes:

[0172] The sending module 710 is further used to send a first data acquisition request to the server, where the first data acquisition request is used to request to acquire first data, where the first data is data used in formulating a plan during an attack and defense simulation, and the server is used to collect the first data through a sensor, where the sensor is deployed inside the server;

[0173] The receiving module 720 is further configured to receive the first data fed back by the server;

[0174] The plan formulation module 750 is used to formulate an attack simulation plan based on the first data, and generate the attack instruction based on the attack simulation plan.

[0175] In an optional embodiment, the plan formulation module 750 is also used to determine the attack target of the attack node based on the first data, and the attack target is used to represent the expected attack effect achieved by the attack node; and generate the attack instruction based on the attack target.

[0176] In an optional embodiment, the sending module 710 is further used to send a detection instruction to the data node, where the detection instruction is used to instruct the data node to detect and defend against the attack behavior executed by the attack node.

[0177] In an optional embodiment, before the sending module 710, the plan formulation module 750 is further used to formulate a defense simulation plan based on the first data, and generate the detection instruction based on the defense simulation plan.

[0178] In an optional embodiment, the plan formulation module 750 is also used to determine the defense target of the defense detection tool based on the first data, and the defense target is used to represent the expected defense effect achieved by the defense detection tool, and the defense target includes the detection range of the defense detection tool in the data node; the detection instruction is generated based on the defense target.

[0179] In an optional embodiment, the generation module 740 is also used to obtain a preset matrix framework, which contains descriptive information of multiple attack technologies; based on the attack technology data and the matrix framework, generate a matrix framework coverage table, and the matrix framework coverage table is used to count the attack technologies successfully defended by the defense system; based on the attack result report and the defense result report, determine the first technology category of the attack technology successfully defended by the defense system, and record the attack technology data corresponding to the first technology category in the matrix framework coverage table.

[0180] In summary, the evaluation device of the defense system provided by the present application can evaluate the effect of the defense system by simulating the network attack and defense confrontation process, and can timely discover the security loopholes of the defense system. Sending an attack instruction to the attack node, so that the attack node attacks the data node and receives the attack result report sent by the attack node, can determine the attack effect of the attack node; when the attack node attacks the data, the attack behavior it performs is converted into specific attack technology data, and the defense result report generated by the data node when defending against the attack behavior from the attack node is received. Based on the attack technology data, the attack result report and the defense result report, the specific situation of the attack technology that the defense system can defend can be determined, and the accuracy can be improved when evaluating the defense effect of the defense system.

[0181] It should be noted that the evaluation device for the defense system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the evaluation device for the defense system provided in the above embodiment and the evaluation method embodiment of the defense system belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0182] Fig. 9The block diagram of the structure of a computer device 900 provided by an exemplary embodiment of the present application is shown. The computer device 900 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) player, a laptop computer or a desktop computer. The computer device 900 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0183] Typically, the computer device 900 includes a processor 901 and a memory 902 .

[0184] The processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0185] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one instruction, which is used to be executed by the processor 901 to implement the evaluation method of the defense system provided in the method embodiment of the present application.

[0186] In some embodiments, the computer device 900 further includes other components, which can be understood by those skilled in the art. Fig. 9 The structure shown in the figure does not constitute a limitation on the computer device 900, and the computer device 900 may include more or less components than those shown in the figure, or combine some components, or adopt a different arrangement of components.

[0187] Optionally, the computer readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid state drive (SSD), or an optical disk. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0188] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the defense system evaluation method as described in any of the above embodiments of the present application.

[0189] An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the defense system evaluation method as described in any of the above embodiments of the present application.

[0190] The present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the evaluation method of the defense system described in any of the above embodiments.

[0191] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0192] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for evaluating a defense system, characterized in that: The method comprises: Sending an attack instruction to an attack node, wherein the attack instruction is used to instruct the attack node to perform a data attack on a data node, wherein the data node includes a defense detection tool, and the defense detection tool is used to perform a defense detection on the attack received by the data node and generate a defense result report; receiving an attack result report sent by the attack node, wherein the attack result report includes attack behavior data when the attack node performs a data attack on the data node; Acquiring attack technical data, wherein the attack technical data is obtained by converting the attack behavior performed by the attack node when performing a data attack on the data node into technical data; receiving a defense result report sent by the data node, wherein the defense result report includes a result of a defense detection tool in the data node defending against the attack behavior; An evaluation result is generated based on the attack result report, the attack technology data and the defense result report, and the evaluation result is used to evaluate the defense effect of the defense detection tool in data attack defense.

2. The method according to claim 1, characterized in that The obtaining of attack technology data includes: Obtaining the attack behavior data in the attack result report; performing technical transformation on the attack behavior data to obtain the attack technical data; or, The attack technology data sent by the server is received, and the server is used to receive the attack result report sent by the attack node, and convert the attack behavior data in the attack result report into the attack technology data.

3. The method according to claim 1, characterized in that Before sending the attack instruction to the attack node, the method further includes: Sending a first data acquisition request to a server, where the first data acquisition request is used to request to acquire first data, where the first data is data used in formulating a plan during an attack and defense simulation, and the server is used to collect the first data through a sensor, where the sensor is deployed inside the server; receiving the first data fed back by the server; An attack simulation plan is formulated based on the first data, and the attack instruction is generated based on the attack simulation plan.

4. The method according to claim 3, characterized in that Formulating an attack simulation plan based on the first data, and generating the attack instruction based on the attack simulation plan, including: Determine an attack target of the attack node based on the first data, where the attack target is used to represent an expected attack effect achieved by the attack node; The attack instruction is generated based on the attack target.

5. The method according to claim 3, characterized in that: Before receiving the defense result report sent by the data node, the method further includes: A detection instruction is sent to the data node, where the detection instruction is used to instruct the data node to detect and defend against the attack behavior performed by the attack node.

6. The method according to claim 5, characterized in that Before sending the detection instruction to the data node, the method further includes: A defense simulation plan is formulated based on the first data, and the detection instruction is generated based on the defense simulation plan.

7. The method according to claim 6, characterized in that The formulating a defense simulation plan based on the first data, and generating the detection instruction based on the defense simulation plan, includes: Determine a defense target of the defense detection tool based on the first data, where the defense target is used to represent an expected defense effect achieved by the defense detection tool, and the defense target includes a detection range of the defense detection tool in the data node; The detection instruction is generated based on the defense target.

8. The method according to any one of claims 1 to 7, characterized in that: The generating of the evaluation result based on the attack result report, the attack technical data and the defense result report includes: Obtaining a preset matrix framework, wherein the matrix framework includes description information of multiple attack techniques; Based on the attack technology data and the matrix framework, a matrix framework coverage table is generated, wherein the matrix framework coverage table is used to count the attack technologies successfully defended by the defense system; A first technology category of the attack technology successfully defended by the defense system is determined based on the attack result report and the defense result report, and the attack technology data corresponding to the first technology category is recorded in the matrix framework coverage table.

9. A defense system evaluation device, characterized in that: The device comprises: A sending module, used to send an attack instruction to an attack node, wherein the attack instruction is used to instruct the attack node to perform a data attack on a data node, wherein the data node includes a defense detection tool, and the defense detection tool is used to perform a defense detection on the attack received by the data node and generate a defense result report; A receiving module, used to receive an attack result report sent by the attack node, wherein the attack result report includes attack behavior data when the attack node performs a data attack on the data node; An acquisition module, used to acquire attack technical data, wherein the attack technical data is obtained by converting the attack behavior performed by the attack node when performing a data attack on the data node into technical data; The receiving module is further used to receive a defense result report sent by the data node, wherein the defense result report includes a result of the defense detection tool in the data node defending against the attack behavior; A generation module is used to generate an evaluation result based on the attack result report, the attack technology data and the defense result report, and the evaluation result is used to evaluate the defense effect of the defense detection tool in data attack defense.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the defense system evaluation method as described in any one of claims 1 to 8.

Images