Network attack and defense training method, equipment and storage medium based on multi-agent collaboration

Abstract

The present application discloses a network attack and defense training method, device and storage medium based on multi-agent collaboration, which relates to the field of electronic digital data processing. The method includes: based on the attack and defense instructions, calling at least one agent corresponding to the attack and defense instructions to perform collaborative defense actions and / or collaborative counter-actions; based on the network environment parameters, instruction execution type and target type corresponding to each of the agents, determining the environment evaluation parameters of the agent; based on the execution results of the attack and defense instructions, determining the execution evaluation parameters of the agent for the attack and defense instructions; based on the environment evaluation parameters and the execution evaluation parameters, updating the behavior strategy of the agent. Therefore, the technical problem that the decision-making method in the related technology cannot adapt to the frequently changing attack and defense environment is effectively solved, and the agent that executes the attack and defense decision is continuously evolved, and the technical effect of improving the attack and defense efficiency is achieved.

Description

Technical Field

[0001] The present application relates to the field of electronic digital data processing, and in particular to a network attack and defense training method, device and storage medium based on multi-agent collaboration. Background Art

[0002] The attack and defense scenarios in network testing require the use of a variety of technologies to simulate actual attack and defense scenarios in order to test the security and vulnerabilities of the network system.

[0003] In the related technologies, the main network attack and defense game decision-making method is the network attack and defense decision-making method based on evolutionary game, which has been widely used in fields such as wireless sensor networks. This method focuses on providing strategy selection guidance for defenders and solving the optimal strategy based on the replication dynamic equation. The evolutionary game method is based on the evolutionary process of rational actors. It assumes that all participants are rational and will choose the strategy that can obtain the maximum benefit.

[0004] However, in actual network attack and defense, attackers and defenders usually have uncertain and non-completely rational behavior patterns, which makes the decision-making methods in related technologies unable to adapt to the frequently changing attack and defense environment. Summary of the invention

[0005] The embodiments of the present application provide a network attack and defense training method, device and storage medium based on multi-agent collaboration, thereby solving the technical problem that the decision-making method in the related technology cannot adapt to the frequently changing attack and defense environment, and realizes the technical effect of continuous evolution of the intelligent agents that execute attack and defense decisions and improving the attack and defense efficiency.

[0006] The present application embodiment provides a network attack and defense training method based on multi-agent collaboration, and the network attack and defense training method based on multi-agent collaboration includes:

[0007] Based on the attack and defense instructions, calling at least one agent corresponding to the attack and defense instructions to perform a collaborative defense action and / or a collaborative counter-action;

[0008] Determine the environment evaluation parameters of the agent based on the network environment parameters, instruction execution type and target type corresponding to each agent;

[0009] Determining an execution evaluation parameter of the agent for the attack and defense instruction based on the execution result of the attack and defense instruction;

[0010] Based on the environmental evaluation parameters and the execution evaluation parameters, the behavior strategy of the agent is updated.

[0011] Optionally, the step of calling at least one agent corresponding to the attack and defense instruction to perform a collaborative defense action and / or a collaborative counter-action based on the attack and defense instruction includes:

[0012] Upon receiving the attack and defense instruction, determining at least one of the intelligent agents according to the attack and defense instruction;

[0013] When the attack and defense instruction is cooperative defense, based on the first intelligent agent attacked among the intelligent agents, executing the cooperative defense action;

[0014] When the attack and defense instruction is a collaborative counterattack, the counterattack action and the execution order to be performed by each of the intelligent agents are determined based on the attack and defense instruction, and the collaborative counterattack action is performed.

[0015] Optionally, when the attack and defense instruction is collaborative defense, the step of executing the collaborative defense action based on the attacked first intelligent agent among the intelligent agents includes:

[0016] When the attack and defense instruction is cooperative defense, determining the first intelligent agent to be attacked among the intelligent agents, and the second intelligent agent for cooperative defense;

[0017] Determine a collaborative defense strategy according to the attack type and attack parameters of the attacker and the type of device where the first agent is located;

[0018] The defense actions to be performed by each of the intelligent agents and the execution order are determined based on the cooperative defense strategy to control the intelligent agents to perform the cooperative defense actions.

[0019] Optionally, after the step of determining the collaborative defense strategy according to the attack type and attack parameters of the attacker and the type of device where the first agent is located, the step further includes:

[0020] When a change is detected in the first agent and / or the attacking party, determining a type of change;

[0021] The collaborative defense strategy is updated based on the change type.

[0022] Optionally, when the attack and defense instruction is a collaborative counterattack, determining the counterattack action and the execution order to be performed by each of the agents based on the attack and defense instruction, and the step of performing the collaborative counterattack action includes:

[0023] When the attack and defense instruction is a collaborative counterattack, determining a counterattack target based on the attack and defense instruction;

[0024] Based on the countermeasure target and the preset behavior strategy of the agent, determining the countermeasure action and the execution order corresponding to each agent;

[0025] According to the encryption algorithm and execution path corresponding to the counter-action, the agent is controlled to execute the collaborative counter-action at the time point corresponding to the execution sequence.

[0026] Optionally, the step of determining the environment evaluation parameters of the agent based on the network environment parameters, instruction execution type and target type corresponding to each agent includes:

[0027] Determining the network environment parameters based on the network fluctuation, network speed average, and delay average of the agent during the execution of the collaborative defense action and / or the collaborative counter-action;

[0028] Determine the instruction execution type and the target type based on the collaborative defense action and / or collaborative counter-action performed by the agent, wherein the target type is a type predefined by the attacker and / or the counter-target;

[0029] The environment evaluation parameter is determined based on the weights corresponding to the network environment parameter, the instruction execution type and the target type.

[0030] Optionally, the step of determining the execution evaluation parameter of the agent for the attack and defense instruction based on the execution result of the attack and defense instruction includes:

[0031] Obtaining a preset execution evaluation function, wherein the execution evaluation function corresponds to a defense execution result and a counterattack execution result respectively;

[0032] Determine the defense execution result based on the corresponding defense frequency ratio, defense efficiency ratio and defense action execution efficiency when the intelligent agent executes the cooperative defense action; or

[0033] Determine the counter-action execution result based on the counter-action success rate, the degree of damage and the execution efficiency of the counter-action corresponding to the execution of the collaborative counter-action by the agent;

[0034] The execution evaluation parameter is determined based on the defense execution result and / or counter-execution result corresponding to the attack and defense instruction executed by the agent, and the execution evaluation function.

[0035] Optionally, the step of updating the behavior strategy of the agent based on the environmental evaluation parameter and the execution evaluation parameter includes:

[0036] Obtaining a corresponding relationship between the environmental evaluation parameter and the strategy space corresponding to the agent;

[0037] Based on the execution evaluation parameter, the policy space corresponding to the environmental evaluation parameter is updated.

[0038] In addition, the present application also proposes a network attack and defense training device based on multi-agent collaboration, and the network attack and defense training device based on multi-agent collaboration includes a memory, a processor, and a network attack and defense training program based on multi-agent collaboration stored in the memory and executable on the processor. When the processor executes the network attack and defense training program based on multi-agent collaboration, the steps of the network attack and defense training method based on multi-agent collaboration as described above are implemented.

[0039] In addition, the present application also proposes a computer-readable storage medium, on which is stored a network attack and defense training program based on multi-agent collaboration. When the network attack and defense training program based on multi-agent collaboration is executed by a processor, the steps of the network attack and defense training method based on multi-agent collaboration as described above are implemented.

[0040] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0041] Because it adopts the method of calling at least one intelligent agent corresponding to the attack and defense instruction based on the attack and defense instruction to perform collaborative defense actions and / or collaborative counter-actions; determining the environmental evaluation parameters of the intelligent agent based on the network environment parameters, instruction execution type and target type corresponding to each of the intelligent agents; determining the execution evaluation parameters of the intelligent agent for the attack and defense instruction based on the execution result of the attack and defense instruction; and updating the behavior strategy of the intelligent agent based on the environmental evaluation parameters and the execution evaluation parameters. Therefore, it effectively solves the technical problem that the decision-making method in the related technology cannot adapt to the frequently changing attack and defense environment, realizes the continuous evolution of the intelligent agent that executes the attack and defense decision, and improves the technical effect of attack and defense efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flowchart of the first embodiment of the network attack and defense training method based on multi-agent collaboration of the present application;

[0043] Figure 2 This is a flow chart of steps S210-S230 in Embodiment 2 of the network attack and defense training method based on multi-agent collaboration of the present application;

[0044] Figure 3 This is a flow chart of steps S231-S233 in Embodiment 2 of the network attack and defense training method based on multi-agent collaboration of the present application;

[0045] Figure 4 This is a flowchart of Embodiment 4 of the network attack and defense training method based on multi-agent collaboration of the present application;

[0046] Figure 5 This is a schematic diagram of the hardware structure involved in an embodiment of the network attack and defense training equipment based on multi-agent collaboration in this application. DETAILED DESCRIPTION

[0047] In the related art, the agent first identifies and evaluates potential threats by monitoring and analyzing network traffic, system logs, and other related data, including detecting malware, abnormal behavior, login failures, port scanning, etc. Then, the agent takes preventive measures to protect the network system based on the identified threats, including configuring and managing firewall rules, installing and updating security patches, strengthening access control, disabling unnecessary services, etc. The agent also monitors network activities in real time, using technologies such as intrusion detection systems (IDS) and intrusion prevention systems (IPS) to detect intrusion behaviors and potential attacks. Once a threat is detected, the agent responds quickly and tries to prevent further attacks. However, in the above technical solution, each agent executes attack and defense decisions alone, and cannot adapt to the frequently changing attack and defense environment. The main technical solution adopted in the embodiment of the present application is: determining at least one agent based on attack and defense instructions, and controlling the agent to collaboratively execute corresponding actions according to attack instructions or defense instructions, and co-evolving the execution strategy of each agent in each environment according to the execution results and parameters such as the network environment to update the overall behavior decision of the agent. Thereby achieving the technical effect of continuous evolution of the agent and improving the attack and defense efficiency.

[0048] In order to better understand the above technical solution, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0049] Embodiment 1

[0050] The present application discloses a network attack and defense training method based on multi-agent collaboration. Figure 1 , the network attack and defense training method based on multi-agent collaboration includes:

[0051] Step S110, based on the attack and defense instructions, calling at least one intelligent agent corresponding to the attack and defense instructions to perform collaborative defense actions and / or collaborative counter-actions.

[0052] In this embodiment, the attack and defense instructions are issued by the host computer, or generated according to the preset trigger instructions. For example, when the intelligent agent is detected to be attacked, the corresponding defense instructions are generated. When the defense performance of the target needs to be tested, an attack instruction or a counter-instruction for the target is generated. After receiving the attack and defense instructions, the intelligent agent performs the corresponding collaborative defense action and / or collaborative counter-action based on the type of the attack and defense instructions and the current network environment, through the corresponding behavior strategy, and the specific content corresponding to the attack and defense instructions.

[0053] As an optional implementation, the attack and defense instructions are attack instructions, and the agents perform cooperative counter-actions according to the received attack instructions. The cooperative execution is a part of the actions performed by each agent, or a link, to jointly complete the attack task against several targets.

[0054] As an optional implementation, the attack and defense instructions are defense instructions, and the intelligent agents cooperate to perform collaborative defense actions based on the received defense instructions.

[0055] As another optional implementation, the attack and defense instructions are counter instructions, and the intelligent agent partially performs collaborative defense actions based on the received counter instructions, and another part performs collaborative counter actions against the attacker.

[0056] Step S120, determining the environment evaluation parameters of the agent based on the network environment parameters, instruction execution type and target type corresponding to each agent.

[0057] In this embodiment, the network environment parameters, instruction execution type and target type are the parameters collected for each agent when the corresponding action is executed for the attack and defense instructions. The network environment parameters identify the quality of the network in which the agent is located when executing the corresponding action. The instruction execution type is an attack and defense instruction, which is an attack instruction, a defense instruction or a counter instruction. The counter instruction can be regarded as a combination of a defense instruction and an attack instruction. The target type is the type information of the attacker or the attacked party. It includes the device type, attack type, defense type, network address, etc. Then, the environmental evaluation parameters are generated according to the above parameters, that is, there are environmental evaluation parameters for each agent.

[0058] Furthermore, when executing attack and defense instructions, the intelligent agent obtains the current network situation, and determines the action that the intelligent agent ultimately needs to perform based on the instruction execution type corresponding to the attack and defense instructions and the execution strategy obtained by matching the network situation with the environmental evaluation parameters.

[0059] Step S130, determining the execution evaluation parameters of the agent for the attack and defense instructions based on the execution results of the attack and defense instructions.

[0060] In this embodiment, the execution result is the execution result of the attack and defense instruction, including the attack result and the defense result. For different instruction execution types, the corresponding execution result evaluation scheme is determined, and the execution evaluation parameters of each intelligent agent for the attack and defense instruction are determined according to the data corresponding to the execution result.

[0061] Step S140: updating the behavior strategy of the agent based on the environmental evaluation parameters and the execution evaluation parameters.

[0062] In this embodiment, the behavior strategy is the strategy for the agent to execute the attack and defense instructions. According to the behavior strategy corresponding to each network situation, and then according to the specific execution content in the attack and defense instructions, the behavior strategy determined based on the network environment determines the final execution strategy of the agent.

[0063] As an optional implementation, for each agent, the corresponding behavior strategy is determined based on the environment evaluation parameter corresponding to the agent, and based on the evaluation threshold corresponding to the execution evaluation parameter and the behavior strategy, when the execution evaluation parameter belongs to the interval corresponding to the evaluation threshold, the behavior strategy is not updated. When the execution evaluation parameter is less than the interval corresponding to the evaluation threshold, the behavior strategy is updated to improve the attack and defense efficiency of the agent; when the execution evaluation parameter is greater than the interval corresponding to the evaluation threshold, the execution evaluation parameter is used as the new evaluation threshold.

[0064] As an example of the present application, based on the attack and defense instructions, the agent is called to perform collaborative defense actions and / or collaborative counter-actions. The host computer issues attack and defense instructions by communicating with the agent. According to the preset trigger instruction or when the agent is detected to be attacked, the corresponding defense instruction is generated. Alternatively, when the defense performance of the target needs to be tested, an attack instruction or counter-instruction for the target is generated. After the agent receives the attack and defense instruction, it performs collaborative defense actions and / or collaborative counter-actions according to the type of instruction, network environment and behavior strategy. According to the content of the attack and defense instruction, the agent can perform attack tasks, defense measures or counter-attack against the attacker. Determine the environmental evaluation parameters of the agent. Each agent collects network quality and other relevant parameters according to the network environment in which it is located when executing the action. Based on the collected parameters, generate environmental evaluation parameters. A machine learning algorithm or a rule-based method can be used to match the network environment and instruction type with the preset evaluation parameters to generate the environmental evaluation parameters of each agent. Determine the execution evaluation parameters of the agent for the attack and defense instructions. According to the execution results of the attack and defense instructions, including the attack results and the defense results, determine the execution evaluation parameters of each agent for the attack and defense instructions. Various data can be considered, such as attack success rate, defense effect, etc. Update the behavior strategy of the agent. Update the behavior strategy of each agent according to the environmental evaluation parameters and the execution evaluation parameters. The final execution strategy of the agent can be determined based on the behavior strategy corresponding to each network situation and the specific content in the attack and defense instructions. Based on the evaluation threshold corresponding to the execution evaluation parameter and the behavior strategy, when the execution evaluation parameter is in the interval corresponding to the evaluation threshold, the behavior strategy is not updated. When the execution evaluation parameter is lower than the evaluation threshold, the behavior strategy is updated to improve the attack and defense efficiency. When the execution evaluation parameter is higher than the evaluation threshold, the execution evaluation parameter is used as the new evaluation threshold. The agent determines the final action to be performed based on the updated behavior strategy and the current network status.

[0065] For example, in a secure environment of an online game, attackers gain an advantage by discovering vulnerabilities and creating cheating tools, while defenders deploy basic security measures to prevent them. In this testing process, the intelligent agent acts as both the attacker and the defender.

[0066] When the agent acts as an attacker, it develops new tools, i.e. new attack actions, as it adapts to the defender's detection mechanism corresponding to the attack action. When the agent acts as a defender, it continuously updates the coordinated defense actions based on the attacker's attack actions and the defense results, and upgrades its detection system by introducing complex behavioral analysis and machine learning models. This continuous interaction between attack and defense forms a dynamic feedback loop, leading to the continuous evolution of both sides' technology and strategies. This co-evolution not only improves the security of the game, but also improves the efficiency and intelligence of defense measures, reduces the occurrence of cheating, and demonstrates that system security and complexity can be enhanced through interaction and adaptation in an adversarial environment.

[0067] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0068] Because it adopts the method of calling at least one intelligent agent corresponding to the attack and defense instruction based on the attack and defense instruction to perform collaborative defense actions and / or collaborative counter-actions; determining the environmental evaluation parameters of the intelligent agent based on the network environment parameters, instruction execution type and target type corresponding to each of the intelligent agents; determining the execution evaluation parameters of the intelligent agent for the attack and defense instruction based on the execution result of the attack and defense instruction; and updating the behavior strategy of the intelligent agent based on the environmental evaluation parameters and the execution evaluation parameters. Therefore, it effectively solves the technical problem that the decision-making method in the related technology cannot adapt to the frequently changing attack and defense environment, realizes the continuous evolution of the intelligent agent that executes the attack and defense decision, and improves the technical effect of attack and defense efficiency.

[0069] Based on Example 1, Example 4 of the present application proposes a network attack and defense training method based on multi-agent collaboration, see Figure 2 , step S110 includes:

[0070] Step S210, upon receiving the attack and defense instruction, determining at least one of the intelligent agents according to the attack and defense instruction.

[0071] In this embodiment, at least one intelligent agent jointly executes the attack and defense instructions, that is, each intelligent agent executes part of the attack and defense instructions.

[0072] When an attack and defense command corresponds to a completed action, the action is divided into several blocks, each of which corresponds to at least one intelligent agent. The above intelligent agents jointly complete the action corresponding to the attack and defense command.

[0073] Step S220, when the attack and defense instruction is cooperative defense, the cooperative defense action is performed based on the first intelligent agent attacked among the intelligent agents.

[0074] In this embodiment, when receiving the attack and defense instruction, the instruction content is first parsed to determine the agents that need to participate in the collaborative defense and collaborative counterattack. When the attack and defense instruction is collaborative defense, the collaborative defense action is performed according to the first agent attacked specified in the instruction. This can be achieved by calling the defense function interface of the agent. For example, the agent can perform defense actions such as updating firewall rules and filtering traffic according to the instruction.

[0075] Optionally, step S220 includes:

[0076] Step S221, when the attack and defense instruction is cooperative defense, determine the first intelligent agent to be attacked among the intelligent agents, and the second intelligent agent for cooperative defense.

[0077] In this embodiment, the number of first agents is at least one, that is, several agents can be attacked at the same time. That is, when a network attack is detected, a corresponding defense instruction is generated based on the attacked agent, the attacked agent is used as the first agent, and the other agents that cooperate in defense are used as the second agent.

[0078] Step S222, determining a collaborative defense strategy based on the attack type and attack parameters of the attacker and the type of device where the first agent is located.

[0079] In this embodiment, the attack type is determined by analyzing the relevant parameters of the attacker when conducting a network attack, and the corresponding parameters are collected to generate attack parameters according to the parameter template corresponding to the attack type. The agent is deployed on the device entity, such as each terminal, or server, host computer, etc., to determine the device type deployed by the first agent, that is, the attacked agent. According to the attack type, attack parameters and device type, the overall collaborative defense strategy is determined. This strategy determines the overall defense plan and does not specify the specific matters that each agent needs to do.

[0080] Step S223, determining the defense action to be performed by each of the intelligent agents and the execution order based on the collaborative defense strategy, so as to control the intelligent agents to perform the collaborative defense action.

[0081] In this embodiment, based on the determined collaborative defense strategy and the first agent and the second agent, the first action parameter to be executed by the first agent and the second action parameter to be executed by the second agent are determined according to the collaborative defense strategy; based on the action parameters and the behavior strategy corresponding to the agent, the defense action to be executed by each agent and the execution order of each agent are determined, wherein the agent can execute different defense actions at different execution nodes. Then all agents are completed to jointly execute the collaborative defense action.

[0082] As an optional implementation, when the attack and defense instruction is collaborative defense, first determine the first attacked agent and the second agent for collaborative defense. The attacked first agent refers to the agent that is first attacked in the attack. Then, determine the collaborative defense strategy based on the attack type, attack parameters and device type of the attacker. The collaborative defense strategy may include but is not limited to the following: selecting appropriate defense actions, determining the execution order of defense, allocating resources, etc. Based on the collaborative defense strategy, determine the defense actions and execution order performed by each agent. This can be done by exchanging information and communicating. Agents can share attack information, defense strategies and execution status to ensure that the execution of each defense action can be effectively coordinated. Control the agent to perform collaborative defense actions. According to the determined execution order, call the defense function interface of the agent in sequence to perform collaborative defense actions. Defense actions may include but are not limited to the following: updating firewall rules, filtering attack traffic, isolating attacked hosts, etc.

[0083] Optionally, after step S222, the following steps are included:

[0084] When a change is detected in the first intelligent agent and / or the attacking party, the type of change is determined; and the collaborative defense strategy is updated based on the type of change.

[0085] In this embodiment, the target of the attacker may change. Therefore, when the collaborative defense action has not been completed, if the attacker's attack target is detected, or the attacker itself has changed, the collaborative defense strategy is adjusted based on the type of change.

[0086] Step S230, when the attack and defense instruction is a collaborative counterattack, determine the counterattack action and execution order to be performed by each of the intelligent agents based on the attack and defense instruction, and perform the collaborative counterattack action.

[0087] In this embodiment, when the attack and defense instruction is a collaborative counterattack, the counteractions and execution order performed by each agent are first determined according to the instruction. The execution order of the counteractions can be determined using an algorithm or rule. For example, the most suitable agent can be selected to perform the counteraction based on information such as the location, capabilities, and status of the agent. According to the determined execution order, the counteraction function interface of the agent is called in sequence to perform the collaborative counteraction. The counteraction may include quickly isolating the attacked host, limiting the attack traffic, starting the intrusion detection system, etc. Coordination and synchronization of counteractions. In order to maximize the effect of collaborative counteraction, the counteractions of each agent need to be coordinated and synchronized during the execution of the counteraction. Coordination and synchronization can be achieved by exchanging information and communicating. For example, agents can share attack information, counteraction strategies, and execution status to ensure that the execution of each counteraction can be effectively coordinated.

[0088] Alternatively, see Figure 3 , step S230 includes:

[0089] Step S231, when the attack and defense instruction is a collaborative counterattack, a counterattack target is determined based on the attack and defense instruction.

[0090] In this embodiment, the counter-target is the target that the agent needs to attack. That is, in the network attack and defense training, the agent acts as the attacker to test the defense performance of the defender.

[0091] Step S232, based on the countermeasure target and the preset behavior strategy of the agent, determine the countermeasure action and the execution order corresponding to each agent.

[0092] Step S233, according to the encryption algorithm and execution path corresponding to the counter-action, control the agent to perform the collaborative counter-action at the time point corresponding to the execution sequence.

[0093] In this embodiment, collaborative counter-actions are performed based on attack and defense instructions. When the attack and defense instructions are collaborative counter-actions, the counter-action target is first determined according to the instructions. The counter-action target refers to the attacker or the attack behavior. By analyzing the key information in the attack instruction, the target that needs to be countered can be determined. Based on the counter-action target and the preset behavior strategy of the agent, the corresponding counter-actions of each agent and their execution order are determined. The preset behavior strategy is a set of counter-action strategies for different attack targets set in advance by the agent, including specific counter-actions and their execution order. According to the encryption algorithm and execution path corresponding to the counter-action, the agent is controlled to perform collaborative counter-actions at the time point corresponding to the execution order. The execution path refers to the execution path set by the agent according to the encryption algorithm and execution order, which is used to ensure the accurate execution of the counter-action.

[0094] For example, assume that there are two agents A and B in a network system, and the attack and defense instructions are collaborative counterattack. The preset behavior strategies of agents A and B are as follows:

[0095] Agent A presets the behavior strategy: Counteraction 1: Block the attack source IP address. Counteraction 2: Remove the malware. Execution order: Action 1 -> Action 2. Agent B presets the behavior strategy: Counteraction 1: Block the attack traffic. Counteraction 2: Restore the attacked device status. Execution order: Action 2 -> Action 1. When the attack and defense instructions are collaborative countermeasures, the steps for executing collaborative countermeasures are as follows: According to the attack and defense instructions, the countermeasure targets are determined to be the blocking of malware propagation and attack traffic. Agent A executes action 1 to block the attack source IP address according to the preset behavior strategy, and continues to execute action 2 according to the execution order to remove the malware. Agent B executes action 2 to restore the attacked device status according to the preset behavior strategy, and then executes action 1 to block the attack traffic.

[0096] When the attack and defense instruction is received, at least one of the intelligent agents is determined according to the attack and defense instruction; when the attack and defense instruction is collaborative defense, based on the first intelligent agent attacked among the intelligent agents, the collaborative defense action is performed; when the attack and defense instruction is collaborative counterattack, the counteraction and execution order to be performed by each of the intelligent agents are determined based on the attack and defense instruction, and the collaborative counteraction action is performed. Multiple intelligent agents are implemented to collaboratively conduct attack and defense tests, respectively as attackers and defenders, and perform corresponding collaborative defense actions and / or collaborative attack actions. That is, in this process, the status of the intelligent agent as an attacker and defender can be switched at any time. The system's response speed and execution efficiency to attack and defense instructions are further improved.

[0097] Based on the first embodiment, the third embodiment of the present application proposes a network attack and defense training method based on multi-agent collaboration, and step S120 includes:

[0098] Step S310, determining the network environment parameters based on the network fluctuation, average network speed and average delay of the agent during the execution of the collaborative defense action and / or collaborative counter-action.

[0099] In this embodiment, network environment parameters are determined to monitor network fluctuations during the execution of collaborative defense and countermeasure actions by the agent, including network jitter, packet loss rate, and delay fluctuations.

[0100] Calculate the average network speed and average delay, which can be measured using network monitoring tools or by the agent itself. Based on these parameters, the network status of the agent in a specific network environment can be obtained.

[0101] Step S320, determining the instruction execution type and the target type based on the collaborative defense action and / or collaborative counter-action performed by the agent, wherein the target type is a type predefined by the attacker and / or the counter-target.

[0102] In this embodiment, the instruction execution type and target type are determined, the collaborative defense and counter-attack actions performed by the agent are monitored, and the execution type thereof, such as whether to perform defense or counter-attack, is determined.

[0103] Determine the target type, that is, determine whether the agent takes action against the attacker or the counterattack target, and choose from pre-defined types.

[0104] Step S330, determining the environment evaluation parameter based on the weights corresponding to the network environment parameter, the instruction execution type, and the target type.

[0105] In this embodiment, the environmental evaluation parameter is determined. In order to determine the environmental evaluation parameter, the network environment parameter, the instruction execution type, and the weight corresponding to the target type need to be combined and calculated. This can be achieved by a predefined formula or algorithm. For example, a weighted average method can be used to calculate the value of the environmental evaluation parameter according to the weights of different network environment parameters, instruction execution types, and target types.

[0106] Exemplarily, the defense strategy is adjusted according to network conditions and instruction types. During execution, the system monitors network fluctuations, including jitter, packet loss rate, and delay changes. At the same time, the types of instructions executed by the system include defense and countermeasures, and the target types include attackers and countermeasure targets. For each network environment parameter, instruction execution type, and target type, the system determines the corresponding weights so that the environmental evaluation parameters can be calculated based on these parameters. For example, for network environment parameters, the system may give a higher weight to network fluctuations because this directly affects the real-time response capability of the system; for instruction execution types and target types, the system may make dynamic adjustments based on the current security threat situation.

[0107] Through comprehensive evaluation of these parameters, the intelligent defense system can adopt the most appropriate defense strategy according to the current network environment and security requirements to maximize network security. At the same time, it accurately evaluates the network environment of the intelligent agent when executing attack and defense instructions to improve the matching accuracy of the behavior strategy.

[0108] Based on Example 1, Example 4 of the present application proposes a network attack and defense training method based on multi-agent collaboration, see Figure 4 , step S130 includes:

[0109] Step S410, obtaining a preset execution evaluation function, wherein the execution evaluation function corresponds to a defense execution result and a counter-execution result respectively.

[0110] In this embodiment, in the system design phase, execution evaluation functions for defense execution results and countermeasure execution results are predefined. These functions can be mathematical models based on a series of indicators, which are used to evaluate the effectiveness and efficiency of defense actions and countermeasure actions.

[0111] Step S420, determining the defense execution result based on the corresponding defense number ratio, defense efficiency ratio and execution efficiency of the defense action when the intelligent agent performs the collaborative defense action.

[0112] Step S430, alternatively, the counter-action execution result is determined based on the counter-action success rate, the degree of damage, and the execution efficiency of the counter-action corresponding to the execution of the collaborative counter-action by the agent.

[0113] In this embodiment, the defense execution result is determined based on the proportion of defense times, defense efficiency, and execution efficiency of the defense action when the agent performs collaborative defense actions. The defense times ratio refers to the proportion of successful defense times to the total defense times, and the defense efficiency ratio refers to the proportion of successful defense efficiency to the efficiency of all defenses. Alternatively, the counter-execution result is determined based on the counter-attack success rate, the degree of damage, and the execution efficiency of the counter-attack when the agent performs collaborative counter-attack actions. The counter-attack success rate refers to the proportion of the number of successful counter-attacks, and the degree of damage refers to the degree of impact on the attacker after a successful counter-attack.

[0114] Step S440, determining the execution evaluation parameters based on the defense execution result and / or counter-execution result corresponding to the attack and defense instruction executed by the agent, and the execution evaluation function.

[0115] In this embodiment, the above-mentioned defense execution results and / or counter-execution results are passed into a preset execution evaluation function to calculate corresponding execution evaluation parameters. These parameters can be used to measure the effect and performance of the intelligent agent in executing the attack and defense instructions.

[0116] As an optional implementation, the effectiveness of defense actions and counter-actions is evaluated based on the execution results, and the next strategy is adjusted based on the evaluation parameters. For defense execution results: Assume that the system determines the percentage of defense times and defense efficiency by monitoring the number of times the agent performs collaborative defense actions and the efficiency of successful defense. Then, these data are passed into the preset defense execution evaluation function, and the corresponding defense execution evaluation parameters are calculated to measure the effectiveness and performance of the agent's execution of attack and defense instructions. For counter-execution results, if the system executes a collaborative counter-action, the counter-execution results can also be determined based on the counter-success rate, the degree of damage, and the execution efficiency of the counter-action. Then these data are passed into the preset counter-execution evaluation function to calculate the corresponding counter-execution evaluation parameters.

[0117] By acquiring and analyzing execution evaluation parameters, the intelligent defense system can continuously optimize collaborative defense and countermeasure strategies to improve the ability to combat cyber attacks.

[0118] The corresponding execution evaluation function is determined according to the action type executed by the agent when executing the attack and defense strategy, and the corresponding defense execution result and counter-attack execution result are determined. Then, according to the execution evaluation parameter and the corresponding execution result, the execution evaluation parameter is determined to evaluate the execution result of the attack and defense command of the agent, so as to improve the execution efficiency of the agent for the command and the response speed when facing attack or defense.

[0119] Based on the first embodiment, the fifth embodiment of the present application proposes a network attack and defense training method based on multi-agent collaboration, and step S140 includes:

[0120] Step S510, obtaining the corresponding relationship between the environmental evaluation parameter and the strategy space corresponding to the agent.

[0121] In this embodiment, the strategy space is a set of strategies for a certain environmental evaluation parameter in the behavior strategy of the intelligent agent. That is, the behavior strategy can be regarded as a set of strategy spaces, and the strategy space is mapped to the environmental evaluation parameter, that is, the network situation.

[0122] Step S520: Based on the execution evaluation parameter, update the policy space corresponding to the environment evaluation parameter.

[0123] As an optional implementation method, the corresponding relationship between the environmental evaluation parameters and the agent's strategy space is obtained. During the system design phase, it is necessary to determine the mapping relationship between the environmental evaluation parameters and the agent's strategy space. The strategy space refers to the set of various behavioral strategies that the agent may adopt, while the environmental evaluation parameters are parameters used to describe the current environmental conditions. The agent should adopt different strategies under different environmental evaluation parameters. This mapping relationship needs to be clearly defined at the beginning of system design. Based on the execution evaluation parameters, the strategy space corresponding to the environmental evaluation parameters is updated. By analyzing the execution evaluation parameters, the effect and performance of the agent execution corresponding to the current environment can be determined. In response to changes in the execution evaluation parameters, the strategy space corresponding to the environmental evaluation parameters can be updated. This means that it is necessary to adjust the strategy set adopted by the agent in different environmental conditions according to the current execution effect to improve the agent's ability to cope with different situations.

[0124] For example, different defense strategies need to be adopted in different network environments. The system's policy space may include a set of defense strategies for different types of network attacks. The correspondence between the environmental evaluation parameters and the policy space is obtained, and the policy space of the agent under different environmental evaluation parameters is clearly defined during the system design phase. For example, when facing a DDoS (Distributed Denial of Service) attack, the strategy space that the agent may adopt includes limiting connections entering the network, excluding malicious traffic, etc. Based on the execution evaluation parameters, the policy space corresponding to the environmental evaluation parameters is updated: by analyzing the execution evaluation parameters, if it is found that a certain defense strategy currently adopted shows better performance in terms of execution effect, the weight of this strategy under a specific environmental evaluation parameter can be updated to increase its influence in the policy space.

[0125] By adopting the correspondence between the environment evaluation parameter and the strategy space corresponding to the agent, the strategy space corresponding to the environment evaluation parameter is updated based on the execution evaluation parameter. With continuous behavior evaluation and feedback, the system dynamically adjusts the strategy set adopted by the agent in different environments to better adapt to different network attack situations.

[0126] This application also proposes a network attack and defense training device based on multi-agent collaboration, referring to Figure 5 , Figure 5 This is a schematic diagram of the structure of a network attack and defense training device based on multi-agent collaboration in the hardware operating environment involved in the embodiment of the present application.

[0127] like Figure 5As shown, the network attack and defense training device based on multi-agent collaboration may include: a processor 1001, such as a core processor (Central Processing Unit, CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-VolatileMemory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0128] Those skilled in the art will understand that Figure 5 The structure shown in does not constitute a limitation on the network attack and defense training equipment based on multi-agent collaboration, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0129] Optionally, the memory 1005 is electrically connected to the processor 1001, and the processor 1001 can be used to control the operation of the memory 1005, and can also read the data in the memory 1005 to realize network attack and defense training based on multi-agent collaboration.

[0130] Alternatively, if Figure 5 As shown, the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module, and a network attack and defense training program based on multi-agent collaboration.

[0131] Optionally, in Figure 5 In the network attack and defense training device based on multi-agent collaboration shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the network attack and defense training device based on multi-agent collaboration of the present application can be set in the network attack and defense training device based on multi-agent collaboration.

[0132] like Figure 5As shown, the network attack and defense training device based on multi-agent collaboration calls the network attack and defense training program based on multi-agent collaboration stored in the memory 1005 through the processor 1001, and executes the relevant steps of the network attack and defense training method based on multi-agent collaboration provided in the embodiment of the present application:

[0133] Based on the attack and defense instructions, calling at least one agent corresponding to the attack and defense instructions to perform a collaborative defense action and / or a collaborative counter-action;

[0134] Determine the environment evaluation parameters of the agent based on the network environment parameters, instruction execution type and target type corresponding to each agent;

[0135] Determining an execution evaluation parameter of the agent for the attack and defense instruction based on the execution result of the attack and defense instruction;

[0136] Based on the environmental evaluation parameters and the execution evaluation parameters, the behavior strategy of the agent is updated.

[0137] Optionally, the processor 1001 may call a network attack and defense training program based on multi-agent collaboration stored in the memory 1005, and further perform the following operations:

[0138] Upon receiving the attack and defense instruction, determining at least one of the intelligent agents according to the attack and defense instruction;

[0139] When the attack and defense instruction is cooperative defense, based on the first intelligent agent attacked among the intelligent agents, executing the cooperative defense action;

[0140] When the attack and defense instruction is a collaborative counterattack, the counterattack action and the execution order to be performed by each of the intelligent agents are determined based on the attack and defense instruction, and the collaborative counterattack action is performed.

[0141] Optionally, the processor 1001 may call a network attack and defense training program based on multi-agent collaboration stored in the memory 1005, and further perform the following operations:

[0142] When the attack and defense instruction is cooperative defense, determining the first intelligent agent to be attacked among the intelligent agents, and the second intelligent agent for cooperative defense;

[0143] Determine a collaborative defense strategy according to the attack type and attack parameters of the attacker and the type of device where the first agent is located;

[0144] The defense actions to be performed by each of the intelligent agents and the execution order are determined based on the cooperative defense strategy to control the intelligent agents to perform the cooperative defense actions.

[0145] Optionally, the processor 1001 may call a network attack and defense training program based on multi-agent collaboration stored in the memory 1005, and further perform the following operations:

[0146] When a change is detected in the first agent and / or the attacking party, determining a type of change;

[0147] The collaborative defense strategy is updated based on the change type.

[0148] Optionally, the processor 1001 may call a network attack and defense training program based on multi-agent collaboration stored in the memory 1005, and further perform the following operations:

[0149] When the attack and defense instruction is a collaborative counterattack, determining a counterattack target based on the attack and defense instruction;

[0150] Based on the countermeasure target and the preset behavior strategy of the agent, determining the countermeasure action and the execution order corresponding to each agent;

[0151] According to the encryption algorithm and execution path corresponding to the counter-action, the agent is controlled to execute the collaborative counter-action at the time point corresponding to the execution sequence.

[0152] Optionally, the processor 1001 may call a network attack and defense training program based on multi-agent collaboration stored in the memory 1005, and further perform the following operations:

[0153] Determining the network environment parameters based on the network fluctuation, network speed average, and delay average of the agent during the execution of the collaborative defense action and / or the collaborative counter-action;

[0154] Determine the instruction execution type and the target type based on the collaborative defense action and / or collaborative counter-action performed by the agent, wherein the target type is a type predefined by the attacker and / or the counter-target;

[0155] The environment evaluation parameter is determined based on the weights corresponding to the network environment parameter, the instruction execution type and the target type.

[0156] Optionally, the processor 1001 may call a network attack and defense training program based on multi-agent collaboration stored in the memory 1005, and further perform the following operations:

[0157] Obtaining a preset execution evaluation function, wherein the execution evaluation function corresponds to a defense execution result and a counterattack execution result respectively;

[0158] Determine the defense execution result based on the corresponding defense frequency ratio, defense efficiency ratio and defense action execution efficiency when the intelligent agent executes the cooperative defense action; or

[0159] Determine the counter-action execution result based on the counter-action success rate, the degree of damage and the execution efficiency of the counter-action corresponding to the execution of the collaborative counter-action by the agent;

[0160] The execution evaluation parameter is determined based on the defense execution result and / or counter-execution result corresponding to the attack and defense instruction executed by the agent, and the execution evaluation function.

[0161] Optionally, the processor 1001 may call a network attack and defense training program based on multi-agent collaboration stored in the memory 1005, and further perform the following operations:

[0162] Obtaining a corresponding relationship between the environmental evaluation parameter and the strategy space corresponding to the agent;

[0163] Based on the execution evaluation parameter, the policy space corresponding to the environmental evaluation parameter is updated.

[0164] In addition, an embodiment of the present application also proposes a computer-readable storage medium, on which a network attack and defense training program based on multi-agent collaboration is stored. When the network attack and defense training program based on multi-agent collaboration is executed by a processor, the relevant steps of any embodiment of the network attack and defense training method based on multi-agent collaboration as described above are implemented.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0169] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present application may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0170] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0171] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A network attack and defense training method based on multi-agent collaboration, characterized in that: The network attack and defense training method based on multi-agent collaboration includes: Upon receiving an attack and defense instruction, determining at least two of the intelligent agents according to the attack and defense instruction; When the attack and defense instruction is a collaborative defense, determine the first intelligent agent attacked among the intelligent agents, and the second intelligent agent used for collaborative defense; determine the collaborative defense strategy according to the attack type and attack parameters of the attacker, and the type of device where the first intelligent agent is located; determine the defense action and execution order performed by each of the intelligent agents based on the collaborative defense strategy, so as to control the intelligent agents to perform collaborative defense actions, wherein the intelligent agents share attack information, defense strategies and execution status to ensure that the execution of each defense action can be effectively coordinated; if the attack and defense instruction is a counter-attack instruction, the intelligent agent partially executes the collaborative defense action according to the received counter-attack instruction, and the other part executes the collaborative counter-action against the attacker, wherein the intelligent agents share the attack information, counter-attack strategy and execution status to ensure that the execution of each counter-action can be effectively coordinated; Determining network environment parameters based on network fluctuations, network speed averages, and delay averages of the agent during execution of the collaborative defense action and / or collaborative counteraction action; Determine the instruction execution type and the target type based on the collaborative defense action and / or collaborative counter-action performed by the agent, wherein the target type is a type predefined by the attacker and / or the counter-target; Determining an environment evaluation parameter based on the network environment parameter, the instruction execution type, and the weights corresponding to the target type; Determining an execution evaluation parameter of the agent for the attack and defense instruction based on the execution result of the attack and defense instruction; Obtaining a corresponding relationship between the environmental evaluation parameter and the strategy space corresponding to the agent; Based on the execution evaluation parameter, the policy space corresponding to the environmental evaluation parameter is updated.

2. The network attack and defense training method based on multi-agent collaboration as claimed in claim 1, characterized in that: After the step of determining at least two of the intelligent agents according to the attack and defense instructions upon receiving the attack and defense instructions, the method further comprises: When the attack and defense instruction is a collaborative counterattack, the counterattack action and the execution order to be performed by each of the intelligent agents are determined based on the attack and defense instruction, and the collaborative counterattack action is performed.

3. The network attack and defense training method based on multi-agent collaboration as claimed in claim 1, characterized in that: After the step of determining the collaborative defense strategy according to the attack type and attack parameters of the attacker and the type of device where the first agent is located, the method further includes: When a change is detected in the first agent and / or the attacking party, determining a type of change; The collaborative defense strategy is updated based on the change type.

4. The network attack and defense training method based on multi-agent collaboration as claimed in claim 2, characterized in that: When the attack and defense instruction is a collaborative counterattack, the counterattack action and the execution order performed by each of the agents are determined based on the attack and defense instruction, and the step of performing the collaborative counterattack action includes: When the attack and defense instruction is a collaborative counterattack, determining a counterattack target based on the attack and defense instruction; Based on the countermeasure target and the preset behavior strategy of the agent, determining the countermeasure action and the execution order corresponding to each agent; According to the encryption algorithm and execution path corresponding to the counter-action, the agent is controlled to execute the collaborative counter-action at the time point corresponding to the execution sequence.

5. The network attack and defense training method based on multi-agent collaboration as claimed in claim 1, characterized in that: The step of determining the execution evaluation parameter of the agent for the attack and defense instruction based on the execution result of the attack and defense instruction comprises: Obtaining a preset execution evaluation function, wherein the execution evaluation function corresponds to a defense execution result and a counterattack execution result respectively; Determine the defense execution result based on the corresponding defense frequency ratio, defense efficiency ratio and defense action execution efficiency when the intelligent agent executes the cooperative defense action; or Determine the counter-action execution result based on the counter-action success rate, the degree of damage and the execution efficiency of the counter-action corresponding to the execution of the collaborative counter-action by the agent; The execution evaluation parameter is determined based on the defense execution result and / or counter-execution result corresponding to the attack and defense instruction executed by the agent, and the execution evaluation function.

6. A network attack and defense training device based on multi-agent collaboration, characterized in that: It includes a memory, a processor, and a network attack and defense training program based on multi-agent collaboration that is stored in the memory and can be run on the processor. When the processor executes the network attack and defense training program based on multi-agent collaboration, it implements the steps of the network attack and defense training method based on multi-agent collaboration as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a network attack and defense training program based on multi-agent collaboration, and when the network attack and defense training program based on multi-agent collaboration is executed by the processor, the steps of the network attack and defense training method based on multi-agent collaboration as described in any one of claims 1 to 5 are implemented.

Images