A network attack and defense simulation method and simulation system based on reinforcement learning

Abstract

The application provides a network attack and defense simulation method and simulation system based on reinforcement learning, which comprises the following steps: generating a simulation network topology structure according to the attack and defense requirements of a network attack and defense to be simulated, and establishing a network attack and defense environment model according to the network environment; the simulation network topology structure comprises the node attributes of the topology nodes of the simulation network topology structure; obtaining corresponding attack and defense algorithm models according to the simulation network topology structure and the network attack and defense environment model; obtaining a network scene graph through rendering according to the simulation network topology structure and the network attack and defense environment model, obtaining the action information of intelligent agents participating in the attack and defense simulation according to the attack and defense algorithm models, and displaying the action information in the network scene graph. By presetting a plurality of different attack and defense algorithm models, the corresponding attack and defense algorithm model can be quickly and accurately selected according to the actual attack and defense simulation requirements, and different attack and defense simulations can be realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of machine learning, and particularly relates to a network attack and defense simulation method and simulation system based on reinforcement learning. BACKGROUND

[0002] In today's highly digitized and interconnected information age, network security is facing unprecedented complexity and severity of challenges. Network attack means continues to evolve, not only frequently updated but also increasingly sophisticated, which requires network defense strategies not only to have high flexibility and adaptability, but also to be able to respond quickly to unknown threats. Traditional network security defense measures, such as firewalls, intrusion detection systems, etc., rely on pre-set rules or known attack features for protection, and often struggle to cope with new or complex attack schemes, which are easily bypassed or ineffective.

[0003] Based on the dynamic network attack and defense simulation environment, reinforcement learning emerges as the times require and shows great potential in the field of network security. Reinforcement learning, a branch of machine learning that focuses on decision-making and optimization, learns the best action strategy through interaction with the environment, providing a new way to adapt to the ever-changing field of network security. Applied to network security defense, reinforcement learning can iteratively optimize strategies through real-time feedback to identify and respond to emerging threats.

[0004] Currently, although compared with traditional network security means, the development of network attack and defense strategies based on reinforcement learning is still in its infancy, but it has shown obvious progress and potential. Especially the reinforcement learning-oriented network attack and defense strategy training platform, which establishes a simulated network attack and defense environment, enables network security researchers and practitioners to conduct iterative testing and optimization of attack and defense strategies in a safe virtual environment. Such a training platform can simulate network attack and defense behaviors in the real world on the one hand, and improve the ability to respond to unknown attacks through continuous learning on the other hand, while also helping to discover potential vulnerabilities in network systems and provide data support for security engineering. SUMMARY

[0005] The main purpose of the embodiments of the present application is to provide a network attack and defense simulation method and simulation system based on reinforcement learning, which can quickly and accurately select the corresponding attack and defense algorithm model according to the actual attack and defense simulation needs by pre-setting multiple different attack and defense algorithm models, and realize different attack and defense simulations.

[0006] In a first aspect, a network attack and defense simulation method based on reinforcement learning is provided, the simulation method comprising:

[0007] generate a simulation network topology according to attack and defense requirements of a network attack and defense to be simulated, and establish a network attack and defense environment model according to the network environment, the simulation network topology comprising: node attributes of topological nodes of the simulation network topology;

[0008] obtain a corresponding attack and defense algorithm model according to the simulation network topology and the network attack and defense environment model;

[0009] obtain network scene graphs through rendering according to the simulation network topology and the network attack and defense environment model, obtain action information of agents participating in the attack and defense simulation according to the attack and defense algorithm model, and display the action information in the network scene graphs.

[0010] In one possible implementation, the simulation method further comprises:

[0011] obtain a simulation result of the attack and defense simulation.

[0012] In another possible implementation, the generating a simulation network topology according to attack and defense requirements of a network attack and defense to be simulated comprises:

[0013] obtain attack and defense data corresponding to the attack and defense requirements, the attack and defense data comprising: text type parameter data, table type parameter data;

[0014] generate a simulation network topology according to the attack and defense data.

[0015] In another possible implementation, the obtaining a corresponding attack and defense algorithm model according to the simulation network topology and the network attack and defense environment model comprises:

[0016] obtaining a corresponding first trained algorithm model according to the simulation network topology and the network attack and defense environment model; or

[0017] obtaining a corresponding untrained algorithm model according to the simulation network topology and the network attack and defense environment model, training the training algorithm model, and obtaining a second trained algorithm model.

[0018] In a second aspect, a network attack and defense simulation system based on reinforcement learning is provided, and the simulation system comprises:

[0019] a topological structure and environment model construction module, configured to generate a simulation network topology according to attack and defense requirements of a network attack and defense to be simulated, and establish a network attack and defense environment model according to the network environment, the simulation network topology comprising: node attributes of topological nodes of the simulation network topology;

[0020] an attack and defense algorithm model obtaining module, configured to obtain a corresponding attack and defense algorithm model according to the simulation network topology and the network attack and defense environment model;

[0021] a display module, configured to acquire a network scene graph by rendering according to the simulation network topology and the network attack and defense environment model, acquire action information of an agent participating in the attack and defense simulation according to the attack and defense algorithm model, and display the action information in the network scene graph.

[0022] In a possible implementation, the simulation system further includes:

[0023] a simulation result acquisition module, configured to acquire a simulation result of the attack and defense simulation.

[0024] In another possible implementation, the generating of the simulation network topology according to the attack and defense requirement of the network attack and defense to be simulated includes:

[0025] acquiring attack and defense data corresponding to the attack and defense requirement, the attack and defense data including text type parameter data and table type parameter data;

[0026] generating the simulation network topology according to the attack and defense data.

[0027] In another possible implementation, the acquiring of the corresponding attack and defense algorithm model according to the simulation network topology and the network attack and defense environment model includes:

[0028] acquiring a first trained algorithm model corresponding to the simulation network topology and the network attack and defense environment model; or

[0029] acquiring an untrained algorithm model corresponding to the simulation network topology and the network attack and defense environment model, training the trained algorithm model, and acquiring a second trained algorithm model.

[0030] In a third aspect, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the network attack and defense simulation method based on reinforcement learning provided in the first aspect when executing the program.

[0031] In a fourth aspect, a non-transitory computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the network attack and defense simulation method based on reinforcement learning provided in the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the description of the embodiments of the present application will be briefly introduced.

[0033] Figure 1 a flowchart of the network attack and defense simulation method based on reinforcement learning provided in the first embodiment of the present application;

[0034] Figure 2 A structural diagram of a network attack and defense simulation system based on reinforcement learning provided for a second embodiment of the present application is provided.

[0035] Figure 3 An entity structure schematic diagram of an electronic device provided for the present application is provided.

[0036] Detailed Implementation

[0037] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present disclosure.

[0038] The embodiments of the present disclosure provide a network attack and defense simulation method and simulation system based on reinforcement learning. Specifically, a network attack and defense simulation method based on reinforcement learning of the embodiments of the present disclosure can be executed by a computer device, which can be a terminal or a server and the like. The terminal can be a terminal device such as a smart phone, a tablet computer, a notebook computer, a touch screen, a game console, a personal computer (PC, Personal Computer), a personal digital assistant (PDA), and the like. The terminal can also include a client, which can be an application client, a browser client carrying an application program, or an instant messaging client, and the like. The server can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms.

[0039] For example, when the network attack and defense simulation method based on reinforcement learning is run on a terminal, the terminal device stores an application and is used to present an operation interface or an application page provided by the application. The terminal device is used to interact with the user through a graphical user interface, a merchant page or a page associated in other application programs, for example, by downloading and installing the application on the terminal device and running. The terminal device can provide the graphical user interface, the merchant page or the page associated in other application programs to the user in various ways, for example, can render and display on the display screen of the terminal device, or present the graphical user interface through holographic projection. For example, the terminal device can include a touch display screen and a processor, the touch display screen is used to present the graphical user interface and receive the operation instruction generated by the user acting on the graphical user interface, and the processor is used to run the application, generate the graphical user interface, respond to the operation instruction and control the display of the graphical user interface on the touch display screen.

[0040] The embodiment of the present disclosure provides a network attack and defense simulation method and simulation system based on reinforcement learning, which can be executed by a terminal or a server. The embodiment of the present disclosure takes a network attack and defense simulation method and simulation system based on reinforcement learning executed by a terminal as an example for illustration. The terminal can include a touch display screen and a processor (of course, the terminal can also use a mouse, a keyboard and other peripherals as input devices, which are only taken as an example of the touch display screen here), and the touch display screen is used to present a graphical user interface and receive an operation instruction generated by a user acting on the graphical user interface. When the user operates the graphical user interface through the touch display screen, the graphical user interface can control the content of the terminal locally by responding to the received operation instruction, or control the content of the server at the other end by responding to the received operation instruction. For example, the operation instruction generated by the user acting on the graphical user interface includes an instruction for starting an application, and the processor is configured to start the application after receiving the instruction for starting the application provided by the user. In addition, the processor is configured to render and draw the graphical user interface associated with the application on the touch display screen. The touch display screen is a multi-touch screen capable of sensing multiple points on the screen simultaneously performing touch or sliding operations. When the graphical user interface detects the touch operation of the user using the finger on the graphical user interface, the graphical user interface controls the display of the action corresponding to the touch operation in the graphical user interface.

[0041] As Figure 1 Fig. 1 shows a flowchart of a network attack and defense simulation method based on reinforcement learning provided by an embodiment of the present disclosure, and the simulation method includes:

[0042] In step 101, a simulation network topology structure is generated according to the attack and defense requirements of the network attack and defense to be simulated, and a network attack and defense environment model is established according to the network environment, wherein the simulation network topology structure includes node attributes of topological nodes of the simulation network topology structure.

[0043] In this step, the simulation network topology structure is generated based on the requirements of the user participating in the network attack and defense simulation and the network environment structure requirements, and the topological nodes of the simulation network topology structure are configured with attributes based on this, and then the network attack and defense environment model is established. Thus, a basis is provided for subsequent attack and defense game training and visualization work, and at the same time, the parameters related to the network environment are also provided with the services of extraction, conversion and analysis of the corresponding data of the parameters.

[0044] The simulation network topology structure is generated according to the attack and defense requirements of the network attack and defense to be simulated, including:

[0045] The attack and defense data corresponding to the attack and defense requirements are obtained, and the attack and defense data includes text type parameter data and table type parameter data.

[0046] The simulation network topology structure is generated according to the attack and defense data.

[0047] In this embodiment, the attack and defense data includes text type parameter data and table type parameter data, and the source of the attack and defense data can be local stored attack and defense data or manually input attack and defense data of the user.

[0048] In step 102, the corresponding attack and defense algorithm model is obtained according to the simulation network topology structure and the network attack and defense environment model.

[0049] In this step, the simulation network topology structure and the network attack and defense environment model define the related parameters required for this attack and defense simulation, and the corresponding attack and defense algorithm model is obtained according to the related parameters.

[0050] The corresponding attack and defense algorithm model is obtained according to the simulation network topology structure and the network attack and defense environment model, including:

[0051] The first trained algorithm model corresponding to the simulation network topology structure and the network attack and defense environment model is obtained; or

[0052] The untrained algorithm model corresponding to the simulation network topology structure and the network attack and defense environment model is obtained, the trained algorithm model is trained, and the second trained algorithm model is obtained.

[0053] In the embodiment of the present application, a plurality of different algorithms are stored in the local database, including trained algorithm models and untrained models. In order to quickly form an attack and defense simulation, a first trained algorithm model is preferentially selected for attack and defense simulation. If there is no corresponding trained algorithm model, a corresponding algorithm model is selected from the untrained algorithm models, the algorithm model is trained to obtain a second trained algorithm model, and the attack and defense simulation is performed through the second trained algorithm model.

[0054] Specifically, a GUI interface and an interface calling interface are used to provide an interface for adjusting training parameters for different specific tasks in different scenarios, so that the user can select the action mode and strategy of the attack and defense agent in the network environment set by the user, and the algorithm training and learning parameters can be modified to achieve the required training intensity of the user and complete the experimental purpose.

[0055] In step 103, the network scene graph is obtained by rendering according to the simulation network topology and the network attack and defense environment model, the action information of the agent participating in the attack and defense simulation is obtained according to the attack and defense algorithm model, and the action information is displayed in the network scene graph.

[0056] In this step, for the simulation network topology and the network attack and defense environment model, and the selected attack and defense algorithm model, the corresponding data needs to be displayed in an intuitive way, that is, the data is visualized. Specifically, the simulation network topology and the network attack and defense environment model are rendered into a corresponding network scene graph, the action information of the attack agent and the defense agent participating in the attack and defense simulation is obtained according to the attack and defense algorithm model, and the action information is displayed in the network scene graph.

[0057] In the embodiment of the present application, the simulation network topology is generated according to the attack and defense requirements of the network attack and defense to be simulated, and the network attack and defense environment model is established according to the network environment. The simulation network topology includes: node attributes of the topology nodes of the simulation network topology; the corresponding attack and defense algorithm model is obtained according to the simulation network topology and the network attack and defense environment model; the network scene graph is obtained by rendering according to the simulation network topology and the network attack and defense environment model, the action information of the agent participating in the attack and defense simulation is obtained according to the attack and defense algorithm model, and the action information is displayed in the network scene graph. By presetting a plurality of different attack and defense algorithm models, the corresponding attack and defense algorithm model can be quickly and accurately selected according to the actual attack and defense simulation requirements, and different attack and defense simulations can be realized.

[0058] As an optional embodiment of the present application, the simulation method further comprises:

[0059] The simulation result of the attack and defense simulation is obtained.

[0060] As Figure 2 Fig. 1 shows a structure diagram of a network attack and defense simulation system based on reinforcement learning provided by an embodiment of the present application, and the simulation system comprises:

[0061] A topology structure and environment model construction module 201 is configured to generate a simulation network topology structure according to attack and defense requirements of a network attack and defense to be simulated, and to establish a network attack and defense environment model according to the network environment, wherein the simulation network topology structure comprises node attributes of topology nodes of the simulation network topology structure.

[0062] In this step, a simulation network topology structure is generated based on requirements of a user participating in network attack and defense simulation and network environment structure requirements, and on this basis, attributes of topology nodes of the simulation network topology structure are configured, and a network attack and defense environment model is established. Thus, a basis support is provided for subsequent attack and defense game training and visualization work, and at the same time, services of extraction, conversion and analysis of data corresponding to parameters related to the network environment are provided.

[0063] The attack and defense requirements comprise at least one of the following: a network attack and defense scenario, a network attack and defense parameter, and a network attack and defense environment.

[0064] The attack and defense data comprises text type parameter data and table type parameter data.

[0065] The simulation network topology structure is generated according to the attack and defense data.

[0066] In this embodiment, the attack and defense data comprises text type parameter data and table type parameter data, and the source of the attack and defense data can be attack and defense data already stored locally or attack and defense data manually input by a user.

[0067] An attack and defense algorithm model acquisition module 202 is configured to acquire a corresponding attack and defense algorithm model according to the simulation network topology structure and the network attack and defense environment model.

[0068] In this step, the simulation network topology structure and the network attack and defense environment model define related parameters required for this attack and defense simulation, and a corresponding attack and defense algorithm model is acquired according to the related parameters.

[0069] The attack and defense algorithm model acquisition module 202 is configured to acquire a corresponding attack and defense algorithm model according to the simulation network topology structure and the network attack and defense environment model.

[0070] A first trained algorithm model corresponding to the simulation network topology structure and the network attack and defense environment model is acquired; or

[0071] According to the simulation network topology and the network attack and defense environment model, a corresponding untrained algorithm model is acquired, the trained algorithm model is trained, and a second trained algorithm model is acquired.

[0072] In the embodiment of the application, a plurality of different algorithms are stored in a local database, including trained algorithm models and untrained models.

[0073] The specific implementation is that a GUI interface and an interface calling interface are used to provide an interface for adjusting training parameters for different specific tasks in different scenarios, so that a user can select the action mode and strategy of an attack and defense intelligent agent in a network environment that has been set, and the algorithm training and learning parameters can be modified to achieve different training intensities required by the user and complete the experimental purpose.

[0074] The display module 203 is configured to acquire a network scene graph through rendering according to the simulation network topology and the network attack and defense environment model, acquire action information of intelligent agents participating in the attack and defense simulation according to the attack and defense algorithm model, and display the action information in the network scene graph.

[0075] In this step, for the simulation network topology and the network attack and defense environment model, and the selected attack and defense algorithm model, the corresponding data needs to be displayed in an intuitive way, that is, the data needs to be visualized.

[0076] In the embodiment of the application, a simulation network topology is generated according to the attack and defense requirements of a network attack and defense to be simulated, a network attack and defense environment model is established according to the network environment, the simulation network topology includes node attributes of topological nodes of the simulation network topology, an attack and defense algorithm model is acquired according to the simulation network topology and the network attack and defense environment model, a network scene graph is acquired through rendering according to the simulation network topology and the network attack and defense environment model, action information of intelligent agents participating in the attack and defense simulation is acquired according to the attack and defense algorithm model, and the action information is displayed in the network scene graph.

[0077] As an optional embodiment of the present invention, the simulation system further includes:

[0078] The simulation result acquisition module is used to acquire the simulation results of the attack and defense simulation.

[0079] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor, communication interface, and memory communicate with each other through the communication bus. The processor can call logical instructions in the memory to execute a network attack and defense simulation method based on reinforcement learning. This method includes: generating a simulated network topology structure according to the attack and defense requirements of the network to be simulated, and establishing a network attack and defense environment model according to the network environment. The simulated network topology structure includes: node attributes of the topology nodes of the simulated network topology structure; obtaining a corresponding attack and defense algorithm model according to the simulated network topology structure and the network attack and defense environment model; obtaining a network scene graph through rendering according to the simulated network topology structure and the network attack and defense environment model; obtaining the action information of the intelligent agents participating in the attack and defense simulation according to the attack and defense algorithm model; and displaying the action information in the network scene graph.

[0080] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] In another aspect, the embodiments of the present application also provide a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, and the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the network attack and defense simulation method based on reinforcement learning provided by the above-mentioned method embodiments, and the method comprises the following steps: generating a simulation network topology according to attack and defense requirements of a network attack and defense to be simulated, and establishing a network attack and defense environment model according to the network environment, the simulation network topology comprises the following: node attributes of topological nodes of the simulation network topology; obtaining corresponding attack and defense algorithm models according to the simulation network topology and the network attack and defense environment model; obtaining a network scene graph through rendering according to the simulation network topology and the network attack and defense environment model, obtaining action information of an agent participating in the attack and defense simulation according to the attack and defense algorithm models, and displaying the action information in the network scene graph.

[0082] In another aspect, the embodiments of the present application also provide a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the network attack and defense simulation method based on reinforcement learning provided by the above-mentioned embodiments, and the method comprises the following steps: generating a simulation network topology according to attack and defense requirements of a network attack and defense to be simulated, and establishing a network attack and defense environment model according to the network environment, the simulation network topology comprises the following: node attributes of topological nodes of the simulation network topology; obtaining corresponding attack and defense algorithm models according to the simulation network topology and the network attack and defense environment model; obtaining a network scene graph through rendering according to the simulation network topology and the network attack and defense environment model, obtaining action information of an agent participating in the attack and defense simulation according to the attack and defense algorithm models, and displaying the action information in the network scene graph.

[0083] It should be understood that, although each step in the flowchart of the accompanying drawings is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other sequences. Moreover, at least part of the steps in the flowchart of the accompanying drawings can comprise multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps.

[0084] The above only describes some implementation manners of the present application, and it should be noted that, for those skilled in the art, without departing from the principle of the present application, some improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A network attack and defense simulation method based on reinforcement learning, characterized in that, The simulation method includes: A simulated network topology is generated based on the attack and defense requirements of the network to be simulated, and a network attack and defense environment model is established based on the network environment. The simulated network topology includes the node attributes of the topology nodes of the simulated network topology. Based on the simulated network topology and network attack and defense environment model, obtain the corresponding attack and defense algorithm model; Based on the simulated network topology and network attack and defense environment model, a network scene graph is obtained by rendering. Based on the attack and defense algorithm model, the action information of the intelligent agents participating in the attack and defense simulation is obtained, and the action information is displayed in the network scene graph. Obtain the simulation results of the attack and defense simulation; The step of obtaining the corresponding attack and defense algorithm model based on the simulated network topology and network attack and defense environment model includes: The first trained algorithm model is obtained based on the simulated network topology and network attack / defense environment model; or... Based on the simulated network topology and network attack and defense environment model, obtain the corresponding untrained algorithm model, train the trained algorithm model, and obtain the second trained algorithm model.

2. The simulation method as described in claim 1, characterized in that, The process of generating a simulated network topology based on the attack and defense requirements of the network to be simulated includes: Obtain the attack and defense data corresponding to the attack and defense requirements. The attack and defense data includes: text-type parameter data and tabular parameter data. A simulated network topology is generated based on the attack and defense data.

3. A network attack and defense simulation system based on reinforcement learning, characterized in that, The simulation system includes: The topology and environment model building module is used to generate a simulated network topology based on the attack and defense requirements of the network to be simulated, and to establish a network attack and defense environment model based on the network environment. The simulated network topology includes the node attributes of the topology nodes of the simulated network topology. The attack and defense algorithm model acquisition module is used to acquire corresponding attack and defense algorithm models based on the simulated network topology and network attack and defense environment model. Acquiring the corresponding attack and defense algorithm model based on the simulated network topology and network attack and defense environment model includes: acquiring a first trained algorithm model based on the simulated network topology and network attack and defense environment model; or, acquiring a non-trained algorithm model based on the simulated network topology and network attack and defense environment model, training the trained algorithm model, and acquiring a second trained algorithm model. The display module is used to obtain a network scene graph by rendering based on the simulated network topology and network attack and defense environment model, obtain the action information of the intelligent agents participating in the attack and defense simulation based on the attack and defense algorithm model, and display the action information in the network scene graph. The simulation result acquisition module is used to acquire the simulation results of the attack and defense simulation.

4. The simulation system as described in any one of claims 3, characterized in that, The process of generating a simulated network topology based on the attack and defense requirements of the network to be simulated includes: Obtain the attack and defense data corresponding to the attack and defense requirements. The attack and defense data includes: text-type parameter data and tabular parameter data. A simulated network topology is generated based on the attack and defense data.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the network attack and defense simulation method based on reinforcement learning as described in any one of claims 1-2.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the network attack and defense simulation method based on reinforcement learning as described in any one of claims 1-2.

Images