A simulation training method, device and equipment for land combat unmanned equipment

Abstract

The present invention discloses a simulation training method, device and equipment for land combat unmanned equipment, comprising: creating a land combat confrontation scenario of multi-agent unmanned equipment, defining the scenario information, observation space, action space, winning and losing conditions and reward function of the red party agent and the blue party agent in the land combat confrontation scenario; constructing a network training framework for multi-agent reinforcement learning, including a MAPPO network and a QMIX network, to obtain a multi-agent reinforcement learning model; wherein the MAPPO network is used to provide instant feedback and gradient information for strategy adjustment; the QMIX network is used to achieve collaboration between multi-agents and evaluation of global strategies; and iteratively training the multi-agent reinforcement learning model by initializing and updating the network parameters of the MAPPO network and the QMIX network, until the training is stopped when preset conditions are met, to obtain a multi-agent collaboration system.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-agent reinforcement learning, and in particular to a simulation training method, device and equipment for land combat unmanned equipment. Background Art

[0002] In the confrontational scenes of urban land warfare, the coordinated combat capability of unmanned equipment (such as drones, unmanned vehicles, etc.) is crucial. The urban terrain is complex and changeable, including high-rise buildings, narrow streets, open squares, etc. These terrain features pose severe challenges to the mobility and concealment of unmanned equipment, as well as the field of vision and firepower coverage in combat. In addition, the densely distributed buildings and obstacles in the city further restrict the passage of unmanned equipment, increasing the difficulty of combat.

[0003] What is more complicated is that the battlefield environment in urban confrontation scenarios is in dynamic change, such as building collapse, street closure, enemy tactics and firepower deployment changes, etc. These unpredictable changes require unmanned equipment to have a high degree of adaptability and flexibility. At the same time, unmanned equipment needs to perform a wide variety of tasks, such as reconnaissance, strike, cover, etc., and there may be interdependence and conflict between these tasks, which requires efficient collaboration between intelligent agents.

[0004] However, existing unmanned equipment multi-agent collaborative training has obvious shortcomings in facing these challenges. Especially in the problem of credit allocation, collaborative tasks often involve close cooperation between multiple agents, and the contribution of each agent to the overall performance is difficult to directly quantify. Therefore, how to fairly and accurately allocate credit (i.e., evaluate the contribution of each agent to the overall performance) is crucial to improving the training efficiency and effectiveness of collaborative strategies. In addition, high-dimensional state space, dynamically changing environment, multi-task learning and strategy diversity are also technical problems that need to be solved urgently. Summary of the invention

[0005] In view of this, the purpose of the present invention is to propose a simulation training method, device and equipment for land warfare unmanned equipment, aiming to solve how to effectively solve the credit allocation problem in multi-agent training in urban land warfare confrontation scenarios, optimize the collaboration and interaction relationship between multiple agents, and improve learning efficiency.

[0006] To achieve the above object, the present invention provides a simulation training method for land combat unmanned equipment, the method comprising:

[0007] Create a land combat confrontation scenario of multi-agent unmanned equipment, and define the scenario information, observation space, action space, winning and losing conditions, and reward function of the red agent and the blue agent in the land combat confrontation scenario; the scenario information includes the number of entities and initial positions of the red agent and the blue agent; the observation space includes the local observation space and the global observation space of the red agent; the action space includes the continuous actions and discrete actions of the red agent, and the reward function includes the process reward and the final reward;

[0008] Construct a network training framework for multi-agent reinforcement learning, including MAPPO network and QMIX network, to obtain a multi-agent reinforcement learning model; the MAPPO network is used to provide instant feedback and gradient information for strategy adjustment; the QMIX network is used to achieve collaboration between multiple agents and evaluation of global strategies;

[0009] By initializing and updating the network parameters of the MAPPO network and the QMIX network, the multi-agent reinforcement learning model is iteratively trained until the preset conditions are met and the training is stopped to obtain a multi-agent collaborative system.

[0010] Preferably, the local observation space includes the type, name, longitude, latitude, altitude, speed, direction, maneuverable direction, attackable target, and survival status of the Red Agent;

[0011] The global observation space includes the local observation space of the red agent, all the information of the blue agent and the environmental information; wherein all the information of the blue agent includes the type, name, longitude, latitude, altitude, speed, remaining ammunition quantity, and survival status; the environmental information includes the location of obstacles, the relative distance between the red and blue parties, and the relative distance between the red agent and the friendly party;

[0012] The action space is designed by using the local observation space of the Red Agent and predefined actions;

[0013] The winning and losing conditions include:

[0014] If the Red Agent or the Blue Agent loses 70% of its manpower, it is considered defeated.

[0015] If the distance between the Red Agent and the farthest Blue Agent is less than the preset distance threshold, and the Blue Agent loses 50% of its manpower, the Red Agent is considered the winner;

[0016] If the duration of the confrontation reaches the set maximum value, the red agent is deemed defeated.

[0017] Preferably, the MAPPO network is constructed based on an actor-critic architecture, including a policy network and a value network, wherein the policy network includes an input layer, a hidden layer, and an output layer, wherein:

[0018] The state vector of the land battle scene is received through the input layer. The state vector includes the local observation space of all red team agents.

[0019] In the hidden layer, a local state feature in a state vector is captured by a first hidden layer, and a feature is extracted and dimension is reduced by a second hidden layer, and a recurrent neural network is introduced in the second hidden layer to capture time dependency;

[0020] Output the decision made by the Red Agent in the current state through the output layer;

[0021] The value network includes an input layer, a hidden layer and an output layer, wherein:

[0022] Receive the global observation space through the input layer;

[0023] The global state features of the global observation space are extracted through the hidden layer, and the QMIX network is introduced into the hidden layer to capture the collaboration and interaction relationship between multiple agents;

[0024] The value estimate of the current state is output through the output layer.

[0025] Preferably, the QMIX network includes an agent network, a super network and a hybrid network, the agent network is constructed based on a GRU recurrent neural network, the super network is constructed using a multi-layer perceptron, and the hybrid network is constructed using a sampled feedforward neural network; wherein,

[0026] Inputting the local observation space and current action of the Red Agent into the Agent Network, and outputting the local Q-value function of the Red Agent;

[0027] After the global observation space is input into the hypernetwork for preprocessing, it is linearly transformed by MLP and nonlinearly transformed by activation function to output a global vector;

[0028] The local Q value function is input into the hybrid network, and the global vector is used as the weight of the hybrid network, and the global Q value function is output.

[0029] Preferably, the initializing and updating the network parameters of the MAPPO network and the QMIX network includes:

[0030] Network parameters , , , And the memory pool D is initialized;

[0031] Each Red Agent executes the policy function corresponding to the policy network The generated action gets the reward value and , , , Stored in memory pool D;

[0032] Every In each cycle, group B data is selected from memory pool D to calculate the gradient. and gradient , and through the gradient and gradient The network parameters of the neural network and network parameters to update.

[0033] Preferably, the In each cycle, group B data is selected from memory pool D to calculate the gradient. and gradient ,include:

[0034] pass Calculate and obtain the network parameters of the actor network corresponding to the i-th red agent gradient ,in,

[0035] ,

[0036] ,

[0037] ,

[0038] ,

[0039] ,

[0040] ,

[0041] In the formula, represents the gradient operator, B represents the number of samples selected for one training, represents the advantage function of the i-th red agent, Used to describe the difference between the new and old control strategies, represents a small positive number, represents the updated policy function of the i-th red agent, represents the current policy function of the i-th red agent, Represents the critic network of the i-th red agent based on the current parameters The output value of represents the discount factor, which is a number between 0 and 1 and is used to calculate the current value of future rewards. rt represents the reward value of immediate feedback, and T is the time step. represents the state value function;

[0042] pass Calculate and get the network parameters of the critic network corresponding to the i-th red agent gradient ;

[0043] in, represents the joint action of all red agents, , represents the action value function of the i-th red agent, which is the output of the agent network. is a nonlinear function used to combine individual action-value functions into a joint action-value function, i.e., a feedforward neural network. Represents the output of the QMIX network: the joint action-value function.

[0044] Preferably, the gradient and gradient The network parameters of the neural network and Updates include:

[0045] Update the network parameters of the actor network by using the gradient ascent method , get the updated network parameters ;

[0046] Update the network parameters of the critic network by using the gradient descent method , get the updated network parameters ,in,

[0047] ,

[0048] ,

[0049] In the formula, represents the learning rate of the actor network, represents the learning rate of the critic network, Represents the weight parameters of the value network before updating.

[0050] To achieve the above object, the present invention further provides a simulation training device for land combat unmanned equipment, the device comprising:

[0051] A scenario creation unit is used to create a land combat confrontation scenario of multi-agent unmanned equipment, and define the scenario information, observation space, action space, winning and losing conditions, and reward function of the red agent and the blue agent in the land combat confrontation scenario; wherein the scenario information includes the number of entities and initial positions of the red agent and the blue agent; the observation space includes the local observation space and the global observation space of the red agent; the action space includes the continuous actions and discrete actions of the red agent, and the reward function includes the process reward and the final reward;

[0052] The framework construction unit is used to construct a network training framework for multi-agent reinforcement learning, including the MAPPO network and the QMIX network, to obtain a multi-agent reinforcement learning model; the MAPPO network is used to provide instant feedback and gradient information for strategy adjustment; the QMIX network is used to achieve collaboration between multiple agents and evaluation of global strategies;

[0053] The model training unit is used to iteratively train the multi-agent reinforcement learning model by initializing and updating the network parameters of the MAPPO network and the QMIX network, and stop the training when the preset conditions are met to obtain a multi-agent collaborative system.

[0054] In order to achieve the above-mentioned objectives, the present invention also proposes a simulation training device for land combat unmanned equipment, comprising a processor, a memory and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a simulation training method for land combat unmanned equipment as described in the above-mentioned embodiment.

[0055] In order to achieve the above-mentioned purpose, the present invention also proposes a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of a simulation training method for land combat unmanned equipment as described in the above embodiment.

[0056] In order to achieve the above-mentioned purpose, the present invention also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of a simulation training method for land combat unmanned equipment as described in the above-mentioned embodiment.

[0057] Beneficial effects:

[0058] The above scheme, by constructing a network training framework for multi-agent reinforcement learning and introducing the QMIX network in the MAPPO network, can more comprehensively consider the mutual influence and dependency between agents, optimize the collaboration and interaction between multiple agents, and improve the coordinated combat capability and adaptability of unmanned equipment in complex urban terrain, thereby improving the performance of the overall system.

[0059] The above scheme, the definition of local observation space and global observation space and the detailed action space design, enable the intelligent agent to understand the current environmental state more comprehensively and make accurate decisions, which enhances the adaptability of unmanned equipment to complex and changing environments and improves the overall combat effectiveness; by defining clear winning and losing conditions and reward functions, and adopting efficient training algorithms and network structures, the training efficiency and effect are significantly improved.

[0060] In the above scheme, the MAPPO network is built based on the actor-critic architecture, which can provide instant feedback and gradient information for strategy adjustment, helping the agent to quickly adapt to environmental changes during training, optimize strategies, and improve learning efficiency. By introducing the QMIX network into the MAPPO network, which includes the agent network, super network, and hybrid network, the effective fusion of the local action value function and the global action value function is achieved, which helps to optimize the collaboration and interaction between multiple agents and improve the overall collaborative combat capability.

[0061] The above scheme ensures the stability and effectiveness of the training process by initializing and updating the network parameters. It ensures the continuous optimization of the model during training by calculating the network parameter gradients and using the gradient ascent method and gradient descent method to update the network parameters of the actor network and the critic network respectively, which helps to improve the training efficiency and effect, and enables the intelligent agent to more accurately evaluate its own strategy and make optimization adjustments.

[0062] In the above scheme, the value decomposition method corresponding to the QMIX network can quantify the contribution of each agent by defining a clear decomposition form (the relationship between the local Q value function of the agent and the overall Q function). During the training process, the strategy of each agent can be evaluated more accurately, and reasonable credit allocation can be made based on the evaluation results. This helps to form effective cooperation and coordination between agents, so that the multi-agent system can learn the optimal strategy more efficiently. Each agent only needs to focus on its own action selection and the corresponding action value function, so it can learn and optimize in parallel, which not only speeds up the learning speed, but also improves the quality of the learned strategy. At the same time, due to the combination of value decomposition and MAPPO, the global state information can be used to optimize the strategy of each agent during the training process, and each agent makes decisions independently based on its own local observations during execution, further improving the learning efficiency and the adaptability of the strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 A flowchart of a simulation training method for land combat unmanned equipment provided in accordance with an embodiment of the present invention.

[0065] Figure 2 A schematic diagram of the network architecture of a MAPPO network provided in one embodiment of the present invention.

[0066] Figure 3 A schematic diagram of the network architecture of a QMIX network provided in an embodiment of the present invention.

[0067] Figure 4 A schematic diagram of a flow chart of iterative model training according to an embodiment of the present invention.

[0068] Figure 5 A schematic structural diagram of a simulation training device for land combat unmanned equipment provided in one embodiment of the present invention.

[0069] The realization of the purpose, functional features and advantages of the invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0071] The content of the present invention is described in detail below with reference to embodiments.

[0072] This scheme can fully consider the characteristics of confrontational combat scenarios, such as the complexity of urban terrain and the interdependence between agents, so as to optimize the training effect of the Red Agent. It can also realize information sharing, task allocation and coordinated combat between agents. By introducing the value decomposition method, the performance and stability of the algorithm are further improved, the solution space is explored more effectively, the convergence speed is increased, and the possibility of finding the global optimal solution is increased.

[0073] QMIX network: It is mainly used to learn the relationship between the joint action value function and the local action value function, and combine the local action value function into the global action value function through the hybrid network. It focuses on the collaboration between multiple agents and the evaluation of global strategies.

[0074] MAPPO's value network: It focuses on providing instant feedback and gradient information for strategy adjustment to the MAPPO algorithm.

[0075] Complementarity: The QMIX network and the MAPPO value network are complementary in the algorithm. The QMIX network provides an estimate of the joint action value function, which helps the MAPPO algorithm consider the collaboration between multiple agents and the optimization of the global strategy when updating the strategy. The MAPPO value network helps the agents make immediate feedback and adjustments during the decision-making process.

[0076] Reference Figure 1 The figure is a flow chart of a simulation training method for land combat unmanned equipment provided by one embodiment of the present invention.

[0077] In this embodiment, the method includes:

[0078] S11, create a land combat confrontation scenario of multi-agent unmanned equipment, define the scene information, observation space, action space, winning and losing conditions and reward function of the red agent and the blue agent in the land combat confrontation scenario; wherein the scene information includes the number of entities and the initial positions of the red agent and the blue agent; the observation space includes the local observation space and the global observation space of the red agent; the action space includes the continuous actions and discrete actions of the red agent, and the reward function includes the process reward and the final reward.

[0079] Furthermore, the local observation space includes the type (including tanks, support vehicles, unmanned vehicles, and drones) of the Red Party’s intelligent agent, its name, longitude, latitude, altitude, speed, direction, maneuverable direction, attackable targets, and survival status;

[0080] The global observation space includes the local observation space of the Red Agent, all the information of the Blue Agent, and the environmental information; wherein, all the information of the Blue Agent includes the type, name, longitude, latitude, altitude, speed, remaining ammunition, and survival status; the environmental information includes the location of obstacles, the relative distance between the Red and Blue Agents, and the relative distance between the Red Agent and the friendly agent (other Red Agents);

[0081] The action space is designed by using the local observation space of the Red Agent and predefined actions;

[0082] The winning and losing conditions include:

[0083] If the Red Agent or the Blue Agent loses 70% of its manpower, it is considered defeated.

[0084] If the distance between the Red Agent and the farthest Blue Agent is less than the preset distance threshold, and the Blue Agent loses 50% of its manpower, the Red Agent is considered the winner;

[0085] If the duration of the confrontation reaches the set maximum value, the red agent is deemed defeated.

[0086] In this embodiment, scene information such as the number of entities and initial positions of the red and blue sides in the land combat confrontation scenario is determined. The red side's intelligent agents include: tanks, support vehicles, unmanned vehicles, and drones. Among them, the support vehicles are mainly rear support and are good at long-range attacks and reconnaissance; tanks are good at close-range attacks; unmanned vehicles are good at concealed enemy engagement and reconnaissance; drones are mainly used for reconnaissance. The blue side's intelligent agents include two tanks, two infantry fighting vehicles, a helicopter, and two infantry squads. Each infantry squad is located on the roof of a high-rise building to facilitate the discovery of red side intelligence. In this scenario, the red side is the attacker and the blue side is the defender. The red side intends to occupy the blue side's position. The main purpose of training the red side's multi-agents is to find a collaborative strategy between the red side's intelligent agents. The higher the winning rate of occupying the blue side's position, the better.

[0087] Furthermore, the local observation space and global observation space of the Red Party agent are determined. Specifically, here we select as much information as possible that the simulation can provide as the local observation space of the Red Party agent. For example, the local observation space of the Red Party entity includes the entity's type, name, longitude, latitude, altitude, speed, direction, maneuverable direction, strikeable target, survival status, etc. The global observation space includes the local observation space of all Red Party entities, all information of the Blue Party agent (including type, name, longitude, latitude, altitude, speed, remaining ammunition, survival status), and some environmental information (including the location of obstacles, the relative distance between the Red and Blue parties, the relative distance between the Red Party entity and the friendly party, etc.).

[0088] Determine the action space of the Red Agent, including continuous actions such as direction and speed, as well as discrete actions such as whether to attack the Blue target and whether to choose to attack the Blue target. The action space of the agent is designed by combining the local observation space of the Red Agent and predefined actions (defined inside the simulation platform entity model). The Red Agent can maneuver, attack, and scout. In addition, when attacking, the target to be attacked and the ammunition to be used must be given. Discretize the direction of entity maneuver, select the discrete granularity according to the training granularity, and select the commonly used height as the reconnaissance height of the drone.

[0089] Determine the winning and losing conditions and reward function of the scenario, where the reward function includes process rewards and final rewards. Design the winning and losing conditions according to the purpose of the confrontation, including:

[0090] If either the red or blue team loses 70% of its manpower first, it will be considered defeated.

[0091] If the distance between the Red Team and the farthest Blue Team entity is less than a certain threshold, and the Blue Team loses 50% of its manpower, the Red Team is deemed to have won;

[0092] If the confrontation time reaches the set maximum value, the red side is considered defeated.

[0093] The reward function is designed as a process reward: a negative reward for being attacked and a positive reward for attacking the blue team; the final reward: a positive reward for victory and a negative reward for failure. If victory is achieved, the reconnaissance time, confrontation time, and remaining troops are used as adjustment factors for the final reward to affect the final reward of victory. The impact of reconnaissance time, confrontation time, and remaining troops on the reward is designed as a piecewise function, which is adjusted according to the training scenario.

[0094] S12, construct a network training framework for multi-agent reinforcement learning, including MAPPO network and QMIX network, to obtain a multi-agent reinforcement learning model; among them, MAPPO network is used to provide instant feedback and gradient information for strategy adjustment; QMIX network is used to achieve collaboration among multiple agents and evaluation of global strategies.

[0095] Further, in step S12, the MAPPO network is constructed based on the actor-critic architecture, including a policy network and a value network, wherein the policy network includes an input layer, a hidden layer and an output layer, wherein:

[0096] The state vector of the land battle scene is received through the input layer. The state vector includes the local observation space of all red team agents.

[0097] In the hidden layer, a local state feature in a state vector is captured by a first hidden layer, and a feature is extracted and dimension is reduced by a second hidden layer, and a recurrent neural network is introduced in the second hidden layer to capture time dependency;

[0098] Output the decision made by the Red Agent in the current state through the output layer;

[0099] The value network includes an input layer, a hidden layer and an output layer, wherein:

[0100] Receive the global observation space through the input layer;

[0101] The global state features of the global observation space are extracted through the hidden layer, and the QMIX network is introduced into the hidden layer to capture the collaboration and interaction relationship between multiple agents;

[0102] The value estimate of the current state is output through the output layer.

[0103] refer to Figure 2 In this embodiment, the MAPPO network is constructed based on the actor-critic architecture. The network architecture of the MAPPO network includes a policy network and a value network. The network structure of the policy network includes:

[0104] Input layer: A fully connected layer is used to receive the state vector from the environment (land combat scenario), which contains the local observation space of all red team agents.

[0105] Hidden Layer:

[0106] First hidden layer: Use a fully connected layer with a large number of neurons (e.g., 512 neurons) to capture useful features in the state vector (local state features);

[0107] Second hidden layer: Use another fully connected layer with a medium number of neurons (e.g., 256 neurons) to further extract features and reduce dimensions; introduce a recurrent neural network structure (such as LSTM) in this second hidden layer to capture time dependencies.

[0108] Using ReLU (Rectified Linear Unit) activation function between networks can speed up training and alleviate the gradient vanishing problem.

[0109] Output layer: It is a fully connected layer with the same number of neurons as the dimension of the action space. The output layer is usually followed by a softmax activation function to ensure that the sum of the output probability distribution is 1 and each probability value is between 0 and 1. Finally, the possible decisions made by the red agent in the current state are output, so as to use Strategy makes strategic choices for decision making.

[0110] The network structure of the value network includes:

[0111] Input layer: It is a fully connected layer. Its network is the same as the policy network, but it receives the global observation space (that is, the local observation space of all red agents, all information of the blue agent, and part of the environment information) as input.

[0112] Hidden layer: Similar to the hidden layer in the policy network, state features are extracted through two fully connected layers. The value-decomposed QMIX network is introduced in the hidden layer. By adding the value-decomposed QMIX network to the value network of the MAPPO network, the collaboration and interaction between multiple agents can be better captured, so that more effective strategies can be learned, which helps to improve the overall performance of the multi-agent system in a collaborative environment.

[0113] Output layer: A single neuron that outputs a scalar value representing the value estimate of the current state; a linear activation function is used to directly output the value estimate.

[0114] Further, in step S12, the QMIX network includes an agent network, a super network and a hybrid network, the agent network is constructed based on a GRU recurrent neural network, the super network is constructed using a multi-layer perceptron, and the hybrid network is constructed using a sampled feedforward neural network; wherein,

[0115] Inputting the local observation space and current action of the Red Agent into the Agent Network, and outputting the local Q-value function of the Red Agent;

[0116] After the global observation space is input into the hypernetwork for preprocessing, it is linearly transformed by MLP and nonlinearly transformed by activation function to output a global vector;

[0117] The local Q value function is input into the hybrid network, and the global vector is used as the weight of the hybrid network, and the global Q value function is output.

[0118] refer to Figure 3 In this embodiment, the actor-critic architecture of the MAPPO network is retained and the value decomposition method of the QMIX network is integrated into the value network. Specifically, the critic network is no longer just a central value function, but is able to use the structure of the QMIX network to decompose the global value and take into account the contribution of each agent.

[0119] The QMIX network includes an agent network, a hypernetwork, and a hybrid network. The agent network uses a recurrent neural network based on GRU, which is a variant of LSTM (Long Short-Term Memory Network) with similar memory capabilities but a simpler structure. By inputting the information of the local observation space of the red agent and the current action into the agent network, the local Q value function of the red agent is output.

[0120] The hypernetwork uses a multi-layer perceptron (MLP) as the design form of the hypernetwork. MLP can capture more complex nonlinear relationships, which may improve the performance of the QMIX network. The relationship between the global Q-value function and the local Q-value function of each agent must satisfy monotonicity, so the weights of the hybrid network are designed to be non-negative. Therefore, before the last layer of the MLP (i.e., the output layer), a non-negative activation function (such as ReLU) is used to output the weights of the hybrid network. The bias can be similar to the network of weights, but the bias value is not restricted to be non-negative. The input of the hypernetwork is the information of the global observation space. It is first preprocessed (such as normalization) to ensure that it is suitable for neural network training; then the input data is linearly transformed through the MLP and nonlinearly transformed through the activation function. After the MLP processing, the hypernetwork will output a vector, which is reshaped or otherwise transformed (transposed, dimensionally expanded, etc.) to match the requirements of the hybrid network. Thus, the output of the hypernetwork is used as the weight of the hybrid network.

[0121] Hybrid network: Generally, a feedforward neural network is used as a hybrid network. Here, the performance is optimized by increasing the network depth, using different activation functions for each layer, and adjusting the dimension of each layer of the network. The following is the specific design form:

[0122] Input layer: receives the local Q-value functions of n red agents.

[0123] Hidden layer 1: A fully connected layer is used, and the dimension is adjusted according to the complexity of the task, for example, it is set to 256 or 512. The activation function uses ReLU, which helps solve the gradient disappearance problem and has high computational efficiency.

[0124] Hidden layer 2: The network is similar to hidden layer 1. The activation function can be Tanh, which can limit the output value between -1 and 1, which helps the nonlinear expression of the model.

[0125] Hidden layer 3: The network is similar to hidden layer 1, and can use ReLU variants such as Leaky ReLU, ParametricReLU and other activation functions.

[0126] Output layer: A fully connected layer is used to ensure that the output layer can receive all information from the previous layer and process it accordingly. The output of the output layer is the output of the hybrid network, which is the global Q-value function: it represents the overall value that all agents can obtain under a given joint action and global state.

[0127] Through its unique hybrid network design, the QMIX algorithm is able to learn a global action value function (global Q-value function) that accurately estimates the total reward given a global state and joint action. This means that when multiple agents perform tasks together, they can learn how to collaborate with each other to maximize the total reward. In addition, this is very important for the MAPPO algorithm, because when the MAPPO algorithm updates its strategy, it usually relies on the value function to evaluate the quality of the current strategy, which helps to reduce the variance of the MAPPO algorithm when updating its strategy. Finally, the hybrid network implicitly considers the impact of each agent's action when evaluating the value of the joint action. This design enables the MAPPO algorithm to achieve efficient and stable strategy learning in a multi-agent environment.

[0128] S13, by initializing and updating the network parameters of the MAPPO network and the QMIX network, the multi-agent reinforcement learning model is iteratively trained until the training is stopped when the preset conditions are met, and a multi-agent collaborative system is obtained.

[0129] Furthermore, in step S13, the initialization and updating of network parameters of the MAPPO network and the QMIX network include:

[0130] S13-1, network parameters , , , And the memory pool D is initialized; among them, , , , They are the value function, the strategy used, the value network, and the network parameters of the strategy network;

[0131] S13-2, the agent (red agent i) executes the policy function corresponding to the policy network The generated action gets the reward value and , , , Stored in memory pool D; where, , , , They represent the state, action, reward of the red agent i at time t and the state at time t+1 respectively;

[0132] S13-3, every In each cycle, group B data is selected from memory pool D to calculate the gradient. and gradient , and through the gradient and gradient The network parameters of the neural network and network parameters Update; among them, the network parameters of the neural network and network parameters They are the network parameters of the actor network. And the network parameters of the critic network .

[0133] Further, in step S13-3, the In each cycle, group B data is selected from memory pool D to calculate the gradient. and gradient ,include:

[0134] pass Calculate and obtain the network parameters of the actor network corresponding to the i-th red agent gradient ,in,

[0135] ,

[0136] ,

[0137] ,

[0138] ,

[0139] ,

[0140] ,

[0141] In the formula, represents the gradient operator, B represents the number of samples selected for one training, represents the advantage function of the i-th red agent, Used to describe the difference between the new and old control strategies, represents a small positive number, represents the updated policy function of the i-th red agent, represents the current policy function of the i-th red agent, Represents the critic network of the i-th red agent based on the current parameters The output value of Represents the discount factor, a number between 0 and 1, used to calculate the current value of future rewards, r t represents the reward value of immediate feedback, T is the time step, represents the state value function;

[0142] pass Calculate and get the network parameters of the critic network corresponding to the i-th red agent gradient ;

[0143] in, represents the joint action of all red agents, , represents the action value function of the i-th red agent, which is the output of the agent network. is a nonlinear function used to combine individual action-value functions into a joint action-value function, i.e., a feedforward neural network. Represents the output of the QMIX network: the joint action-value function.

[0144] Furthermore, the gradient and gradient The network parameters of the neural network and Updates include:

[0145] Update the network parameters of the actor network by using the gradient ascent method , get the updated network parameters ;

[0146] Update the network parameters of the critic network by using the gradient descent method , get the updated network parameters ,in,

[0147] ,

[0148] ,

[0149] In the formula, represents the learning rate of the actor network, represents the learning rate of the critic network, Represents the weight parameters of the value network before updating.

[0150] Reference Figure 4 In this embodiment, first, the network parameters of the actor network are initialized. and the network parameters of the critic network Given each red agent i according to the state and the policy function Maximize discount rewards , Represents the policy function of each red agent i. The expression is:

[0151] (1)

[0152] in, is the reward discount factor, The larger it is, the more important the future reward is; for The reward value of the red agent i at this moment.

[0153] The maximum discount reward will be solved The problem is transformed into finding the optimal dispersion strategy, so that the counterattack plan generated based on the optimal dispersion strategy can maximize the destruction of the blue side, minimize the loss of the red side, and shorten the confrontation time.

[0154] The given policy function set is:

[0155] Then the joint strategy function , given the action-value function for:

[0156] (2).

[0157] Given a state value function for:

[0158] (3)

[0159] In the formula, a represents the policy function All possible values ​​of the action generated.

[0160] Given the advantage function expression for:

[0161] (4).

[0162] The MAPPO algorithm continuously interacts with the environment, collects environmental states and calculates advantage functions, gradually training the optimal joint strategy. ,Right now:

[0163] Further use of the central evaluation function Evaluating the action-value function Good or bad.

[0164] The above formulas (1)-(4) are several basic concepts and their corresponding formula expressions. The subsequent parameter update process is implemented based on these basic concepts.

[0165] Use gradient ascent method to update the network parameters of the actor network , the network parameters of the actor network of the red agent i gradient for:

[0166] (5)

[0167] (6)

[0168] In the formula, B is the number of samples selected for one training. is the advantage function of the red agent i, is the learning rate of the actor network, Used to describe the differences between the new and old control strategies.

[0169] (7)

[0170] (8)

[0171] (9)

[0172] (10)

[0173] (11)

[0174] In the formula Yes An estimated value of is the updated policy function of the agent; For the Red Agent The current policy function; For the Red Agent The critic network is based on the current parameters , and The output value of ; T is the time step.

[0175] At the same time, the gradient descent method is used to update the network parameters of the critic network , the network parameters of the critic network of the red agent i gradient for:

[0176] (12)

[0177] (13)

[0178] In the formula, Output of QMIX: joint action-value function.

[0179] During the training process, the agent updates the policy network using the decision sequence obtained by exploring the environment and periodically updates the target policy network parameters. .

[0180] The process of MAPPO algorithm includes: first initialize the network parameters , , , and memory pool D (experience replay pool); then the agent executes the policy network The generated action gets the reward value and , , , Stored in memory pool D; finally, every cycles are selected from memory pool D Group data used for calculation and , and update the network parameters of the neural network through the above formula (6) and formula (13) and .Right now:

[0181] (1) Initialization , , , ;

[0182] (2) Initialize memory pool D;

[0183] (3) Start training until m training times:

[0184] (3-1) A round of confrontation begins:

[0185] a. Each agent i performs Generate Action ;

[0186] b. Execute action ,calculate And update the status to get ;

[0187] c. , , , Stored in memory pool D;

[0188] The confrontation ended

[0189] (3-2) Calculated by formula (10) ;

[0190] (3-4) Calculated by formula (9) ;

[0191] (3-4) Sampling from memory pool D:

[0192] a. Randomly select group B data from D;

[0193] b. Calculate by formula (5) ;

[0194] c. Calculate by formula (12) ;

[0195] d. Update by formula (6) ;

[0196] e. Update by formula (13) ;

[0197] Calculate the sampled data gradient and update the network parameters;

[0198] After m rounds of training, the training is completed.

[0199] Then start the reinforcement learning training, and stop the training when the red team’s win rate is no less than 90%, the loss curve tends to be stable, and the reward tends to be stable.

[0200] In the MAPPO algorithm, the policy network and value network are trained through a centralized training and distributed execution architecture. During the training phase, the value network of all agents can access the global state and action information of other agents in order to learn a more accurate value function. In the execution phase, each agent only uses the local state and strategy it observes to select actions.

[0201] When the QMIX network is added to the MAPPO algorithm, the QMIX network acts as a value decomposition network to estimate the joint action value function. The agent network in the QMIX network is responsible for processing the local observations of each agent and outputting the local Q-value function; the hybrid network is responsible for combining these local Q-value functions into a global Q-value function; and the hypernetwork is used to generate the weights of the hybrid network to achieve the decomposition and combination of the value function.

[0202] During training, the parameter updates of the policy network and value network of the MAPPO network depend on the empirical data collected from the environment, including states, actions, rewards, etc. At the same time, the parameter updates of the QMIX network also depend on these empirical data, especially the data used to calculate the joint action value function. Therefore, during training, the parameter updates of these networks are interrelated.

[0203] The parameter update of the value network of the MAPPO network can be achieved by minimizing the error in the value function estimate, which is usually the difference between the value network's prediction and the actual reward obtained. In the hybrid algorithm with the QMIX network, another part of this error is the difference between the joint action value function predicted by the QMIX network and the actual reward obtained. Using the difference between the joint action value output by the QMIX network and the true reward as part of the value network loss function has the following benefits:

[0204] Improve learning efficiency: Taking the difference between the joint action value output by QMIX and the true return as part of the loss function can enable the value network to learn the relationship between the joint action and the return more quickly, thereby improving learning efficiency.

[0205] Enhanced generalization ability: Since QMIX can handle interactions and collaborations between multiple agents, using the joint action value output by the QMIX network as part of the loss function can make the value network better generalize to different multi-agent environments.

[0206] Optimizing the global policy: By using the difference between the joint action value and the true reward as part of the value network’s loss function, the value network can be forced to estimate the value of the global state more accurately, thus providing more accurate guidance for the optimization of the policy network.

[0207] The policy network's parameter updates are achieved by maximizing the advantage function, which represents the degree of advantage of taking a certain action relative to the average action under the current policy. The advantage function is calculated based on the predicted value of the value network.

[0208] For the QMIX network, its parameter update is achieved by minimizing the estimated error of the joint action value function. This error is calculated by comparing the joint action value function predicted by the QMIX network with the actual reward obtained. During training, the parameter update of the QMIX network will rely on the empirical data collected by MAPPO, especially those containing joint actions and global rewards.

[0209] The design of the QMIX network has a significant effect on the back-propagation algorithm of the MAPPO algorithm itself to update the network parameters. It provides an effective joint action value function estimation, improves the stability and efficiency of policy updates, and promotes collaboration between multiple agents.

[0210] Using the QMIX network non-value decomposition algorithm, the relationship between the global Q-value function and the Q-value function of each agent satisfies monotonicity, which means that the global Q-value function can be regarded as a combination or transformation of the Q-value functions of each agent, and this combination or transformation maintains monotonicity. In this way, during the training process, when optimizing the behavior of each agent by maximizing the global Q-value function, it can be ensured that the local strategy of each agent is moving towards the direction of the global optimal solution. It ensures the consistency between the global optimization goal and the local optimization goal, thereby improving the stability and efficiency of the algorithm.

[0211] Reference Figure 5 Shown is a schematic structural diagram of a simulation training device for land combat unmanned equipment provided by an embodiment of the present invention.

[0212] In this embodiment, the device 20 includes:

[0213] The scenario creation unit 21 is used to create a land combat confrontation scenario of multi-agent unmanned equipment, and define the scenario information, observation space, action space, winning and losing conditions, and reward function of the red agent and the blue agent in the land combat confrontation scenario; wherein the scenario information includes the number of entities and initial positions of the red agent and the blue agent; the observation space includes the local observation space and the global observation space of the red agent; the action space includes the continuous action and the discrete action of the red agent, and the reward function includes the process reward and the final reward;

[0214] A framework construction unit 22 is used to construct a network training framework for multi-agent reinforcement learning, including a MAPPO network and a QMIX network, to obtain a multi-agent reinforcement learning model; wherein the MAPPO network is used to provide instant feedback and gradient information for strategy adjustment; and the QMIX network is used to achieve collaboration among multiple agents and evaluation of global strategies;

[0215] The model training unit 23 is used to iteratively train the multi-agent reinforcement learning model by initializing and updating the network parameters of the MAPPO network and the QMIX network, and stop the training when the preset conditions are met to obtain a multi-agent collaborative system.

[0216] Each unit module of the device 20 can respectively execute the corresponding steps in the above method embodiment, so each unit module will not be described in detail here. Please refer to the description of the above corresponding steps for details.

[0217] The embodiment of the present invention also provides a land combat unmanned equipment simulation training device, the device includes the land combat unmanned equipment simulation training device as described above, wherein the land combat unmanned equipment simulation training device can be used Figure 5 The structure of the embodiment can be executed accordingly. Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects. For details, please refer to the relevant records in the above embodiments, which will not be repeated here.

[0218] The device includes: a mobile phone, a digital camera, a tablet computer, or other devices with a camera function, or a device with an image processing function, or a device with an image display function. The device may include components such as a memory, a processor, an input unit, a display unit, and a power supply.

[0219] Among them, the memory can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as an image playback function, etc.), etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory may also include a memory controller to provide the processor and the input unit with access to the memory.

[0220] The input unit can be used to receive input digital or character or image information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit of this embodiment includes not only a camera, but also a touch-sensitive surface (such as a touch display) and other input devices.

[0221] The display unit can be used to display information input by the user or information provided to the user and various graphical user interfaces of the device, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit may include a display panel, and optionally, the display panel may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface may cover the display panel, and when the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor to determine the type of touch event, and then the processor provides a corresponding visual output on the display panel according to the type of touch event.

[0222] The embodiment of the present invention further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory in the above embodiment; or a computer-readable storage medium that exists independently and is not installed in a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement Figure 1 The computer readable storage medium may be a read-only memory, a disk or an optical disk, etc.

[0223] The embodiment of the present invention also provides a computer program product, including a computer program / instruction, which is loaded and executed by a processor to implement Figure 1 A simulation training method for land combat unmanned equipment is shown.

[0224] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, equipment embodiment and storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0225] Furthermore, in this document, the terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device that includes the element.

[0226] The above description shows and describes the preferred embodiments of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the invention, through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not depart from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A simulation training method for land combat unmanned equipment, characterized in that: The method comprises: Create a land combat confrontation scenario of multi-agent unmanned equipment, and define the scenario information, observation space, action space, winning and losing conditions, and reward function of the red agent and the blue agent in the land combat confrontation scenario; the scenario information includes the number of entities and initial positions of the red agent and the blue agent; the observation space includes the local observation space and the global observation space of the red agent; the action space includes the continuous actions and discrete actions of the red agent, and the reward function includes the process reward and the final reward; Construct a network training framework for multi-agent reinforcement learning, including MAPPO network and QMIX network, to obtain a multi-agent reinforcement learning model; the MAPPO network is used to provide instant feedback and gradient information for strategy adjustment; the QMIX network is used to achieve collaboration between multiple agents and evaluation of global strategies; Furthermore, the MAPPO network is constructed based on the actor-critic architecture, including a policy network and a value network, wherein the policy network includes an input layer, a hidden layer and an output layer, wherein: The state vector of the land battle scene is received through the input layer. The state vector includes the local observation space of all red team agents. In the hidden layer, a local state feature in a state vector is captured by a first hidden layer, and a feature is extracted and dimension is reduced by a second hidden layer, and a recurrent neural network is introduced in the second hidden layer to capture time dependency; Output the decision made by the Red Agent in the current state through the output layer; The value network includes an input layer, a hidden layer and an output layer, wherein: Receive the global observation space through the input layer; The global state features of the global observation space are extracted through the hidden layer, and the QMIX network is introduced into the hidden layer to capture the collaboration and interaction relationship between multiple agents; Output the value estimate of the current state through the output layer; Furthermore, the QMIX network includes an agent network, a super network and a hybrid network, wherein the agent network is constructed based on a GRU recurrent neural network, the super network is constructed using a multi-layer perceptron, and the hybrid network is constructed using a sampled feedforward neural network; wherein, Inputting the local observation space and current action of the Red Agent into the Agent Network, and outputting the local Q-value function of the Red Agent; After the global observation space is input into the hypernetwork for preprocessing, it is linearly transformed by MLP and nonlinearly transformed by activation function to output a global vector; Inputting the local Q value function into the hybrid network, using the global vector as the weight of the hybrid network, and outputting the global Q value function; By initializing and updating the network parameters of the MAPPO network and the QMIX network, the multi-agent reinforcement learning model is iteratively trained until the preset conditions are met and the training is stopped to obtain a multi-agent collaborative system.

2. The method for simulating training of unmanned land combat equipment according to claim 1, characterized in that: The local observation space includes the type, name, longitude, latitude, altitude, speed, direction, maneuverable direction, attackable targets, and survival status of the Red Agent; The global observation space includes the local observation space of the red agent, all the information of the blue agent and the environmental information; wherein all the information of the blue agent includes the type, name, longitude, latitude, altitude, speed, remaining ammunition quantity, and survival status; the environmental information includes the location of obstacles, the relative distance between the red and blue parties, and the relative distance between the red agent and the friendly party; The action space is designed by using the local observation space of the Red Agent and predefined actions; The winning and losing conditions include: If the Red Agent or the Blue Agent loses 70% of its manpower, it is considered defeated. If the distance between the Red Agent and the farthest Blue Agent is less than the preset distance threshold, and the Blue Agent loses 50% of its manpower, the Red Agent is considered the winner; If the duration of the confrontation reaches the set maximum value, the red agent is deemed defeated.

3. The method for simulating training of unmanned land combat equipment according to claim 1, characterized in that: The initialization and updating of network parameters of the MAPPO network and the QMIX network include: Network parameters , , , And the memory pool D is initialized; Each Red Agent executes the policy function corresponding to the policy network The generated action gets the reward value and , , , Stored in memory pool D; Every In each cycle, group B data is selected from memory pool D to calculate the gradient. and gradient , and through the gradient and gradient The network parameters of the neural network and network parameters to update.

4. The method for simulating training of unmanned land combat equipment according to claim 3, characterized in that: The In each cycle, group B data is selected from memory pool D to calculate the gradient. and gradient ,include: pass Calculate and obtain the network parameters of the actor network corresponding to the i-th red agent gradient ,in, ， ， ， ， ， In the formula, represents the gradient operator, B represents the number of samples selected for one training, represents the advantage function of the i-th red agent, Used to describe the difference between the new and old control strategies, represents a small positive number, represents the updated policy function of the i-th red agent, represents the current policy function of the i-th red agent, Represents the critic network of the i-th red agent based on the current parameters The output value of Represents the discount factor, a number between 0 and 1, used to calculate the current value of future rewards, r t Represents the reward value of immediate feedback, T is the time step; pass Calculate and get the network parameters of the critic network corresponding to the i-th red agent gradient ; in, represents the joint action of all red agents, , represents the action value function of the i-th red agent, which is the output of the agent network. is a nonlinear function used to combine individual action-value functions into a joint action-value function, i.e., a feedforward neural network. Represents the output of the QMIX network: the joint action-value function.

5. The method for simulating training of unmanned land combat equipment according to claim 4, characterized in that: The gradient and gradient The network parameters of the neural network and network parameters Updates include: Update the network parameters of the actor network by using the gradient ascent method , get the updated network parameters ; Update the network parameters of the critic network by using the gradient descent method , get the updated network parameters ,in, ， ， In the formula, represents the learning rate of the actor network, represents the learning rate of the critic network, Represents the weight parameters of the value network before updating.

6. A simulation training device for land combat unmanned equipment, characterized in that: The device comprises: A scenario creation unit is used to create a land combat confrontation scenario of multi-agent unmanned equipment, and define the scenario information, observation space, action space, winning and losing conditions, and reward function of the red agent and the blue agent in the land combat confrontation scenario; wherein the scenario information includes the number of entities and initial positions of the red agent and the blue agent; the observation space includes the local observation space and the global observation space of the red agent; the action space includes the continuous actions and discrete actions of the red agent, and the reward function includes the process reward and the final reward; The framework construction unit is used to construct a network training framework for multi-agent reinforcement learning, including the MAPPO network and the QMIX network, to obtain a multi-agent reinforcement learning model; the MAPPO network is used to provide instant feedback and gradient information for strategy adjustment; the QMIX network is used to achieve collaboration between multiple agents and evaluation of global strategies; Furthermore, the MAPPO network is constructed based on the actor-critic architecture, including a policy network and a value network, wherein the policy network includes an input layer, a hidden layer and an output layer, wherein: The state vector of the land battle scene is received through the input layer. The state vector includes the local observation space of all red team agents. In the hidden layer, a local state feature in a state vector is captured by a first hidden layer, and a feature is extracted and dimension is reduced by a second hidden layer, and a recurrent neural network is introduced in the second hidden layer to capture time dependency; Output the decision made by the Red Agent in the current state through the output layer; The value network includes an input layer, a hidden layer and an output layer, wherein: Receive the global observation space through the input layer; The global state features of the global observation space are extracted through the hidden layer, and the QMIX network is introduced into the hidden layer to capture the collaboration and interaction relationship between multiple agents; Output the value estimate of the current state through the output layer; Furthermore, the QMIX network includes an agent network, a super network and a hybrid network, wherein the agent network is constructed based on a GRU recurrent neural network, the super network is constructed using a multi-layer perceptron, and the hybrid network is constructed using a sampled feedforward neural network; wherein, Inputting the local observation space and current action of the Red Agent into the Agent Network, and outputting the local Q-value function of the Red Agent; After the global observation space is input into the hypernetwork for preprocessing, it is linearly transformed by MLP and nonlinearly transformed by activation function to output a global vector; Inputting the local Q value function into the hybrid network, using the global vector as the weight of the hybrid network, and outputting the global Q value function; The model training unit is used to iteratively train the multi-agent reinforcement learning model by initializing and updating the network parameters of the MAPPO network and the QMIX network, and stop the training when the preset conditions are met to obtain a multi-agent collaborative system.

7. A simulation training device for land combat unmanned equipment, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a simulation training method for land combat unmanned equipment as described in any one of claims 1 to 5.

8. A computer program product, comprising a computer program / instruction, wherein the computer program / instruction is executed by a processor to implement the steps of a simulation training method for land combat unmanned equipment as described in any one of claims 1 to 5.

Images