Intelligent cooperative game simulation modeling method for heterogeneous combat unit

Abstract

The invention discloses an intelligent cooperative game simulation modeling method for a heterogeneous combat unit, and relates to the technical field of military simulation modeling and computer simulation, and the method comprises the following units: an environment construction layer, an intelligent agent decision-making layer, a cooperative game layer and a performance evaluation layer. The problem that a traditional collaborative decision-making method is poor in adaptability is effectively solved, the environment construction layer simulates real-time changes of geography, weather, electromagnetic and enemy environments through a dynamic random process, and the intelligent agent decision-making layer constructs a differentiated deep reinforcement learning model for functional differences of heterogeneous combat units. A priority experience playback mechanism and a TD3 strategy are introduced into an improved DDPG algorithm, so that the utilization efficiency of important experience samples is improved, the problem of over-estimation of a critic network is relieved, the system gets rid of limitation of a traditional preset rule, the dynamic nature and uncertainty of a battlefield can be dynamically responded, and the self-adaptive decision-making capability of a heterogeneous combat unit in complex enemies and environments is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of military simulation modeling and computer simulation technology, and in particular relates to a simulation modeling method for intelligent collaborative game between heterogeneous combat units. Background Technology

[0002] In modern information warfare, combat systems exhibit the significant characteristics of "multi-domain integration and heterogeneous collaboration." The combat capabilities of a single combat unit are no longer sufficient to meet the demands of complex battlefield environments, making collaborative operations among heterogeneous combat units a core means of enhancing overall combat effectiveness. For example, in air defense and missile defense missions, it is necessary to integrate early warning satellites, ground radars, fighter jets, air defense missile systems, and unmanned reconnaissance vehicles into a collaborative combat system to achieve a closed-loop process of "detection-identification-tracking-strike-assessment." In amphibious landing operations, it is necessary to coordinate the actions of unmanned obstacle-clearing vehicles, amphibious armored vehicles, shipborne helicopters, and logistics support ships to ensure the efficient advancement of the landing mission.

[0003] The core challenges of collaborative game theory among heterogeneous combat units lie in two aspects: First, the heterogeneity of functions, communication protocols, and decision-making logic among units makes it difficult to uniformly transmit and execute collaborative instructions. Second, the battlefield environment is dynamic (e.g., changes in enemy troop deployment, sudden changes in weather conditions), uncertain (e.g., lack of intelligence information, equipment malfunctions), and adversarial (e.g., enemy interference, deception, and attacks), requiring the combat system to possess real-time adaptive decision-making capabilities. Traditional collaborative decision-making methods mainly rely on manually preset rules or optimization algorithms based on mathematical programming, which have the following limitations:

[0004] Poor adaptability: The preset rules are only applicable to specific combat scenarios. When the battlefield environment deviates from the preset conditions, the decision-making effect drops sharply, making it difficult to cope with complex and ever-changing enemy situations and environments.

[0005] Insufficient optimization capabilities: Mathematical programming methods (such as linear programming and integer programming) are prone to getting trapped in local optima when dealing with high-dimensional, nonlinear combat decision-making problems, and the computational complexity increases exponentially with the number of combat units, which cannot meet the needs of real-time decision-making.

[0006] Weak coordination: Traditional methods focus on task allocation for individual units, neglecting dynamic interaction and resource sharing between units, making it difficult to achieve global optimization of overall combat effectiveness.

[0007] To overcome the aforementioned bottlenecks, the application of artificial intelligence technology, especially reinforcement learning, in the field of combat decision-making has attracted widespread attention. Existing research has seen some attempts to apply reinforcement learning to the autonomous decision-making of single combat units, but these have not considered the cooperative game relationships between heterogeneous units. A few cooperative decision-making schemes, while introducing multi-agent reinforcement learning, suffer from problems such as dimensional explosion in the state and action spaces, conflicting reward functions among agents, and unstable training processes, making them difficult to apply to simulation modeling and decision optimization in large-scale heterogeneous combat systems. Therefore, there is an urgent need for a cooperative game simulation modeling method and system that can integrate deep reinforcement learning and game theory, adapt to the characteristics of heterogeneous combat units, and possess efficient training and real-time decision-making capabilities. Summary of the Invention

[0008] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0009] This invention is a simulation modeling method for intelligent collaborative game of heterogeneous combat units, comprising the following units: environment construction layer, agent decision-making layer, collaborative game layer, and effectiveness evaluation layer.

[0010] Furthermore, the environment construction layer is responsible for constructing a simulated combat environment and entities that closely resemble the real battlefield, providing an interactive scenario for intelligent agent decision-making and collaborative game. The environment construction layer includes: a combat entity modeling module, a battlefield environment modeling module, and an interaction rule modeling module.

[0011] The combat entity modeling module constructs an entity model library based on the physical characteristics, functional parameters, and combat capabilities of heterogeneous combat units. For example, fighter jet models include parameters such as maximum speed, range, type of weapons carried, and radar detection range; UAV models include parameters such as endurance, payload capacity, and communication range. An object-oriented design approach is adopted to define attribute (e.g., position, state, capability) and behavior (e.g., movement, detection, attack) interfaces for each entity.

[0012] The battlefield environment modeling module is responsible for constructing models of the geographical environment (terrain, landforms, altitude), meteorological environment (wind speed, visibility, precipitation), electromagnetic environment (electromagnetic interference intensity, frequency range), and enemy situation environment (enemy troop deployment, equipment performance, tactical intentions). It employs dynamic stochastic processes to simulate real-time changes in the environment; for example, it generates enemy troop movement probabilities using a Markov chain model and simulates electromagnetic interference using a noise model.

[0013] The interaction rule modeling module is used to define the interaction logic between combat entities, including detection rules (such as the relationship between radar detection probability and distance and angle), strike rules (such as the relationship between weapon damage probability and hit accuracy), communication rules (such as communication delay and data packet loss rate), and coordination rules (such as mission handover conditions and resource sharing mechanisms).

[0014] Furthermore, the agent decision layer is responsible for constructing differentiated deep reinforcement learning decision models based on the functional differences of heterogeneous combat units, enabling autonomous decision-making for each unit. This layer configures an independent agent for each combat unit, consisting of an actor network (policy execution network) and a critic network (policy evaluation network), trained using an improved DDPG algorithm. The design includes: state space design, action space design, and an improved DDPG algorithm.

[0015] The state space design defines differentiated state vectors for different types of combat units. For example, the state vector of a reconnaissance agent includes "current position, remaining battery power, detection range, number of detected targets, and communication status"; the state vector of an attack agent includes "current position, remaining ammunition, fuel level, weapon range, and target lock status". The state vectors are updated in real time through the environment modeling layer.

[0016] The action space design is used to design continuous or discrete action spaces based on the functional positioning of the combat unit. For example, the action space of a fighter jet is continuous (such as flight speed, heading angle, and altitude change rate); the action space of an air defense missile system is discrete (such as "launch missile", "switch target", and "standby").

[0017] The improved DDPG algorithm introduces a priority experience replay mechanism on the basis of traditional DDPG to improve the utilization efficiency of important experience samples; adopts a double delay update (TD3) strategy to alleviate the overestimation problem of the critic network and improve training stability; and designs a personalized learning rate for each agent to accelerate the policy convergence speed of the dominant unit in response to the ability differences of heterogeneous units.

[0018] Furthermore, the collaborative game layer is responsible for coordinating the decision-making behavior of each agent to achieve global collaborative optimization of heterogeneous combat units. The collaborative game layer includes a game strategy generation module and a collaborative coordination module.

[0019] The game strategy generation module is based on the fusion of non-cooperative and cooperative game theory, constructing a game model of "local autonomous decision-making + global collaborative optimization". First, each agent generates an initial strategy based on its own state and local objectives (non-cooperative game stage). Then, a virtual dominant agent is introduced to optimize and adjust the initial strategies with the goal of maximizing overall combat effectiveness (cooperative game stage). The virtual dominant agent collects the state and strategy information of each agent and uses a Nash equilibrium optimization algorithm to solve for the globally optimal strategy combination.

[0020] The collaborative coordination module is used to resolve resource conflicts and task allocation problems among multiple agents. ① Resource coordination: A distributed resource scheduling algorithm is adopted to achieve dynamic resource sharing based on the remaining resources (such as ammunition, fuel, and electricity) of each unit and task requirements. For example, when a fighter jet runs out of ammunition, it coordinates nearby missile systems to provide support. ② Task coordination: Based on an improved task allocation model using the Hungarian algorithm, the optimal allocation of tasks is achieved by combining the capability matching degree (such as strike accuracy and detection range) of each unit with task priority. ③ Communication coordination: A hierarchical communication protocol is adopted, with high-speed encrypted communication between core units and broadcast communication between edge units to reduce communication latency and data conflicts.

[0021] Furthermore, the effectiveness evaluation layer is responsible for quantitatively evaluating the effectiveness of collaborative game strategies, providing feedback for agent training, and providing a basis for optimizing combat plans. The effectiveness evaluation layer includes: an indicator system construction module, a real-time evaluation module, and a result analysis module.

[0022] The indicator system construction module is used to establish a multi-dimensional combat effectiveness indicator system, including core indicators (mission success rate, target destruction rate, friendly loss rate), efficiency indicators (combat response time, resource utilization rate) and stability indicators (strategy convergence speed, anti-interference capability).

[0023] The real-time evaluation module is used to collect status data and interaction data of each combat unit through sensor models during the simulation process. It uses a comprehensive evaluation algorithm that combines the Analytic Hierarchy Process (AHP) and the entropy weight method to calculate the effectiveness index value in real time and convert it into a reward signal for the agent, which is used to update the parameters of the reinforcement learning model.

[0024] The results analysis module generates an effectiveness evaluation report after the simulation, including indicator values, trend curves, and strategy optimization suggestions. By comparing effectiveness indicators under different strategies, it provides decision support for combat commanders.

[0025] Furthermore, the simulation modeling method includes the following steps:

[0026] Step S1, Scene Configuration: The user inputs the combat mission type (such as air defense, landing, anti-submarine), the number and type of heterogeneous combat units, and battlefield environment parameters (such as terrain and enemy situation) through the system interface. The environment modeling layer generates the initial simulation scene based on the configuration information.

[0027] Step S2, Agent Initialization: Based on the type of combat unit, assign a corresponding agent model to each unit, initialize the parameters of the actor and critic networks (such as weights and biases), and set training parameters such as learning rate and reward function coefficients;

[0028] Step S3, Simulation Interaction and Strategy Learning: The environment modeling layer updates the battlefield state in real time and feeds it back to the agent decision-making layer; each agent generates an initial action strategy based on the current state; the collaborative game layer optimizes the strategy combination through a virtual leading agent and coordinates the actions of each unit; the performance evaluation layer calculates real-time performance indicators and generates reward signals to feed back to the agents; the agents update network parameters by improving the DDPG algorithm.

[0029] Step S4, Strategy Iteration and Optimization: Repeat step S3 until the agent's strategy converges (i.e., the performance index tends to stabilize) or the preset number of training iterations are reached, and output the optimal cooperative game strategy.

[0030] Step S5, Performance Evaluation and Result Output: Simulation verification is performed on the optimal strategy. The performance evaluation layer generates a comprehensive evaluation report. Users can adjust the scenario parameters or training parameters according to the report and restart the simulation process.

[0031] Furthermore, the improved DDPG algorithm structure is as follows:

[0032] The improved DDPG algorithm employs an actor-critic dual-network structure. The actor network uses a 3-layer fully connected neural network, taking the agent's state vector as input and outputting the action policy. The critic network uses a 4-layer fully connected neural network, taking both the state and action vectors as input and outputting the policy's Q-value (action value). The algorithm stores the agent's interaction experience (s, a, r, s') in a priority experience replay pool, prioritizing the sampling of high-priority experience samples (such as those with larger reward values ​​or larger Q-value errors) for training. A TD3 policy is used to delay updates to the critic network, avoiding overestimation.

[0033] Furthermore, the optimization steps of the cooperative game strategy are as follows:

[0034] Policy generation phase: Each agent generates an initial policy P1, P2, ..., Pn based on its own state (n is the number of combat units);

[0035] Strategy evaluation phase: The virtual dominant agent collects local performance indicators corresponding to each strategy and calculates the global performance value E;

[0036] Strategy adjustment phase: If E does not reach the optimal threshold, the parameters of each strategy are adjusted based on the Nash equilibrium optimization algorithm to generate a new strategy;

[0037] Strategy output phase: Repeated evaluation and adjustment until E reaches its optimum, and output the final collaborative strategy.

[0038] The present invention has the following beneficial effects:

[0039] 1. This invention effectively solves the problem of poor adaptability of traditional collaborative decision-making methods. The environment construction layer simulates the real-time changes of geographical, meteorological, electromagnetic and enemy situation environments through dynamic stochastic processes. The intelligent agent decision-making layer constructs differentiated deep reinforcement learning models for the functional differences of heterogeneous combat units. The improved DDPG algorithm introduces a priority experience replay mechanism and TD3 strategy, which not only improves the utilization efficiency of important experience samples, but also alleviates the overestimation problem of the critic network. This enables the system to break free from the limitations of traditional preset rules, dynamically respond to the dynamics and uncertainties of the battlefield, and enhance the adaptive decision-making capability of heterogeneous combat units in complex enemy situations and environments.

[0040] 2. This invention overcomes the bottlenecks of insufficient optimization capabilities and weak collaboration in traditional methods. The collaborative game layer integrates the ideas of non-cooperative and cooperative games, and achieves global strategy optimization by using virtual dominant agents and Nash equilibrium optimization algorithms. At the same time, it solves the problems of resource conflicts, task matching and communication delays among multiple agents by using distributed resource scheduling, improved Hungarian algorithm for task allocation and hierarchical communication protocols. It avoids the local optima and exponential complexity of traditional mathematical programming, realizes global collaboration of heterogeneous combat units, meets the needs of real-time decision-making and ensures the maximization of overall combat effectiveness.

[0041] 3. This invention overcomes the shortcomings of existing solutions in lacking quantitative feedback and iterative optimization mechanisms. The effectiveness evaluation layer constructs a multi-dimensional system containing core, efficiency, and stability indicators. It uses an algorithm that combines AHP and entropy weighting to calculate indicators in real time and convert them into agent reward signals, providing support for reinforcement learning model updates to accelerate policy convergence. After simulation, a report containing indicator values, trend curves, and optimization suggestions is generated, which not only helps the agent to iteratively optimize its policies but also provides data-driven decision-making basis for combat commanders, forming a closed loop of simulation-evaluation-optimization and improving the guidance of simulation modeling for optimizing actual combat plans.

[0042] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a system architecture diagram of the intelligent collaborative game simulation modeling method for heterogeneous combat units according to the present invention;

[0045] Figure 2 This is a flowchart of a simulation modeling method for intelligent collaborative game theory of heterogeneous combat units according to the present invention.

[0046] Figure 3 This is a structural diagram of the DDPG algorithm for a simulation modeling method of intelligent collaborative game between heterogeneous combat units according to the present invention.

[0047] Figure 4 This is a flowchart of the collaborative game strategy optimization process in the intelligent collaborative game simulation modeling method for heterogeneous combat units according to the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Please see Figure 1-4 As shown, this invention is a simulation modeling method for intelligent collaborative game theory of heterogeneous combat units. It is divided into four core layers: environment construction layer, agent decision-making layer, collaborative game theory layer, and effectiveness evaluation layer. Each layer interacts with other systems through data interfaces. The overall system architecture is as follows: Figure 1 As shown.

[0050] The environment construction layer and the agent decision-making layer form a closed loop. The model provides the agent with the state and individual rewards, and the agent outputs actions to the environment, driving the evolution of the environment state.

[0051] The decision-making layer and the coordination layer interact closely. The decision-making layer uploads local experience data, while the coordination layer is responsible for global learning and distributes the updated policy parameters to all agents.

[0052] The effectiveness evaluation layer obtains the final adversarial results and data from the environment layer, analyzes and evaluates them, and its evaluation conclusions can guide the strategy optimization of the collaborative game layer, forming a complete loop of iterative optimization.

[0053] 1. Environment Construction Layer

[0054] The environment construction layer is used to build simulated combat environments and entities that closely resemble real battlefields, providing interactive scenarios for intelligent agent decision-making and collaborative game theory. It mainly includes the following modules:

[0055] (1) Combat Entity Modeling Module

[0056] Based on the physical characteristics, functional parameters, and combat capabilities of heterogeneous combat units, an entity model library is constructed. For example, fighter jet models include parameters such as maximum speed, range, weapon types, and radar detection range; UAV models include parameters such as endurance, payload capacity, and communication range. An object-oriented design approach is adopted to define interfaces for attributes (such as position, state, and capabilities) and behaviors (such as movement, detection, and attack) for each entity.

[0057] (2) Battlefield Environment Modeling Module

[0058] Models are constructed for the geographical environment (topography, landforms, altitude), meteorological environment (wind speed, visibility, precipitation), electromagnetic environment (electromagnetic interference intensity, frequency range), and enemy situation environment (enemy troop deployment, equipment performance, tactical intentions). Dynamic stochastic processes are used to simulate real-time changes in the environment. For example, Markov chain models are used to generate enemy troop movement probabilities, and noise models are used to simulate electromagnetic interference.

[0059] (3) Interaction rule modeling module

[0060] Define the interaction logic between combat entities, including detection rules (such as the relationship between radar detection probability and distance and angle), strike rules (such as the relationship between weapon damage probability and hit accuracy), communication rules (such as communication delay and data packet loss rate), and coordination rules (such as mission handover conditions and resource sharing mechanisms).

[0061] 2. Agent Decision Layer

[0062] The agent decision layer constructs differentiated deep reinforcement learning decision models to address the functional differences among heterogeneous combat units, enabling autonomous decision-making for each unit. This layer configures an independent agent for each combat unit, consisting of an actor network (policy enforcement network) and a critic network (policy evaluation network), trained using an improved DDPG algorithm. The specific design is as follows:

[0063] (1) State-space design

[0064] Different state vectors are defined for different types of combat units. For example, the state vector of a reconnaissance agent includes "current position, remaining battery power, detection range, number of detected targets, and communication status"; the state vector of a strike agent includes "current position, remaining ammunition, fuel level, weapon range, and target lock status". The state vectors are updated in real time through the environment modeling layer.

[0065] (2) Action space design

[0066] Based on the functional positioning of the combat unit, the action space can be designed as either continuous or discrete. For example, the action space of a fighter jet is continuous (such as flight speed, heading angle, and rate of change of altitude); the action space of an air defense missile system is discrete (such as "launch missile", "switch targets", and "standby").

[0067] (3) Improved DDPG algorithm

[0068] Based on the traditional DDPG, a priority experience replay mechanism is introduced to improve the utilization efficiency of important experience samples; a double delay update (TD3) strategy is adopted to alleviate the overestimation problem of the critic network and improve training stability; and personalized learning rates are designed for each agent to accelerate the policy convergence speed of the dominant unit in response to the ability differences of heterogeneous units.

[0069] 3. Collaborative Game Layer

[0070] The collaborative game layer is used to coordinate the decision-making behavior of various agents and realize global collaborative optimization of heterogeneous combat units. Its core includes a game strategy generation module and a collaborative coordination module, which are implemented as follows:

[0071] (1) Game strategy generation module

[0072] Based on the fusion of non-cooperative and cooperative game theory, a game model of "local autonomous decision-making + global collaborative optimization" is constructed. First, each agent generates an initial strategy based on its own state and local objectives (non-cooperative game phase). Then, a virtual dominant agent is introduced to optimize and adjust the initial strategies with the goal of maximizing overall combat effectiveness (cooperative game phase). The virtual dominant agent collects the state and strategy information of each agent and uses a Nash equilibrium optimization algorithm to solve for the globally optimal strategy combination.

[0073] (2) Collaborative Coordination Module

[0074] To address resource conflicts and task allocation issues among multiple agents: ① Resource Coordination: Employing a distributed resource scheduling algorithm, dynamic resource sharing is achieved based on the remaining resources (e.g., ammunition, fuel, electricity) of each unit and task requirements. For example, when a fighter jet runs out of ammunition, nearby missile systems are coordinated for support. ② Task Coordination: A task allocation model based on an improved Hungarian algorithm is used, combining the capability matching degree (e.g., strike accuracy, detection range) of each unit with task priority to achieve optimal task allocation. ③ Communication Coordination: A hierarchical communication protocol is adopted, with high-speed encrypted communication between core units and broadcast communication between edge units, reducing communication latency and data conflicts.

[0075] 4. Performance Evaluation Layer

[0076] The performance evaluation layer is used to quantitatively evaluate the effectiveness of cooperative game strategies, providing feedback for agent training and a basis for optimizing combat plans. It mainly includes:

[0077] (1) Indicator System Construction Module

[0078] Establish a multi-dimensional operational effectiveness indicator system, including core indicators (mission success rate, target destruction rate, friendly loss rate), efficiency indicators (operation response time, resource utilization rate) and stability indicators (strategy convergence speed, anti-jamming capability).

[0079] (2) Real-time evaluation module

[0080] During the simulation, the status and interaction data of each combat unit are collected through the sensor model. A comprehensive evaluation algorithm that combines the Analytic Hierarchy Process (AHP) and the entropy weight method is used to calculate the effectiveness index value in real time and convert it into the reward signal of the agent for the parameter update of the reinforcement learning model.

[0081] (3) Results Analysis Module

[0082] After the simulation, an effectiveness evaluation report is generated, including indicator values, trend curves, and strategy optimization suggestions. By comparing the effectiveness indicators under different strategies, decision support is provided to combat commanders.

[0083] The simulation modeling method flow of the present invention is as follows: Figure 2 As shown:

[0084] Step 1: Scene Configuration:

[0085] Users input the type of combat mission (such as air defense, landing, anti-submarine), the number and type of heterogeneous combat units, and battlefield environment parameters (such as terrain and enemy situation) through the system interface. The environment modeling layer generates an initial simulation scenario based on the configuration information.

[0086] Step 2: Agent initialization:

[0087] Based on the type of combat unit, assign a corresponding intelligent agent model to each unit, initialize the parameters (such as weights and biases) of the actor and critic networks, and set training parameters such as learning rate and reward function coefficients.

[0088] Step 3: Simulation Interaction and Policy Learning

[0089] The environment modeling layer updates the battlefield status in real time and feeds it back to the agent decision-making layer; each agent generates an initial action strategy based on the current status; the cooperative game layer optimizes the strategy combination through a virtual leading agent and coordinates the actions of each unit; the performance evaluation layer calculates real-time performance indicators and generates reward signals to feed back to the agents, which update network parameters by improving the DDPG algorithm.

[0090] Step 4: Iterative optimization of the strategy:

[0091] Repeat step 3 until the agent's strategy converges (i.e., the performance index tends to stabilize) or the preset number of training iterations are reached, and output the optimal cooperative game strategy.

[0092] Step 5: Performance Evaluation and Results Output:

[0093] The optimal strategy is verified through simulation. The performance evaluation layer generates a comprehensive evaluation report, which users can use to adjust scenario parameters or training parameters and restart the simulation process.

[0094] 1. Improve the DDPG algorithm structure

[0095] The improved DDPG algorithm adopts an actor-critic dual-network structure, such as... Figure 3 As shown in the diagram, the actor network employs a 3-layer fully connected neural network, with the agent's state vector as input and the action policy as output. The critic network employs a 4-layer fully connected neural network, with the state vector and action vector as input and the Q-value (action value) of the policy as output. The agent's interaction experience (s, a, r, s') is stored in a priority experience replay pool, and high-priority experience samples (such as samples with larger reward values ​​or larger Q-value errors) are sampled first for training. A TD3 policy is used to delay updates to the critic network to avoid overestimation.

[0096] 2. Collaborative Game Strategy Optimization Process

[0097] The optimization process of collaborative game strategy is as follows: Figure 4 As shown, it specifically includes:

[0098] Policy generation phase: Each agent generates an initial policy P1, P2, ..., Pn based on its own state (n is the number of combat units).

[0099] Strategy evaluation phase: The virtual dominant agent collects local performance indicators corresponding to each strategy and calculates the global performance value E.

[0100] Strategy adjustment phase: If E does not reach the optimal threshold, the parameters of each strategy are adjusted based on the Nash equilibrium optimization algorithm to generate new strategies P1', P2', ..., Pn'.

[0101] Strategy output phase: Repeated evaluation and adjustment until E reaches its optimum, and output the final collaborative strategy.

[0102] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0103] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A simulation modeling method for intelligent collaborative game theory among heterogeneous combat units, characterized in that: It includes the following units: environment construction layer, agent decision-making layer, collaborative game layer, and performance evaluation layer.

2. The simulation modeling method for intelligent collaborative game theory of heterogeneous combat units according to claim 1, characterized in that, The environment construction layer is responsible for constructing a simulated combat environment and entities that closely resemble the real battlefield, providing an interactive scenario for intelligent agent decision-making and collaborative game. The environment construction layer includes: a combat entity modeling module, a battlefield environment modeling module, and an interaction rule modeling module. The combat entity modeling module constructs an entity model library based on the physical characteristics, functional parameters, and combat capabilities of heterogeneous combat units; it adopts an object-oriented design method to define attributes and behavioral interfaces for each entity. The battlefield environment modeling module is responsible for constructing models of the geographical environment, meteorological environment, electromagnetic environment, and enemy situation environment; and uses dynamic stochastic processes to simulate real-time changes in the environment. The interaction rule modeling module is used to define the interaction logic between combat entities, including detection rules, attack rules, communication rules, and coordination rules.

3. The simulation modeling method for intelligent collaborative game theory of heterogeneous combat units according to claim 2, characterized in that, The agent decision layer is responsible for constructing differentiated deep reinforcement learning decision models based on the functional differences of heterogeneous combat units, enabling autonomous decision-making for each unit. This layer configures an independent agent for each combat unit, consisting of an actor network and a critic network, and is trained using an improved DDPG algorithm. The design includes: state space design, action space design, and an improved DDPG algorithm. The state space design is used to define differentiated state vectors for different types of combat units; The action space design is used to design continuous or discrete action spaces according to the functional positioning of the combat unit; The improved DDPG algorithm introduces a priority experience replay mechanism on the basis of traditional DDPG.

4. The simulation modeling method for intelligent collaborative game theory of heterogeneous combat units according to claim 3, characterized in that, The collaborative game layer is responsible for coordinating the decision-making behavior of each agent to achieve global collaborative optimization of heterogeneous combat units. The collaborative game layer includes a game strategy generation module and a collaborative coordination module. The game strategy generation module is based on the fusion of non-cooperative and cooperative game theory, and constructs a game model of local autonomous decision-making and global collaborative optimization. Each agent generates an initial strategy based on its own state and local objectives. A virtual dominant agent is introduced to optimize and adjust the initial strategy with the goal of maximizing overall combat effectiveness. The virtual dominant agent collects state and policy information from each agent; The collaborative coordination module is used to resolve resource conflicts and task allocation problems among multiple agents, including: Resource coordination: A distributed resource scheduling algorithm is adopted to achieve dynamic resource sharing based on the remaining resources of each unit and the task requirements; Task collaboration: Based on an improved task allocation model using the Hungarian algorithm, the optimal allocation of tasks is achieved by combining the capability matching degree of each unit with the task priority. Communication and coordination: A layered communication protocol is adopted, with high-speed encrypted communication between core units and broadcast communication between edge units to reduce communication latency and data conflicts.

5. The simulation modeling method for intelligent collaborative game theory of heterogeneous combat units according to claim 4, characterized in that, The effectiveness evaluation layer is responsible for quantitatively evaluating the effectiveness of collaborative game strategies, providing feedback for agent training, and providing a basis for optimizing combat plans. The effectiveness evaluation layer includes: an indicator system construction module, a real-time evaluation module, and a result analysis module. The indicator system construction module is used to establish a multi-dimensional combat effectiveness indicator system, including core indicators, efficiency indicators, and stability indicators. The real-time evaluation module is used to collect status data and interaction data of each combat unit through sensor models during the simulation process. It uses a comprehensive evaluation algorithm that combines the analytic hierarchy process and the entropy weight method to calculate the effectiveness index value in real time and convert it into a reward signal for the agent, which is used to update the parameters of the reinforcement learning model. The results analysis module is used to generate an effectiveness evaluation report after the simulation, including indicator values, trend curves, and strategy optimization suggestions; by comparing the effectiveness indicators under different strategies, it provides decision support for combat commanders.

6. The simulation modeling method for intelligent collaborative game theory of heterogeneous combat units according to claim 5, characterized in that, The simulation modeling method includes the following steps: Step S1, Scene Configuration: The user inputs the combat mission type, the number and type of heterogeneous combat units, and battlefield environment parameters through the system interface. The environment modeling layer generates the initial simulation scene based on the configuration information. Step S2, Agent Initialization: Based on the type of combat unit, assign a corresponding agent model to each unit, initialize the parameters of the actor and critic networks, and set the learning rate, reward function coefficients, and training parameters. Step S3, Simulation Interaction and Strategy Learning: The environment modeling layer updates the battlefield state in real time and feeds it back to the agent decision-making layer; each agent generates an initial action strategy based on the current state; the collaborative game layer optimizes the strategy combination through a virtual leading agent and coordinates the actions of each unit; the performance evaluation layer calculates real-time performance indicators and generates reward signals to feed back to the agents; the agents update network parameters by improving the DDPG algorithm. Step S4, Strategy Iteration and Optimization: Repeat step S3 until the agent's strategy converges or the preset number of training iterations are reached, and output the optimal cooperative game strategy. Step S5, Performance Evaluation and Result Output: Simulation verification is performed on the optimal strategy. The performance evaluation layer generates a comprehensive evaluation report. Users can adjust the scenario parameters or training parameters according to the report and restart the simulation process.

7. The simulation modeling method for intelligent collaborative game theory of heterogeneous combat units according to claim 6, characterized in that, The improved DDPG algorithm structure is as follows: The improved DDPG algorithm adopts an actor-critic dual-network structure. The actor network uses a 3-layer fully connected neural network, with the agent's state vector as input and the action policy as output. The critic network uses a 4-layer fully connected neural network, with the state vector and action vector as input and the policy's Q-value as output. The interaction experience of the agent is stored in a priority experience replay pool, and high-priority experience samples are sampled first for training. The TD3 policy is used to delay the update of the critic network to avoid overestimation problems.

8. The simulation modeling method for intelligent collaborative game theory of heterogeneous combat units according to claim 7, characterized in that, The optimization steps for the cooperative game strategy are as follows: Policy generation phase: Each agent generates an initial policy based on its own state; Strategy evaluation phase: The virtual dominant agent collects local performance indicators corresponding to each strategy and calculates the global performance value E; Strategy adjustment phase: If E does not reach the optimal threshold, the parameters of each strategy are adjusted based on the Nash equilibrium optimization algorithm to generate a new strategy; Strategy output phase: Repeated evaluation and adjustment until E reaches its optimum, and output the final collaborative strategy.

Images