A dynamic game comprehensive evaluation method for system combat capability based on deep learning

Abstract

The present invention discloses a comprehensive evaluation method for the dynamic game of system combat capability based on deep learning, which relates to the technical field of combat capability evaluation. The present invention dynamically extracts key features of time and space in the battlefield environment through multi-sensor fusion technology combined with time-series convolutional networks and graph neural networks, updates the situation model in real time, and utilizes proximal strategy optimization PPO and transfer learning technology to accelerate the optimization process of unmanned cluster grouping and task allocation. By defining multi-task collaborative efficiency indicators, the adaptive adjustment of grouping schemes to complex task requirements is achieved; in terms of strategy optimization, a multi-agent game model driven by deep reinforcement learning is constructed, which comprehensively considers enemy and friendly resource allocation, dynamic environmental changes and uncertainty factors, effectively generates optimal strategies, and eliminates inefficient schemes through combat scenario simulation; at the same time, through real-time monitoring and dynamic planning mechanisms, it responds to emergencies such as target changes and emergency task insertion, thereby improving the continuity and collaborative efficiency of task execution.

Description

Technical Field

[0001] The present invention relates to the technical field of combat capability assessment, and in particular to a comprehensive assessment method of system combat capability dynamic game based on deep learning. Background Art

[0002] As modern battlefields accelerate their development towards intelligence and informatization, unmanned swarms, as key representatives of intelligent equipment, are playing an increasingly important role in reconnaissance and surveillance, target strike, and communication relay due to their high autonomy, flexibility, and scalability.

[0003] However, when existing dynamic game comprehensive evaluation methods are applied in dynamic environments, their combat capability evaluation models are mostly based on static assumptions, resulting in a lack of real-time response capabilities. In actual combat environments, information flow, command flow, and action flow are intertwined, and dynamic factors have a great impact on battlefield games. Traditional evaluation methods are difficult to reflect dynamic characteristics in a timely manner. Secondly, in multi-level, distributed games, traditional methods have high computational complexity and poor real-time performance, and cannot meet the needs of high-frequency decision-making. The game model's ability to optimize resource allocation and coordination strategies is also constrained, making it difficult to generate optimal solutions that adapt to complex task requirements.

[0004] At the same time, grouping and task allocation mainly rely on manual rules, lack intelligence and adaptability, and are unable to respond quickly to dynamic tasks; traditional methods have poor adaptability to new forces such as unmanned swarms, and cannot provide support for their autonomous coordination capabilities and strategy optimization in enemy-friendly interaction scenarios. Therefore, there is an urgent need for a comprehensive evaluation solution for system combat capability dynamic game based on deep learning to solve such problems. Summary of the Invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] The present invention provides a deep learning-based dynamic game comprehensive evaluation method for system combat capability to solve the problems of weak real-time performance of traditional methods, insufficient adaptability to intelligent unmanned clusters, and limited dynamic game and strategy optimization capabilities.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] The embodiment of the present invention provides a method for comprehensive evaluation of system combat capability based on deep learning dynamic game, which includes:

[0009] Step S1: collect battlefield environment data through multi-sensor fusion technology, and use temporal convolutional network and graph neural network models to convert the collected environment data into a real-time battlefield situation model;

[0010] Step S2: Using the proximal migration optimization (PPO) deep learning algorithm combined with migration learning technology, the unmanned swarm is grouped and optimized based on the battlefield situation model, and the grouping and task allocation plan of the unmanned swarm is dynamically generated;

[0011] Step S3: Based on the unmanned swarm formation and task allocation plan generated in step S2, a multi-agent game model driven by deep reinforcement learning is constructed. The game model comprehensively considers the resource allocation of both sides, the dynamic changes of the environment, and uncertainty factors, simulates the dynamic changes of the resource allocation and behavior of both sides, and generates the optimal strategy.

[0012] Step S4: Convert the optimal strategy generated in step S3 into specific execution instructions for the unmanned swarm, issue them to each combat unit in real time, and monitor the execution process in real time after issuance;

[0013] In step S5, based on the feedback data obtained from monitoring in step S4, the battlefield situation model, formation plan and game strategy are optimized, an iterative closed loop based on reinforcement learning is constructed, the model and strategy logic are updated, and the feedback data is used to optimize the multi-task collaborative efficiency indicators.

[0014] As an optimal solution of the method for comprehensive evaluation of system combat capability dynamic game based on deep learning described in the present invention, the battlefield environment data includes ground situation, air platform information and electromagnetic environment characteristics.

[0015] As a preferred solution of the method for comprehensive evaluation of system combat capability dynamic game based on deep learning described in the present invention, the step of converting the collected environmental data into a real-time battlefield situation model by using a temporal convolutional network and a graph neural network model is as follows:

[0016] The collected battlefield environment data is represented as X t :

[0017]

[0018] Among them, X t represents the multimodal input data set at time t, Represents the feature vector of ground situation data at time t, represents the feature vector of the aerial platform information at time t, represents the electromagnetic environment characteristic vector at time t,

[0019] To X t Perform standardization processing, the standardization formula is:

[0020]

[0021] Among them, x' t Represents the normalized input data, xt Represents the input data before normalization, μ represents the mean of the data feature, and σ represents the standard deviation of the data feature;

[0022] The one-dimensional temporal convolutional network is used to extract the dynamic features of the time dimension. The convolution calculation formula is:

[0023] H t =ReLU(W c *X t +b c ),

[0024] Among them, H t Represents the hidden state feature matrix after convolution, W c Represents the convolution kernel weight matrix, * represents the one-dimensional convolution operation, b c Represents the bias vector, ReLU represents the ReLU activation function,

[0025] Construct the graph structure of the battlefield environment, which is represented as:

[0026] G=(V,E),

[0027] Among them, G represents the battlefield map, V represents the set of graph nodes, which contains different sources of each feature, ground situation, air platform and electromagnetic environment nodes, and E represents the set of graph edges, which represents the association, communication relationship or geometric proximity relationship between nodes.

[0028] The adjacency matrix A defines the connectivity of the graph, and the feature update formula of the graph neural network is:

[0029] H′ t =softmax(AH t W g +b g ),

[0030] Among them, H' t represents the node feature matrix after the graph neural network is updated, A represents the adjacency matrix, and defines the connection relationship of the graph. t Represents the convolution feature matrix of the input, W g represents the weight matrix of the graph neural network, b g represents the bias vector,

[0031] Map the output features of the graph neural network to the battlefield situation model space through the fully connected layer:

[0032] M t =W m H′ t +b m ,

[0033] Among them, M t represents the battlefield situation model at time t, Wm Represents the weight matrix of the fully connected layer, H' t Represents the node feature matrix output by the graph neural network, b m Represents the bias vector.

[0034] As a preferred solution of the deep learning-based system combat capability dynamic game comprehensive evaluation method described in the present invention, wherein: in step S2, multi-task collaborative effectiveness indicators are defined, including completion rate, response time, and resource utilization;

[0035] During the formation optimization process of step S2, the coordination mode between the UAV and the ground unmanned platform is adjusted in real time.

[0036] As a preferred solution of the method for comprehensive evaluation of system combat capability based on deep learning in the present invention, the steps of using the proximal transfer optimization (PPO) deep learning algorithm combined with transfer learning technology to optimize the grouping of unmanned swarms based on the battlefield situation model and dynamically generate the grouping and task allocation plan of the unmanned swarm are as follows:

[0037] The battlefield situation model M generated by step S1 t As input, the optimization objective function J is constructed for the grouping and task allocation optimization of the unmanned swarm. The objective function is:

[0038] J=α·R c -β·T r +γ·U r ,

[0039] Among them, J represents the optimization target value, multi-task collaborative efficiency index, R c It represents the task completion rate, which is defined as the ratio of the number of completed tasks to the total number of tasks. r represents the task response time, which is defined as the weighted sum of the time required to complete the task, U r It represents resource utilization, which is defined as the ratio of actual resource usage to total resource capacity. α, β, and γ represent weight coefficients.

[0040] The initial formation of the unmanned swarm is represented as

[0041]

[0042] in, represents the initial grouping set, U g Represents the set of ground unmanned platforms, U a Represents a collection of unmanned aerial platforms;

[0043] The proximal strategy optimization PPO algorithm is combined with transfer learning technology for dynamic optimization. The optimization objective formula of PPO is:

[0044]

[0045] Among them, L(θ) represents the objective function of the strategy update, E t represents the expected value calculation at time t, r t (θ) represents the strategy probability ratio, which describes the ratio of the distribution of new and old strategies. Represents the advantage function, which is used to evaluate the pros and cons of the current strategy. Clip represents the clipping operation, which is defined as: if r t If (θ) exceeds the interval [1-∈, 1+∈], the endpoint value of the interval is taken, otherwise it remains unchanged, and ∈ represents the clipping threshold;

[0046] Based on the optimization results, the optimal unmanned cluster formation is dynamically generated and task allocation scheme

[0047] in, represents the optimal unmanned cluster formation scheme, Represents an optional unmanned cluster grouping set, represents the optimal task allocation solution, Represents all possible task allocation schemes, T r Indicates task response time;

[0048] Define the coordination mode function based on the battlefield situation model M t and optimal formation Derived collaborative model:

[0049]

[0050] in, represents the collaborative mode at the moment, ATT represents the collaborative mode derivation function based on the attention mechanism, and dynamically adjusts the communication and task collaboration of different unmanned platforms through weights.

[0051] The calculation formula of the attention mechanism is:

[0052]

[0053] Among them, Q, K, V represent the query matrix, key matrix and value matrix respectively, d k Indicates the dimensions of the bond matrix.

[0054] As a preferred solution of the method for comprehensive evaluation of system combat capability based on deep learning in the present invention, wherein: in step S3, a combat scenario simulation is performed based on the optimal strategy, the combat effectiveness under different strategies is evaluated, and inefficient strategies are eliminated;

[0055] The uncertainties include environmental disturbances, communication coordination obstacles, and enemy strategy adjustments.

[0056] As a preferred embodiment of the method for comprehensive evaluation of system combat capability dynamic game based on deep learning described in the present invention, the steps of synthesizing the resource allocation of both sides, dynamic changes in the environment, and uncertainty factors, simulating the dynamic changes in resource allocation and behavior of both sides, and generating the optimal strategy are as follows:

[0057] Define the input of the game model based on the battlefield situation model M t Express the resource status of both sides S t :

[0058]

[0059] Among them, S t represents the resource status at time t, Indicates our resources, including unmanned clusters and task status, Represents enemy resources, including enemy units and capability assessments,

[0060] Based on S t , establish a multi-agent game model and define the strategy set of both sides:

[0061] Π={π f ,π e},

[0062] Among them, Π represents the strategy set, π f represents our strategy, π e Indicates enemy strategy;

[0063] The goal of the agent is to optimize its own strategy to maximize profit or minimize loss.

[0064] Define the profit function U to evaluate the effect of the strategy:

[0065]

[0066] Among them, U f Represents our profit function, U e represents the enemy’s payoff function, Indicates our task completion rate, represents the enemy's mission completion rate, C f Indicates our resource consumption, C e represents the enemy's resource consumption, and λ represents the resource consumption weight coefficient;

[0067] In view of the dynamic changes in the battlefield environment, the state transfer function is used to represent the game process, which is expressed as:

[0068] St+1 =f(S t ,π f ,π e ,ΔE),

[0069] Among them, S t+1 represents the resource state at the next moment, f represents the state transition function, ΔE represents the dynamic disturbance, including communication barriers and environmental changes,

[0070] Using deep reinforcement learning methods, we can solve the optimal strategy of the game model:

[0071]

[0072] in, represents our optimal strategy, represents the enemy's optimal strategy, E represents the expected value calculation,

[0073] Using the value function V(S t ) to optimize the strategy, the value function is defined as:

[0074]

[0075] Among them, V(S t ) indicates that in state S t The cumulative revenue under , γ represents the discount factor, weighing the current revenue and future revenue, and T represents the maximum number of steps in the game process.

[0076] As a preferred solution of the method for comprehensive evaluation of system combat capability based on deep learning in the present invention, wherein: in step S4, the execution process is monitored to obtain feedback data, including task completion status, resource consumption and execution effect;

[0077] In step S4, during the execution process, the strategy is adjusted according to unexpected events, including target changes or insertion of new tasks.

[0078] As a preferred solution of the method for comprehensive evaluation of system combat capability dynamic game based on deep learning described in the present invention, wherein: during the execution process, the step of adjusting the strategy for emergencies is:

[0079] Based on the optimal strategy generated in step S3 Issue specific execution instructions C t :

[0080]

[0081] Among them, C t represents the specific execution instruction at time t, g represents the instruction generation function, represents our optimal strategy, Mt represents the battlefield situation model at time t,

[0082] During the execution of the instruction, the feedback data F t :

[0083] F t ={P t ,R t ,E t},

[0084] Among them, F t represents the feedback data set at time t, P t Indicates the task completion status, R t Indicates resource consumption, E t Indicates the execution effect;

[0085] Detect unexpected events ΔT by dynamically evaluating feedback data t :

[0086] ΔT t =h(F t ,δ),

[0087] Where ΔT t represents the emergency event at time t, h represents the emergency event detection function, F t Represents feedback data, δ represents the event threshold, which is used to determine whether an abnormality occurs;

[0088] Emergencies include: adjustments to the number or location of mission targets, and the addition of new urgent tasks during mission execution;

[0089] After detecting an emergency, dynamic programming is used to adjust the strategy. The adjustment formula is:

[0090]

[0091] in, represents the optimal strategy after adjustment, π represents all possible strategies, E represents the expected value calculation, U f represents the profit function, ΔT t Indicates an emergency condition;

[0092] Generate new execution instructions based on the adjusted strategy:

[0093]

[0094] Among them, C' t Indicates the specific execution instructions after adjustment. Represents the optimal strategy M after adjustment t Represents the current battlefield situation model, ΔT tIndicates the conditions of an emergency.

[0095] Repeat the monitoring, testing, and adjustment process until the mission is completed or the battlefield situation changes fundamentally.

[0096] As a preferred solution of the method for comprehensive evaluation of system combat capability dynamic game based on deep learning described in the present invention, the step of optimizing the battlefield situation model, formation scheme and game strategy is as follows:

[0097] Collect the feedback data monitored in step S4 and integrate them into a unified data set F:

[0098]

[0099] Among them, F represents the total set of feedback data, F t Represents the feedback data at time t, including the task completion status P t , resource consumption R t , Execution Effect E t , T represents the total time length of the feedback cycle,

[0100] According to the feedback data F, the battlefield situation model M is updated using the recursive function t Optimize, the optimization formula is:

[0101] M t+1 =φ(M t ,F t ),

[0102] Among them, M t+1 represents the updated battlefield situation model, φ represents the update function of the situation model, and adjusts the feature weights in the model based on the feedback data. t Represents the current situation model, F t represents the feedback data at time t,

[0103] The core of the situation model update is the feature weight adjustment function, which is expressed as:

[0104] w i,t+1 =w i,t +η·Δw i,t ,

[0105] Among them, w i,t+1 represents the updated i-th feature weight, w i,t represents the current i-th feature weight, η represents the learning rate, Δw i,t Indicates the adjusted value of the feature, calculated from the feedback data,

[0106] Optimize the unmanned swarm formation plan based on the updated results of the situation model

[0107]

[0108] in, It represents the optimized unmanned cluster formation plan. represents the set of all possible grouping schemes, J represents the multi-task collaborative effectiveness index function, which is calculated based on feedback data and the latest situation model.

[0109] Using deep reinforcement learning technology to improve game strategies and Perform iterative optimization:

[0110]

[0111] in, represents our optimal strategy after update, represents the updated enemy's optimal strategy, S t Indicates the current state, U f ,U e Denote our and enemy's profit functions respectively,

[0112] Utilize the optimized battlefield situation model, formation plan, and game strategy to form a closed-loop iteration:

[0113]

[0114] in, represents the optimized dynamic closed loop set, M t+1 represents the optimized battlefield situation model, It represents the optimized unmanned cluster formation plan. Represents the optimal strategy after optimization.

[0115] The beneficial effects of the present invention are as follows: the present invention, through the combination of multi-sensor fusion technology with temporal convolutional networks and graph neural networks, dynamically extracts key features of time and space in the battlefield environment, updates the situation model in real time, improves the accuracy of data processing and the real-time performance of the model, and uses proximal strategy optimization PPO and transfer learning technology to accelerate the optimization process of unmanned cluster grouping and task allocation, and realizes the adaptive adjustment of grouping schemes to complex task requirements by defining multi-task collaborative efficiency indicators; in terms of strategy optimization, a multi-agent game model driven by deep reinforcement learning is constructed, which comprehensively considers enemy and friendly resource allocation, dynamic environmental changes and uncertainty factors, effectively generates optimal strategies and eliminates inefficient schemes through combat scenario simulation, thereby enhancing the applicability of decision-making; at the same time, through real-time monitoring and dynamic planning mechanisms, it responds to emergencies such as target changes and emergency task insertion, improves the continuity and collaborative efficiency of task execution, and feedback closed-loop optimization dynamically updates the battlefield situation model, grouping scheme and game strategy through feature weight adjustment and deep reinforcement learning iterative mechanism, and gradually improves the adaptability and robustness of the evaluation system.

[0116] In summary, the present invention effectively solves the disadvantages of traditional methods in real-time, adaptability, uncertainty processing and intelligent collaboration, and can comprehensively improve the collaborative combat capabilities of unmanned clusters in complex and changeable battlefield environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0117] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0118] Figure 1 This is a flow chart of the deep learning-based comprehensive evaluation method for system combat capability dynamic game. DETAILED DESCRIPTION

[0119] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0120] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0121] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0122] Example 1, with reference to Figure 1 This embodiment provides a method for comprehensive evaluation of system combat capability based on deep learning dynamic game theory, including the following steps:

[0123] Step S1: collect battlefield environment data through multi-sensor fusion technology, and use temporal convolutional network and graph neural network models to convert the collected environment data into a real-time battlefield situation model;

[0124] Battlefield environment data includes ground situation, air platform information and electromagnetic environment characteristics;

[0125] The steps of converting the collected environmental data into a real-time battlefield situation model using a temporal convolutional network and a graph neural network model are as follows:

[0126] The collected battlefield environment data is represented as X t :

[0127]

[0128] Among them, X t represents the multimodal input data set at time t, Represents the feature vector of ground situation data at time t, represents the feature vector of the aerial platform information at time t, represents the electromagnetic environment characteristic vector at time t,

[0129] To X t Perform standardization processing, the standardization formula is:

[0130]

[0131] Among them, x' t Represents the normalized input data, x t Represents the input data before normalization, μ represents the mean of the data feature, and σ represents the standard deviation of the data feature;

[0132] The one-dimensional temporal convolutional network is used to extract the dynamic features of the time dimension. The convolution calculation formula is:

[0133] H t =ReLU(W c *X t +b c ),

[0134] Among them, H t Represents the hidden state feature matrix after convolution, W c Represents the convolution kernel weight matrix, * represents the one-dimensional convolution operation, b c Represents the bias vector, ReLU represents the ReLU activation function,

[0135] Construct the graph structure of the battlefield environment, which is represented as:

[0136] G=(V,E),

[0137] Among them, G represents the battlefield map, V represents the set of graph nodes, which contains different sources of each feature, ground situation, air platform and electromagnetic environment nodes, and E represents the set of graph edges, which represents the association, communication relationship or geometric proximity relationship between nodes.

[0138] The adjacency matrix A defines the connectivity of the graph, and the feature update formula of the graph neural network is:

[0139] H t =softmax(AH t W g +b g ),

[0140] Among them, H' t represents the node feature matrix after the graph neural network is updated, A represents the adjacency matrix, and defines the connection relationship of the graph. t Represents the convolution feature matrix of the input, W g represents the weight matrix of the graph neural network, b g represents the bias vector,

[0141] Map the output features of the graph neural network to the battlefield situation model space through the fully connected layer:

[0142] M t =W m H′ t +b m ,

[0143] Among them, M t represents the battlefield situation model at time t, W m Represents the weight matrix of the fully connected layer, H' t Represents the node feature matrix output by the graph neural network, b m represents the bias vector;

[0144] Specifically, multimodal data processing and joint modeling technology are carried out here, the dynamic features of the time dimension are extracted using a temporal convolutional network, and the correlation between battlefield nodes is captured through a graph neural network.

[0145] Step S2: Using the proximal migration optimization (PPO) deep learning algorithm combined with migration learning technology, the unmanned swarm is grouped and optimized based on the battlefield situation model, and the grouping and task allocation plan of the unmanned swarm is dynamically generated;

[0146] In step S2, multi-task collaborative performance indicators are defined, including completion rate, response time, and resource utilization;

[0147] During the grouping optimization process of step S2, the coordination mode between the UAV and the ground unmanned platform is adjusted in real time;

[0148] By using the proximal migration optimization PPO deep learning algorithm combined with transfer learning technology, the unmanned swarm is optimized based on the battlefield situation model. The steps of dynamically generating the unmanned swarm grouping and task allocation plan are as follows:

[0149] The battlefield situation model M generated by step S1 t As input, the optimization objective function J is constructed for the grouping and task allocation optimization of the unmanned swarm. The objective function is:

[0150] J=α·R c -β·T r +γ·U r ,

[0151] Among them, J represents the optimization target value, multi-task collaborative efficiency index, R c It represents the task completion rate, which is defined as the ratio of the number of completed tasks to the total number of tasks. r represents the task response time, which is defined as the weighted sum of the time required to complete the task, U r It represents resource utilization, which is defined as the ratio of actual resource usage to total resource capacity. α, β, and γ represent weight coefficients.

[0152] The initial formation of the unmanned swarm is represented as

[0153]

[0154] in, represents the initial grouping set, U g Represents the set of ground unmanned platforms, U a Represents a collection of unmanned aerial platforms;

[0155] The proximal strategy optimization PPO algorithm is combined with transfer learning technology for dynamic optimization. The optimization objective formula of PPO is:

[0156]

[0157] Among them, L(θ) represents the objective function of the strategy update, E trepresents the expected value calculation at time t, r t (θ) represents the strategy probability ratio, which describes the ratio of the distribution of new and old strategies. Represents the advantage function, which is used to evaluate the pros and cons of the current strategy. Clip represents the clipping operation, which is defined as: if r t If (θ) exceeds the interval [1-∈, 1+∈], the endpoint value of the interval is taken, otherwise it remains unchanged, and ∈ represents the clipping threshold;

[0158] Based on the optimization results, the optimal unmanned cluster formation is dynamically generated and task allocation scheme

[0159] in, represents the optimal unmanned cluster formation scheme, Represents an optional unmanned cluster grouping set, represents the optimal task allocation solution, Represents all possible task allocation schemes, T r Indicates task response time;

[0160] Define the coordination mode function based on the battlefield situation model M t and optimal formation Derived collaborative model:

[0161]

[0162] in, represents the collaborative mode at time t, ATT represents the collaborative mode derivation function based on the attention mechanism, and dynamically adjusts the communication and task collaboration of different unmanned platforms through weights.

[0163] The calculation formula of the attention mechanism is:

[0164]

[0165] Among them, Q, K, V represent the query matrix, key matrix and value matrix respectively, d k represents the dimension of the bond matrix;

[0166] Specifically, the objective function J is used to optimize the grouping and task allocation scheme of the unmanned cluster, and the PPO algorithm is used to dynamically update the strategy. At the same time, a collaborative mode derivation function based on the attention mechanism is introduced to enhance the task collaboration capability between unmanned platforms, enabling them to have higher adaptability and collaborative efficiency in complex battlefield environments.

[0167] Step S3: Based on the unmanned swarm formation and task allocation plan generated in step S2, a multi-agent game model driven by deep reinforcement learning is constructed. The game model comprehensively considers the resource allocation of both sides, the dynamic changes of the environment, and uncertainty factors, simulates the dynamic changes of the resource allocation and behavior of both sides, and generates the optimal strategy.

[0168] In step S3, a combat scenario simulation is performed based on the optimal strategy to evaluate the combat effectiveness under different strategies and eliminate inefficient strategies;

[0169] Uncertainty factors include environmental disturbances, communication coordination barriers, and enemy strategy adjustments;

[0170] Taking into account the resource allocation, dynamic changes in the environment, and uncertainty factors of both sides, the steps to generate the optimal strategy are as follows:

[0171] Define the input of the game model based on the battlefield situation model M t Express the resource status of both sides S t :

[0172]

[0173] Among them, S t represents the resource status at time t, Indicates our resources, including unmanned clusters and task status, Represents enemy resources, including enemy units and capability assessments,

[0174] Based on S t , establish a multi-agent game model and define the strategy set of both sides:

[0175] Π={π f ,π e},

[0176] Among them, Π represents the strategy set, π f represents our strategy, π e Indicates enemy strategy;

[0177] The goal of the agent is to optimize its own strategy to maximize profit or minimize loss.

[0178] Define the profit function U to evaluate the effect of the strategy:

[0179]

[0180] Among them, U f Represents our profit function, U e represents the enemy’s payoff function, Indicates our task completion rate, represents the enemy's mission completion rate, C f Indicates our resource consumption, C e represents the enemy's resource consumption, and λ represents the resource consumption weight coefficient;

[0181] In view of the dynamic changes in the battlefield environment, the state transfer function is used to represent the game process, which is expressed as:

[0182] S t+1 =f(S t ,π f ,π e ,ΔE),

[0183] Among them, S t+1 represents the resource state at the next moment, f represents the state transition function, ΔE represents the dynamic disturbance, including communication barriers and environmental changes,

[0184] Using deep reinforcement learning methods, we can solve the optimal strategy of the game model:

[0185]

[0186] in, represents our optimal strategy, represents the enemy's optimal strategy, E represents the expected value calculation,

[0187] Using the value function V(S t ) to optimize the strategy, the value function is defined as:

[0188]

[0189] Among them, V(S t ) indicates that in state S t The cumulative revenue under the game is , γ represents the discount factor, weighing the current revenue and future revenue, and T represents the maximum number of steps in the game process;

[0190] Specifically, deep reinforcement learning optimization strategy is used here to solve decision-making problems in complex dynamic environments;

[0191] Step S4: Convert the optimal strategy generated in step S3 into specific execution instructions for the unmanned swarm, issue them to each combat unit in real time, and monitor the execution process in real time after issuance;

[0192] In step S4, the execution process is monitored to obtain feedback data, including task completion status, resource consumption, and execution effect;

[0193] In step S4, during the execution process, the strategy is adjusted according to unexpected events, such as target changes or the insertion of new tasks;

[0194] During the implementation process, the steps to adjust the strategy for emergencies are:

[0195] Based on the optimal strategy generated in step S3 Issue specific execution instructions C t :

[0196]

[0197] Among them, C t represents the specific execution instruction at time t, g represents the instruction generation function, represents our optimal strategy, M t represents the battlefield situation model at time t,

[0198] During the execution of the instruction, the feedback data F t :

[0199] F t ={P t ,R t ,E t},

[0200] Among them, F t represents the feedback data set at time t, P t Indicates the task completion status, R t Indicates resource consumption, E t Indicates the execution effect;

[0201] Detect unexpected events ΔT by dynamically evaluating feedback data t :

[0202] ΔT t =h(F t ,δ),

[0203] Where ΔT t represents the emergency event at time t, h represents the emergency event detection function, F t Represents feedback data, δ represents the event threshold, which is used to determine whether an abnormality occurs;

[0204] Emergencies include: adjustments to the number or location of mission targets, and the addition of new urgent tasks during mission execution;

[0205] After detecting an emergency, dynamic programming is used to adjust the strategy. The adjustment formula is:

[0206]

[0207] in, represents the optimal strategy after adjustment, π represents all possible strategies, E represents the expected value calculation, U f represents the profit function, ΔTt Indicates an emergency condition;

[0208] Generate new execution instructions based on the adjusted strategy:

[0209]

[0210] Among them, C' t Indicates the specific execution instructions after adjustment. Represents the optimal strategy M after adjustment t Represents the current battlefield situation model, ΔT t Indicates the conditions of an emergency.

[0211] Repeat the monitoring, testing, and adjustment process until the mission is completed or the battlefield situation changes fundamentally;

[0212] Specifically, by monitoring feedback data in real time, detecting emergencies, and combining dynamic planning and strategy updates to generate adjusted optimal instructions, this step strengthens the unmanned swarm's ability to respond to emergencies, allowing it to maintain the continuity of task execution in a highly dynamic environment.

[0213] Step S5: Based on the feedback data obtained from the monitoring in step S4, the battlefield situation model, formation plan, and game strategy are optimized, an iterative closed loop based on reinforcement learning is constructed, the model and strategy logic are updated, and the feedback data is used to optimize the multi-task collaborative effectiveness indicators;

[0214] The steps to optimize the battlefield situation model, formation plan and game strategy are as follows:

[0215] Collect the feedback data monitored in step S4 and integrate them into a unified data set F:

[0216]

[0217] Among them, F represents the total set of feedback data, F t Represents the feedback data at time t, including the task completion status P t , resource consumption R t , Execution Effect E t , T represents the total time length of the feedback cycle,

[0218] According to the feedback data F, the battlefield situation model M is updated using the recursive function t Optimize, the optimization formula is:

[0219] M t+1 =φ(M t ,F t ),

[0220] Among them, M t+1represents the updated battlefield situation model, φ represents the update function of the situation model, and adjusts the feature weights in the model based on the feedback data. t Represents the current situation model, F t represents the feedback data at time t,

[0221] The core of the situation model update is the feature weight adjustment function, which is expressed as:

[0222] w i,t+1 =w i,t +η·Δw i,t ,

[0223] Among them, w i,t+1 represents the updated i-th feature weight, w i,t represents the current i-th feature weight, η represents the learning rate, Δw i,t Indicates the adjusted value of the feature, calculated from the feedback data,

[0224] Optimize the unmanned swarm formation plan based on the updated results of the situation model

[0225]

[0226] in, It represents the optimized unmanned cluster formation plan. represents the set of all possible grouping schemes, J represents the multi-task collaborative effectiveness index function, which is calculated based on feedback data and the latest situation model.

[0227] Using deep reinforcement learning technology to improve game strategies and Perform iterative optimization:

[0228]

[0229] in, represents our optimal strategy after update, represents the updated enemy's optimal strategy, S t Indicates the current state, U f ,U e Denote our and enemy's profit functions respectively,

[0230] Utilize the optimized battlefield situation model, formation plan, and game strategy to form a closed-loop iteration:

[0231]

[0232] in, represents the optimized dynamic closed loop set, M t+1 represents the optimized battlefield situation model, It represents the optimized unmanned cluster formation plan. represents the optimal strategy after optimization;

[0233] Specifically, in this step, a dynamic closed loop of battlefield situation model, unmanned cluster formation and game strategy is formed through the optimization process driven by feedback data. The feature weight adjustment mechanism is used to improve the ability to describe environmental dynamics, and the formation plan is dynamically optimized in combination with the latest situation and task feedback.

[0234] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A deep learning-based dynamic game-based comprehensive evaluation method for system combat capability, characterized by: include, Step S1: collect battlefield environment data through multi-sensor fusion technology, and use temporal convolutional network and graph neural network models to convert the collected environment data into a real-time battlefield situation model; Step S2: Using the proximal migration optimization (PPO) deep learning algorithm combined with migration learning technology, the unmanned swarm is grouped and optimized based on the battlefield situation model, and the grouping and task allocation plan of the unmanned swarm is dynamically generated; Step S3: Based on the unmanned swarm formation and task allocation plan generated in step S2, a multi-agent game model driven by deep reinforcement learning is constructed. The game model comprehensively considers the resource allocation of both sides, the dynamic changes of the environment, and uncertainty factors, simulates the dynamic changes of the resource allocation and behavior of both sides, and generates the optimal strategy. Step S4: Convert the optimal strategy generated in step S3 into specific execution instructions for the unmanned swarm, issue them to each combat unit in real time, and monitor the execution process in real time after issuance; In step S5, based on the feedback data obtained from monitoring in step S4, the battlefield situation model, formation plan and game strategy are optimized, an iterative closed loop based on reinforcement learning is constructed, the model and strategy logic are updated, and the feedback data is used to optimize the multi-task collaborative efficiency indicators.

2. The method for comprehensive evaluation of system combat capability based on deep learning and dynamic game theory as claimed in claim 1, characterized in that: The battlefield environment data includes ground situation, air platform information and electromagnetic environment characteristics.

3. The method for comprehensive evaluation of system combat capability based on deep learning and dynamic game theory as claimed in claim 2, characterized in that: The steps of converting the collected environmental data into a real-time battlefield situation model using a temporal convolutional network and a graph neural network model are as follows: The collected battlefield environment data is represented as X t : Among them, X t represents the multimodal input data set at time t, Represents the feature vector of ground situation data at time t, represents the feature vector of the aerial platform information at time t, represents the electromagnetic environment characteristic vector at time t, To X t Perform standardization processing, the standardization formula is: Among them, x' t Represents the normalized input data, x t Represents the input data before normalization, μ represents the mean of the data feature, and σ represents the standard deviation of the data feature; The one-dimensional temporal convolutional network is used to extract the dynamic features of the time dimension. The convolution calculation formula is: H t =ReLU(W c *X t +b c ), Among them, H t Represents the hidden state feature matrix after convolution, W c Represents the convolution kernel weight matrix, * represents the one-dimensional convolution operation, b c Represents the bias vector, ReLU represents the ReLU activation function, Construct the graph structure of the battlefield environment, which is represented as: G=(V,E), Among them, G represents the battlefield map, V represents the set of graph nodes, which contains different sources of each feature, and E represents the set of graph edges, which represents the relationship between nodes. The adjacency matrix A defines the connectivity of the graph, and the feature update formula of the graph neural network is: H′ t =softmax(AH t W g +b g ), Among them, H' t represents the node feature matrix after the graph neural network is updated, A represents the adjacency matrix, and defines the connection relationship of the graph. t Represents the convolution feature matrix of the input, W g represents the weight matrix of the graph neural network, b g represents the bias vector, Map the output features of the graph neural network to the battlefield situation model space through the fully connected layer: M t =W m H′ t +b m , Among them, M t represents the battlefield situation model at time t, W m Represents the weight matrix of the fully connected layer, H' t Represents the node feature matrix output by the graph neural network, b m Represents the bias vector.

4. The method for comprehensive evaluation of system combat capability based on deep learning and dynamic game theory as claimed in claim 3, characterized in that: In step S2, multi-task collaborative performance indicators are defined, including completion rate, response time, and resource utilization; During the formation optimization process of step S2, the coordination mode between the UAV and the ground unmanned platform is adjusted in real time.

5. The method for comprehensive evaluation of system combat capability based on deep learning and dynamic game theory as claimed in claim 4, characterized in that: The steps of using the proximal transfer optimization PPO deep learning algorithm combined with transfer learning technology to optimize the grouping of unmanned clusters based on the battlefield situation model and dynamically generate the grouping and task allocation plan of the unmanned cluster are as follows: The battlefield situation model M generated by step S1 t As input, the optimization objective function J is constructed for the grouping and task allocation optimization of the unmanned swarm. The objective function is: J=α·R c -β·T r +γ·U r , Among them, J represents the optimization target value, multi-task collaborative efficiency index, R c It represents the task completion rate, which is defined as the ratio of the number of completed tasks to the total number of tasks. r represents the task response time, which is defined as the weighted sum of the time required to complete the task, U r It represents resource utilization, which is defined as the ratio of actual resource usage to total resource capacity. α, β, and γ represent weight coefficients. The initial formation of the unmanned swarm is represented as in, represents the initial grouping set, U g Represents the set of ground unmanned platforms, U a Represents a collection of unmanned aerial platforms; The proximal strategy optimization PPO algorithm is combined with transfer learning technology for dynamic optimization. The optimization objective formula of PPO is: Among them, L(θ) represents the objective function of the strategy update, E t represents the expected value calculation at time t, r t (θ) represents the strategy probability ratio, which describes the ratio of the distribution of new and old strategies. Represents the advantage function, which is used to evaluate the pros and cons of the current strategy. Clip represents the clipping operation, which is defined as: if r t If (θ) exceeds the interval [1-∈, 1+∈], the endpoint value of the interval is taken, otherwise it remains unchanged, and ∈ represents the clipping threshold; Based on the optimization results, the optimal unmanned cluster formation is dynamically generated and task allocation scheme in, represents the optimal unmanned cluster formation scheme, Represents an optional unmanned cluster grouping set, represents the optimal task allocation solution, Represents all possible task allocation schemes, T r Indicates task response time; Define the coordination mode function based on the battlefield situation model M t and optimal formation Derived collaborative model: in, represents the collaborative mode at time t, ATT represents the collaborative mode derivation function based on the attention mechanism, and dynamically adjusts the communication and task collaboration of different unmanned platforms through weights. The calculation formula of the attention mechanism is: Among them, Q, K, V represent the query matrix, key matrix and value matrix respectively, d k Indicates the dimensions of the bond matrix.

6. The method for comprehensive evaluation of system combat capability based on deep learning and dynamic game theory as claimed in claim 5, characterized in that: In step S3, a combat scenario simulation is performed based on the optimal strategy to evaluate the combat effectiveness under different strategies and eliminate inefficient strategies; The uncertainties include environmental disturbances, communication coordination obstacles, and enemy strategy adjustments.

7. The method for comprehensive evaluation of system combat capability based on deep learning and dynamic game theory as claimed in claim 6, characterized in that: The steps of synthesizing the resource allocation of both sides, the dynamic changes of the environment and the uncertainty factors, simulating the resource allocation and dynamic changes of the behavior of both sides and generating the optimal strategy are as follows: Define the input of the game model based on the battlefield situation model M t Express the resource status of both sides S t : Among them, S t represents the resource status at time t, Indicates our resources, including unmanned clusters and task status, Represents enemy resources, including enemy units and capability assessments, Based on S t , establish a multi-agent game model and define the strategy set of both sides: Π={π f ,p e }, Among them, Π represents the strategy set, π f represents our strategy, π e Indicates enemy strategy; The goal of the agent is to optimize its own strategy to maximize profit or minimize loss. Define the profit function U to evaluate the effectiveness of the strategy: Among them, U f Represents our profit function, U e represents the enemy’s payoff function, Indicates our task completion rate, represents the enemy's mission completion rate, C f Indicates our resource consumption, C e represents the enemy's resource consumption, and λ represents the resource consumption weight coefficient; In view of the dynamic changes in the battlefield environment, the state transfer function is used to represent the game process, which is expressed as: S t+1 =f(S t ,p f ,p e ,ΔE), Among them, S t+1 represents the resource state at the next moment, f represents the state transition function, ΔE represents the dynamic disturbance, including communication barriers and environmental changes, Using deep reinforcement learning methods, we can solve the optimal strategy of the game model: in, represents our optimal strategy, represents the enemy's optimal strategy, E represents the expected value calculation, Utilize the value function V(S t ) to optimize the strategy, the value function is defined as: Among them, V(S t ) indicates that in state S t The cumulative revenue under , γ represents the discount factor, weighing the current revenue and future revenue, and T represents the maximum number of steps in the game process.

8. The method for comprehensive evaluation of system combat capability based on dynamic game theory based on deep learning as claimed in claim 7, characterized in that: In step S4, the execution process is monitored to obtain feedback data, including task completion status, resource consumption, and execution effect; In step S4, during the execution process, the strategy is adjusted according to unexpected events, including target changes or insertion of new tasks.

9. The method for comprehensive evaluation of system combat capability based on deep learning and dynamic game theory as claimed in claim 8, characterized in that: During the execution process, the steps for adjusting the strategy for emergencies are: Based on the optimal strategy generated in step S3 Issue specific execution instructions C t : Among them, C t represents the specific execution instruction at time t, g represents the instruction generation function, represents our optimal strategy, M t represents the battlefield situation model at time t, During the execution of the instruction, the feedback data F t : F t ={P t ,R t ,E t }, Among them, F t represents the feedback data set at time t, P t Indicates the task completion status, R t Indicates resource consumption, E t Indicates the execution effect; Detect unexpected events ΔT by dynamically evaluating feedback data t : ΔT t =h(F t ,δ), Where, ΔT t represents the emergency event at time t, h represents the emergency event detection function, F t Represents feedback data, δ represents the event threshold, which is used to determine whether an abnormality occurs; Emergencies include: adjustments to the number or location of mission targets, and the addition of new urgent tasks during mission execution; After detecting an emergency, dynamic programming is used to adjust the strategy. The adjustment formula is: in, represents the optimal strategy after adjustment, π represents all possible strategies, E represents the expected value calculation, U f represents the profit function, ΔT t Indicates an emergency condition; Generate new execution instructions based on the adjusted strategy: Among them, C' t Indicates the specific execution instructions after adjustment. Represents the optimal strategy M after adjustment t Represents the current battlefield situation model, ΔT t Indicates the conditions of an emergency. Repeat the monitoring, testing, and adjustment process until the mission is completed or the battlefield situation changes fundamentally.

10. A method for comprehensive evaluation of system combat capability based on deep learning and dynamic game theory as claimed in claim 9, characterized in that: The steps of optimizing the battlefield situation model, formation plan and game strategy are as follows: Collect the feedback data monitored in step S4 and integrate them into a unified data set F: Among them, F represents the total set of feedback data, F t Represents the feedback data at time t, including the task completion status P t , resource consumption R t , Execution Effect E t , T represents the total time length of the feedback cycle, According to the feedback data F, the battlefield situation model M is updated using the recursive function t Optimize, the optimization formula is: M t+1 =φ(M t ,F t ), Among them, M t+1 represents the updated battlefield situation model, φ represents the update function of the situation model, and adjusts the feature weights in the model based on the feedback data. t Represents the current situation model, F t represents the feedback data at time t, The core of the situation model update is the feature weight adjustment function, which is expressed as: In i,t+1 =in i,t +η Δw i,t , Among them, w i,t+1 represents the updated i-th feature weight, w i,t represents the current i-th feature weight, η represents the learning rate, Δw i,t Indicates the adjusted value of the feature, calculated from the feedback data, Optimize the unmanned swarm formation plan based on the updated results of the situation model in, It represents the optimized unmanned cluster formation plan. represents the set of all possible grouping schemes, J represents the multi-task collaborative effectiveness index function, which is calculated based on feedback data and the latest situation model. Using deep reinforcement learning technology to improve game strategies and Perform iterative optimization: in, represents our optimal strategy after update, represents the updated enemy's optimal strategy, S t Indicates the current state, U f ,U e Denote our and enemy's profit functions respectively, Utilize the optimized battlefield situation model, formation plan, and game strategy to form a closed-loop iteration: in, Represents the optimized dynamic closed loop set, M t+1 represents the optimized battlefield situation model, It represents the optimized unmanned cluster formation plan. Represents the optimal strategy after optimization.

Images