System combat ability dynamic game comprehensive evaluation method based on deep learning

Abstract

The invention discloses a system combat ability dynamic game comprehensive evaluation method based on deep learning, and relates to the technical field of combat ability evaluation.According to the method, key features of time and space in a battlefield environment are dynamically extracted by combining a multi-sensor fusion technology with a time sequence convolutional network and a graph neural network, a situation model is updated in real time, and the dynamic game of the system combat ability is comprehensively evaluated. The optimization process of unmanned cluster marshalling and task allocation is accelerated by utilizing a near-end strategy optimization PPO and transfer learning technology, and adaptive adjustment of a marshalling scheme on complex task requirements is realized by defining a multi-task collaborative efficiency index; in the aspect of strategy optimization, a multi-agent game model driven by deep reinforcement learning is constructed, friend or foe resource allocation, dynamic environment change and uncertainty factors are integrated, an optimal strategy is effectively generated, and low-efficiency schemes are eliminated through combat scene simulation; and meanwhile, through a real-time monitoring and dynamic planning mechanism, emergencies such as target change and emergency task insertion are dealt with, and the task execution continuity and cooperative efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of combat capability evaluation, and particularly to a dynamic game comprehensive evaluation method for system combat capability based on deep learning. Background Art

[0002] With the accelerating development of the modern battlefield towards intelligence and informatization, the unmanned cluster, as a key representative of intelligent equipment, is playing an increasingly important role in the fields of reconnaissance and surveillance, target strike, and communication relay due to its high autonomy, flexibility, and scalability.

[0003] When the existing dynamic game comprehensive evaluation methods are applied in a dynamic environment, since most of their combat capability evaluation models are based on static assumptions, they lack real-time response capabilities; in the actual combat environment, the information flow, command flow, and action flow are intertwined, and dynamic factors have a great impact on battlefield games. Traditional evaluation methods are difficult to reflect dynamic characteristics in a timely manner; secondly, in multi-level and distributed games, traditional methods have high computational complexity and poor real-time performance, unable to meet the needs of high-frequency decision-making. The optimization capabilities of game models for resource allocation and collaborative strategies are also restricted, and it is difficult to generate optimal solutions that adapt to complex task requirements.

[0004] At the same time, the formation and task allocation mainly rely on manual rules, with insufficient intelligence and adaptability, and it is difficult to quickly respond to dynamic tasks; traditional methods have poor adaptability to new forces such as unmanned clusters and cannot provide support for their autonomous cooperation capabilities and strategy optimization in the scenario of interaction between the enemy and us. Therefore, there is an urgent need for a dynamic game comprehensive evaluation scheme for system combat capability based on deep learning to solve such problems. Summary of the Invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] The present invention provides a dynamic game comprehensive evaluation method for system combat capability based on deep learning to solve the problems of weak real-time performance of traditional methods, insufficient adaptation to intelligent unmanned clusters, and limited dynamic game and strategy optimization capabilities.

[0007] To solve the above technical problems, the present invention provides the following technical solutions:

[0008] An embodiment of the present invention provides a dynamic game comprehensive evaluation method for system combat capability based on deep learning, which includes:

[0009] Step S1, collecting battlefield environment data through multi-sensor fusion technology, and using a temporal convolutional network and a graph neural network model to convert the collected environment data into a real-time battlefield situation model;

[0010] Step S2: Using proximal transfer to optimize the PPO deep learning algorithm in combination with transfer learning technology, optimize the formation of the unmanned cluster based on the battlefield situation model, and dynamically generate the formation and task allocation plan for the unmanned cluster;

[0011] Step S3: Based on the unmanned cluster formation and task allocation plan generated in Step S2, construct a multi-agent game model driven by deep reinforcement learning. The game model comprehensively considers the resource allocation of both the enemy and our side, the dynamic changes of the environment, and uncertain factors, simulates the dynamic changes of the resource allocation and behavior of both the enemy and our side, and generates the optimal strategy;

[0012] Step S4: Convert the optimal strategy generated in Step S3 into specific execution instructions for the unmanned cluster, and send them to each combat unit in real time. After sending, monitor the execution process in real time;

[0013] Step S5: Based on the feedback data obtained by monitoring in Step S4, optimize the battlefield situation model, formation plan, and game strategy, construct an iterative closed loop based on reinforcement learning, update the model and strategy logic, and use the feedback data to optimize the multi-task cooperation effectiveness index.

[0014] As a preferred solution of the dynamic game comprehensive evaluation method for the system combat capability based on deep learning described in the present invention, wherein: the battlefield environment data includes ground situation, air platform information, and electromagnetic environment characteristics.

[0015] As a preferred solution of the dynamic game comprehensive evaluation method for the system combat capability based on deep learning described in the present invention, wherein: the steps of using a temporal convolutional network and a graph neural network model to convert the collected environmental data into a real-time battlefield situation model are as follows,

[0016] Represent the collected battlefield environment data as X t :

[0017]

[0018] Among them, X t represents the multi-modal input data set at time t, represents the ground situation data feature vector at time t, represents the air platform information feature vector at time t, represents the electromagnetic environment feature vector at time t,

[0019] Perform standardization processing on X t The standardization formula is:

[0020]

[0021] Among them, x' t represents the normalized input data, xt denotes the input data before normalization, μ denotes the mean of data features, and σ denotes the standard deviation of data features;

[0022] Use a one-dimensional time series convolutional network to extract dynamic features in the time dimension. The convolution calculation formula is:

[0023] H t = ReLU(W c * X t + b c ),

[0024] where H t denotes the hidden state feature matrix after convolution, W c denotes the convolutional kernel weight matrix, * denotes the one-dimensional convolution operation, b c denotes the bias vector, and ReLU denotes the ReLU activation function.

[0025] Construct the graph structure of the battlefield environment, denoted as:

[0026] G = (V, E),

[0027] where G denotes the battlefield graph, V denotes the set of graph nodes, including different sources of various features, such as ground situation, air platform, and electromagnetic environment nodes, and E denotes the set of graph edges, representing the associations, communication relationships, or geometric proximity relationships between nodes.

[0028] The adjacency matrix A defines the connection relationship of the graph. The feature update formula of the graph neural network is:

[0029] H' t = softmax(AH t W g + b g ),

[0030] where H' t denotes the node feature matrix updated by the graph neural network, A denotes the adjacency matrix, defining the connection relationship of the graph, H t denotes the input convolutional feature matrix, W g denotes the weight matrix of the graph neural network, and b g denotes the bias vector.

[0031] Map the output features of the graph neural network to the battlefield situation model space through a fully connected layer:

[0032] M t = W m H' t + b m ,

[0033] where M t denotes the battlefield situation model at time t, and Wm Denote the weight matrix of the fully connected layer, H' t Denote the node feature matrix output by the graph neural network, b m Denote the bias vector.

[0034] As a preferred solution of the comprehensive evaluation method for the dynamic game of the system combat ability based on deep learning according to the present invention, wherein: in step S2, define the multi-task cooperation effectiveness indexes, including the completion rate, response time, and resource utilization rate;

[0035] During the formation optimization process of step S2, the cooperation mode of the unmanned aerial vehicle and the ground unmanned platform is adjusted in real time.

[0036] As a preferred solution of the comprehensive evaluation method for the dynamic game of the system combat ability based on deep learning according to the present invention, wherein: the step of using the proximal transfer optimization PPO deep learning algorithm combined with the transfer learning technology to optimize the formation of the unmanned cluster based on the battlefield situation model and dynamically generate the formation and task allocation plan of the unmanned cluster is as follows,

[0037] Use the battlefield situation model M generated in step S1 t As the input, construct the optimization objective function J for the formation and task allocation optimization of the unmanned cluster, and the objective function is:

[0038] J = α·R c -β·T r +γ·U r ,

[0039] wherein, J represents the optimization objective value, the multi-task cooperation effectiveness index, R c represents the task completion rate, defined as the ratio of the number of completed tasks to the total number of tasks, T r represents the task response time, defined as the weighted sum of the time required to complete the task, U r represents the resource utilization rate, defined as the ratio of the actual usage amount of resources to the total resource capacity, and α, β, γ represent the weight coefficients,

[0040] The initial formation of the unmanned cluster is represented as

[0041]

[0042] wherein, represents the initial formation set, U g represents the set of ground unmanned platforms, U a represents the set of aerial unmanned platforms;

[0043] Use the proximal policy optimization PPO algorithm combined with the transfer learning technology for dynamic optimization, and the optimization target formula of PPO is:

[0044]

[0045] Among them, L(θ) represents the objective function for policy update, and E t represents the calculation of the expected value at time t, and r t (θ) represents the policy probability ratio, which describes the ratio of the new and old policy distributions. represents the advantage function, which is used to evaluate the quality of the current policy. clip represents the clipping operation, which is defined as follows: if r t (θ) exceeds the interval [1 - ∈, 1 + ∈], then take the endpoint value of the interval, otherwise remain unchanged. ∈ represents the clipping threshold.

[0046] Based on the optimization results, dynamically generate the optimal unmanned cluster formation and the task assignment plan

[0047] Among them, represents the optimal unmanned cluster formation plan, represents the set of optional unmanned cluster formations, represents the optimal task assignment plan, represents all possible task assignment plans, and T r represents the task response time;

[0048] Define the collaborative mode function based on the battlefield situation model M t and the optimal formation to deduce the collaborative mode:

[0049]

[0050] Among them, represents the collaborative mode at time, and ATT represents the collaborative mode deduction function based on the attention mechanism, which dynamically adjusts the communication and task collaboration of different unmanned platforms through weights.

[0051] The calculation formula of the attention mechanism is:

[0052]

[0053] Among them, Q, K, and V represent the query matrix, key matrix, and value matrix respectively, and d k represents the dimension of the key matrix.

[0054] As a preferred embodiment of the dynamic game comprehensive evaluation method for the combat capability of a system based on deep learning according to the present invention, in step S3, based on the optimal strategy, simulate the combat scenario, evaluate the combat effectiveness under different strategies, and eliminate inefficient strategies.

[0055] The uncertainty factors include environmental disturbances, communication and cooperation obstacles, and enemy strategy adjustments.

[0056] As an optimal solution of the dynamic game comprehensive evaluation method for the combat capability of a system based on deep learning according to the present invention, wherein: the steps of simulating the dynamic changes of the resource allocations and behaviors of both the enemy and us by integrating the resource allocations of both the enemy and us, the dynamic changes of the environment, and the uncertainty factors to generate an optimal strategy are as follows.

[0057] Define the input of the game model, based on the battlefield situation model M t Express the resource states S of both the enemy and us t :

[0058]

[0059] Wherein, S t represents the resource state at time t, represents our resources, including the unmanned cluster and the mission status, represents the enemy's resources, including the enemy units and the ability assessment,

[0060] Based on S t , establish a multi-agent game model and define the strategy sets of both the enemy and us:

[0061] Π = {π f , π e},

[0062] Wherein, Π represents the strategy set, π f represents our strategy, and π e represents the enemy's strategy;

[0063] Define the goal of the agent as: optimizing its own strategy to maximize the benefit or minimize the loss,

[0064] Define the benefit function U to evaluate the strategy effect:

[0065]

[0066] Wherein, U f represents our benefit function, and U e represents the enemy's benefit function, represents our mission completion rate, represents the enemy's mission completion rate, C f represents our resource consumption, and C e represents the enemy's resource consumption, and λ represents the resource consumption weight coefficient;

[0067] For the dynamic changes of the battlefield environment, use the state transition function to represent the game process, which is expressed as:

[0068] S t+1 = f(S t , π f , π e , ΔE),

[0069] where S t+1 represents the resource state at the next moment, f represents the state transition function, and ΔE represents dynamic disturbances, including communication obstacles and environmental changes,

[0070] Solve the optimal strategy of the game model through the deep reinforcement learning method:

[0071]

[0072] where, represents our optimal strategy, represents the optimal strategy of the enemy, and E represents the expected value calculation,

[0073] Use the value function V(S t ) to optimize the strategy, and the value function is defined as:

[0074]

[0075] where V(S t ) represents the cumulative reward in state S t , γ represents the discount factor, weighing the current reward and future rewards, and T represents the maximum number of steps in the game process.

[0076] As a preferred solution of the dynamic game comprehensive evaluation method for system combat capabilities based on deep learning according to the present invention, wherein: in step S4, monitor the execution process to obtain feedback data, including task completion, resource consumption, and execution effects;

[0077] In step S4, during the execution process, adjust the strategy for emergencies, and the emergencies include target changes or insertion of new tasks.

[0078] As a preferred solution of the dynamic game comprehensive evaluation method for system combat capabilities based on deep learning according to the present invention, wherein: the step of adjusting the strategy for emergencies during the execution process is,

[0079] Based on the optimal strategy generated in step S3 issue specific execution instructions C t :

[0080]

[0081] where C t represents the specific execution instruction at time t, and g represents the instruction generation function, Denote our optimal strategy as M t Denote the battlefield situation model at time t

[0082] During the execution of the instruction, monitor the feedback data F t :

[0083] F t ={P t , R t , E t},

[0084] where F t denotes the set of feedback data at time t, P t denotes the task completion situation, R t denotes the resource consumption situation, and E t denotes the execution effect;

[0085] Detect the sudden event ΔT by dynamically evaluating the feedback data t :

[0086] ΔT t =h(F t , δ),

[0087] where ΔT t denotes the sudden event at time t, h denotes the sudden event detection function, F t denotes the feedback data, and δ denotes the event threshold for judging whether an abnormality occurs;

[0088] Sudden events include: adjustment of the number or location of task objectives, and addition of emergency tasks during the task execution;

[0089] After detecting the sudden event, adjust the strategy using dynamic programming. The adjustment formula is:

[0090]

[0091] where denotes the adjusted optimal strategy, π denotes all possible strategies, E denotes the expected value calculation, and U f denotes the revenue function, and ΔT t denotes the sudden event condition;

[0092] Generate a new execution instruction based on the adjusted strategy:

[0093]

[0094] where C' t denotes the adjusted specific execution instruction, denotes the adjusted optimal strategy M tRepresents the current battlefield situation model, ΔT t Represents the conditions for emergencies

[0095] Repeat the monitoring, detection, and adjustment process until the task is completed or the battlefield situation undergoes a fundamental change.

[0096] As an optimal solution of the dynamic game comprehensive evaluation method for the combat capability of the system based on deep learning described in the present invention, wherein: the steps of optimizing the battlefield situation model, formation plan, and game strategy are as follows

[0097] Collect the feedback data monitored in step S4 and integrate it into a unified data set F

[0098]

[0099] Among them, F represents the total set of feedback data, F t Represents the feedback data at time t, including the task completion situation P t , resource consumption situation R t , execution effect E t , T represents the total time length of the feedback period

[0100] According to the feedback data F, use the recursive update function to optimize the battlefield situation model M t The optimization formula is

[0101] M t+1 = φ(M t , F t ),

[0102] Among them, M t+1 Represents the updated battlefield situation model, φ represents the update function of the situation model, adjusts the feature weights in the model based on the feedback data, M t Represents the situation model at the current moment, F t Represents the feedback data at time t

[0103] The core of the situation model update is the feature weight adjustment function, which is expressed as

[0104] w i,t+1 = w i,t + η·Δw i,t ,

[0105] Among them, w i,t+1 Represents the updated i-th feature weight, w i,t Represents the current i-th feature weight, η represents the learning rate, Δw i,t Represents the adjustment value of the feature, calculated from the feedback data

[0106] Optimize the formation plan of the unmanned cluster according to the updated result of the situation model

[0107]

[0108] Among them, represents the optimized formation plan of the unmanned cluster, represents the set of all possible formation plans, J represents the multi-task collaborative efficiency index function, which is calculated based on the feedback data and the latest situation model,

[0109] Utilize the deep reinforcement learning technology to iteratively optimize the game strategies and :

[0110]

[0111] Among them, represents the updated optimal strategy of our side, represents the updated optimal strategy of the enemy side, S t represents the current state, U f ,U e represent the benefit functions of our side and the enemy side respectively,

[0112] Utilize the optimized battlefield situation model, formation plan and game strategy to form a closed-loop iteration:

[0113]

[0114] Among them, represents the optimized dynamic closed-loop set, M t+1 represents the optimized battlefield situation model, represents the optimized formation plan of the unmanned cluster, represents the optimized optimal strategy.

[0115] The beneficial effects of the present invention are as follows: By combining multi-sensor fusion technology with temporal convolutional networks and graph neural networks, the present invention dynamically extracts key features of time and space in the battlefield environment, updates the situation model in real time, improves the accuracy of data processing and the real-time performance of the model. Using proximal policy optimization (PPO) and transfer learning techniques, it accelerates the optimization process of unmanned cluster formation and task allocation, and realizes the adaptive adjustment of the formation plan to complex task requirements by defining multi-task collaborative efficiency indicators. In terms of policy optimization, a multi-agent game model driven by deep reinforcement learning is constructed, which comprehensively considers the allocation of enemy and friendly resources, dynamic environmental changes and uncertainty factors, effectively generates optimal strategies and eliminates inefficient solutions through combat scenario simulation, enhancing the applicability of decision-making. At the same time, through real-time monitoring and dynamic planning mechanisms, it responds to emergencies such as target changes and the insertion of urgent tasks, improves the continuity of task execution and collaborative efficiency. The feedback closed-loop optimization dynamically updates the battlefield situation model, formation plan and game strategy through feature weight adjustment and deep reinforcement learning iteration mechanisms, gradually improving the adaptability and robustness of the evaluation system.

[0116] In summary, the present invention effectively solves the disadvantages of traditional methods in terms of real-time performance, self-adaptability, uncertainty processing and intelligent collaboration, and can comprehensively improve the collaborative combat ability of unmanned clusters in complex and changeable battlefield environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0117] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0118] Figure 1 It is a schematic flow chart of the dynamic game comprehensive evaluation method for the combat capability of the system based on deep learning of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0119] In order to make the above objects, features and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific embodiments of the present invention in conjunction with the drawings in the specification.

[0120] Many specific details are set forth in the following description in order to provide a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0121] Secondly, the "one embodiment" or "embodiment" mentioned herein refers to specific features, structures, or characteristics that may be included in at least one implementation manner of the present invention. The "in one embodiment" that appears in different places in this specification does not all refer to the same embodiment, nor is it an individual or selectively mutually exclusive embodiment with other embodiments.

[0122] Embodiment 1, referring to Figure 1 , this embodiment provides a dynamic game comprehensive evaluation method for the combat capability of a system based on deep learning, including the following steps:

[0123] Step S1, collect battlefield environment data through multi-sensor fusion technology, and use a temporal convolutional network and a graph neural network model to convert the collected environmental data into a real-time battlefield situation model;

[0124] The battlefield environment data includes ground situation, air platform information, and electromagnetic environment characteristics;

[0125] The steps of using a temporal convolutional network and a graph neural network model to convert the collected environmental data into a real-time battlefield situation model are as follows:

[0126] Represent the collected battlefield environment data as X t :

[0127]

[0128] Among them, X t represents the multi-modal input data set at time t, represents the ground situation data feature vector at time t, represents the air platform information feature vector at time t, represents the electromagnetic environment feature vector at time t,

[0129] Perform normalization processing on X t , and the normalization formula is:

[0130]

[0131] Among them, x' t represents the normalized input data, x t represents the input data before normalization, μ represents the mean of the data features, and σ represents the standard deviation of the data features;

[0132] Use a one-dimensional temporal convolutional network to extract the dynamic features in the time dimension, and the convolution calculation formula is:

[0133] H t = ReLU(W c *X t +b c ),

[0134] Among them, H t represents the hidden state feature matrix after convolution, W c represents the convolutional kernel weight matrix, * represents the one-dimensional convolution operation, b c represents the bias vector, and ReLU represents the ReLU activation function.

[0135] Construct the graph structure of the battlefield environment, which is expressed as:

[0136] G = (V, E),

[0137] Among them, G represents the battlefield graph, V represents the set of graph nodes, including different sources of various features, ground situation, air platform, and electromagnetic environment nodes, and E represents the set of graph edges, representing the associations, communication relationships, or geometric proximity relationships between nodes.

[0138] The adjacency matrix A defines the connection relationship of the graph, and the feature update formula of the graph neural network is:

[0139] H t = softmax(AH t W g + b g ),

[0140] Among them, H' t represents the node feature matrix after being updated by the graph neural network, A represents the adjacency matrix, which defines the connection relationship of the graph, H t represents the input convolutional feature matrix, W g represents the weight matrix of the graph neural network, b g represents the bias vector.

[0141] Map the output features of the graph neural network to the battlefield situation model space through the fully connected layer:

[0142] M t = W m H′ t + b m ,

[0143] Among them, M t represents the battlefield situation model at time t, W m represents the weight matrix of the fully connected layer, H' t represents the node feature matrix output by the graph neural network, b m represents the bias vector;

[0144] Specifically, multi-modal data processing and joint modeling techniques are carried out here. The temporal convolutional network is used to extract the dynamic features in the time dimension, and the graph neural network is used to capture the associations between battlefield nodes.

[0145] Step S2: Optimize the PPO deep learning algorithm using proximal transfer and combine it with transfer learning technology. Based on the battlefield situation model, optimize the formation of the unmanned cluster and dynamically generate the formation and task allocation plan for the unmanned cluster.

[0146] In step S2, define the multi-task collaboration effectiveness indicators, including completion rate, response time, and resource utilization rate.

[0147] During the formation optimization process in step S2, adjust the collaboration mode of the unmanned aerial vehicles and ground unmanned platforms in real time.

[0148] The steps of using the proximal transfer optimized PPO deep learning algorithm combined with transfer learning technology to optimize the formation of the unmanned cluster based on the battlefield situation model and dynamically generate the formation and task allocation plan for the unmanned cluster are as follows:

[0149] Use the battlefield situation model M generated in step S1 t As the input, construct an optimization objective function J for optimizing the formation and task allocation of the unmanned cluster. The objective function is:

[0150] J = α·R c -β·T r +γ·U r ,

[0151] Among them, J represents the optimization objective value, the multi-task collaboration effectiveness indicator, R c represents the task completion rate, defined as the ratio of the number of completed tasks to the total number of tasks, T r represents the task response time, defined as the weighted sum of the time required to complete the task, U r represents the resource utilization rate, defined as the ratio of the actual usage amount of resources to the total resource capacity. α, β, γ represent the weight coefficients.

[0152] The initial formation of the unmanned cluster is represented as

[0153]

[0154] Among them, represents the initial formation set, U g represents the set of ground unmanned platforms, U a represents the set of aerial unmanned platforms;

[0155] Use the proximal policy optimization (PPO) algorithm combined with transfer learning technology for dynamic optimization. The optimization target formula of PPO is:

[0156]

[0157] Among them, L(θ) represents the objective function for policy update, E tDenotes the calculation of the expected value at time t, r t (θ) represents the policy probability ratio, describing the ratio of the new and old policy distributions, Denotes the advantage function, used to evaluate the pros and cons of the current policy, clip represents the clipping operation, defined as: if r t (θ) exceeds the interval [1 - ∈, 1 + ∈], then take the endpoint value of the interval, otherwise remain unchanged, ∈ represents the clipping threshold;

[0158] Based on the optimization results, dynamically generate the optimal unmanned cluster formation and task assignment scheme

[0159] Among them, Denotes the optimal unmanned cluster formation plan, Denotes the set of optional unmanned cluster formations, Denotes the optimal task assignment plan, Denotes all possible task assignment plans, T r Denotes the task response time;

[0160] Define the collaborative mode function based on the battlefield situation model M t and the optimal formation Derive the collaborative mode:

[0161]

[0162] Among them, Denotes the collaborative mode at time t, ATT represents the collaborative mode derivation function based on the attention mechanism, which dynamically adjusts the communication and task collaboration of different unmanned platforms through weights,

[0163] The calculation formula of the attention mechanism is:

[0164]

[0165] Among them, Q, K, V represent the query matrix, key matrix, and value matrix respectively, d k Denotes the dimension of the key matrix;

[0166] Specifically, optimize the formation and task assignment plan of the unmanned cluster through the objective function J, and dynamically update the policy using the PPO algorithm; at the same time, introduce the collaborative mode derivation function based on the attention mechanism to enhance the task collaboration ability between unmanned platforms, making it have higher adaptability and collaborative efficiency in complex battlefield environments.

[0167] Step S3: Based on the unmanned cluster formation and task assignment scheme generated in Step S2, construct a multi-agent game model driven by deep reinforcement learning. The game model comprehensively considers the resource allocation of both sides, the dynamic changes of the environment, and uncertain factors, simulates the dynamic changes of the resource allocation and behavior of both sides, and generates the optimal strategy.

[0168] In Step S3, conduct combat scenario simulation based on the optimal strategy, evaluate the combat effectiveness under different strategies, and eliminate inefficient strategies.

[0169] Uncertain factors include environmental disturbances, communication and coordination obstacles, and enemy strategy adjustments.

[0170] The steps of comprehensively considering the resource allocation of both sides, the dynamic changes of the environment, and uncertain factors, simulating the dynamic changes of the resource allocation and behavior of both sides, and generating the optimal strategy are as follows:

[0171] Define the input of the game model, based on the battlefield situation model M t Express the resource status S of both sides t :

[0172]

[0173] Among them, S t represents the resource status at time t, represents our resources, including the unmanned cluster and task status, represents the enemy's resources, including enemy units and ability assessments,

[0174] Based on S t , establish a multi-agent game model and define the strategy sets of both sides:

[0175] Π = {π f , π e},

[0176] Among them, Π represents the strategy set, π f represents our strategy, and π e represents the enemy's strategy;

[0177] Define the goal of the agent as: optimizing its own strategy to maximize the benefit or minimize the loss,

[0178] Define the benefit function U to evaluate the strategy effect:

[0179]

[0180] Among them, U f represents our benefit function, and U e represents the enemy's benefit function, represents our task completion rate, Indicates the enemy's mission completion rate, C f Indicates our side's resource consumption, C e Indicates the enemy's resource consumption, and λ represents the resource consumption weight coefficient;

[0181] In response to the dynamic changes in the battlefield environment, the game process is represented using a state transition function, expressed as:

[0182] S t+1 = f(S t , π f , π e , ΔE),

[0183] where S t+1 represents the resource state at the next moment, f represents the state transition function, and ΔE represents dynamic disturbances, including communication obstacles and environmental changes,

[0184] The optimal strategy of the game model is solved through the deep reinforcement learning method:

[0185]

[0186] where, represents our side's optimal strategy, represents the enemy's optimal strategy, and E represents the expected value calculation,

[0187] The value function V(S t ) is used for policy optimization, and the value function is defined as:

[0188]

[0189] where V(S t ) represents the cumulative reward in state S t , γ represents the discount factor, weighing the current reward and future rewards, and T represents the maximum number of steps in the game process;

[0190] Specifically, here the policy is optimized through deep reinforcement learning to solve the decision-making problem in a complex dynamic environment;

[0191] Step S4: Convert the optimal policy generated in step S3 into specific execution instructions for the unmanned cluster and send them to each combat unit in real time. After sending, monitor the execution process in real time;

[0192] In step S4, feedback data is obtained by monitoring the execution process, including task completion, resource consumption, and execution effect;

[0193] In step S4, during the execution process, the policy is adjusted for emergencies, and the emergencies include target changes or insertion of new tasks;

[0194] During the execution process, the steps for adjusting the strategy in response to emergencies are as follows:

[0195] Based on the optimal strategy generated in step S3 Issue a specific execution instruction C t :

[0196]

[0197] where C t represents the specific execution instruction at time t, g represents the instruction generation function, represents our optimal strategy, and M t represents the battlefield situation model at time t.

[0198] During the execution of the instruction, monitor the feedback data F t :

[0199] F t ={P t , R t , E t},

[0200] where F t represents the set of feedback data at time t, P t represents the task completion situation, R t represents the resource consumption situation, and E t represents the execution effect;

[0201] Detect emergencies ΔT by dynamically evaluating the feedback data t :

[0202] ΔT t =h(F t , δ),

[0203] where ΔT t represents the emergency at time t, h represents the emergency detection function, F t represents the feedback data, and δ represents the event threshold for judging whether an abnormality occurs;

[0204] Emergencies include: adjustment of the quantity or location of the task objective, and addition of an urgent task during the task execution;

[0205] After detecting an emergency, adjust the strategy using dynamic programming. The adjustment formula is:

[0206]

[0207] where, represents the adjusted optimal strategy, π represents all possible strategies, E represents the expected value calculation, and U f represents the benefit function, and ΔTt Indicates emergency conditions;

[0208] Generate new execution instructions based on the adjusted strategy:

[0209]

[0210] where C' t represents the specific adjusted execution instruction, represents the adjusted optimal strategy M t represents the current battlefield situation model, ΔT t represents the conditions of the emergency,

[0211] Repeat the monitoring, detection, and adjustment process until the task is completed or the battlefield situation undergoes a fundamental change;

[0212] Specifically, by real-time monitoring of feedback data, detecting emergencies, and combining dynamic programming and strategy updates to generate adjusted optimal instructions, this step strengthens the response ability of the unmanned cluster to emergencies, enabling it to maintain the continuity of task execution in a highly dynamic environment.

[0213] Step S5: Based on the feedback data obtained from monitoring in step S4, optimize the battlefield situation model, formation plan, and game strategy, construct an iterative closed-loop based on reinforcement learning, update the model and strategy logic, and use the feedback data to optimize the multi-task collaboration effectiveness index;

[0214] The steps for optimizing the battlefield situation model, formation plan, and game strategy are as follows:

[0215] Collect the feedback data monitored in step S4 and integrate it into a unified data set F:

[0216]

[0217] where F represents the total set of feedback data, F t represents the feedback data at time t, including the task completion situation P t 、resource consumption situation R t 、execution effect E t , T represents the total time length of the feedback cycle,

[0218] According to the feedback data F, use the recursive update function to optimize the battlefield situation model M t The optimization formula is:

[0219] M t+1 =φ(M t ,F t ),

[0220] where M t+1Denote the updated battlefield situation model as $\varPhi$, and $\varphi$ represents the update function of the situation model, which adjusts the feature weights in the model based on the feedback data, $M$ t represents the situation model at the current moment, $F$ t represents the feedback data at time $t$,

[0221] The core of the situation model update is the feature weight adjustment function, which is expressed as:

[0222] $w$ i,t+1 $=$ $w$ i,t $+$ $\eta\cdot\Delta w$ i,t ,

[0223] where $w$ i,t+1 represents the updated $i$-th feature weight, $w$ i,t represents the current $i$-th feature weight, $\eta$ represents the learning rate, $\Delta w$ i,t represents the adjustment value of the feature, which is calculated from the feedback data,

[0224] Optimize the formation plan of the unmanned cluster according to the situation model update result

[0225]

[0226] where, represents the optimized formation plan of the unmanned cluster, represents the set of all possible formation plans, $J$ represents the multi-task cooperation efficiency index function, which is calculated based on the feedback data and the latest situation model,

[0227] Use the deep reinforcement learning technique to iteratively optimize the game strategies and :

[0228]

[0229] where, represents the updated optimal strategy of our side, represents the updated optimal strategy of the enemy side, $S$ t represents the current state, $U$ f , $U$ e respectively represent the benefit functions of our side and the enemy side,

[0230] Use the optimized battlefield situation model, formation plan and game strategy to form a closed-loop iteration:

[0231]

[0232] where, represents the optimized dynamic closed-loop set, $M$ t+1 represents the optimized battlefield situation model, represents the optimized unmanned cluster formation plan, represents the optimized optimal strategy;

[0233] Specifically, in this step, through the optimization process driven by feedback data, a dynamic closed-loop of the battlefield situation model, unmanned cluster formation, and game strategy is formed. The feature weight adjustment mechanism is used to enhance the ability to describe the environmental dynamics, and the formation plan is dynamically optimized in combination with the latest situation and task feedback.

[0234] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A dynamic game comprehensive evaluation method for system combat capability based on deep learning, characterized by: include, Step S1, collecting battlefield environment data through multi-sensor fusion technology, and using temporal convolutional network and graph neural network models to convert the collected environment data into a real-time battlefield situation model; Step S2, using the proximal migration optimization PPO deep learning algorithm combined with migration learning technology, optimizes the grouping of the unmanned cluster based on the battlefield situation model, and dynamically generates the grouping and task allocation plan of the unmanned cluster; Step S3, based on the unmanned swarm formation and task allocation scheme generated in step S2, a multi-agent game model driven by deep reinforcement learning is constructed. The game model integrates the resource allocation of both sides, the dynamic changes of the environment and uncertainty factors, simulates the resource allocation and dynamic changes of the behavior of both sides, and generates the optimal strategy; Step S4, converting the optimal strategy generated in step S3 into specific execution instructions of the unmanned swarm, sending them to each combat unit in real time, and monitoring the execution process in real time after sending them; Step S5, based on the feedback data obtained through monitoring in step S4, optimize the battlefield situation model, formation plan and game strategy, build an iterative closed loop based on reinforcement learning, update the model and strategy logic, and use the feedback data to optimize the multi-task collaborative performance indicators.

2. A method for comprehensive evaluation of system combat capability based on deep learning in dynamic game theory as claimed in claim 1, characterized in that: The battlefield environment data includes ground situation, air platform information and electromagnetic environment characteristics.

3. A method for comprehensive evaluation of system combat capability dynamic game based on deep learning as claimed in claim 2, characterized in that: The steps of using the temporal convolutional network and the graph neural network model to convert the collected environmental data into a real-time battlefield situation model are: The collected battlefield environment data is represented as X t : Among them, X t represents the multimodal input data set at time t, Represents the feature vector of ground situation data at time t, represents the feature vector of the aerial platform information at time t, represents the electromagnetic environment characteristic vector at time t, X t Perform standardization, the standardization formula is: Among them, x' t represents the normalized input data, x t represents the input data before normalization, μ represents the mean of the data feature, and σ represents the standard deviation of the data feature; The one-dimensional temporal convolutional network is used to extract the dynamic features of the time dimension. The convolution calculation formula is: H t =ReLU(W c *X t +b c ), Among them, H t Represents the hidden state feature matrix after convolution, W c represents the convolution kernel weight matrix, * represents a one-dimensional convolution operation, b c represents the bias vector, ReLU represents the ReLU activation function, The graph structure of the battlefield environment is constructed as follows: G=(V,E), Among them, G represents the battlefield map, V represents the set of graph nodes, which contains different sources of each feature, and E represents the set of graph edges, which represents the relationship between nodes. The adjacency matrix A defines the connection relationship of the graph, and the feature update formula of the graph neural network is: H′ t =softmax(AH t W g +b g ), Among them, H' t represents the node feature matrix after being updated by the graph neural network, A represents the adjacency matrix, which defines the connection relationship of the graph, and H t Represents the input convolution feature matrix, W g represents the weight matrix of the graph neural network, b g represents the bias vector, Map the output features of the graph neural network to the battlefield situation model space through the fully connected layer: M t =W m H′ t +b m , Among them, M t represents the battlefield situation model at time t, W m represents the weight matrix of the fully connected layer, H' t Represents the node feature matrix output by the graph neural network, b m Represents the bias vector.

4. A method for comprehensive evaluation of system combat capability dynamic game based on deep learning as claimed in claim 3, characterized in that: In step S2, multi-task collaborative performance indicators are defined, including completion rate, response time, and resource utilization; During the formation optimization process of step S2, the coordination mode of the UAV and the ground unmanned platform is adjusted in real time.

5. A method for comprehensive evaluation of system combat capability dynamic game based on deep learning as claimed in claim 4, characterized in that: The steps of using the proximal transfer optimization PPO deep learning algorithm combined with transfer learning technology to optimize the grouping of unmanned clusters based on the battlefield situation model and dynamically generate the grouping and task allocation plan of the unmanned cluster are as follows: Using the battlefield situation model M generated in step S1 t As input, the optimization objective function J is constructed for the grouping and task allocation optimization of the unmanned swarm. The objective function is: J=α·R c -β·T r +γ·U r , Among them, J represents the optimization target value, multi-task collaborative efficiency index, R c It represents the task completion rate, which is defined as the ratio of the number of completed tasks to the total number of tasks. r represents the task response time, which is defined as the weighted sum of the time required to complete the task, U r It represents resource utilization, which is defined as the ratio of actual resource usage to total resource capacity. α, β, and γ represent weight coefficients. The initial formation of the unmanned swarm is represented as in, represents the initial grouping set, U g represents the set of ground unmanned platforms, U a Represents a collection of aerial unmanned platforms; The proximal strategy optimization PPO algorithm is combined with transfer learning technology for dynamic optimization. The optimization target formula of PPO is: Among them, L(θ) represents the objective function of the strategy update, E t represents the expected value calculation at time t, r t (θ) represents the strategy probability ratio, which describes the ratio of the distribution of new and old strategies. represents the advantage function, which is used to evaluate the pros and cons of the current strategy. Clip represents the clipping operation, which is defined as: if r t If (θ) exceeds the interval [1-∈, 1+∈], the interval endpoint value is taken, otherwise it remains unchanged, and ∈ represents the clipping threshold; Based on the optimization results, the optimal unmanned swarm formation is dynamically generated and task allocation scheme in, represents the optimal unmanned swarm formation scheme, Represents an optional unmanned cluster grouping set, represents the optimal task allocation solution, represents all possible task allocation schemes, T r Indicates the task response time; Define the coordination mode function based on the battlefield situation model M t and optimal grouping Derivation of collaborative mode: in, represents the collaborative mode at time t, ATT represents the collaborative mode derivation function based on the attention mechanism, and dynamically adjusts the communication and task collaboration of different unmanned platforms through weights. The calculation formula of the attention mechanism is: Among them, Q, K, and V represent the query matrix, key matrix, and value matrix respectively. k Indicates the dimensions of the key matrix.

6. A method for comprehensive evaluation of system combat capability dynamic game based on deep learning as claimed in claim 5, characterized in that: In step S3, a combat scenario simulation is performed based on the optimal strategy, the combat effectiveness under different strategies is evaluated, and inefficient strategies are eliminated; The uncertain factors include environmental disturbances, communication coordination obstacles, and enemy strategy adjustments.

7. A method for comprehensive evaluation of system combat capability dynamic game based on deep learning as claimed in claim 6, characterized in that: The steps of synthesizing the resource allocation of both sides, the dynamic changes of the environment and the uncertainty factors, simulating the dynamic changes of the resource allocation and behavior of both sides, and generating the optimal strategy are as follows: Define the input of the game model, based on the battlefield situation model M t Express the resource status of both sides S t : Among them, S t represents the resource status at time t, Indicates our resources, including unmanned clusters and task status. Represents enemy resources, including enemy units and capability assessments, Based on S t , establish a multi-agent game model and define the strategy set of both sides: Π={π f ,p e }, Among them, Π represents the strategy set, π f represents our strategy, π e Indicates enemy strategy; The goal of the intelligent agent is to optimize its own strategy to maximize profit or minimize loss. Define the profit function U to evaluate the effect of the strategy: Among them, U f represents our profit function, U e represents the enemy’s payoff function, Indicates our task completion rate, represents the enemy's mission completion rate, C f Indicates our resource consumption, C e represents the enemy's resource consumption, λ represents the resource consumption weight coefficient; In view of the dynamic changes in the battlefield environment, the state transfer function is used to represent the game process, which is expressed as: S t+1 =f(S t ,p f ,p e ,ΔE), Among them, S t+1 represents the resource state at the next moment, f represents the state transition function, ΔE represents the dynamic disturbance, including communication obstacles and environmental changes, Through deep reinforcement learning methods, we can solve the optimal strategy of the game model: in, represents our optimal strategy, represents the enemy's optimal strategy, E represents the expected value calculation, Using the value function V(S t ) to optimize the strategy, the value function is defined as: Among them, V(S t ) indicates that in state S t The cumulative revenue under , γ represents the discount factor, weighing the current revenue and future revenue, and T represents the maximum number of steps in the game process.

8. A method for comprehensive evaluation of system combat capability dynamic game based on deep learning as claimed in claim 7, characterized in that: In step S4, the execution process is monitored to obtain feedback data, including task completion status, resource consumption, and execution effect; In step S4, during the execution process, the strategy is adjusted according to unexpected events, including target changes or insertion of new tasks.

9. A method for comprehensive evaluation of system combat capability dynamic game based on deep learning as claimed in claim 8, characterized in that: During the execution process, the steps of adjusting the strategy for emergencies are: Based on the optimal strategy generated in step S3 Issue specific execution instructions C t : Among them, C t represents the specific execution instruction at time t, g represents the instruction generation function, represents our optimal strategy, M t represents the battlefield situation model at time t, During the execution of the instruction, monitor the feedback data F t : F t ={P t ,R t ,E t }, Among them, F t represents the feedback data set at time t, P t Indicates the task completion status, R t Indicates resource consumption, E t Indicates the execution effect; Detect unexpected events ΔT by dynamically evaluating feedback data t : ΔT t =h(F t ,δ), Where, ΔT t represents the emergency event at time t, h represents the emergency event detection function, F t represents feedback data, δ represents the event threshold, which is used to determine whether an abnormality occurs; Emergencies include: adjustments to the number or location of mission targets, and the addition of new emergency tasks during mission execution; After detecting an emergency, dynamic programming is used to adjust the strategy. The adjustment formula is: in, represents the optimal strategy after adjustment, π represents all possible strategies, E represents the expected value calculation, U f represents the revenue function, ΔT t Indicates an emergency condition; Generate new execution instructions based on the adjusted strategy: Among them, C' t Indicates the specific execution instructions after adjustment. Represents the optimal strategy M after adjustment t Represents the current battlefield situation model, ΔT t Indicates the conditions of an emergency. Repeat the monitoring, testing, and adjustment process until the mission is completed or the battlefield situation changes fundamentally.

10. A method for comprehensive evaluation of system combat capability dynamic game based on deep learning as claimed in claim 9, characterized in that: The steps of optimizing the battlefield situation model, formation scheme and game strategy are as follows: Collect the feedback data monitored in step S4 and integrate them into a unified data set F: Where F represents the total set of feedback data, F t Represents the feedback data at time t, including the task completion status P t , Resource consumption R t 、Execution effect E t , T represents the total time length of the feedback cycle, According to the feedback data F, the battlefield situation model M is updated using a recursive function t Optimize, the optimization formula is: M t+1 =φ(M t ,F t ), Among them, M t+1 represents the updated battlefield situation model, φ represents the update function of the situation model, and adjusts the feature weights in the model based on the feedback data. t represents the current situation model, F t represents the feedback data at time t, The core of situation model update is the feature weight adjustment function, which is expressed as: In i,t+1 =in i,t +η Δw i,t , Among them, w i,t+1 represents the updated i-th feature weight, w i,t represents the current i-th feature weight, η represents the learning rate, Δw i,t represents the adjusted value of the feature, calculated from the feedback data, Optimize the grouping plan of unmanned swarm based on the updated results of the situation model in, It represents the optimized unmanned cluster formation scheme. represents the set of all possible grouping schemes, J represents the multi-task collaborative effectiveness index function, which is calculated based on feedback data and the latest situation model. Using deep reinforcement learning technology to improve game strategies and Perform iterative optimization: in, represents our optimal strategy after update, represents the updated enemy's optimal strategy, S t Indicates the current state, U f ,U e Represent our and enemy’s profit functions respectively, Using the optimized battlefield situation model, formation plan and game strategy, a closed-loop iteration is formed: in, represents the optimized dynamic closed loop set, M t+1 represents the optimized battlefield situation model, It represents the optimized unmanned cluster formation scheme. Represents the optimal strategy after optimization.

Images