An artificial intelligence-based combat task planning method and system

Abstract

Embodiments of the present application provide a combat task planning method and system based on artificial intelligence. The method comprises: designing a combat task planning artificial intelligence (AI) solving environment, wherein the combat task planning AI solving environment comprises 7 state characteristics, 5 decision actions and 1 comprehensive evaluation reward function; establishing a multi-layer deep combat decision neural network model; training the multi-layer deep combat decision neural network model by using an IMPALA reinforcement learning algorithm; and applying a result of the training to construct an intelligent combat task planning system, wherein the intelligent combat task planning system automatically generates a combat scheme, thereby solving a battle-level combat task planning solving problem.

Description

Technical Field

[0001] This invention relates to the fields of military operations research and artificial intelligence, specifically to an AI-based operational mission planning method and system. Background Technology

[0002] Operational mission planning, as the brain of the military information system, primarily involves planning and designing operational processes, task organization, force coordination, battlefield deployment, and weapon combination and application, based on the operational intentions of higher command and constrained by operational resources and rules. This process generates operational plans and schemes using scientific planning methods and computer tools. Operational mission planning is divided into four levels according to the command hierarchy of the service recipient: weapon platform level, tactical level, operational level, and strategic level.

[0003] With the continuous development of new weapons, technologies, and operational concepts, modern warfare is increasingly exhibiting a trend towards high-end development. On the one hand, it is characterized by large-scale and deep coordination, meaning that the size of participating forces, operational areas, weapon types, and quantities exceed those of the past, the number of targets has also increased significantly, and the coordinated use of different units, areas, and weapons has a significant impact on operational effectiveness, resulting in enormous pressure on operational mission planning. On the other hand, it is characterized by fast pace and intense confrontation, meaning that the battlefield situation changes rapidly, and weapons face complex fire interception and electronic jamming environments, requiring operational plans to be dynamically adjusted and generated accordingly, placing very high demands on the efficiency of operational mission planning. Therefore, facing large-scale, fast-paced modern high-end warfare, traditional operational mission planning methods such as mathematical programming or biomimetic evolution are difficult to model and have slow computation speeds, and can no longer meet the requirements of campaign-level operational mission planning.

[0004] Traditional operational mission planning methods include mathematical programming and biomimetic evolutionary methods. Mathematical programming methods, such as queuing theory, dynamic programming, storage theory, branch and bound, and backtracking, are difficult to model and prone to the curse of dimensionality when solving large-scale operational mission planning problems. Biomimetic evolutionary methods, such as genetic algorithms, particle swarm optimization, and differential evolution, are highly stochastic, lack generalization ability, and tend to converge prematurely when solving large-scale operational mission planning problems, resulting in poor optimization performance and long computation times.

[0005] In recent years, artificial intelligence technology has made significant breakthroughs, especially deep reinforcement learning. Because it organically combines the perception capabilities of deep learning with the decision-making capabilities of reinforcement learning, it is considered an important pathway towards general artificial intelligence and has been successfully applied in fields such as games, robot control, and parameter optimization, as seen in AlphaGo, StarCraft II, and Dota. Combat mission planning is essentially a combat decision-making problem based on battlefield situational awareness. The continuous development of deep reinforcement learning has opened up new avenues for the automated and autonomous solution of combat mission planning problems. Currently, it is mainly used for weapon platform-level planning problems (such as flight path planning, penetration decision planning, and guidance method planning) and tactical-level planning problems (such as UAV mission allocation and vehicle unit fire allocation), employing mainstream deep reinforcement learning algorithms such as Deep Q Networks (DQN), Proximal Policy Optimization (PPO), and SAC (Soft actor-critic).

[0006] For mainstream deep reinforcement learning algorithms such as DQN, PPO, and SAC, due to the large number of decision variables, complex constraints, and huge solution space of campaign-level combat mission planning problems, the state space and action space of their deep reinforcement learning models are huge. The DQN algorithm cannot achieve continuous action decision-making and the model training stability is insufficient. The PPO algorithm has low sample utilization and long training time. The SAC algorithm has poor parallelism and slow convergence speed. None of them can be applied to solving highly complex campaign-level combat mission planning problems. Summary of the Invention

[0007] This invention addresses the problem of planning campaign-level combat missions under the large-scale and fast-paced conditions of modern high-end warfare by proposing an artificial intelligence-based combat mission planning method and system.

[0008] In a first aspect, embodiments of the present invention provide an artificial intelligence-based combat mission planning method, which includes the following steps:

[0009] In a first aspect, embodiments of the present invention provide an artificial intelligence-based combat mission planning method, characterized in that the method includes the following steps:

[0010] S1. Design an AI-based environment for operational mission planning;

[0011] S2. Establish a multi-layered deep combat decision-making neural network model;

[0012] S3. The multi-layer deep combat decision neural network model is trained using the IMPALA (important weighted actor-learner architecture) reinforcement learning algorithm;

[0013] S4. The training results are used to construct an intelligent combat mission planning system, which automatically generates combat plans.

[0014] Furthermore, the AI ​​solution environment for designing combat mission planning described in step S1 includes design state characteristics, design action set, and design comprehensive evaluation reward function.

[0015] Furthermore, the status characteristics include target status, target marker status, troop status, troop marker status, operational area status, operational area marker status, and weapon type marker status, wherein...

[0016] The target status specifically includes: target attributes, which include the current target's sequence number and type, the damage level of each target, and the number of weapon types of the target;

[0017] The target status indicator includes: all valid targets;

[0018] Troop status, specifically including: mission troop attributes, which include the current troop's sequence number, the maximum number of weapons that can be fired in a single wave, the maximum number of available operational areas, and the operational area used;

[0019] Troop status indicators, specifically including: all weapon types available to the unit;

[0020] The operational area status includes: which unit is using the operational area, and the types and quantities of remaining weapons.

[0021] The status of the combat area indicators specifically includes: the availability status of each combat area;

[0022] The weapon type status includes the weapon type assigned to each target.

[0023] Furthermore, the action set includes actions related to target engagement, weapon type, troop movements, operational area, and weapon quantity.

[0024] The action to strike the target indicates the target of the current decision-making operational group.

[0025] The weapon type action indicates the type of weapon used by the combat group making the current decision;

[0026] Troop movements refer to the troops deployed by the operational group making the current decision.

[0027] Operational area actions indicate the operational area in which the operational group is currently conducting operations;

[0028] The weapon quantity action indicates the number of weapons the combat group is using to strike the target in the current decision.

[0029] Furthermore, the comprehensive evaluation reward function includes a combat effectiveness achievement reward function, a combat loss reduction reward function, and a combat risk management reward function, which is obtained according to the following formula:

[0030]

[0031] Where R is the reward value;

[0032] τ1 is the weight of the reward for achieving combat effectiveness, τ2 is the weight of the reward for reducing combat losses, and τ3 is the weight of the reward for managing combat risks. The weights are preset according to the actual situation.

[0033] r1 is the reward function for achieving operational effectiveness, r2 is the reward function for reducing operational losses, and r3 is the reward function for managing operational risks.

[0034] Furthermore, the multi-layer deep combat decision neural network model mentioned in step S2 includes a model input part and a model output part. The model output part includes a value network and a policy network. The establishment of the multi-layer deep combat decision neural network model includes the following steps:

[0035] S21. The input part receives three state features of the combat mission planning AI solution environment;

[0036] S22. The input part uses a fully connected neural network to convert the three state features into a consistent dimension and concatenates them into a feature vector, which is then input into the core network composed of a GRU recurrent neural network for time series feature extraction.

[0037] S23. The value network obtains the time series feature vector output by the core network, calculates the action value using a multi-layer fully connected neural network, and outputs the action value of the three state features.

[0038] S24. The target action policy neural network in the policy network obtains the time series feature vector output by the core network and the target flag state feature corresponding to the target state feature. The decision action filtering is realized through the target flag state feature, and the target feature vector is output.

[0039] The strategy network is composed of a target action strategy neural network, a weapon type action strategy neural network, a troop action strategy neural network, a combat area action strategy neural network, and a weapon quantity action strategy neural network connected in sequence, with the output of the preceding action strategy neural network serving as the input of the following action strategy neural network.

[0040] Furthermore, the target action policy neural network implementing decision action filtering in step S24 includes the following steps:

[0041] S241. Perform matrix multiplication on the time series feature vector output by the core network and the target flag state feature;

[0042] S242. Use a one-dimensional convolutional network to extract one-dimensional features of the sequence, and add the one-dimensional features of the sequence to the action masking vector mask;

[0043] S243. Calculate the probability distribution logits of decision actions using the Softmax logistic regression model, randomly select decision actions based on the probability distribution, and output the decision actions.

[0044] S244. The decision action is converted into a one-hot vector, which is then passed through a two-layer fully connected neural network to obtain the Embedding feature vector.

[0045] Furthermore, the training of the multi-layer deep combat decision neural network model in step S3 includes the following steps:

[0046] S31. Obtain operational intent and planning scenarios through the operational mission planning application module, wherein the operational mission planning application module is a component of the intelligent operational mission planning system.

[0047] S32. Construct a cluster of combat decision-making intelligent agents, wherein each combat decision-making intelligent agent includes an AI solution environment for combat mission planning and a multi-layer deep combat decision-making neural network model;

[0048] S33. Train each of the aforementioned combat decision-making agents one by one, generate samples, and store the samples in a sample sequence;

[0049] S34. After all the combat decision agents have been trained, the sample sequence obtained is stored in a sample set. If the sample set reaches the specified number requirement, the multi-layer deep combat decision neural network model is updated and output. Otherwise, the training of a single combat decision agent continues.

[0050] S35. Training ends, and the trained combat decision-making agent is obtained. The conditions for ending the training include either manual termination or termination after reaching a specified number of training sessions.

[0051] Furthermore, the step S33 of training each of the individual combat decision-making agents includes the following steps:

[0052] S331, the initialization state features of the AI ​​solution environment for the combat mission planning S t ;

[0053] S332, the combat mission planning AI solution environment will use the state features S t The output is fed into the multi-layer deep combat decision neural network model;

[0054] S333, The multi-layer deep combat decision neural network model receives the state feature S t The action set A is calculated. t ;

[0055] S334, The combat mission planning AI solution environment obtains the action set A. t It executes combat decision-making actions, calculates and processes the damage status of the targets, and adjusts the number of corresponding types of weapons in the combat mission planning AI solution environment based on the results of the execution of combat decision-making actions, and changes the troop status and combat area status.

[0056] S335, the state characteristics S of the AI ​​solution environment for combat mission planning returning subsequent decision actions. t+1 Given the reward value r and the end label d, generate samples (S). t A t ,r,S t+1 ,d).

[0057] Secondly, embodiments of the present invention provide an intelligent combat mission planning system, including a combat mission planning application module and a combat mission planning AI algorithm module;

[0058] The combat mission planning application module is used to: provide combat intentions and planning scenarios to the combat mission planning AI algorithm module, wherein the combat intentions and planning scenarios are issued by the commander or planner to the combat mission planning application module through human-machine delivery;

[0059] The combat mission planning AI algorithm module is used to automatically generate multiple combat schemes and plans for the combat mission planning application module. The combat mission planning AI algorithm module integrates a multi-layer deep combat decision neural network model and a combat mission planning AI solution environment.

[0060] The multi-layer deep combat decision neural network model and the combat mission planning AI solution environment are established according to one of the methods in steps S1 to S3 of the first aspect.

[0061] This invention designs an AI solution environment for combat mission planning, establishes a multi-layer deep combat decision neural network model, trains the multi-layer deep combat decision neural network model using the IMPALA reinforcement learning algorithm, and applies the training results to construct an intelligent combat mission planning system, thereby enabling the intelligent combat mission planning system to quickly and automatically generate combat plans. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the specific embodiments of the present invention, the drawings used in the description of the specific implementation or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating the overall implementation of an embodiment of the present invention;

[0064] Figure 2 This is a schematic diagram of the multi-layer deep combat decision neural network model structure according to an embodiment of the present invention;

[0065] Figure 3 This is a schematic diagram of a target action strategy neural network according to an embodiment of the present invention;

[0066] Figure 4 This is a schematic diagram of the composition structure and information flow of an intelligent combat mission planning system according to an embodiment of the present invention. Detailed Implementation

[0067] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method or system that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods or devices.

[0069] Figure 1 This is a flowchart illustrating the overall implementation of an embodiment of the present invention, as follows: Figure 1 As shown, an artificial intelligence-based combat mission planning method and system includes the following steps:

[0070] S1. Design an AI solution environment for combat mission planning, wherein the AI ​​solution environment for combat mission planning includes 7 state features, 5 decision actions and 1 comprehensive evaluation reward function;

[0071] S2. Establish a multi-layered deep combat decision-making neural network model;

[0072] S3. The multi-layer deep combat decision neural network model is trained using the IMPALA reinforcement learning algorithm.

[0073] S4. The training results are used to construct an intelligent combat mission planning system, which automatically generates combat plans.

[0074] In step S1, the design of the AI ​​solution environment for operational mission planning includes design state characteristics, design action set, and design comprehensive evaluation reward function.

[0075] 1. Design an AI to solve for the state characteristics S of the operational mission planning environment. t The state features include the target status, target marker status, troop status, troop marker status, operational area status, operational area marker status, and weapon type marker status. The state space S of the AI ​​solution environment... t Represented by the following array:

[0076] S t =[S mb ,S mz ,S bd ,S bz ,S dy ,S dz ,S wq ]

[0077] Among them, S mb To strike the target status, S mz To strike the target's status, S bd For troop status, S bz For troop marking status, S dy For operational area status, S dz For the status of the operational area, S wq The weapon type status is determined by the following calculation method:

[0078] ① Target status S mb

[0079] Record the target attributes. For the M targets requested by the superior, each target contains the following four attribute information:

[0080] a. Use an M-dimensional one-hot vector to represent the sequence number of the current target;

[0081] b. Each target has a clearly defined damage level from a superior authority;

[0082] c. Use an N-dimensional one-hot vector to represent the type of the current target;

[0083] d. Use a vector of length R to represent the number of weapon types of the target.

[0084] In summary, a two-dimensional matrix of M×(R+N+1) is used to describe the state S of the target. mb If the number of targets initialized in the environment is less than M, the missing target attributes are padded with 0 to maintain the consistency of the matrix dimensions.

[0085] ② Target status indicator S mz

[0086] The status of all valid targets is recorded. The target status is described by a vector [T1, T2, ..., TM] of length M. mz T i =1 indicates that the target has completed the allocation of firepower resources.

[0087] ③ Troop Status S bd

[0088] Record mission unit attributes. For the available S units, each unit contains the following attributes:

[0089] a. Current unit sequence number, represented by a one-dimensional vector of length S;

[0090] b. The maximum number of weapons that can be fired in a single wave by the current unit (FSDL) bc ;

[0091] c. The maximum number of operational areas currently available to the troops, T;

[0092] d. The operational area currently used by the troops is represented by a P-dimensional one-hot vector, where P is the sum of the number of available operational areas for all troops.

[0093] In summary, a two-dimensional matrix S×(T×P+S+1) is used to describe the troop state S. bd .

[0094] ④ Troop marking status S bz

[0095] Record all weapon types usable by the unit. Since a unit may operate from multiple independent operational areas, and each operational area can deploy various weapons, an S×R two-dimensional matrix can be used to describe the unit's status S. bz .

[0096] ⑤ Operational Area Status S dy

[0097] Record which unit is using the operational area and the remaining weapon types and quantities. For each operational area, use a one-hot vector of length P to represent the current operational area number, a one-hot vector of length S to represent which unit is using the operational area, and a vector of length R to store the remaining quantity of each type of weapon in the operational area. Then, the operational area state is a one-dimensional vector of length P × (P + S + R).

[0098] ⑥ Operational Area Marker Status S dz

[0099] Record the availability status of each operational area. Represent the operational area number using a one-hot vector of length P, and describe the operational area status S using a two-dimensional matrix of length S×(T×P). dz .

[0100] ⑦ Weapon type indicator status S wq

[0101] Record the weapon type assigned to each target. A two-dimensional matrix M×R is used to describe the weapon type flag state S. wq The element Swq in the i-th row and j-th column of the matrix. ij =1 indicates using the j-th type of weapon to strike the i-th target, Swq ij =0 indicates that the j-th type of weapon was not used to attack the i-th target.

[0102] 2. Design the action set A for the AI ​​solution environment of the combat mission planning, including five actions: target attack action, weapon type action, troop action, combat area action, and weapon quantity action. Specifically:

[0103]

[0104] Among them, A t Actions to strike targets indicate the target of the operational group making the current decision; A w The weapon type action indicates the type of weapon used by the combat group making the current decision; A a For troop actions, it indicates the troops being called up by the current operational group making the decision; A r For operational area actions, it indicates the operational area where the operational group currently making the decision is to operate; A n The weapon quantity action indicates the number of weapons the combat group will use to strike the target in the current decision; K is the maximum number of weapons that can be deployed to strike the target.

[0105] 3. Design a comprehensive evaluation reward function for the AI ​​solution environment of combat mission planning. The comprehensive evaluation reward function includes a reward function for achieving combat effectiveness, a reward function for reducing combat losses, and a reward function for managing combat risks. The comprehensive evaluation reward function is obtained according to the following formula:

[0106]

[0107] Where R is the reward value;

[0108] τ1, τ2, and τ3 are the weights of rewards for achieving operational effectiveness, reducing operational losses, and managing operational risks, respectively. These weights are usually determined by the commanders based on the decision-making intent.

[0109] r1 is the reward function for achieving optimal combat results, indicating that the core objective of combat task allocation is to achieve the best combat effect. r1 is calculated according to the following formula:

[0110]

[0111] Where, p i Let v be the probability that the i-th target is destroyed. i Let be the value coefficient of the i-th target.

[0112] r2 is the reward function for reducing combat losses. By rationally combining and using troops and combat areas, and optimizing weapon attack paths, losses caused by enemy fire interception and electronic jamming can be reduced. r2 is calculated according to the following formula:

[0113]

[0114] Where, N i Let i be the number of the i-th type of weapon used. Let be the probability that the i-th weapon is intercepted by the opponent's firepower. Let be the probability that the i-th weapon is electronically interfered with by the opponent;

[0115] r3 is the operational risk management reward function. By adjusting the allocation relationship between operational area, weapons, and targets, it can reduce collateral damage caused by weapon debris falling into or attack routes passing through sensitive areas. r3 is calculated according to the following formula:

[0116]

[0117] Where, N jbfq σ represents the number of sensitive areas. i To determine whether the weapon's attack path passes through the i-th sensitive area, σ i =1 indicates passing through, and vice versa; ω i The importance factor for the i-th sensitive area is usually assigned by the commander or planner according to the type of sensitive area.

[0118] In step S2, the multi-layer deep combat decision neural network model includes a model input part and a model output part. The model output part includes a value network and a policy network, such as... Figure 2 As shown, MLP refers to a fully connected network, Conv1d refers to a one-dimensional convolutional network, GRU refers to a gated recurrent unit neural network, Core refers to a core network composed of GRUs, Value refers to decision value, Encoder refers to the feature encoding obtained after feature extraction using the network, and Embedding refers to the feature vector after special processing using the network; the establishment of the multi-layer deep combat decision neural network model includes the following steps:

[0119] S21. The input part receives three state features of the combat mission planning AI solution environment, the three state features including the target status, the troop status, and the combat area status;

[0120] S22. The input part uses an MLP fully connected neural network to convert the three state features into encoder feature codes with consistent dimensions, and concatenates them into a feature vector, which is then input into the Core network composed of GRU recurrent neural networks for time series feature extraction.

[0121] S23. The value network obtains the time series feature vector output by the Core network, uses an MLP multilayer fully connected neural network to calculate the action value and outputs the action value Value of the three state features.

[0122] S24. The target action policy neural network in the policy network obtains the time series feature vector output by the Core network and the target flag state feature corresponding to the target state feature. The decision action filtering is achieved through the target flag state feature, and the Embedding target feature vector is output.

[0123] The strategy network is composed of a target action strategy neural network, a weapon type action strategy neural network, a troop action strategy neural network, a combat area action strategy neural network, and a weapon quantity action strategy neural network connected in sequence, with the output of the preceding action strategy neural network serving as the input of the following action strategy neural network.

[0124] In step S24, as Figure 3 As shown, the target action policy neural network implements decision action filtering by including the following steps:

[0125] S241. Perform matrix multiplication on the time series feature vector output by the Core network and the target flag state feature;

[0126] S242. Use a Conv1d one-dimensional convolutional network to extract one-dimensional features of the sequence, and add the one-dimensional features of the sequence to the action masking vector mask;

[0127] S243. Calculate the probability distribution of decision actions using the Softmax logistic regression model, randomly select target decision actions based on the probability distribution, and output the target decision actions.

[0128] S244. The target decision action is converted into a one-hot vector, and then passed through a two-layer fully connected neural network (FC2 and FC3) to obtain the Embedding target feature vector.

[0129] To implement action filtering, a mask is designed to exclude invalid actions from the action probability calculation during decision action probability calculation, thus ensuring that invalid actions are blocked during forward execution and backward propagation. The mask is obtained from the target flag, and the specific method is as follows:

[0130]

[0131] In the formula, when the target flag is 0, it means that the action is a valid action.

[0132] Furthermore, the construction methods for troop strategy neural networks, operational area strategy neural networks, weapon type strategy neural networks, weapon quantity strategy neural networks, and target action strategy neural networks are basically the same, such as... Figure 3 As shown, it will not be elaborated upon here.

[0133] In step S3, training a multi-layer deep combat decision neural network model using the IMPALA reinforcement learning algorithm includes the following steps:

[0134] S31. Obtain operational intent and planning scenarios through the operational mission planning application module, wherein the operational mission planning application module is a component of the intelligent operational mission planning system.

[0135] S32. Construct a cluster of combat decision-making intelligent agents, wherein each combat decision-making intelligent agent includes an AI solution environment for combat mission planning and a multi-layer deep combat decision-making neural network model;

[0136] S33. Train each of the aforementioned combat decision-making agents one by one, generate samples, and store the samples in a sample sequence;

[0137] S34, the (S) generated in step S33 above t A t ,r,S t+1d) The sample sequence constitutes a complete training sample. The training samples generated by multiple combat decision-making agents are stored in the sample set. If the number of samples in the sample set reaches a specified number, all samples are retrieved, the loss function is solved, and backpropagation is performed to update the network model. If the number of samples is insufficient, sample collection continues from step S33.

[0138] S35. Repeat the above steps until the training termination condition is met, resulting in a well-trained combat decision-making agent. The training termination condition includes either manual termination or termination after reaching a specified number of training iterations.

[0139] In step S33, the step of training each of the individual combat decision agents one by one includes:

[0140] S331, the initialization state features of the AI ​​solution environment for the combat mission planning S t ;

[0141] S332, the combat mission planning AI solution environment will use the state features S t The output is fed into the multi-layer deep combat decision neural network model;

[0142] S333, The multi-layer deep combat decision neural network model receives the state feature S t The action set A is calculated. t ;

[0143] S334, The combat mission planning AI solution environment obtains the action set A. t It executes combat decision-making actions, calculates and processes the damage status of the targets, and adjusts the number of corresponding types of weapons in the combat mission planning AI solution environment based on the results of the execution of combat decision-making actions, and changes the troop status and combat area status.

[0144] S335, the state characteristics S of the AI ​​solution environment for combat mission planning returning subsequent decision actions. t+1 Given the reward value r and the end label d, generate samples (S). t A t ,r,S t+1 ,d), determine whether the task assignment for all targets has been completed. If not, jump to step S331 to continue; if completed, continue to the next step.

[0145] In step S4, the intelligent combat mission planning system includes a combat mission planning application module and a combat mission planning AI algorithm module, such as... Figure 4 As shown.

[0146] The combat mission planning application module is used to: provide combat intentions and planning scenarios to the combat mission planning AI algorithm module. The combat intentions and planning scenarios are issued by the commander or planner to the combat mission planning application module through human-machine delivery.

[0147] The GPU computing server cluster is used to provide hardware support for the rapid computation of the combat mission planning AI algorithm module.

[0148] The combat mission planning AI algorithm module is used to automatically generate multiple combat schemes and plans for the combat mission planning application module. The combat mission planning AI algorithm module is integrated with a multi-layer deep combat decision neural network model and a combat mission planning AI solution environment. The multi-layer deep combat decision neural network model and the combat mission planning AI solution environment are established according to one of the methods in steps S1 to S3. The combat mission planning AI solution environment provides state features, as well as state features and reward values ​​of subsequent decision actions to the multi-layer deep combat decision neural network model. The multi-layer deep combat decision neural network model outputs decision actions to the combat mission planning AI solution environment.

[0149] Furthermore, this embodiment of the invention also employs the particle swarm optimization algorithm to solve the same campaign-level operational mission planning problem. To ensure the validity of the experimental results comparison, the input to the particle swarm optimization algorithm is completely identical to the input to the IMPALA deep reinforcement learning algorithm. For 1000 sets of operational mission planning problem cases, after the algorithm solves the problem, the operational plan is output, and the average reward score is calculated. The test results of the two algorithms are shown in Table 1.

[0150] Table 1 Comparison of test results between IMPALA deep reinforcement learning algorithm and particle swarm optimization algorithm

[0151] algorithm Average test time (s) Reward points Particle Swarm Optimization 200 0.96 IMPALA Deep Reinforcement Learning Algorithm 5 0.97

[0152] Therefore, the IMPALA deep reinforcement learning algorithm provided in this embodiment of the invention is 40 times more efficient than the traditional particle swarm optimization algorithm.

[0153] In summary, the AI-based combat mission planning method and system provided in this embodiment of the invention designs an AI solution environment for combat mission planning, establishes a multi-layer deep combat decision neural network model, trains the multi-layer deep combat decision neural network model using the IMPALA reinforcement learning algorithm, and applies the training results to construct an intelligent combat mission planning system, thereby realizing the rapid and automatic generation of combat plans by the intelligent combat mission planning system.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An artificial intelligence-based combat mission planning method, characterized by, The method comprises the following steps: S1, designing a combat task planning artificial intelligence (AI) solving environment, wherein the combat task planning AI solving environment comprises designed state features, a designed action set, and a designed comprehensive evaluation reward function; The state features comprise a strike target state, a strike target marker state, a troop state, a troop marker state, a combat region state, a combat region marker state, and a weapon type marker state; S2, establishing a multi-layer deep combat decision neural network model; The multi-layer deep combat decision neural network model comprises a model input part and a model output part, wherein the model output part comprises a value network and a policy network; The establishment of the multi-layer deep combat decision neural network model comprises the following steps: S21, the input part receives three state features of the combat task planning AI solving environment, wherein the three state features comprise a strike target state, a troop state, and a combat region state; S22, the input part converts the three state features into a consistent dimension by using a fully connected neural network, and splices them into a feature vector to input a core network composed of a GRU recurrent neural network for time series feature extraction; S23, the value network obtains the time series feature vector output by the core network, calculates the action value by using a multi-layer fully connected neural network, and outputs the action value of the three state features; S24, a target action policy neural network in the policy network obtains the time series feature vector output by the core network and the target marker state feature corresponding to the target state feature, realizes decision action filtering through the target marker state feature, and outputs a target feature vector; The policy network is composed of a target action policy neural network, a weapon type action policy neural network, a troop action policy neural network, a combat region action policy neural network, and a weapon quantity action policy neural network connected in sequence, wherein the output of a previous action policy neural network is used as the input of a subsequent action policy neural network; The target action policy neural network in step S24 realizes decision action filtering, comprising the following steps: S241, performing matrix multiplication operation on the time series feature vector output by the core network and the target marker state feature; S242, extracting a sequence one-dimensional feature by using a one-dimensional convolution network, and adding the sequence one-dimensional feature and an action mask vector mask; S243, calculating a decision action probability distribution logits by using a Softmax logistic regression model, randomly selecting a decision action based on the probability distribution, and outputting the decision action; S244, converting the decision action into a one-hot vector, passing it through two fully connected layer neural networks, and obtaining an Embedding feature vector; S3, training the multi-layer deep combat decision neural network model by using an IMPALA reinforcement learning algorithm; S4, constructing an intelligent combat task planning system by using the training result, and automatically generating a combat scheme by the intelligent combat task planning system.

2. The method of claim 1, wherein, The strike target state specifically includes: a strike target attribute, the strike target attribute including a serial number and a type of a current strike target, a damage level of each strike target, and a number of weapon types of the strike target; The strike target flag state includes: all valid strike targets; The force state specifically includes: a task force attribute, the force attribute including a serial number of a current force, a maximum number of weapons that can be launched in a single wave, a maximum number of available combat regions, and a used combat region; The force flag state specifically includes: all available weapon types of the force; The combat region state specifically includes: a remaining weapon type and a number of a combat region used by which force; The combat region flag state specifically includes: an available state of each combat region; The weapon type flag state specifically includes: a weapon type allocated for each target.

3. The method of claim 1, wherein, The action set includes: a strike target action, a weapon type action, a force action, a combat region action, and a weapon number action; The strike target action indicates a strike target of a combat group of a current decision; The weapon type action indicates a weapon type used by the combat group of the current decision; The force action indicates a force called by the combat group of the current decision; The combat region action indicates a combat region in which the force of the combat group of the current decision operates; The weapon number action indicates a number of weapons used by a strike target of the combat group of the current decision.

4. The method of claim 1, wherein, The comprehensive evaluation reward function includes a combat effect realization reward function, a combat loss reduction reward function, and a combat risk control reward function, and the comprehensive evaluation reward function is obtained according to the following formula: ， wherein, is a reward value; , a weight for implementing a reward for combat effectiveness, a weight for reducing a reward for combat loss, a weight for controlling a reward for combat risk, which is pre-set according to actual conditions; a reward function for combat effectiveness, a reward function for combat loss reduction, a reward function for combat risk management.

5. The method of claim 1, wherein, The training of the multi-layer deep combat decision neural network model in step S3 includes the following steps: S31, obtaining a combat intention and a planning scene through a combat task planning application module, wherein the combat task planning application module is a component of the intelligent combat task planning system; S32, constructing a combat decision agent cluster, wherein each combat decision agent includes a combat task planning artificial intelligence (AI) solving environment and a multi-layer deep combat decision neural network model; S33, training each combat decision agent one by one, generating a sample, and storing the sample in a sample sequence; S34, storing the sample sequence obtained after training all combat decision agents in a sample set, if the sample set reaches a specified number requirement, updating and outputting the multi-layer deep combat decision neural network model, otherwise, continuing to train each combat decision agent; S35, training is completed, and a trained combat decision agent is obtained, and the training completion condition includes two kinds of human termination or reaching a specified number of training times.

6. The method of claim 5, wherein, The training of each combat decision agent one by one in step S33 includes the following steps: S331, initialize a state feature of the combat mission planning artificial intelligence (AI) solution environment ; S332, the combat mission planning artificial intelligence (AI) solving environment will the state features output to the multi-layer deep combat decision neural network model; S333、the multi-layer deep combat decision-making neural network model receives the state features , and calculates an action set ; S334, the combat mission planning artificial intelligence (AI) solving environment acquires the action set and executes a combat decision action, calculates and processes a damage state of a strike target, adjusts the number of weapons of a corresponding type in the combat mission planning artificial intelligence (AI) solving environment according to a result of the execution of the combat decision action, and changes a force state and a combat region state; S335、the combat mission planning artificial intelligence (AI) solving environment returns the state features of the postorder decision action , the reward value , and the end label , and generates a sample .

7. An intelligent combat task planning system, including a combat task planning application module and a combat task planning artificial intelligence (AI) algorithm module, characterized in that: The combat mission planning application module is configured to provide a combat intent and a planning scene to a combat mission planning artificial intelligence (AI) algorithm module, the combat intent and the planning scene being issued by a commander or a planner to the combat mission planning application module through human-computer interaction. The combat mission planning AI algorithm module is configured to automatically generate a plurality of combat schemes and plans for the combat mission planning application module, the combat mission planning AI algorithm module being integrated by a multi-layer deep combat decision neural network model and a combat mission planning AI solving environment. The multi-layer deep combat decision neural network model and the combat mission planning AI solving environment are established according to the method of any one of claims 1-6.

Images