Multi-aircraft collaborative attack target allocation decision-making method based on near-end strategy optimization

Abstract

The invention discloses a multi-aircraft cooperative attack target allocation decision-making method based on near-end strategy optimization, and belongs to the technical field of multi-aircraft cooperative combat. The method comprises the following steps: constructing a target allocation optimization problem model; designing a deep reinforcement learning environment of a multi-aircraft collaborative strike target allocation problem, constructing a deep network model, and training the network model by adopting a near-end strategy optimization algorithm; and by applying the target distribution model, online solving of a collaborative strike target distribution problem is realized, a distribution scheme and expected total strike income are output, and the decision-making efficiency in a dynamic battlefield environment is improved. According to the method, the Actor-Critic network architecture is constructed and the training method is optimized, so that the solving efficiency of the model is remarkably improved under complex constraint conditions and large-scale application scenes, a high-precision allocation scheme can be quickly generated according to actual requirements of different scales, and real-time online solving of a collaborative strike target allocation problem is effectively supported.

Description

Technical Field

[0001] The present invention relates to the technical field of reinforcement learning, and in particular to a multi-aircraft cooperative strike target allocation decision-making method based on proximal policy optimization. Background Art

[0002] With the rapid development of information technology, multi-aircraft cooperative combat is gradually becoming a powerful means to enhance combat capabilities in modern battlefields. As an important form of cooperative combat, multi-aircraft cooperative strike targets, existing solution methods have low solution efficiency and are difficult to achieve online real-time decision-making when facing complex constraint conditions and large-scale scenarios.

[0003] Since reinforcement learning uses the method of an agent interacting with the environment to obtain rewards to learn strategies, applying it to the target allocation problem has the advantage of not requiring self-setting of sample labels compared with traditional deep learning methods. And how to construct the Markov process of reinforcement learning is the key to applying reinforcement learning to the target allocation problem. Aiming at the stability problems existing in the training process of existing reinforcement learning methods, the proximal policy optimization algorithm under the Actor-Critic architecture is adopted, which can effectively improve the training problem, accelerate the reward convergence speed, and improve the final solution accuracy of the model. In view of this, the present invention proposes a multi-aircraft cooperative strike target allocation decision-making method based on proximal policy optimization. Summary of the Invention

[0004] The purpose of the present invention is to propose a multi-aircraft cooperative strike target allocation decision-making method based on proximal policy optimization to solve the problems raised in the background art. The present invention first constructs an optimization problem model for target allocation, then designs a deep reinforcement learning environment for the multi-aircraft cooperative strike target allocation problem, constructs a deep network model, adopts the proximal policy optimization algorithm to train the network model, and finally applies the target allocation model to realize the online solution of the cooperative strike target allocation problem, output the allocation plan and the expected overall strike benefit, and improve the decision-making efficiency in the dynamic battlefield environment. The trained model can quickly provide high-quality allocation plans in scenarios of different scales.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A multi-aircraft cooperative strike target allocation decision-making method based on proximal policy optimization includes the following:

[0007] S1. Construct an optimization problem model for multi-aircraft cooperative strike target allocation;

[0008] S2. Design a deep reinforcement learning environment for the multi - vehicle collaborative strike target allocation problem and construct a deep network model. Adopt the proximal policy optimization algorithm to train the constructed network model to obtain a multi - vehicle collaborative strike target allocation model;

[0009] S3. Apply the multi - vehicle collaborative strike target allocation model to output the allocation plan and the expected overall strike benefit.

[0010] Preferably, the S1 specifically includes the following content:

[0011] Considering the damage probability and the cost of the aircraft, the specific optimization problem model F is expressed as:

[0012]

[0013] Among them, M represents the number of our aircraft; N represents the number of enemy aircraft; represents the value of variable X when the function F(X) takes the maximum value; [x ij M×N is a decision variable matrix of M rows and N columns. The first constraint condition indicates that x ij is a binary decision variable. If the i - th aircraft is assigned to the j - th target, then x ij = 1, otherwise x ij = 0, p ij ∈[0,1], which is the damage probability of the i - th aircraft hitting the j - th target; v j > 0 represents the threat level of the j - th target; c i > 0 is the cost of the i - th aircraft. The second constraint condition indicates that an aircraft can be assigned to at most one target.

[0014] Preferably, in S2, the design of the deep reinforcement learning environment for the multi - vehicle collaborative strike target allocation problem includes the design of state, action, and reward models, specifically:

[0015] Define the set of our aircraft as U = {u i | i = 1,2,…,M}, the total number of aircraft is M, and there are l u types of aircraft, satisfying where m k represents the number of the k - th type of aircraft; u i =(c i ,dp i ), where c i > 0 represents the cost of the i - th aircraft, and dp i > 0 represents the ammunition carried by the aircraft; the set of enemy targets is T = {t j |j = 1,2,L,N}, the total number of targets is N, and there are l​t A kind of target, satisfying where n k represents the quantity of the k-th kind of target; t j =(v j , h j ), where v j >0 represents the threat degree of the j-th target, and h j >0 represents the health degree of the target. Based on the above, calculate the damage probability p ij , and the formula is as follows:

[0016]

[0017] where dp i represents the ammunition quantity carried by the i-th aircraft; h j represents the health degree of the j-th target; thus, design the state, action, and reward models;

[0018] The design of the state model specifically includes: when allocating the i-th aircraft to a target, set the state as:

[0019]

[0020] where o i has a dimension of 1 and represents the remaining quantity of the total aircraft, expressed as the ratio of the cost of the allocated aircraft to the cost of the total aircraft; d i has a dimension of l t and represents the quantity of aircraft allocated to each type of target, expressed as the ratio of the cost of the aircraft allocated to each target to the cost of the total aircraft; has a dimension of l u and represents the one-hot encoding of the aircraft types;

[0021] The design of the action model specifically includes: when allocating an aircraft to the j-th target, set the action as:

[0022]

[0023] where n j has a dimension of l u and represents the cost of each type of aircraft allocated to the current target; v j has a dimension of 1 and represents the threat degree of the current target; has a dimension of l t and represents the one-hot encoding of the target types;

[0024] The design of the reward model specifically includes: set the joint reward function of the single-step reward and the global reward:

[0025] r = αr s +(1 - α)rg (5)

[0026] Among them, α ∈ [0, 1], representing the adjustment factor; r s and r g respectively represent the single-step reward and the global reward, and:

[0027]

[0028] Among them, F represents the joint optimization model; represents the allocation scheme after single-step allocation; represents the allocation scheme before single-step allocation; represents the final allocation scheme after completion; M is the total number of aircraft.

[0029] Preferably, the construction of the deep network model in S2 includes constructing an Actor network and a Critic network based on the forward network model;

[0030] Among them, the Actor network consists of an input layer, three hidden layers and an output layer. The number of nodes in the hidden layers is 64, 128, 64 respectively. The dimension of the input layer is 2(1 + l u + l t ), the dimension of the output layer is 1, and the activation function is selected as leakyrelu(0.1), which is used to evaluate the state-action pair <s, a>;

[0031] The Critic network consists of an input layer, three hidden layers and an output layer. The number of nodes in the hidden layers is 64, 128, 64 respectively. The dimension of the input layer is 1 + l u + l t , the dimension of the output layer is 1, and the activation function is selected as leaky relu(0.1), which is used to evaluate the state s;

[0032] For the Actor network, after the values corresponding to all possible actions a corresponding to the current state s are output by this network, the softmax function is used to convert them into a probability distribution for subsequent action selection and network parameter update.

[0033] Preferably, the proximal policy optimization algorithm is adopted in S2 to train the constructed deep network model, including:

[0034] The proximal policy optimization algorithm is used to update the network parameters of the Actor network, which is defined as follows:

[0035]

[0036] Among them, θ represents the Actor network parameters; represents the value of θ when the objective function is maximized, mean represents taking the average of the sampled data; min represents taking the smaller of the two; π θ (a t |s t ) and respectively represent the new policy and the old policy, that is, the probability distribution of selecting each action under this policy for state s t ; A t is the advantage function obtained by generalized advantage estimation:

[0037]

[0038] where γ ∈ [0, 1] is the discount factor, λ ∈ [0, 1] is the hyperparameter of generalized advantage estimation; Δ t+l is the temporal difference error at t + l; the clip function is the clipping function, and its definition is:

[0039]

[0040] The clip function is used to limit the range of the ratio of the new and old policies; ε is the limiting factor;

[0041] For the Critic network, the network parameters are updated according to the TD error, and its TD error optimization objective is defined as follows:

[0042]

[0043] where ω represents the Critic network parameters; represents the value of ω when the objective function is minimized; represents taking the average of the sampled data; r t represents the current reward; γ is the reward discount factor; V ω (s) represents the output of the Critic network; t + 1 represents the next moment, and t represents the current moment.

[0044] Preferably, the S3 specifically includes the following content:

[0045] Input the cost, ammunition quantity, type of the aircraft and the value, health, type data of the target into the trained multi-aircraft cooperative strike target allocation decision model, and output the allocation plan of the aircraft to the target, the expected revenue and the decision-making time.

[0046] Compared with the prior art, the present invention provides a multi-aircraft cooperative strike target allocation decision method based on proximal policy optimization, having the following beneficial effects:

[0047] (1) The present invention trains a network model based on the Proximal Policy Optimization (PPO) algorithm. This method limits the ratio of the new and old policies by introducing a truncation mechanism, making the new policy and the old policy approximate. The trained model can solve problems faster and with higher quality compared to existing reinforcement learning models.

[0048] (2) In the network structure of the present invention, the state and action vectors are jointly input at the input layer of the Actor network, enabling the trained model to adapt to problem inputs of different scales and quickly and efficiently give allocation solutions.

[0049] (3) The present invention designs a joint reward function based on single-step rewards and global rewards. After a round of allocation results is completed, the global reward can be updated to the single-step rewards of each state, thereby improving the training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings involved in the embodiments are briefly introduced below. Obviously, the drawings in the following description are only schematic illustrations of some embodiments of the present invention, and those skilled in the art can also construct other forms of drawings based on these drawings without creative efforts.

[0051] Figure 1 FIG. is the overall flowchart of the multi-aircraft cooperative strike target allocation decision-making method based on proximal policy optimization proposed according to the present invention;

[0052] Figure 2 FIG. is a schematic diagram of the reinforcement learning framework proposed in Embodiment 1 of the present invention;

[0053] Figure 3 FIG. is a schematic diagram of the Actor network structure proposed in Embodiment 1 of the present invention;

[0054] Figure 4 FIG. is a schematic diagram of the Critic network structure proposed in Embodiment 1 of the present invention;

[0055] Figure 5 FIG. is a schematic diagram of the change curve of the final profit distribution during the training process proposed in Embodiment 1 of the present invention;

[0056] Figure 6 FIG. is the allocation solution given by the decision-making model under different problem scales in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them.

[0058] Example 1:

[0059] Please refer to Figure 1 , this example proposes a multi - vehicle collaborative strike target allocation decision - making method based on Proximal Policy Optimization (PPO), which specifically includes the following steps:

[0060] Step 1: Construct an optimization problem model for multi - vehicle collaborative strike target allocation

[0061] The constructed optimization problem model considers both the damage probability and the vehicle cost. The specific optimization problem model F can be expressed as:

[0062]

[0063] Among them, M represents the number of our side's vehicles, N represents the number of enemy vehicles, represents the value of variable X when function F(X) takes the maximum value, [x ij M×N is a decision - variable matrix of M rows and N columns. The first constraint indicates that x ij is a binary decision variable. If the i - th vehicle is assigned to the j - th target, then x ij = 1; otherwise, x ij = 0, p ij ∈[0, 1], which is the damage probability of the i - th vehicle hitting the j - th target, v j > 0, which is the threat degree of the j - th target, c i > 0 is the cost of the i - th vehicle. The second constraint indicates that a vehicle can be assigned to at most one target.

[0064] Step 2: Design a deep reinforcement learning environment for the multi - vehicle collaborative strike target allocation problem, construct a deep network model, and adopt the Proximal Policy Optimization algorithm to train the network model

[0065] It is necessary to construct the target allocation problem into a multi - stage sequential decision - making problem. Each step of decision - making completes the target allocation of one vehicle. When the target allocation of all vehicles is completed, a round of sequential decision - making is completed, and a Markov process is established to meet the requirements of reinforcement learning. In this example, our side's vehicle set is U = {u i |i = 1, 2, …, M}, the total number of vehicles is M, and there are l u types of vehicles, satisfying where m k represents the number of the k - th type of vehicle, u i =(c i , dp i ), where c i > 0 represents the cost of the vehicle, dp i ​> 0 represents the amount of ammunition carried by the aircraft, and the set of enemy targets is T = {t j | j = 1, 2, L, N}, the total number of targets is N, and there are l t types of targets, satisfying where n k represents the number of the k-th type of target, and t j = (v j , h j ), where v j > 0 represents the threat level of the target, and h j > 0 represents the health level of the target; the formula for calculating the damage probability p ij is as follows:

[0066]

[0067] Specifically, in this example, there are 2 types of aircraft in total for our side, with 36 in number. u i corresponds to two values of (1, 6), (1.25, 6); there are 4 types of targets in total for the enemy side, with 18 in number. t j corresponds to four values of (4, 1), (6, 2), (8, 4), (16, 8). The number of each type of our aircraft and enemy targets is uncertain during each training to improve the final generalization of the model.

[0068] Next, construct the state, action, and reward models: When allocating the i-th aircraft to a target, set the state as:

[0069]

[0070] Among them, o i has a dimension of 1 and represents the remaining amount of the total aircraft, expressed as the ratio of the cost of the allocated aircraft to the cost of the total aircraft. d i has a dimension of l t and represents the number of aircraft allocated to each type of target, expressed as the ratio of the cost of the aircraft allocated to each target to the cost of the total aircraft. has a dimension of l u and represents the one-hot encoding of the aircraft types; when allocating an aircraft to the j-th target, set the action as:

[0071]

[0072] Among them, n j has a dimension of l u and represents the cost of each type of aircraft allocated to the current target; v j has a dimension of 1 and represents the threat level of the current target. has a dimension of l t, a one-hot encoding representing the target type; set the combined reward function for the single-step reward and the global reward as:

[0073] r = αr s +(1 - α)r g (5)

[0074] where α ∈ [0, 1] is the adjustment factor; r s and r g represent the single-step reward and the global reward respectively, and:

[0075]

[0076]

[0077] where F is the joint optimization model; represents the allocation plan after single-step allocation, represents the allocation plan before single-step allocation; represents the final completed allocation plan, and M is the total number of aircraft. Specifically, in this example, the number of our two types of aircraft is 24, 12, and the number of the enemy's 4 types of targets is 9, 3, 5, 1. Then our total cost is 39. When about to allocate the 4th aircraft to the 3rd target, and three aircraft of the first type have been previously allocated to the first, second, and third types of targets, the corresponding state at this time is s = (12 / 13, (1 / 13, 1 / 13, 1 / 13, 0), (1, 0)); if the 3rd target selected is the first type of target and no aircraft has been allocated to this target yet, the corresponding action is a = ((0, 0), 4, (1, 0, 0, 0)); the corresponding single-step reward is calculated by the formula of r s , and after completing one round of allocation, the global reward r g is updated to the single-step rewards of each state to obtain the combined reward.

[0078] Then construct a deep network model, construct an Actor network and a Critic network based on the forward network model. Among them, the Actor network needs to interact with the environment and learn better strategies under the guidance of the Critic network. The Critic network learns a value function to help the Actor update its strategy. Figure 2 and Figure 3The structural diagrams of two networks are given. In terms of network structure parameters, the learning rate of the Actor network is set to 1e-3, the number of nodes in the hidden layers of the Actor network is 64, 128, 64, the learning rate of the Critic network is 1e-2, the number of nodes in the hidden layers of the Critic network is 64, 128, 64, and the activation function is selected as leaky relu(0.1). During the training process, the two networks optimize their respective objective functions, and the two cooperate with each other to enable the allocation strategy to converge in a relatively short time.

[0079] Finally, the proximal policy optimization algorithm is adopted to update the network parameters. The specific reinforcement learning architecture refers to Figure 4 As shown, the proximal policy optimization algorithm is used to determine the policy update amplitude during the training process, avoiding overly drastic policy updates, which can better adapt to environmental changes and conform to the application scenario of the aircraft cluster dynamically perceiving the position and type of the target and then adjusting the cooperative strike strategy. Specifically, for the Actor network, its optimization objective is defined as follows:

[0080]

[0081] where θ represents the Actor network parameters, represents the value of θ when the objective function is maximized, represents taking the average value of the sampled data, min represents taking the smaller of the two, and π θ (a t |s t ) and represent the new policy and the old policy respectively, that is, the probability distributions of selecting each action in the state s under this policy, and A t is the advantage function obtained by generalized advantage estimation: t For which, γ ∈ [0, 1] is the discount factor, λ ∈ [0, 1] is the hyperparameter of generalized advantage estimation, Δ

[0082]

[0083] where, γ ∈ [0, 1] is the discount factor, λ ∈ [0, 1] is the hyperparameter of generalized advantage estimation, Δ t+l is the temporal difference error at t + l, and the clip function is the clipping function, and its definition is:

[0084]

[0085] is used to limit the range of the ratio of the new and old policies, ε is the limiting factor, and ε is set to 0.2 in this example; for the Critic, the network parameters are updated according to the TD error, and its TD error optimization objective is:

[0086]

[0087] Among them, ω represents the parameters of the Critic network, represents the value of ω when the objective function is minimized, represents taking the average value of the sampled data, r t represents the current reward, V ω (s) represents the output of the Critic network, t + 1 represents the next moment, t represents the current moment, and γ is the reward discount factor. In this example, γ is set to 0.98.

[0088] Step 3: Apply the multi - aircraft collaborative strike target allocation model to output the allocation plan and the expected overall strike benefit.

[0089] In this example, according to the steps described above, a multi - aircraft collaborative strike target allocation decision model trained based on the proximal policy optimization algorithm can be obtained. As Figure 5 is the change curve of the allocated final benefit during the model training process. The model was tested 50 times on multiple different examples with 36 of our aircraft and 18 enemy targets, and better results were obtained compared with existing reinforcement learning algorithms (such as DQN, etc.). In terms of the average value of the allocated final benefit, the method of the present invention is 79.10649, and the existing DQN algorithm is 78.31408. In terms of the standard deviation, the method of the present invention is 2.37637, and the existing DQN algorithm is 2.55247; in terms of the average value of the single - round solution time, the method of the present invention is 0.28906s, and the existing DQN algorithm is 0.30118s. While the solution time is reduced, the solution quality is also improved. In addition, the model trained by this method can also give high - quality allocation plans for problems of various scales, Figure 6 shows the allocation results under different scales.

[0090] The above - mentioned is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and all should be covered by the protection scope of the present invention.

Claims

1. A multi-aircraft cooperative strike target assignment decision-making method based on proximal policy optimization, characterized in that It includes the following content: S1. Construct an optimization problem model for multi-aircraft cooperative strike target allocation; S2. Design a deep reinforcement learning environment for multi-aircraft cooperative strike target allocation problem and construct a deep network model. Adopt the proximal policy optimization algorithm to train the constructed network model to obtain a multi-aircraft cooperative strike target allocation model; S3. Apply the multi-aircraft cooperative strike target allocation model to output the allocation plan and the expected overall strike benefit.

2. The method according to claim 1, wherein The specific content of S1 includes the following: Considering the damage probability and aircraft cost, the specific optimization problem model F is expressed as: Among them, M represents the number of our aircraft; N represents the number of enemy aircraft; represents the value of variable X when the function F(X) takes the maximum value; [x ij M×N is a decision variable matrix of M rows and N columns. The first constraint condition indicates that x ij is a binary decision variable. If the i-th aircraft is assigned to the j-th target, then x ij = 1; otherwise, x ij = 0, p ij ∈[0,1] is the damage probability of the i-th aircraft hitting the j-th target; v j > 0 represents the threat level of the j-th target; c i > 0 is the cost of the i-th aircraft. The second constraint condition indicates that an aircraft can be assigned to at most one target.​ 3. The method according to claim 1, characterized in that The design of the deep reinforcement learning environment for the multi-aircraft cooperative strike target allocation problem in S2 includes the design of the state, action, and reward models, specifically: Define the set of our aircraft as U = {u i | i = 1, 2, …, M}, where M is the total number of aircraft, and there are l u types of aircraft, satisfying where m k represents the number of the k-th type of aircraft; u i = (c i , dp i ), where c i > 0 represents the cost of the i-th aircraft, and dp i > 0 represents the amount of ammunition carried by the aircraft; the set of enemy targets is T = {t j | j = 1, 2, …, N}, where N is the total number of targets, and there are l t types of targets, satisfying where n k represents the number of the k-th type of target; t j = (v j , h j ), where v j > 0 represents the threat level of the j-th target, and h j > 0 represents the health of the target. Based on the above, calculate the damage probability p ij , and the formula is as follows: where dp i represents the amount of ammunition carried by the i-th aircraft, and h j represents the health of the j-th target; based on this, the state, action, and reward models are designed; The design of the state model specifically includes: when allocating the i-th aircraft to a target, the state is set as: Among them, o i The dimension is 1, representing the remaining amount of the total aircraft, expressed as the ratio of the cost of the allocated aircraft to the cost of the total aircraft; d i The dimension is l t , representing the number of aircraft allocated to each type of target, expressed as the ratio of the cost of the aircraft allocated to each target to the cost of the total aircraft; The dimension is l u , representing the one-hot encoding of the aircraft types; The design of the action model specifically includes: when allocating an aircraft to the j-th target, the action is set as: where n j is of dimension l u , representing the costs of various types of aircraft assigned to the current target; v j is of dimension 1, representing the threat level of the current target; is of dimension l t , representing the one-hot encoding of the target type; The design of the reward model specifically includes: setting a joint reward function for single-step reward and global reward: r＝αr s +(1-α)r g (5) where α ∈ [0, 1] represents a regulation factor; r s and r g represent a single-step reward and a global reward respectively, and: Among them, F represents the joint optimization model; represents the allocation plan after single-step allocation; represents the allocation plan before single-step allocation; represents the final allocation plan; M is the total number of aircraft.

4. The method according to claim 1, characterized in that, The construction of the deep network model in S2 includes constructing an Actor network and a Critic network based on the forward network model; Among them, the Actor network consists of an input layer, three hidden layers and an output layer. The number of nodes in the hidden layers are 64, 128, and 64 respectively. The dimension of the input layer is 2(1 + l u + l t ), and the dimension of the output layer is 1. The activation function is selected as leaky relu(0.1), which is used to evaluate the state-action pair <s, a>; The Critic network consists of an input layer, three hidden layers, and an output layer. The number of nodes in the hidden layers is 64, 128, and 64 respectively, and the dimension of the input layer is 1 + l u + l t , and the dimension of the output layer is 1. The activation function is chosen as leaky relu(0.1) and is used to evaluate the state s; For the Actor network, after the values corresponding to all possible actions a corresponding to the current state s are output by the network, they are transformed into a probability distribution using the softmax function for subsequent action selection and network parameter update.

5. The method according to claim 1, characterized in that, The adoption of the proximal policy optimization algorithm to train the constructed deep network model in S2 includes: Adopt the proximal policy optimization algorithm to update the network parameters of the Actor network, which is defined as follows: where $\theta$ represents the Actor network parameters; denotes the value of $\theta$ when the objective function is maximized, denotes taking the average of the sampled data; min denotes taking the smaller of the two; $\pi$ θ (a t |s t ) and denote the new policy and the old policy respectively, that is, the probability distributions of selecting each action in the state s t under this policy; A t is the advantage function obtained by generalized advantage estimation: where γ∈[0,1] is the discount factor and λ∈[0,1] is the hyperparameter of generalized advantage estimation; Δ t+l is the temporal difference error at time t + l; the clip function is the clipping function, and its definition is: The clip function is used to limit the range of the ratio of the new and old policies; ε is the limiting factor; For the Critic network, update the network parameters according to the TD error, and its TD error optimization target is defined as follows: Among them, ω represents the parameters of the Critic network; represents the value of ω when the objective function is minimized; represents taking the average value of the sampled data; r t represents the current reward; γ is the reward discount factor; V ω (s) represents the output of the Critic network; t + 1 represents the next moment, and t represents the current moment.

6. The method according to claim 1, characterized in that The specific content of S3 includes the following: Input the cost, ammunition quantity, type of the aircraft, and the value, health status, type data of the target into the trained multi-aircraft cooperative strike target allocation decision model to output the allocation plan of the aircraft to the target, the expected benefit, and the decision-making time.

Images