Fire distribution method, device and medium comprising target grouping and reinforcement learning

Abstract

The present application relates to the technical field of firepower distribution, in particular to a firepower distribution method, device and medium containing target grouping and reinforcement learning. The present application contains a firepower distribution method with collateral damage grouping and deep reinforcement learning, establishes a target grouping strategy based on damage area, and then considers the collateral damage of the aircraft to the target; carries out target damage effect evaluation based on sight point optimization in the target group, improves the accuracy and rationality of damage effect calculation; finally, a deep reinforcement learning model of firepower distribution is established, which maximizes the damage cost-effectiveness ratio while ensuring the expected damage effect on the target.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of firepower distribution, in particular to a firepower distribution method, device and medium comprising target grouping and reinforcement learning. BACKGROUND

[0002] Firepower distribution refers to reasonably allocating firepower resources and determining an optimal distribution scheme according to target value, aircraft performance, target threat and other constraint conditions in combination with real-time situation information, and is one of the core problems of cooperative combat command decision-making. Firepower distribution is an NP-complete problem with multiple parameters and constraints, and with the increase of aircraft types and quantity and target types and quantity, the number of possible solutions increases exponentially, and traditional optimization algorithms are often low in solving efficiency and are prone to local optimal solution, so how to improve the solving speed and accuracy of firepower distribution has become a big problem to be solved.

[0003] The existing firepower distribution method is prone to local optimal solution and slow in solving speed, and is difficult to meet the requirement of rapid planning decision-making in modern strong uncertain environment; the existing firepower distribution method basically assumes that the attack effect of a certain aircraft on a certain target is a constant value or a rough estimated value based on a simple empirical formula when modeling, and the attack effect under different aiming points can be very different, which will inevitably affect the rationality of firepower distribution; the existing firepower distribution method does not consider the damage effect of an aircraft on an unallocated target, however, the damage range of some aircraft is large, which will bring about collateral damage to the targets near the allocated target, and ignoring the collateral damage effect directly leads to waste of firepower resources. SUMMARY

[0004] Therefore, the present application provides a firepower distribution method, device and medium comprising target grouping and reinforcement learning, which can maximize the damage efficiency-cost ratio while ensuring the expected damage effect on the target.

[0005] To achieve the above-mentioned purpose, the present application provides a firepower distribution method comprising target grouping and reinforcement learning, establishes a target grouping strategy based on damage range, and then considers the collateral damage of an aircraft to a target; carries out target damage effect evaluation based on aiming point optimization in the target group; finally, establishes a deep reinforcement learning model of firepower distribution by taking the platform position of each aircraft, the type of carried aircraft, the number of carried aircraft, the target grouping situation, the type and position of each group target as the model input, and taking the firepower distribution result as the model output, so as to maximize the damage efficiency-cost ratio while ensuring the expected damage effect on the target.

[0006] The target grouping strategy based on damage range is that the maximum damage radius of an aircraft is taken as a grouping distance threshold, a DSCAN clustering method based on relative distance is adopted to realize the grouping of multiple targets, and a target group, which is the basic unit of firepower distribution, is formed.

[0007] Wherein, by establishing a targeting point planning model and a damage effect evaluation model for each target group, target damage effect evaluation based on targeting point optimization is carried out in the target group; the targeting point planning model refers to planning the optimal targeting point of each aircraft for the single-group target of the fire distribution basic unit, realizing the maximum damage to the target under the premise of the specified aircraft launch platform, aircraft type and quantity, target type and quantity; the damage effect evaluation model refers to calculating the aircraft-target intersection condition and evaluating the target damage under the specified targeting point, serving as the judgment basis for the pros and cons of the target allocation result.

[0008] Wherein, the targeting point planning method adopts a heuristic algorithm or an integer programming algorithm for solution, and the damage effect evaluation is realized by looking up a damage database table in advance, using an empirical formula, a proxy model or precise finite element calculation.

[0009] Wherein, based on the input of the model, a state space S = {W, T} of deep reinforcement learning is constructed, wherein W = {w1,..., w j ,…,w m} is the state information of all available m aircraft platforms,

[0010] is the state information of the aircraft platform j, lon w 、lat w and H w are the longitude, latitude and altitude of the aircraft platform, respectively, type w i and n w i are the type and quantity of the i-th aircraft in the aircraft platform, respectively, is the cost of the i-th aircraft; T = {t1,..., t j ,…,t q} is the state information of all q groups of targets, is the state information of the j-th group of targets, wherein lon t 、lat t and are the longitude, latitude and altitude of the target, respectively, type t i is the type of the target i in the group, V i t is the value of the target i, P i * is the expected damage probability of the target i.

[0011] Based on the output of the model, an action space A = [a1,..., a m ,…,aq ], wherein is the fire distribution result for the mth group of targets, is the wth i aircraft platform uses n i type i aircraft to attack the mth group of targets

[0012] Wherein, the deep neural network model is designed by using the Transformer, the Encoder module based on the Transformer and the Decoder module are constructed, the Encoder module encodes the state space of indefinite length into the embedded feature map of fixed size, the Decoder module decodes the embedded feature map into the action space of specified size, the deep neural network model is constructed by combining the Actor-Critic architecture.

[0013] Wherein, the reward function is designed by combining the aiming point optimization and damage effect evaluation.

[0014] Wherein, after the deep reinforcement learning model is constructed, the deep reinforcement learning model is iterated and updated continuously by the continuous offline training and the feedback of the exercise data.

[0015] The present application provides an electronic device, which comprises a processor, a memory for storing executable instructions of the processor; the processor is used for reading the executable instructions from the memory and executing the instructions to realize the fire distribution method comprising target grouping and reinforcement learning according to the present application.

[0016] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program is used for executing the fire distribution method comprising target grouping and reinforcement learning according to any one of the present application.

[0017] Advantages:

[0018] 1. The method of the present application is a fire distribution method comprising collateral damage grouping and deep reinforcement learning, a target grouping strategy based on damage area is established, and then the collateral damage of the aircraft to the target is considered; the target damage effect evaluation based on the aiming point optimization is carried out in the target group, the accuracy and rationality of the damage effect calculation are improved; finally, the deep reinforcement learning model of the fire distribution is established, the expected damage effect to the target is ensured while the damage cost-effectiveness ratio is maximized.

[0019] 2.The method breaks through the limitation of traditional rough estimation with poor accuracy, and builds a targeting point planning model and a damage effect evaluation model for each target. The targeting point planning model refers to a single target group of a basic unit of fire distribution. Under the premise of a specified aircraft launch platform, aircraft type and quantity, target type and quantity, the optimal targeting point of each aircraft is planned to achieve maximum damage to the target. The damage effect evaluation model refers to the calculation of the aircraft-target intersection condition under the specified targeting point to quickly evaluate the target damage situation as the basis for judging the pros and cons of the target distribution result, which can accurately evaluate the pros and cons of the fire distribution result. Among them, the targeting point planning method can be quickly solved by using traditional heuristic algorithms or integer programming algorithms, while the damage effect evaluation can be achieved by traditional methods, such as looking up the damage database table in advance, or using empirical formulas, proxy models or precise finite element calculation.

[0020] 3.In the present application, a deep neural network model is designed using Transformer, an Encoder and Decoder module based on Transformer are constructed, the Encoder module encodes the state space of indefinite length into a fixed size embedding feature map, the Decoder module decodes the embedding feature map into a specified size action space, a deep neural network model is constructed by combining the Actor-Critic architecture to solve the problem that the input and output of the model are both indefinite length data.

[0021] 4.In the method of the present application, the reward function is designed by combining targeting point optimization and damage effect evaluation, which can more accurately reflect the pros and cons of the fire distribution result.

[0022] 5.In the method of the present application, the weight of the reward function is adjusted reasonably to guide the deep reinforcement learning model to make the optimal fire distribution decision.

[0023] 6.After the deep reinforcement learning model is constructed, the model is continuously iterated and updated through offline training and feedback of the exercise data to ensure the rationality and reliability of the fire distribution result. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 It is a schematic diagram of the overall architecture of the fire distribution based on deep reinforcement learning in the method of the present application.

[0025] Figure 2 It is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0026] The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0027] The application provides a firepower distribution method containing target grouping and reinforcement learning, establishes a target grouping strategy based on damage range, and further considers the collateral damage of an aircraft to a target; target damage effect evaluation based on a sighting point optimization is carried out in the target group, so that the accuracy and rationality of damage effect calculation are improved; finally, a deep reinforcement learning model of firepower distribution is established, which maximizes the damage cost ratio while ensuring the expected damage effect on the target, and the specific implementation steps of the method are as follows:

[0028] Step one, considering the target grouping of collateral damage.

[0029] In order to ensure the full use of firepower resources, the collateral damage of an aircraft to the allocated target and the collateral damage of the aircraft to the adjacent target of the allocated target are considered, and a target grouping method considering collateral damage is constructed based on the power field and damage range of the aircraft. In order to ensure the rationality of grouping, the maximum damage radius of the aircraft is used as the grouping distance threshold, and the DSCAN clustering method based on relative distance is used to realize the rapid grouping of multiple targets, and form the basic unit of firepower distribution-target group.

[0030] Step two, for each target group, a sighting point planning and damage effect evaluation model is established.

[0031] In order to accurately evaluate the pros and cons of firepower distribution results, break through the limitation of traditional rough estimation value with poor accuracy, a sighting point planning and damage effect evaluation model is established for each target group.

[0032] The sighting point planning model refers to planning the optimal sighting point of each aircraft for the single-target group of the basic unit of firepower distribution under the premise of specified aircraft launch platform, aircraft type and quantity, target type and quantity, and realizing the maximum damage to the target. The damage effect evaluation model refers to calculating the aircraft-target intersection condition under the specified sighting point, quickly evaluating the target damage, and serving as the judgment basis for the pros and cons of the target allocation result. The sighting point planning method here can be quickly solved by using traditional heuristic algorithm or integer programming algorithm, and the damage effect evaluation can be realized by using traditional method, such as looking up the damage database table in advance, or using experience formula, proxy model or precise finite element calculation.

[0033] Step three, a deep reinforcement learning model of firepower distribution is established.

[0034] The platform position of each aircraft, the type of carried aircraft, the number of carried aircraft, the target grouping situation, the type and position of each group target are used as the input of the model, and the firepower distribution result is used as the output of the model, and the overall architecture of firepower distribution is as shown in Figure 1

[0035] Based on the input of the model, the state space S={W,T} of deep reinforcement learning is constructed, wherein​

[0036] W = {w1,...,w} j ,…,w m} represents the status information of all m available aircraft platforms.

[0037] For the status information of aircraft platform j, lon w lat w and H w These represent the longitude, latitude, and altitude of the aircraft platform, respectively, and type. w i and n w i These represent the type and number of the i-th type of aircraft in the aircraft platform. The cost of the i-th type of aircraft; T = {t1,...,t} j ,…,t q} represents the state information of all q groups of targets. Let lon be the state information of the j-th target group. t lat t and These represent the target's longitude, latitude, and altitude, respectively, and type. t i V represents the type of target i within this group. i t For the value of target i, P i * Let be the expected damage probability of target i.

[0038] Based on the model's output, construct the action space A = [a1,...,a2] for deep reinforcement learning. m ,...,a q ],in This represents the firepower allocation result for the m-th target group. For the wth i Each aircraft platform uses n i type i Type m aircraft strikes the m-th group of targets.

[0039] Since the input and output of the model are both data of variable length, a deep neural network model is designed using Transformer. Encoder and Decoder modules based on Transformer are constructed. The Encoder module encodes the variable-length state space into a fixed-size embedded feature map, and the Decoder module decodes the embedded feature map into an action space of a specified size. Combined with the Actor-Critic architecture, a deep neural network model is constructed.

[0040] To more accurately reflect the pros and cons of the fire distribution result, the reward function is designed in combination with the aiming point optimization and damage effect evaluation. For a group of fire distribution schemes The aiming point optimization and damage effect evaluation are performed for each group of targets, and the group of targets t m The damage probability P of each target t in the group t The reward function R is designed as follows:

[0041] R = λ1R1 + λ2R2

[0042] R1 = sum(P t ≥ P t * )

[0043]

[0044] Wherein, λ1 and λ2 are reward weights; N T is the total number of all targets; N W is the total number of all aircraft in the fire distribution result; R1 is a damage target satisfaction reward function, which aims to consider whether the fire distribution result can meet the expected damage probability of each target; R2 is a fire distribution result cost-effectiveness reward function, which aims to consider the cost-effectiveness (i.e. the ratio of the income and the cost) of the fire distribution result. By reasonably adjusting the weights of the reward functions, the deep reinforcement learning model can be guided to make the optimal fire distribution decision.

[0045] Step four, training and use of the deep reinforcement learning model.

[0046] After the deep reinforcement learning model is constructed, the deep reinforcement learning model is iterated and updated through continuous offline training and exercise data feedback, so as to ensure the rationality and reliability of the fire distribution result.

[0047] The embodiment of the application further provides an electronic device, Figure 2The structure of the electronic device provided by the embodiments of the present application is shown. For example, the electronic device 20 can include a processor 21, a memory 22, and a transmission device 23. The processor is configured to execute the fire distribution method including target grouping and reinforcement learning mentioned in the above embodiments. The processor and the memory can be connected by a bus or other means. The transmission device can be connected with the processor and the memory by wired or wireless means. The memory, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions / modules corresponding to the fire distribution method including target grouping and reinforcement learning in the embodiments of the present application. The processor executes various functions and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in the memory, that is, implements the fire distribution method including target grouping and reinforcement learning in the above method embodiments. The memory can include a program storage area and a data storage area. The program storage area can store an operating system and application programs required by at least one function; and the data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory can optionally include a memory remotely arranged with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The one or more modules are stored in the memory and executed by the processor to execute the fire distribution method including target grouping and reinforcement learning in the embodiments.

[0048] As another aspect, the present application also provides a computer readable storage medium, which can be the computer readable storage medium included in the device described in the above embodiments; or can exist independently and not be assembled into the device. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field. The computer readable storage medium stores one or more programs used by one or more processors to execute the fire distribution method including target grouping and reinforcement learning described in the present application.

[0049] In conclusion, the above is only a preferred embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A fire distribution method comprising target grouping and reinforcement learning, characterized by, A target grouping strategy based on damage range is established, and the collateral damage of the aircraft to the target is considered; a target damage effect evaluation based on the optimization of aiming points is carried out in the target group; finally, a deep reinforcement learning model of fire distribution is established, taking the position of each aircraft platform, the type of carried aircraft, the number of carried aircraft, the grouping situation of the target, the type and position of each target group as the input of the model, and the fire distribution result as the output of the model, so as to maximize the damage cost-effectiveness ratio while ensuring the expected damage effect on the target; Based on the model input, construct the state space S = {W, T} of the deep reinforcement learning, where W = {w1,..., w j ,…,w m} is the state information of all available m aerial vehicle platforms, is the state information of aerial vehicle platform j, lon w , lat w , and H w are the longitude, latitude, and altitude of the aerial vehicle platform, respectively, type w i and n w i are the type and number of the i-th aerial vehicle in the aerial vehicle platform, respectively, is the cost of the i-th aerial vehicle; T = {t1,..., t j ,…,t q} is the state information of all q groups of targets, is the state information of the j-th group of targets, where lon t i , lat t i , and H are the longitude, latitude, and altitude of target i in the group, respectively, type t i is the type of target i in the group, V i t is the value of target i, P i * is the expected damage probability of target i; Based on the output of the model, construct the action space A = [a1,..., a m ,...,a q ] of the deep reinforcement learning, where is the fire distribution result for the mth group of targets, is the w i th aircraft platform using n i type i aircraft to attack the mth group of targets; The reward function is designed in combination with the aim point optimization and damage assessment for a set of fire distribution schemes The aim point optimization and damage assessment are performed for each set of targets, and the set of targets is outputted as T m The damage probability P of each target t in T t The reward function R is designed as follows: R = λ1R1 + λ2R2 R1 = sum(P t ≥ P t * ) wherein λ1 and λ2 are reward weights; N T is the total number of all targets; N W is the total number of all aircraft in the fire distribution result; R1 is a damage target satisfaction reward function, which aims to consider whether the fire distribution result can meet the expected damage probability of each target; R2 is a fire distribution result cost-effectiveness reward function, which aims to consider the cost-effectiveness of the fire distribution result.

2. The method of claim 1, wherein, The target grouping strategy based on damage range is established: taking the maximum damage radius of the aircraft as the grouping distance threshold, using the DSCAN clustering method based on relative distance to realize the grouping of multiple targets, and forming the basic unit of fire distribution, i.e. target group.

3. The method of claim 2, wherein, By establishing an aiming point planning model and a damage effect evaluation model for each target group, a target damage effect evaluation based on aiming point optimization is carried out in the target group; the aiming point planning model refers to planning the optimal aiming point of each aircraft for the single target of the basic unit of fire distribution under the premise of specified aircraft launch platform, aircraft type and quantity, target type and quantity, so as to achieve maximum damage to the target; the damage effect evaluation model refers to calculating the aircraft-target intersection condition under the specified aiming point, evaluating the target damage situation, and taking it as the judgment basis for the advantages and disadvantages of the target distribution result.

4. The method of claim 3, wherein The aiming point planning method uses heuristic algorithm or integer programming algorithm for solution, and the damage effect evaluation is realized by looking up the damage database table in advance, using empirical formula, proxy model or precise finite element calculation.

5. The method of claim 1, wherein, A deep neural network model is designed using Transformer, and an Encoder and Decoder module based on Transformer is constructed, the Encoder module encodes the state space of indefinite length into a fixed size embedding feature map, and the Decoder module decodes the embedding feature map into a specified size action space, combined with the Actor-Critic architecture, a deep neural network model is constructed.

6. The method of claim 5, wherein, After the deep reinforcement learning model is constructed, the deep reinforcement learning model is iterated and updated through continuous offline training and exercise data feedback.

7. An electronic device, comprising: The electronic device includes a processor, a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to realize the fire distribution method comprising target grouping and reinforcement learning of any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the fire distribution method comprising target grouping and reinforcement learning of any one of claims 1-6.

Images