A dynamic target assignment method based on semi-Markov processes and hybrid graph pointer networks

By employing a dynamic target allocation method based on semi-Markov processes and hybrid graph pointer networks, the problems of high-dimensional situational understanding and complex temporal reasoning in air-to-air combat are solved. This method achieves efficient target allocation under electronic jamming conditions, thereby improving the mission success rate and resource utilization efficiency in air-to-air combat.

CN123113334APending Publication Date: 2026-11-17BEIHANG UNIV +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611275935.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-21
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing air-to-air combat decision-making algorithms struggle to effectively address the requirements of high-dimensional situational understanding, complex temporal reasoning, and real-time solution when facing electronic interference, becoming a technical bottleneck restricting the development of intelligent air-to-air combat.

Method used

A dynamic target allocation method based on semi-Markov process and hybrid graph pointer network is adopted. The situation is represented by graph neural network, and a robust reward function is designed by combining pointer network and reinforcement learning. The allocation strategy is optimized by using proximal policy optimization algorithm.

Benefits of technology

In an environment of fluctuating information, a long-term strategic allocation strategy was implemented, which improved the model's ability to understand the topological relationships of complex scenarios, handled the combination allocation of variable numbers of entity units, reduced computation time, and improved the success rate of missions and the efficiency of resource utilization in air-to-air combat.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123113334A_ABST
    Figure CN123113334A_ABST
Patent Text Reader

Abstract

This invention discloses a dynamic target allocation method based on a semi-Markov process and a hybrid graph pointer network, belonging to the field of weapon target allocation technology. It includes the following steps: S1, modeling the simulation environment as an event-driven semi-Markov decision process; S2, constructing a graph neural network situation encoder containing disturbance operators to achieve efficient representation of high-dimensional situational awareness on the field; S3, using a pointer network as a decoder, transforming the combinatorial optimization problem of target allocation into a sequential decision process through an attention mechanism; S4, designing a reward function to address the reputation allocation issues caused by sparse rewards and disturbance misdirection in adversarial decision-making; S5, constructing a proximal policy optimization algorithm for end-to-end training; and S6, testing and evaluating the algorithm's performance under various adversarial scenarios. This invention effectively solves the problems of insufficient battlefield situational awareness representation, dimensional explosion in the combined action space, and difficulty in long-term reputation allocation in existing methods.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of weapon target allocation technology, specifically relating to a dynamic target allocation method based on a semi-Markov process and a hybrid graph pointer network. Background Technology

[0002] In modern air-to-air combat, strike effectiveness depends not only on the performance of a single platform, but also on the efficiency and quality of collaboration among multiple platforms within a formation. In high-speed, highly contested dynamic environments, solving the Dynamic Weapon Target Assignment (DWTA) problem is one of the key aspects of achieving intelligent command and control.

[0003] The DWTA problem requires comprehensive consideration of the multi-dimensional states of both adversaries within an extremely short decision window, including the number of units, motion parameters, value, and resource constraints such as weapon inventory and strike probability. The fundamental goal of formulating coordinated strike strategies for multiple weapon platforms is to maximize the comprehensive strike effectiveness of formation combat under complex constraints. However, existing decision-making algorithms struggle to adequately address the requirements of high-dimensional situational understanding, complex temporal reasoning, and real-time solution when facing electronic jamming, becoming one of the technical bottlenecks restricting the development of intelligent air-to-air combat. This invention aims to address the problems of insufficient battlefield situational representation, exploding spatial dimensions of combined actions, and difficulties in long-term reputation allocation in existing methods by proposing an adaptive dynamic allocation strategy based on a semi-Markov decision process, integrating graph neural networks, pointer networks, and reinforcement learning methods. Summary of the Invention

[0004] This invention aims to overcome the shortcomings of the prior art and provides a dynamic target allocation method based on semi-Markov processes and hybrid graph pointer networks.

[0005] A dynamic target assignment method based on a semi-Markov process and a hybrid graph pointer network includes the following steps: S1, based on a semi-Markov decision process, models an air-to-air combat simulation environment; S2, based on graph neural network modeling situational representation, injects random perturbation terms that increase with the intensity of interference into node features to simulate a non-ideal information environment; S3, based on pointer networks, establishes a target allocation action space to realize target allocation scheme modeling based on the sequential decision of target-to-weapon direction; S4, Design a reward function that comprehensively considers interference robustness to solve the reputation allocation under sparse rewards in air-to-air adversarial decision-making; S5, construct a near-end policy optimization algorithm for end-to-end training, optimize and iterate the target allocation scheme decision strategy; S6 tests and evaluates the algorithm in various adversarial scenarios to verify its performance.

[0006] Furthermore, in step S1: The air-to-air confrontation process is modeled as an event-driven semi-Markov decision process. Under this framework, the evolution of the confrontation process is decoupled into two time scales: a continuous physical simulation scale and a discrete strategy decision scale. In a confrontation, since there are multiple rounds of target allocation scheme decisions and weapon launch and strike processes, the agent only triggers decisions at key event points to determine the weapon target allocation scheme, and then conducts scenario environment simulation until it transitions to the next decision state.

[0007] Furthermore, in step S2: Using a graph structure To represent the instantaneous situation at time t, each node in the graph Representing a surviving field entity, including platforms, weapons, and targets, node features encode its entity type, kinematics, and value attributes, and perturbation functions are superimposed on the continuous state variables of position and velocity based on the field's disturbance environment; the edges of the graph Directed edges, constructed as fully connected elements, represent the relationships between different entities in the field. Edge features explicitly describe the geometric features of the relative distance, relative velocity, and relative angle between node pairs.

[0008] Furthermore, in step S3: The action space based on pointer networks will include actions Defined as a target-to-weapon mapping dictionary At decision moment t, the intelligent agent assigns specific weapons to each surviving target based on the current situation on the field.

[0009] Furthermore, in step S4: The reward function comprehensively considers the characteristics of immediate feedback and delayed rewards on the field. At the decision-making moment, it introduces a launch cost penalty and a shaping reward based on a dynamic hit probability model to guide the agent to carry out low-cost strikes under advantageous geometric conditions. During the adversarial scenario simulation, the reward function accumulates the value reward of striking the target and applies a continuous time step penalty to incentivize a quick end to the adversarial scenario. During the adversarial scenario simulation, if our side still achieves effective destruction under strong interference conditions, an additional robust decision reward is given. If there are still targets alive and weapons are exhausted at the end of the adversarial scenario, a final penalty based on the value of the remaining targets will be applied, thereby strengthening the concept of global victory or defeat.

[0010] Furthermore, in step S5: The Proximal Policy Optimization (PPO) algorithm is used to optimize policy and value. Based on the characteristics of the semi-Markov decision process framework, the advantage function calculation formula in the PPO algorithm is designed. The overall loss function includes policy loss, value loss, and entropy regularization.

[0011] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. In this invention, the reward function is designed under the SMDP framework by taking advantage of the characteristics of instantaneous launch and delayed strike. Compared with the standard Markov process (MDP) that makes decisions at fixed time steps, it better solves the problems of "buffer overflow" and "reputation allocation failure" caused by time scale mismatch. This enables the model to learn allocation strategies with a long-term strategic vision in the context of fluctuating information, thus achieving framework innovation.

[0012] 2. The combination of GNN and Ptr-Nets significantly improves the model's ability to understand the topological relationships of complex scenarios and its generalization performance in interference scenarios. It can also better handle the combination and allocation of variable numbers of entity units. It has achieved good performance in tests of different adversarial scenarios, realizing network framework innovation.

[0013] 3. This invention is based on reinforcement learning to solve the dynamic weapon target allocation problem. Compared with traditional optimization algorithms such as dynamic programming, genetic algorithm, and particle swarm optimization algorithm, which are based on "static" optimization based on "snapshots" of the field, it can better handle the strong temporal correlation problem in highly dynamic air combat. At the same time, by using a trained model, the computation time is greatly reduced. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 This is a schematic diagram of the framework of the present invention; Figure 2 This is a schematic diagram of the adversarial simulation environment based on the semi-Markov decision process of this invention. Figure 3 This is a schematic diagram of an air-to-air combat scenario according to Embodiment 1 of the present invention; Figure 4 This is a graph showing the change in the reward function during the training process of Embodiment 1 of the present invention; Figure 5 This is a statistical chart of the reasons for battlefield termination in the training results of Embodiment 1 of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0016] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0017] It should be noted that the labels and letters in the following figures represent similar items, therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0018] A dynamic target assignment method based on a semi-Markov process and a hybrid graph pointer network includes the following steps: Step 1: Model the air combat process as an event-driven semi-Markov decision process. Within this framework, the evolution of the adversarial process is decoupled into two time scales: a continuous physical simulation scale and a discrete strategy decision scale. In a single engagement, there may be multiple rounds of target allocation scheme decisions and weapon launches. The agent only triggers decisions at key event points to determine the weapon target allocation scheme, followed by pure scenario environment simulation until the transition to the next decision state.

[0019] Step 2: Use a graph structure To represent the instantaneous situation at time t, each node in the graph Representing a surviving field entity, such as a platform, weapon, or target, node features encode its entity type, kinematics, value, and other attributes, and explicitly include observations affected by perturbations; the edges of the graph Directed edges, constructed as fully connected components, represent the relationships between different entities in the field. Edge features explicitly describe the geometric situational characteristics of node pairs, such as relative distance, relative velocity, and relative angle. Situational encoding is implemented based on a graph neural network (GNN). The target information acquired by the system is superimposed with perturbations according to the real-time interference intensity. The GNN learns how to extract true situational features from noise through multi-layer aggregation.

[0020] Step 3: Based on pointer networks (Ptr-Nets), represent the actions of the target allocation scheme. Defined as a target-to-weapon mapping dictionary At decision moment t, the intelligent agent assigns specific weapons to each surviving target based on the current situation on the field.

[0021] Step 4: Design a composite reward function that comprehensively considers the characteristics of immediate feedback and delayed rewards. At the decision-making moment, a launch cost penalty and a shaping reward based on a dynamic hit probability model are introduced to guide the agent to conduct low-cost strikes under advantageous geometric conditions. During the adversarial scenario simulation, the reward function accumulates the value reward of striking the target and applies a continuous time-step penalty to incentivize a rapid end to the adversarial scenario. During the adversarial scenario simulation, if our side still achieves effective destruction under strong interference conditions, an additional robust decision-making reward is given to reflect the guidance of the success rate of adversarial interference; if there are still targets surviving and weapons are exhausted at the end of the adversarial scenario, a final penalty based on the value of the remaining targets will be applied to strengthen the concept of global victory or defeat.

[0022] Step 5: Use the Proximal Policy Optimization (PPO) algorithm to optimize the policy and value. The advantage function calculation formula in the PPO algorithm needs to be designed according to the characteristics of the SMDP framework. The overall loss function includes three parts: policy loss, value loss and entropy regularization.

[0023] Step 6: Test and evaluate the algorithm's performance in various adversarial scenarios of different scales.

[0024] Example 1 This invention provides a dynamic target allocation method based on a semi-Markov process and a hybrid graph pointer network, comprising the following steps: Step 1: Model the air combat process as an event-driven semi-Markov decision process. Within this framework, the evolution of the adversarial process is decoupled into two time scales: a continuous physical simulation scale and a discrete strategy decision scale. In a single engagement, there may be multiple rounds of target allocation scheme decisions and weapon launches. The agent only triggers decisions at key event points, determining the weapon target allocation scheme, followed by pure scenario environment simulation until transitioning to the next decision state. The air combat simulation process is as follows: Figure 2 As shown.

[0025] Step 2: Use a graph structure To represent the instantaneous situation at time t, each node in the graph Representing a surviving field entity, such as a platform, weapon, or target, node features encode its entity type, kinematics, value, and other attributes, and explicitly include observations affected by perturbations; the edges of the graph Directed edges, constructed as fully connected components, represent the relationships between different entities in the field. Edge features explicitly describe the geometric situational characteristics of node pairs, such as relative distance, relative velocity, and relative angle. Situational encoding is implemented based on a graph neural network (GNN). The target information acquired by the system is superimposed with perturbations according to the real-time interference intensity. The GNN learns how to extract true situational features from noise through multi-layer aggregation.

[0026] Step 3: Based on pointer networks (Ptr-Nets), represent the actions of the target allocation scheme. Defined as a target-to-weapon mapping dictionary At decision time t, the intelligent agent assigns specific weapons to each surviving target based on the current situation on the field, thus deciding on the allocation plan.

[0027] Step 4: Design a composite reward function that comprehensively considers the characteristics of immediate feedback and delayed rewards. At the decision-making moment, a launch cost penalty and a shaping reward based on a dynamic hit probability model are introduced to guide the agent to conduct low-cost strikes under advantageous geometric conditions. During the adversarial scenario simulation, the reward function accumulates the value reward of striking the target and applies a continuous time-step penalty to incentivize a rapid end to the adversarial scenario. During the adversarial scenario simulation, if our side still achieves effective destruction under strong interference conditions, an additional robust decision-making reward is given to reflect the guidance of the success rate of adversarial interference; if there are still targets surviving and weapons are exhausted at the end of the adversarial scenario, a final penalty based on the value of the remaining targets will be applied to strengthen the concept of global victory or defeat.

[0028] Step 5: Use the Proximal Policy Optimization (PPO) algorithm to optimize the policy and value. The advantage function calculation formula in the PPO algorithm needs to be designed according to the characteristics of the SMDP framework. The overall loss function includes three parts: policy loss, value loss and entropy regularization.

[0029] Step 6: Test and evaluate the algorithm's performance in various adversarial scenarios of different scales.

[0030] After the above steps Figure 3 This demonstrates an air-to-air combat scenario involving weapon target allocation. The scenario is set up with three friendly platforms, each carrying two different types of weapons, for a total of six; facing four enemy targets, including two high-value targets. The enemy targets do not attack, but only evade attacks. In each round of weapon target allocation decision-making, each target can be engaged by a maximum of one weapon at a time. When the number of weapons exceeds the number of targets, the mission in the air-to-air combat is successfully defined as destroying all enemy targets before weapons are exhausted.

[0031] Approximately 9636 training and iteration sessions were conducted in air-to-air combat scenarios. Figure 4The results show that the reward curve steadily rises until it reaches a convergent state. Figure 5 The results show that the proposed objective assignment algorithm can achieve a task success rate of 71.2% in the 3v4 scenario during the final convergence phase, which verifies the model's learning ability under complex constraints.

[0032] In the testing phase, test scenarios of different operational scales are designed as shown in Table 1, and the operational effectiveness evaluation indicators are defined as follows: 1. Value Destruction Rate: This assesses the ratio of the actual value of enemy targets destroyed to the total value of enemy targets that could be destroyed in a confrontation.

[0033] 2. High-value destruction rate: Assess the percentage of high-value targets destroyed by the opposing side in a confrontation.

[0034] 3. Weapon efficiency: Evaluate the average number of targets that each weapon can destroy in a combat scenario.

[0035] 4. Average decision time: The computation time for evaluating a single target allocation scheme.

[0036] The experimental results are shown in Table 2. It can be seen that the method proposed in this invention maintains a high value destruction rate and high value destruction efficiency in adversarial scenarios of different scales. Especially in scenarios where weapon resources are scarce, it can still learn a good target allocation scheme decision strategy and utilize limited weapon resources to strike enemy targets. This indicates that the proposed method can effectively manage limited weapon resources. At the same time, the algorithm also consumes relatively little computation time in decision-making, remaining on the order of milliseconds, demonstrating strong potential for engineering applications.

[0037] Table 1 Test Scenario Definition Table 2 Results of the Operational Scale Comparison Experiment Therefore, it can be concluded that the present invention can effectively solve the problem of dynamic weapon target allocation in air-to-air combat.

[0038] The above description constitutes an embodiment of the present invention. The foregoing descriptions are preferred embodiments of the present invention. Unless there is a clear contradiction or a prerequisite for a particular preferred embodiment, the preferred embodiments can be arbitrarily combined and used. The embodiments and specific parameters described are merely for clearly illustrating the verification process of the invention and are not intended to limit the scope of patent protection of the present invention. The scope of patent protection of the present invention is still determined by its claims. Similarly, any equivalent structural changes made based on the description and drawings of the present invention should also be included within the scope of protection of the present invention.

Claims

1. A dynamic target allocation method based on semi-Markov processes and hybrid graph pointer networks, characterized in that, Includes the following steps: S1, based on a semi-Markov decision process, models an air-to-air combat simulation environment; S2, based on graph neural network modeling situational representation, injects random perturbation terms that increase with the intensity of interference into node features to simulate a non-ideal information environment; S3, based on pointer networks, establishes a target allocation action space to realize target allocation scheme modeling based on the sequential decision of target-to-weapon direction; S4, Design a reward function that comprehensively considers interference robustness to solve the reputation allocation under sparse rewards in air-to-air adversarial decision-making; S5, construct a near-end policy optimization algorithm for end-to-end training, optimize and iterate the target allocation scheme decision strategy; S6 tests and evaluates the algorithm in various adversarial scenarios to verify its performance.

2. The dynamic target allocation method based on a semi-Markov process and a hybrid graph pointer network according to claim 1, characterized in that, In step S1: The air-to-air confrontation process is modeled as an event-driven semi-Markov decision process. Under this framework, the evolution of the confrontation process is decoupled into two time scales: a continuous physical simulation scale and a discrete strategy decision scale. In a confrontation, since there are multiple rounds of target allocation scheme decisions and weapon launch and strike processes, the agent only triggers decisions at key event points to determine the weapon target allocation scheme, and then conducts scenario environment simulation until it transitions to the next decision state.

3. The dynamic target allocation method based on a semi-Markov process and a hybrid graph pointer network as described in claim 1, characterized in that, In step S2: Using a graph structure To represent the instantaneous situation at time t, each node in the graph Representing a surviving field entity, including platforms, weapons, and targets, node features encode its entity type, kinematics, and value attributes, and perturbation functions are superimposed on the continuous state variables of position and velocity based on the field's disturbance environment; the edges of the graph Directed edges, constructed as fully connected elements, represent the relationships between different entities in the field. Edge features explicitly describe the geometric features of the relative distance, relative velocity, and relative angle between node pairs.

4. The dynamic target allocation method based on a semi-Markov process and a hybrid graph pointer network according to claim 1, characterized in that, In step S3: The action space based on pointer networks will include actions Defined as a target-to-weapon mapping dictionary At decision moment t, the intelligent agent assigns specific weapons to each surviving target based on the current situation on the field.

5. A dynamic target allocation method based on a semi-Markov process and a hybrid graph pointer network as described in claim 1, characterized in that, In step S4: The reward function comprehensively considers the characteristics of immediate feedback and delayed rewards on the field. At the decision-making moment, it introduces a launch cost penalty and a shaping reward based on a dynamic hit probability model to guide the agent to carry out low-cost strikes under advantageous geometric conditions. During the adversarial scenario simulation, the reward function accumulates the value reward of striking the target and applies a continuous time step penalty to incentivize a quick end to the adversarial scenario. During the adversarial scenario simulation, if our side still achieves effective destruction under strong interference conditions, an additional robust decision reward is given. If there are still targets alive and weapons are exhausted at the end of the adversarial scenario, a final penalty based on the value of the remaining targets will be applied, thereby strengthening the concept of global victory or defeat.

6. The dynamic target allocation method based on a semi-Markov process and a hybrid graph pointer network according to claim 1, characterized in that, In step S5: The Proximal Policy Optimization (PPO) algorithm is used to optimize policy and value. Based on the characteristics of the semi-Markov decision process framework, the advantage function calculation formula in the PPO algorithm is designed. The overall loss function includes policy loss, value loss, and entropy regularization.