A target threat assessment method based on a graph neural network and deep reinforcement learning

Abstract

The application discloses a target threat evaluation method based on a graph neural network and deep reinforcement learning, belongs to the technical field of individual intelligent decision-making, and is applied to a multi-target autonomous confrontation combat scene of individual teams. The application realizes the method as follows: 1. threat element extraction and preprocessing; 2. two-dimensional soft rule evaluation of action threat and firepower threat; 3. intuition fuzzy feature quantization; 4. multi-target relationship modeling of a graph attention network by fusing intuition fuzzy constraints; 5. intelligent decision-making based on rule guidance and proximal policy optimization; 6. real-time output of a priority disposal target by an intelligent agent based on a current situation vector, so that end-to-end autonomous threat evaluation and decision-making are realized. Compared with the prior art, the application solves the technical problem of accurately evaluating the threat degree of multiple targets in a complex battlefield environment.

Description

Technical Field

[0001] This invention relates to a target threat assessment method based on graph neural networks and deep reinforcement learning, belonging to the field of individual soldier intelligent decision-making technology, and applied to multi-target autonomous combat scenarios in individual soldier squads. Background Technology

[0002] In multi-target battlefield environments, accurately assessing the threat level of each target and prioritizing engagement of high-threat targets is crucial for enhancing combat effectiveness. Target threat assessment requires comprehensive information across multiple dimensions, including operational status, weapon type, distance, speed, spatial location, and detection confidence.

[0003] In existing technologies, traditional threat assessment methods mainly have the following shortcomings:

[0004] (1) Although the rule-based weighted method is highly interpretable and has good real-time performance, it is limited by artificial rules and fixed weights, making it difficult to model the complex interaction relationship between multiple targets, and lacks explicit processing of detection uncertainty and dynamic understanding of battlefield situation.

[0005] (2) When the target is close, the midpoint of the bottom of the detection box cannot accurately correspond to the actual foot position, resulting in a significant deviation in distance estimation. When the target is far away, the small perturbation of the pixel coordinates will cause a large distance measurement error after being amplified by the projection model.

[0006] (3) Single-frame action recognition results often have jitter and brief misjudgments. If the original label is used directly, the threat score will jump drastically between adjacent frames, affecting the stability of decision-making.

[0007] (4) Existing intelligent evaluation methods rely on a large amount of domain knowledge and large-scale high-quality labeled data, and have weak interpretability; reinforcement learning methods have high simulation environment construction costs, slow training convergence speed, and are difficult to directly transfer to real scenarios.

[0008] Therefore, there is an urgent need for an intelligent threat assessment method that can accurately assess the threat level of multiple targets in complex battlefield environments, while possessing high interpretability, strong adaptability, and real-time performance. Summary of the Invention

[0009] The purpose of this invention is to address the technical problem of accurately assessing the threat level of multiple targets in complex battlefield environments by proposing a target threat assessment method based on graph neural networks and deep reinforcement learning. This method can comprehensively assess the threat level of targets by integrating multi-dimensional information, adaptively learn the interaction relationships between multiple targets, and ensure both real-time performance and interpretability while maintaining decision-making accuracy.

[0010] The objective of this invention is achieved through the following technical solution:

[0011] This invention discloses a target threat assessment method based on graph neural networks and deep reinforcement learning, comprising the following steps:

[0012] Step 1: Threat Element Extraction and Preprocessing; Obtain multi-dimensional basic information for each target from the output of the multi-target detection and tracking model, including action state, weapon type, movement speed, spatial location, and detection confidence; perform parallel preprocessing on the multi-dimensional basic information, including temporal action information correction, target distance estimation, and occlusion ratio estimation; Specifically, perform dynamic threshold-driven temporal action information correction on the action state; calculate the target distance using a perspective projection-based distance estimation method, and correct the ranging result using an error compensation model; estimate the target occlusion ratio based on tracking matching cost or target loss state; the temporal action information correction is used to reduce jitter in multi-frame images; the target distance estimation is used to estimate target distance information from two-dimensional images; the occlusion ratio estimation is used to estimate the degree of target occlusion.

[0013] The temporal action information correction is as follows: for each tracked target, continuously record the action labels output by the detection network. and the corresponding detection confidence level Forming a time series ,in This indicates the original test results. To correspond to the confidence level; the judgment threshold is dynamically adjusted based on the historical statistics of the confidence level, and a high threshold is calculated. and low threshold Perform state consistency judgment and use an asymmetric acknowledgment frame threshold for state verification;

[0014] In step 1, the high threshold and low threshold Represented as:

[0015] ;

[0016] ;

[0017] in, and These are the mean and standard deviation of the confidence sequence of the most recent L frames, respectively. and These are the 75th percentile and the 25th percentile, respectively. These are the weighting coefficients. It is a multiple of the standard deviation;

[0018] In step 1, the consistency judgment of the execution state is as follows: if the detection result of the current frame is inconsistent with the state of the previous frame, then only if the confidence level is higher than the previous frame's state will the execution state be consistent. The results of this test will be adopted only if the results are consistent; if they are consistent, then as long as the confidence level is higher than [a certain value], the results will be adopted. That is, maintain the original state; otherwise, perform a state flip.

[0019] In step 1, the state verification using an asymmetric confirmation frame threshold means that the transition from a non-firing state to a firing state must be continuous. A frame is detected as firing, and the transition from firing state to non-firing state requires consecutive frames. The frame was detected as non-shooting, in which ,in , ;

[0020] The target distance estimation method specifically involves: obtaining the camera intrinsic parameter matrix K using the Zhang Zhengyou calibration method; selecting the bottom center point of the target detection box as the foot point based on the pinhole camera imaging model; establishing the perspective projection mapping relationship between the pixel coordinates of the imaging plane and the world coordinate system; and deriving the world coordinates of the foot point based on the pinhole camera model. Find the closed-form solution and calculate the horizontal distance between the target and the camera optical center. When the target's foot point is close to the lower edge of the image, the target's pixel width is introduced as an auxiliary ranging criterion to enhance the robustness of near-range estimation; real distance data is collected using a laser rangefinder to construct a quadratic polynomial error compensation model. Post-compensation is performed on the initially estimated distance; among which, , , These are the error compensation coefficients for the fitted estimate;

[0021] In step 1, the world coordinates of the foot point The closed-form solution is:

[0022]

[0023] in, The angle between the camera's optical axis and the horizontal plane; The horizontal coordinates of the target foot point on the image; Vertical coordinates of the target foot point on the image; equivalent focal length , and principal point coordinates , ; The altitude of the camera in the world coordinate system; The coordinates of the target footpoint in the world coordinate system;

[0024] The occlusion ratio is estimated as follows: when the target and the detection result match normally, the occlusion ratio is... Where C is the matching cost; when the target is marked as temporarily lost, ;

[0025] Step 2: Two-dimensional soft rule assessment of action threat and fire threat; construct two assessment dimensions, action threat and fire threat, and calculate the action threat value and fire threat value for each target separately; introduce a confidence weighting factor. Motion cost weighting factor By combining an adaptive weight adjustment mechanism based on population statistical characteristics, a preliminary threat score for each target is generated. ;

[0026] The confidence weighting factor is: Used to reflect the reliability of perceived information; among which, The confidence level output for target detection;

[0027] The motion cost weighting factor The distance from the image center is calculated by the Euclidean distance between the target pixel coordinates and the image center, which reflects the efficiency of aiming at the target.

[0028] The adaptive weight adjustment mechanism based on group statistical characteristics dynamically adjusts the action threat weight according to the group statistical characteristics. and firepower threat weight ;

[0029] The initial threat score Represented as: ;in, Scoring for action threat; Scoring for firepower threat;

[0030] The threat value is calculated as follows: when the target is firing, the movement speed is negatively correlated with the threat level, with lower speed or a stationary state indicating a higher threat level; when the target is not firing, the movement speed is positively correlated with the threat level, with higher speed suggesting that the target is rapidly approaching.

[0031] The fire threat value is calculated as follows: a base threat value is set for each type of weapon, with a gun having the highest base weight, a rocket launcher having the second highest base weight, and no weapon having a base weight of zero; a distance threat factor is calculated by combining the target distance and the effective range, and the fire threat of targets outside the effective range is zero; within the effective range, the closer the target is, the higher the threat level.

[0032] The adaptive weight adjustment mechanism based on group statistical characteristics is as follows: when the proportion of shooting targets in the scene is high, the fusion weight of the action threat dimension is appropriately increased; when the proportion of armed targets is high, the fusion weight of the firepower threat dimension is increased, and the adjustment is made in combination with the average target distance.

[0033] Step 3: Intuitive Fuzzy Feature Quantization; Using intuitionistic fuzzy set theory, weapon type, action state, detection confidence, and occlusion ratio information are quantified into intuitionistic fuzzy membership degrees. and hesitation , as the feature embedding of the target node;

[0034] In step 3, intuitive fuzzy membership degree and hesitation The calculation is as follows:

[0035] ;

[0036] ;

[0037] in, To test the confidence level, This is the weapon's base threat value. For behavioral confidence, Set the value to 1 if the target is in an attack state, otherwise set it to 0. The target occlusion ratio obtained in step 1; ensured by truncation. and ;

[0038] Step 4: Modeling Multi-Target Relationships in a Graph Attention Network by Integrating Intuitive Fuzzy Constraints. The target set at each time step is explicitly modeled as a fully connected graph structure, with each target node containing a threat feature vector and the intuitive fuzzy feature embedding. A graph attention network is used to aggregate node features. The intuitive fuzzy membership degree and hesitation degree are introduced as attention constraints during the multi-head attention coefficient calculation process. This adaptively learns the mutual influence between targets and extracts the interaction relationship features between them. Average pooling is performed on the output embeddings of all nodes to obtain graph-level embeddings. These are then concatenated with 3D global statistical features and projected to generate a situation vector that integrates local interaction information and macro-battlefield statistical features. The 3D global statistical features include the proportion of shooting targets, the proportion of weapon-wielding targets, and the average target distance.

[0039] The graph attention network employs a two-layer GAT convolutional structure. Specifically: in feature mapping, the original target node features are mapped to a 64-dimensional latent space through a multilayer perceptron to obtain an initial node feature matrix; in fully connected graph construction, each target is treated as an independent graph node, and all nodes are connected pairwise to form a fully connected graph structure; in the first GAT convolutional layer, a multi-head attention mechanism is used to compute node... To his neighbors Attention coefficient The neighbor information is weighted and aggregated accordingly, and the updated node representation is output after passing through the ReLU activation function. On the second layer of GAT convolution, the node features output by the first layer are further aggregated to output the final 64-dimensional node embedding vector. In the generation of the situation vector, all node embeddings are averaged to obtain graph-level embeddings, which are then concatenated with the three-dimensional global statistical features and projected onto the 128-dimensional situation vector.

[0040] Step 5: Intelligent decision-making based on rule guidance and proximal policy optimization; using the situation vector as state input, the decision result is output using the proximal policy optimization method; the action space includes observation / waiting and prioritizing the handling of a certain target; a two-stage training mechanism is adopted in the training phase: the first stage is behavior cloning pre-training, using preliminary threat scoring. The first stage involves prioritizing targets based on their current threat level, using the top-ranked target as the action label to train the policy network, incorporating the decision-making biases of soft rule-based methods. The second stage is reinforcement training for proximal policy optimization, autonomously optimizing long-term decision benefits in a dynamic interactive environment. The reward function integrates disposition benefits, time costs, and decision penalties, and the disposition benefits are correlated with the initial threat score. They are positively correlated;

[0041] The behavior cloning pre-training involves collecting a state-action pair dataset in a dynamic simulation environment using an adaptive rule-weighted threat assessment method. Each state corresponds to the optimal action as determined by the rule-based method; and the policy network of the near-end policy optimization algorithm is pre-trained using the cross-entropy loss function; after pre-training, the normal training process of near-end policy optimization is initiated.

[0042] In step 5, the reward function Represented as:

[0043] ;

[0044] in, Successful handling of the target grants a positive reward based on the target's weapon type: +1 for no weapon, +3 for rocket, and +5 for gun; an additional +1.5 if the target is in an attack state; and an additional +1 if the target is also a high-threat target according to the rules. As a time penalty, a penalty of -0.5 is applied for each step, which accumulates with the number of steps. Penalties for failure to execute a move: -0.25 penalty for failure to execute a move; -3 penalty for selecting an invalid move; -0.5 penalty for each remaining target attacked.

[0045] In step 5, the proximal policy optimization method adopts an Actor-Critic architecture, where the Actor network and the Critic network share the global situation representation extracted by the graph attention module; the Actor network takes the situation vector as input, passes through two hidden layers, and outputs the action probability distribution through a Softmax layer; the Critic network takes the situation vector as input and outputs a single scalar value estimate; during training, the policy loss and value loss are calculated separately, and the shared and independent parameters of the two networks are jointly updated.

[0046] Step 6: The intelligent agent outputs priority targets for action in real time based on the current situation vector, realizing end-to-end autonomous threat assessment and decision-making;

[0047] The application of a target threat assessment method in a multi-target autonomous combat scenario for individual soldiers and squads includes the following steps:

[0048] Step 1: Using the method described above, extract threat elements and preprocess the real-time data output by the multi-target detection and tracking model to obtain the action state after time-series correction, the target distance after error compensation, and the occlusion ratio parameters.

[0049] Step 2: Perform two-dimensional soft rule evaluation, intuitionistic fuzzy feature quantification, graph attention relationship modeling, and proximal policy optimization decision-making using the method described above to generate a global situation vector and output the threat priority ranking results of each target;

[0050] Step 3: Apply the decision results of the priority targets to the actual combat scenario of multi-target autonomous confrontation between individual soldiers and squads.

[0051] Compared with existing technologies, it has the following beneficial effects:

[0052] 1. This invention uses graph attention networks to explicitly model the collaborative and interactive relationships among multiple targets, overcoming the limitations of traditional rule-based methods that score targets independently, and improving the accuracy of threat assessment.

[0053] 2. This invention utilizes intuitionistic fuzzy set theory to unify the modeling of uncertainties in detection and tracking, enabling the agent to adaptively suppress high uncertainty information and enhance the robustness of decision-making.

[0054] 3. This invention fully absorbs the reasonable decision-making tendency of rule-based methods through behavioral cloning pre-training, which accelerates the convergence speed while ensuring early performance, and solves the problem of low efficiency in training deep reinforcement learning from scratch.

[0055] 4. The dynamic threshold-driven temporal action information correction method of the present invention effectively suppresses random jitter and instantaneous anomalies in the detection results, and significantly improves the temporal stability of threat assessment.

[0056] 5. The distance estimation method based on perspective projection of this invention, combined with pixel width-assisted ranging and error polynomial compensation, achieves high-precision monocular ranging across the entire distance range.

[0057] 6. The adaptive weight adjustment mechanism of this invention dynamically optimizes decision parameters based on the battlefield situation, enabling threat assessment to have environmental adaptability. Attached Figure Description

[0058] Figure 1 This is an overall flowchart of the target threat assessment method of the present invention;

[0059] Figure 2 A schematic diagram of the process for a dynamic threshold-driven time-series action information correction method;

[0060] Figure 3 This is a schematic diagram of a geometric model for target distance estimation based on perspective projection;

[0061] Figure 4 A schematic diagram of a two-dimensional assessment framework for action threat and firepower threat;

[0062] Figure 5 A schematic diagram of the multi-objective relationship modeling process based on graph neural networks;

[0063] Figure 6 This is a schematic diagram of the overall architecture and learning process of an intelligent decision-making network. Detailed Implementation

[0064] To better illustrate the purpose and advantages of this invention, the invention will be further described below with reference to the accompanying drawings and examples. It should be noted that the implementation of this invention is not limited to the following embodiments, and any modifications or alterations made to this invention will fall within the scope of protection of this invention.

[0065] Example

[0066] like Figure 1 As shown in the figure, the specific implementation steps of the target threat assessment method based on graph neural networks and deep reinforcement learning in this embodiment are as follows:

[0067] Step 1: Threat Element Extraction and Preprocessing. Basic information for each target is extracted from the output of the multi-target detection and tracking model, including action state (firing / not firing), weapon type (unarmed / armed / armed rocket launcher), velocity (estimated by Kalman filter), spatial location (bounding box normalized pixel coordinates), and confidence level. Dynamic threshold-driven temporal action information correction is performed on the action state, and perspective projection-based estimation and error compensation are performed on the target distance.

[0068] Step 2: Two-Dimensional Soft Rule Threat Assessment. Calculate the action threat value and fire threat value for each target separately. Action threat is determined based on the target's action state and speed, while fire threat is determined based on weapon type and target distance. Simultaneously, group statistical characteristics (proportion of firing targets, proportion of armed targets) are calculated to adaptively adjust the fusion weights of action and fire threats.

[0069] Step 3: Intuitive Fuzzy Feature Quantization. Information such as weapon type, action state, and detection confidence for each target is quantified into intuitive fuzzy membership degree and hesitation degree, serving as supplementary features for target nodes in the subsequent graph neural network.

[0070] Step 4: Graph Neural Network Relationship Modeling. A fully connected graph is constructed, with each target as a node. Node features consist of a 14-dimensional vector, including action state (2D), weapon type (3D), behavior confidence, detection confidence, target velocity, target distance, normalized pixel coordinates (2D), occlusion ratio, intuitionistic fuzzy membership, and intuitionistic fuzzy hesitation. A two-layer multi-head graph attention network is used to encode the node features, obtaining the embedding vector for each node. Average pooling is performed on all node embeddings to obtain graph-level embeddings, which are then concatenated with 3D global statistical features (proportion of shooting targets, proportion of targets wielding weapons, and average target distance), and projected through a fully connected layer to form a situation vector.

[0071] Step 5: Proximal Policy Optimization Decision. The Actor network takes the situation vector as input, passes through two hidden layers, and outputs the action probability distribution through a Softmax layer. Action 0 represents observation and waiting, and action i (i≥1) represents prioritizing the i-th target. The Critic network takes the same situation vector as input and outputs a state value estimate. The PPO algorithm is used for policy optimization, including policy pruning loss, value function loss, and entropy regularization term.

[0072] Step 6: Decision Output. At each decision point, the agent outputs the priority target to be dealt with based on the current situation vector. The round ends if all targets have been dealt with or the maximum number of steps has been reached.

[0073] Timing action information correction:

[0074] like Figure 2 As shown, the specific implementation method of the timing action information correction method is as follows:

[0075] For each tracked target, continuously record the action labels output by the detection network. and the corresponding detection confidence level This forms a time series. A high threshold is dynamically calculated based on the target's historical confidence statistic. and low threshold :

[0076]

[0077]

[0078] in, and These are the mean and standard deviation of the confidence sequence of the most recent L frames, respectively. and These are the 75th percentile and the 25th percentile, respectively. These are the weighting coefficients. It is a multiple of the standard deviation.

[0079] The original detection results are corrected based on a dynamic threshold: if the detection result of the current frame is inconsistent with the state of the previous frame, correction is only performed when the confidence level is higher than the threshold. The results of this test will be adopted only if the results are consistent; if they are consistent, then as long as the confidence level is higher than [a certain value], the results will be adopted. That is, to maintain the original state.

[0080] An asymmetric confirmation frame rate strategy is adopted: the transition from non-firing to firing requires continuous firing. The frame is detected as firing; a transition from firing to non-firing requires a continuous sequence. Frame-agnostic firing. This asymmetric design reflects the tactical need for greater sensitivity to attack states—preferring rapid response to emerging firing threats while remaining cautious about confirmed firing states to avoid misinterpretations due to brief obstruction or detection errors.

[0081] Target distance estimation:

[0082] like Figure 3 As shown, the specific implementation method of the target distance estimation method based on perspective projection is as follows:

[0083] (1) Camera calibration: The camera intrinsic parameter matrix was obtained using Zhang Zhengyou's calibration method. Including equivalent focal length , and principal point coordinates , .

[0084] (2) Foot point projection calculation: The bottom center point of the target detection box is selected as the foot point, and its projection point on the image is... Based on the pinhole camera model, the world coordinates of the foot point. The closed-form solution is:

[0085]

[0086] Horizontal distance between the target and the optical center of the camera .

[0087] (3) Near-field auxiliary correction: When the target foot point is close to the lower edge of the image (normalized ordinate v is greater than the threshold 0.85), the pixel width of the target is introduced. As an auxiliary distance measurement basis, through The distance is estimated and fused with the foot point projection solution to enhance the robustness of the near distance estimation.

[0088] (4) Error polynomial compensation: Real distance data is collected by a laser rangefinder, and a quadratic polynomial error model is constructed. Post-compensation is applied to the initial distance estimate to improve ranging accuracy across the entire distance range.

[0089] Two-dimensional soft rule evaluation:

[0090] like Figure 4 As shown, the specific implementation method for two-dimensional soft rule evaluation is as follows:

[0091] (1) Action threat calculation: When the target is firing, the movement speed is negatively correlated with the threat level—lower speed or a stationary state indicates that the target is aiming and firing steadily, and the threat level is higher; when the target is not firing, the movement speed is positively correlated with the threat level—higher speed suggests that the target is rapidly approaching our position, and the threat level is increased. A piecewise function is used to calculate the action threat value based on the movement speed. .

[0092] (2) Firepower Threat Calculation: A base threat value is set for each type of weapon: Armed = 1.0 (highest, as firearms have effective lethality at most combat distances), Rocket = 0.7 (second highest, as rocket launchers, although causing significant single-shot damage, have a slow reload time), No Weapon = 0.0. The distance threat factor is calculated by combining the target distance d and the effective range R. Targets beyond the effective range pose no fire threat; within the effective range, the closer the distance, the greater the threat.

[0093] (3) Weighting factor: confidence weighting factor Reflects the reliability of perceived information; motion cost weighting factor The Euclidean distance between the target pixel coordinates and the image center is calculated. The closer a target is to the image center, the more attention it deserves.

[0094] (4) Adaptive weight adjustment mechanism: Dynamically adjust the action threat weight based on the statistical characteristics of the group. and firepower threat weight When the proportion of shooting targets in the scene is high, appropriately increase the weight of the action threat dimension; when the proportion of armed targets is high, increase the weight of the firepower threat dimension, and adjust it in conjunction with the average target distance.

[0095] (5) Final Threat Score: .

[0096] Graph neural network relationship modeling:

[0097] like Figure 5 As shown, the specific implementation method for graph neural network relationship modeling is as follows:

[0098] (1) Graph construction: Suppose that there are a total of at the current time. Track targets and construct a fully connected graph. ,in For the target node set, This is the set of edges connecting all pairs of nodes. Each target node... eigenvectors It consists of 14 components: 2D action state (one-hot encoding shooting / non-shooting), 3D weapon type (one-hot encoding no weapon / armed / rocket launcher), 1D behavior confidence, 1D detection confidence, 1D target velocity, 1D target distance, 2D normalized pixel coordinates, 1D occlusion ratio, 1D intuitive fuzzy membership, and 1D intuitive fuzzy hesitation.

[0099] (2) Feature encoding: The original target node features are mapped to a 64-dimensional latent space through a multilayer perceptron: The MLP consists of two fully connected layers with the ReLU activation function in between.

[0100] (3) Graph attention calculation: The first layer of GAT adopts a K=4 head attention mechanism to calculate the attention of nodes. Attention coefficients for neighbor j. The representation of node i after being updated by the k-th attention head is:

[0101]

[0102] The results of multiple heads were spliced ​​together to obtain The second layer of GAT uses single-head attention, outputting a final 64-dimensional node embedding vector. .

[0103] (4) Status vector generation: Average pooling is performed on all node embeddings to obtain graph-level embeddings. , with 3D global statistical features (proportion of shooting targets) Proportion of armed targets Average target distance The data is spliced ​​together and projected onto a 128-dimensional situation vector through a fully connected layer. .

[0104] Deep reinforcement learning decision-making:

[0105] like Figure 6As shown, the specific implementation methods of deep reinforcement learning decision-making and learning processes are as follows:

[0106] (1) State space: 128-dimensional situation vector output by the graph neural network It integrates the individual characteristics of the target, the interaction relationships between targets, and global battlefield statistics.

[0107] (2) Action Space: Action 0 indicates observation and waiting (no action is taken for the time being, waiting for more complete information); Action Indicates priority to handle the first There are multiple targets. The size of the action space changes dynamically with the number of targets in the scene.

[0108] (3) Reward function: ,in:

[0109] Successful handling reward A positive bonus is awarded based on the target's weapon type: +1 for no weapon, +3 for rocket, and +5 for gun; an additional +1.5 is awarded if the target is in an attack (shooting) state; and an additional +1 is awarded if the target is also a high-threat target according to the rules.

[0110] Time penalty A penalty of -0.5 is imposed for each step to encourage the agent to make quick decisions and avoid procrastination.

[0111] Penalties for failure to handle A penalty of -0.25 is imposed for failure to handle a target; a penalty of -3 is imposed for selecting an invalid action (such as handling a target that has already been handled); a penalty of -0.5 is imposed for each remaining target attacked at the end of the round.

[0112] (4) Training process: The training is divided into two stages. The first stage is behavior cloning pre-training: using an adaptive rule-weighted threat assessment method to collect state-action pair datasets in a dynamic simulation environment. Each state corresponds to a rule-based approach that considers the optimal action; the cross-entropy loss function is used. The policy network for PPO is pre-trained. The second stage is normal PPO training: after pre-training, the standard PPO training process is adopted, using the Actor-Critic architecture. The Actor network and the Critic network share the global situation representation extracted by the graph attention module. The Actor network takes the situation vector as input, passes through two hidden layers (256-dimensional and 128-dimensional), and outputs the action probability distribution through a Softmax layer; the Critic network takes the same situation vector as input, passes through two hidden layers, and outputs a single scalar state value estimate.

[0113] The objective function of the PPO algorithm is as follows:

[0114]

[0115] in The probability ratio of the strategies. This is the generalized advantage estimate (GAE). The clipping factor. Total loss function. ,in The mean squared error loss of the value function. This is the policy entropy regularization term.

[0116] Key hyperparameter: learning rate Discount factor GAE parameters Cutting factor The total number of training steps is 500,000, and the number of rounds in the behavior cloning pre-training phase is 100.

[0117] (5) Reasoning and deployment: After training, the agent outputs the priority targets for handling in real time based on the current situation vector, realizing end-to-end autonomous threat assessment and decision-making.

[0118] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target threat assessment method based on graph neural networks and deep reinforcement learning, characterized in that: Includes the following steps, Step 1: Threat element extraction and preprocessing; obtain multi-dimensional basic information of each target from the output of the multi-target detection and tracking model, including action state, weapon type, movement speed, spatial position and detection confidence. The multidimensional basic information is preprocessed in parallel with temporal motion information correction, target distance estimation, and occlusion ratio estimation. Specifically, dynamic threshold-driven temporal motion information correction is performed on the motion state; the target distance is calculated using a perspective projection-based distance estimation method, and the ranging result is corrected using an error compensation model; the target occlusion ratio is estimated based on tracking matching cost or target loss state; the temporal motion information correction is used to reduce jitter in multi-frame images; the target distance estimation is used to estimate target distance information from two-dimensional images; and the occlusion ratio estimation is used to estimate the degree of target occlusion. The temporal action information correction involves continuously recording the action labels output by the detection network for each tracked target. and the corresponding detection confidence level Forming a time series ,in This indicates the original test results. To correspond to the confidence level; the judgment threshold is dynamically adjusted based on the historical statistics of the confidence level, and a high threshold is calculated. and low threshold Perform state consistency judgment and use an asymmetric acknowledgment frame threshold for state verification; The target distance estimation method specifically involves: obtaining the camera intrinsic parameter matrix K using the Zhang Zhengyou calibration method; selecting the bottom center point of the target detection box as the foot point based on the pinhole camera imaging model; establishing the perspective projection mapping relationship between the pixel coordinates of the imaging plane and the world coordinate system; and deriving the world coordinates of the foot point based on the pinhole camera model. Find the closed-form solution and calculate the horizontal distance between the target and the camera optical center. When the target's foot point is close to the lower edge of the image, the target's pixel width is introduced as an auxiliary ranging criterion to enhance the robustness of near-range estimation; real distance data is collected using a laser rangefinder to construct a quadratic polynomial error compensation model. Post-compensation is performed on the initially estimated distance; among which, , , These are the error compensation coefficients for the fitted estimate; The occlusion ratio is estimated as follows: when the target and the detection result match normally, the occlusion ratio is... Where C is the matching cost; when the target is marked as temporarily lost, ; Step 2: Two-dimensional soft rule assessment of action threat and fire threat; construct two assessment dimensions, action threat and fire threat, and calculate the action threat value and fire threat value for each target separately; introduce a confidence weighting factor. Motion cost weighting factor By combining an adaptive weight adjustment mechanism based on population statistical characteristics, a preliminary threat score for each target is generated. ; The confidence weighting factor is: Used to reflect the reliability of perceived information; among which, The confidence level output for target detection; The motion cost weighting factor The distance from the image center is calculated by the Euclidean distance between the target pixel coordinates and the image center, which reflects the efficiency of aiming at the target. The adaptive weight adjustment mechanism based on group statistical characteristics dynamically adjusts the action threat weight according to the group statistical characteristics. and firepower threat weight ; The initial threat score Represented as: ;in, Scoring for action threat; Scoring for firepower threat; The threat value is calculated as follows: when the target is firing, the movement speed is negatively correlated with the threat level, with lower speed or a stationary state indicating a higher threat level; when the target is not firing, the movement speed is positively correlated with the threat level, with higher speed suggesting that the target is rapidly approaching. The fire threat value is calculated by setting a base threat value for each type of weapon, with a gun having the highest base weight, a rocket launcher having the second highest base weight, and no weapon having a base weight of zero; a distance threat factor is calculated by combining the target distance and the effective range, with targets outside the effective range having a fire threat of zero; within the effective range, the closer the target is, the higher the threat level. The adaptive weight adjustment mechanism based on group statistical characteristics is as follows: when the proportion of shooting targets in the scene is high, the fusion weight of the action threat dimension is appropriately increased; when the proportion of armed targets is high, the fusion weight of the firepower threat dimension is increased, and the adjustment is made in combination with the average target distance. Step 3: Intuitive Fuzzy Feature Quantization; Using intuitionistic fuzzy set theory, weapon type, action state, detection confidence, and occlusion ratio information are quantified into intuitionistic fuzzy membership degrees. and hesitation , as the feature embedding of the target node; Step 4: Integrate intuitionistic fuzzy constraints to model multi-target relationships using a graph attention network; explicitly model the target set at each time step as a fully connected graph structure, with each target node containing a threat feature vector and the intuitionistic fuzzy feature embedding; use a graph attention network to aggregate node features, introducing the intuitionistic fuzzy membership degree and hesitation degree as attention constraints during the multi-head attention coefficient calculation process, adaptively learning the mutual influence between targets and extracting the interaction relationship features between targets; perform average pooling on the output embeddings of all nodes to obtain graph-level embeddings, concatenate them with three-dimensional global statistical features, and then perform feature projection to generate a situation vector that integrates local interaction information and macro-battlefield statistical features; the three-dimensional global statistical features include the proportion of shooting targets, the proportion of weapon-wielding targets, and the average target distance; The graph attention network employs a two-layer GAT convolutional structure. Specifically: in feature mapping, the original target node features are mapped to a 64-dimensional latent space through a multilayer perceptron to obtain an initial node feature matrix; in fully connected graph construction, each target is treated as an independent graph node, and all nodes are connected pairwise to form a fully connected graph structure; in the first GAT convolutional layer, a multi-head attention mechanism is used to compute node... To his neighbors Attention coefficient The neighbor information is weighted and aggregated accordingly, and the updated node representation is output after passing through the ReLU activation function. On the second layer of GAT convolution, the node features output by the first layer are further aggregated to output the final 64-dimensional node embedding vector. In the generation of the situation vector, all node embeddings are averaged to obtain graph-level embeddings, which are then concatenated with the three-dimensional global statistical features and projected onto the 128-dimensional situation vector. Step 5: Intelligent decision-making based on rule guidance and proximal policy optimization; using the situation vector as state input, the decision result is output using the proximal policy optimization method; the action space includes observation / waiting and prioritizing the handling of a certain target; a two-stage training mechanism is adopted in the training phase: the first stage is behavior cloning pre-training, using preliminary threat scoring. The first stage involves ranking targets by threat level under the current situation, using the top-ranked target as the action label to train the policy network, incorporating the decision-making bias of soft rule methods. The second stage is proximal policy optimization and reinforcement training, autonomously optimizing long-term decision benefits in a dynamic interactive environment. The reward function integrates disposition benefits, time costs, and decision penalties, and the disposition benefits are correlated with the initial threat score. They are positively correlated; The behavior cloning pre-training involves collecting a state-action pair dataset in a dynamic simulation environment using an adaptive rule-weighted threat assessment method. Each state corresponds to the optimal action as determined by the rule-based method; and the policy network of the near-end policy optimization algorithm is pre-trained using the cross-entropy loss function; after pre-training, the normal training process of near-end policy optimization is initiated. Step 6: The intelligent agent outputs priority targets for action in real time based on the current situation vector, realizing end-to-end autonomous threat assessment and decision-making.

2. The target threat assessment method based on graph neural networks and deep reinforcement learning as described in claim 1, characterized in that: In step 1, the high threshold and low threshold Represented as, ； ； in, and Here, represents the mean and standard deviation of the confidence sequence for the most recent L frames, respectively. and These are the 75th percentile and the 25th percentile, respectively. These are the weighting coefficients. It is a multiple of the standard deviation.

3. The target threat assessment method based on graph neural networks and deep reinforcement learning as described in claim 1, characterized in that: In step 1, the consistency judgment of the execution state is as follows: if the detection result of the current frame is inconsistent with the state of the previous frame, then only if the confidence level is higher than the previous frame's state will the execution state be determined. The results of this test will be adopted only if the results are consistent; if they are consistent, then as long as the confidence level is higher than [a certain value], the results will be adopted. That is, maintain the original state, otherwise perform a state flip.

4. The target threat assessment method based on graph neural networks and deep reinforcement learning as described in claim 1, characterized in that: In step 1, the state verification using an asymmetric confirmation frame threshold means that the transition from a non-firing state to a firing state requires continuous... A frame is detected as firing, and the transition from firing state to non-firing state requires consecutive frames. The frame was detected as non-shooting, in which ,in , .

5. The target threat assessment method based on graph neural networks and deep reinforcement learning as described in claim 1, characterized in that: In step 1, the world coordinates of the foot point The closed-form solution is, in, The angle between the camera's optical axis and the horizontal plane; The horizontal coordinates of the target foot point on the image; Vertical coordinates of the target foot point on the image; equivalent focal length , and principal point coordinates , ; The altitude of the camera in the world coordinate system; The coordinates of the target footpoint in the world coordinate system.

6. The target threat assessment method based on graph neural networks and deep reinforcement learning as described in claim 1, characterized in that: In step 3, intuitive fuzzy membership degree and hesitation Calculated as, ； ； in, To test the confidence level, The base threat value for the weapon. For behavioral confidence, Set the value to 1 if the target is in an attack state, otherwise set it to 0. The target occlusion ratio obtained in step 1; ensured by truncation. and .

7. The target threat assessment method based on graph neural networks and deep reinforcement learning as described in claim 1, characterized in that: In step 5, the reward function Represented as, ； in, Successful handling of the target grants a positive reward based on the target's weapon type: +1 for no weapon, +3 for rocket, and +5 for gun; an additional +1.5 if the target is in an attack state; and an additional +1 if the target is also a high-threat target according to the rules. As a time penalty, a penalty of -0.5 is applied for each step, which accumulates with the number of steps. Penalties for failure to handle: -0.25 penalty for failure to handle; -3 penalty for selecting an invalid action; -0.5 penalty for each remaining target attacked.

8. The target threat assessment method based on graph neural networks and deep reinforcement learning as described in claim 1, characterized in that: In step 5, the proximal policy optimization method adopts an Actor-Critic architecture, where the Actor network and the Critic network share the global situation representation extracted by the graph attention module; the Actor network takes the situation vector as input, passes through two hidden layers, and outputs the action probability distribution through a Softmax layer; the Critic network takes the situation vector as input and outputs a single scalar value estimate; during training, the policy loss and value loss are calculated separately, and the shared and independent parameters of the two networks are jointly updated.

9. A target threat assessment method applied in a multi-target autonomous combat scenario for individual soldiers and squads, characterized by employing the method described in any one of claims 1-8 in the application of the method in a multi-target autonomous combat scenario for individual soldiers and squads: Includes the following steps, Step 1: Using the method described above, extract threat elements and preprocess the real-time data output by the multi-target detection and tracking model to obtain the action state after time-series correction, the target distance after error compensation, and the occlusion ratio parameters. Step 2: Perform two-dimensional soft rule evaluation, intuitionistic fuzzy feature quantification, graph attention relationship modeling, and proximal policy optimization decision-making using the method described above to generate a global situation vector and output the threat priority ranking results of each target; Step 3: Apply the decision results of the priority targets to the actual combat scenario of multi-target autonomous confrontation between individual soldiers and squads.

Images