An air target tracking evaluation method based on infrared search

Abstract

The application discloses an air target tracking evaluation method based on infrared search, relates to the technical field of target tracking, and comprises the following steps: collecting an infrared image sequence in real time and eliminating background images, converting a target enhanced image into a binary image, randomly determining a pixel point with a value of 1, selecting eight neighbor pixels adjacent to the pixel point, connecting the pixel points with the same value of 1, and forming a connected domain; acquiring all the connected domains, calculating the characteristic parameters of each connected domain for screening, marking the screened connected domains as candidate targets, assigning an exclusive ID to each candidate target, forming a candidate target list, performing background elimination and adaptive threshold segmentation on the collected infrared image sequence, extracting the candidate targets, and constructing a tracking chain containing appearance and motion features, so that continuous and stable tracking of the targets is realized. In view of the problem that identities are easily confused in multi-target tracking, a graph neural network is introduced to globally optimize identity distribution of target nodes with associated ambiguity, and identity switching is effectively inhibited.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, and in particular to an infrared search-based method for tracking and evaluating aerial targets. Background Technology

[0002] In the field of infrared search and track (IRST) technology, the circular scan infrared search system is an important passive detection method. It acquires infrared image sequences by continuously scanning the airspace, thereby realizing target detection and tracking.

[0003] However, existing technologies face numerous challenges in handling air target tracking and assessment. First, due to the mechanical rotation of the periodic scanning system, the revisit period for the same specific airspace is relatively long (typically about 1 second), leading to severe information loss when tracking fast-moving targets and making it difficult to establish continuous and stable target trajectories. Second, when a target encounters brief obstruction, signal attenuation, or intersects with other targets, traditional tracking algorithms based on motion models or simple feature matching are prone to target loss or identity confusion, and the re-identification process is slow and inefficient, severely compromising the integrity of the trajectory.

[0004] Furthermore, existing methods largely rely on single-frame image detection or simple inter-frame correlation, lacking in-depth mining and utilization of target spatiotemporal context information, and are unable to cope with multi-target, high-density tracking scenarios in complex battlefield environments. In terms of damage assessment, existing technologies are usually based only on intuitive interpretation at the image level or simple changes in motion state, resulting in insufficient accuracy and low reliability of assessment results. Summary of the Invention

[0005] This application provides an infrared search-based method for tracking and evaluating aerial targets, which solves the problems of discontinuous target information caused by long revisit cycles, target occlusion, or cross-flight issues leading to identity switching and trajectory breakage in existing technologies. It achieves a refined, accurate, continuous, and stable tracking effect for aerial targets based on multi-source information fusion.

[0006] This application provides an infrared search-based method for tracking and evaluating aerial targets, including: S1: Real-time acquisition of infrared image sequences and removal of background images to obtain target enhancement images; conversion of target enhancement images into binary images, random determination of a pixel with a value of 1, selection of its 8 neighboring pixels, and connection of pixels with the same value of 1 to form a connected component; S2: Obtain all connected components, calculate the feature parameters of each connected component for filtering, mark the filtered connected components as candidate targets, assign a unique ID, and form a candidate target list; the feature parameters include area, average gray level, centroid position, and shape features, and set a corresponding threshold range for each feature parameter for filtering. If each feature meets the corresponding set threshold range, it means that it passes the filtering; if any feature parameter does not meet the corresponding set threshold range, it means that it does not pass the filtering. S3: Obtain a list of candidate targets, and extract a fixed-size image patch centered on the centroid of each candidate target. Input the patch into a convolutional neural network and output an appearance feature vector. Initialize a uniform motion model and generate a motion feature vector. Form a tracking chain based on the appearance feature vector and the motion feature vector. The tracking chain includes a unique ID, tracking confidence and its historical maximum value, historical state sequence, appearance feature vector, and motion feature vector. S4: Update the tracking confidence based on real-time conditions. If the tracking confidence drops to 0, mark the corresponding tracking chain as to be deleted and store the last appearance feature vector and motion feature vector of the candidate target into the missing target feature library. If the tracking confidence does not decrease for 5 consecutive frames, mark the tracking chain as a stable chain, mark the candidate target as a stable target, and perform tracking evaluation.

[0007] Furthermore, the method also includes: S5: obtaining the candidate target list of the current frame, calculating the similarity between each candidate target and the stable chain, and if the similarity is greater than 0.6, it is suspected to match; recording the number of stable chains that are suspected to match the candidate target, and if the number is 1, adding it to the matching stable chain; if the number is less than 1, marking it as a target to be identified; if the number is greater than 1, marking it as a multi-source target; re-analyzing the multi-source target and the target to be identified based on the dynamic grading rules and performing tracking and evaluation.

[0008] Furthermore, the dynamic grading rule includes: S51: For multi-source targets: extract candidate targets corresponding to all stable chains that are suspected to match in the current frame as associated targets, define the multi-source targets and several associated targets as nodes, and represent each node with a feature vector, including appearance features, motion features and historical state sequence; connect any two nodes to form an edge and use the similarity between nodes as the edge weight to form a graph structure; perform message passing through multiple iterations for each edge, and after traversal, each node updates its own feature vector according to the aggregated message; S52: Concatenate the updated feature vectors of the two endpoints of any edge to form an edge feature vector. Input the edge feature vector into the edge classifier and output the probability value. After traversal, obtain the association probability of each edge and form an association probability matrix. Determine the stable chain to which each multi-source target belongs based on the association probability matrix.

[0009] Furthermore, the dynamic grading rule also includes: S53: For the target to be identified, a dynamic neighborhood region is defined with the target to be identified as the center; all stable targets and reference objects within the dynamic neighborhood region are identified, and the relative spatial relationship vector between the target to be identified and each reference object is calculated; the micro-motion trajectory of the target to be identified in the current frame and short period is extracted, and the initial motion features are calculated; the reference objects, relative spatial relationship vectors, and initial motion features of the target to be identified are uniformly encoded to form a prior vector; a behavior pattern library is generated based on historical data analysis, and the prior vector of the target to be identified is matched with the behavior pattern library to determine the most consistent motion pattern; the short period refers to a time range period of 1-5 frames.

[0010] Furthermore, the dynamic neighborhood region refers to a spatial range dynamically determined based on sensor performance, target kinematic characteristics, and tactical scenario, centered on the current position of the target to be identified. By reviewing the intermediate processing results within a short period, it searches for any suspected points that have not yet been definitively identified and treats them as motion changes of the target to be identified. The suspected points are then connected in chronological order to form a micro-motion trajectory. Based on the micro-motion trajectory, the instantaneous motion state of the target is estimated using a kinematic model, and the angular displacement and total time interval from the trajectory start point to the end point are calculated. The initial motion features include the initial velocity vector, acceleration trend, and motion curvature.

[0011] Furthermore, forming a priori vector includes: normalizing the relative spatial relationship vector; sequentially concatenating the normalized relative spatial relationship vectors of all reference objects into a spatial relationship feature vector; normalizing and concatenating the calculated initial motion features to obtain a motion feature vector; and directly concatenating the spatial relationship feature vector and the motion feature vector to form the final priori vector.

[0012] One or more technical solutions provided in this application have at least the following technical effects or advantages: By introducing a tracking chain management mechanism, discrete inter-frame detection points are associated into continuous trajectories with unique identities. By utilizing the correlation information in the time dimension, prediction and interpolation can be performed based on historical information when the target encounters brief occlusion or signal flicker, which greatly reduces the probability of target loss and ensures the integrity of the trajectory. By employing an adaptive threshold based on multiple features to filter candidate targets and combining it with deep appearance features extracted by a convolutional neural network, the accuracy of target detection is greatly enhanced, and false alarms and missed detections are reduced. For scenarios where multiple targets intersect or fly close together, which are prone to identity confusion, the optimal identity allocation is achieved by fusing global context information through message passing and node updates, effectively solving the identity switching problem. For newly emerging or temporarily unidentifiable targets, a rapid identification scheme based on spatiotemporal context awareness is proposed. By analyzing the relative relationship between the target and surrounding stable reference points, extracting its micro-motion trajectory, and calculating motion features, a priori vector is formed and matched with a behavior pattern library to quickly infer its possible motion patterns or tactical intentions. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of a method for tracking and evaluating air targets based on infrared search, as described in an embodiment of the present invention. Detailed Implementation

[0014] To facilitate understanding of the present invention, a more complete description of this application will be given below with reference to the accompanying drawings, which illustrate preferred embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to enable a more thorough and complete understanding of the disclosure of the present invention.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0016] Example 1: As Figure 1 As shown, an infrared search-based method for tracking and evaluating aerial targets includes the following steps: S1: Real-time acquisition of infrared image sequences; identification of background images based on several frames of infrared image sequences; removal of background images to obtain target enhancement images; conversion of the target enhancement images into binary images; random determination of a pixel with a value of 1; selection of its eight neighboring pixels; and connection of pixels with the same value of 1 to form a connected component.

[0017] The infrared search device acquires a continuous sequence of infrared images within its search range. Essentially, this is a low-frame-rate video stream. The infrared search device can utilize a circular infrared search and track (IRST) system, a passive airborne optoelectronic device based on infrared radiation detection. It performs a continuous 360-degree horizontal rotation scan and has a fixed field of view vertically. Each rotation (one full scan) acquires a panoramic infrared image covering 360 degrees, or a sequence of image frames covering different azimuths, output in chronological order. Due to the limitations of mechanical rotation speed, the revisit period for the same specific point in space is approximately one second. Therefore, the processed data is a sequence of infrared image frames with circles or seconds as the basic time unit, serving as the raw data source.

[0018] Based on image data from the current frame and several previous frames, a background image that best represents static or slowly changing conditions is calculated, excluding fast-moving targets. Specifically, a moving average method is used. Assume that the most recent N frames are stored (e.g., N=10). For each pixel (x,y) in the image, combined with the acquisition timestamp t, the background estimate B(x,y,t) is calculated as follows: B(x,y,t)=α*I(x,y,t)+(1-α)*B(x,y,t-1), where I(x,y,t) is the grayscale value of that pixel in the current frame, B(x,y,t-1) is the background estimate from the previous time step, and α is the learning rate (set to 0.05). The learning rate is a hyperparameter between 0 and 1, used to control the weight distribution between new observations and historical background estimates during the update process. A larger value indicates greater trust in the current frame's observations, while a smaller value indicates greater reliance on historical background estimation. Setting it to 0.05 is an optimization trade-off based on the system characteristics of periodic IRST (revisit period approximately 1 second). In practical applications, this needs to be dynamically set according to scene change detection and pixel changes; this application does not impose specific restrictions here. Each pixel is processed independently, using its historical grayscale value sequence to predict the current background value. Moving targets, due to their brief appearance and large differences from historical values, are averaged out in the background image and thus not included. Subtracting the estimated background image from the current original image yields the difference image, i.e., the target enhancement image. In the target enhancement image, the values ​​of the background region approach 0, used to highlight the target region. The target enhancement image is a grayscale image; the value of each pixel represents the degree of difference between that point and the background, with a larger value indicating a higher probability that the point is a target.

[0019] The target image for enhancement is converted into a binary image. A sliding window and a step size are pre-defined. A fixed-size window, centered on the coordinates of the pixel to be processed or an arbitrarily selected pixel on the image edge, is defined and slid across the target image according to the step size until the entire image is covered. After each movement, the mean and standard deviation of the grayscale values ​​of all pixels within the sliding window are calculated. Once the sliding window covers the entire target image, a list of statistical values ​​is obtained, including the coordinates, timestamp, mean, and standard deviation of the sliding window. The step size must be smaller than the length of the sliding window to ensure complete coverage of the target image.

[0020] The local threshold for each sliding window is calculated using the adaptive threshold formula T = μ + k * σ, where μ is the mean, σ is the standard deviation, and k is a constant factor used to control sensitivity. This constant is pre-set based on the statistical confidence interval concept and engineering experience; for example, k = 3 corresponds to a 99.7% confidence level, resulting in a low false alarm rate but potentially missing weak targets. k = 5 corresponds to a higher confidence level, reducing the likelihood of missed alarms, but potentially increasing false alarms. In practical applications, fine-tuning is required based on the noise level of the specific sensor and actual needs. Each sliding window corresponds to a local threshold. The grayscale value of the center pixel is compared with the calculated local threshold. If it is greater than the local threshold, it is determined to be a target pixel, and its value is set to 1 in the binary image; otherwise, if it is not greater than the local threshold, it is determined to be a background pixel, and its value is set to 0 (represented as black) in the binary image. By performing threshold comparisons on every pixel in the target enhancement image, a binary image is finally obtained. A binary image is an image where pixel values ​​are only 0 (representing background) or 1 (representing target).

[0021] A pixel with a value of 1 is randomly selected, and its eight neighboring pixels are chosen. Pixels with the same value of 1 are connected to form a connected component. Specifically, this application selects eight directions as adjacency, meaning that a pixel is considered adjacent to its direct neighbors in eight directions: top, bottom, left, right, top-left, top-right, bottom-left, and bottom-right. Eight-way connectivity better groups diagonally adjacent pixels belonging to the same target together, forming a complete cluster and avoiding fragmentation into multiple smaller parts. In practical applications, to improve data processing efficiency, four-way connectivity can also be selected, meaning only adjacent pixels in the top, bottom, left, and right directions are chosen.

[0022] S2: Obtain all connected components, calculate the feature parameters of each connected component for filtering, mark the filtered connected components as candidate targets, assign a unique ID, and form a candidate target list. The feature parameters include area, average gray level, centroid position, and shape features. In practical applications, it is necessary to set a corresponding threshold range for each feature parameter for filtering. If each feature meets the corresponding threshold range, it means that the filtering is passed; if any feature parameter does not meet the corresponding threshold range, it means that the filtering is not passed.

[0023] Specifically, the area threshold is not a fixed value; its setting directly depends on the spatial resolution of the imaging system and the expected physical size and distance of the target. The main purpose of the lower area threshold is to suppress noise. Random noise in the image may form extremely small connected regions with an area of ​​1-2 pixels. According to the point spread function, an ideal point target will also occupy a small spot area on the image plane (usually with a diameter greater than 3 pixels). Therefore, connected regions with excessively small areas are statistically highly likely to be noise. The minimum area threshold should be greater than the maximum possible area of ​​noise point clusters under specific operating conditions, specifically determined through experiments. Multiple frames of images are acquired under a uniform background without targets (such as a clear sky), connected component analysis is performed, the areas of all detected connected components are counted, and their probability distribution is calculated. The lower threshold is set to the area value at the 99th quantile of this distribution (such as 9 pixels). The upper limit aims to filter out large-sized interference objects that are not of interest, such as clouds or flocks of birds. Its setting depends on the maximum imaging size of the farthest target expected in the tactical scenario, and is specifically set according to the size of interference objects that are likely to appear in the actual application scenario, or determined through the above experiments.

[0024] The grayscale threshold corresponding to the average grayscale is used to ensure that the target has a sufficient signal-to-noise ratio. Its setting is based on statistical detection and actual setting. The grayscale value of the target pixel should be significantly higher than the grayscale fluctuation level of the local background. For each candidate connected component, a background ring is defined around it. With the centroid of the connected component as the center, an annular region with an inner diameter of 1.5 times the radius of the circumcircle of the connected component and an outer diameter of 3 times is taken. The grayscale values ​​of all pixels in this background ring are calculated as the grayscale threshold.

[0025] The threshold for the centroid position is based on the target's azimuth and pitch angles, ensuring that the azimuth and pitch angles are within the sensor's effective field of view. For example, for a circular scanning system, the azimuth angle should be between 0° and 360°, and the pitch angle should be between -5° and +60° (set according to specific equipment parameters).

[0026] Shape characteristics include aspect ratio and circularity. Ideally, distant point targets should be approximately circular (determined by the point spread function of the optical system), while cloud edges, power lines, and other obstructions typically exhibit elongated or irregular shapes. For aspect ratio, the aspect ratio of the smallest bounding rectangle of the connected region is calculated. For point targets, this value should be close to 1, with a set range such as [0.5, 2.0] to exclude excessively narrow noise. The degree of relaxation depends on target maneuverability and imaging quality. For circularity, the ideal value is 1. The actual lower threshold can be set based on the impact of optical distortion and noise on contour extraction. By imaging known point targets (such as stars used for calibration or long-range UAVs), the statistical distribution of their circularity is calculated, and the lower limit is set to the 5th percentile of the distribution to ensure that 95% of real targets are not mistakenly deleted.

[0027] S3: Obtain a list of candidate targets. For each candidate target, extract a fixed-size image patch centered on its centroid. Input this patch into a lightweight convolutional neural network and output a 128-dimensional appearance feature vector of the target. Initialize a uniform motion model for the target, setting the initial velocity to 0, and generate a motion feature vector. Form a tracking chain based on the appearance feature vector and the motion feature vector.

[0028] In some embodiments, a fixed-size image patch is cropped centered on the centroid of each candidate target to ensure data processing efficiency and data feature representation capability. The size is set based on all candidate targets in the candidate target list, sequentially determining the suspected area of ​​each candidate target to obtain the maximum area. The preset fixed-size area should be greater than the maximum area, with a value range between (1, 1.5] times the maximum area. This ensures coverage of the target and a small number of adjacent background pixels, providing the convolutional neural network with contextual information to distinguish the target from background noise. For distant point targets (possibly only 3x3 pixels), a 32x32 patch ensures the target is located in the central region with sufficient background surrounding it for the network to learn discriminative features. Excessively large image patches significantly increase the computational load on the convolutional neural network, affecting system real-time performance. For example, with a set size of 32x32 pixels, the cropped infrared image patch is normalized (e.g., scaling pixel values ​​from 0-255 to between 0-1). The image patch is then fed into a pre-trained CNN, consisting of multiple stacked convolutional layers. Each layer uses multiple small convolutional kernels (filters) to perform sliding convolution operations on the input image or the feature map of the previous layer, extracting local features (such as edges, corners, and textures). Each convolution is followed by a non-linear activation function (such as ReLU) to introduce a non-linear transformation and enhance the model's expressive power. Pooling layers (such as max pooling) are periodically inserted. These pooling layers downsample local regions, retaining the most salient features while reducing data size, increasing the network's robustness to slight changes in target location, and reducing computational cost. After several rounds of convolution-activation-pooling, the resulting feature map is flattened into a one-dimensional vector and input into one or more fully connected layers. Finally, the convolutional neural network compresses the features to a specified dimension (128 dimensions in this example), resulting in the extracted appearance feature vector. The above uses a lightweight convolutional neural network as the analysis model to generate appearance feature vectors for candidate targets; further details are omitted here.

[0029] Appearance feature vectors are highly abstract feature representations that collectively encode the visual attributes of the original image patch. Different dimensions of the network may be sensitive to specific patterns of the target (such as brightness distribution, shape contour, and texture characteristics). The entire vector uniquely represents the appearance of the target in the feature space. When a target reappears after being occluded, or when identity confusion occurs, its position and motion state may have changed drastically, but its appearance features remain relatively stable. By comparing the target features detected in the current frame with historical features stored in the tracking chain, high-precision identity matching is performed, achieving intelligent re-identification. In scenes with dense or overlapping targets, relying solely on positional information for association is prone to errors. Combining appearance feature similarity can significantly improve the accuracy of data association and reduce the identity switching rate.

[0030] In some embodiments, a uniform motion model is initialized for the target. In the initial stage, since only one frame of detection information is available, the true motion state of the target cannot be known. Therefore, the most basic and commonly used assumption is a uniform linear motion model, that is, assuming that the target maintains a constant velocity magnitude and direction for a short period of time. Since only one frame of data is available, the velocity cannot be calculated. Therefore, the initial velocity of the target is set to 0 in both the azimuth and pitch directions. At the same time, a state vector containing position and velocity is initialized, for example, using a Kalman filter. The motion feature vector is a low-dimensional vector, mainly used to quickly compare the consistency of motion. The motion information is updated in real time in subsequent frames, including instantaneous angular velocity, average angular velocity (the average value of the velocity over the past N frames, reflecting the recent motion trend), and motion direction (the motion trend in the azimuth).

[0031] A tracking chain is formed based on appearance feature vectors and motion feature vectors. The tracking chain includes a unique ID, tracking confidence level and its historical maximum value, historical state sequence, appearance feature vector, and motion feature vector. The tracking confidence level quantifies the degree of certainty that the target represented by the current tracking chain truly exists and is correctly identified. When a new candidate target passes detection screening and is added to the tracking chain, an initial confidence level is assigned based on the detection strength at the time of its creation; the higher the detection strength, the higher the initial confidence level. The initial confidence level can be designed as a function based on the target's signal-to-noise ratio (SNR), representing the ratio of the candidate target's SNR to the SNR threshold set in the detection process. For example, a powerful target has a higher initial confidence level (e.g., 0.8) at creation, while a weak target starts with a lower confidence level (e.g., 0.4), thus measuring the target's authenticity.

[0032] S4: Update the tracking confidence based on real-time conditions. If the tracking confidence drops to 0, mark the corresponding tracking chain as to be deleted and store the last appearance feature vector and motion feature vector of the candidate target into the missing target feature library for re-identification. If the tracking confidence does not decrease for 5 consecutive frames (i.e., remains unchanged or increases), mark the tracking chain as a stable chain and the candidate target as a stable target.

[0033] The tracking confidence is updated in real time, including: if the motion trajectory of a candidate target is predicted in the next frame based on the motion prediction model, the predicted trajectory is compared with the actual trajectory; firstly, the similarity of the appearance features between the predicted target and the actual detected target is calculated based on the appearance feature vector. If the similarity is lower than a preset threshold (0.6 in this application, but can be set according to actual needs, the purpose is to perform preliminary screening based on appearance feature similarity), they are directly determined to be different targets, and no overlap calculation is performed; if the similarity is higher than the preset threshold, the position overlap is calculated; if the overlap (equal to or greater than 0.7) indicates a successful prediction, meaning a detection target has been found for the tracking chain, and its confidence is significantly increased (tracking confidence incremented by one); if the overlap is greater than 0.4 and less than 0.7, it is marked and the original tracking confidence is maintained; if the overlap is less than or equal to 0.4, it indicates a prediction failure, and the tracking confidence is decremented by one. The highest confidence reached during the lifecycle of the tracking chain is recorded as the historical maximum value. The prediction model used here can be trained using a Long Short-Term Memory (LSTM) network as the base model. Training and validation using candidate target list data is sufficient, and this application will not elaborate further. There are various methods for calculating overlap. Overlap refers to the degree of positional overlap between the predicted bounding box of the predicted motion trajectory and the detected bounding box of the actual motion trajectory. In this application, the overlap is calculated as the ratio of the intersection area of ​​the two bounding boxes (predicted bounding box and detected bounding box) to the union area of ​​the two bounding boxes, with a value range of (0,1]. A larger value indicates a higher degree of positional overlap. The predicted bounding box is a rectangular box extrapolated from the motion prediction model based on the historical state of the tracking chain, representing the target's position in the current frame. The detected bounding box is the target's bounding rectangle obtained from the candidate target detection process in the previous frame. All bounding boxes use the same coordinate system. For point targets, the bounding box is set as a rectangle centered on the centroid with a fixed size (e.g., 32×32 pixels).

[0034] The historical state sequence is a chronologically ordered queue that records a snapshot of the tracking chain's state for each frame throughout the candidate target's entire lifecycle. This includes, but is not limited to: timestamps, target position (transformed coordinates, converting all candidate target sensor coordinate systems to a unified coordinate system), motion state (such as instantaneous velocity and acceleration, depending on the specific operational model), and observation information (raw observations at the current moment, such as area and signal-to-noise ratio). The historical state sequence provides the temporal context of the target's behavior, improving tracking robustness, enabling it to handle brief disturbances and missed detections, and allowing for smoothing and interpolation based on historical trajectories. It also improves prediction accuracy, helping to more accurately estimate target dynamic model parameters, reduce prediction errors, and ultimately achieve trajectory-level diagnostics. Furthermore, it records the evolution of the target's kinematics and dynamics over time.

[0035] In some embodiments, each candidate target corresponds to a tracking chain in the initial state. If candidate targets with the same appearance feature vector and motion feature vector exist consecutively in the obtained candidate target list, they will be merged into a tracking chain.

[0036] Existing technologies cannot utilize temporal correlation information when processing each frame of an image, cannot handle complex scenes such as target occlusion, intersection, and flickering, and cannot perform trajectory-based damage assessment. In this embodiment, discrete, frame-independent detection points are associated into continuous, uniquely identified trajectories, i.e., tracking chains corresponding to candidate targets.

[0037] By forming a tracking chain, each newly emerging target is assigned a unique and stable identity, enabling the system to distinguish target A from target B in subsequent frames, rather than treating them all as indiscriminate candidate targets. Recording the target's historical trajectory (position, velocity, etc.) and features allows for prediction of the target's most likely location in the next frame based on historical information, greatly improving the efficiency and accuracy of search and association. When a target is briefly occluded or noise causes a single frame to be missed, the historical information (such as motion models and appearance features) stored in the tracking chain can serve as memory, enabling fast and accurate re-identification when the target reappears, preventing trajectory interruption. The historical state sequence focuses on the dynamic behavior of the target over time and is the foundation for prediction and trajectory analysis. Feature vectors focus on the target's static intrinsic attributes and are key to identity recognition and matching. In the tracking chain, the historical sequence ensures continuous tracking of a moving entity, while the feature vectors ensure the accuracy of tracking; the two complement each other.

[0038] In this embodiment, robust detection and continuous tracking of aerial targets solve the core problem of discontinuous target trajectories and easy loss due to the isolation of inter-frame information in existing technologies. The moving average method effectively estimates and eliminates background, significantly highlighting potential targets; adaptive thresholding and connected component analysis accurately extract candidate targets from complex backgrounds; and by constructing a tracking chain containing appearance and motion features for each candidate target, independent detection points in a single frame are successfully associated with continuous trajectories with unique identifiers.

[0039] In Example 2, although the addition of historical state sequences and tracking chains in the above examples ensures real-time tracking of target objects, problems still exist in multi-target tracking. These problems include assigning different IDs to segments of the trajectory of the same real target, or incorrectly associating trajectory segments of two different real targets with the same ID. For example, when the flight paths of two targets intersect, the tracking algorithm may mistakenly identify the subsequent trajectory of target A as a continuation of target B, causing the ID of target A to jump to target B after the intersection point, resulting in an identity switch. Furthermore, in practical applications, target objects often disappear from the field of view due to brief occlusion, signal attenuation, or other reasons. When they reappear, the recognition efficiency when re-associating them with the original tracking chain (or adding a stabilizing chain) decreases, leading to misidentification of the target.

[0040] The method further includes: S5: Obtaining the current frame candidate target list, calculating the similarity between each candidate target and a stable chain, and indicating a possible match if the similarity is greater than 0.6; recording the number of stable chains that are suspected of matching the candidate target, adding the matching stable chain if the number is 1; marking the target as to be identified if the number is less than 1; and marking the target as a multi-source target if the number is greater than 1. The current frame candidate target list represents all currently detected candidate targets, including historically detected and still active targets, as well as newly detected candidate targets; here, a unified analysis is performed on each target.

[0041] The dynamic grading rule includes: S51: Extracting candidate targets corresponding to all stable chains that are suspected of matching as associated targets in the current frame; defining multi-source targets and several associated targets as nodes; each node is represented by a feature vector, including appearance features, motion features, and historical state sequence; connecting any two nodes to form an edge and using the similarity between nodes as the edge weight to form a graph structure. Message passing is performed on each edge through multiple iterations. After traversal, each node updates its own feature vector based on the aggregated message.

[0042] Specifically, there are a total of L iterations, at the [number]th [level]. In the layer iteration, for each edge (Connecting nodes) and ), generate from arrive message vector : in, It is a learnable message function (such as a simple linear transformation, which is not specifically limited in this application). and These are nodes and exist Feature vector representation of layer iteration, It is the weight of the edge (i.e., the similarity between nodes); node Aggregate all neighboring nodes The sent message was received. ;node Combining the features of its previous layer with the aggregated messages, the updated feature representation is as follows: ,in, This is the update function formed by the learnable neural network. After the first layer, the node features contain information about its first-order neighbors. After the second layer, the node features further contain information about its second-order neighbors (neighbors of neighbors). Finally, in the... Layers, features of each node It has incorporated the indirect influence of all nodes within the maximum L-hop range of the entire graph. This allows nodes to perceive the complex dependencies in the graph structure. For example, when two tracking chain nodes TA and TB are both highly similar to a candidate target node CX, through message passing, TA and TB can indirectly perceive each other's existence and competitive relationship, thus reflecting this global constraint in feature updates and avoiding the incorrect assignment of CX to either of them.

[0043] S52: Concatenate the updated feature vectors of the two endpoints of any edge to form an edge feature vector. Input the edge feature vector into the edge classifier and output the probability value. After traversal, obtain the association probability of each edge and form an association probability matrix. Determine the stable chain to which each multi-source target belongs based on the association probability matrix.

[0044] Specifically, this embodiment requires the use of a readout function, which is a module in a graph neural network used to generate task-specific outputs from updated node features. In this scheme, its task is to evaluate the probability that candidate edges (i.e., possible associated edges between candidate target nodes and tracking chain nodes) in the graph represent the same identity.

[0045] The edge feature vector is: , and These are nodes and The final feature vector representation after L layers of message passing.

[0046] The output probability value needs to be determined based on the edge classifier. A shallow neural network, such as a multilayer perceptron (MLP), is selected as the edge classifier, and a scalar score is calculated based on the MLP. And map scalar scores to association probabilities: , It's the edge The association probability of a node and The probability of corresponding to the same target.

[0047] The correlation probability matrix is ​​a two-dimensional matrix P, where rows correspond to all candidate target nodes and columns correspond to all tracking chain nodes; matrix elements It is the correlation probability between candidate target i and tracking chain j. For positions where no candidate edges exist, the probability value is set to 0 or a minimum value.

[0048] Assume there are two candidate targets to be identified in the current frame. and 2 stable chains And there is ambiguity in their relationship (e.g.) and , (They are all similar); the node set is Establish candidate edges: (connect ), (connect ), (connect ), (connect After two layers of GNN message passing, each node obtains the final feature that incorporates global context information. The probability is calculated as follows: For the edge ; For the edge ; Similarly, we obtain the edges. ; This leads to the generation of an association probability matrix: According to the correlation probability matrix: It is highly likely to belong to , It is highly likely to belong to For multi-source targets that still cannot be identified, mark them as targets to be identified.

[0049] The target to be identified refers to a target whose behavioral trajectory cannot be determined and whose behavioral intention cannot be predicted. Therefore, it is necessary to analyze and judge the target to be identified. However, existing technologies require the acquisition of certain behavioral data before analysis, resulting in low recognition and processing efficiency and an inability to quickly identify and determine its movement patterns.

[0050] The method further includes: S53: For the target to be identified, a dynamic neighborhood region is defined with the target to be identified as the center; all stable targets and reference objects within the dynamic neighborhood region are identified, and the relative spatial relationship vector between the target to be identified and each reference object is calculated; the micro-motion trajectory of the target to be identified in the current frame and short period is extracted, and the initial motion features are calculated; the reference objects, relative spatial relationship vectors, and initial motion features of the target to be identified are uniformly encoded to form a prior vector; a behavior pattern library is generated based on historical data analysis, and the prior vector of the target to be identified is matched with the behavior pattern library to determine the most consistent motion pattern and identity hypothesis.

[0051] The dynamic proximity region refers to a spatial range dynamically determined based on sensor performance, target kinematic characteristics, and tactical scenario, centered on the current position of the target to be identified. This region is used to capture other entities (stable targets, reference objects) that may interact with the target being identified. Its range setting needs to balance two aspects: including sufficient contextual information and avoiding the introduction of excessive irrelevant noise. The steps are as follows: First, determine the basic angle unit based on the angular resolution of the infrared sensor. The angular resolution determines the minimum angular interval between two point targets that the sensor can distinguish, and is the basic unit of spatial measurement. Second, determine the region size. The region size should be able to accommodate typical tactical actions and the range that the target may move within a short period (such as one or two scan cycles). Specifically, it is necessary to calculate the horizontal azimuth and elevation angle ranges, and dynamically adjust the size of the proximity region based on these two ranges. This application does not impose specific limitations here. Third, generate a rectangular region, i.e., the dynamic proximity region, centered on the current centroid azimuth (upper and lower limits) and elevation angle (upper and lower limits) of the target to be identified.

[0052] The reference objects refer to significant terrestrial or astronomical landmarks, such as the edge of a stationary cloud, a known ground heat source, or a star of constant brightness. The reference objects are considered spatial neighbors of the target to be identified. The number of reference objects is not specifically limited, but is optimally between [3,5]. The spatial relationship vector is... ,in, and These are the relative angles of azimuth and elevation, respectively. It is a relative distance estimated based on the angle difference.

[0053] The short period refers to a period within an extremely short time (within a time range of 1-5 frames), such as the time range of the most recent 3 frames. It is used to quickly identify motion features within a short period. After determining the short period, the intermediate processing results within that short period (e.g., the most recent 3 frames) are reviewed to search for any unidentified potential points (i.e., potential motion signs). These are considered as motion changes of the target to be identified. The potential points are connected in chronological order to form a micro-motion trajectory. Based on the brief micro-motion trajectory, the instantaneous motion state of the target is estimated using a kinematic model. The angular displacement and total time interval from the trajectory's starting point to its ending point are calculated. The initial motion features include the initial velocity vector (horizontal azimuth angular velocity and pitch angular velocity), acceleration trend, and motion curvature (estimated by fitting trajectory points or calculating changes in continuous steering angles, with the rate of change of the direction angle of adjacent line segments used as a metric).

[0054] The behavioral pattern library is derived by unsupervised learning or expert annotation of a large amount of historical tracking data (including data on confirmed identities and target behaviors) to summarize typical tactical behavioral patterns. It collects trajectory segments and corresponding contextual information of the target from detection to final identity confirmation from historical data. A prior vector is calculated for each trajectory segment, and these prior vectors are clustered using a clustering algorithm. Each cluster center represents a common tactical behavioral pattern prototype. The automatically discovered clusters are manually reviewed and labeled, assigning them tactical meanings, such as escort patterns, formation flying patterns, and interception flight path patterns. Each pattern is described by a representative prior vector prototype and its tactical label.

[0055] The reference objects, relative spatial relationship vectors, and initial motion features of the target to be identified are uniformly encoded to form a prior vector. The relative spatial relationship vectors are normalized, and the normalized relative spatial relationship vectors of all reference objects are concatenated sequentially into a long vector, i.e., the spatial relationship feature vector. If the number of reference objects is not fixed, only the first n closest ones can be used, or a pooling operation (such as averaging) can be used to obtain a fixed-dimensional representation. The calculated initial motion features are normalized and concatenated to obtain the motion feature vector. The spatial relationship feature vector and the motion feature vector are directly concatenated to form the final prior vector. The prior vector has a fixed dimension and contains key contextual information about the target to be identified within its spatiotemporal environment.

[0056] For the prior vector of the target to be identified, calculate its similarity (e.g., cosine similarity) with each prototype in the pattern library. Find the pattern with the highest similarity score, and use the tactical behavior description represented by the pattern as the most consistent inference of the movement law of the target to be identified.

[0057] The method further includes: determining the coordinates of the detonation point based on the optimized list of stable target trajectories, the detonation command signal, and sensor signals; calculating the distance from the detonation point to the predicted position of each stable target, the predicted position being extrapolated from the target's motion state (velocity, acceleration) just before detonation. In explosion detection, the method of matching area change rate and azimuth and elevation angle of the explosion point is used to detect whether the interceptor missile has detonated, and multi-dimensional damage features are extracted, including: calculating the maximum decrease in confidence after the event; analyzing the trajectory points after the event, calculating the approximate entropy of the trajectory, quantifying the randomness and unpredictability of the trajectory, and the trajectory of the destroyed target will be in a disordered state; counting the number of frames with reduced elevation angle in the most recent N frames (e.g., N=5) after the event, calculating the elevation angle change rate, if its value exceeds the theoretical fall rate caused by gravitational acceleration, it strongly indicates loss of power; analyzing the rate of change of the target velocity vector (magnitude and direction) before and after the event, the damaged target may have a sharp decrease in speed but the direction remains unchanged (power loss), while the velocity vector of the destroyed target may be completely disordered; the target image point area diffusion rate, that is, the rate of change of the pixel area occupied by the target in the image, the destroyed target may have a rapidly expanding image point area due to disintegration and then dissipation.

[0058] Based on the extracted multidimensional damage features, all feature values ​​are normalized to the [0,1] interval, where 0 represents no impact and 1 represents the maximum damage index. A fuzzy subset (e.g., "slight," "moderate," "severe") is defined for each feature, and a fuzzy rule base is established. Through fuzzy inference and defuzzification, a continuous damage index is output. The damage level of the current stable target is determined based on the damage index and a pre-determined level classification. Damage levels include destroyed, damaged, and not hit. The specific level classification rules need to be dynamically set based on the results of historical data statistical analysis. As a simple data statistical analysis method, this application does not impose specific limitations here. The final state of the stable target and the assessment probability (the reliability of the result is assessed based on the multidimensional damage features) are obtained. Damage assessment refers to the technical process of using target state information obtained by sensors, combined with battlefield events (such as the detonation of our interceptor missiles), to quantitatively judge and probabilistically predict the degree of damage to a specific aerial target after an attack.

[0059] In this embodiment, a graph neural network is used to globally optimize the identity assignment of multi-source targets with ambiguous associations. A message-passing mechanism is employed to fuse global spatiotemporal constraints, significantly reducing the identity switching error rate in complex scenarios such as multi-target intersection and parallelism. Simultaneously, for the target to be identified, rapid intelligent identification is performed by analyzing its spatiotemporal context and behavioral patterns, greatly improving the speed and accuracy of re-acquisition. This solves the technical challenges of identity confusion and low re-identification efficiency after target occlusion disappears in multi-target tracking; it enhances robustness in highly dynamic and adversarial environments, ensuring the accuracy and completeness of multi-target tracking trajectories, representing a key upgrade for achieving precise situational awareness and damage assessment.

[0060] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for tracking and evaluating aerial targets based on infrared search, characterized in that, include: S1: Real-time acquisition of infrared image sequences and removal of background images to obtain enhanced target images; The target enhancement image is converted into a binary image. A pixel with a value of 1 is randomly selected. The eight neighboring pixels of the same value are selected and connected to form a connected component. S2: Obtain all connected components, calculate the feature parameters of each connected component for filtering, mark the filtered connected components as candidate targets, assign a unique ID, and form a candidate target list; S3: Obtain a list of candidate targets, and extract a fixed-size image patch centered on the centroid of each candidate target. Input the patch into a convolutional neural network and output an appearance feature vector. Initialize a uniform motion model and generate a motion feature vector. Form a tracking chain based on the appearance feature vector and the motion feature vector. The tracking chain includes a unique ID, tracking confidence and its historical maximum value, historical state sequence, appearance feature vector, and motion feature vector. S4: Update the tracking confidence based on real-time conditions. If the tracking confidence drops to 0, mark the corresponding tracking chain as to be deleted and store the last appearance feature vector and motion feature vector of the candidate target into the missing target feature library. If the tracking confidence does not decrease for 5 consecutive frames, the tracking chain is marked as a stable chain, the candidate target is marked as a stable target, and tracking evaluation is performed.

2. The method as described in claim 1, characterized in that, The method further includes: S5: obtaining the candidate target list of the current frame, calculating the similarity between each candidate target and the stable chain, and if the similarity is greater than 0.6, it is suspected to match; recording the number of stable chains that are suspected to match the candidate target, and if the number is 1, adding the matching stable chain; if the number is less than 1, marking it as a target to be identified; if the number is greater than 1, marking it as a multi-source target; re-analyzing the multi-source target and the target to be identified based on the dynamic hierarchical rules and performing tracking and evaluation.

3. The method as described in claim 2, characterized in that, The dynamic grading rules include: S51: For multi-source targets: extract candidate targets corresponding to all stable chains that are suspected to match in the current frame as associated targets, define multi-source targets and several associated targets as nodes, and represent each node with a feature vector, including appearance features, motion features and historical state sequence; connect any two nodes to form an edge and use the similarity between nodes as the edge weight to form a graph structure; perform message passing through multiple iterations for each edge, and after traversal, each node updates its own feature vector according to the aggregated message; S52: Concatenate the updated feature vectors of the two endpoints of any edge to form an edge feature vector. Input the edge feature vector into the edge classifier and output the probability value. After traversal, obtain the association probability of each edge and form an association probability matrix. Determine the stable chain to which each multi-source target belongs based on the association probability matrix.

4. The method as described in claim 2, characterized in that, The dynamic grading rule further includes: S53: For the target to be identified, a dynamic neighborhood region is defined with the target to be identified as the center; all stable targets and reference objects within the dynamic neighborhood region are identified, and the relative spatial relationship vector between the target to be identified and each reference object is calculated; the micro-motion trajectory of the target to be identified in the current frame and short period is extracted, and the initial motion features are calculated; the reference objects, relative spatial relationship vectors, and initial motion features of the target to be identified are uniformly encoded to form a prior vector; a behavior pattern library is generated based on historical data analysis, and the prior vector of the target to be identified is matched with the behavior pattern library to determine the most consistent motion pattern; the short period refers to a time range period of 1-5 frames.

5. The method as described in claim 1, characterized in that, Converting the target enhancement image into a binary image includes: pre-setting a sliding window and a movement step size; sliding the image on the target enhancement image according to the movement step size, centered on the coordinates of the current pixel to be processed; calculating the mean and standard deviation of the gray values ​​of all pixels within the sliding window after each movement to obtain a list of statistical values; each sliding window corresponds to a local threshold; comparing the gray value of the center pixel with the calculated local threshold; if it is greater than the local threshold, it is determined to be a target pixel and its value is set to 1 in the binary image; if it is not greater than the local threshold, it is determined to be a background pixel and its value is set to 0 in the binary image. The statistical list includes the coordinates, timestamp, mean, and standard deviation of the sliding window, and the step size must be smaller than the length of the sliding window.

6. The method as described in claim 1, characterized in that, The historical state sequence is a queue arranged in chronological order, recording the state snapshot of each frame of the tracking chain throughout the entire life cycle of the candidate target, including timestamp, target position, motion state and observation information; The tracking confidence level is used to quantify the degree of certainty that the target represented by the current tracking chain truly exists and is correctly identified; When a new candidate target passes the detection screening and is created into a tracking chain, it is assigned an initial confidence level based on the detection strength at the time of its creation. The higher the detection strength, the higher the initial confidence level.

7. The method as described in claim 1, characterized in that, The tracking confidence is updated based on real-time conditions, including: predicting the motion trajectory of the candidate target in the next frame based on the motion prediction model, and comparing the prediction result with the actual motion trajectory; if the overlap is greater than 0.7, the tracking confidence is incremented by one; if the overlap is greater than 0.4 and less than 0.7, it is marked and the original tracking confidence is maintained; if the overlap is less than 0.4, it indicates that the prediction has failed, and the tracking confidence is decremented by one.

8. The method as described in claim 4, characterized in that, The dynamic neighborhood region refers to a spatial range that is dynamically determined based on sensor performance, target kinematic characteristics, and tactical scenario, with the current position of the target to be identified as the center. By reviewing the intermediate processing results within a short period, it searches for any suspected points that have not yet been definitively identified and treats them as motion changes of the target to be identified. The suspected points are then connected in chronological order to form a micro-motion trajectory. Based on the micro-motion trajectory, the instantaneous motion state of the target is estimated using a kinematic model, and the angular displacement and total time interval from the trajectory start point to the end point are calculated. The initial motion features include the initial velocity vector, acceleration trend, and motion curvature.

9. The method as described in claim 4, characterized in that, The process of forming a prior vector includes: normalizing the relative spatial relationship vector; sequentially concatenating the normalized relative spatial relationship vectors of all reference objects into a spatial relationship feature vector; normalizing and concatenating the calculated initial motion features to obtain a motion feature vector; and directly concatenating the spatial relationship feature vector and the motion feature vector to form the final prior vector.

10. The method as described in claim 1, characterized in that, The feature parameters include area, average gray level, centroid position, and shape features. A corresponding threshold range is set for each feature parameter for filtering. If each feature meets the corresponding threshold range, it means that the filtering is passed; if any feature parameter does not meet the corresponding threshold range, it means that the filtering is not passed.

Images