Airplane target tracking method, system and computer program product under space-air background

Abstract

An aircraft target tracking method, system and computer program product in the aerospace background relates to the technical field of target tracking, solves the problems of large prediction error, low tracking accuracy and trajectory matching failure of aircraft targets caused by strong maneuverability in the aerospace background. A detection frame set is obtained; a predicted trajectory set is output based on an IMM model; in the first stage of association based on the Hyper-Hungarian trajectory association strategy, the high-resolution detection frame set and the predicted trajectory set are matched using a hypergraph model; for successfully matched trajectories, the trajectories are updated; for unsuccessfully matched detection frames, new trajectories are initialized; in the second stage of association based on the Hyper-Hungarian trajectory association strategy, the unsuccessfully matched trajectories and the low-resolution detection frame set are matched again; for successfully matched trajectories, the trajectories are updated; for unsuccessfully matched trajectories, the trajectories are marked as lost.

Description

Technical Field

[0001] This invention relates to the field of target tracking technology, specifically to aircraft target tracking methods, systems, and computer program products in the context of aerospace. Background Technology

[0002] Target tracking is one of the core tasks in the field of computer vision. It integrates complex theories from multiple disciplines and has developed rapidly with continuous technological iteration. It is widely used in key areas such as military reconnaissance, aerospace defense, and autonomous driving, and has significant practical significance and application value. Especially in aircraft target tracking tasks in aerospace contexts, the long distance, high speed, and strong maneuverability of the target, as well as complex environmental factors such as cloud interference and changes in lighting, make high-precision tracking extremely challenging.

[0003] From the perspective of existing tracking technologies, TBD (Tracking-by-Detection) is the mainstream paradigm. Among them, the ByteTrack algorithm (target detection tracking algorithm) is widely used due to its efficiency. However, existing technologies still have significant shortcomings when dealing with aerospace aircraft targets. For example, algorithms such as Sort (multi-target tracking algorithm) and Bytetrack typically use Kalman filtering with a single-mode uniform velocity model for state prediction, which is difficult to adapt to the significant acceleration changes in various motion phases of military aircraft, such as takeoff, landing, circling, and sharp turns. This ultimately leads to a large deviation between the predicted bounding box and the detected bounding box, resulting in tracking loss problems.

[0004] Existing association strategies mainly rely on IoU (Intersection over Union) for target matching. However, in aerospace scenarios, when aircraft targets are occluded, fly in cross-space, or have small target sizes or low background contrast, the IoU value will drop significantly or even become invalid, failing to accurately reflect the association between targets. This leads to frequent IDS (Identity Switching) or trajectory interruptions. Furthermore, most algorithms are based on pairwise first-order association methods, ignoring the relatively stable geometric relationships between multiple targets, resulting in a lack of effective global constraint support when local features fail.

[0005] Therefore, it is urgent to overcome the above-mentioned defects in the process of aircraft target tracking in order to improve the accuracy of aircraft target tracking in the context of air and space. Summary of the Invention

[0006] This invention aims to solve the problems of large prediction errors, low tracking accuracy, and trajectory matching failures caused by the strong maneuverability of aircraft targets in the context of aerospace.

[0007] The aircraft target tracking method under aerospace background described in this invention includes the following steps:

[0008] Step S1: Detect the aircraft video sequence in the aerospace background to be processed using the detector to obtain a set of detection boxes; the set of detection boxes includes a set of high-resolution detection boxes and a set of low-resolution detection boxes.

[0009] Step S2: Output the predicted trajectory set based on the IMM model;

[0010] Step S3: Based on the first-stage association of the Hyper-Hungarian trajectory association strategy, the hypergraph model is used to match the set of high-resolution detection boxes and the set of predicted trajectories. For successfully matched trajectories, the trajectory is updated; for unmatched detection boxes, the new trajectory is initialized.

[0011] Step S4: Based on the Hyper-Hungarian trajectory association strategy, the second-stage association is performed, and the trajectories that failed to match in step S3 are matched with the low-scoring detection box set again. For the trajectories that match successfully, the trajectories are updated, and for the trajectories that fail to match successfully, they are marked as lost.

[0012] Furthermore, in one embodiment of the present invention, in step S3, the hypergraph model is constructed, and constraints are set for the matching scheme of the hypergraph model.

[0013] Based on the constraints, the first-order similarity measure is calculated as the weight of the second-order hyperedge, and the higher-order structural consistency is calculated as the weight of the fourth-order hyperedge.

[0014] A continuous hypergraph scoring matrix is ​​constructed based on first-order similarity measurement and higher-order structural consistency, and a cost matrix is ​​defined.

[0015] The cost matrix is ​​discretized to obtain the optimal matching result, and incorrect matches are filtered out by a matching threshold.

[0016] Furthermore, in one embodiment of the present invention, the construction of the hypergraph model, specifically setting constraints on the matching scheme of the hypergraph model, includes:

[0017] The hypergraph model defines the vertex sets of trajectory nodes and detection nodes. ,in, For trajectory nodes, Define a set of second-order hyperedges for each detection box node. A single second-order hyperedge To characterize the one-to-one matching hypothesis between trajectories and detections, a fourth-order hyperedge set is defined. A single fourth-order hyperedge To characterize the higher-order structural relationship between trajectory pairs and detection pairs, a complete set of hyperedges is defined for the hypergraph. Then the hypergraph model ;

[0018] Based on the aforementioned hypergraph model, under the one-to-one constraint of satisfying the hypergraph matching objective, a matching scheme that maximizes the sum of hyperedge weights is found:

[0019] ;

[0020] The constraints of the matching scheme are as follows:

[0021] ;

[0022] ;

[0023] in, The matching matrix is ​​the matching scheme. For the number of trajectories, The number of detection frames. And the constraints apply to the matching matrix All elements are true. The weight of the second-order hyperedge. The weight of the fourth-order hyperedge. This is the maximum value.

[0024] Furthermore, in one embodiment of the present invention, the first-order similarity metric integrates IoU geometric consistency, IMM motion consistency based on Mahalanobis distance, and Euclidean distance weights for the center point.

[0025] Furthermore, in one embodiment of the present invention, the higher-order structural consistency specifically refers to:

[0026] For any two trajectories and , and the two candidate matching detection boxes and , The difference between the relative displacement vectors between trajectories and the relative displacement vectors between detection boxes is calculated. Structural compatibility is measured using an exponential decay function. The association results from the previous frame are used as anchoring constraints to encode higher-order information.

[0027] ;

[0028] in, For higher-order structural consistency, Two trajectories and The relative displacement vector between them Two detection boxes and The relative displacement vector between them Measuring the difference between two relative displacement vectors reflects the matching hypothesis. In terms of structural compatibility, This is a structural scale parameter that controls the sensitivity of structural consistency.

[0029] Furthermore, in one embodiment of the present invention, the construction of the continuous hypergraph scoring matrix involves adding the first-order similarity measure to a weighted higher-order structural consistency term, and using the matching matrix of the previous frame to transform the higher-order term into a linear scoring term.

[0030] ;

[0031] in, For higher-order structural consistency, and For any two trajectories, and , and There are two detection boxes, and , As a first-order similarity measure, For trajectory nodes, For the detection box node, This is the correlation result matrix of the previous frame.

[0032] The aircraft target tracking system under aerospace background described in this invention is based on the aircraft target tracking method under aerospace background described above, and includes the following modules:

[0033] Module S1 detects the aircraft video sequence in the aerospace background to be processed based on the detector and obtains a set of detection boxes; the set of detection boxes includes a set of high-resolution detection boxes and a set of low-resolution detection boxes.

[0034] Module S2 outputs a set of predicted trajectories based on the IMM model;

[0035] Module S3 is the first-stage association based on the Hyper-Hungarian trajectory association strategy. It uses a hypergraph model to match the set of high-resolution detection boxes and the set of predicted trajectories. For successfully matched trajectories, the trajectory is updated; for unmatched detection boxes, the new trajectory is initialized.

[0036] Module S4 is the second-stage association based on the Hyper-Hungarian trajectory association strategy. It re-matches the trajectories that failed to match in Module S3 with the set of low-scoring detection boxes. For trajectories that successfully match, the trajectory is updated, and for trajectories that fail to match, they are marked as lost.

[0037] The computer program product of the present invention includes a computer program or instructions, which, when executed by a processor, implement the aircraft target tracking method under aerospace background described above.

[0038] This invention aims to solve the problems of large prediction errors, low tracking accuracy, and trajectory matching failures caused by the high maneuverability of aircraft targets in aerospace environments. Specific beneficial effects include:

[0039] 1. The aircraft target tracking method under aerospace background described in this invention is based on the IMM model and the Hyper-Hungarian trajectory association strategy. On the basis of the hypergraph model, a transformation strategy for complex hypergraph matching is implemented, which solves the problems of large prediction error, low tracking accuracy and trajectory matching failure caused by the strong maneuverability of aircraft targets under aerospace background.

[0040] 2. The aircraft target tracking method under the aerospace background described in this invention proposes an IMM model, which can accurately capture the dynamic changes of aircraft targets in different flight stages such as cruise, turning, and diving through adaptive switching and weighted fusion of CV model and CA model. It can significantly reduce the error between the predicted position and the actual position in the X and Y directions and avoid tracking loss due to the failure of single model assumptions.

[0041] 3. The aircraft target tracking method under aerospace background described in this invention proposes a Hyper-Hungarian trajectory association strategy that not only utilizes IoU information but also introduces IMM motion consistency based on Mahalanobis distance and Euclidean distance weights for the center point, solving the problems of small-sized targets being sensitive to IoU and large detection deviations under low contrast. Furthermore, by introducing higher-order structural consistency of fourth-order hyperedges and using the relative geometric relationships between multiple targets as global constraints, even in the event of severe occlusion or intersecting flight, correct trajectory association can be maintained through the structural stability of neighboring targets, significantly reducing the number of identity switching.

[0042] 4. The aircraft target tracking method under aerospace background described in this invention transforms the complex hypergraph matching problem into a combination strategy of constructing a continuous scoring matrix and discretizing it using the Hungarian algorithm. This avoids the direct solution of NP-hard problems, significantly improving tracking robustness while maintaining a high frame rate to meet real-time requirements. Attached Figure Description

[0043] The above and / or additional methods and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0044] Figure 1 This is a flowchart of the aircraft target tracking method under aerospace background as described in Implementation Method 1;

[0045] Figure 2 This is a schematic diagram illustrating the principle and process of the IMM model described in Implementation Method 1;

[0046] Figure 3 This is a schematic diagram of the hypergraph structure in the Hyper-Hungarian trajectory association strategy described in Implementation Method 1;

[0047] Figure 4 This is the overall calculation flowchart of the Hyper-Hungarian trajectory association strategy described in Implementation Method 1. Detailed Implementation

[0048] Various embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. The embodiments described with reference to the drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0049] Implementation Method 1: This implementation method aims to solve the technical problems existing in the prior art, such as... Figure 1 As shown, a method for aircraft target tracking under aerospace background is proposed, including the following steps:

[0050] Step S1: Obtain and detect aircraft video sequences against a sky-and-air background. Specifically, input the aircraft video sequences against a sky-and-air background, perform frame-by-frame detection using the YOLOv8 detector (YOLO model v8 version), and output detection results including aircraft target category, confidence score, and bounding box coordinates. To fully utilize the detection results, the strategy of the ByteTrack algorithm is followed, with a confidence threshold... Divided into high-resolution detection box sets and low-scoring detection box set In this embodiment, the aircraft category dataset from the ImageNet2015 VID dataset is used. This dataset covers aircraft targets under different lighting conditions, backgrounds, shooting angles, and flight attitudes, providing a wealth of data samples.

[0051] Step S2: Construct the IMM (Interactive Multi-Model) model. Specifically, the core idea of ​​the IMM model is to simultaneously use multiple different motion models and embed these motion models into multiple filters used in parallel to predict the target state. Therefore, each filter corresponds to a different motion model, and the final state estimate is obtained through weighted fusion to track the target, thereby improving target tracking performance. For the aircraft target tracking task in the aforementioned aerospace context, the CV (Constant Velocity Motion Model) and CA (Constant Acceleration Motion Model) models were selected. Regarding model selection, the CV model is suitable for the aircraft during constant speed taxiing or cruise, while the CA model is suitable for the aircraft during acceleration or deceleration during turns. It predicts the state of the matched trajectory from the previous frame and outputs the predicted trajectory set and corresponding state covariance for the current frame.

[0052] In step S2, the key point is that the aircraft target in the aforementioned aerospace background has strong mobility and will have different acceleration characteristics at different stages, such as takeoff, landing, cruise, and turning. To address these acceleration characteristics, the IMM model constructed in step S2, through the characteristics of the CV and CA models, can simultaneously handle multiple motion modes of the target, improving prediction accuracy. Figure 2 As shown, it includes the following steps:

[0053] Step S2.1, Input Interaction: This step is used to transmit information between various models, especially when performing state estimation between multiple models, where information exchange between models is necessary. The goal of the interaction is to calculate the prior model probability of each model, that is, based on the state estimates and posterior probabilities of the sub-models CV and CA at the previous time step, calculate the mixed initial state and covariance, which serve as the input to each sub-model at the current time step.

[0054] The input interaction in step S2.1 includes the following steps:

[0055] Step S2.11, define the model transition probability matrix. To investigate the motion characteristics of aircraft targets against an aerospace background, a CV (CV) model and a CA (CA) model with a dimension of 2 are used. For the target from the motion model To motion model transition probability, motion model and motion model Both could be either CA or CV models;

[0056] Set the model transition probability matrix for:

[0057] ;

[0058] The set model transition probability matrix There is a 95% probability that the target will maintain the current movement mode in the next moment, and a 5% probability that it will switch modes.

[0059] Step S2.12, calculate the mixed initial probability. This probability represents the probability given that the motion model at the previous time step was... Under the condition that the motion model switches to the current time step The conditional probability is calculated using the following formula:

[0060] ;

[0061] in, For at any time Motion Model The posterior probability, This is the normalization constant;

[0062] Step S2.13, calculate the motion model after interaction. initial state and initial covariance This is the core of the IMM model. It achieves information exchange by weighted mixing of the previous time-series estimates from various models. The calculation formula is as follows:

[0063] ;

[0064] ;

[0065] in, and respectively motion model At any moment State estimation and covariance This is the matrix transpose. After completing the above calculations, each transition motion model... At any moment All initializations have been completed and initial values ​​have been obtained;

[0066] Step S2.2, Sub-model State Filtering Prediction: Each sub-model independently performs Kalman filtering prediction using the mixed initial values ​​obtained in step S2.1; for the CV model, the state vector is... ,in, The coordinates of the center of the target bounding box. For width, For high, Center coordinates The rate of change in direction, Center coordinates velocity in the direction, The width change rate, For a high rate of change, the state transition matrix Based on the assumption of uniform velocity, for the CA model, the state vector is: ,in, for acceleration in the direction, for Acceleration in direction, state transition matrix The motion model was constructed based on the assumption of uniform acceleration; subsequently, the motion models were calculated respectively. At any moment State estimation Covariance The calculation formula is:

[0067] ;

[0068] ;

[0069] in, For motion model The state transition matrix represents the state transition from time 1 to 2. At the time The state change, The process noise covariance matrix describes the motion noise of the target.

[0070] Step S2.3, Measurement and Model State Update: Calculate the innovation residual based on the observed values ​​of the detection boxes in the current frame. Innovative covariance matrix Kalman gain Estimating the state Covariance And calculate the likelihood value of each sub-model. Then, Bayes' theorem is used to update the posterior probabilities of each sub-model. This enables adaptive perception of the target's current motion pattern, and the calculation formula is as follows:

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] ;

[0076] ;

[0077] ;

[0078] in, For the current moment Motion Model The observation vector, For the current moment Motion Model The observation matrix For the current moment Motion Model The observation noise covariance matrix, observation vector Dimensions Given the identity matrix, the estimated state of each independent sub-model is obtained after the above calculations are performed. Covariance ;

[0079] Step S2.4, Model State Fusion: Based on the updated model probabilities, the state estimates and covariances of each sub-model are weighted and fused to obtain the final estimated state. and the final covariance This is used for subsequent trajectory association, and the calculation formula is:

[0080] ;

[0081] Step S3 involves performing the first-stage association using the Hyper-Hungarian trajectory association strategy. Specifically, a hypergraph scoring matrix is ​​constructed, incorporating first-order similarity metrics and high-order structural consistency. The predicted trajectory set from Step S2 is then compared with the high-scoring detection boxes from Step S1. Perform a match and update the status of the successfully matched trajectories;

[0082] In step S3, the Hungarian algorithm, which is part of the data association step in the existing ByteTrack algorithm, is first used to perform association matching between the detection boxes of the preceding and following frames and the existing trajectory based on IoU, including high-resolution detection boxes. Initial association with the trajectory, and the set of low-resolution detection boxes. Secondary association with unmatched trajectories is performed, but matching pairs with trajectories are detected through IoU thresholding to maintain trajectory continuity. The IoU metric method has low computational complexity, scale invariance unaffected by target size, and effectively reflects the spatial overlap of bounding boxes.

[0083] However, in practical applications, it has been found that when the target is severely occluded, the intersection area may be close to 0, which makes it impossible to reflect the positional correlation between the two boxes and results in matching failure. It is extremely sensitive to the positional deviation of small targets, because the detection box area is small, and even a slight positional shift will cause the IoU value to drop significantly, thus resulting in matching errors.

[0084] To address the aforementioned technical issues, a Hyper-Hungarian trajectory association strategy based on hypergraph matching was used to replace the existing Hungarian algorithm. The construction of the Hyper-Hungarian trajectory association strategy is as follows: Figure 4 As shown, it includes the following steps:

[0085] Step S3.1, construct as follows Figure 3 The "trajectory-detection" hypergraph model shown, in which, , , , , , , , and These are the different vertices of the hypergraph structure, and all the different vertices together constitute the vertex set of the hypergraph. , , , and These are the different hyperedges of the hypergraph, and all the different hyperedges together constitute the hyperedge set of the hypergraph. The vertex set of the trajectory node and the detection node is also defined. ,in, For trajectory nodes, Define the detection box nodes; define a set of second-order hyperedges. A single second-order hyperedge Connect a trajectory and a detection box to represent the one-to-one matching hypothesis of "trajectory-detection"; define a set of fourth-order hyperedges. A single fourth-order hyperedge Connecting two trajectories and two detection boxes represents the structural compatibility between two matching hypotheses, characterizes the higher-order structural relationship of the "trajectory pair-detection pair", and defines the complete set of hyperedges of the hypergraph. Thus, the hypergraph model exists. Based on the aforementioned hypergraph model, and under the one-to-one constraint of satisfying the hypergraph matching objective, a matching scheme that maximizes the sum of hyperedge weights is found, as shown in the following formula:

[0086] ;

[0087] The constraints of the matching scheme are as follows:

[0088] ;

[0089] ;

[0090] in, Let be the matching matrix of the matching scheme. For the number of trajectories, The number of detection frames. And the constraints apply to the matrix All elements are true;

[0091] Specifically, The weights of the second-order hyperedges reflect the trajectory. With detection box Local matching similarity between them The weight of the fourth-order hyperedge reflects the degree of structural consistency between the two pairs of matching hypotheses. Furthermore, the matching problem, which includes quadratic terms, is a quadratic distribution problem and is an NP (nondeterministic polynomial) hard problem. The computational complexity of directly solving it increases exponentially with the number of vertices.

[0092] The above equation represents the specific matching scheme for the hypergraph model. Then, the matching scheme shown in the above equation is solved to obtain the matching matrix. Then, let this matching matrix... Replace the matching matrix in the data association step of the existing ByteTrack algorithm. .

[0093] However, directly solving the above matching scheme involves quadratic terms, which fall under the category of discrete optimization problems involving quadratic distributions. The computational complexity increases exponentially with the number of vertices, necessitating alternative methods. A continuous solution strategy based on temporal propagation is employed. This strategy uses the matching results of the previous frame as anchoring constraints on the temporal relationships of higher-order terms, transforming the original discrete optimization problem into the construction of a continuous scoring matrix. This abandons discrete optimization, directly constructing the hypergraph scoring matrix based on hyperedge weights. Then, the Hungarian algorithm is executed on this continuous scoring matrix to achieve efficient discretization and obtain the final matching matrix. Specifically:

[0094] Step S3.2 mainly calculates the first-order similarity measure. As weights for second-order hyperedges, existing first-order similarity measures include IoU geometric consistency. However, applying it to calculate the first-order similarity metric in this embodiment has the problem of being sensitive to positional deviations of small-sized targets, leading to a significant drop in the IoU value. To overcome this technical problem, this embodiment integrates IMM motion consistency based on Mahalanobis distance on the basis of IoU geometric consistency. The first-order similarity metric was calculated, but based on the results, it was found that although it could overcome the aforementioned technical problems, it still exhibited the technical issue of drastic scale changes in aircraft targets due to variations in flight altitude and shooting angle. Therefore, the Euclidean distance weight of the center point was incorporated again. Compensation is performed to comprehensively characterize the degree of matching between the trajectory and the detection box in terms of geometric overlap, motion state, and spatial position. Specifically:

[0095] The IoU geometric consistency mentioned above retains the traditional area overlap metric, and the calculation formula is as follows:

[0096] ;

[0097] in, The predicted bounding box of the predicted trajectory obtained based on the IMM model constructed in step S2 in the current frame. The IoU value is the boundary of the detection box output by the detector. The IoU value ranges from [0,1]. The larger the value, the higher the degree of spatial overlap.

[0098] The motion consistency of the IMM model based on Mahalanobis distance utilizes the innovative covariance matrix output by the IMM model constructed in step S2. Calculate the Mahalanobis distance and convert it into similarity. This step measures the degree of agreement between the detection result and the trajectory motion model by calculating the statistical distance between the current detection box observation and the trajectory prediction state, i.e., the final posterior state estimate output by the IMM model. The calculation formula is as follows:

[0099] ;

[0100] in, For detection box The observation vector contains the center coordinates and size information of the bounding box. For trajectory The state prediction vector after fusion by the IMM model The observation matrix;

[0101] Considering that aircraft targets in aerospace scenarios may undergo drastic scale changes due to variations in flight altitude and shooting angle, a compensation term based on the Euclidean distance weighted by the center point is introduced to calculate the Euclidean distance between the center points of the predicted bounding box and the detected bounding box. This distance is processed using an exponential decay function, and the calculation formula is as follows:

[0102] ;

[0103] in, For trajectory Predict the center point coordinates of the bounding box. For detection box The coordinates of the center point, , is a distance scale parameter used to control the sensitivity to distance decay;

[0104] Combining the above three components, the trajectory is obtained. With detection box The first-order similarity measure between them is given by the following formula:

[0105] ;

[0106] in, These are empirical weighting parameters used to balance the contributions of the three components to the final similarity, and are subject to constraints. ;

[0107] Step S3.3, calculate the consistency of higher-order structures As the weight of a fourth-order hyperedge, for any two trajectories and , and the two candidate matching detection boxes and , The difference between the "relative displacement vector between trajectories" and the "relative displacement vector between detection boxes" is calculated. Structural compatibility is measured using an exponential decay function. The association results from the previous frame are used as anchoring constraints to encode higher-order information. The calculation formula is as follows:

[0108] ;

[0109] Among them, higher-order structure consistency The physical meaning lies in the assumed trajectory Matching detection box trajectory Matching detection box , Two trajectories and The relative displacement vectors between predicted positions characterize the spatial relative relationship of trajectory pairs in the current frame. Two detection boxes and The relative displacement vectors between them characterize the spatial relative relationship of the detection pairs. Measuring the difference between two relative displacement vectors reflects the matching hypothesis. In terms of structural compatibility, These are structural scale parameters that control the sensitivity to structural consistency.

[0110] Step S3.4: Construct a continuous hypergraph rating matrix. To avoid the NP-hard problem of solving the hypergraph model described in step S3.1, a time-transfer relaxation strategy is used to construct the continuous hypergraph rating matrix. The first-order similarity metric is added to the weighted higher-order structural consistency term, and the matching matrix of the previous frame is used. Transform higher-order terms into linear scoring terms to construct a continuous hypergraph scoring matrix. The calculation formula is:

[0111] ;

[0112] in, The correlation result matrix of the previous frame is obtained by solving the Hungarian algorithm at the previous time step. These are higher-order information weighting coefficients used to control the strength of the influence of structural constraints on the final score. The rating matrix is ​​for a continuous hypergraph.

[0113] Subsequently, the cost matrix is ​​defined. Its elements To track Detection box The cost of establishing a matching relationship is related to the continuous hypergraph scoring matrix. Transform into a cost matrix This transforms the problem of maximizing similarity into the problem of minimizing cost, making it conform to the solution form of the Hungarian algorithm.

[0114] Step S3.5: Apply the Hungarian algorithm to the cost matrix. Discretization is performed to obtain the optimal matching result that satisfies the one-to-one constraint, and incorrect matches are filtered out by a matching threshold. Specifically, in the cost matrix... The Hungarian algorithm is then executed to find the optimal bipartite matching that satisfies the one-to-one constraint, which can be formalized as follows:

[0115] ;

[0116] in, For a moment The matching matrix, For trajectory With detection box Establish a matching relationship in the current frame. For the number of trajectories, The number of detection boxes is represented by the matching matrix output by the Hungarian algorithm. Although the one-to-one constraint is satisfied, it may contain spurious matches with too low similarity, and the matching matrix will then be analyzed. Threshold filtering is performed, and the calculation formula is as follows:

[0117] ;

[0118] in, These are the elements of the final matching matrix after filtering. The matching threshold indicates that a match is considered reliable only when the overall similarity between the trajectory and the detection box exceeds the threshold; otherwise, even if the Hungarian algorithm pairs them, they should be considered invalid matches and discarded.

[0119] Step S4: Match the unmatched trajectories from step S3 with the low-scoring detection boxes from step S1. Perform matching again using IoU or Hyper-Hungarian trajectory association strategies to uncover real targets that are occluded or blurred in low-confidence detection boxes;

[0120] Step S5, Trajectory Update and Management. Specifically, based on the association results of step S3, for successfully matched trajectories, the current frame detection box is used to update the state and model probability through the measurement update step of the IMM model; for unmatched detection boxes, if they are high-resolution boxes, they are initialized as new trajectories; based on the association results of step S4, for successfully matched trajectories, the current frame detection box is used to update the state and model probability through the measurement update step of the IMM model; for unmatched trajectories, they are marked as lost, and if they are unmatched for 30 consecutive frames, they are deleted, and then the final aircraft target tracking result is output.

[0121] To better illustrate the aircraft target tracking method under aerospace background described in this embodiment, the following examples provide a detailed description:

[0122] The tracking results of the existing ByteTrack algorithm and the method described in this embodiment are visually compared. For aircraft target tracking in video sequences, the tracking results of each frame are extracted, and the frame number is marked in the upper left corner of each frame's tracking result.

[0123] When using the existing ByteTrack algorithm for target tracking, an ID switch occurred in frame 15. Target 4, originally identified as IDs 1, 2, 3, and 4, underwent two ID switches, changing its ID to 6 and losing its original ID. In frame 27, the original target 4, now target 6, experienced another ID switch, changing its ID to 7. During subsequent movement, the aircraft's flight angle and direction continuously changed until severe occlusion occurred in frame 199. At this point, the original target 4, now target 7, experienced an ID switch, changing its ID to 8. However, the method described in this implementation maintains continuous and stable tracking without any ID switches or trajectory loss.

[0124] Therefore, the method described in this embodiment solves the problems of large prediction errors, low tracking accuracy, and trajectory matching failures caused by the strong maneuverability of aircraft targets in the prior art by using the IMM model and the Hyper-Hungarian trajectory association strategy, as well as the combination strategy of transforming the complex hypergraph matching problem into a combination strategy of continuous scoring matrix construction and discretization solution of the Hungarian algorithm. The cooperation and interaction between the various strategies solve the problems of large prediction errors, low tracking accuracy, and trajectory matching failures caused by the strong maneuverability of aircraft targets in the prior art.

[0125] Implementation Method 2: The aircraft target tracking system under aerospace background described in this implementation method is based on the aircraft target tracking method under aerospace background described in Implementation Method 1, and includes the following modules:

[0126] Module S1 detects the aircraft video sequence in the aerospace background to be processed based on the detector and obtains a set of detection boxes; the set of detection boxes includes a set of high-resolution detection boxes and a set of low-resolution detection boxes.

[0127] Module S2, based on the IMM model multi-mode prediction module, outputs a set of predicted trajectories;

[0128] Module S3 is the first-stage association based on the Hyper-Hungarian trajectory association strategy. It uses a hypergraph model to match the set of high-resolution detection boxes and the set of predicted trajectories. For successfully matched trajectories, the trajectory is updated; for unmatched detection boxes, the new trajectory is initialized.

[0129] Module S4 is the second-stage association based on the Hyper-Hungarian trajectory association strategy. It re-matches the trajectories that failed to match in Module S3 with the set of low-scoring detection boxes. For trajectories that successfully match, the trajectory is updated, and for trajectories that fail to match, they are marked as lost.

[0130] Implementation Method 3: A computer program product described in this implementation method includes a computer program or instructions, which, when executed by a processor, implement the aircraft target tracking method in the context of aerospace background as described in Implementation Method 1.

[0131] The above provides a detailed description of the aircraft target tracking method, system, and computer program product under aerospace background proposed in this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for tracking aircraft targets in aerospace context, characterized in that, Includes the following steps: Step S1: Detect the aircraft video sequence in the aerospace background to be processed using the detector to obtain a set of detection boxes; the set of detection boxes includes a set of high-resolution detection boxes and a set of low-resolution detection boxes. Step S2: Output the predicted trajectory set based on the IMM model; Step S3: Based on the first-stage association of the Hyper-Hungarian trajectory association strategy, the hypergraph model is used to match the set of high-resolution detection boxes and the set of predicted trajectories. For successfully matched trajectories, the trajectory is updated; for unmatched detection boxes, the new trajectory is initialized. Step S4: Based on the Hyper-Hungarian trajectory association strategy, the second-stage association is performed, and the trajectories that failed to match in step S3 are matched with the low-scoring detection box set again. For the trajectories that match successfully, the trajectory is updated, and for the trajectories that fail to match successfully, they are marked as lost. The IMM model includes the CV model and the CA model; In step S3, the hypergraph model is constructed, and constraints are set for the matching scheme of the hypergraph model. Based on the constraints, the first-order similarity measure is calculated as the weight of the second-order hyperedge, and the higher-order structural consistency is calculated as the weight of the fourth-order hyperedge. A continuous hypergraph scoring matrix is ​​constructed based on first-order similarity measurement and higher-order structural consistency, and a cost matrix is ​​defined. The cost matrix is ​​discretized to obtain the optimal matching result, and incorrect matches are filtered out by a matching threshold. The construction of the continuous hypergraph scoring matrix involves adding the first-order similarity measure to a weighted higher-order structural consistency term, and then using the matching matrix from the previous frame to transform the higher-order term into a linear scoring term. ； in, For higher-order structural consistency, and For any two trajectories, and , and There are two detection boxes, and , As a first-order similarity measure, For trajectory nodes, For the detection box node, This is the correlation result matrix of the previous frame.

2. The aircraft target tracking method under aerospace background according to claim 1, characterized in that, The construction of the hypergraph model, and the setting of constraints on the matching scheme of the hypergraph model, specifically includes: The hypergraph model defines the vertex sets of trajectory nodes and detection nodes. ,in, For trajectory nodes, Define a set of second-order hyperedges for each detection box node. A single second-order hyperedge To characterize the one-to-one matching hypothesis between trajectories and detections, a fourth-order hyperedge set is defined. A single fourth-order hyperedge To characterize the higher-order structural relationship between trajectory pairs and detection pairs, a complete set of hyperedges is defined for the hypergraph. Then the hypergraph model ; Based on the aforementioned hypergraph model, under the one-to-one constraint of satisfying the hypergraph matching objective, a matching scheme that maximizes the sum of hyperedge weights is found: ； The constraints of the matching scheme are as follows: ； ； in, The matching matrix is ​​the matching scheme. For the number of trajectories, The number of detection frames. And the constraints apply to the matching matrix All elements are true. The weight of the second-order hyperedge. The weight of the fourth-order hyperedge. This is the maximum value.

3. The aircraft target tracking method under aerospace background according to claim 1, characterized in that, The first-order similarity metric integrates IoU geometric consistency, IMM motion consistency based on Mahalanobis distance, and Euclidean distance weights for the center point.

4. The aircraft target tracking method under aerospace background according to claim 1, characterized in that, The aforementioned higher-order structural consistency specifically refers to: For any two trajectories and , and the two candidate matching detection boxes and , The difference between the relative displacement vectors between trajectories and the relative displacement vectors between detection boxes is calculated. Structural compatibility is measured using an exponential decay function. The association results from the previous frame are used as anchoring constraints to encode higher-order information. ； in, For higher-order structural consistency, Two trajectories and The relative displacement vector between them Two detection boxes and The relative displacement vector between them Measuring the difference between two relative displacement vectors reflects the matching hypothesis. In terms of structural compatibility, This is a structural scale parameter that controls the sensitivity of structural consistency.

5. An aircraft target tracking system under aerospace background, wherein the system is implemented based on the aircraft target tracking method under aerospace background as described in claim 1, characterized in that, Includes the following modules: Module S1 detects the aircraft video sequence in the aerospace background to be processed based on the detector and obtains a set of detection boxes; the set of detection boxes includes a set of high-resolution detection boxes and a set of low-resolution detection boxes. Module S2 outputs a set of predicted trajectories based on the IMM model; Module S3 is the first-stage association based on the Hyper-Hungarian trajectory association strategy. It uses a hypergraph model to match the set of high-resolution detection boxes and the set of predicted trajectories. For successfully matched trajectories, the trajectory is updated; for unmatched detection boxes, the new trajectory is initialized. Module S4 is the second-stage association based on the Hyper-Hungarian trajectory association strategy. It re-matches the trajectories that failed to match in Module S3 with the set of low-scoring detection boxes. For trajectories that successfully match, the trajectory is updated, and for trajectories that fail to match, they are marked as lost.

6. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the aircraft target tracking method in the aerospace background as described in any one of claims 1 to 4.

Images