Airport multi-target tracking method and system based on context perception and depth consistency constraint

Abstract

The application relates to the technical field of computer vision, and particularly discloses an airport multi-target tracking method and system based on context perception and depth consistency constraint. First, a multi-task joint strategy of target detection and self-supervised monocular depth estimation is constructed, and the geometric structure perception ability for an airport scene is enhanced through shared feature representation; second, a context perception matching module is designed, a neighborhood context and a historical consistency constraint are introduced, and the correlation accuracy of similar targets is improved; finally, a depth consistency discrimination module is designed, monocular depth geometric features are used to assist target identity discrimination, and target ID switching and mis-correlation in a shielding scene are effectively reduced. The method effectively improves the track stability and identity maintenance ability in a complex airport environment, and provides a more robust solution for multi-target tracking in an airport monitoring scene.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to an airport multi-target tracking method and system based on context awareness and depth consistency constraints. Background Technology

[0002] Multi-Object Tracking (MOT) is a core task in computer vision, aiming to continuously locate multiple objects in a video sequence while maintaining their consistent identities. In recent years, MOT has made significant progress in fields such as intelligent transportation, public safety, and autonomous driving. Under the tracking-by-detection paradigm, a series of efficient and practical methods have emerged, such as ByteTrack, which recovers occluded targets by incorporating low-confidence detection boxes into the association process, and BoT-SORT, which further enhances the robustness of association based on motion modeling and appearance matching. Although these methods have made significant progress on public datasets, they often fail to meet practical needs when directly applied to airport surveillance scenarios. Airport ground environments are complex, containing both large targets such as aircraft and vehicles, and small targets such as pedestrians and baggage carts, with significant differences in target scale. Simultaneously, frequent occlusion of smaller targets by larger ones, and the overlap of multiple similar-looking targets in the same frame, often result in incomplete or blurry detection results, easily leading to identity switching and trajectory interruptions. These characteristics make airport scenarios a typical challenge in MOT applications.

[0003] Despite researchers proposing various methods to improve MOT performance, significant shortcomings remain under complex occlusion conditions. Motion model-based methods, such as SORT, are widely used due to their simple structure and high efficiency, but their reliance on linear motion assumptions leads to rapid accumulation of prediction errors under prolonged occlusion or nonlinear motion conditions, making trajectory recovery difficult. DeepSORT further enhances this by introducing appearance features to improve association accuracy; however, its robustness remains limited in airport scenarios with highly similar appearances or significant differences in target scale. StrongSORT, a significant upgrade to the SORT series, is based on the tracking-by-detection paradigm. Building upon motion and appearance feature matching, it effectively mitigates the ID switching problem caused by occlusion by introducing NSA nonlinear Kalman filtering, MC motion compensation, and improved cost matrix and trajectory management mechanisms, demonstrating superior tracking performance in complex scenarios. However, this method still suffers from appearance feature distortion under heavy occlusion conditions, its motion prediction has limited adaptability to frequently changing occlusion relationships in airports, and it still faces a certain risk of mismatch in densely packed similar targets. OC-SORT further optimizes the Kalman filter modeling and outperforms traditional methods in occluded scenarios. However, since it mainly relies on motion and observation information and lacks explicit modeling of context structure and geometric constraints, it still has performance bottlenecks under dense occlusion conditions.

[0004] In addition to the improvements based on the SORT series mentioned above, recent studies have also attempted to enhance the robustness of online multi-target tracking by explicitly modeling the occlusion structure. For example, MSPNet prioritizes visible targets and gradually recovers occluded targets through motion-guided spatial awareness and hierarchical association strategies; SMILEtrack strengthens the ability to distinguish between similar targets under occlusion conditions by introducing a similarity learning mechanism from Siamese networks. These methods alleviate the trajectory interruption problem caused by occlusion to some extent, but in complex environments such as airport ground with large differences in target scale, dynamic changes in occlusion morphology, and a high density of similar targets, they are still susceptible to appearance degradation and lack of geometric constraints.

[0005] On the other hand, end-to-end methods for joint detection and tracking (such as FairMOT, MOTR, and Transformer-based methods) have achieved high accuracy on public datasets by sharing feature representations. For example, TrackForme maintains target identity consistency in the temporal dimension through a tracking-by-attention mechanism, while MeMOTR further introduces long-term memory enhancement to improve reconnection capability after occlusion. However, these methods are usually heavily reliant on large-scale, high-quality training data. In real-world, complex scenarios such as airport surveillance, the stability of association remains difficult to guarantee due to diverse occlusion patterns, uneven target scale distribution, and significant domain differences. Summary of the Invention

[0006] To address the shortcomings and problems of current multi-target tracking methods in real-world, complex scenarios such as airports, this invention provides a multi-target tracking method and system for airports based on context awareness and depth consistency constraints.

[0007] The airport multi-target tracking method based on context awareness and depth consistency constraints provided by this invention includes the following steps: Step 1: Input the input image into the feature extraction backbone network for feature encoding. Then, input the encoded features in parallel into the self-supervised training monocular depth estimation network and the detector based on the Transformer architecture to generate depth maps and high-precision target detection boxes. Step 2: Obtain the set of detected targets and the set of historical trajectories in the current frame, and use a Kalman filter to predict the motion state of the trajectory; Step 3: Integrate the target's appearance features, motion priors, contextual distribution differences, and historical trajectories, and perform preliminary matching and association; Step 4: Perform deep consistency judgment on the targets that failed the initial matching and association in Step 3 to complete the secondary matching and association; Step 5: Update the trajectory status and output the multi-target tracking results based on the preliminary matching and secondary matching results.

[0008] The aforementioned airport multi-target tracking method based on context awareness and depth consistency constraints, in step three, calculates the neighborhood context distance and historical consistency distance, and models them in a unified manner with appearance distance and telekinetic prior, constructing a context-aware matching cost function to achieve preliminary matching association.

[0009] The aforementioned airport multi-target tracking method based on context awareness and depth consistency constraints includes the following steps in its neighborhood context distance calculation method: (1) Given the set of targets detected in a frame as Each target Having feature vectors and spatial location To construct a spatial neighborhood for each target

[0010] In the formula: Indicate target and Euclidean distance, It is the neighborhood search radius, which controls the size of the context region; (2) Calculate the feature difference between the current target and each target in the domain. and weighting coefficients ;

[0011] In the formula: This represents the inner product of two eigenvectors. The distance is a Euclidean norm; the larger the distance, the less similar the appearances of the two targets.

[0012] In the formula, It indicates the degree to which the control spatial distance affects the neighborhood weight, reflecting the prior assumption of spatial locality; (3) Calculate the neighborhood context distance of the target. ,

[0013]

[0014] Where: context distance For the goal The larger the value of the local feature difference measure, the more heterogeneous and complex the neighborhood is.

[0015] The above-mentioned airport multi-target tracking method based on context awareness and depth consistency constraints includes the following steps in its historical consistency distance calculation method: (1) Assume detection The appearance feature vector is trajectory In the current frame Recently One effective appearance feature is Calculate its historical appearance mean and fluctuation.

[0016] In the formula: >0 indicates a numerically stable term; (2) Calculate the normalized appearance deviation between the detection and the trajectory history template.

[0017] (3) Based on the missed detection ratio within a time window of length w Calculate the trajectory reliability factor;

[0018] In the formula: >0 The rate at which control reliability decreases with the proportion of missed detections; (4) Define the time decay factor.

[0019] In the formula: This represents the number of frames since the last valid observation of the trajectory. Control the decay rate; (5) Calculate the historical consistency distance based on appearance deviation, trajectory reliability factor, and defined time decay factor.

[0020] In the formula, For the Sigmoid function, Controlling the steepness of the curve, This is the deviation threshold. Smaller... A value close to 1 indicates a high degree of consistency between the detection and the trajectory in historical patterns, while a value close to 1 indicates a significant difference.

[0021] The aforementioned airport multi-target tracking method based on context awareness and depth consistency constraints uses the following initial matching cost function for context awareness:

[0022] In the formula: The appearance distance based on ReID features measures the difference between the detection and the trajectory in the visual feature space. To reflect the consistency of motion state based on the IoU distance between the predicted position and the detection box; The neighborhood context distance quantifies the degree of semantic feature difference between the candidate target and its spatial neighborhood; The historical consistency distance characterizes the stability differences between candidate targets and trajectories in long-term appearance patterns.

[0023] The above-mentioned airport multi-target tracking method based on context awareness and depth consistency constraints, in step four, performs depth consistency discrimination on targets that fail to be initially matched and associated by calculating the depth consistency score between candidate detections and historical trajectories in the depth dimension, constructing a quadratic cost function, and re-matching the detections and trajectories that failed to be initially associated based on the quadratic association results.

[0024] The above-mentioned airport multi-target tracking method based on context awareness and depth consistency constraints uses the following formula for calculating the depth consistency score:

[0025] In the formula: For candidate detection and trajectory The depth consistency score between them; Candidate Detection The corresponding depth value; Candidate Detection The corresponding depth value; This is the preset depth smoothing coefficient.

[0026] The aforementioned airport multi-target tracking method based on context awareness and depth consistency constraints uses the following quadratic cost function:

[0027] In the formula: The cost of secondary matching after introducing deep consistency constraints; The initial matching cost obtained from the context-aware matching cost function; For candidate detection and trajectory The depth consistency score between them.

[0028] This invention also provides an airport multi-target tracking system based on context awareness and depth consistency constraints, comprising: The feature extraction module includes a shared ResNet18 feature extraction backbone network for extracting features from the input image, and a monocular depth estimation network Lite-Mono and a Transformer-based detector RT-DETR connected to the output of the backbone network, which output depth maps and target detection boxes in parallel. The motion state prediction module, with a built-in Kalman filter, is used to predict the motion state of historical trajectories and obtain the predicted trajectory. The multimodal matching module includes a context-aware matching module and a depth consistency discrimination module. The context-aware matching module is used to fuse multimodal information and construct a comprehensive cost function for preliminary association. The depth consistency discrimination module is used to calculate a depth consistency score based on the depth estimates of candidate detections and historical trajectories when the context-aware matching module fails to match, and to perform secondary discrimination and association. The target tracking output module is used to update the trajectory status and output multi-target tracking results based on the association results of the context-aware matching module and the depth consistency discrimination module.

[0029] The aforementioned airport multi-target tracking system based on context awareness and depth consistency constraints includes multimodal information such as appearance features, motion information, neighborhood feature distribution differences, and historical stability information. Compared with existing technologies, the beneficial effects of this invention are as follows: This invention's method effectively improves the trajectory stability and identity preservation capabilities of multi-target tracking in airport monitoring scenarios through multi-task joint modeling, context-aware association, and deep geometric consistency constraints, providing a feasible and robust solution to multi-target tracking problems in complex traffic and security scenarios. The method constructs a multi-task joint training framework, enhancing the model's perception of the complex geometric structure of airport scenes through shared feature representations, providing effective spatial constraints for target association, and improving trajectory continuity and identity preservation capabilities overall. Secondly, addressing the problem of mis-association of similar targets in dense scenes, a context-aware matching (CAM) module is designed. This module explicitly introduces neighborhood context information and historical consistency constraints during the association stage, effectively mitigating association ambiguities caused by occlusion and target interaction. Furthermore, to enhance the model's discriminative ability in occluded and blurred appearance scenarios, a proposed Depth-Consistency Discriminator (DCD) module is used. This module utilizes geometric consistency information provided by monocular depth estimation to assist in identity discrimination, significantly reducing the ID switching rate in complex scenes and providing a valuable supplement to traditional appearance-based methods. Experimental results on a self-built airport multi-object tracking dataset demonstrate that the proposed method achieves stable and competitive performance improvements on key metrics such as HOTA, AssA, and IDF1. Comparative experiments with several representative methods verify the robustness and generalization ability of this method in complex airport environments. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the multi-target tracking system architecture of the present invention; Figure 2 This is a schematic diagram of the depth mean and consistency detection module architecture of the present invention; Figure 3 This is the loss curve for the joint training of target detection and depth estimation in this invention; Figure 4 This is for the gradual optimization of depth estimation results during the training process of the system of this invention; Figure 5 Decompose the tracking performance curves for each type of target; Figure 6 Visualize the results of different methods for comparison. Detailed Implementation

[0031] Multi-Object Tracking (MOT), a key task in computer vision, aims to continuously detect and track multiple objects in a video sequence, assigning each object a unique identifier and maintaining its consistency over time. In recent years, with the rapid development of deep learning technology, MOT has made significant progress, with various algorithms emerging to address numerous challenges in complex scenes, such as object occlusion, appearance similarity, object disappearance and reappearance, and detection errors. These include: (1) A two-stage detection-tracking method These methods explicitly divide the MOT task into two independent stages: object detection and data association. First, the target location is obtained through a detection model, and then cross-frame matching is achieved through an association strategy. SORT (Simple Online and Realtime Tracking) is a classic example of this paradigm. It uses Kalman filtering to predict the target's position in the next frame and combines it with the Hungarian algorithm to match the predicted bounding box with the detected bounding box based on the Intersection over Union (IOU). This algorithm has a simple structure and strong real-time performance, but it relies solely on motion information and is prone to ID switching in scenes with target occlusion or similar appearances.

[0032] DeepSORT introduces deep appearance features on top of SORT, extracting color, texture, and other features of the target through networks such as ResNet-50 and fusing them with motion features. The data association stage employs a three-level strategy: first, motion matching is performed using IOU to filter invalid detection boxes; then, appearance feature cosine similarity is calculated to complete appearance matching; finally, cascaded matching is used for targets with long-term occlusion, prioritizing matching based on trajectory age to reduce the probability of erroneous associations, significantly improving tracking accuracy in complex scenes.

[0033] Building upon this foundation, researchers have further improved motion modeling and association mechanisms. OC-SORT enhances trajectory stability under occlusion conditions by optimizing Kalman filter prediction and data association strategies. CenterTrack models target tracking as a keypoint detection and displacement prediction problem, achieving continuous target tracking within a single-stage framework, providing a method different from traditional data association for the detection-tracking paradigm. Furthermore, TradeS utilizes historical trajectory information to guide target detection backwards, enhancing temporal consistency to some extent, but its overall performance still highly depends on the quality of the detection results.

[0034] Besides association strategies based on local matching, some studies have attempted to solve the multi-object association problem from a global optimization perspective. Early work used minimum cost flow and graph optimization methods to jointly model targets across multiple frames, laying the theoretical foundation for subsequent data association methods based on graph neural networks. Building on this, GCNNMatch explicitly models the relationships between targets using graph convolutional networks, achieving better association results in complex interaction and occlusion scenarios.

[0035] (2) Joint learning method for detection and tracking These methods break down the traditional two-stage boundary, performing detection and tracking tasks simultaneously within a unified framework. They reduce information loss and improve efficiency through shared feature extraction. JDE (Joint Detection and Embedding) is a pioneer in this type of method. Based on YOLOv3, it redesigns the coupled prediction head, simultaneously performing object detection and re-identification (Re-ID) feature extraction in a single network. It directly extracts embedding features from shared features through 1×1 convolutional layers, achieving joint learning of detection and embedding. However, JDE has limitations in handling the inherent differences between the three tasks: detection, classification, and Re-ID.

[0036] FairMOT, based on the CenterNet Anchor-Free architecture, achieves "fair" sharing of detection and Re-ID features, meaning the same feature serves both tasks simultaneously. By optimizing its network architecture and training strategy, it significantly improves tracking accuracy while maintaining real-time performance, demonstrating outstanding performance on multiple public datasets and setting a benchmark for joint learning methods.

[0037] MOTR transforms the tracking problem into a sequence-to-sequence prediction task, using a Transformer structure to model the spatiotemporal information of targets and achieve end-to-end multi-target tracking. MOTIP focuses on modeling target interaction behavior, optimizing tracking performance in interactive scenarios by analyzing information such as relative positions and velocities between targets. Tracktor constructs trajectories based on detection results and performs analysis and processing. QD-Track improves detection accuracy and small target detection capabilities by refining the detection network, providing a more reliable foundation for tracking. MOTDT integrates target dynamics and appearance information to build complex models to cope with changes in target state, all of which expand the solution approach of MOT within the joint learning framework.

[0038] Based on this, this paper proposes a multi-target tracking framework that integrates monocular depth estimation and target detection, aiming to improve the accuracy of target association and trajectory stability in complex airport scenarios. The invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0039] This invention provides a method and system for multi-target tracking at airports based on context awareness and depth consistency constraints, such as... Figure 1 As shown, the overall framework mainly consists of the following parts: feature extraction module, joint depth and detection module, multimodal matching module, and target tracking output module.

[0040] First, the input image is feature-encoded through a shared feature extraction network, ResNet18. The extracted features are then fed in parallel into two sub-networks: one is Lite-Mono, a self-supervised monocular depth estimation network, which generates a depth map through image reconstruction; the other is RT-DETR, a detector based on the Transformer architecture, which generates high-precision object detection boxes.

[0041] Subsequently, detection and depth information jointly participate in the target tracking stage. The system first uses a Kalman filter (KF) to predict the target's motion state; then, through the context-aware matching (CAM) module, it fuses multimodal information such as appearance features (ReID), IoU, and context distribution differences to perform preliminary association.

[0042] The Context-Aware Matching Module (CAM) models the contextual features of targets in their spatial, temporal, and adjacent semantic environments, assisting in the construction of more discriminative matching representations. This module can be seamlessly embedded into a two-stage matching process. In the first stage, it integrates structural cues to improve initial matching accuracy, and in the second stage, it assists in recovering potential matching pairs of targets after occlusion. This module can improve the robustness of data association in multi-target tracking, especially in complex scenarios where targets are highly similar, occlusion is frequent, or detection errors are large. It effectively addresses the problem that traditional multi-target matching methods often rely on appearance features or temporal trajectory information in a single frame image, but neglect the contextual associations between targets in terms of spatial distribution, semantic adjacency, and motion consistency.

[0043] The context-aware matching module includes a Neighborhood Context Distance (NCD) calculation unit and a Historical Consistency Distance (HCD) calculation unit. The Neighborhood Context Distance calculation unit models the differences between a target and its spatial neighbors in the semantic feature space, thereby capturing the local context structure to improve the discriminative power and robustness of the matching. This effectively addresses the problem that traditional multi-target tracking matching strategies often rely on appearance feature similarity (such as ReID vectors) and motion state (such as IoU or uniform velocity assumptions). However, in complex scenes, such as densely packed targets, highly similar appearances, or occluded airport scenes, matching based solely on individual target features can easily lead to false associations. The calculation process is as follows: Given the set of targets detected in a frame Each target Having feature vectors and spatial location First, construct the spatial neighborhood for each target: (1) in, Indicate target and Euclidean distance, It is the neighborhood search radius, which controls the size of the context region.

[0044] To characterize the feature differences between the current target and its neighbors, this paper uses normalized cosine distance: (2) in, This represents the inner product of two eigenvectors. The distance is a Euclidean norm; the larger the distance, the less similar the appearances of the two targets.

[0045] To highlight the importance of spatially closer neighbors for context modeling, a Gaussian kernel is introduced as a weighting function: (3) in, This indicates the degree to which spatial distance affects neighborhood weights, reflecting the prior assumption of spatial locality.

[0046] Therefore, the goal The neighborhood context distance is averaged by the weighted feature differences of its neighbors, and is expressed as:

[0047]

[0048] Wherein, the context distance For the goal The larger the value of the local feature difference measure, the more heterogeneous and complex the neighborhood is.

[0049] In the actual matching process, for candidate matching pairs ( We jointly model individual feature similarity and contextual consistency, defining the final matching score as: (6) in, Indicates the current goal and candidate targets Feature distance; and These represent the context distance between the two targets; , These are adjustable hyperparameters, controlling the contributions of feature similarity and contextual consistency to the matching score, respectively.

[0050] This scoring function, while ensuring the effectiveness of traditional feature matching, introduces the constraint of "contextual consistency" to penalize matching pairs with large differences in semantic neighborhood structure, thereby effectively reducing false associations caused by high similarity in target appearance or detection drift.

[0051] The historical consistency distance calculation unit quantifies the degree of consistency between candidate detections and trajectories in terms of historical observation stability. Unlike metrics that rely solely on current frame features, the historical consistency distance calculation unit focuses on the persistence of the target's appearance and detection reliability within a certain time window, and measures the compatibility of the current detection with this historical pattern. This effectively compensates for the shortcomings of instantaneous information such as IoU (motion prior) and ReID (appearance matching) in long-term temporal associations. Specifically: Set up detection The appearance feature vector is trajectory In the current frame Recently One effective appearance feature is Then its historical appearance mean and volatility are defined as: (7) in, >0 indicates a numerically stable term.

[0052] The normalized appearance deviation between the detection and the trajectory history template is: (8) This normalization method can dynamically adjust the deviation scale based on the historical fluctuations of the trajectory itself, thereby maintaining comparability between trajectories with different appearance stability.

[0053] In addition, HCD introduces a trajectory reliability factor, based on the length of... Missed detection rate within the time window calculate: (9) in, >0 represents the rate at which control reliability decreases with the proportion of missed detections.

[0054] To suppress the influence of outdated trajectories, a time decay factor is defined: (10) in, This represents the number of frames since the last valid observation of the trajectory. Control the decay rate.

[0055] Ultimately, the historical consistency distance is defined as: (11) in, For the Sigmoid function, Controlling the steepness of the curve, This is the deviation threshold. Smaller... A value close to 1 indicates a high degree of consistency between the detection and the trajectory in historical patterns, while a value close to 1 indicates a significant difference.

[0056] Finally, after defining the Neighborhood Context Distance (NCD) and History Consistency Distance (HCD), the multimodal matching and tracking module compares them with the traditional appearance distance (…). ) and motion priors ( Unified modeling is used to construct the final context-aware matching cost function: (12) in, To measure the difference between the detection and the trajectory in the visual feature space, based on the appearance distance (such as cosine distance) of ReID features; To reflect the consistency of motion state based on the IoU distance between the predicted position and the detection box; Neighborhood Context Distance (NCD) quantifies the degree of semantic difference between a candidate target and its spatial neighborhood; The Historical Consistency Distance (HCD) characterizes the stability differences between candidate targets and trajectories in long-term appearance patterns. This cost function, while maintaining the effectiveness of traditional features and motion information, introduces two complementary constraints: context and historical consistency, thereby significantly improving the robustness of matching in complex situations such as highly similar appearances, intersecting motion trajectories, or detection drift.

[0057] In the target matching process, appearance feature similarity (CAM) in the first stage can effectively complete the trajectory association of most targets. However, in some scenarios, such as occlusion, target scale changes, or unclear appearance features, relying solely on appearance feature matching often fails. Therefore, based on the initial association, a depth-consistency discriminator (DCD) module is further introduced as a supplementary mechanism in the second stage to measure the depth structure consistency between matched targets, thereby enhancing the robustness of matching in occlusion and scale change scenarios.

[0058] Specifically, the DCD module assists in discrimination by comparing the consistency between candidate detections and historical trajectories in the depth dimension. Let the candidate detections... With trajectory The initial matching cost is Then, in the DCD stage, its new cost function (quadratic cost function) is defined as:

[0059] In the formula: The cost of secondary matching after introducing deep consistency constraints; The initial matching cost obtained from the context-aware matching cost function; For candidate detection and trajectory The depth consistency score between them.

[0060] in, The depth consistency score represents the depth similarity between the candidate detection and the trajectory, and is defined as follows:

[0061] In the formula: For candidate detection and trajectory The depth consistency score between them; Candidate Detection The corresponding depth value; Candidate Detection The corresponding depth value; This is the preset depth smoothing coefficient.

[0062] Deep consistency score is modeled using an exponential function. The value approaches 1 when the depth difference is small, but rapidly decays to 0 when the depth difference is large. This form ensures that the DCD module has strong tolerance for candidates with small depth differences, while effectively eliminating candidates with large depth differences. This design can compensate for feature ambiguity by using geometric constraints when appearance information is insufficient for discrimination. Especially in scenarios where multiple targets have similar appearances or partial occlusion, DCD helps to significantly improve the robustness and accuracy of trajectory matching.

[0063] To achieve both high-precision self-supervised monocular depth estimation and efficient object detection within a unified framework, this paper employs a multi-task learning paradigm. The front-end feature extraction network (ResNet18) for both tasks is shared to reduce computational overhead and enable collaborative learning of low-level visual features. After feature sharing, the two tasks are fed into their respective independent encoder-decoder structures to preserve task-specific representational capabilities and avoid over-coupling of features.

[0064] The depth estimation branch is based on the Lite-Mono architecture, whose core idea is to achieve self-supervised learning of depth through temporally adjacent frames. Specifically, it utilizes input frames... With adjacent frames Disparity prediction is performed and optimized using image reconstruction constraints. The loss function for this branch is... It consists of three parts: (13) in, Photometric Loss is used to maintain the consistency between the reconstructed image and the original image. Edge-aware Smoothness Loss is used to suppress noise in the depth map; This is called Geometry Consistency Loss, which is used to handle occluded regions and improve the stability of depth prediction.

[0065] The object detection branch employs an RT-DETR architecture to achieve efficient end-to-end detection. Shared backbone features undergo high-level semantic enhancement via an independent encoder in the detection branch, and are then input into the detection head to directly output the object category and bounding box parameters. The loss function of the detection branch... Defined as: (14) in, Focal Loss is used to alleviate the imbalance between positive and negative samples; Includes IoU Loss and L1 regression loss, used to accurately regress the position and size of the detection box.

[0066] In multi-task training, the loss scale and convergence speed of different tasks often differ. Using fixed weights for fusion can easily lead to one task dominating the optimization process, thus affecting overall performance. Therefore, this paper introduces an uncertainty-based weighted loss strategy. By modeling task-dependent uncertainty (homoscedastic uncertainty), task weights are automatically learned during training. The overall loss function is: (15) in, and ...

[0067] Experimental Example: To verify the effectiveness of the proposed method, this paper constructs a multi-target tracking dataset for airport scenes. This dataset originates from real airport scenarios, covering typical situations such as aircraft taxiing, ground vehicle operation, and personnel activities, and is characterized by a wide variety of targets, complex scenes, and frequent occlusion.

[0068] Video data was annotated frame-by-frame using the DarkLabel tool, with annotation categories including airplanes, vehicles, and people. Each target was provided with a bounding box annotation, maintaining consistency in target identity to meet the requirements of multi-target tracking tasks. The dataset used in this paper contains 16 video segments, totaling approximately 80,000 frames. To ensure the objectivity and generalization ability of the evaluation, we divided the dataset into video sequences to avoid the same scene appearing in both training and testing. Specifically, the dataset was divided into 12 training sets, 2 validation sets, and 2 test sets (approximately an 8:1:1 ratio). The training set was used for model learning, the validation set for hyperparameter selection, and the test set was used only for final performance evaluation and results reporting, thus ensuring the fairness and reproducibility of the experimental results. The construction of this dataset provides fundamental data support for airport surface monitoring and multi-target tracking research. Compared to common general datasets such as MOT17 and MOT20, this dataset is more challenging in terms of large target scale variations, high appearance similarity, and significant depth relationships, making it more suitable for validating the effectiveness of the proposed depth consistency-consistent tracking method.

[0069] To comprehensively evaluate the performance of the proposed method in multi-target tracking tasks at airports, this paper adopts HOTA (Higher Order Tracking Accuracy) as the main evaluation metric. Compared with traditional metrics, HOTA achieves a more reasonable balance between detection accuracy and trajectory association performance, thus more objectively reflecting the overall performance of the proposed method in complex scenarios. Meanwhile, to verify the effectiveness of the proposed Deep Consistency Discriminant (DCD) module in trajectory association, we pay particular attention to the AssA (Association Accuracy) and IDF1 (ID F1-score) metrics. AssA directly measures the accuracy of trajectory association, while IDF1 reflects the consistency of target identity maintenance throughout the sequence; both highlight the advantages of this method in complex occlusion and appearance-similar scenarios.

[0070] Furthermore, it should be noted that MOTA is highly correlated with detection performance, and therefore is only comparable when different methods use the same detection results (public tracking settings). Considering that the joint training framework proposed in this paper optimizes both detection and depth estimation tasks, the MOTA metric is only used as a supplementary reference, while HOTA, AssA, and IDF1 better reflect the advantages of our method.

[0071] In this experiment, we performed end-to-end training on the proposed joint framework of self-supervised monocular depth estimation and object detection. Figure 3The graph illustrates how the loss function changes with the number of iterations during training. The left graph shows the object detection loss, and the right graph shows the depth estimation loss. The overall trend of the curves shows that the model's loss value decreases rapidly in the early stages of training and then stabilizes within a short number of iterations, indicating that the network has good convergence and high training efficiency. In particular, in the first few thousand iterations, the loss value exhibits an exponential decreasing trend, demonstrating that the network can quickly capture low-level features and geometric constraints in the image.

[0072] As training progresses, the losses from both tasks show a smooth downward trend with relatively small fluctuations. This indicates that joint optimization of multiple tasks did not introduce negative interference but instead played a positive complementary role. Notably, depth estimation and object detection exhibit a certain synergistic effect during joint training: on the one hand, depth information provides precise geometric constraints for the detection module, making the bounding box regression of objects more stable and significantly improving localization accuracy; on the other hand, the high-level semantic features learned by the detection module, in turn, provide stronger contextual information for depth estimation, helping the model maintain high robustness in depth prediction even in scenes with texture discontinuities, lighting variations, or occlusions.

[0073] To further verify the effectiveness of the proposed method during training, we visualized the depth estimation results at different iteration stages, such as... Figure 4 As shown, in the early stages of training (step 100), the predicted depth map is generally blurry, object boundaries are not clear, and there are significant deviations in the scene geometry. As training progresses (step 10000), the depth prediction gradually stabilizes, the outlines of the ground, aircraft, and surrounding facilities become clearly discernible, and the sense of depth hierarchy becomes more reasonable. This phenomenon indicates that the model can continuously optimize its depth representation capabilities within the framework of self-supervised and multi-task joint training, thereby effectively improving its geometric understanding of complex scenes and providing more reliable prior information for subsequent object detection and multi-object tracking tasks.

[0074] The performance of different categories in multi-target tracking was also tested, and the results are shown in Table 1 below.

[0075] Table 1 shows that different categories exhibit varying performance in multi-object tracking tasks. For the airplane category, all metrics reach high levels, with HOTA at 77, IDF1 and MOTA at 78.56 and 86.34 respectively, indicating that the model can achieve relatively accurate and stable tracking results in this category. In contrast, the person category performs relatively poorly, with HOTA at only 54. Although IDF1 and MOTA reach 70.13 and 73.62 respectively, they are still significantly lower than other categories overall. This is mainly due to challenges such as significant appearance differences and frequent occlusion of pedestrians in the scene. The vehicle category's performance falls between the two, with HOTA at 62 and MOTA at 83.18, indicating that the model has some robustness in vehicle tracking tasks, but there is still room for improvement in the association accuracy (AssA=57).

[0076] Overall, the model achieved HOTA=64.33, AssA=62.33, IDF1=73.51, and MOTA=81.05 on the Overall dataset, indicating that the proposed method can achieve relatively balanced performance in joint tracking of multiple target classes. The higher IDF1 and MOTA values ​​suggest good overall accuracy in detection and tracking, while the lower HOTA and AssA values ​​further reflect room for optimization in target association and trajectory consistency. These results validate the effectiveness of the proposed method within the joint framework of self-supervised depth estimation and target detection.

[0077] The changes in HOTA and its sub-indices of the model across the three categories of airplane, person, and vehicle under different thresholds are shown in the following figures. Figure 5 As shown.

[0078] Depend on Figure 5As can be seen, the model achieved the best performance in the airplane category, with a HOTA of 0.77. Both localization accuracy (LocA=0.92) and detection recall (DetRe=0.92) remained at a high level, indicating that the method can fully leverage its advantages in scenarios with clear structures and well-defined target boundaries, achieving stable and accurate detection and tracking. In the vehicle category, the model also demonstrated strong detection and localization capabilities, with outstanding detection accuracy (DetA=0.68), detection precision (DetPr=0.81), and localization accuracy (LocA=0.84), reflecting the model's good adaptability in terms of vehicle target appearance consistency and boundary discrimination. Despite the large number of vehicles and more complex scenes, the overall results remained stable, demonstrating the method's robustness. In the person category, the model maintained reasonable performance in terms of association consistency and detection stability, with a HOTA of 0.53 and localization accuracy (LocA=0.78), indicating that even under conditions of frequent occlusion and significant appearance differences, the model can still maintain an effective understanding of the scene geometry. This further validates the generalization ability of the proposed method across different categories and diverse scenarios.

[0079] Overall, the experimental results for the three types of targets show that the method can fully leverage its advantages on structurally regular targets (such as airplanes and vehicles), while also demonstrating robust performance on more challenging targets (such as people).

[0080] To further verify the overall performance of the method of the present invention, this experimental example uses multiple models for comparison to verify their performance on the airport multi-target tracking dataset. The results are shown in Table 2 below.

[0081]

[0082] As shown in Table 2, the HOTA of our method reaches 64.33, which is higher than SORT, DeepSORT, ByteTrack, and OC-SORT, indicating that it achieves a better balance between detection accuracy and trajectory correlation. In particular, the AssA metric improves from 60.0 in OC-SORT to 62.33, demonstrating that our method can significantly enhance the consistency of identity preservation in scenarios with complex occlusion and high target similarity. The IDF1 metric also reaches 73.51, ensuring the stability of target identity while avoiding frequent trajectory interruptions. Although the MOTA is not significantly different from OC-SORT, considering that our framework simultaneously performs detection and depth estimation training, the results still have high reference value.

[0083] To further verify the effectiveness of the method in real-world scenarios, the visualization results of OC-SORT and the method presented in this paper on airport surveillance videos were compared. Figure 6 As shown in Frame 1, both methods can detect and track the main target, but the method in this paper is more detailed than OC-SORT and can accurately detect the person near the apron, demonstrating a stronger ability to perceive small targets.

[0084] As the video sequence progresses to Frames 600 and 650, the aircraft takes off and gradually becomes occluded. Under these complex conditions, our proposed method maintains target ID consistency; the originally assigned ID6 remains unchanged, correctly identifying the same target even after partial occlusion and changes in viewpoint. In contrast, OC-SORT exhibits an ID switching issue when faced with occlusion: the original ID6 is incorrectly switched to ID8, resulting in discontinuous tracking trajectories. This ID switching phenomenon significantly reduces tracking accuracy and stability in long-term tracking tasks. The visualization results above demonstrate that our proposed method exhibits stronger robustness and ID preservation capabilities in complex environments. This advantage is consistent with the improvement in metrics such as IDF1 shown in quantitative experiments, further proving the practicality and superiority of our proposed method in real-world scenarios.

[0085] (4) Ablation test To verify the contribution of each component to the overall performance, we examined the performance of removing neighborhood context distance consistency (CAM w / o NCD), removing history consistency distance (CAM w / o HCD), retaining only context-aware matching (CAM), retaining only depth consistency discrimination (DCD), and the complete model (Ours) under the same training and evaluation settings. The results are shown in Table 3.

[0086]

[0087] As shown in Table 3, the model of this invention achieved the best performance in all four indicators (HOTA 64.33, AssA 62.33, IDF1 73.51, MOTA 81.05), which is an improvement of +1.23, +1.43, +0.91, and +0.85 respectively compared with the configuration using only CAM. This indicates that the introduction of NCD, HCD and DCD on the basis of CAM can synergistically improve the overall quality of detection-correlation.

[0088] From a component perspective, NCD, targeting dense scenes and highly similar appearances, significantly enhances the ability to discriminate local structures. When NCD is removed (CAM w / o NCD), HOTA and AssA drop significantly (approximately -2.6 and -2.7 compared to Ours), indicating that spatial neighborhood semantics and relative relationships are crucial for suppressing mismatches. HCD, through stability modeling within a time window, alleviates occasional distortions in single-frame appearance / motion metrics in long-term temporal associations. After removing HCD (CAM w / o HCD), the reduction in AssA and IDF1 is more pronounced (approximately -2.1 and -1.6 compared to Ours), consistent with its direct effect on trajectory consistency and identity preservation. DCD, as a two-stage supplementary judgment mechanism, provides additional geometric consistency constraints on occlusion, scale-changing, and appearance-weak discriminative examples. When used alone (DCD), its overall benefit is limited (slightly lower than CAM), but when combined with CAM into the complete framework, it can bring stable gains on difficult examples, making Ours' overall performance optimal. This phenomenon indicates that CAM is responsible for the high recall and strong discrimination of the backbone matching, while DCD "fills in the gaps" in edge and mismatched samples, and the two complement each other.

[0089] The above description is only a preferred embodiment of the present invention and does not limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An airport multi-target tracking method based on context awareness and depth consistency constraints, characterized in that: Includes the following steps: Step 1: Input the input image into the feature extraction backbone network for feature encoding. Then, input the encoded features in parallel into the self-supervised training monocular depth estimation network and the detector based on the Transformer architecture to generate depth maps and high-precision target detection boxes. Step 2: Obtain the set of detected targets and the set of historical trajectories in the current frame, and use a Kalman filter to predict the motion state of the trajectory; Step 3: Integrate the target's appearance features, motion priors, contextual distribution differences, and historical trajectories, and perform preliminary matching and association; Step 4: Perform deep consistency judgment on the targets that failed the initial matching and association in Step 3 to complete the secondary matching and association; Step 5: Update the trajectory status and output the multi-target tracking results based on the preliminary matching and secondary matching results.

2. The airport multi-target tracking method based on context awareness and depth consistency constraints according to claim 1, characterized in that: In step three, the neighborhood context distance and historical consistency distance are calculated and modeled in a unified manner with appearance distance and telekinetic prior, and a context-aware matching cost function is constructed to achieve preliminary matching association.

3. The airport multi-target tracking method based on context awareness and depth consistency constraints according to claim 2, characterized in that: The neighborhood context distance calculation method includes the following steps: (1) Given the set of targets detected in a frame as Each target Having feature vectors and spatial location To construct a spatial neighborhood for each target ； In the formula: Indicate target and Euclidean distance, It is the neighborhood search radius, which controls the size of the context region; (2) Calculate the feature difference between the current target and each target in the domain. and weighting coefficients ; ； In the formula: This represents the inner product of two eigenvectors. The distance is a Euclidean norm; the larger the distance, the less similar the appearances of the two targets. ； In the formula, It indicates the degree to which the control spatial distance affects the neighborhood weight, reflecting the prior assumption of spatial locality; (3) Calculate the neighborhood context distance of the target. , ； ； Where: context distance For the goal The larger the value of the local feature difference measure, the more heterogeneous and complex the neighborhood is.

4. The airport multi-target tracking method based on context awareness and depth consistency constraints according to claim 1, characterized in that: The historical consistency distance calculation method includes the following steps: (1) Assume detection The appearance feature vector is trajectory In the current frame Recently One effective appearance feature is Calculate its historical appearance mean and fluctuation. ； In the formula: >0 indicates a numerically stable term; (2) Calculate the normalized appearance deviation between the detection and the trajectory history template. ； (3) Based on the missed detection ratio within a time window of length w Calculate the trajectory reliability factor; ； In the formula: >0 The rate at which control reliability decreases with the proportion of missed detections; (4) Define the time decay factor. ； In the formula: This represents the number of frames since the last valid observation of the trajectory. Control the decay rate; (5) Calculate the historical consistency distance based on appearance deviation, trajectory reliability factor, and defined time decay factor. ； In the formula, For the Sigmoid function, Controlling the steepness of the curve, This is the deviation threshold. Smaller... A value close to 1 indicates a high degree of consistency between the detection and the trajectory in historical patterns, while a value close to 1 indicates a significant difference.

5. The airport multi-target tracking method based on context awareness and depth consistency constraints according to claim 2, characterized in that: The context-aware initial matching cost function is: ； In the formula: The appearance distance based on ReID features measures the difference between the detection and the trajectory in the visual feature space. To reflect the consistency of motion state based on the IoU distance between the predicted position and the detection box; The neighborhood context distance quantifies the degree of semantic feature difference between the candidate target and its spatial neighborhood; The historical consistency distance characterizes the stability differences between candidate targets and trajectories in long-term appearance patterns.

6. The airport multi-target tracking method based on context awareness and depth consistency constraints according to claim 1, characterized in that: In step four, the deep consistency judgment for targets that failed the initial matching and association is carried out by calculating the deep consistency score between candidate detections and historical trajectories in the depth dimension, constructing a quadratic cost function, and re-matching the detections and trajectories that failed the initial association based on the results of the quadratic association.

7. The airport multi-target tracking method based on context awareness and depth consistency constraints according to claim 6, characterized in that: The formula for calculating the depth consistency score is as follows: ； In the formula: For candidate detection and trajectory The depth consistency score between them; Candidate Detection The corresponding depth value; Candidate Detection The corresponding depth value; This is the preset depth smoothing coefficient.

8. The airport multi-target tracking method based on context awareness and depth consistency constraints according to claim 6, characterized in that: The quadratic cost function is: ； In the formula: The cost of secondary matching after introducing deep consistency constraints; The initial matching cost obtained from the context-aware matching cost function; For candidate detection and trajectory The depth consistency score between them.

9. An airport multi-target tracking system based on context awareness and depth consistency constraints, characterized in that: include: The feature extraction module includes a feature extraction backbone network for extracting features from the input image, a monocular depth estimation network connected to the output of the backbone network, and a detector based on the Transformer architecture, which outputs depth maps and target detection boxes in parallel. The motion state prediction module, with a built-in Kalman filter, is used to predict the motion state of historical trajectories and obtain the predicted trajectory. The multimodal matching module includes a context-aware matching module and a depth consistency discrimination module. The context-aware matching module is used to fuse multimodal information and construct a comprehensive cost function for preliminary association. The depth consistency discrimination module is used to calculate a depth consistency score based on the depth estimates of candidate detections and historical trajectories when the context-aware matching module fails to match, and to perform secondary discrimination and association. The target tracking output module is used to update the trajectory status and output multi-target tracking results based on the association results of the context-aware matching module and the depth consistency discrimination module.

10. The airport multi-target tracking system based on context awareness and depth consistency constraints according to claim 9, characterized in that: The multimodal information includes appearance features, motion information, differences in the distribution of neighborhood features, and historical stability information.

Images