Airport aircraft target tracking method and system based on low-orbit satellite joint monitoring

Abstract

The present invention discloses a method and system for tracking airport aircraft targets based on low-orbit satellite joint surveillance. The method includes the following steps: S100. Data preprocessing: performing image processing on satellite remote sensing images corresponding to the airport area acquired through joint surveillance by multiple satellites, mapping the satellite remote sensing images captured by different satellites into a unified geographic coordinate system, achieving a unique mapping of pixels to geographic locations; S200. Target detection: identifying aircraft targets in satellite remote sensing images using a target detection network (YOLOv7) algorithm, automatically identifying and locating targets from the image data; and S300. Target tracking: using a two-stage target tracking method based on YOLOv7 and DeepSORT, continuously tracking multiple targets through the target tracking algorithm network combined with target detection result slices. This method can effectively improve the accuracy of target identification and tracking.

Description

Technical Field

[0001] The present invention relates to the fields of aerospace and satellite technology, and in particular to an airport aircraft target tracking method based on low-orbit satellite joint monitoring. Background Art

[0002] In recent years, with the rapid development of commercial space technology, the number of satellites in orbit around the world has increased rapidly, especially low-orbit commercial Internet satellites have been deployed on a large scale. At the same time, the construction of low-orbit commercial remote sensing satellite constellations is also accelerating. my country's "Jilin-1" satellite constellation has broken the record for the largest number of satellites launched in a single time by China's aerospace industry. There are currently more than 100 satellites in orbit. Its goal is to achieve the ability to revisit any location in the world within 10 minutes through the deployment of a large-scale satellite network, significantly improving the real-time and continuity of regional observations.

[0003] As important hubs for national and regional transportation, the safety and operational status of airports are crucial to national security and economic operations. Continuous monitoring of airports can effectively prevent and control various emergencies, while also enhancing efficiency, optimizing aircraft ground operations and scheduling, and reducing delays. Low-orbit commercial remote sensing constellations can achieve continuous relay observations of the same area, providing a new technical means for real-time monitoring of airports and making continuous tracking of aircraft targets possible. Current research on aircraft target tracking in the field of satellite remote sensing is mainly based on single satellite video, with a short target tracking duration, and mainly uses video multi-target tracking methods. Among traditional video multi-target tracking methods, detection-based optical flow technology has difficulty achieving real-time tracking due to its high computational requirements. While the frame difference method can provide faster tracking speeds, it is more sensitive to factors such as changes in ambient light, image noise, and target deformation, resulting in insufficient tracking accuracy.

[0004] With the rise of deep learning, deep learning-based object tracking algorithms have developed rapidly, and two-stage object tracking algorithms that combine object detection networks have emerged. These methods first use an object detector to identify objects in video frames, and then use object tracking algorithms to maintain the continuity and identity of the objects in the video sequence. For example, the SORT algorithm is based on the Kalman filter and the Hungarian algorithm. It first uses object detection algorithms such as Faster R-CNN to identify the object, then uses the Kalman filter to predict the object's position in the current frame. Finally, the Hungarian algorithm uses the Intersection over Union (IoU) distance to associate the detected objects with existing tracks, thereby achieving continuous tracking of the object in the video sequence. The ByteTrack algorithm uses the high-performance YOLOX detector to perform object detection and the BYTE algorithm to associate detection boxes with tracks. Alternatively, these two-stage object tracking algorithms combine object detection with the subsequent tracking process, effectively improving tracking accuracy and robustness, making them particularly suitable for complex video scenes. Current research has made many improvements to address the difficulties of object tracking. LCB-YOLOv5 enhances feature extraction by introducing the LSM and C3 modules to obtain more multi-scale global information. This method can enhance the model's ability to grasp objects of varying scales in satellite remote sensing imagery. However, it requires high computing power and a long training time. By improving the MDNet network and using transfer learning and specialized convolutional layers to extract aircraft features, good tracking accuracy and real-time performance were achieved. However, there is still room for improvement in adaptability and tracking stability in complex environments.

[0005] Aircraft target tracking based on low-orbit satellite joint surveillance is a multi-target tracking problem across satellites. Its core lies in the joint processing of multi-source satellite information, enabling continuous target tracking through the fusion of target information between different satellites. Unlike video-based target tracking tasks, target tracking under joint satellite surveillance still faces several difficulties: First, the long satellite imaging intervals and varying time delays result in no overlap between moving targets in adjacent images, making matching difficult; second, targets in satellite remote sensing images are very small relative to the overall image size, making detection difficult; and third, remote sensing images obtained from different satellites exhibit discontinuous differences in perspective, radiation, and other factors, which hinder target tracking. Summary of the Invention

[0006] In response to the problems existing in the prior art, the purpose of the present invention is to provide a technical method for obtaining continuous high-resolution satellite remote sensing images through joint monitoring of low-orbit satellites and continuously tracking aircraft targets in ground airports, thereby effectively identifying the dynamic information of aircraft targets.

[0007] To achieve the above object, the present invention provides an airport aircraft target tracking method based on low-orbit satellite joint monitoring, the method comprising the following steps:

[0008] S100. Data preprocessing: Perform image processing on satellite remote sensing images of the corresponding airport area acquired through joint surveillance by multiple satellites, so that the satellite remote sensing images captured by different satellites are mapped into a unified geographic coordinate system, achieving a unique mapping of pixels to geographic locations;

[0009] S200. Target Detection: Identify aircraft targets in satellite remote sensing images using the YOLOv7 target detection network algorithm, automatically identifying and locating targets from image data.

[0010] S300. Target Tracking: Use a two-stage target tracking method based on YOLOv7 and DeepSORT to continuously track multiple targets through the target tracking algorithm network combined with target detection result slices.

[0011] Furthermore, step S100 specifically includes:

[0012] S101. Data Acquisition: Utilize remote sensing satellite constellation joint surveillance to obtain original remote sensing data images corresponding to the airport area;

[0013] S102. Image Registration: Process remote sensing images from different satellites and ensure that they are correctly aligned in geographic space. Image registration technology can be used to map images taken by different satellites into a unified geographic coordinate system, achieving a unique mapping of pixels to geographic locations.

[0014] Furthermore, in step S200, the YOLOv7 network uses the darknet deep learning framework to implement end-to-end training of the input image; during prediction, the network outputs a predicted bounding box, which contains five prediction parameters: x, y, w, h and confidence, where x and y represent the coordinates of the target detection box, w and h are the width and height of the target detection box, and confidence is the confidence level.

[0015] Furthermore, the prediction box with the highest confidence is selected as the final detection result through non-maximum suppression.

[0016] Furthermore, the convolutional attention mechanism module CBAM is integrated into YOLOv7 to enhance the perception ability of the model. CBAM integrates two attention mechanisms in sequence: channel attention and spatial attention.

[0017] Furthermore, global average pooling and global maximum pooling operations are performed to obtain global spatial information, and then the dependencies between channels are learned through a shared network layer to generate a channel attention map, which is calculated as follows:

[0018] (1)

[0019] in, is the input feature map, is the channel attention map, is the sigmoid activation function, Represents a multilayer perceptron.

[0020] Furthermore, the spatial attention module is implemented by maximizing and averaging the feature maps to generate a spatial attention map that highlights important spatial areas. The formula of the spatial attention map is as follows:

[0021] (2)

[0022] in, Indicates that a 7×7 convolution kernel is used for convolution to merge the feature maps of average pooling and maximum pooling to generate a spatial attention map. .

[0023] Furthermore, step S300 specifically includes:

[0024] S301. Calculate the time difference: Different satellite remote sensing images have different shooting times, and the time difference between adjacent images is not fixed. Calculate and give the time difference between adjacent images;

[0025] S302. Feature extraction: First, the target image slice is captured according to the detection frame, and its appearance and size information is retained; then, the boundary filling and depth feature extraction are performed on each target slice;

[0026] S303. State prediction: Based on the time difference between images, the state prediction matrix is ​​used to predict the state of the aircraft target at the next moment, including position and speed; the accuracy and robustness of the tracking process are ensured by setting relevant thresholds.

[0027] Furthermore, the DeepSORT algorithm is used to implement target matching and tracking, with the following specific improvements:

[0028] (1) Introduction of time variables: Based on the original DeepSORT, When a new satellite image is input, the Kalman filter will get a new As the time difference, a new state update matrix is ​​generated;

[0029] (2) Mahalanobis distance matching optimization: The limit of Mahalanobis distance is enlarged so that more predicted position boxes can be retained, avoiding the situation where the target in the image moves far while the predicted box only moves a short distance from the original position;

[0030] (3) Slice scale feature preservation: In order to make the obtained feature vector preserve more target size and appearance information, the target slice is border-filled. This way, while ensuring the same input image size, the size and original appearance of the target aircraft are retained. When calculating the cosine distance matrix, the cosine distance of the same target will be smaller than the cosine distance of other targets. Lowering the cosine distance threshold helps to screen out paired targets.

[0031] (4) IOU matching optimization: The IOU matching threshold is enlarged so that the algorithm does not delete moving targets, thereby ensuring the tracking recall rate.

[0032] On the other hand, the present invention provides an airport aircraft target tracking system under joint monitoring of low-orbit satellites, the system including a data preprocessing module, a target detection module and a target tracking module, for implementing the airport aircraft target tracking method under joint monitoring of low-orbit satellites according to the present invention.

[0033] Beneficial effects:

[0034] This invention uses a two-stage target tracking method and system based on YOLOv7 and DeepSORT, and has been adaptively improved according to the application scenario. Using image data of the same airport area captured by multiple "Jilin-1" satellites in relay mode, it achieves effective tracking of airport aircraft targets, enhances the real-time perception of airport operating status, and can provide technical support for airport safety management, flight scheduling, emergency response and other fields. It also further verifies the huge application value of low-orbit satellite constellations. The method of the present invention can integrate data from multiple satellites at different time points, achieving continuous monitoring of airport activities and demonstrating the huge application potential of large-scale low-orbit satellite constellations in the field of target monitoring in the future. It has high recognition and tracking accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 The following is a flowchart showing the implementation process of aircraft target tracking based on satellite joint monitoring according to the present invention;

[0036] Figure 2 A schematic diagram of filling pixels with objects of different sizes according to the present invention is shown;

[0037] Figure 3 A schematic diagram of the target position of the same aircraft on the runway in three adjacent images according to the present invention is shown;

[0038] Figure 4 The DeepSORT feature extraction network structure according to the present invention is shown;

[0039] Figure 5 A schematic diagram showing comparison of target detection results according to the present invention is shown;

[0040] Figure 6 An initial tracking effect diagram of target tracking according to the present invention and a schematic diagram of a moving target are shown. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solution of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0043] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0044] The following combination Figures 1-6 The specific embodiments of the present invention are described in detail. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0045] The airport aircraft target tracking system based on low-orbit satellite joint monitoring according to the present invention includes three parts: data preprocessing module, target detection module and target tracking module. By improving the targets in different states, it can continuously track the moving and stationary targets in the image. The basic implementation framework is as follows: Figure 1 In the present invention, a low-orbit satellite refers to a satellite with an orbital altitude of less than 2000 kilometers.

[0046] The method for tracking an airport aircraft target by low-orbit satellite joint monitoring according to the present invention comprises the following steps:

[0047] S100. Data preprocessing: performing image processing on satellite remote sensing images of the corresponding airport area acquired through joint surveillance by multiple satellites, so that the satellite remote sensing images taken by different satellites are mapped into a unified geographic coordinate system, achieving a unique mapping of pixels to geographic locations.

[0048] S200. Target Detection: This algorithm uses the YOLOv7 target detection network to identify aircraft targets in satellite remote sensing imagery, automatically identifying and locating targets from the image data. This provides initial data for subsequent tracking. The higher the accuracy of the target detection algorithm, the more accurate and reliable the subsequent tracking.

[0049] S300. Target Tracking: Use a two-stage target tracking method based on YOLOv7 and DeepSORT to continuously track multiple targets through the target tracking algorithm network combined with target detection result slices.

[0050] Step S100 specifically includes:

[0051] S101. Data acquisition: Utilize remote sensing satellite constellation joint monitoring to obtain original remote sensing data images corresponding to the airport area; in a specific embodiment of the present invention, the satellite constellation consists of 6 remote sensing satellites distributed in low orbit.

[0052] S102. Image Registration: This is a fundamental step in target tracking, processing images from different satellites and ensuring they are correctly aligned in geographic space. Using high-precision image registration technology, images captured by different satellites can be mapped to a unified geographic coordinate system, achieving a unique mapping of pixels to geographic locations. This step is crucial for subsequent target detection and tracking, as it ensures that images acquired at different time points can be accurately compared. In this embodiment, the image registration technology used is ENVI (The Environment for Visualizing Images).

[0053] In step S200, the present invention uses the YOLOv7 algorithm to detect aircraft at airports. Because aircraft targets against the backdrop of airports in satellite remote sensing images are small in pixel size and numerous, YOLOv7 significantly improves target detection performance by introducing convolutional reparameterization and dynamic label assignment. This algorithm achieves superior detection performance for aircraft detection tasks.

[0054] Like other networks in the YOLO family, the YOLOv7 network uses the Darknet deep learning framework for end-to-end training of input images. During prediction, the network outputs a predicted bounding box, which contains five parameters: x, y, w, h, and confidence. x and y represent the coordinates of the object detection box, and w and h represent the width and height of the object detection box. Finally, the prediction box with the highest confidence is selected as the final detection result using the non-maximum suppression algorithm.

[0055] In real airports, there are often many types of aircraft targets, with varying sizes. In contrast, small targets, whose length and width are less than 0.1 of the original image, are less distinct and difficult to distinguish from the background. This makes it more difficult for the target detection network to detect small targets, resulting in significant missed detections. In this invention, small targets refer to targets whose length and width are less than 0.1 of the original image.

[0056] To address this issue, the present invention makes the following improvements: In YOLOv7, the backbone network generates features of three different dimensions, corresponding to different network depths. The network feature maps are output in sizes of 80×80, 40×40, and 20×20 pixels, respectively, for detecting small objects smaller than 50×50 pixels, medium-sized objects larger than 50×50 and smaller than 70×70 pixels, and large objects larger than 70×70 pixels. The present invention applies a Convolutional Attention Mechanism (CBAM) to these three features. By using the features enhanced by CBAM, the model can focus on key parts of the feature maps and channels, better identifying and localizing small objects in images.

[0057] To address the problem of small-sized target detection, the present invention integrates the Convolutional Attention Mechanism Module (CBAM) into YOLOv7 to enhance the model's perception of small-sized targets and improve the performance of small-sized target detection. CBAM is a lightweight attention mechanism module that can be directly integrated into the CNN network used for feature extraction in YOLOv7 to enhance feature expression capabilities. First, the three-scale feature maps output after the convolutional layer are identified in the YOLOv7 backbone network, and then the CBAM module is integrated. CBAM sequentially integrates two attention mechanisms: channel attention and spatial attention. Channel attention can highlight feature channels that are more important for target detection by considering the feature importance between different channels. It usually obtains global spatial information through global average pooling and global maximum pooling operations, and then learns the dependencies between channels through a shared network layer to generate a channel attention map. The calculation method is:

[0058] (1)

[0059] in, is the input feature map, is the channel attention map, is the sigmoid activation function, Represents a multilayer perceptron.

[0060] Building on channel attention, the spatial attention module further focuses on more important locations in the image. During training, the network automatically learns to assign higher weights to regions in the spatial attention map through backpropagation, gradually focusing the model's attention on areas more likely to contain small objects, enhancing its ability to detect small aircraft targets. The spatial attention module achieves this by maximizing and averaging the feature maps, generating a spatial attention map (Equation 2) that highlights important spatial regions containing small objects.

[0061] (2)

[0062] in, Indicates that a 7×7 convolution kernel is used for convolution to merge the feature maps of average pooling and maximum pooling to generate a spatial attention map. .

[0063] In the airport detection scenario, for small targets, CBAM will increase the weight of feature maps containing valid targets during training, so that the model pays more attention to the corresponding feature maps to highlight the features related to small-sized targets, such as shape and position. Spatial attention will analyze the amount of information at each spatial position on the feature map and emphasize positions with high corresponding values, making areas containing small targets easier to detect.

[0064] Step S300 specifically includes:

[0065] S301. Calculating Time Difference: Considering that different satellite remote sensing images are captured at different times and the time difference between adjacent images is not fixed, it is necessary to calculate and present the time difference between adjacent images based on the imaging time information contained in the images. Currently, common position prediction methods mostly use uniform speed calculations. Determining the time difference between adjacent images helps accurately predict the target's movement trajectory during target tracking.

[0066] S302. Feature extraction: Feature extraction is a key step in target tracking. First, the target image slices are intercepted according to the coordinate points given by the detection box, without stretching or cropping to preserve its appearance and size information, which is crucial for subsequent target matching. Then, the boundary filling and depth feature extraction are performed on each target slice, such as Figure 2As shown in the figure, targets of different sizes are given the size that needs to be input into the CNN network in the end, and grayscale pixels are filled on the periphery so that the targets retain their own scale information on the basis of unified size when input into the network, thereby obtaining more unique target feature information in preparation for target matching and tracking.

[0067] Regarding slice-scale feature preservation, DeepSORT's Re-Identification (ReID) feature extraction in this paper uses a CNN. This requires the input image to be uniformly sized. To ensure that the resulting feature vector retains more information about the object's size and appearance, the object slices are padded. This preserves the size and appearance of the target aircraft while maintaining the same input image size. When calculating the cosine distance matrix, the cosine distance of the same object is often smaller than the cosine distances of other objects. Lowering the cosine distance threshold helps filter out paired objects.

[0068] S303. State Prediction: Based on the time difference between images, the target's movement distance will vary between them. The diagonal values ​​in the state prediction matrix are modified based on the given time difference. This, combined with the current position and velocity information, improves the accuracy of the target's predicted position. This step comprehensively considers the target's spatial and temporal changes to accurately track the moving target. Furthermore, relevant thresholds are set to ensure the accuracy and robustness of the tracking process.

[0069] Unlike traditional video target tracking, in multi-satellite joint surveillance, due to significant temporal differences between satellites, moving targets may not overlap in consecutive image frames. This makes position-based IOU matching methods inapplicable. DeepSORT is an advanced target tracking algorithm that leverages both the motion and appearance characteristics of a target, compensating for the performance limitations of traditional methods such as IOU (Intersection-over-Union) tracking in complex scenarios. To adapt to multi-satellite joint aircraft target tracking applications, this paper improves DeepSORT so that even when there is no positional overlap between targets, the network can achieve high-precision target tracking by comparing their appearance features.

[0070] DeepSORT improves the SORT algorithm by introducing an additional cascade matching before IOU matching to enhance tracking performance. It is particularly suitable for handling situations where there is large motion between targets in consecutive frames or the target is occluded.

[0071] The core of cascade matching is to calculate the cost matrix (Cost Matrix), which is mainly composed of two parts: (1) Mahalanobis distance and cosine distance are used to calculate the similarity matrix to obtain the cost matrix; (2) Gate matrix is ​​used to limit the excessive values ​​in the cost matrix, and the Mahalanobis distance and cosine distance are limited by giving the gating threshold (gating_threshold) and maximum distance (max_distance) respectively.

[0072] Since the target may move very quickly in real-world situations, IOU matching may cause the target to be lost. Generally speaking, the target movement distance between the upper and lower frames will not be too far, so the closer the coordinates are, the more likely they are the same target. DeepSORT uses the Mahalanobis distance to calculate the distance between detection frames. The covariance matrix S is obtained using the first four dimensions x, y, w, and h of the detection frame, which gives the Mahalanobis distance similarity calculation formula:

[0073] (3)

[0074] Indicates the The position of the detection box, Indicates the If the Mahalanobis distance of each association is less than the specified threshold, it means the association is successful.

[0075] DeepSORT not only considers the distance between boxes, but also introduces a deep neural network to extract the appearance features of the target. This is the origin of Deep in DeepSORT. Its network structure is as follows Figure 4 As shown. A large number of residual network structures are used to extract the input 128×64 size target slices into 128-dimensional feature vectors, and finally use L2 regularization to obtain the final feature vector. These feature vectors contain the appearance information of the target, enabling the algorithm to recognize the appearance of the same target in different frames. DeepSORT uses the cosine distance between the extracted feature vectors for target matching. The cosine distance measures the angular difference between two feature vectors rather than their Euclidean distance, so that the matching process focuses more on the direction of the vector rather than its size. This method effectively identifies similar targets in the feature space, even if their positions in the image space have changed significantly. The cosine distance can be calculated using the following formula:

[0076] (4)

[0077] in, and are the magnitudes of the vectors respectively.

[0078] The DeepSORT algorithm first uses the state predicted by the Kalman filter for preliminary matching, then uses cascade matching to correct the results of the preliminary matching, and finally solves the final matching problem through the Hungarian algorithm.

[0079] In summary, DeepSORT provides a powerful algorithmic framework to handle the challenges in satellite remote sensing image tracking by combining appearance feature extraction and cosine distance matching.

[0080] Currently, DeepSORT is mainly used for video multi-target tracking tasks. Since the video has a high frame rate, the movement of the target is continuous in this task scenario, so the tracking trajectory can be established through the overlapping parts of the target. However, in cross-satellite remote sensing image target tracking, a track list can be created for stationary targets in this way, but for moving targets, due to the long time interval between satellites, the same target often does not have overlapping parts and the position differences are large, which makes the original DeepSORT completely unable to effectively track such targets. At the same time, it will have a certain impact on the originally available stationary target tracking, and new ID switching situations will occur. Therefore, the present invention adaptively improves DeepSORT to achieve target matching tracking in response to this technical problem. For step S303, the specific improvements are as follows:

[0081] Time variable introduction: based on the original DeepSORT When a new satellite image is input, the Kalman filter will get a new As the time difference, a new state update matrix is ​​generated, which can make the predicted target position more accurate.

[0082] In the original DeepSORT algorithm, the Kalman filter is used to estimate the position and velocity of the target. The traditional state transition matrix usually assumes that the time interval (dt) between each prediction is fixed, for example:

[0083] (5)

[0084] Typically, the time interval depends on the frame rate. However, in practical applications, the time interval between image acquisitions may be inconsistent due to various reasons (such as system delay or frame loss), which can lead to inaccurate predictions.

[0085] To solve this problem, the concept of dynamic time interval (dt) is added to the Kalman filter part of DeepSORT, allowing dynamic adjustment based on actual input. The improved state transition matrix is ​​as follows:

[0086] (6)

[0087] is a variable that can be set according to the actual time interval. This gives us a new state prediction formula:

[0088] ⑺

[0089] in By adding a time variable to the state transition matrix, the Kalman filter can more accurately predict the target's position when presented with image data of varying time delays. This dynamic adjustment significantly improves the accuracy of position estimation, especially when encountering long time delays. This significantly enhances the algorithm's adaptability and robustness, making it more responsive to changes and challenges in real-world applications.

[0090] like Figure 3 As shown in the figure, the same aircraft target on the runway has obvious differences in movement distances in three adjacent images. The original Kalman filter can only predict uniform motion. After introducing the time variable, the predicted position of different displacement distances can be obtained according to the time interval between two adjacent images, thereby improving the prediction accuracy in the joint satellite surveillance scenario.

[0091] Mahalanobis distance matching optimization: Due to the large positional shifts of moving targets, this invention amplifies the Mahalanobis distance limit. Considering the specific application scenarios, we maximize the Mahalanobis distance limit, using the maximum distance in the original solution as the threshold. This allows more predicted position boxes to be retained, preventing the target from moving far in the image while the predicted box only moves a short distance from its original position. This is combined with the subsequent low-threshold cosine distance matching results to filter out redundant targets.

[0092] IOU matching optimization: When creating the track list, it is important to note that DeepSORT creates tracking tracks through IOU matching, which prevents it from adding completely non-overlapping objects in consecutive images to the track list. IOU matching is very effective for stationary objects and can fully suppress the occurrence of ID switching. Therefore, this paper increases the IOU matching threshold so that the algorithm does not delete moving objects, thereby ensuring the tracking recall rate.

[0093] This paper uses two datasets to train the YOLOv7 object detector: the Mar20 dataset and the custom-built "Jilin-1" satellite dataset. The Mar20 dataset is a high-resolution remote sensing aircraft dataset, containing 3,842 images and 22,341 instances. Its primary purpose is to enhance the detection model's ability to extract features from aircraft appearance. The custom-built "Jilin-1" satellite dataset is a lower-resolution satellite remote sensing image dataset, containing 2,503 images and 35,083 instances. Its primary purpose is to enhance the model's ability to detect small objects.

[0094] In this example, satellite remote sensing images captured in relays by six Jilin-1 satellites were selected to verify the performance of the aircraft target detection and tracking algorithm. Specific satellite imaging information is shown in Table 1 below. The common imaging area is an airport. The satellite images are L1 level. After preprocessing such as orthorectification and image registration, a spatially aligned satellite image sequence was obtained. The images depict 81 aircraft targets, four of which are in motion.

[0095] Because satellite remote sensing images have large pixels, we set the image reading and input to sliding window cropping. The detection results are merged and uniformly filtered using non-maximum suppression before tracking the aircraft. The main threshold parameter settings are: confidence threshold 0.75, cosine distance threshold 0.007, and IoU threshold 1.

[0096] In object detection research, key metrics for evaluating model performance include Average Precision (AP), Recall, and the F1 score. Average Precision measures the average precision of the model at different recall levels. By calculating the area under the precision-recall curve for each category, it provides a single value that comprehensively reflects the model's performance across categories. Recall measures the model's ability to identify all positive samples. A higher value indicates that the model misses fewer positive samples. The F1 score is the harmonic average of precision and recall, providing a balanced indicator between the two. It is particularly suitable for scenarios where both precision and recall are equally important. These evaluation metrics reflect the performance of the object detection model from different perspectives.

[0097] In multi-object tracking tasks, Multiple Object Tracking Accuracy (MOTA) and Multiple Object Tracking Precision (MOTP) are two core evaluation metrics used to quantify the performance of object tracking algorithms. MOTA comprehensively considers three types of errors: missed detections (misses), false positives (false positives), and identity switches (ID switches).

[0098] The multi-target tracking accuracy MOTA can be expressed by the following formula:

[0099] (8)

[0100] in, It's in time The number of missed detections, It's in time The number of false positives, It's in time The number of identity exchanges, It's in time The actual number of targets.

[0101] In the present invention, the distance between the detection box and the center point of the matching box is preferably calculated using the MOTP indicator, and the formula is as follows:

[0102] (9)

[0103] in, It's in time The distance between the i-th matching pair in , is the number of target pairs to be matched. Since the distance calculation is the Euclidean distance between the center points, the smaller the MOTP indicator value, the higher the accuracy.

[0104] The present invention also considers several indicators derived from the detection task, such as IDP, IDR and IDF1, namely precision, recall and F1 Score, to evaluate the ability of the tracking model to maintain identity.

[0105] In order to maximize the recall rate of aircraft target detection, ensure that the target can be tracked later, and reduce the impact of single target re-detection on the subsequent target tracking process, the present invention preferably sets the detection confidence to 0.75. Figure 5 As shown in the figure, the left side is the original YOLOv7 detection result. It can be found that the smaller aircraft target is not detected, while the result on the right side with the integrated CBAM module can detect the target, significantly reducing the missed detection rate.

[0106] The quantitative evaluation results of the two methods are shown in Table 2 below. After integrating the CBAM module, all indicators are significantly improved.

[0107] Table 2 Comparison of target detection quantization results

[0108]

[0109] Due to the large time difference between adjacent images in the satellite joint surveillance scenario, the detection frames of moving targets may not overlap in the two images. In this case, the original DeepSORT cannot create a tracker for moving targets and cannot track such targets in the results. Therefore, it is not used as a comparison in the experimental results. However, the improved DeepSORT tracking algorithm of the present invention can create a tracker for moving targets without affecting stationary targets, and the effect is as follows: Figure 4 As shown, the moving targets are highlighted, and it can be seen that the creation of the tracking list has been completed and the corresponding tracking ID number has been assigned, which also proves the effectiveness of the method of the present invention in the satellite joint monitoring mission.

[0110] Figure 5 The quantitative results of the improved tracking algorithm of the present invention and the original DeepSORT method are compared. The smaller the MOTP index value, the higher the accuracy. Since MOTP itself is used to measure the position difference between the tracking frame and the detection frame, it is more determined by the detection result, so the difference is not obvious. By observing the other indicators, it can be found that the method of the present invention has more than doubled the accuracy in the application scenario of satellite joint monitoring based on the existing DeepSORT, and can achieve continuous tracking of aircraft targets in complex scenarios of satellite joint monitoring. The effect is as follows Figure 6 As shown, Figure 6 The two figures correspond to Figure 2 In the first two frames of the image, the aircraft target has a large movement distance, while Figure 6 In the two images, the target still achieves correct ID assignment after moving long distances in two consecutive images, completing the target tracking task.

[0111] To address the problem of tracking aircraft targets at airports under joint surveillance by low-orbit satellites, a two-stage approach combining a modified YOLOv7 and DeepSORT is proposed. By integrating the CBAM attention module into the YOLOv7 network, this approach improves the detection accuracy of small aircraft targets in satellite remote sensing imagery. Simultaneously, the DeepSORT algorithm is improved to ensure continuous tracking of aircraft targets in long-duration image series. Experimental validation of the algorithm using remote sensing imagery from the Jilin-1 satellite demonstrates that the proposed method outperforms other commonly used methods and can integrate data from multiple satellites at different time points, enabling continuous monitoring of airport activities. This demonstrates the enormous potential for future large-scale low-orbit satellite constellations in target surveillance.

[0112] Any process or method described in the flowchart of the present invention or in other ways herein can be understood as representing a module, segment or portion of code including one or more executable instructions for implementing specific logical functions or process steps, which can be implemented in any computer-readable medium for use by an instruction execution system, device or apparatus. The computer-readable medium can be any medium that stores, communicates, propagates or transmits a program for use by an execution system, device or apparatus, including read-only memory, magnetic disk or optical disk, etc.

[0113] Throughout this specification, reference to terms such as "embodiment" and "example" indicates that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, those skilled in the art may combine or integrate different embodiments or examples described in this specification, as well as features therein, without creating any inconsistency.

[0114] Although the above content has shown and described the embodiments of the present invention, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present invention. Ordinary technicians in this field can perform update operations such as changes, modifications, replacements and variations on the above embodiments within the scope of the present invention.

Claims

1. A method for tracking airport aircraft targets based on low-orbit satellite joint monitoring, characterized in that: The method comprises the following steps: S100. Data preprocessing: Perform image processing on satellite remote sensing images of the corresponding airport area acquired through joint surveillance by multiple satellites, so that the satellite remote sensing images captured by different satellites are mapped into a unified geographic coordinate system, achieving a unique mapping of pixels to geographic locations; S200. Target Detection: Identify aircraft targets in satellite remote sensing images using the YOLOv7 target detection network algorithm, automatically identifying and locating targets from image data. S300. Object Tracking: Utilizes a two-stage object tracking method based on YOLOv7 and DeepSORT, achieving continuous tracking of multiple objects through the object tracking algorithm network combined with object detection result slices. In step S200, the YOLOv7 network uses the darknet deep learning framework to implement end-to-end training on the input image. During prediction, the network outputs a predicted bounding box, which contains five prediction parameters: x, y, w, h, and confidence, where x and y represent the coordinates of the target detection box, w and h are the width and height of the target detection box, and confidence is the confidence level. Integrate the convolutional attention mechanism module (CBAM) into YOLOv7 to enhance the model's perception capabilities. CBAM sequentially integrates two attention mechanisms: channel attention and spatial attention. Global spatial information is obtained through global average pooling and global maximum pooling operations, and then the dependencies between channels are learned through a shared network layer to generate a channel attention map. The calculation method is: （1） in, is the input feature map, is the channel attention map, is the sigmoid activation function, represents a multilayer perceptron; The spatial attention module is implemented by maximizing and averaging the feature map to generate a spatial attention map that highlights important spatial areas. The formula for the spatial attention map is as follows: （2） in, Indicates that a 7×7 convolution kernel is used for convolution to merge the feature maps of average pooling and maximum pooling to generate a spatial attention map. .

2. The method for tracking airport aircraft targets based on low-orbit satellite joint monitoring according to claim 1, characterized in that: Step S100 specifically includes: S101. Data Acquisition: Utilize remote sensing satellite constellation joint surveillance to obtain original remote sensing data images corresponding to the airport area; S102. Image Registration: Process remote sensing images from different satellites and ensure that they are correctly aligned in geographic space. Through image registration technology, images taken by different satellites are mapped to a unified geographic coordinate system to achieve a unique mapping of pixels to geographic locations.

3. The method for tracking airport aircraft targets based on low-orbit satellite joint monitoring according to claim 1, characterized in that: The prediction box with the highest confidence is selected as the final detection result through non-maximum suppression.

4. The method for tracking airport aircraft targets based on low-orbit satellite joint monitoring according to claim 1, characterized in that: Step S300 specifically includes: S301. Calculate the time difference: Different satellite remote sensing images have different shooting times, and the time difference between adjacent images is not fixed. Calculate and give the time difference between adjacent images; S302. Feature extraction: First, the target image slice is captured according to the detection frame, and its appearance and size information is retained; then, the boundary filling and depth feature extraction are performed on each target slice; S303. State prediction: Based on the time difference between images, the state prediction matrix is ​​used to predict the state of the aircraft target at the next moment, including position and speed; the accuracy and robustness of the tracking process are ensured by setting relevant thresholds.

5. The method for tracking airport aircraft targets based on low-orbit satellite joint monitoring according to claim 4, characterized in that: Use the DeepSORT algorithm to achieve target matching and tracking, as follows: Time variable introduction: based on the original DeepSORT When a new satellite image is input, the Kalman filter will get a new As the time difference, a new state update matrix is ​​generated; Mahalanobis distance matching optimization: The Mahalanobis distance limit is increased to retain more predicted position boxes, preventing the target from moving far in the image while the predicted box only moves a short distance from its original position. Slice-scale feature preservation: To preserve more target size and appearance information in the obtained feature vector, the target slices are padded. This preserves the size and original appearance of the target aircraft while maintaining the same input image size. This ensures that when calculating the cosine distance matrix, the cosine distance of the same target is smaller than the cosine distance of other targets. Lowering the cosine distance threshold helps to filter out paired targets. IOU matching optimization: The IOU matching threshold is enlarged so that the algorithm does not delete moving targets, thereby ensuring the tracking recall rate.

6. An airport aircraft target tracking system based on low-orbit satellite joint monitoring, characterized in that: The system includes a data preprocessing module, a target detection module and a target tracking module, which are used to implement the airport aircraft target tracking method based on low-orbit satellite joint monitoring according to any one of claims 1 to 5.

Images