Unmanned aerial vehicle long-time tracking method and system

Abstract

The invention discloses an unmanned aerial vehicle long-time tracking method and system. The method comprises the following steps: S1, designing a three-level complementary algorithm framework based on detection YOLO, multi-target tracking DeepSort and single-target tracking ECO; s2, in long-time tracking of the unmanned aerial vehicle, real-time target detection is realized through a YOLO algorithm, and a detection result is output according to frames; s3, a DeepSort algorithm carries out filtering processing on a detection result based on a historical track, a unique ID is distributed to each real target, and multi-target track information is output; s4, the ECO algorithm aligns the multi-target trajectory information with the ECO tracking result; and S5, if the tracked target is lost by the ECO algorithm, executing a hovering tracking strategy, keeping the pod pointing to the known absolute position until the target is detected again, and recovering closed-loop tracking. The invention discloses an unmanned aerial vehicle long-time tracking method and system, which can effectively deal with a complex scene where a target is located, avoid tracking drift or loss caused by limitation of a single algorithm, and realize continuous and stable tracking of the target.

Description

Technical Field

[0001] The present invention relates to a drone tracking method and a system thereof. Background Art

[0002] With the rapid development of drone and computer vision technologies, air-to-ground target tracking based on drone platforms has become a research focus in the field of intelligent perception. Compared to traditional short-term tracking systems, long-term tracking systems must achieve continuous and stable tracking of mission targets in complex and dynamic environments. Tracking failures can lead to mission interruption, data loss, and even security risks. Therefore, long-term tracking capability is a core indicator of the reliability and intelligence level of drone systems.

[0003] Currently, long-term robust tracking of air-to-ground image targets using drones faces multiple technical and engineering challenges. Despite extensive research on air-to-ground target tracking by drones, existing approaches face significant limitations in practical system deployment. For one thing, most studies remain limited to theoretical verification in static scenarios, lacking robustness assessments for the complex environments of highly dynamic drone flight. Furthermore, existing technologies primarily focus on optimizing image algorithms, lacking research on integrating drone platform resources from a system-level perspective. This results in ineffective recovery mechanisms after target loss, and a lack of a systematic solution for long-term tracking tasks.

[0004] From the above, we can see that it is necessary to provide a long-term tracking method for drones to solve the following problems:

[0005] (1) The problem of coordinated optimization of the real-time performance of multiple UAV tasks and limited computing power. Existing UAVs need to handle multiple tasks simultaneously, such as flight control, target detection, tracking, and communication. However, they are limited by the GPU / CPU computing power and power consumption of lightweight hardware. The traditional single-threaded or fixed resource allocation mode is prone to problems such as resource preemption between tasks, computing delays, and the inability to ensure the real-time performance of key tasks.

[0006] (2) The problem of complementary fusion caused by the lack of robustness of a single algorithm in long-term tracking. The traditional single-target tracking algorithm has problems such as poor adaptability to target deformation, long-term tracking model drift and cumulative error. The multi-target tracking algorithm relies on detection results and has limited ability to handle complex occlusions. The target detection algorithm has defects such as high false alarm rate and lack of correlation between frames.

[0007] (3) The problem of being unable to efficiently retrieve a target based on spatiotemporal continuity after it is lost. In existing technologies, when a target is lost in an image due to long-term occlusion, perspective switching, etc., traditional algorithms rely on short-term appearance feature matching and lack the ability to continuously lock the target's three-dimensional spatial position. Summary of the Invention

[0008] An object of the present invention is to solve at least the above problems and / or disadvantages and to provide at least the advantages which will be described hereinafter.

[0009] In order to achieve these objectives and other advantages of the present invention, a method for long-term tracking of a drone is provided, comprising:

[0010] S1. Design a three-level complementary algorithm framework based on detection YOLO-multi-target tracking DeepSort-single target tracking ECO;

[0011] S2. In the long-term tracking of drones, the YOLO algorithm in the three-level complementary algorithm framework is used to achieve real-time target detection and output the detection results frame by frame;

[0012] S3, the DeepSort algorithm in the three-level complementary algorithm framework filters the detection results based on historical trajectories, assigns a unique ID to each real target, and outputs multi-target trajectory information;

[0013] S4. The ECO algorithm in the three-level complementary algorithm framework uses a three-level matching method to align the multi-target trajectory information with the ECO tracking results. If the match is successful, the DeepSort trajectory is used to correct the ECO state, otherwise the tracked target is marked as potentially lost;

[0014] S5. If the ECO algorithm fails to match successfully for multiple consecutive frames and the tracking target is not redetected, a loss response is triggered, and a circling tracking strategy is executed with the last known absolute position of the target as the center, and the pod is kept pointing to the known absolute position until the target is redetected and closed-loop tracking is resumed.

[0015] Preferably, S1 also includes: building a multi-task asynchronous parallel long-term tracking architecture system, the three-level complementary algorithm framework is designed at the image algorithm level of the long-term tracking architecture system, and at the system level of the long-term tracking architecture system, a circling tracking strategy based on geographic location is designed.

[0016] Preferably, in S2, the YOLO algorithm reduces the probability of missed detection by lowering the detection confidence threshold.

[0017] Preferably, in S3, the DeepSort algorithm establishes a feature vector corresponding to the target by extracting the apparent features, and then compares the features between different frames to determine whether the targets in different frames are the same target;

[0018] The DeepSort algorithm predicts the target's motion trajectory through the Kalman filter motion model to complete the filtering of the detection results.

[0019] Preferably, in S4, the ECO algorithm determines the tracking target using the initialization candidate target information provided in the multi-target trajectory information during the tracking initialization phase, and continuously updates the state of the tracking target during the tracking process, while initializing a tracker for the tracking target and regularly updating the tracker template.

[0020] Preferably, in S4, the three-level matching refers to target category matching, ID matching, and DIOU distance matching.

[0021] Preferably, in S4, the hovering tracking strategy refers to performing real-time calculation based on the position information of the UAV and the tracking target after determining that the target is blocked or image tracking fails, so as to obtain the line of sight direction of the tracking target relative to the UAV and the yaw angle and pitch angle of the tracking target relative to the optoelectronic pod;

[0022] By controlling the pod servo through the yaw and pitch angles, the camera optical axis points to the absolute position before the target is lost, thereby enabling the pod to continuously lock and track the target's geographic location, waiting for the target to reappear in the image screen and complete the re-detection of the target.

[0023] Preferably, if the optoelectronic payload device is integrated with a laser rangefinder, the position information of the tracking target is represented by the following formula:

[0024]

[0025] In the above formula, is the absolute position of the tracking target in the ground reference system, To track the target's unit sight direction in the camera coordinate system, is the transformation relationship from camera system to machine system at the current moment, is the coordinate system from the camera to the ground reference system, Dist is the distance between the tracking target and the camera;

[0026] Otherwise, the position information of the tracking target is represented by the following formula:

[0027]

[0028] In the above formula, is the coordinate of the target sight direction in the ground reference system, and Height is the current height of the drone, p z is a vector The component on the z-axis.

[0029] Preferably, in S4, after triggering the loss response, an adaptive initialization strategy may be triggered to reselect a suitable target from the initialization candidate target information provided in the multi-target trajectory information to initialize the tracker, so as to resume tracking of the target;

[0030] Among them, the reinitialization of the tracker must meet the following three conditions:

[0031] Condition 1: The time interval from the last initialization reaches the set threshold;

[0032] Condition 2: The multi-target tracking results match the single-target tracking results successfully;

[0033] Condition 2: Based on the matching results, the tracking results are detected to be drifting.

[0034] A system is applied to a long-term tracking method for a UAV, comprising: a target detection thread, a multi-target tracking thread, a single target tracking thread, a flight control thread, a comprehensive thread, and an RTSP thread working asynchronously and in parallel;

[0035] The single target tracking thread includes a target tracking module, a tracking decision module and a target matching module.

[0036] The present invention has at least the following beneficial effects:

[0037] First, the present invention can significantly improve the long-term tracking stability: by complementing the advantages of detection (YOLO), multi-target tracking (DeepSort) and single-target tracking (ECO), it can effectively cope with complex scenarios such as target appearance changes, occlusions, and scale mutations, avoid tracking drift or loss caused by the limitations of a single algorithm, and achieve continuous and stable tracking of the target.

[0038] Secondly, the present invention can achieve efficient utilization of computing resources: the multi-task asynchronous parallel architecture allocates GPU / CPU resources to resolve the contradiction between the limited computing power of drones and the real-time performance of multi-tasks. While ensuring the real-time performance of key tasks such as flight control and pod control, it improves the efficiency of target detection and tracking and makes full use of computing resources.

[0039] Third, the present invention can enhance the recovery capability after the target is lost: based on the target absolute position solution and the UAV geographic location circling tracking strategy, when the target is blocked or image tracking fails, the target spatial position is continuously locked by adjusting the pod viewing angle or the UAV flight path, which significantly improves the probability of recovering the lost target and reduces the risk of mission interruption.

[0040] Fourthly, the present invention has strong adaptability in complex scenarios: the system integrates algorithm-level robustness and system-level maneuverability, can maintain stable tracking in dense obstacle environments and highly dynamic flight conditions, and meet the needs of multiple scenarios such as counter-terrorism and search and rescue.

[0041] Other advantages, objectives and features of the present invention will be reflected in part from the following description and will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a schematic diagram of the architecture of the multi-task asynchronous parallel long-term tracking system for unmanned aerial vehicles of the present invention;

[0043] Figure 2 This is a visualization diagram of the long-term tracking results of the present invention in different scenarios. DETAILED DESCRIPTION

[0044] The present invention will be described in further detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0045] The present invention proposes a long-term tracking system for air-to-ground image targets for UAVs. By leveraging the complementary advantages of detection and tracking, as well as hovering tracking of the UAV's geographic location, the system effectively improves the long-term tracking capability of the swarm UAV system for mission targets in highly dynamic and complex scenarios.

[0046] Specifically, the long-term tracking system of the present invention proposes a design of a multi-task asynchronous parallel long-term tracking architecture. This architecture achieves efficient coordination between tasks such as detection, tracking, and control by dynamically allocating GPU / CPU resources, effectively alleviating the contradiction between limited onboard computing resources and the real-time requirements of multiple tasks. At the image algorithm level, a complementary detection and tracking algorithm framework is constructed. By integrating the high detection rate of YOLOv10, the multi-target association capability of DeepSort, and the real-time tracking advantage of ECO, the long-term tracking capability of the algorithm is enhanced. At the UAV system level, a circling tracking strategy based on geographic location is designed. By comprehensively utilizing the status information and maneuverability of the UAV, it breaks through the limitations of the camera's field of view and provides system-level support for continuous tracking tasks.

[0047] Furthermore, the multi-task asynchronous parallel long-term tracking architecture of the UAV of the present invention adopts a multi-task heterogeneous parallel architecture to more efficiently utilize the CPU, GPU and other computing resources of the onboard processor, avoiding the situation where computing resources are idle due to waiting for other tasks; in addition, multi-task asynchronous parallelism can improve the reliability and fault tolerance of the UAV system while ensuring the real-time performance of tasks such as UAV flight control and pod control.

[0048] like Figure 1As shown, in the long-term tracking system of the present invention, the long-term tracking architecture includes: a target detection thread, a multi-target tracking thread, a single target tracking thread, a flight control thread, a comprehensive thread and an RTSP thread, and the six threads work asynchronously and in parallel.

[0049] Among them, the long-term tracking architecture integrates algorithm-level complementary advantages and system-level collaborative control. The core structure serving target tracking includes three threads: target detection thread, multi-target tracking thread, and single-target tracking thread. Efficient collaboration is achieved through dynamic resource allocation and data interaction.

[0050] In the specific implementation, based on the existing YOLO series target detection, DeepSort multi-target tracking and ECO single target tracking, this paper uses the respective strengths of the algorithms to complement each other and constructs the main architecture of the image target long-term tracking system. When designing the main architecture, we first analyze the respective advantages and disadvantages of YOLO target detection, DeepSort multi-target tracking and ECO single target tracking: a) YOLO mainly uses GPU for calculation. The advantages are high target recognition ability, inter-frame irrelevant and no cumulative error. The disadvantage is that there is a certain false alarm rate; b) DeepSort needs to use GPU to extract deep features and implement inter-frame matching and filtering based on CPU. The advantage is that it takes into account the appearance characteristics and kinematic model, which can be used to filter the false alarm of the detector and provide initial values ​​for ECO. The disadvantage is that it needs to rely on the detection results; c) ECO mainly uses CPU processor for calculation. The advantage is that the algorithm is fast and suitable for closed-loop control. The disadvantage is that it is not adaptable to target deformation, there is cumulative error, and model drift will occur in long-term tracking.

[0051] Based on the above analysis, we use the YOLO target detection algorithm as the front end and improve the target detection capability by reducing the confidence level. The target detection results are used as the input of the DeepSort algorithm. The appearance characteristics and kinematic models in the DeepSort algorithm can filter out false alarms and unstable detection results in the detection results, and compensate for the impact of reducing the target detection confidence level while retaining the target detection capability. The ECO algorithm is used as a single target tracker, and the multi-target tracking results are used for initialization to perform real-time closed-loop tracking of the target. The ECO tracker is corrected using the multi-target tracking results according to the set rules to prevent drift. In addition, the improvement of target detection performance will also promote the multi-target tracking effect. Figure 2 The closed-loop tracking results of the UAV on the ground target in different scenarios ( Figure 2(#Frame t* represents the number of image frames, and a red five-pointed star indicates the task target) It can be seen that the long-term tracking system designed in this invention maintains the consistency of multi-target tracking IDs while tracking the task target in a closed loop; maintains high tracking performance in both ordinary road scenes and wild scenes; and can cope with difficult scenarios such as size changes, target rotation, dense targets, and interfering targets.

[0052] Furthermore, the adaptive initialization strategy of the single target tracker is also taken into consideration when designing the main architecture. In order to fully utilize the respective advantages of detection, multi-target tracking and single target tracking algorithms and complement each other, we designed an automatic initialization and reinitialization strategy for the single target tracker: First, the long-term tracking system uses the multi-target tracking results as candidate targets, and can design automatic selection criteria for task targets based on factors such as detection confidence, target size, target category, and distance from the image center. During the actual flight process, the UAV system can autonomously select the task target for closed-loop tracking according to the pre-set criteria. Secondly, as Figure 1 As shown in the figure, a target matching module is designed in the single target tracking thread. The multi-target tracking results and the single target tracking results are matched by target category matching, multi-target tracking ID matching, and DIOU distance matching, and finally a matching result is obtained. In addition, the reinitialization of the tracker must meet the following three conditions:

[0053] First, the time interval from the last initialization reaches the set threshold;

[0054] Second, the multi-target tracking results are successfully matched with the multi-target tracking results;

[0055] Third, the tracking result drift is monitored based on the matching results.

[0056] Since single-target tracking and multi-target tracking are in asynchronous parallel threads, the time asynchrony of each thread needs to be considered when matching the DIOU (DIOU is a loss function in target detection, mainly used to measure the distance between the predicted box and the real box) distance and judging whether the tracking results have drifted. Specifically, the DIOU distance requirement should be appropriately relaxed during DIOU distance matching, because there is a small deviation in the center position of the target box at the two moments; when judging whether the tracking result has drifted, because the match has been successful, the influence of the deviation in the center position of the box is removed, and only the change in the posture, area, etc. of the target is considered. The tracking can be judged by first moving the centers of the two boxes to the same, and then calculating the IOU of the two boxes to determine whether the tracking has drifted.

[0057] From the above content, it can be seen that the main framework of the present invention fully utilizes the respective advantages of detection, multi-target tracking, and single target tracking algorithms to complement each other, which can not only improve the target detection capability and multi-target tracking capability, but also prevent the model drift of the target tracking algorithm, thereby realizing long-term tracking of the target at the image algorithm level.

[0058] In specific implementation, the functions of each thread are as follows:

[0059] The target detection thread primarily utilizes the onboard processor GPU resources for target detection. In practical applications, this thread is a critical step in the system's initial target identification. It uses the YOLO algorithm to achieve real-time target detection. To better adapt to complex situations that may arise in real-world applications, we have lowered the detection confidence threshold. This reduces the probability of missed detections, enabling more possible targets to be detected. The subsequent multi-target tracking thread can filter out these false alarms.

[0060] After completing target detection, this thread outputs the target's bounding box and category information. The bounding box accurately identifies the target's location in the image, while the category information indicates the target's category. This information serves as input data for the subsequent multi-target tracking thread.

[0061] The second multi-target tracking thread tracks multiple targets based on the detection results and passes them to the tracking thread. In practical applications, the multi-target tracking thread is primarily responsible for further processing the detection results output by the target detection thread to achieve inter-frame target association and stable trajectory tracking. This thread uses the DeepSort algorithm.

[0062] Appearance feature extraction is a key component of the DeepSort algorithm. It extracts and analyzes the appearance features of a target to create a unique feature vector for each target. These feature vectors can be compared across frames to determine whether the targets in different frames are the same. The Kalman filter motion model is used to predict the target's trajectory, predicting its likely position in the next frame based on its position and velocity in the current frame. By combining these two methods, the DeepSort algorithm can accurately associate targets across frames even when the target is occluded or its appearance changes.

[0063] After receiving data from the target detection thread, this thread first filters the detection results using surface features and the Kalman filter motion model to remove targets that may be false alarms. It then assigns a unique ID to each real target, enabling continuous identification and tracking during subsequent tracking. Finally, it outputs stable multi-target trajectories, which serve as a reference for initializing candidate targets and trajectory corrections in the single-target tracking thread.

[0064] 3. The single target tracking thread is the core part of the entire system for accurately tracking specific targets. It consists of three sub-modules: the target tracking module, the target matching module, and the tracking decision module. The target tracking module is mainly responsible for initializing the tracker, updating the target status, and updating the template. The target matching module is responsible for matching the single target tracking results with the multi-target tracking results. The tracking decision module mainly responds to the control instructions of the integrated thread, determines whether the tracking target is lost, and whether reinitialization is required.

[0065] Specifically, the target tracking module implements real-time closed-loop tracking based on the ECO algorithm. During the tracker initialization phase, the target to be tracked is determined based on the initial candidate target information provided by the multi-target tracking thread, and a tracker is initialized for that target. During the tracking process, the target's state is continuously updated, and its position in the next frame is predicted based on its position and appearance in the current frame. Furthermore, to adapt to changes in the target's appearance, the tracker's template is regularly updated, enabling the tracker to better track the target.

[0066] The main function of the target matching module is to align the multi-target tracking results output by the multi-target tracking thread with the tracking results of the single-target tracking thread through three-level matching: target category, DeepSortID, and DIOU distance. First, a preliminary match is performed based on the target category information to ensure that the matched targets belong to the same category. Then, further matching is performed using the unique ID assigned to the target by DeepSort. Only targets with the same ID are considered to be the same target. Finally, the degree of target position matching is determined by calculating the DIOU distance. The DIOU distance comprehensively considers the degree of overlap of the target bounding boxes and the distance between the center points, and can more accurately measure the similarity between two targets. Through this three-level matching, the accuracy of target matching can be effectively improved, ensuring that the single-target tracking thread can accurately track the correct target.

[0067] The tracking decision module is responsible for determining whether the target is lost and triggering the adaptive initialization strategy when necessary. The tracking decision module determines whether the target is lost based on the matching results of the target matching module and the tracking confidence of the tracker.

[0068] If the target is lost, the tracking decision module triggers an adaptive initialization strategy. Based on the current situation, the adaptive initialization strategy re-initializes the tracker by selecting a suitable target from the candidate targets provided by the multi-target tracking thread, or attempts to search in areas where the target is likely to appear to resume tracking. This allows the system to quickly resume tracking if the target is lost, improving system robustness and reliability.

[0069] Furthermore, during the UAV air-to-ground target tracking process, due to the complexity of ground targets and the environment, difficult scenarios such as target obscuration or hiding may occur. In such situations, especially when the obscuration lasts for a long time, existing long-term target tracking algorithms are still unable to cope with it. This invention improves the probability of the UAV recovering a lost target from the UAV system level by calculating the absolute position of the target and tracking the UAV's geographical location in a hovering manner. The specific operation method is as follows:

[0070] 1. Absolute positioning of ground targets

[0071] According to the image target tracking results, the unit line of sight direction of the target in the camera coordinate system can be calculated The transformation relationship from the camera system to the aircraft system at the current moment is: The specific value can be calculated from the position and attitude parameters of the UAV. If the optoelectronic payload is integrated with a laser rangefinder and measures the distance Dist from the target to the camera, the absolute position of the target in the ground reference system is It can be expressed as:

[0072]

[0073] If there is no laser rangefinder, it can be assumed that the target is located on a flat ground with a height of zero, and the rough position of the target can be solved based on the trigonometric relationship. First, the coordinate representation of the target's sight direction in the ground reference system can be obtained by transformation Then the absolute position of the target in the ground reference system can be expressed as:

[0074]

[0075] Where Height is the current height of the drone, p z is a vector Component along the z-axis.

[0076] 2. Drone geographic location tracking

[0077] According to the spatiotemporal continuity of physical events, when a target disappears from an image due to occlusion or other factors, its absolute position in three-dimensional space is still near its position before it was lost, and it will most likely reappear near the lost position, but it cannot be observed from the current image's line of sight. Therefore, after determining that the target is occluded or image tracking fails, the target's line of sight relative to the drone can be calculated in real time based on the position information of the drone and the target, and the yaw and pitch angles of the target relative to the optoelectronic pod can be further calculated. Based on these angles, the pod servo can be controlled so that the camera's optical axis points to the target's absolute position, thereby achieving continuous lock-on tracking of the target's geographic location, waiting for the target to reappear in the image.

[0078] Furthermore, the system can control the drone to hover and track the target's absolute position, providing the camera with different viewing angles and increasing the probability of retrieving the target. When the target reappears, the image detection and tracking algorithm can quickly retrieve the lost target, enabling long-term tracking of the target.

[0079] Fourth, the flight control thread is mainly responsible for collecting drone sensor information and controlling the drone flight based on the sensor information;

[0080] 5. The RTSP thread is mainly responsible for the image encoding of the detection processing results and RTSP streaming;

[0081] 6. The integrated thread is mainly responsible for the communication between the onboard processor and various onboard hardware devices, including the communication with the pod, flight controller and communication link equipment. At the same time, this thread also needs to complete the information integration processing, UAV self-residence task allocation, target positioning solution and pod closed-loop tracking control of the collected information.

[0082] The workflow of the system of the present invention includes:

[0083] 1. Initialization phase

[0084] The integration thread receives mission instructions from the ground station, determines the target category, and initiates the target detection and multi-target tracking threads. The multi-target tracking thread screens candidate targets based on the detection results, automatically selects the primary target based on detection confidence, target size, and distance from the image center, and triggers the single-target tracking thread to initialize the ECO tracker.

[0085] 2. Real-time tracking stage

[0086] Each time the target detection thread outputs a detection result, the multi-target tracking thread filters false alarms based on historical trajectories and outputs a stable target ID and position. The single-target tracking thread uses the target matching module to align the ECO tracking results with the DeepSort trajectory. If a match is successful, the DeepSort trajectory is used to correct the ECO state to prevent model drift. If a match fails, the tracking decision module marks the target as "potentially lost."

[0087] 3. Target loss recovery phase

[0088] If single-target tracking fails to match multiple target trajectories for multiple consecutive frames and the detection thread fails to re-detect the target, a "loss response" is triggered: the integration thread instructs the drone to execute a circling tracking strategy centered on the target's last known absolute position, while the pod maintains its position and expands its field of view. Once the target is re-detected, the ECO tracker is immediately reinitialized to resume closed-loop tracking.

[0089] The innovative effects of the present invention are:

[0090] 1) Heterogeneous parallel architecture innovation: Design a multi-task asynchronous parallel long-term tracking architecture to achieve efficient coordination of detection, tracking, control and other tasks, break through the resource bottleneck of traditional single-threaded processing, and improve the reliability and fault tolerance of the UAV system.

[0091] 2) Algorithm fusion and complementary advantages: Construct a three-level complementary algorithm framework of "detection-multi-target tracking-single target tracking", utilizing the high detection rate of YOLOv10, the target association capability of DeepSort and the real-time tracking advantages of ECO, and through adaptive initialization and correction strategies, make up for the defects of a single algorithm and form a full-process robust tracking capability.

[0092] 3) System-level target retrieval strategy: A geolocation-based circling tracking strategy is proposed, combining the target's absolute position solution with the UAV's maneuverability control. This strategy breaks through the camera's field of view limitations from the perspective of hardware and algorithm collaboration, providing systematic support for long-term tracking.

[0093] 4) Lightweight and real-time optimization: Targeting the lightweight hardware characteristics of drones, task scheduling optimization and algorithm parameter adaptation are used within limited computing power to achieve a balance between high performance and low power consumption, thus meeting the needs of project implementation.

[0094] 5) Adaptive decision-making mechanism: Design a single target tracker initialization and reinitialization strategy, automatically calibrate the tracking results through multi-dimensional matching (target category, ID, DIOU distance) and drift detection algorithm, and improve the system's autonomous decision-making ability.

[0095] The above solution is only an illustration of a preferred embodiment, but is not limited thereto. When implementing the present invention, appropriate replacements and / or modifications can be made according to user needs.

[0096] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and exemplary embodiments. They can be applied to a variety of fields suitable for the present invention. Further modifications will be readily apparent to those skilled in the art. Therefore, the present invention is not limited to the specific details and illustrations shown and described herein without departing from the general concept defined by the claims and their equivalents.

Claims

1. A method for long-term tracking of a drone, characterized in that: include: S1. Design a three-level complementary algorithm framework based on detection YOLO-multi-target tracking DeepSort-single target tracking ECO; S2. In the long-term tracking of drones, the YOLO algorithm in the three-level complementary algorithm framework is used to achieve real-time target detection and output the detection results frame by frame; S3, the DeepSort algorithm in the three-level complementary algorithm framework filters the detection results based on historical trajectories, assigns a unique ID to each real target, and outputs multi-target trajectory information; S4. The ECO algorithm in the three-level complementary algorithm framework uses a three-level matching method to align the multi-target trajectory information with the ECO tracking results. If the match is successful, the DeepSort trajectory is used to correct the ECO state, otherwise the tracked target is marked as potentially lost; S5. If the ECO algorithm fails to match successfully for multiple consecutive frames and the tracking target is not redetected, a loss response is triggered, and a circling tracking strategy is executed with the last known absolute position of the target as the center, and the pod is kept pointing to the known absolute position until the target is redetected and closed-loop tracking is resumed.

2. The long-term tracking method for a UAV according to claim 1, wherein: S1 also includes: building a multi-task asynchronous parallel long-term tracking architecture system, the three-level complementary algorithm framework is designed at the image algorithm level of the long-term tracking architecture system, and at the system level of the long-term tracking architecture system, a circling tracking strategy based on geographic location is designed.

3. The long-term tracking method for a UAV according to claim 1, wherein: In S2, the YOLO algorithm reduces the probability of missed detection by lowering the detection confidence threshold.

4. The long-term tracking method for a UAV according to claim 1, wherein: In S3, the DeepSort algorithm creates a feature vector corresponding to the target by extracting the apparent features, and then compares the feature vectors between different frames to determine whether the targets in different frames are the same target. The DeepSort algorithm predicts the target's motion trajectory through the Kalman filter motion model to complete the filtering of the detection results.

5. The long-term tracking method for a UAV according to claim 1, wherein: In S4, the ECO algorithm determines the tracking target based on the initialization candidate target information provided in the multi-target trajectory information during the tracking initialization phase, and continuously updates the state of the tracking target during the tracking process. At the same time, a tracker is initialized for the tracking target, and the tracker template is regularly updated.

6. The long-term tracking method for a UAV according to claim 1, wherein: In S4, the three-level matching refers to target category matching, ID matching, and DIOU distance matching.

7. The long-term tracking method for a UAV according to claim 1, wherein: In S4, the hovering tracking strategy refers to performing real-time calculation based on the position information of the UAV and the tracking target after determining that the target is blocked or image tracking fails, so as to obtain the line of sight direction of the tracking target relative to the UAV and the yaw angle and pitch angle of the tracking target relative to the optoelectronic pod; By controlling the pod servo through the yaw and pitch angles, the camera optical axis points to the absolute position before the target is lost, thereby enabling the pod to continuously lock and track the target's geographic location, waiting for the target to reappear in the image screen and complete the re-detection of the target.

8. The long-term tracking method for a UAV according to claim 7, wherein: If the optoelectronic payload device is integrated with a laser rangefinder, the position information of the tracked target is represented by the following formula: In the above formula, is the absolute position of the tracking target in the ground reference system, To track the target's unit sight direction in the camera coordinate system, is the transformation relationship from camera system to machine system at the current moment, is the coordinate system from the camera to the ground reference system, Dist is the distance between the tracking target and the camera; Otherwise, the position information of the tracking target is represented by the following formula: In the above formula, is the coordinate of the target sight direction in the ground reference system, and Height is the current height of the drone, p z is a vector The component on the z-axis.

9. The long-term tracking method for a UAV according to claim 1, wherein: In S4, after the loss response is triggered, the adaptive initialization strategy can be triggered to reselect a suitable target from the initialization candidate target information provided in the multi-target trajectory information to initialize the tracker and resume tracking of the target; Among them, the reinitialization of the tracker must meet the following three conditions: Condition 1: The time interval from the last initialization reaches the set threshold; Condition 2: The multi-target tracking results match the single-target tracking results successfully; Condition 2: Based on the matching results, the tracking results are detected to be drifting.

10. A system, applied to the long-term tracking method of a UAV according to any one of claims 1 to 8, characterized in that: include: Asynchronous and parallel working target detection thread, multi-target tracking thread, single target tracking thread, flight control thread, integrated thread and RTSP thread; The single target tracking thread includes a target tracking module, a tracking decision module and a target matching module.

Images