A method and apparatus for tracking a drone
By combining the YOLOv8m target detection model with laser ranging and Kalman filtering to form a three-anchor anti-disturbance tracking mechanism, the problem of target tracking anti-disturbance of UAVs in complex environments is solved, and the UAVs are accurately tracked and stably controlled.
Patent Information
- Application Number
- CN202611212264.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-11
- Publication Date
- 2026-11-10
AI Technical Summary
Existing drone target tracking technologies have poor anti-interference capabilities in complex environments. They are prone to losing targets, especially in scenarios with drone shaking, tree shadows, overlapping targets, or confusing backgrounds. Furthermore, pure visual tracking has limited accuracy, large distance calculation errors, and reduced tracking precision.
The YOLOv8m target detection model is combined with laser ranging and Kalman filtering to construct a three-anchored anti-disturbance tracking mechanism consisting of visual feature anchors, spatial anchors, and motion anchors. The ECO tracker runs in parallel and dynamically fuses visual, laser, and motion data to achieve weighted fusion of multi-source data and occlusion recovery.
It improves the tracking and anti-interference capabilities of UAVs in complex environments, ensures accurate target tracking, constructs a complete tracking closed loop, enhances the synergy of identification, tracking, and tracing, and guarantees the precise control of UAVs.
Smart Images

Figure CN123067553A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a UAV tracking method and apparatus based on target recognition and tracking algorithms. Background Technology
[0002] With the rapid development of the low-altitude economy and drone technology, drone target recognition, tracking, and tracing technologies have been widely applied in outdoor security, daily inspections, operational supervision, and emergency rescue. The core requirement is to achieve accurate identification, stable tracking, and efficient drone tracking of various targets in complex environments. Current target recognition and tracking algorithms mainly include traditional target recognition methods (manual features and classifiers), deep learning methods for target recognition (two-stage detectors, single-stage detectors), single-target tracking, and multi-target tracking algorithms. These algorithms ensure stable target tracking, keeping the target always within the frame. Combined with visual information, the target position is calculated, and the drone's position is controlled via a closed-loop flight control system to achieve autonomous target tracking.
[0003] By installing an onboard computer on a drone to create a hardware environment, relying on visual processing, and using target recognition models such as YoLoV8 as the target recognition module, along with tracking algorithms such as BoT-SORT as the tracking module, the drone achieves target recognition and tracking functions. Finally, by combining visual information with camera models, motion estimation, and coordinate transformation, the relative position between the target and the drone is calculated, and then the drone's position is controlled through a flight control closed loop to achieve target tracking.
[0004] However, pure vision tracking is prone to target loss in scenarios with slight drone shaking, tree occlusion, target overlap, or blurred backgrounds. Furthermore, pure vision tracking relies on strong priors, limiting accuracy. It requires knowledge of the target's true physical dimensions to accurately calculate depth; changes in target pose disrupt the mapping between pixel size and true size, increasing distance calculation errors and reducing tracking accuracy. Additionally, feature matching between left and right frames is prone to failure when the target is occluded or moving rapidly, leading to interruptions or errors in depth calculation. Summary of the Invention
[0005] In view of the above problems, the present invention provides a drone tracking method and apparatus to overcome or at least partially solve the above problems. It addresses the shortcomings of existing technologies in terms of poor tracking immunity and weak coordination between identification, tracking, and tracing. It enhances tracking immunity in complex environments, effectively addressing scenarios such as jitter, occlusion, and background obfuscation. Deep coordination between identification, tracking, and tracing constructs a complete closed-loop tracking link, ensuring accurate drone tracking.
[0006] This invention provides the following solution: A drone tracking method, comprising: When the detection confidence of the YOLOv8m target detection model reaches a preset threshold, the initialization of visual feature anchors, spatial anchors, motion anchors, and the core module are completed simultaneously. Visual feature anchor initialization includes extracting the feature map output from the last layer of the backbone network of the YOLOv8m target detection model, cropping the feature blocks corresponding to the target region, performing global average pooling and normalization, and storing them as target-specific visual feature anchors. Spatial anchor initialization includes reading the target spatial data output by the laser ranging module as spatial positioning reference anchors. Motion anchor initialization includes calculating the initial motion velocity based on the preliminary target positioning coordinates output by the YOLOv8m target detection model, and initializing the Kalman filter model to predict the motion trajectory for future multiple frames, serving as motion trend anchors. The core module startup includes importing the visual feature anchors into the ECO tracker for initialization and starting the ECO tracker and the Kalman filter to run in parallel. Each frame of image is dynamically fused based on a distance-layered strategy, which integrates the real-time visual tracking position and response score output by the ECO tracker, the real-time spatial coordinates transformed image pixel coordinates output by the laser ranging module, and the motion prediction position output by the Kalman filter based on the motion state of the previous frame, and outputs fused coordinates. When the response score is lower than a preset threshold, the target is determined to be occluded. The visual tracking result is paused, and the system switches to a joint prediction mode of spatial anchor points and motion anchor points, outputting a temporary tracking position. The target coordinates obtained by the last laser ranging before occlusion are used as the center, and a search radius is set according to the straight-line distance between the target and the UAV to form a search area. The visual feature anchor points stored in the initialization phase are called, and the YOLOv8m target detection model detects candidate targets and extracts candidate features within the search area, calculating the similarity between the candidate features and the visual feature anchor points. When the similarity reaches a preset threshold, the target is locked, the data of each anchor point is updated, and the three-anchor fusion tracking is resumed. If a predetermined number of consecutive frames fail to match, the tracking is terminated and the global detection of the YOLOv8m target detection model is restarted. After each frame of image tracking is completed, the visual feature anchor point, the spatial anchor point, and the motion anchor point are updated synchronously.
[0007] Preferably, the global average pooling and normalization process includes: A 256-dimensional feature vector is obtained through global average pooling and then normalized using L2. The target spatial data includes the target latitude and longitude. Target height and the straight-line distance between the target and the drone .
[0008] Preferably: Calculating the initial velocity and initializing the Kalman filter model, including: Preliminary target location coordinates output by the YOLOv8m target detection model Calculate the target's initial velocity:
[0009] Among them, the initial value frame ; Initialize the Kalman filter model and set the state vector. Process noise covariance Observation noise covariance It can predict the motion trajectory of the next 3 frames.
[0010] Preferably: the response score being lower than a preset threshold includes a response score less than 0.6; the similarity reaching a preset threshold includes a cosine similarity greater than or equal to 0.85; and the consecutive predetermined number of frames includes 10 consecutive frames.
[0011] Preferably, the conversion process of the real-time spatial coordinates output by the laser ranging module into image pixel coordinates is based on the camera's participation in the UAV's attitude, realizing the mapping between spatial data and the pixel domain.
[0012] Preferably, the distance layering strategy and the calculation method of the fused coordinates are as follows: When the straight-line distance D between the target and the drone is less than 50 meters, it is considered a close-range scene, and the visual weight is determined as follows. Laser weight is Motion weights are fusion coordinates ;like Then YOLOv8m local re-detection will be initiated, only when... The surrounding 200×200 pixel area is detected to calibrate the ECO tracking template. This is the actual distance after pixel coordinate transformation; When the straight-line distance D is greater than or equal to 50 meters and less than or equal to 1000 meters, it is considered a mid-range scene, and the visual weight is determined as follows. Laser weight is Motion weights are The Kalman filter dynamically adjusts the filter gain coefficient to achieve weighted fusion of the three data. When the straight-line distance D is greater than 1000 meters, it is considered a long-distance scene, and the visual weight is determined as follows. Laser weight is Motion weights are The visual tracking position is corrected based on the spatial coordinates of the laser ranging module. The drift error; like Then temporarily adjust the motion weights. Increase the weight of laser data to suppress inter-frame jump anomalies.
[0013] Preferably, the formula for calculating the temporary tracking position includes: The setting of the search radius based on the straight-line distance between the target and the UAV includes setting the search radius to 50 pixels when the straight-line distance D is less than 500 meters and to 100 pixels when the straight-line distance D is greater than or equal to 500 meters; the similarity includes cosine similarity, and the formula for calculating the cosine similarity includes... .
[0014] Preferably, the visual feature anchor point update includes: Based on the new target features extracted by the YOLOv8m target detection model in the latest frame, a moving average strategy is used for updating, as shown in the formula: That is, 80% of historical features and 20% of new features are retained; the spatial anchor point is updated by directly replacing it with the real-time spatial data output by the laser ranging module; the motion anchor point is updated by using the Kalman filter prediction-update iteration process to fuse coordinates with the current frame. Optimize motion state vector .
[0015] Preferably, the method further includes fusing the YOLOv8m target detection model recognition results, the stable tracking position output by the three-anchor tracking module, and the laser ranging spatial data to generate a real-time target status report. The real-time target status report includes the target position, movement speed, occlusion status, and relative distance information to the UAV. The UAV flight control system receives the target's real-time status report, calculates the relative motion trend based on the difference between the target's latitude, longitude, altitude and the UAV's own position, and generates flight attitude adjustment commands, camera fine-tuning commands and path planning suggestions; After the UAV executes the control command, the YOLOv8m target detection model and the three-anchor tracking module provide real-time feedback on the target's position deviation and tracking stability in the image. If the target deviates from the center of the image by more than 30% or the tracking response score decreases, the flight control system, in conjunction with the laser ranging spatial data, fine-tunes the flight or camera parameters again, forming a closed-loop adjustment of recognition-tracking-positioning-control-feedback.
[0016] A drone tracking device for performing the above-described drone tracking method, the device comprising: The tracking initialization unit is used to simultaneously complete visual feature anchor point initialization, spatial anchor point initialization, motion anchor point initialization, and core module startup when the detection confidence of the YOLOv8m target detection model reaches a preset threshold. The visual feature anchor point initialization includes extracting the feature map output from the last layer of the backbone network of the YOLOv8m target detection model, cropping the feature blocks corresponding to the target region, performing global average pooling and normalization processing, and storing them as target-specific visual feature anchor points. The spatial anchor point initialization includes reading the target spatial data output by the laser ranging module as spatial positioning reference anchor points. The motion anchor point initialization includes calculating the initial motion velocity based on the preliminary target positioning coordinates output by the YOLOv8m target detection model, and initializing the Kalman filter model to predict the motion trajectory in future multiple frames, serving as motion trend anchor points. The core module startup involves importing the visual feature anchor points into the ECO tracker to complete initialization, and starting the ECO tracker and the Kalman filter to run in parallel. The multi-source data real-time fusion unit is used to dynamically fuse the visual tracking position and response score output in real time by the ECO tracker, the image pixel coordinates after real-time spatial coordinate transformation output by the laser ranging module, and the motion prediction position output by the Kalman filter based on the motion state of the previous frame for each frame image based on a distance layering strategy, and output fused coordinates. The occlusion determination and rapid recovery unit is used to determine that the target is occluded when the response score is lower than a preset threshold, suspend the dominance of visual tracking results, switch to the joint prediction mode of spatial anchor points and motion anchor points, and output a temporary tracking position; using the target coordinates obtained by the last laser ranging before occlusion as the center, a search radius is set according to the straight-line distance between the target and the UAV to form a search area; the visual feature anchor points stored in the initialization phase are called, and the YOLOv8m target detection model detects candidate targets and extracts candidate features in the search area, and calculates the similarity between the candidate features and the visual feature anchor points; when the similarity reaches a preset threshold, the target is locked, the data of each anchor point is updated, and the three-anchor fusion tracking is resumed; if a predetermined number of consecutive frames fail to match, the tracking is terminated and the global detection of the YOLOv8m target detection model is restarted; The tracking state update unit is used to synchronously update the visual feature anchor point, the spatial anchor point, and the motion anchor point after each frame of image tracking is completed.
[0017] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This application provides a drone tracking method and apparatus. The method improves tracking immunity through a three-anchor anti-interference mechanism and a multi-source fusion strategy, effectively addressing scenarios such as shaking, occlusion, and background obfuscation. It constructs a complete tracking closed loop, achieving deep collaboration between identification, tracking, laser ranging, and flight control, ensuring accurate drone tracking. Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0019] Figure 1 This is a flowchart of a drone tracking method provided in an embodiment of the present invention; Figure 2 This is a flowchart of the YoloV8m algorithm provided in an embodiment of the present invention; Figure 3 This is a flowchart of the three-anchor anti-disturbance tracking algorithm provided in an embodiment of the present invention; Figure 4 This is an overall flowchart of the tracking method provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a drone tracking device provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of a drone tracking device provided in an embodiment of the present invention. Detailed Implementation
[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.
[0021] See Figure 1 This invention provides a drone tracking method, such as... Figure 1 As shown, the method may include: S101: When the detection confidence of the YOLOv8m target detection model reaches a preset threshold, the initialization of visual feature anchors, spatial anchors, motion anchors, and the core module are completed simultaneously. The initialization of visual feature anchors includes extracting the feature map output from the last layer of the backbone network of the YOLOv8m target detection model, cropping the feature blocks corresponding to the target region, and storing them as target-specific visual feature anchors after global average pooling and normalization. The initialization of spatial anchors includes reading the target spatial data output by the laser ranging module as spatial positioning reference anchors. The initialization of motion anchors includes calculating the initial motion velocity based on the preliminary target positioning coordinates output by the YOLOv8m target detection model, and initializing the Kalman filter model to predict the motion trajectory of future multiple frames as motion trend anchors. The core module startup includes importing the visual feature anchors into the ECO tracker to complete the initialization, and starting the ECO tracker and the Kalman filter to run in parallel. In specific implementation, the embodiments of this application may provide the following: the global average pooling and normalization processing includes: A 256-dimensional feature vector is obtained through global average pooling and then normalized using L2. The target spatial data includes the target latitude and longitude. Target height and the straight-line distance between the target and the drone .
[0022] Furthermore, the initial velocity and the Kalman filter model are calculated, including: Preliminary target location coordinates output by the YOLOv8m target detection model Calculate the target's initial velocity:
[0023] Among them, the initial value frame ; Initialize the Kalman filter model and set the state vector. Process noise covariance Observation noise covariance It can predict the motion trajectory of the next 3 frames.
[0024] The conversion process of the real-time spatial coordinates output by the laser ranging module into image pixel coordinates is based on the camera's participation in the UAV's attitude, realizing the mapping between spatial data and the pixel domain.
[0025] S102: For each frame of image, based on a distance-layered strategy, the visual tracking position and response score output in real time by the ECO tracker, the image pixel coordinates after real-time spatial coordinate transformation output by the laser ranging module, and the motion prediction position output by the Kalman filter based on the motion state of the previous frame are dynamically fused to output fused coordinates; In specific implementation, the embodiments of this application may provide that the response score is lower than a preset threshold, including the response score being less than 0.6; the similarity reaches a preset threshold, including the cosine similarity being greater than or equal to 0.85; and the consecutive predetermined number of frames includes 10 consecutive frames.
[0026] The distance layering strategy and the calculation method of the fused coordinates are as follows: When the straight-line distance D between the target and the drone is less than 50 meters, it is considered a close-range scene, and the visual weight is determined as follows. Laser weight is Motion weights are fusion coordinates ;like Then YOLOv8m local re-detection will be initiated, only when... The surrounding 200×200 pixel area is detected to calibrate the ECO tracking template. This is the actual distance after pixel coordinate transformation; When the straight-line distance D is greater than or equal to 50 meters and less than or equal to 1000 meters, it is considered a mid-range scene, and the visual weight is determined as follows. Laser weight is Motion weights are The Kalman filter dynamically adjusts the filter gain coefficient to achieve weighted fusion of the three data. When the straight-line distance D is greater than 1000 meters, it is considered a long-distance scene, and the visual weight is determined as follows. Laser weight is Motion weights are The visual tracking position is corrected based on the spatial coordinates of the laser ranging module. The drift error; like Then temporarily adjust the motion weights. Increase the weight of laser data to suppress inter-frame jump anomalies.
[0027] S103: When the response score is lower than a preset threshold, it is determined that the target is in an occluded state, the visual tracking result is paused, and the system switches to a joint prediction mode of spatial anchor points and motion anchor points, outputting a temporary tracking position; the target coordinates obtained by the last laser ranging before occlusion are used as the center, and a search radius is set according to the straight-line distance between the target and the UAV to form a search area; the visual feature anchor points stored in the initialization phase are called, and the YOLOv8m target detection model detects candidate targets and extracts candidate features in the search area, calculating the similarity between the candidate features and the visual feature anchor points; when the similarity reaches a preset threshold, the target is locked, the data of each anchor point is updated, and the three-anchor fusion tracking is resumed; if a predetermined number of consecutive frames fail to match, the tracking is terminated and the global detection of the YOLOv8m target detection model is restarted; in specific implementation, the embodiment of this application can provide the calculation formula of the temporary tracking position including The setting of the search radius based on the straight-line distance between the target and the UAV includes setting the search radius to 50 pixels when the straight-line distance D is less than 500 meters and to 100 pixels when the straight-line distance D is greater than or equal to 500 meters; the similarity includes cosine similarity, and the formula for calculating the cosine similarity includes... .
[0028] S104: After each frame of image tracking is completed, the visual feature anchor point, the spatial anchor point, and the motion anchor point are updated synchronously.
[0029] The visual feature anchor point update includes: Based on the new target features extracted by YOLOv8m in the latest frame, a moving average strategy is used for updating, as shown in the formula: That is, 80% of historical features and 20% of new features are retained; the spatial anchor point is updated by directly replacing it with the real-time spatial data output by the laser ranging module; the motion anchor point is updated by using the Kalman filter prediction-update iteration process to fuse coordinates with the current frame. Optimize motion state vector .
[0030] It also includes integrating the YOLOv8m target detection model recognition results, the stable tracking position output by the three-anchor tracking module, and the laser ranging spatial data to generate a real-time target status report. The real-time target status report includes the target position, movement speed, occlusion status, and relative distance information to the UAV. The UAV flight control system receives the target's real-time status report, calculates the relative motion trend based on the difference between the target's latitude, longitude, altitude and the UAV's own position, and generates flight attitude adjustment commands, camera fine-tuning commands and path planning suggestions; After the UAV executes the control command, the YOLOv8m target detection model and the three-anchor tracking module provide real-time feedback on the target's position deviation and tracking stability in the image. If the target deviates from the center of the image by more than 30% or the tracking response score decreases, the flight control system, in conjunction with the laser ranging spatial data, fine-tunes the flight or camera parameters again, forming a closed-loop adjustment of recognition-tracking-positioning-control-feedback.
[0031] The UAV tracking method provided in this application uses the YOLOv8m target detection model as the target recognition module. Relying on the mature feature extraction capability and detection accuracy of the YOLOv8m target detection model, it quickly outputs the preliminary positioning coordinates and target type of the target, providing reliable initialization input for the tracking module.
[0032] A three-anchored anti-disturbance tracking mechanism based on vision, laser ranging, and motion is adopted. With the ECO tracking algorithm as the core, it deeply integrates the target visual features output by the YOLOv8m target detection model, the high-precision spatial data obtained by the laser ranging module, and the motion trajectory predicted by Kalman filtering to construct a three-dimensional anchored constraint tracking framework, thereby improving tracking stability and anti-disturbance.
[0033] It achieves closed-loop tracking control. The flight control system receives target status reports, calculates the relative motion trend based on the difference between the target's latitude, longitude, altitude and the UAV's own position, and generates flight attitude adjustment commands to ensure accurate target tracking. It also builds an identification-tracking-tracking collaborative mechanism to form a complete link.
[0034] The drone tracking method provided in the embodiments of this application will be described in detail below.
[0035] The YOLOv8m target recognition model used in this embodiment is based on a single-stage target detection framework. It extracts multi-scale features through a backbone network, fuses features from different levels through a neck network, and outputs the target's bounding box coordinates, category, and confidence score through a head network. This module leverages its high-precision detection capabilities to provide the tracking module with initial target positions and features, while balancing detection accuracy and real-time performance through TensorRT hardware acceleration optimization.
[0036] like Figure 2 The algorithm flow of the YOLOv8m target recognition model is shown below: 1. Image preprocessing: The input image is scaled, normalized, and channel-converted before being input into the model. 2. Feature Extraction: CSPDarknet outputs raw feature maps at 3 scales. 3. Feature Fusion: PAN-FPN performs cross-scale fusion on the original feature maps, outputting three fused feature maps. 4. Target Prediction: The Head layer infers from the fused feature map and outputs the target coordinates, confidence score, and category. 5. Post-processing: Duplicate detection boxes are removed by non-maximum suppression (NMS), and targets with a confidence level ≥ 0.7 are output to the tracking module.
[0037] The three-anchor anti-interference tracking algorithm adopted in this application is a multi-source data fusion tracking scheme for complex UAV tracking scenarios. It constructs a three-dimensional constraint system of "visual feature anchor points, spatial positioning reference anchor points, and motion anchor points," fusing visual tracking, laser ranging, and motion prediction data to achieve strong anti-interference tracking. Its core logic leverages the complementarity of multi-dimensional data: visual features provide appearance recognition, laser ranging provides an absolute spatial reference, and motion prediction provides trend constraints. Through dynamic fusion strategies and anti-occlusion mechanisms, it addresses pain points in traditional tracking such as visual drift, occlusion failure, and insufficient long-range accuracy.
[0038] Anchor point modeling principle: 1. Visual Feature Anchor Points: Based on the deep semantic feature extraction capability of the YOLOv8m object detection model backbone network (CSPDarknet), a 256-dimensional high-dimensional feature vector of the target is obtained. After L2 normalization, feature scale differences are eliminated, forming a target-specific visual identifier, providing a unified benchmark for subsequent feature matching and ensuring the stability of appearance recognition.
[0039] 2. Spatial Positioning Reference Anchor Point: The laser ranging module directly outputs high-precision spatial data of the target (latitude, longitude, altitude, and straight-line distance D), without the need for additional coordinate conversion, and directly serves as the absolute positioning reference. This data has no inter-frame drift characteristics, which can effectively correct the cumulative error of visual tracking and provide reliable spatial constraints for long-distance tracking and occlusion recovery.
[0040] 3. Motion Anchor Point: Based on the target's initial positioning coordinates and the frame interval (Δt=1 / 30s), the initial velocity is calculated using the assumption of uniform motion, and the Kalman filter model is initialized. Utilizing the Kalman filter's "prediction-update" iterative mechanism, dynamic estimation of the target's trajectory is achieved, constraining inter-frame coordinate jumps and ensuring tracking continuity.
[0041] Multi-source data fusion principle Based on the straight-line distance D measured by laser ranging, the fusion weights of the three types of data are dynamically adjusted to adapt to the differences in data source reliability under different distance scenarios: 1. Close-range scenes (D<100m): The target's visual features are clear and the ECO tracker outputs high accuracy, so visual data is the primary factor, and laser data is used to verify deviations.
[0042] 2. Mid-range scenarios (100m≤D≤1000m): The reliability of visual features and laser data is balanced. Through equal weight fusion and dynamic gain adjustment, positioning accuracy and stability are balanced.
[0043] 3. Long-distance scenarios (D>100m): Blurred visual features cause tracking to drift easily. Laser ranging data has better stability, so laser data is the primary method and visual data is used as an auxiliary calibration method.
[0044] Principle of occlusion recovery: When occlusion occurs, visual features become ineffective. Rapid recovery is achieved through a dual mechanism of "spatial constraint + feature matching": the search area is adaptively defined based on the distance using historical reliable coordinates of laser ranging as the center, narrowing the target search range; and cosine similarity matching is performed between pre-stored visual feature anchor points and candidate target features to accurately lock the target and avoid recovery failure caused by mismatch.
[0045] Based on the above algorithms and principles, such as Figure 3 , Figure 4 As shown, the implementation steps of the UAV tracking method provided in this application embodiment are as follows: Step 1: Tracking initialization (triggering condition: confidence of YOLOv8m object detection model ≥ 0.7).
[0046] Simultaneously complete the import of data from the three major anchor points and start the core modules to provide a baseline for tracking: Visual feature anchor point initialization: Extract the feature map output from the last layer of the backbone network of the YOLOv8m object detection model, crop the feature block corresponding to the target region, obtain a 256-dimensional feature vector through global average pooling, and store it as a target-specific visual feature anchor point through L2 normalization.
[0047] ,in (Normalization) Spatial positioning reference anchor point initialization: Directly read the target spatial data output by the laser ranging module, including latitude and longitude. ,high The straight-line distance (D) from the drone is used as the spatial positioning reference anchor point.
[0048] Motion anchor point initialization: Initial target positioning coordinates output by YOLOv8m Calculate the target's initial velocity:
[0049] Initial value frame:
[0050] Initialize the Kalman filter model and set the state vector. Process noise covariance Observation noise covariance It predicts the motion trajectory of the next 3 frames and uses it as an anchor point for motion trends.
[0051] Core module startup: Import visual feature anchor points into the ECO tracker to complete initialization, and start the ECO and Kalman filter to run in parallel.
[0052] Step 2: Real-time fusion of multi-source data (executed per frame).
[0053] Based on a distance-based hierarchical strategy, three types of data are dynamically fused to output the optimal tracking coordinates. Data collection: ECO tracker outputs real-time visual tracking position. And the response score s (characterizing the confidence level of visual tracking).
[0054] The laser ranging module outputs real-time spatial coordinates, which are then converted into image pixel coordinates. (The conversion process is based on the camera's participation in the drone's attitude, ensuring accurate mapping between spatial data and the pixel domain.)
[0055] Kalman filtering, based on the motion state of the previous frame, first performs a "prediction" step to output the predicted motion position. Then use Perform the "Update" step to optimize the motion model.
[0056] Weighting and fusion calculation: Close-range scenes (D<50m): Fusion weights , , fusion coordinates ;like (Convert pixel coordinates to actual distance), start YOLOv8m local re-detection (only when...) (200×200 pixel area around the perimeter) calibrate the ECO tracking template.
[0057] Mid-range scenes (50m≤D≤1000m): Fusion weights , , By dynamically adjusting the filter gain coefficient through Kalman filtering, weighted fusion of the three data points is achieved, balancing the bias.
[0058] Long-distance scenes (D>1000m): Fusion weights , , Based on laser ranging spatial coordinate correction To minimize drift error and ensure tracking accuracy.
[0059] Anomaly Validation: If (Inter-frame jump), temporary adjustment ( Increase the weight of laser data to suppress abnormal jumps.
[0060] Step 3: Occlusion detection and quick recovery.
[0061] Occlusion Detection: When the response score of the ECO tracker (s<0.6), the target is determined to be occluded. The visual tracking results are paused, and the system switches to a joint prediction mode of "spatial positioning reference anchor point + motion anchor point" to output a temporary tracking position. .
[0062] Adaptive search area delineation: Using the target coordinates obtained from the last laser ranging before occlusion as the center, the search radius is set according to the straight-line distance D—50 pixels when D < 500m, and 100 pixels when D ≥ 500m, forming a circular search area to narrow the search range for candidate targets.
[0063] Feature matching and target recovery: YOLOv8m uses the visual feature anchors stored during the initialization phase to detect candidate targets and extract features in real time within the search area, calculating the cosine similarity between the candidate features and the visual feature anchors (formula: (...) When the similarity is ≥0.85, the target is immediately locked, the three anchor points data are updated, and the three-anchor fusion tracking is resumed; if no match is found for 10 consecutive frames, the tracking is terminated and the global detection of the YOLOv8m target detection model is restarted.
[0064] Step 4: Track state updates (executed every frame).
[0065] After each frame of tracking is completed, the data of the three main anchor points is updated synchronously to ensure the real-time performance and accuracy of the data: Visual feature anchor point update: Based on the new target features extracted by the latest frame YOLOv8m object detection model, a moving average strategy is used to update the anchor points (retaining 80% of historical features + 20% of new features), formula: ( ) Adapt to subtle changes in the target's appearance (such as posture adjustments and lighting changes).
[0066] Spatial positioning reference anchor point update: directly replaces it with real-time spatial data output by the laser ranging module to ensure the timeliness of the spatial reference.
[0067] Motion anchor point update: The coordinates are fused using the current frame through a Kalman filter "prediction-update" iterative process. Optimize motion state vector ( This improves the accuracy of motion prediction in the next frame.
[0068] Complete drone tracking chain implementation: The YOLOv8m target detection model's recognition results, the stable tracking position output by the three-anchor tracking module, and the high-precision spatial data from laser ranging are used to generate a real-time target status report, which includes information such as target position, movement speed, occlusion status, and relative distance to the UAV.
[0069] Flight control command generation: The UAV flight control system receives the target status report, calculates the relative motion trend based on the difference between the target's latitude, longitude, altitude and the UAV's own position, and generates flight attitude adjustment commands (horizontal angle, pitch angle, yaw angle, etc.), camera fine-tuning commands and path planning suggestions to ensure accurate target tracking.
[0070] Dynamic feedback correction: After the UAV executes control commands, the YOLOv8m target detection model recognition and three-anchor tracking module provide real-time feedback on the target's position deviation and tracking stability in the image. If the target deviates from the center of the image by more than 30% or the tracking response score decreases, the flight control system combines the laser ranging spatial data to fine-tune the flight or camera parameters again, forming a closed-loop adjustment of "recognition-tracking-positioning-control-feedback" to ensure continuous and accurate tracking.
[0071] Understandably, many improvements have been made to the YOLOv8 object detection model, such as YOLOv8m and YOLOv8n. Different algorithms can be selected for training and deployment based on the computing power requirements of different devices. In addition, algorithms have been optimized for different recognition backgrounds and accuracy requirements by modifying the structure of the YOLOv8 algorithm, such as introducing the CBAM attention mechanism and using BiFPN for multi-scale feature fusion.
[0072] Tracking algorithms can also employ various improvements, such as CSRF (Collaborative Spatial Regression Forests) and MOSSE (Minimum Output Sum of Squared Error Filter). The CSRF algorithm utilizes multiple feature channels, including color and edges, to enhance its adaptability to changes in the target's appearance. The MOSSE algorithm trains the filter by minimizing the squared output error, resulting in low computational cost.
[0073] In summary, the UAV tracking method provided in this application improves tracking immunity through a three-anchor anti-interference mechanism and a multi-source fusion strategy, effectively addressing scenarios such as jitter, occlusion, and background obfuscation. It constructs a complete tracking closed loop, achieving deep collaboration between identification, tracking, laser ranging, and flight control, ensuring accurate UAV tracking.
[0074] See Figure 5 The present invention can also provide a drone tracking device, such as... Figure 5 As shown, the device may include: The tracking initialization unit 501 is used to simultaneously complete the initialization of visual feature anchors, spatial anchors, motion anchors, and the core module when the detection confidence of the YOLOv8m target detection model reaches a preset threshold. The visual feature anchor initialization includes extracting the feature map output from the last layer of the backbone network of the YOLOv8m target detection model, cropping the feature blocks corresponding to the target region, performing global average pooling and normalization, and storing them as target-specific visual feature anchors. The spatial anchor initialization includes reading the target spatial data output by the laser ranging module as spatial positioning reference anchors. The motion anchor initialization includes calculating the initial motion velocity based on the preliminary target positioning coordinates output by the YOLOv8m target detection model, and initializing the Kalman filter model to predict the motion trajectory in future multiple frames, serving as motion trend anchors. The core module startup involves importing the visual feature anchors into the ECO tracker to complete initialization, and starting the ECO tracker and the Kalman filter to run in parallel. The multi-source data real-time fusion unit 502 is used to dynamically fuse the visual tracking position and response score output in real time by the ECO tracker, the image pixel coordinates after real-time spatial coordinate transformation output by the laser ranging module, and the motion prediction position output by the Kalman filter based on the motion state of the previous frame for each frame image based on a distance layering strategy, and output fused coordinates. The occlusion determination and fast recovery unit 503 is used to determine that the target is occluded when the response score is lower than a preset threshold, suspend the dominance of visual tracking results, switch to the joint prediction mode of spatial anchor points and motion anchor points, and output a temporary tracking position; taking the target coordinates obtained by the last laser ranging before occlusion as the center, and setting the search radius according to the straight-line distance between the target and the UAV to form a search area; calling the visual feature anchor points stored in the initialization stage, the YOLOv8m target detection model detects candidate targets and extracts candidate features in the search area, and calculates the similarity between the candidate features and the visual feature anchor points; when the similarity reaches a preset threshold, the target is locked, the data of each anchor point is updated, and the three-anchor fusion tracking is resumed; if a predetermined number of consecutive frames fail to match, the tracking is terminated and the global detection of the YOLOv8m target detection model is restarted; The tracking state update unit 504 is used to synchronously update the visual feature anchor point, the spatial anchor point, and the motion anchor point after each frame of image tracking is completed.
[0075] This invention can also provide a drone tracking device, the device including a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of the above-described drone tracking method according to the instructions in the program code.
[0076] like Figure 6 As shown in the figure, an embodiment of the present invention provides a drone tracking device, which may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, the memory 11, and the communication interface 12 all communicate with each other through the communication bus 13.
[0077] In this embodiment of the invention, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.
[0078] The processor 10 can call programs stored in the memory 11. Specifically, the processor 10 can execute operations in the embodiments of the drone tracking method.
[0079] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 11 stores at least a program for implementing the following functions: When the detection confidence of the YOLOv8m target detection model reaches a preset threshold, the initialization of visual feature anchors, spatial anchors, motion anchors, and the core module are completed simultaneously. Visual feature anchor initialization involves extracting the feature map output from the last layer of the backbone network of the YOLOv8m target detection model, cropping the feature blocks corresponding to the target region, performing global average pooling and normalization, and storing them as target-specific visual feature anchors. Spatial anchor initialization involves reading the target spatial data output by the laser ranging module as spatial positioning reference anchors. Motion anchor initialization involves calculating the initial motion velocity based on the preliminary target positioning coordinates output by the YOLOv8m target detection model, and initializing the Kalman filter model to predict the motion trajectory for future multiple frames, serving as motion trend anchors. The core module startup involves importing the visual feature anchors into the ECO tracker to complete initialization, and then starting the ECO tracker and the Kalman filter to run in parallel. Each frame of image is dynamically fused based on a distance-layered strategy, which integrates the real-time visual tracking position and response score output by the ECO tracker, the real-time spatial coordinates transformed image pixel coordinates output by the laser ranging module, and the motion prediction position output by the Kalman filter based on the motion state of the previous frame, and outputs fused coordinates. When the response score is lower than a preset threshold, the target is determined to be occluded. The visual tracking result is paused, and the system switches to a joint prediction mode of spatial anchor points and motion anchor points, outputting a temporary tracking position. The target coordinates obtained by the last laser ranging before occlusion are used as the center, and a search radius is set according to the straight-line distance between the target and the UAV to form a search area. The visual feature anchor points stored in the initialization phase are called, and the YOLOv8m target detection model detects candidate targets and extracts candidate features within the search area, calculating the similarity between the candidate features and the visual feature anchor points. When the similarity reaches a preset threshold, the target is locked, the data of each anchor point is updated, and the three-anchor fusion tracking is resumed. If a predetermined number of consecutive frames fail to match, the tracking is terminated and the global detection of the YOLOv8m target detection model is restarted. After each frame of image tracking is completed, the visual feature anchor point, the spatial anchor point, and the motion anchor point are updated synchronously.
[0080] In one possible implementation, the memory 11 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function (such as file creation or data read / write). The data storage area may store data created during use, such as initialization data.
[0081] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.
[0082] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.
[0083] Of course, it should be noted that, Figure 6 The structure shown does not constitute a limitation on the drone tracking device in the embodiments of the present invention. In practical applications, the drone tracking device may include more than Figure 6 More or fewer components as shown, or combinations of certain components.
[0084] This invention also provides a computer-readable storage medium for storing program code for executing the steps of the above-described UAV tracking method.
[0085] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0086] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0087] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0088] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. A method for tracking unmanned aerial vehicles (UAVs), characterized in that, The method includes: When the detection confidence of the YOLOv8m target detection model reaches a preset threshold, the initialization of visual feature anchors, spatial anchors, motion anchors, and the core module are completed simultaneously. Visual feature anchor initialization includes extracting the feature map output from the last layer of the backbone network of the YOLOv8m target detection model, cropping the feature blocks corresponding to the target region, performing global average pooling and normalization, and storing them as target-specific visual feature anchors. Spatial anchor initialization includes reading the target spatial data output by the laser ranging module as spatial positioning reference anchors. Motion anchor initialization includes calculating the initial motion velocity based on the preliminary target positioning coordinates output by the YOLOv8m target detection model, and initializing the Kalman filter model to predict the motion trajectory for future multiple frames, serving as motion trend anchors. The core module startup includes importing the visual feature anchors into the ECO tracker for initialization and starting the ECO tracker and the Kalman filter to run in parallel. Each frame of image is dynamically fused based on a distance-layered strategy, which integrates the real-time visual tracking position and response score output by the ECO tracker, the real-time spatial coordinates transformed image pixel coordinates output by the laser ranging module, and the motion prediction position output by the Kalman filter based on the motion state of the previous frame, and outputs fused coordinates. When the response score is lower than a preset threshold, the target is determined to be occluded. The visual tracking result is paused, and the system switches to a joint prediction mode of spatial anchor points and motion anchor points, outputting a temporary tracking position. The target coordinates obtained by the last laser ranging before occlusion are used as the center, and a search radius is set according to the straight-line distance between the target and the UAV to form a search area. The visual feature anchor points stored in the initialization phase are called, and the YOLOv8m target detection model detects candidate targets and extracts candidate features within the search area, calculating the similarity between the candidate features and the visual feature anchor points. When the similarity reaches a preset threshold, the target is locked, the data of each anchor point is updated, and the three-anchor fusion tracking is resumed. If a predetermined number of consecutive frames fail to match, the tracking is terminated and the global detection of the YOLOv8m target detection model is restarted. After each frame of image tracking is completed, the visual feature anchor point, the spatial anchor point, and the motion anchor point are updated synchronously.
2. The UAV tracking method according to claim 1, characterized in that, The global average pooling and normalization process includes: A 256-dimensional feature vector is obtained through global average pooling and then normalized using L2. The target spatial data includes the target latitude and longitude. Target height and the straight-line distance between the target and the drone .
3. The UAV tracking method according to claim 1, characterized in that, Calculate the initial velocity and initialize the Kalman filter model, including: Preliminary target location coordinates output by the YOLOv8m target detection model Calculate the target's initial velocity: Among them, the initial value frame ; Initialize the Kalman filter model and set the state vector. Process noise covariance Observation noise covariance It can predict the motion trajectory of the next 3 frames.
4. The UAV tracking method according to claim 1, characterized in that, The response score being lower than a preset threshold includes a response score less than 0.6; the similarity reaching a preset threshold includes a cosine similarity greater than or equal to 0.85; the consecutive predetermined number of frames includes 10 consecutive frames.
5. The UAV tracking method according to claim 1, characterized in that, The conversion process of the real-time spatial coordinates output by the laser ranging module into image pixel coordinates is based on the camera's participation in the UAV's attitude, realizing the mapping between spatial data and the pixel domain.
6. The UAV tracking method according to claim 1, characterized in that, The distance layering strategy and the calculation method of the fused coordinates are as follows: When the straight-line distance D between the target and the drone is less than 50 meters, it is considered a close-range scene, and the visual weight is determined as follows. Laser weight is Motion weights are fusion coordinates ;like Then YOLOv8m local re-detection will be initiated, only when... The surrounding 200×200 pixel area is detected to calibrate the ECO tracking template. This is the actual distance after pixel coordinate transformation; When the straight-line distance D is greater than or equal to 50 meters and less than or equal to 1000 meters, it is considered a mid-range scene, and the visual weight is determined as follows. Laser weight is Motion weights are The Kalman filter dynamically adjusts the filter gain coefficient to achieve weighted fusion of the three data. When the straight-line distance D is greater than 1000 meters, it is considered a long-distance scene, and the visual weight is determined as follows. Laser weight is Motion weights are The visual tracking position is corrected based on the spatial coordinates of the laser ranging module. The drift error; like Then temporarily adjust the motion weights. Increase the weight of laser data to suppress inter-frame jump anomalies.
7. The UAV tracking method according to claim 1, characterized in that, The formula for calculating the temporary tracking position includes: The setting of the search radius based on the straight-line distance between the target and the UAV includes setting the search radius to 50 pixels when the straight-line distance D is less than 500 meters and to 100 pixels when the straight-line distance D is greater than or equal to 500 meters; the similarity includes cosine similarity, and the formula for calculating the cosine similarity includes... .
8. The UAV tracking method according to claim 1, characterized in that, The visual feature anchor point update includes: Based on the new target features extracted by the YOLOv8m target detection model in the latest frame, a moving average strategy is used for updating, as shown in the formula: That is, 80% of historical features and 20% of new features are retained; the spatial anchor point is updated by directly replacing it with the real-time spatial data output by the laser ranging module; the motion anchor point is updated by using the Kalman filter prediction-update iteration process to fuse coordinates with the current frame. Optimize motion state vector .
9. The UAV tracking method according to claim 1, characterized in that, It also includes integrating the YOLOv8m target detection model recognition results, the stable tracking position output by the three-anchor tracking module, and the laser ranging spatial data to generate a real-time target status report. The real-time target status report includes the target position, movement speed, occlusion status, and relative distance information to the UAV. The UAV flight control system receives the target's real-time status report, calculates the relative motion trend based on the difference between the target's latitude, longitude, altitude and the UAV's own position, and generates flight attitude adjustment commands, camera fine-tuning commands and path planning suggestions; After the UAV executes the control command, the YOLOv8m target detection model and the three-anchor tracking module provide real-time feedback on the target's position deviation and tracking stability in the image. If the target deviates from the center of the image by more than 30% or the tracking response score decreases, the flight control system, in conjunction with the laser ranging spatial data, fine-tunes the flight or camera parameters again, forming a closed-loop adjustment of recognition-tracking-positioning-control-feedback.
10. A drone tracking device, characterized in that, The apparatus for performing the drone tracking method according to any one of claims 1-9, the apparatus comprising: The tracking initialization unit is used to simultaneously complete visual feature anchor point initialization, spatial anchor point initialization, motion anchor point initialization, and core module startup when the detection confidence of the YOLOv8m target detection model reaches a preset threshold. The visual feature anchor point initialization includes extracting the feature map output from the last layer of the backbone network of the YOLOv8m target detection model, cropping the feature blocks corresponding to the target region, performing global average pooling and normalization processing, and storing them as target-specific visual feature anchor points. The spatial anchor point initialization includes reading the target spatial data output by the laser ranging module as spatial positioning reference anchor points. The motion anchor point initialization includes calculating the initial motion velocity based on the preliminary target positioning coordinates output by the YOLOv8m target detection model, and initializing the Kalman filter model to predict the motion trajectory in future multiple frames, serving as motion trend anchor points. The core module startup involves importing the visual feature anchor points into the ECO tracker to complete initialization, and starting the ECO tracker and the Kalman filter to run in parallel. The multi-source data real-time fusion unit is used to dynamically fuse the visual tracking position and response score output in real time by the ECO tracker, the image pixel coordinates after real-time spatial coordinate transformation output by the laser ranging module, and the motion prediction position output by the Kalman filter based on the motion state of the previous frame for each frame image based on a distance layering strategy, and output fused coordinates. The occlusion determination and rapid recovery unit is used to determine that the target is occluded when the response score is lower than a preset threshold, suspend the dominance of visual tracking results, switch to the joint prediction mode of spatial anchor points and motion anchor points, and output a temporary tracking position; using the target coordinates obtained by the last laser ranging before occlusion as the center, a search radius is set according to the straight-line distance between the target and the UAV to form a search area; the visual feature anchor points stored in the initialization phase are called, and the YOLOv8m target detection model detects candidate targets and extracts candidate features in the search area, and calculates the similarity between the candidate features and the visual feature anchor points; when the similarity reaches a preset threshold, the target is locked, the data of each anchor point is updated, and the three-anchor fusion tracking is resumed; if a predetermined number of consecutive frames fail to match, the tracking is terminated and the global detection of the YOLOv8m target detection model is restarted; The tracking state update unit is used to synchronously update the visual feature anchor point, the spatial anchor point, and the motion anchor point after each frame of image tracking is completed.
