An unmanned aerial vehicle tracking method based on continuous local frame matching

Abstract

The application discloses a kind of unmanned plane tracking methods based on continuous local frame matching, belong to unmanned plane tracking technical field.Method includes: target identification and selection to video stream;Control the target to be centered in turn, and dynamically save continuous N frame image and corresponding turntable pitch, yaw angle;Utilize turntable angle restriction to carry out matching and purification to the background feature of continuous frame, splice image and record target position sequence;Based on the sequence, predict target next frame position by Kalman filter;Calculate the intersection and union ratio of predicted frame and actual frame, and judge target to be in normal tracking, follow wrong or shield state in combination with feature matching value, and execute corresponding correction operation;Finally, loop update information.The application improves the accuracy, anti-interference ability and continuity of unmanned plane tracking in complex scene by continuous multi-frame information fusion and trajectory prediction correction mechanism.

Description

Technical Field

[0001] This invention relates to the field of drone tracking technology, and in particular to a drone tracking method based on continuous local frame matching. Background Technology

[0002] Due to their low cost, low-altitude, slow speed, and high stealth capabilities, drones, while offering convenience, can also be used for illegal activities, posing a threat to public safety, aviation order, and sensitive areas. Therefore, establishing an efficient and reliable low-altitude defense system to accurately track and handle illegal drones has become an urgent need.

[0003] Currently, common drone target tracking solutions typically integrate the camera onto a rotatable gimbal (or turntable), and control the turntable's rotation via an edge computing platform to keep the target continuously centered in the camera's field of view, enabling continuous tracking and potential strikes. However, this solution faces several challenges in practical applications. First, both the drone and the camera-equipped turntable are in motion during tracking, causing drastic changes in the camera's perspective, resulting in rapid alterations to the target's appearance, scale, and background in the image. Second, similar targets in complex environments (such as birds or other drones) can easily interfere with the tracking algorithm, causing the tracker to "track the wrong" target. Furthermore, situations where the target is temporarily obscured by obstacles such as buildings and trees also frequently occur.

[0004] A core flaw in existing solutions lies in the vulnerability of their "single-frame decision-making." If a tracking error occurs in a frame due to interference from similar targets or occlusion, control commands will cause the turntable to rotate incorrectly, resulting in the real target rapidly moving out of the camera's field of view. Lacking effective global context and historical trajectory information for backtracking and correction, the system often struggles to recover from such errors, leading to complete tracking failure. This severely impacts the robustness, continuous operation capability, and final strike accuracy of the UAV tracking system. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a UAV tracking method based on continuous local frame matching. By fusing and matching information from multiple consecutive frames and combining turntable motion information, a more stable global scene understanding is constructed. Furthermore, by utilizing trajectory prediction and tracking state diagnosis mechanisms, effective correction can be made when tracking errors or occlusions occur, thereby significantly improving the accuracy, anti-interference ability, and overall robustness of UAV tracking in complex scenarios.

[0006] The objective of this invention is achieved through the following technical solution: a UAV tracking method based on continuous local frame matching, comprising the following steps: S1. Target Recognition and Selection Stage: The video stream mode is used to identify the drone target. If a single drone target is identified, it is set as the tracking target; if multiple drone targets are identified, one target is selected as the tracking target according to preset rules. S2, Target Tracking and Frame Sequence Management Stage: Track the selected target and control the turntable equipped with the camera to position the target at the center of the camera image; starting from the first frame of tracking, save N consecutive video images to form a video frame sequence; S3, Angle Information Recording and Continuous Frame Matching Stage: Record the pitch and yaw angles of the turntable as it rotates continuously during image acquisition, and obtain the pitch angle sequence and yaw angle sequence for each frame of video information; then perform feature matching on the continuous frames in the video frame sequence to obtain a complete stitched image containing the tracked target, and record the position sequence of the tracked target in the complete stitched image; S4. Trajectory Prediction Stage: Based on the position sequence of the tracked target, the trajectory prediction algorithm is used to predict the predicted position of the tracked target in the next frame, and the predicted position is used as a constraint to continuously track the target. S5. Tracking Status Judgment and Correction Stage: Calculate the intersection-over-union ratio (IoU) between the actual tracked target position and the predicted position in the next frame. I Based on intersection-union ratio I The calculation results determine the target tracking status, and the corresponding tracking correction operation is performed according to the different statuses. S6. Cyclic Update Phase: Release the complete stitched image of the tracked target and update the video frame information, pitch angle sequence information and yaw angle sequence information. Then return to step S3 and cycle through steps S3 to S5 until a stop tracking command is received or the target is determined to have been lost for a long time.

[0007] Preferably, the preset rule is: calculating the distance from each identified target to the center point of the video frame image. L i : L i =| x i - cx |+| y i - cy |, among which x i Let x be the x-coordinate of the center point of the i-th target. y i Let be the ordinate of the center point of the i-th target. cx The x-coordinate of the center point of the video frame image. cy The ordinate of the center point of the video frame image; Select L iThe target with the smallest value is selected as the tracking target.

[0008] Preferably, in the S2 target tracking and frame sequence management stage, the number of frames N is dynamically calculated based on the memory required for video imaging, the memory resources of the computing platform, and the movement speed of the UAV. ,in p The preset percentage value, R To calculate the total memory usage of the platform, m This represents the memory usage of a single frame of an image.

[0009] Preferably, feature matching for consecutive frames includes the following sub-steps: S31. Perform SIFT feature extraction on consecutive frames in the video frame sequence, and divide the feature points of each frame into the drone part and the background part. The drone feature points are denoted as follows: P U UAV feature vector f U Background feature points are denoted as P B The background feature vector is denoted as f B ; S32. Based on background feature points and background feature vectors, perform feature matching on adjacent frames to generate a matching matrix. M B and the rotation matrix formed by rotating the turntable. M R As a constraint; ,in Yi This represents the yaw angle of the turntable rotation corresponding to adjacent frame images. Pi This represents the pitch angle of the turntable rotation corresponding to adjacent frame images. i The frame number is 1 < 1. i ≤N; S33. Calculate the matching matrix M B With rotation matrix M R similarity R s ,like R s Greater than the preset threshold T Then, feature points with a matching degree lower than the preset matching degree in the background features are removed, and matching is repeated until the matching matrix is ​​obtained. M B Satisfy rotation matrix M R Constraints; S34. Calculate each adjacent frame image sequentially until a complete stitched image is formed, and record the position sequence of the tracked target in the complete stitched image.

[0010] Preferably, the trajectory prediction algorithm is the Kalman filter algorithm.

[0011] Preferably, the S5 tracking status judgment and correction stage further includes the following steps: The predicted location is denoted as ( x p , y p , w p , h p ),in x p To predict the x-coordinate of the top-left corner of the target bounding box, y p To predict the ordinate of the top-left corner of the target bounding box, w p To predict the width of the target bounding box, h p To predict the height of the target bounding box; The actual location of the tracked target is denoted as ( x t , y t , w t , h t )in x t The x-coordinate of the top left corner of the actual tracking target bounding box. y t The actual tracking target bounding box's top-left ordinate. w t To track the actual target bounding box width, h t This is the actual height of the target bounding box being tracked. Calculate the center point positions of the predicted bounding box and the ground truth tracked bounding box: , , , ,in To predict the x-coordinate of the center point of the target bounding box, To predict the ordinate of the center point of the target bounding box, To accurately track the x-coordinate of the center point of the target bounding box, To accurately track the ordinate of the center point of the target bounding box; when or Then the intersection and union ratio I =0, otherwise ; Set the intersection over union threshold ThI and the feature matching threshold ThT; When I ≥ ThI, it is determined that the tracking is successful, and the next frame is processed continuously; When I < ThI, calculate the feature matching value PM of the target area in the current frame and the previous frame; if PM ≤ ThT, it is determined that a similar interference target is tracked, and the tracking target box is recalculated according to the predicted position obtained in step S4; if PM > ThT, it is determined that the tracking target is occluded, and the tracking target box is updated with the predicted position obtained in step S4.

[0012] Preferably, recalculating the tracking target box according to the predicted position includes the following steps: Predict the target prediction box of the current frame according to the target position of the previous frame, obtain the current frame target position search box with a region twice the length and width of the target prediction box of the current frame as the search range, perform template matching within the current frame target position search box, find the most similar target position, and judge the similarity of the most similar target position. If it is greater than or equal to the preset relocalization threshold, it is determined that the relocalization is successful.

[0013] The beneficial effects of the present invention are: 1) Through the closed-loop mechanism of "continuous local frame matching" and "trajectory prediction - state correction", the present invention fundamentally changes the fragile mode of traditional solutions that rely on single-frame decision-making. By saving and analyzing information of multiple consecutive frames (N frames) and combining the turntable movement angle constraint for feature matching, the system constructs a more stable and global understanding of the target movement trajectory and the scene background. When similar target interference (tracking error) or short-term occlusion occurs, the system can perform intelligent diagnosis and correction based on historical trajectory prediction and feature matching values, rather than following the interference object incorrectly or losing the target, thereby greatly reducing the probability of tracking failure and ensuring the long-term continuous and stable operation of the tracking task.

[0014] 2) In the initial target selection stage, the present invention adopts the principle of the closest distance to the image center, preferentially selects the target most likely to be at the stable tracking starting point, and improves the accuracy of initial locking. Secondly, using the accurate pitch angle and yaw angle sequences of the turntable as geometric constraints, the background features of consecutive frames are matched and screened, greatly improving the accuracy of image stitching and the recording of the UAV position sequence.

[0015] 3) The number of frames N is dynamically calculated based on the computing platform's memory and the drone's movement speed, achieving optimal utilization of effective historical information under limited hardware resources. This avoids memory overflow and ensures that fast-moving targets have enough frames for trajectory analysis. In the feature matching stage, the turntable rotation matrix (MR) is introduced as a hard constraint to verify and refine background feature matching (MB), effectively eliminating false matches caused by repeated textures or moving objects in the scene, and enhancing the algorithm's anti-interference ability in complex dynamic backgrounds.

[0016] 4) This invention calculates the intersection-union ratio (I) between the predicted bounding box and the actual tracking bounding box, as well as the feature matching value (PM) of the target region, and sets dual thresholds (ThI, ThT) for judgment. This allows for a clear distinction between whether the target is "replaced by a similar object" or "partially / completely occluded." For "mistracking," the system reinitializes the tracking bounding box with the predicted position and discards the erroneous target; for "occlusion," the system updates the tracking bounding box with the predicted position, maintaining lock on the target's potential location. This differentiated correction strategy significantly improves the system's success rate in resuming tracking.

[0017] 5) By releasing the stitched image, after completing trajectory prediction and state judgment in the current cycle, the memory occupied by the processed historical image data (stitched image) is released in a timely manner. The system only needs to maintain the valid information of the most recent N frames for decision-making. By releasing old data and incorporating new data, the recycling of memory resources is achieved, ensuring that the algorithm can run in real time on resource-limited edge computing devices for a long time, meeting the strict real-time requirements of actual deployment.

[0018] 6) This invention is an algorithm-level improvement scheme. Its core lies in the deeper fusion processing and intelligent decision-making of data (video stream, turntable angle) collected by existing cameras, turntables, and computing platforms. Therefore, this method can be directly applied to existing UAV tracking system hardware platforms without adding special sensors or modifying hardware. It achieves a leapfrog improvement in system performance at the lowest additional cost, and has high industrial application value and promotion prospects. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram illustrating the recalculation of the tracking target bounding box based on the predicted location. Figure 3 Set up a diagram to indicate the search scope; Figure 4 A schematic diagram illustrating the relocation of the tracking target. Detailed Implementation

[0020] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] See Figures 1-4 This invention provides a technical solution: a UAV tracking method based on continuous local frame matching, comprising the following steps: S1. Target Recognition and Selection Stage: The video stream mode is used to identify the drone target. If a single drone target is identified, it is set as the tracking target; if multiple drone targets are identified, one target is selected as the tracking target according to preset rules. S2, Target Tracking and Frame Sequence Management Stage: Track the selected target and control the turntable equipped with the camera to position the target at the center of the camera image; starting from the first frame of tracking, save N consecutive video images to form a video frame sequence; S3, Angle Information Recording and Continuous Frame Matching Stage: Record the pitch and yaw angles of the turntable as it rotates continuously during image acquisition, and obtain the pitch angle sequence and yaw angle sequence for each frame of video information; then perform feature matching on the continuous frames in the video frame sequence to obtain a complete stitched image containing the tracked target, and record the position sequence of the tracked target in the complete stitched image; S4. Trajectory Prediction Stage: Based on the position sequence of the tracked target, the trajectory prediction algorithm is used to predict the predicted position of the tracked target in the next frame, and the predicted position is used as a constraint to continuously track the target. S5. Tracking Status Judgment and Correction Stage: Calculate the intersection-over-union ratio (IoU) between the actual tracked target position and the predicted position in the next frame. I Based on intersection-union ratio I The calculation results determine the target tracking status, and the corresponding tracking correction operation is performed according to the different statuses. S6. Cyclic Update Phase: Release the complete stitched image of the tracked target and update the video frame information, pitch angle sequence information and yaw angle sequence information. Then return to step S3 and cycle through steps S3 to S5 until a stop tracking command is received or the target is determined to have been lost for a long time.

[0022] In this embodiment, the present invention upgrades the traditional "image-based" tracking mode of single-frame tracking to a mode of "global judgment and decision-making" using spatiotemporal context information through a three-level mechanism of "continuous local frame matching to construct stable trajectories," "trajectory prediction look-ahead guidance," and "multi-feature diagnosis and correction." Turntable angle information provides geometric constraints for inter-frame matching, ensuring the accuracy of historical trajectories; prediction based on accurate trajectories provides strong priors for tracking the current frame; and the combination of IoU and appearance feature matching intelligently distinguishes and handles the two most typical tracking failure scenarios of "failed tracking" and "occlusion," thereby achieving a qualitative improvement in tracking robustness in complex real-world environments.

[0023] In S6, releasing the complete stitched image of the tracked target is necessary because continuous local frame matching involves stitching only N consecutive frames at a time. Continuing to stitch beyond N frames would lead to excessive memory usage, reducing the efficiency of subsequent tracking calculations, and potentially causing the program to crash due to insufficient memory. The principle is to release the memory resources occupied by the stitched image of N consecutive frames, but using a queue rule (first-in, first-out) to release the matching matrix, target features, background features, and turntable angle sequence information of adjacent frames. Older information is released while newer information is retained, allowing new image frames to continue matching and stitching the next set of N frames, repeating this process until tracking ends.

[0024] In some embodiments, the preset rule is: calculating the distance from each identified target to the center point of the video frame image. L i : L i =| x i - cx |+| y i - cy |, among which x i Let x be the x-coordinate of the center point of the i-th target. y i Let be the ordinate of the center point of the i-th target. cx The x-coordinate of the center point of the video frame image. cy The ordinate of the center point of the video frame image; Select L i The target with the smallest value is selected as the tracking target.

[0025] In this embodiment, after the system starts, the camera continuously captures images of the airspace in video stream mode. The edge computing platform uses a trained target detection model (such as YOLO, SSD, etc.) to perform real-time analysis of the video stream and identify drone targets in the image.

[0026] If only one drone target is identified in the current frame, it will be directly set as the target that the system needs to track.

[0027] If multiple drone targets are identified in the current frame, one must be selected as the tracking target according to a preset rule. The selection rule is as follows: Calculate the distance from the center point of the bounding rectangle of each identified target to the center point of the current video frame image. Then, select the target with the shortest distance, that is, the target closest to the center of the frame, as the tracking target. This rule is based on the engineering experience that "the target closest to the center of the frame is most likely to be stably tracked initially," thus improving the reliability of the tracking start.

[0028] In some embodiments, during the S2 target tracking and frame sequence management stage, the number of frames N is dynamically calculated based on the memory required for video imaging, the memory resources of the computing platform, and the drone's movement speed. ,in p The preset percentage value, R To calculate the total memory usage of the platform, m This represents the memory usage of a single frame of an image.

[0029] In this embodiment, after setting the tracking target, the system generates control commands to drive the turntable, ensuring that the target remains near the center of the camera's image, thus achieving initial visual servo tracking. Starting from the first frame of the image successfully locked and tracking begins, the system saves N consecutive video frames, forming a video frame sequence. The number of frames N is not a fixed value but is dynamically calculated based on the system's real-time resources and the target's motion state. This calculation must simultaneously satisfy memory constraints (ensuring the frame sequence does not cause memory overflow) and motion coverage constraints (ensuring the saved frames are sufficient to cover a meaningful trajectory of the target in the spatial domain, providing a sufficient time window for subsequent analysis). The system takes the largest integer that simultaneously satisfies both conditions as the value of N. This dynamic management mechanism achieves optimal utilization of information under limited hardware resources.

[0030] In some embodiments, feature matching of consecutive frames includes the following sub-steps: S31. Perform SIFT feature extraction on consecutive frames in the video frame sequence, and divide the feature points of each frame into the drone part and the background part. The drone feature points are denoted as follows: P U UAV feature vector f U Background feature points are denoted as P B The background feature vector is denoted as f B ; S32. Based on background feature points and background feature vectors, perform feature matching on adjacent frames to generate a matching matrix. MB and the rotation matrix formed by rotating the turntable. M R As a constraint; ,in Yi This represents the yaw angle of the turntable rotation corresponding to adjacent frame images. Pi This represents the pitch angle of the turntable rotation corresponding to adjacent frame images. i The frame number is 1 < 1. i ≤N; S33. Calculate the matching matrix M B With rotation matrix M R similarity R s ,like R s Greater than the preset threshold T Then, feature points with a matching degree lower than the preset matching degree in the background features are removed, and matching is repeated until the matching matrix is ​​obtained. M B Satisfy rotation matrix M R Constraints; S34. Calculate each adjacent frame image sequentially until a complete stitched image is formed, and record the position sequence of the tracked target in the complete stitched image.

[0031] In this embodiment, SIFT (Scale Invariant Feature Transform) feature extraction is performed on each frame of the frame sequence to obtain a large number of feature points and their descriptions (feature vectors), and the frames are divided into a drone part and a background part. Then, based on the background feature points and background feature vectors, feature matching is performed on adjacent frames to generate a matching matrix. M B This matrix describes the mapping relationship of background feature points from the previous frame to the current frame.

[0032] Simultaneously, geometric constraints are constructed using the angle information recorded by the turntable. A theoretical rotation constraint matrix can be calculated by the changes in yaw and pitch angles of the turntable between two adjacent frames. M R This matrix represents the ideal projection transformation relationship of the midpoint of the scene from the previous frame to the current frame in the case of pure camera rotation (no translation).

[0033] Calculate the matching matrix M B With rotation matrix M R similarity R s ,like R s Greater than the preset thresholdT If the result is negative, it indicates that the current background matching result deviates significantly from the pure camera rotation model, which may include incorrect matching caused by independent moving objects in the scene (such as birds or clouds).

[0034] At this point, the system will discard feature point pairs with low matching confidence, then re-match and recalculate the similarity. This process is repeated iteratively until the similarity between the obtained background matching matrix and the rotation constraint matrix meets the requirements. This process uses precise turntable angle information as a "hard constraint," effectively filtering out dynamic background interference and obtaining a clean and accurate background matching relationship.

[0035] The purified background matching matrix is ​​used to align and stitch adjacent frames. Calculations are performed sequentially on all adjacent frames in the frame sequence, ultimately stitching all images into a complete image. During the stitching process, the system accurately records the position of the tracked target in the coordinate system of the stitched image within each original frame through coordinate mapping, thus obtaining a precise target position sequence. This sequence reflects the actual movement trajectory of the UAV over a recent period.

[0036] In some embodiments, the trajectory prediction algorithm is a Kalman filter algorithm.

[0037] In this embodiment, the target position sequence obtained in step S3 is input into the Kalman filter algorithm. Based on the target's historical position and velocity information, and combined with the motion equations and observation model, the Kalman filter algorithm predicts the position where the tracked target should appear in the next frame, denoted as the predicted position. This predicted position will serve as an important basis and constraint for subsequent tracking decisions.

[0038] In some embodiments, the S5 tracking status judgment and correction stage further includes the following steps: The predicted location is denoted as ( x p , y p , w p , h p ),in x p To predict the x-coordinate of the top-left corner of the target bounding box, y p To predict the ordinate of the top-left corner of the target bounding box, w p To predict the width of the target bounding box, h p To predict the height of the target bounding box; The actual location of the tracked target is denoted as ( x t ,y t , w t , h t ), where x t is the abscissa of the upper - left corner of the actual tracking target box, y t is the ordinate of the upper - left corner of the actual tracking target box, w t is the width of the actual tracking target box, h t is the height of the actual tracking target box; Calculate the center - point positions of the predicted target box and the actual tracking target box: , , , , where is the abscissa of the center - point of the predicted target box, is the ordinate of the center - point of the predicted target box, is the abscissa of the center - point of the actual tracking target box, is the ordinate of the center - point of the actual tracking target box; When or , then the intersection - over - union I = 0, otherwise ; Set the intersection - over - union threshold ThI and the feature - matching threshold ThT; When I ≥ThI, it is determined that the tracking is successful, and continue to process the next frame; When I <ThI, calculate the feature - matching value PM of the target regions in the current frame and the previous frame; if PM≤ThT, it is determined that a similar interference target is tracked, and recalculate the tracking target box according to the predicted position obtained in step S4; if PM > ThT, it is determined that the tracking target is occluded, and update the tracking target box with the predicted position obtained in step S4.

[0039] In this embodiment, after the system acquires the next - frame image, the target - detection module will give the position of the "actual tracking target" in the current frame. The system compares this actual position with the predicted position obtained in step S4, and calculates the intersection - over - union I of the predicted target box and the actual tracking target box. If the distance between the centers of the two boxes in the horizontal or vertical direction is too large, that is, when or , it is determined that the two boxes have no overlap, and the intersection - over - union I = 0, otherwise calculate the intersection - over - union according to the formula.

[0040] Status Judgment and Correction: Case 1: Tracking is successful. If I ≥ ThI, it indicates that the actual target coincides highly with the predicted position, the tracking status is good, and the system continues to process the subsequent frames.

[0041] Case 2: Tracking the wrong target (similar interference). If I < ThI, it indicates that there may be problems with the tracking. The system further calculates the matching value PM between the features within the region of the "actual tracking target" in the current frame and the features of the true target region in the previous frame; if PM ≤ ThT, it means that the appearance features of the targets in the two consecutive frames are significantly different, and it is very likely that the tracker has lost the true target and instead locked on to a similar-looking interference object. At this time, the system initializes a new tracking target box centered on the position predicted in the previous cycle to forcibly correct the tracker.

[0042] Case 3: The target is occluded. If I < ThI and PM > ThT, it means that the target within the current box is similar to the target in the previous frame in terms of features, but the position deviates from the prediction. This is usually caused by the true target being temporarily partially occluded, resulting in the detection box shifting. At this time, the system updates the current tracking target box with the predicted position to maintain the lock on the potential position of the target and waits for the target to reappear completely.

[0043] After completing the judgment and correction in step S5, the system releases the memory occupied by the processed complete stitched image generated in step S3. Then, it removes the oldest frame data from the video frame sequence, pitch angle sequence, and yaw angle sequence, and incorporates the latest frame data and its angle information to update the entire sequence.

[0044] After the update, the system returns to step S3 and starts the next round of feature matching, trajectory prediction, and status judgment based on the new sequence. This process repeats continuously to achieve continuous and robust tracking of the UAV target until an external stop instruction is received, or the system determines that the target has been lost for a long time in multiple consecutive frames (such as the IoU is continuously 0 and the feature matching value is extremely low).

[0045] In some embodiments, recalculating the tracking target box based on the predicted position includes the following steps: Predict the target prediction box for the current frame based on the target position in the previous frame, obtain the current frame target position search box with a region twice the length and width of the target prediction box of the current frame, perform template matching within the current frame target position search box to find the most similar target position, and judge the similarity of the most similar target position. If it is greater than or equal to the preset relocation threshold, it is determined that the relocation is successful.

[0046] In this embodiment, taking Figure 2 as an example, such as Figures 2-4As shown, after stitching together N frames of images, the target position in the fifth frame is predicted using the target information sequence. If the intersection-union ratio (IU) between the predicted target box and the actual tracking box is less than ThI, and PM ≤ ThT, it is determined that a similar interfering target has been tracked. The target prediction box in the fifth frame is predicted based on the target position in the fourth frame, with the search range being twice the length and width of the predicted target box in the fifth frame (e.g., ...). Figure 3 As shown, the target location search box in the fifth frame is obtained. Template matching is performed within the fifth frame target location search box to find the most similar target location. The similarity of the most similar target location is then judged. If it is greater than or equal to the preset relocation threshold, the relocation is considered successful (e.g., ...). Figure 4 (As shown).

[0047] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A UAV tracking method based on continuous local frame matching, characterized in that: Includes the following steps: S1. Target Recognition and Selection Stage: The video stream mode is used to identify the drone target. If a single drone target is identified, it is set as the tracking target; if multiple drone targets are identified, one target is selected as the tracking target according to preset rules. S2, Target Tracking and Frame Sequence Management Stage: Track the selected target and control the turntable equipped with the camera to position the target at the center of the camera image; starting from the first frame of tracking, save N consecutive video images to form a video frame sequence; S3, Angle Information Recording and Continuous Frame Matching Stage: Record the pitch and yaw angles of the turntable as it rotates continuously during image acquisition, and obtain the pitch angle sequence and yaw angle sequence for each frame of video information; then perform feature matching on the continuous frames in the video frame sequence to obtain a complete stitched image containing the tracked target, and record the position sequence of the tracked target in the complete stitched image; S4. Trajectory Prediction Stage: Based on the position sequence of the tracked target, the trajectory prediction algorithm is used to predict the predicted position of the tracked target in the next frame, and the predicted position is used as a constraint to continuously track the target. S5. Tracking Status Judgment and Correction Stage: Calculate the intersection-over-union ratio (IoU) between the actual tracked target position and the predicted position in the next frame. I Based on intersection-union ratio I The calculation results determine the target tracking status, and the corresponding tracking correction operation is performed according to the different statuses. S6, Cyclic Update Phase: Release the complete stitched image of the tracked target, and update the video frame information, pitch angle sequence information and yaw angle sequence information. Then return to step S3 and cycle through steps S3 to S5 until a stop tracking command is received or it is determined that the target has been lost for a long time. Feature matching of consecutive frames includes the following sub-steps: S31. Perform SIFT feature extraction on consecutive frames in the video frame sequence, and divide the feature points of each frame into the drone part and the background part. The drone feature points are denoted as follows: P U UAV feature vector f U Background feature points are denoted as P B The background feature vector is denoted as f B ; S32. Based on background feature points and background feature vectors, perform feature matching on adjacent frames to generate a matching matrix. M B and the rotation matrix formed by rotating the turntable. M R As a constraint; ,in Yi This represents the yaw angle of the turntable rotation corresponding to adjacent frame images. Pi This represents the pitch angle of the turntable rotation corresponding to adjacent frame images. i The frame number is 1 < 1. i ≤N; S33. Calculate the matching matrix M B With rotation matrix M R similarity R s ,like R s Greater than the preset threshold T Then, feature points with a matching degree lower than the preset matching degree in the background features are removed, and matching is repeated until the matching matrix is ​​obtained. M B Satisfy rotation matrix M R Constraints; S34. Calculate each adjacent frame image sequentially until a complete stitched image is formed, and record the position sequence of the tracked target in the complete stitched image.

2. The UAV tracking method based on continuous local frame matching according to claim 1, characterized in that: The preset rule is as follows: calculate the distance from each identified target to the center point of the video frame image. L i : L i =| x i - cx |+| y i - cy |, among which x i Let x be the x-coordinate of the center point of the i-th target. y i Let be the ordinate of the center point of the i-th target. cx The x-coordinate of the center point of the video frame image. cy The ordinate of the center point of the video frame image; Select L i The target with the smallest value is selected as the tracking target.

3. The UAV tracking method based on continuous local frame matching according to claim 1, characterized in that: In the aforementioned S2 target tracking and frame sequence management stage, the number of frames N is dynamically calculated based on the memory required for video imaging, the memory resources of the computing platform, and the drone's movement speed. ,in p The preset percentage value, R To calculate the total memory usage of the platform, m This represents the memory usage of a single frame of an image.

4. The UAV tracking method based on continuous local frame matching according to claim 1, characterized in that: The trajectory prediction algorithm is the Kalman filter algorithm.

5. The UAV tracking method based on continuous local frame matching according to claim 1, characterized in that: The S5 tracking status judgment and correction stage also includes the following steps: The predicted location is denoted as ( x p , y p , w p , h p ),in x p To predict the x-coordinate of the top-left corner of the target bounding box, y p To predict the ordinate of the top-left corner of the target bounding box, w p To predict the width of the target bounding box, h p To predict the height of the target bounding box; The actual location of the tracked target is denoted as ( x t , y t , w t , h t )in x t The x-coordinate of the top left corner of the actual tracking target bounding box. y t The actual tracking target bounding box's top-left ordinate. w t To track the actual target bounding box width, h t This is the actual height of the target bounding box being tracked. Calculate the center point positions of the predicted bounding box and the ground truth tracked bounding box: , , , ,in To predict the x-coordinate of the center point of the target bounding box, To predict the ordinate of the center point of the target bounding box, To accurately track the x-coordinate of the center point of the target bounding box, To accurately track the ordinate of the center point of the target bounding box; when or Then the intersection and union ratio I =0, otherwise ; Set the intersection-union ratio threshold ThI and the feature matching threshold ThT; when I If the value is ≥ThI, the tracking is considered successful, and the next frame is processed. When I At this time, calculate the feature matching value PM of the target area in the current frame and the previous frame; if PM ≤ ThT, it is determined that a similar interference target is tracked, and the tracking target box is recalculated according to the predicted position obtained in step S4; if PM > ThT, it is determined that the tracking target is occluded, and the tracking target box is updated with the predicted position obtained in step S4.

6. The UAV tracking method based on continuous local frame matching according to claim 5, characterized in that: Recalculating the tracking bounding box based on the predicted location includes the following steps: Based on the target position of the previous frame, predict the target prediction box of the current frame. Use the area twice the length and width of the target prediction box of the current frame as the search range to obtain the target position search box of the current frame. Perform template matching within the target position search box of the current frame to find the most similar target position and judge the similarity of the most similar target position. If it is greater than or equal to the preset relocation threshold, the relocation is judged to be successful.

Images