Unmanned aerial vehicle aerial video vehicle robust tracking method in anti-shielding scene

Abstract

The invention relates to an unmanned aerial vehicle aerial video vehicle robust tracking method in an anti-shielding scene, and the method comprises the steps: obtaining a road traffic video through the fixed visual angle high-altitude shooting of an unmanned aerial vehicle; marking a specific area in the video frame based on the polygon mask, and generating coordinate points of a polygon area; performing target detection on the video frame based on a pre-trained target detection model, and outputting a vehicle detection frame and category information; through adaptive target tracking, shielding area tracking is carried out on video vehicles in a shielding area, non-shielding area tracking is carried out on video vehicles in a non-shielding area, and a vehicle tracking trajectory with a stable ID and a flow statistical result are obtained; and matching a lane for each video vehicle, and outputting a vehicle tracking trajectory with a stable ID and a traffic statistical result, wherein the vehicle tracking trajectory has a serial number of the lane where the tracked vehicle is located. Compared with the prior art, the problems that the vehicle cannot be continuously tracked and the tracking precision is low due to the shielding object are solved.

Description

Technical Field

[0001] This invention relates to the field of transportation engineering, and in particular to a robust vehicle tracking method for drone aerial video in obstruction-resistant scenarios. Background Technology

[0002] With the acceleration of urbanization, the number of cars on urban roads has increased dramatically, often accompanied by various forms of congestion. The rapid development of drone technology has provided a completely new approach to traditional traffic monitoring technology, enabling effective traffic flow statistics and violation tracking. It has significant research and application value for alleviating traffic congestion, preventing traffic accidents, improving road utilization efficiency, and strengthening public safety monitoring.

[0003] Existing vehicle tracking technologies primarily rely on research findings in image processing, computer vision, and computer learning to perform target detection and ID assignment on drone-captured images to achieve vehicle tracking. However, in real-world applications using drone overhead views, vehicles may temporarily disappear due to physical obstructions when passing over road structures such as gantries. Traditional tracking algorithms may erroneously terminate tracking and assign new IDs, meaning the same vehicle may be incorrectly assigned a new ID during continuous tracking, leading to tracking interruptions and statistical errors, affecting the accuracy of vehicle counting, trajectory analysis, and traffic flow statistics. Patent application CN106791277A proposes a vehicle tracking method in video surveillance that relies on manually designed features for video vehicle tracking. It matches vehicles based on the closest distance between consecutive frames (i.e., the same vehicle). However, this tracking method struggles to continuously re-identify vehicles when they pass over obstacles, resulting in low vehicle tracking accuracy.

[0004] Therefore, there is an urgent need to develop robust tracking algorithms that can be used to identify occluded vehicles, providing more accurate data support for intelligent traffic management. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a robust vehicle tracking method for drone aerial video in occlusion scenarios, solving the problems of continuous vehicle tracking and low tracking accuracy caused by occlusion in existing drone aerial video vehicle tracking technologies.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A robust vehicle tracking method for drone aerial video in occlusion-resistant scenarios, the method comprising:

[0008] Road traffic videos were captured by drones from a fixed aerial perspective.

[0009] Based on polygonal masking, specific regions in a video frame are labeled, and coordinate points of the polygonal regions are generated. The specific regions include occlusion areas, background interference areas, and the range of each lane.

[0010] The video frame is used to detect objects based on a pre-trained object detection model, outputting vehicle detection boxes and category information, and then converting them into the input format required by the preset multi-object tracking algorithm.

[0011] Based on vehicle detection boxes and category information, adaptive target tracking is used to perform occlusion region tracking for video vehicles in occluded areas and non-occluded region tracking for video vehicles in non-occluded areas, obtaining vehicle tracking trajectories with stable IDs and traffic statistics results.

[0012] Based on the coordinates of the polygonal region and the vehicle tracking trajectory with a stable ID, lanes are matched for each video vehicle, and the vehicle tracking trajectory with a stable ID containing the lane number of the tracked vehicle and traffic statistics results are output.

[0013] Furthermore, the process of annotating specific regions in a video frame based on polygon masks includes:

[0014] The road traffic video is split into individual video frames. Specific areas, including occluded areas, background interference areas, and lane ranges, are labeled within each video frame. The labeling results are saved in dictionary form, and the coordinates of the specific areas are output as the coordinates of the polygonal regions.

[0015] Furthermore, the pre-trained target detection model is the YOLOv5 model, and the preset multi-target tracking algorithm is the DeepSORT algorithm.

[0016] Furthermore, when performing target detection on video frames based on the pre-trained target detection model, the confidence threshold of the pre-trained target detection model is set to a low confidence threshold of no more than 0.1, resulting in the output of more potential target vehicle detection boxes.

[0017] Furthermore, the process of converting the vehicle detection bounding box and category information into the input format required by the preset multi-object tracking algorithm includes:

[0018] The grid coordinates of the vehicle detection boxes output by the pre-trained target detection model are restored to the original image coordinates to form independent coordinate format detection boxes and category information;

[0019] Each absolute coordinate is converted into a relative coordinate format required by a preset multi-target tracking algorithm, wherein the center point coordinates and width and height of each relative coordinate format are normalized relative to the original image size.

[0020] Furthermore, the adaptive target tracking includes:

[0021] Based on the coordinates of the polygonal region and the vehicle detection box, the intersection-union ratio (IU / R) between the vehicle detection box and the occlusion region is calculated. If the IU / R is greater than a preset occlusion threshold, the video vehicle is determined to be in the occluded region; otherwise, the video vehicle is determined to be in the non-occluded region. Occlusion region tracking is performed for video vehicles in the occluded region, and non-occluded region tracking is performed for video vehicles in the non-occluded region. During the tracking process, a Kalman filter is used to predict and update the vehicle motion state, and the filter parameters are dynamically adjusted according to the target's location to improve the accuracy and robustness of the tracking.

[0022] Furthermore, the non-occlusion region tracking includes:

[0023] Obtain the vehicle detection bounding box and category information of the current frame, and use the Kalman filter to predict the position of the vehicle in the next frame based on the uniform motion model to obtain the vehicle prediction bounding box.

[0024] If the target vehicle in the vehicle detection box is a confirmed tracking target, then the detection box and the prediction box are associated by combining appearance features and motion features;

[0025] If the target vehicle in the vehicle detection box is an unmatched detection box and an unconfirmed tracking target, supplementary matching is performed based on the cross-union ratio, and the filter parameters and appearance feature library of this unconfirmed tracking target are updated. A new ID is assigned to the unmatched detection box. If the continuous supplementary matching fails for more than the preset number of live frames, the detection box and the tracking target are deleted.

[0026] Furthermore, the occlusion area tracking includes:

[0027] Obtain the vehicle detection bounding box and category information of the current frame, and adjust the noise parameters of the Kalman filter based on the feature information of the occluded area to enhance the dependence on target motion prediction in order to predict the position of all detected video vehicles in the next frame and obtain the vehicle prediction bounding box.

[0028] During the detection phase of each frame of the video, the overlap area and intersection-union ratio of the vehicle detection box and the occluded area are calculated. If the overlap area exceeds the threshold, the detection box is filtered out.

[0029] The predicted bounding boxes and detection boxes of confirmed targets are correlated based on Mahalanobis distance, and unmatched detection boxes and unconfirmed tracked targets are removed.

[0030] Furthermore, when outputting vehicle tracking trajectories and traffic statistics results with stable IDs, a visualization interface is also used to display the vehicle tracking trajectories and traffic statistics results with stable IDs. This includes displaying the IDs that match the detection boxes and drawing the corresponding IDs and detection boxes for each vehicle in each frame of the video.

[0031] Furthermore, the process of outputting vehicle tracking trajectories with stable IDs and traffic statistics results, including the lane number of the tracked vehicle, includes:

[0032] Obtain the coordinate data of each lane range and the video frame number, vehicle ID, vehicle category, and vehicle bounding box coordinates in the vehicle tracking trajectory with stable ID;

[0033] Calculate the intersection-over-union ratio (IoU) between each lane profile and each vehicle bounding box, and select the lane with the highest IoU with each vehicle as the lane containing the corresponding vehicle.

[0034] Add the lane ID to the vehicle tracking trajectory with stable ID, and output the vehicle tracking trajectory with stable ID containing the lane number of the tracked vehicle and traffic statistics results;

[0035] If the intersection ratio of a lane with any vehicle is lower than the preset intersection ratio threshold, then the lane is marked as unmatched.

[0036] Compared with the prior art, the beneficial effects of the present invention include:

[0037] 1. This invention achieves real-time, high-precision vehicle tracking by incorporating an adaptive anti-occlusion tracking method based on a deep learning model. This solves the problems of discontinuous vehicle tracking and low tracking accuracy caused by occlusion in existing UAV-based vehicle tracking technologies. The invention uses polygonal masks to label occluded areas, background interference areas, and lane ranges, enabling precise location of key areas and reducing background interference. An adaptive target tracking mechanism is employed to track video vehicles in non-occluded areas and those in occluded areas differently, ensuring the continuity of tracking for the same vehicle and resolving the tracking interruption problem caused by occlusion in traditional algorithms. The invention outputs trajectories with stable IDs, traffic statistics results, and a visualization interface, avoiding duplicate counting of the same vehicle and improving the accuracy of traffic flow statistics and vehicle counting. Furthermore, lane matching outputs results with lane numbers, providing precise data support for intelligent traffic management such as lane-specific traffic flow analysis and lane-level trajectory management.

[0038] 2. This invention uses polygonal masks to mark occluded areas, background interference areas, and lane ranges, which can selectively retain the detection bounding boxes of occluded vehicles, solving the problem that traditional algorithms directly discard targets due to interference from occlusions, and improving the detection efficiency of vehicles in occluded areas.

[0039] 3. When performing target detection on video frames based on a pre-trained target detection model, this invention sets a low confidence threshold to avoid filtering out targets due to partial occlusion of vehicles, ensuring that potential targets are not missed. This solves the problem of traditional high-threshold detection losing targets in occluded scenarios and can retain more potential targets.

[0040] 4. After outputting the vehicle detection box and category information, this invention converts them into the input format required by the preset multi-target tracking algorithm, standardizes the input data format, reduces tracking errors caused by format incompatibility, and ensures the continuity of the detection and tracking process.

[0041] 5. This invention performs standard tracking processing for vehicles in non-occluded areas, combining two-level matching of appearance features and motion features to reduce ID confusion caused by similar vehicle appearances or brief overlaps, ensuring stable tracking in normal scenarios. For vehicles in occluded areas, the occluded areas are first filtered as invalid regions, and then the target is matched only based on motion features. Furthermore, new IDs are prohibited from being assigned in occluded areas. This solves the problem of traditional algorithms erroneously terminating tracking and assigning new IDs when vehicles enter or leave occluded areas, ensuring that the ID of the same vehicle remains stable and avoiding tracking interruptions. Attached Figure Description

[0042] Figure 1 This is a flowchart of the method of the present invention;

[0043] Figure 2 This is a flowchart of the non-occlusion region tracking method of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0045] Example 1

[0046] This embodiment proposes a robust vehicle tracking method for drone aerial video in occlusion-resistant scenarios, the method as follows: Figure 1 As shown, steps S1-S6 are included, as detailed below.

[0047] Step S1: Obtain road traffic video by taking aerial photos from a fixed perspective using a drone.

[0048] Step S2: Based on the polygon mask annotation of a specific region in the video frame, generate the coordinate points of the polygon region.

[0049] In step S2, the specific area includes the occlusion area, the background interference area, and the range of each lane.

[0050] The process of marking specific regions in a video frame using polygon masks includes:

[0051] The road traffic video is broken down into individual video frames. Within each frame, specific regions, including occluded areas, background interference areas, and the range of each lane, are labeled. The labeling results are saved in dictionary form, and the coordinates of the specific regions are output as the coordinates of the polygon regions. Lanes are also numbered during the labeling process for subsequent lane matching.

[0052] Step S3: Target detection is performed on the video frames based on the pre-trained target detection model, and vehicle detection boxes and category information are output and converted into the input format required by the preset multi-target tracking algorithm.

[0053] In this embodiment, the pre-trained target detection model used is the YOLOv5 model, and the preset multi-target tracking algorithm is the DeepSORT algorithm. When performing target detection on UAV aerial video frames based on the YOLOv5 model, the input video frames are first normalized and multi-scale feature maps are output.

[0054] When performing object detection on video frames based on a pre-trained object detection model, the confidence threshold of the pre-trained object detection model is set to a low confidence threshold of no more than 0.1 to output more potential target vehicle detection boxes, in order to ensure the detection effect of occluded vehicles (such as targets that are only partially visible).

[0055] The process of converting vehicle detection bounding boxes and category information into the input format required by a predefined multi-object tracking algorithm includes:

[0056] The grid coordinates of the vehicle detection boxes output by the pre-trained target detection model are restored to the original image coordinates, forming independent coordinate format detection boxes and category information;

[0057] Each absolute coordinate is converted into a relative coordinate format required by a preset multi-target tracking algorithm, wherein the center point coordinates and width and height of each relative coordinate format are normalized relative to the original image size.

[0058] Step S4: Based on the vehicle detection bounding box and category information, adaptive target tracking is used to perform occlusion region tracking for video vehicles in occluded areas and non-occluded region tracking for video vehicles in non-occluded areas.

[0059] Adaptive target tracking includes:

[0060] Based on the coordinates of the polygonal region and the vehicle detection bounding box, the intersection-union ratio (IUR) between the vehicle detection box and the occluded region is calculated. If the IUR is greater than a preset occlusion threshold, the video vehicle is determined to be in an occluded region; otherwise, it is determined to be in an unoccluded region. Occluded region tracking is performed for video vehicles in occluded regions, and unoccluded region tracking is performed for video vehicles in unoccluded regions. During the tracking process, a Kalman filter is used to predict and update the vehicle's motion state, and the filter parameters are dynamically adjusted according to the target's location to improve tracking accuracy and robustness.

[0061] Unobstructed area tracking includes:

[0062] Obtain the vehicle detection bounding box and category information of the current frame, and use the Kalman filter to predict the position of all detected video vehicles in the next frame based on the uniform motion model to obtain the vehicle prediction bounding box.

[0063] If the target vehicle in the vehicle detection box is a confirmed tracking target, then the detection box and the prediction box are associated by combining appearance features and motion features;

[0064] If the target vehicle in the vehicle detection box is an unmatched detection box and an unconfirmed tracking target, supplementary matching is performed based on the cross-union ratio, and the Kalman filter parameters and appearance feature library of this unconfirmed tracking target are updated. A new ID is assigned to the unmatched detection box. If the continuous supplementary matching fails for more than the preset number of live frames, the detection box and the tracking target are deleted.

[0065] The uniform motion model in the Kalman filter is designed directly based on the kinematic assumptions of the physical world. The uniform motion model assumes that the target's velocity remains constant over a short period of time.

[0066] In this embodiment, the uniform motion model is represented by a state vector, which includes position, velocity, and other motion parameters. The vehicle's state vector is: [x,y,a,h,vx,vy,va,vh], where (x,y) represents the coordinates of the center point of the target bounding box, a is the aspect ratio, h is the height, and (vx,vy,va,vh) are the corresponding velocity parameters.

[0067] Occlusion area tracking includes:

[0068] Obtain the vehicle detection bounding box and category information of the current frame, and adjust the noise parameters of the Kalman filter according to the characteristics of the occluded area to enhance the dependence on the target motion prediction, i.e. the uniform motion model, to predict the position of all detected video vehicles in the next frame and obtain the vehicle prediction bounding box.

[0069] During the detection phase of each frame of the video, the overlap area and intersection-union ratio (IOU) between the vehicle detection box and the occluded area are calculated. If the threshold is exceeded, the detection box is filtered out.

[0070] The predicted bounding boxes and detection boxes of confirmed targets are correlated based on Mahalanobis distance, and unmatched detection boxes and unconfirmed tracked targets are removed.

[0071] Adjusting the noise parameters of the Kalman filter enhances the reliance on target motion prediction to predict the next frame position of all detected video vehicles and obtain vehicle prediction boxes. Specifically, this is achieved by modifying the process noise covariance matrix (Q) and measurement noise covariance matrix (R) in the Kalman filter to change the weight distribution between the prediction model and the observation data.

[0072] The Kalman filter achieves state estimation through two core steps: predicting the state at the next moment based on a uniform motion model and correcting the prediction by incorporating actual observed vehicle detection boxes. The weights of these two steps are determined by noise parameters. The process noise covariance matrix (Q) represents the uncertainty of the system model. The larger the Q value, the lower the reliability of the uniform motion model prediction, and the more the filter will rely on the current vehicle detection box. The measurement noise covariance matrix (R) represents the uncertainty of the observed data. The larger the R value, the lower the reliability of the vehicle detection box prediction, and the more the filter will rely on the uniform motion model prediction.

[0073] In occluded scenarios, some or all of the vehicle's features may not be captured by the detector, leading to a decrease in the accuracy of the detection box. In such cases, it is necessary to adjust the parameters to enhance the reliance on historical motion information.

[0074] For example, when occlusion is detected, the diagonal elements of the R matrix can be temporarily increased, i.e., the variance of the observation dimensions such as position and size, to reduce the confidence of the current detection box and reduce the impact of observation on state estimation. When occlusion is detected, the diagonal elements of the Q matrix can also be reduced, especially the velocity-related dimensions. During the occlusion period, the filter will predict the target trajectory more smoothly and reduce the prediction divergence caused by lack of observation.

[0075] The process noise covariance matrix (Q) and measurement noise covariance matrix (R) are initially set based on historical data and experience, and the parameters are dynamically adjusted according to the scenario during operation.

[0076] Step S5: Obtain vehicle tracking trajectories and traffic statistics results with stable IDs, and display the vehicle tracking trajectories and traffic statistics results with stable IDs through a visualization interface.

[0077] The system displays vehicle tracking trajectories and traffic statistics with stable IDs through a visual interface. It shows the IDs that match the output bounding boxes and draws the corresponding IDs and bounding boxes for each vehicle in each frame of the video.

[0078] Traffic statistics can be obtained by counting the number of vehicles with stable IDs (number of detection boxes).

[0079] Step S6: Based on the coordinate points of the polygonal region and the vehicle tracking trajectory with stable ID, match lanes for each video vehicle, and output the vehicle tracking trajectory with stable ID and traffic statistics results, which contain the lane number of the tracked vehicle.

[0080] The process of outputting vehicle tracking trajectories with stable IDs and the lane number of the tracked vehicle, along with traffic statistics, includes:

[0081] Obtain the coordinate data of each lane range and the video frame number, vehicle ID, vehicle category, and vehicle bounding box coordinates in the vehicle tracking trajectory with stable ID;

[0082] Calculate the intersection-over-union ratio (IoU) between each lane profile and each vehicle bounding box, and select the lane with the highest IoU with each vehicle as the lane containing the corresponding vehicle.

[0083] Add the lane ID to the vehicle tracking trajectory with stable ID, and output the vehicle tracking trajectory with stable ID containing the lane number of the tracked vehicle and traffic statistics results;

[0084] If the intersection ratio of a lane with any vehicle is lower than the preset intersection ratio threshold, then the lane is marked as unmatched.

[0085] Step S6 specifically involves loading the contour data annotated with polygon masks from step S2, and reading vehicle detection information from the results obtained in step S5, including frame number, ID, category, and bounding box coordinates. When the intersection-over-union ratio (IoU) is below the threshold (0.0), it is marked as -1 to indicate no match. Finally, the matching result (road ID) is appended to the original vehicle data file to generate a result file with the lane number of the tracked vehicle.

[0086] Below are examples of specific usage processes.

[0087] Track vehicles on a specific road.

[0088] First, control the drone to take high-altitude shots from a fixed perspective to acquire road traffic videos. After acquiring the videos, process each video frame one by one, and mark the occluded areas, background interference areas, and the range of each lane in the video frames with polygonal masks on the images. Save the annotation results in dictionary form to complete the annotation and output of specific areas.

[0089] Target detection based on UAV aerial video frames using the YOLOv5 model: The YOLOv5 model is set to a low confidence threshold, the input video frames are normalized, and multi-scale feature maps are output.

[0090] The grid coordinates output by the model are restored to the original image coordinates, forming independent coordinate format detection boxes and category information. These absolute coordinates are then converted to the relative coordinate format required for DeepSORT tracking, where the center point coordinates and width and height are normalized relative to the original image dimensions.

[0091] Accurate vehicle target tracking is achieved through an adaptive target tracking mechanism: Vehicle detection boxes and category information for the current frame are acquired, and then a Kalman filter is used to predict the next frame position of all tracked targets. Based on a uniform motion model, predicted bounding boxes are generated. For vehicles in non-occluded areas, a two-level strategy is employed. The first level (confirmed tracked targets) integrates appearance features (cosine distance) and motion features (Mahathano distance) to associate detection boxes with predicted boxes. The second level (unmatched detection boxes and unconfirmed targets) performs supplementary matching based on the intersection-over-union ratio (IoU). The matching threshold is set to 0.3. If the IoU of unmatched detection boxes and unconfirmed targets exceeds the threshold, a successful match is achieved. After a successful match, the Kalman filter parameters and appearance feature library of the tracked targets are updated, and new IDs are assigned to unmatched detection boxes. Simultaneously, targets that have failed to match consecutively for more than a preset number of live frames are deleted. For vehicles in occluded areas, if the detection box overlaps with the occluded area, it is marked as invalid and will not participate in the matching. If the overlap does not exceed the limit, the predicted box and detection box of the confirmed target are associated through Mahalanobis distance. In the matching of vehicles in occluded areas, new IDs are not allowed to be assigned to avoid the same vehicle being misjudged as a new target when entering or leaving the occluded area.

[0092] Output: The system outputs vehicle tracking trajectories and traffic statistics results with stable IDs, and displays these results through a visual interface. This mainly includes outputting the IDs that match the detection boxes and drawing the corresponding IDs and detection boxes in each frame of video.

[0093] Lane matching: Load the lane contour data previously annotated with polygon masks and the vehicle detection information output by the system (including frame number, ID, category, and bounding box coordinates). Calculate the intersection-union ratio (IURR) of each lane mask contour with the vehicle bounding box, and select the mask ID with the highest IURR. When the IURR is below the threshold (0.0), it is marked as -1 to indicate no match. Finally, the matching results (mask IDs) are appended to the original vehicle data file to generate a result file with the lane number of the tracked vehicle.

[0094] The actual vehicle trajectories, traffic flow statistics, and the lanes in which the vehicles were located were compared with the predicted results, and the results were basically consistent.

[0095] In summary, this embodiment provides a robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios. By annotating specific regions of video frames using a labeling tool and employing the YOLOv5 model for recognition and output, it effectively preserves some detection bounding boxes of identified targets, preventing target loss. Furthermore, this method utilizes an improved DeepSort model to implement special processing schemes for invalid regions. These invalid regions include occluded areas (e.g., gantry cranes) and background interference areas (e.g., lane markings). By processing these invalid regions, the impact of occlusion on target tracking is eliminated.

[0096] Example 2

[0097] Based on Embodiment 1, this embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for executing the aforementioned robust vehicle tracking method for drone aerial video in anti-occlusion scenarios.

[0098] At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios described above. Of course, in addition to software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0099] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0100] Computer-readable media (AA) include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0101] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A robust vehicle tracking method for drone aerial video in occlusion-resistant scenarios, characterized in that, The method includes: Road traffic videos were captured by drones from a fixed aerial perspective. Based on polygonal masking, specific regions in a video frame are labeled, and coordinate points of the polygonal regions are generated. The specific regions include occlusion areas, background interference areas, and the range of each lane. The video frame is used to detect objects based on a pre-trained object detection model, outputting vehicle detection boxes and category information, and then converting them into the input format required by the preset multi-object tracking algorithm. Based on vehicle detection boxes and category information, adaptive target tracking is used to perform occlusion region tracking for video vehicles in occluded areas and non-occluded region tracking for video vehicles in non-occluded areas, obtaining vehicle tracking trajectories with stable IDs and traffic statistics results. Based on the coordinates of the polygonal region and the vehicle tracking trajectory with a stable ID, lanes are matched for each video vehicle, and the vehicle tracking trajectory with a stable ID containing the lane number of the tracked vehicle and traffic statistics results are output.

2. The robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios according to claim 1, characterized in that, The process of annotating specific regions in a video frame using polygon masks includes: The road traffic video is split into individual video frames. Specific areas, including occluded areas, background interference areas, and lane ranges, are labeled within each video frame. The labeling results are saved in dictionary form, and the coordinates of the specific areas are output as the coordinates of the polygonal regions.

3. The robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios according to claim 1, characterized in that, The pre-trained target detection model is the YOLOv5 model, and the preset multi-target tracking algorithm is the DeepSORT algorithm.

4. The robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios according to claim 1, characterized in that, When performing target detection on video frames based on a pre-trained target detection model, the confidence threshold of the pre-trained target detection model is set to a low confidence threshold of no more than 0.1, resulting in the output of more potential target vehicle detection boxes.

5. A robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios according to claim 1, characterized in that, The process of converting the vehicle detection bounding box and category information into the input format required by the preset multi-object tracking algorithm includes: The grid coordinates of the vehicle detection boxes output by the pre-trained target detection model are restored to the original image coordinates to form independent coordinate format detection boxes and category information; Each absolute coordinate is converted into a relative coordinate format required by a preset multi-target tracking algorithm, wherein the center point coordinates and width and height of each relative coordinate format are normalized relative to the original image size.

6. The robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios according to claim 1, characterized in that, The adaptive target tracking includes: Based on the coordinates of the polygonal region and the vehicle detection box, the intersection-union ratio (IU / R) between the vehicle detection box and the occlusion region is calculated. If the IU / R is greater than a preset occlusion threshold, the video vehicle is determined to be in the occluded region; otherwise, the video vehicle is determined to be in the non-occluded region. Occlusion region tracking is performed on video vehicles in the occluded region, and non-occluded region tracking is performed on video vehicles in the non-occluded region.

7. A robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios according to claim 6, characterized in that, The non-occlusion area tracking includes: Obtain the vehicle detection bounding box and category information of the current frame, and use a preset filter to predict the position of all detected video vehicles in the next frame based on a uniform motion model to obtain the vehicle prediction bounding box. If the target vehicle in the vehicle detection box is a confirmed tracking target, then the detection box and the prediction box are associated by combining appearance features and motion features; If the target vehicle in the vehicle detection box is an unmatched detection box and an unconfirmed tracking target, supplementary matching is performed based on the cross-union ratio, and the filter parameters and appearance feature library of this unconfirmed tracking target are updated. A new ID is assigned to the unmatched detection box. If the continuous supplementary matching fails for more than the preset number of live frames, the detection box and the tracking target are deleted.

8. A robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios according to claim 6, characterized in that, The occlusion area tracking includes: Obtain the vehicle detection bounding box and category information of the current frame, and adjust the noise parameters of the preset filter based on the feature information of the occluded area to enhance the dependence on target motion prediction in order to predict the position of all detected video vehicles in the next frame and obtain the vehicle prediction bounding box. During the detection phase of each frame of the video, the overlap area and intersection-union ratio of the vehicle detection box and the occluded area are calculated. If the overlap area exceeds the threshold, the detection box is filtered out. The predicted bounding boxes and detection boxes of confirmed targets are correlated based on Mahalanobis distance, and unmatched detection boxes and unconfirmed tracked targets are removed.

9. A robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios according to claim 1, characterized in that, When outputting vehicle tracking trajectories and traffic statistics results with stable IDs, the system also displays these results through a visualization interface. This includes showing the IDs that match the detection boxes and drawing the corresponding IDs and detection boxes for each vehicle in each frame of the video.

10. A robust vehicle tracking method for UAV aerial video in occlusion-resistant scenarios according to claim 1, characterized in that, The process of outputting the vehicle tracking trajectory and traffic statistics results with a stable ID and the lane number of the tracked vehicle includes: Obtain the coordinate data of each lane range and the video frame number, vehicle ID, vehicle category, and vehicle bounding box coordinates in the vehicle tracking trajectory with stable ID; Calculate the intersection-over-union ratio (IoU) between each lane profile and each vehicle bounding box, and select the lane with the highest IoU with each vehicle as the lane containing the corresponding vehicle. Add the lane ID to the vehicle tracking trajectory with stable ID, and output the vehicle tracking trajectory with stable ID containing the lane number of the tracked vehicle and traffic statistics results; If the intersection ratio of a lane with any vehicle is lower than the preset intersection ratio threshold, then the lane is marked as unmatched.

Images