Method, system and equipment for detecting road vehicle speed and vehicle distance based on unmanned aerial vehicle vision

Abstract

The invention belongs to the technical field of intelligent traffic, and discloses a road vehicle speed and distance detection method, system and device based on unmanned aerial vehicle vision. The method comprises the steps of tracking a vehicle in a to-be-processed video aerially photographed by an unmanned aerial vehicle in real time through an improved YOLOv11 network and a BotSort algorithm to obtain a vehicle trajectory, and converting a vehicle pixel coordinate detected by each frame into a world coordinate in combination with a conversion matrix H of the pixel coordinate and the world coordinate to form a trajectory sequence of the vehicle in the world coordinate system; and for a certain vehicle i, according to the pixel coordinates at the t-th moment and the (t + t)-th moment in several continuous video frames, corresponding world coordinates are obtained through the conversion matrix H, the displacement between two points is calculated according to an Euclidean distance formula, and the average speed of the vehicle in the time period is calculated. According to the invention, efficient, stable and reliable vehicle speed measurement and vehicle distance calculation can be realized.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation technology, specifically relating to a method, system, and equipment for detecting road vehicle speed and distance based on UAV vision. Background Technology

[0002] With the rapid development of Intelligent Transportation Systems (ITS), higher requirements have been placed on real-time speed monitoring and distance detection of road vehicles.

[0003] Traditional detection methods such as inductive loops and microwave radar suffer from high deployment costs, poor flexibility, and high maintenance difficulty, making them unsuitable for the dynamic monitoring needs of complex traffic scenarios.

[0004] Machine vision-based detection technologies have become a research hotspot in the field of traffic monitoring due to their advantages such as non-contact operation, wide coverage, and rich information. Among them, the monitoring mode of drones equipped with visual sensors has shown great application potential in scenarios such as urban road monitoring, highway inspection, and emergency command for sudden incidents due to its unique advantages such as flexible high-altitude perspective, rapid deployment, and ability to realize large-scale traffic flow situation perception.

[0005] In vehicle detection and tracking technology, the YOLO series of algorithms are widely used in real-time target detection tasks due to their high-efficiency end-to-end detection capabilities. However, in complex road scenarios with dense traffic and frequent occlusion, the detection performance of traditional target detection algorithms degrades when viewed from the high-altitude perspective of UAVs for small target vehicles. Traditional multi-target tracking algorithms (such as DeepSORT) rely on a simple fusion of motion prediction and appearance features, which can easily lead to trajectory loss or identity (ID) switching confusion when the target moves rapidly, changes its posture, or is briefly occluded, resulting in the accumulation of continuous measurement errors in speed and distance.

[0006] In terms of coordinate transformation and parameter measurement, accurately converting pixel coordinates to world coordinates in a UAV vision system is a key technical challenge. Traditional calibration methods rely on high-precision calibration objects or complex computational models, which are cumbersome to deploy and have poor adaptability in real traffic scenarios. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies by providing a method, system, and device for detecting road vehicle speed and distance based on UAV vision. It integrates the improved YOLOv11 algorithm and the BoT-SORT tracking framework, and combines an innovative reference calibration method to achieve efficient vehicle speed measurement and distance calculation, providing a low-cost, high-precision solution for intelligent traffic management and autonomous driving roadside perception systems.

[0008] Specifically, the present invention is implemented using the following technical solutions.

[0009] On one hand, the present invention provides a road vehicle speed detection method based on UAV vision, comprising:

[0010] By improving the YOLOv11 network and using the BotSort algorithm, vehicles in the drone aerial videos are tracked in real time to obtain vehicle trajectories. Then, the pixel coordinates of the detected vehicles in each video frame are transformed using the pixel coordinates to world coordinates transformation matrix H. Convert to world coordinates This forms the trajectory sequence of the vehicle in the world coordinate system. , where t represents a time frame;

[0011] For a given vehicle i, suppose that in several consecutive video frames, the pixel coordinates at times t and t+∆t are respectively and The corresponding world coordinates are obtained through the transformation matrix H between pixel coordinates and world coordinates. and Calculate the displacement d between two points using the following Euclidean distance formula. i :

[0012] Calculate the average speed v of the vehicle during the time interval from t to (t+∆t). i for: .

[0014] Furthermore, the improved YOLOv11 network includes:

[0015] An EMA module is embedded in the C3k2 module of the backbone network to construct an attention-enhanced feature extraction unit;

[0016] The SPPF module in the neck network is upgraded to the SPPF_LSKA multi-scale fusion unit to construct a feature fusion structure that enhances long-range dependencies.

[0017] Based on the existing three-scale detection head, a new micro-target detection head has been added to form a four-level detection system.

[0018] Furthermore, the EMA module in the C3k2 module of the backbone network, which constructs an attention-enhanced feature extraction unit, includes:

[0019] The original features X∈R are input into the improved YOLOv11 network. C×H×W First, the feature maps are divided into multiple sub-feature groups and then reshaped through batch processing. The grouped feature maps are as follows:

[0020]

[0021] in, The feature map after grouping; reshape represents the grouping operation on X; B is the batch dimension; C is the number of feature maps; H is the feature map height; W is the feature map width; g is the number of groups;

[0022] The sub-features in each sub-feature group are fed into two parallel branches, which perform multi-scale feature extraction. One branch is the channel attention branch, which performs spatial average pooling on each sub-feature to extract global statistics. Then, it fuses channel information through 1×1 convolution and uses Sigmoid activation to generate channel attention weights. These weights are multiplied by the original features to achieve dynamic calibration of the channel dimension. The other branch is the spatial attention branch, which applies 3×3 convolution to each sub-feature to capture the local spatial context. Then, it performs average pooling to compress the spatial dimension. Finally, it generates spatial attention weights through Softmax. These weights are broadcast and expanded to 1×H×W for spatial dimension weighting.

[0023] The outputs of the two parallel branches are fused in both spatial and channel dimensions. The channel weights of the channel attention branch are added to the spatial weights of the spatial attention branch, and then normalized by Sigmoid to obtain the joint attention weights. These weights are multiplied by the original features to complete multi-scale attention modulation. The modulation results of all sub-feature groups are merged to restore the original channel dimension C×H×W. Finally, the output is obtained by superimposing it with the original features through residual connection.

[0024] Furthermore, upgrading the SPPF module in the neck network to an SPPF_LSKA multi-scale fusion unit to construct a long-range dependency-enhanced feature fusion structure includes:

[0025] 1-2-1) LSKA dynamic weighted pooling is used to replace the fixed pooling kernel of the SPPF module in the neck network of the original YOLOv11 network. It includes three core branches: a local receptive field branch, which extracts local detail features and captures high-frequency information through 3×3 convolution; a global receptive field branch, which expands the receptive field and models long-distance spatial dependencies through 7×7 convolution; and attention weight generation, which generates channel attention vector α through global average pooling and fully connected layers.

[0026] 1-2-2) Fusing features from different kernels with attention weights generates weighted multi-scale features:

[0027] 1-2-3) Spatial pyramid pooling reconstruction: according to the following formula, the weighted multi-scale features are combined in a step-by-step pooling kernel to capture the multi-granular semantic information of the target through pooling operations at different scales.

[0028] 1-2-4) Output feature recombination restores the pooled features to their original dimensions and connects them with the input feature residuals, ultimately outputting multi-scale features that are fused with local and global attention guidance.

[0029] Furthermore, the addition of a micro-target detection head to the existing three-scale detection head, forming a four-level detection system, includes:

[0030] Based on the original YOLOv11 detection head, a new higher-resolution micro-target detection head is added. This head generates the corresponding original target detection head by fusing shallow feature maps from the backbone network through two layers of bilinear upsampling. Figure 1 / 8 scale feature layer.

[0031] Furthermore, the transformation matrix H between pixel coordinates and world coordinates is obtained in the following way:

[0032] Select a segment of drone aerial video to be processed, and take several frames from the video to calibrate candidate regions; select regions with fixed physical size characteristics from the candidate regions as calibration benchmarks;

[0033] Establish a world coordinate system O−XYZ, with the road surface as the XY plane and the origin O set as a fixed feature point. For a reference object with known fixed physical dimensions, determine its world coordinates through geometric relationships. Select multiple sets of feature point pairs (u i ,v i )↔(X i ,Y i );

[0034] The transformation matrix H between pixel coordinates and world coordinates satisfies the following equation:

[0035]

[0036] Through at least 4 sets of feature point pairs (u i ,v i )↔(X i ,Y i Solve for the unknown parameter h of the transformation matrix H between pixel coordinates and world coordinates. 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 .

[0037] Furthermore, the fixed physical dimension feature is as follows:

[0038] The lane structure features in the candidate region include, but are not limited to, lane length, lane spacing along the road traffic direction, lane width, and lane spacing along the road traffic radial direction; or

[0039] Select a dynamic reference object and obtain its standard dimensions from a preset size database.

[0040] On the other hand, the present invention provides a road vehicle distance detection method based on UAV vision, including:

[0041] By improving the YOLOv11 network and BotSort algorithm, vehicles in the drone aerial video are tracked in real time, the ID of each vehicle is obtained, and the world coordinate information of the road lane lines is combined to determine whether the vehicles are in the same lane.

[0042] For two vehicles j and k in the same lane, extract their pixel coordinates. and Using the transformation matrix H between pixel coordinates and world coordinates, the pixel coordinates are converted into world coordinates respectively. and ;

[0043] The distance between two vehicles j and k in the same lane is calculated using the Euclidean distance formula. :

[0044] .

[0045] On the other hand, the present invention provides a road vehicle speed and distance detection system based on UAV vision, which realizes the above-mentioned road vehicle speed and distance detection method based on UAV vision, including a UAV perception subsystem, a vision processing server, a homography matrix calibration calculation engine, and a motion analysis module;

[0046] The UAV perception subsystem includes a high-definition camera and an edge computing unit; the high-definition camera acquires real-time video streams of the road and transmits them to the vision processing server; the edge computing unit performs image preprocessing on the acquired video streams.

[0047] The vision processing server includes an improved YOLOv11 detection module, a BoT-SORT tracking module, and a candidate region calibration unit. The improved YOLOv11 detection module detects input video frames and outputs the detection results, including the vehicle detection box's position information, category information, and confidence score, which are then sent to the BoT-SORT tracking module. The BoT-SORT tracking module uses an improved Kalman filter to predict the detection results, then reduces coordinate offset caused by camera motion through camera motion compensation, and finally outputs the tracked vehicle ID through a multi-feature fusion matching strategy. The candidate region calibration unit calibrates the road area based on a calibration method.

[0048] The homography matrix calibration calculation engine includes a reference object calibration module and a homography matrix calculator; the reference object calibration module calibrates the physical dimensions of the reference object; the homography matrix calculator solves for the transformation matrix H between pixel coordinates and world coordinates;

[0049] The motion analysis module includes a frame difference speed measurement module and / or a vehicle distance calculation module; the frame difference speed measurement module calculates the displacement change of the vehicle within a set time period by combining the timestamp to determine the vehicle speed; the vehicle distance calculation module combines the transformation matrix H between the pixel coordinates and world coordinates and the tracking vehicle ID to determine the distance between two tracking vehicles in the same lane.

[0050] In another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that: when the computer program is executed by a processor, it implements the steps of the above-mentioned road vehicle speed and distance detection method based on UAV vision.

[0051] The beneficial effects of the method, system, and equipment for detecting road vehicle speed and distance based on UAV vision of the present invention are as follows:

[0052] To improve the performance of the YOLOv11 algorithm in road vehicle detection tasks, the present invention provides a method, system, and device for road vehicle speed and distance detection based on UAV vision, which are specifically improved. The improved YOLOv11 algorithm significantly improves the detection accuracy and speed of road vehicles, and the output detection box coordinates, confidence scores, and categories are more accurate.

[0053] The present invention relates to a method, system, and device for detecting road vehicle speed and distance based on UAV vision. The BotSort algorithm is selected as the core of the tracking module. The BotSort algorithm can directly utilize the high-quality detection results output by the improved YOLOv11 algorithm. Through the joint measurement of cosine distance and Mahalanobis distance, it achieves deep fusion of detection and trajectory, forming a closed-loop optimization of "detection-tracking", which further improves the overall system performance. The BotSort algorithm, through its lightweight design, maintains high MOTA (multi-target tracking accuracy) while having better computational efficiency than similar algorithms, which can meet the real-time processing needs of UAVs and achieve "lightweight equipment".

[0054] In UAV vision inspection systems, accurate conversion from pixel coordinates to world coordinates is a core prerequisite for vehicle speed and distance measurement. This invention presents a method, system, and device for road vehicle speed and distance detection based on UAV vision. Its calibration technique, based on the reference object method, establishes a mapping relationship between pixel coordinates and world coordinates quickly by setting reference objects of known size in the scene and combining methods such as homography matrix solving. This effectively reduces calibration complexity and improves the system's robustness in different monitoring scenarios.

[0055] Whether in the complex traffic environment of urban roads or the long-distance monitoring of highways, the method, system, and equipment for detecting road vehicle speed and distance based on UAV vision of the present invention can achieve stable and reliable speed and distance measurement functions by dynamically updating the coordinate transformation matrix and adjusting the detection and tracking parameters in real time. It meets the real-time data requirements of intelligent transportation systems, provides strong support for traffic flow optimization and accident prevention, and provides a low-cost, high-precision solution for intelligent traffic management and autonomous driving roadside perception systems, with broad application prospects. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the system composition according to an embodiment of the present invention.

[0057] Figure 2 This is a detailed system flowchart of an embodiment of the present invention.

[0058] Figure 3 This is a schematic diagram of the improved YOLOv11 network structure according to an embodiment of the present invention.

[0059] Figure 4 This is a schematic diagram of the EMA added in the improved YOLOv11 according to an embodiment of the present invention.

[0060] Figure 5 This is a schematic diagram of the improved YOLOv11 results in the public dataset VisDrone, based on an embodiment of the present invention.

[0061] Figure 6This is a schematic diagram showing the correspondence between reference features and world coordinates in an embodiment of the present invention.

[0062] Figure 7 This is a schematic diagram of vehicle speed and distance measurement results according to an embodiment of the present invention. Detailed Implementation

[0063] The present invention will now be described in further detail with reference to the embodiments and the accompanying drawings.

[0064] Example 1:

[0065] One embodiment of the present invention is a road vehicle speed and distance detection system based on UAV vision, such as... Figure 1 As shown, it includes a drone perception subsystem, a vision processing server, a homography matrix calibration calculation engine, and a motion analysis module.

[0066] The drone perception subsystem includes a high-definition camera and an edge computing unit. The high-definition camera captures real-time video streams of the road and transmits them to a vision processing server. The edge computing unit, equipped with a high-performance edge computing module (such as the NVIDIA Jetson nano), performs preprocessing on the captured video stream, including image denoising and image enhancement.

[0067] The vision processing server includes an improved YOLOv11 detection module, a BoT-SORT tracking module, and a candidate region calibration unit. The improved YOLOv11 detection module detects input video frames and outputs the detection results, including vehicle bounding box location information, category information, and confidence scores, which are then sent to the BoT-SORT tracking module. The BoT-SORT tracking module uses an improved Kalman filter to predict the detection results, then uses camera motion compensation (CMC) to reduce coordinate offsets caused by camera motion, and finally outputs the tracking vehicle ID through a multi-feature fusion matching strategy, achieving real-time stable vehicle tracking. The candidate region calibration unit calibrates the road area based on a calibration method.

[0068] The homography matrix calibration calculation engine includes a reference object calibration module and a homography matrix calculator. The reference object calibration module is used to calibrate the physical dimensions of reference objects, such as the length of lane dashed lines, lane width, and vehicle length. The homography matrix calculator solves for the transformation matrix H between pixel coordinates and world coordinates, thereby establishing the projection relationship from the pixel coordinate system to the world coordinate system.

[0069] The motion analysis module includes a frame difference speed measurement module and a vehicle distance calculation module. The frame difference speed measurement module calculates the vehicle's displacement changes within a set time period using timestamps to determine the vehicle's speed. The vehicle distance calculation module uses the pixel coordinate to world coordinate transformation matrix H and the tracked vehicle ID to determine the distance between two tracked vehicles in the same lane.

[0070] Example 2:

[0071] One embodiment of the present invention is a method for detecting road vehicle speed and distance based on unmanned aerial vehicle (UAV) vision, such as... Figure 2 As shown, it includes the following steps:

[0072] 1. Real-time vehicle tracking in drone aerial videos is achieved by improving the YOLOv11 network and BotSort algorithm. The tracked vehicle trajectories are used, combined with a pixel coordinate to world coordinate transformation matrix H, to convert the detected vehicle pixel coordinates in each frame. Convert to world coordinates This forms the trajectory sequence of the vehicle in the world coordinate system. , where t represents a time frame.

[0073] In complex traffic scenarios, vehicle targets from the aerial perspective of drones often exhibit small-sized features, making traditional YOLO algorithms prone to missed or false detections. This invention, based on the existing YOLOv11 architecture, specifically optimizes three key components—the backbone network, feature fusion module, and detection head—to address the small target detection problem from the drone's aerial perspective. This includes incorporating the EMA attention mechanism, improving the SPPF module to a SPPF_LSKA multi-scale fusion module, and adding a small target detection head, forming a three-layer architecture of "backbone enhancement - multi-scale fusion - full-scale detection" to improve the model's accuracy in small target detection. The improved YOLOv11 architecture is as follows: Figure 3 As shown.

[0074] Specifically, the improved YOLOv11 network of this invention includes:

[0075] (i) An efficient multi-scale attention module (EMA) is embedded in the C3k2 module of the backbone network to construct an attention-enhanced feature extraction unit, such as... Figure 4 As shown. Specifically includes:

[0076] 1. The feature map X∈R input to the improved YOLOv11 network C×H×W First, the feature maps are divided into multiple sub-feature groups and then reshaped through batch processing to preserve complete channel information and reduce computational complexity. The grouped feature maps are shown below:

[0077]

[0078] in, The feature map after grouping; reshape represents grouping X; B is the batch dimension; C is the number of feature maps; H is the feature map height; W is the feature map width; g is the number of groups (e.g., 32).

[0079] 2. Sub-features in each sub-feature group are entered into two parallel branches, and multi-scale feature extraction is performed on the parallel branches.

[0080] A branch ( Figure 4 The middle and upper branches are channel attention branches: spatial average pooling (AvgPool) is performed on the sub-features to extract global statistics; then channel information is fused through 1×1 convolution, and sigmoid activation is used to generate channel attention weights; these weights are multiplied by the original sub-features to achieve dynamic calibration of the channel dimension;

[0081] Another branch ( Figure 4 The lower-middle branch is the spatial attention branch: it applies 3×3 convolutions to sub-features to capture local spatial context, then performs average pooling to compress the spatial dimension, and finally generates spatial attention weights through Softmax; these weights are broadcast and expanded to 1×H×W for spatial dimension weighting. This branch focuses on spatial features under different receptive fields, improving the ability to capture object edges and small targets.

[0082] 3. Cross-dimensional interaction and feature reorganization. The outputs of the two parallel branches are fused along the spatial and channel dimensions. The channel weights of the channel attention branch are added to the spatial weights of the spatial attention branch, and then normalized using Sigmoid to obtain the joint attention weights. These weights are multiplied by the original features to complete multi-scale attention modulation. The modulation results of all sub-feature groups are merged to restore the original channel dimensions C×H×W. Finally, the output is obtained by superimposing it with the original features through residual connections.

[0083] This invention embeds a high-efficiency multi-scale attention module (EMA) into the C3k2 module of the backbone network, employing a cross-space-channel interaction mechanism to extract local and global contextual information through parallel sub-branches and dynamically calibrate feature map channel weights. By optimizing feature channel weight allocation, it suppresses interference from complex backgrounds and focuses on key vehicle features. To preserve information in each channel and reduce computational overhead, the EMA mechanism reconstructs some channels into batch processing dimensions and groups channel dimensions into multiple sub-features, ensuring a uniform distribution of spatial semantic features within each feature group. Specifically, in addition to encoding global information to recalibrate channel weights in each parallel branch, it further aggregates the output features of two parallel branches through cross-dimensional interaction to capture pixel-level pairwise relationships. By incorporating this attention mechanism, false detections of vehicles and backgrounds (such as trees and shadows) from high-altitude perspectives are reduced; and multi-scale feature enhancement improves the ability to distinguish occluded or blurred vehicles. This method effectively combines global and local, channel and spatial attention while maintaining low computational cost and parameter count, avoiding the information loss caused by channel reduction in traditional methods.

[0084] (ii) Upgrade the SPPF module in the neck network of the original YOLOv11 network to an SPPF_LSKA multi-scale fusion unit to construct a feature fusion structure that enhances long-range dependencies, including:

[0085] 1-2-1) The dynamic weighted pooling of LSKA (Large Separable Kernel Attention) is used to replace the fixed pooling kernel of the SPPF (Spatial Pyramid Pooling-Fast) module in the neck network of the original YOLOv11 network, which contains three core branches:

[0086] (1) Local receptive field branches (small nuclei)

[0087]

[0088] Local detail features are extracted using 3×3 convolutions to capture high-frequency information such as edges and textures. Here, Fgroup represents the input features after grouping.

[0089] (2) Global receptive field branch (large kernel)

[0090]

[0091] Expanding the receptive field through 7×7 convolutions allows for the modeling of long-range spatial dependencies.

[0092] (3) Attention weight generation

[0093] The channel attention vector α is generated through global average pooling and a fully connected layer, as shown in the following equation:

[0094]

[0095] Where W is the learning parameter, used to dynamically allocate the importance of different kernels.

[0096] 1-2-2) Multi-scale feature fusion: Based on the following formula, features from different kernels are fused with attention weights to generate a weighted multi-scale feature F. fused :

[0097]

[0098] This operation preserves the original features through residual connections while enhancing the response in key regions.

[0099] 1-2-3) Spatial pyramid pooling reconstruction, performing step-by-step pooling kernel combination on the fused features:

[0100]

[0101] Capturing multi-granular semantic information F of the target through pooling operations of different scales. pool .

[0102] 1-2-4) Output feature recombination: restore the pooled features to their original dimensions and connect them with the input feature residuals.

[0103]

[0104] The final output integrates multi-scale features guided by local and global attention while maintaining the original dimensionality.

[0105] The serial pooling structure in the original SPPF (Spatial Pyramid Pooling Fast Module) is replaced with the SPPF_LSKA module, and a lightweight large kernel attention (LSKA) unit is introduced. Long-range spatial dependencies are extracted through decomposed large kernel convolution (such as 7×7 depthwise separable convolution), and multi-branch pooling (max pooling and average pooling) is combined to achieve multi-granularity feature fusion. The receptive field is expanded to capture the overall structure of vehicles at a distance, which solves the problem of insufficient multi-scale feature fusion in long-distance scenes by the traditional SPPF module.

[0106] (III) Based on the original three-scale detection head of the original YOLO network, a small target detection head is added to form a four-level detection system.

[0107] The original YOLO network has three detector heads (P3, P4, and P5) from top to bottom, representing their detection capabilities at small, medium, and large scales, respectively. However, for small objects, there are problems such as few features, unclear semantic features, and the tendency for features to be submerged with continuous convolution.

[0108] This invention adds a micro-target detection head to the original three-scale detection heads P3, P4, and P5 of the YOLO network, forming a four-level detection system, specifically including:

[0109] By fusing shallow features from the third layer of the backbone network through two layers of bilinear upsampling (magnifying by 2 times each time) (preserving 1 / 4 of the original resolution information), the corresponding original... Figure 1 / 8 scale feature layers (e.g., 128×128 high-resolution feature layers). Utilizing the high spatial resolution information of shallow features improves the recall rate of small targets; through multi-level detection head collaboration, full-scale coverage from tiny to large vehicles is achieved.

[0110] By designing smaller convolutional kernels and deeper network layers, the ability to extract features from small target vehicles is enhanced. Based on the original YOLOv11 detection head (large, medium, and small target detection layers), a new higher-resolution micro-target detection head is added. This is achieved by upsampling and fusing shallow feature maps from the backbone network to generate a 128×128 feature layer, and by designing dense anchor boxes (scaled to the original size). Figure 1 (8-scale) Coverage of tiny targets. Utilizing the high spatial resolution information of shallow features improves the recall rate of small targets; through multi-level detection head collaboration, full-scale coverage from tiny to large vehicles is achieved.

[0111] Through the above improvements, the optimized YOLOv11 algorithm significantly improves the detection accuracy and speed of road vehicles, providing a more reliable foundation for subsequent vehicle tracking and speed and distance calculation based on the BotSort algorithm.

[0112] This invention uses the VisDrone2019 dataset as the experimental data source. This dataset contains a wealth of road scene images from the perspective of drones, covering samples with different lighting conditions, vehicle scales, and background complexities, making it highly representative and challenging. During training, the number of training rounds was set to 100 to fully evaluate the algorithm's learning ability and performance stability.

[0113] After 100 rounds of training, the improved YOLOv11 algorithm achieved a significant improvement in mAP@0.5. Compared to the original YOLOv11 algorithm, the improved YOLOv11 algorithm achieved a significant improvement in detection accuracy, such as... Figure 5 As shown, the accuracy has improved by 5%.

[0114] In addition to detection, a tracking algorithm is also needed to distinguish each vehicle.

[0115] Preferably, in another embodiment, the BotSort algorithm used in this invention is a high-efficiency multi-object tracking (MOT) algorithm based on depth matching and motion models. Its core idea is to use an improved Kalman filter to predict the detection results of the current frame, then reduce coordinate offsets caused by camera motion through camera motion compensation (CMC), and finally achieve stable tracking through a multi-feature fusion matching strategy. Specifically:

[0116] (1) The location information, category information, confidence score, etc. of the vehicle detection box obtained by the improved YOLOv11 algorithm in each frame are converted into the information required for tracking by the BotSort algorithm (the coordinate information and confidence score (x,y,conf) of the target obtained by YOLO detection, etc.) and sent to the BotSort algorithm. The BotSort algorithm uses an improved Kalman filter (KF) to predict the position of the detected target in the next frame. The KF state vector is defined as [x,y,w,h,v] x ,v y ,v w ,v h ], where (x,y) are the coordinates of the center point, w is the width of the detection box, h is the height of the detection box, and (v x ,v y ,v w ,v h (x, y, w, h) represent the rate of change of the velocity of the target in the x, y, w, and h directions, respectively.

[0117] The BotSort algorithm employs an improved Kalman filter for prediction, replacing the traditional SORT method which only estimates the aspect ratio, thus enhancing its adaptability to changes in vehicle size. Furthermore, the BotSort algorithm dynamically adjusts noise parameters based on the target's motion behavior and matching state, allowing the filter to more "intelligently" balance prediction and observation.

[0118] (2) Camera motion compensation: The affine transformation matrix A between adjacent video frames is calculated by global motion estimation methods (such as the OpenCV method in the OpenCV library), and decomposed into scale rotation matrix S and translation vector T.

[0119] A=[M,T]

[0120] Where M is the scale rotation matrix, used to control image scaling and rotation; T is the translation vector, representing the overall displacement of the camera in the image.

[0121] To make the predicted position closer to the actual motion, the BotSort algorithm also corrects the predicted state and covariance:

[0122] Predicted state correction:

[0123] Covariance Correction:

[0124] in, This represents the Kalman filter's prediction of the target state in the current frame (including center point coordinates, velocity, etc.), where T is the translation vector. covariance matrix It is the covariance matrix of the predicted state. It is the transpose of the scale rotation matrix.

[0125] (3) The following multi-level matching strategy is used to match the detected targets with the predicted targets:

[0126] During tracking, it is necessary to distinguish multiple targets detected in each frame of the image; after predicting the position of the target in the previous frame, the detected target is matched with the predicted target.

[0127] The BotSort algorithm no longer uses a single matching method (such as matching only with IoU), but instead designs a priority-based, multi-stage matching strategy to improve matching accuracy and stability.

[0128] First priority: Matching high-confidence detection boxes based on IoU distance. For detection boxes in the current video frame with confidence scores higher than a set high threshold, IoU (Intersection over Union) distance is used to match the predicted boxes first. If no match is found, the process switches to the second priority matching method.

[0129] Second priority: For targets in the current video frame whose confidence levels fall between a set low threshold and a set high threshold, matching cannot rely solely on IoU because the location may be inaccurate. Therefore, appearance features (ReID) are introduced in conjunction with IoU for joint judgment. The following formula is the distance fusion formula used for judgment:

[0130]

[0131] Wherein, λ is a weighting coefficient used to dynamically balance motion and appearance information. ; This represents the IoU distance between the predicted bounding box and the detected bounding box. The cosine distance between the appearance feature (ReID) and the currently detected feature. If If the result is less than or equal to the preset second priority matching threshold, it is considered a mismatch; otherwise, it is considered a successful match.

[0132] The multi-level matching strategy can effectively solve the problem of identity confusion when trajectories intersect. Compared with traditional greedy matching algorithms (such as IOU matching), it significantly reduces the number of vehicle ID switching and provides a stable trajectory sequence for subsequent vehicle distance calculation.

[0133] In this invention, the BotSort algorithm is chosen as the core of the tracking module, mainly based on the following technical advantages and scenario adaptability:

[0134] (1) Addressing the detection challenges from the perspective of drones:

[0135] When UAVs perform high-altitude imaging, vehicle targets often exhibit small size and dense distribution. Although the improved YOLOv11 detector head enhances the detection accuracy of small targets, the positioning error of the detection box (such as bounding box offset) can still lead to difficulties in tracking and association. The multi-level matching strategy of the BotSort algorithm prioritizes matching high-confidence detection results, which can filter low-confidence detection boxes and reduce the probability of false association.

[0136] Vehicles move dynamically on roads (e.g., changing lanes, crossing lanes), making traditional motion models (such as those assuming uniform motion) prone to failure. The BotSort algorithm's adaptive motion model can learn vehicle motion patterns in real time and dynamically adjust prediction parameters based on information such as the position and speed of detection boxes, significantly improving trajectory continuity.

[0137] (2) Enhance robustness in complex scenarios:

[0138] Urban roads are often subject to interference such as occlusion (e.g., trees or buildings partially obscuring vehicles) and changes in lighting (e.g., shadow areas). The appearance feature matching mechanism of the BotSort algorithm, through deep feature extraction, can maintain feature association even when the target part is partially occluded, reducing the loss of vehicle IDs due to appearance changes.

[0139] For dense multi-vehicle scenarios, the BotSort algorithm's global optimal matching algorithm can effectively solve the identity confusion problem when trajectories intersect. Compared with traditional greedy matching algorithms (such as IOU matching), it significantly reduces the number of vehicle ID switching and provides a stable trajectory sequence for subsequent vehicle distance calculation.

[0140] (3) Co-optimization with improved detection algorithms:

[0141] This invention improves the detection accuracy of vehicles at multiple scales by using an improved YOLOv11 algorithm, resulting in more accurate output of bounding box coordinates, confidence scores, and appearance features. The BotSort algorithm can directly utilize these high-quality detection results and achieve deep fusion of detection and trajectory through a multi-level matching strategy, forming a closed-loop optimization of "detection-tracking" and further improving the overall system performance.

[0142] Unmanned aerial vehicle (UAV) platforms are limited by computing power and power consumption, and need to achieve real-time detection and tracking with limited resources. The BotSort algorithm, through its lightweight design, maintains high MOTA (multi-target tracking accuracy) while having better computational efficiency than similar algorithms (such as FairMOT and DeepSORT), which can meet the real-time processing requirements of UAVs (≥25FPS) and fits the "lightweight equipment" design goal of this invention.

[0143] Preferably, in another embodiment, the pixel coordinate to world coordinate transformation matrix H in step one is obtained based on the homography matrix calibration method of scene reference objects. This method utilizes fixed or dynamic reference objects with known physical dimensions in the road (such as lane dashed line length, lane width, standard vehicle length, etc.) to construct the mapping relationship between pixel coordinates and world coordinates. Specifically, this includes the following steps:

[0144] 1) Extract the features of the reference object and its corresponding world coordinates.

[0145] 1-A) Select a video taken by the drone to be processed, take several frames of the video to calibrate candidate regions, including the area where the road is located and the position of the lane; select a region with fixed physical size characteristics from the candidate regions as the calibration benchmark.

[0146] Preferably, the lane length, lane spacing along the road traffic direction, lane width, and lane spacing along the road traffic radial direction in the candidate area are measured as calibration benchmarks.

[0147] Fixed physical dimension features can be lane structure features, including but not limited to lane dashed lines (single segment length is typically 6m, interval 9m), lane width (standard lane width 3.5m or 3.75m), road edge lines, etc. Alternatively, a dynamic reference object can be selected, and its standard dimensions can be obtained from a pre-set dimension database as a fixed physical dimension feature. For example, selecting a small car as a dynamic reference object, matching the vehicle category (small car) to the pre-set dimension database, yields its standard dimensions as approximately 4m in length and 1.5m in width.

[0148] 1-B) Establish a world coordinate system O−XYZ, with the road surface as the XY plane (Z=0). The origin O can be set as a fixed feature point, such as the point at the top left of the lane area with the same Y value as the first dashed line, like... Figure 6 As shown. For a reference object with known fixed physical dimensions, its world coordinates are... It can be directly determined through geometric relationships. Based on this, multiple sets of feature point pairs (u) can be selected. i ,v i )↔(X i ,Y i ).

[0149] 2) Solve for the transformation matrix H between pixel coordinates and world coordinates.

[0150] The transformation matrix H between pixel coordinates and world coordinates is a homography matrix, describing the projection transformation relationship from the XY plane of the world coordinate system to the image pixel plane, satisfying the following equation:

[0151]

[0152] Through at least 4 sets of feature point pairs (ui ,v i )↔(X i ,Y i The eight unknown parameters of the homography matrix H are solved using the least squares method or the RANSAC algorithm. The specific steps are as follows:

[0153] Constructing a system of linear equations for each pair of feature points, and substituting them into the homography transformation formula, we get:

[0154]

[0155] Transform into homogeneous equation form:

[0156]

[0157] Final Request .

[0158] 2. The vehicle speed is calculated using the time difference method.

[0159] For a given vehicle i, suppose that in several consecutive video frames, the pixel coordinates at times t and (t+∆t) are respectively and After transformation matrix H, the corresponding world coordinates are obtained as follows: and Calculate the displacement d between two points using the following Euclidean distance formula. i :

[0160] Calculate the average speed v of the vehicle during the time interval from t to (t+∆t). i for:

[0161] Example 3:

[0162] Another embodiment of the present invention is a method for measuring vehicle distance, comprising:

[0163] 1. The drone aerial video to be processed is used to track vehicles in real time through an improved Yolov11 network and BotSort algorithm. The ID of each vehicle obtained by tracking is combined with the world coordinate information of the road lane lines to determine whether the vehicles are in the same lane.

[0164] 2. For two vehicles j and k in the same lane, extract their pixel coordinates. and By combining the pixel coordinate to world coordinate transformation matrix H, the pixel coordinates are converted into world coordinates respectively. and .

[0165] III. Calculate the distance between two vehicles j and k in the same lane using the Euclidean distance formula. :

[0166]

[0167] IV. (Optional) Set safe following distance thresholds based on traffic regulations or practical application needs. When the calculated vehicle spacing < When this is triggered, an early warning is provided, offering real-time safety alerts to the intelligent traffic management system.

[0168] Vehicle speed and distance measurement results are as follows Figure 7 As shown.

[0169] In some embodiments, certain aspects of the above-described techniques may be implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly implemented on a non-transitory computer-readable storage medium. The software may include instructions and certain data that, when executed by one or more processors, manipulate one or more processors to perform one or more aspects of the above-described techniques. The non-transitory computer-readable storage medium may include, for example, magnetic or optical disk storage devices, solid-state storage devices such as flash memory, cache, random access memory (RAM), or other non-volatile memory devices. The executable instructions stored on the non-transitory computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats interpreted or otherwise executed by one or more processors.

[0170] Computer-readable storage media can include any storage medium or combination of storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media can include, but are not limited to, optical media (e.g., optical discs (CDs), digital versatile optical discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tapes, or magnetic hard disks), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS) based storage media. Computer-readable storage media can be embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard disk drive), removably attached to a computing system (e.g., an optical disc or universal serial bus-based (USB) flash memory), or coupled to a computer system via a wired or wireless network (e.g., network-accessible storage (NAS)).

[0171] Please note that not all activities or elements described in the general description above are essential, a particular activity or part of the apparatus may not be essential, and one or more further activities or included elements may be performed in addition to those described. Furthermore, the order in which the activities are listed does not necessarily represent the order in which they are performed. Moreover, these concepts have been described with reference to specific embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure as set forth in the following claims. Therefore, the specification and drawings are to be considered illustrative rather than restrictive, and all such modifications are included within the scope of this disclosure.

[0172] The benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, any benefits, advantages, solutions to problems, and any features that may lead to or make any benefit, advantage, or solution more apparent should not be construed as critical, essential, or essential features of any or all claims. Furthermore, the specific embodiments disclosed above are merely illustrative, as the disclosed subject matter can be modified and implemented in different but equivalent ways that would be apparent to those skilled in the art benefiting from the teachings herein. There is no intention to limit the details of the constructions or designs shown herein other than those described in the claims. Therefore, it is apparent that the specific embodiments disclosed above can be altered or modified, and all such changes are considered to be within the scope of the disclosed subject matter.

Claims

1. A method for detecting road vehicle speed based on UAV vision, characterized in that, include: By improving the YOLOv11 network and using the BotSort algorithm, vehicles in the drone aerial videos are tracked in real time to obtain vehicle trajectories. Then, the pixel coordinates of the detected vehicles in each video frame are transformed using the pixel coordinates to world coordinates transformation matrix H. Convert to world coordinates This forms the trajectory sequence of the vehicle in the world coordinate system. , where t represents a time frame; For a given vehicle i, suppose that in several consecutive video frames, the pixel coordinates at times t and t+∆t are respectively and The corresponding world coordinates are obtained through the transformation matrix H between pixel coordinates and world coordinates. and Calculate the displacement d between two points using the following Euclidean distance formula. i : Calculate the average speed v of the vehicle during the time interval from t to (t+∆t). i for: .

2. The road vehicle speed detection method based on UAV vision according to claim 1, characterized in that, The improved YOLOv11 network includes: An EMA module is embedded in the C3k2 module of the backbone network to construct an attention-enhanced feature extraction unit; The SPPF module in the neck network is upgraded to the SPPF_LSKA multi-scale fusion unit to construct a feature fusion structure that enhances long-range dependencies. Based on the existing three-scale detection head, a new micro-target detection head has been added to form a four-level detection system.

3. The road vehicle speed detection method based on UAV vision according to claim 2, characterized in that, The EMA module in the C3k2 module of the backbone network, which constructs an attention-enhanced feature extraction unit, includes: The original features X∈R are input into the improved YOLOv11 network. C×H×W First, the feature maps are divided into multiple sub-feature groups and then reshaped through batch processing. The grouped feature maps are as follows: in, The feature map after grouping; reshape represents the grouping operation on X; B is the batch dimension; C is the number of feature maps; H is the feature map height; W is the feature map width; g is the number of groups; The sub-features in each sub-feature group are fed into two parallel branches, which perform multi-scale feature extraction. One branch is the channel attention branch, which performs spatial average pooling on each sub-feature to extract global statistics. Then, it fuses channel information through 1×1 convolution and uses Sigmoid activation to generate channel attention weights. These weights are multiplied by the original features to achieve dynamic calibration of the channel dimension. The other branch is the spatial attention branch, which applies 3×3 convolution to each sub-feature to capture the local spatial context. Then, it performs average pooling to compress the spatial dimension. Finally, it generates spatial attention weights through Softmax. These weights are broadcast and expanded to 1×H×W for spatial dimension weighting. The outputs of the two parallel branches are fused in both spatial and channel dimensions. The channel weights of the channel attention branch are added to the spatial weights of the spatial attention branch, and then normalized by Sigmoid to obtain the joint attention weights. These weights are multiplied by the original features to complete multi-scale attention modulation. The modulation results of all sub-feature groups are merged to restore the original channel dimension C×H×W. Finally, the output is obtained by superimposing it with the original features through residual connection.

4. The road vehicle speed detection method based on UAV vision according to claim 2, characterized in that, The step of upgrading the SPPF module in the neck network to an SPPF_LSKA multi-scale fusion unit and constructing a long-range dependency-enhanced feature fusion structure includes: 1-2-1) LSKA dynamic weighted pooling is used to replace the fixed pooling kernel of the SPPF module in the neck network of the original YOLOv11 network. It includes three core branches: a local receptive field branch, which extracts local detail features and captures high-frequency information through 3×3 convolution; a global receptive field branch, which expands the receptive field and models long-distance spatial dependencies through 7×7 convolution; and attention weight generation, which generates channel attention vector α through global average pooling and fully connected layers. 1-2-2) Fusing features from different kernels with attention weights generates weighted multi-scale features: 1-2-3) Spatial pyramid pooling reconstruction: according to the following formula, the weighted multi-scale features are combined in a step-by-step pooling kernel to capture the multi-granular semantic information of the target through pooling operations at different scales. 1-2-4) Output feature recombination restores the pooled features to their original dimensions and connects them with the input feature residuals, ultimately outputting multi-scale features that are fused with local and global attention guidance.

5. The road vehicle speed detection method based on UAV vision according to claim 2, characterized in that, The addition of a micro-target detection head to the existing three-scale detection head, forming a four-level detection system, includes: Based on the original YOLOv11 detection head, a new high-resolution micro-target detection head is added. It generates a feature layer corresponding to 1 / 8 scale of the original image by fusing two layers of bilinear upsampling and the shallow feature map of the backbone network.

6. The road vehicle speed detection method based on UAV vision according to claim 1, characterized in that, The transformation matrix H between pixel coordinates and world coordinates is obtained in the following way: Select a segment of drone aerial video to be processed, and take several frames from the video to calibrate candidate regions; select regions with fixed physical size characteristics from the candidate regions as calibration benchmarks; Establish a world coordinate system O−XYZ, with the road surface as the XY plane and the origin O set as a fixed feature point. For a reference object with known fixed physical dimensions, determine its world coordinates through geometric relationships. Select multiple sets of feature point pairs (u i ,v i )↔(X i ,Y i ); The transformation matrix H between pixel coordinates and world coordinates satisfies the following equation: Through at least 4 sets of feature point pairs (u i ,v i )↔(X i ,Y i Solve for the unknown parameter h of the transformation matrix H between pixel coordinates and world coordinates. 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 .

7. The road vehicle speed detection method based on UAV vision according to claim 6, characterized in that, The fixed physical dimension feature is: The lane structure features in the candidate region include, but are not limited to, lane length, lane spacing along the road traffic direction, lane width, and lane spacing along the road traffic radial direction; or Select a dynamic reference object and obtain its standard dimensions from a preset size database.

8. A road vehicle distance detection method based on UAV vision, characterized in that, include: By improving the YOLOv11 network and BotSort algorithm, vehicles in the drone aerial video are tracked in real time, the ID of each vehicle is obtained, and the world coordinate information of the road lane lines is combined to determine whether the vehicles are in the same lane. For two vehicles j and k in the same lane, extract their pixel coordinates. and Using the transformation matrix H between pixel coordinates and world coordinates, the pixel coordinates are converted into world coordinates respectively. and ; The distance between two vehicles j and k in the same lane is calculated using the Euclidean distance formula. : 。 9. A road vehicle speed and distance detection system based on UAV vision, implementing the road vehicle speed detection method based on UAV vision as described in any one of claims 1-7 or the road vehicle distance detection method based on UAV vision as described in claim 8, characterized in that, It includes a drone perception subsystem, a vision processing server, a homography matrix calibration calculation engine, and a motion analysis module; The UAV perception subsystem includes a high-definition camera and an edge computing unit; the high-definition camera collects real-time video streams of the road and transmits them to the vision processing server. The edge computing unit performs image preprocessing on the acquired video stream; The vision processing server includes an improved YOLOv11 detection module, a BoT-SORT tracking module, and a candidate region calibration unit. The improved YOLOv11 detection module detects input video frames and outputs the detection results, including the vehicle detection box's position information, category information, and confidence score, which are then sent to the BoT-SORT tracking module. The BoT-SORT tracking module uses an improved Kalman filter to predict the detection results, then reduces coordinate offset caused by camera motion through camera motion compensation, and finally outputs the tracked vehicle ID through a multi-feature fusion matching strategy. The candidate region calibration unit calibrates the road area based on a calibration method. The homography matrix calibration calculation engine includes a reference object calibration module and a homography matrix calculator; the reference object calibration module calibrates the physical dimensions of the reference object; the homography matrix calculator solves for the transformation matrix H between pixel coordinates and world coordinates; The motion analysis module includes a frame difference speed measurement module and / or a vehicle distance calculation module; the frame difference speed measurement module calculates the displacement change of the vehicle within a set time period by combining the timestamp to determine the vehicle speed; the vehicle distance calculation module combines the transformation matrix H between the pixel coordinates and world coordinates and the tracking vehicle ID to determine the distance between two tracking vehicles in the same lane.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the road vehicle speed detection method based on UAV vision as described in any one of claims 1-7 or the road vehicle distance detection method based on UAV vision as described in claim 8.

Images