Vehicle congestion estimation method based on UAV imagery

Abstract

This application relates to the fields of intelligent traffic monitoring and UAV video analysis, specifically disclosing a method for estimating vehicle congestion based on UAV imagery. First, it utilizes semantic segmentation technology to construct a high-precision region-of-interest (ROI) mask from UAV-viewed road images. Then, it uses the local tangential flow field generated by the mask to replace the global statistical direction, and adaptively spatially sorts vehicles according to the road topology through path integration. Subsequently, combining real-time UAV pose data with a precise photogrammetry model, it inversely maps vehicle pixel coordinates to WGS-84 geodetic coordinates, accurately calculating the actual physical distance between the first and last vehicles on the ground. Finally, by matching a road level congestion threshold, it estimates the congestion level of the queue length. In this way, it moves from constructing a microscopic vehicle topology sequence to quantifying the macroscopic geographical queue length, overcoming estimation errors caused by perspective distortion and curve interference.

Description

Technical Field

[0001] This application relates to the fields of intelligent traffic monitoring and drone video analysis, and more specifically, to a method for estimating vehicle congestion based on drone imagery. Background Technology

[0002] With the continuous acceleration of urbanization and the rapid increase in the number of motor vehicles, traffic congestion has become a key issue restricting urban operational efficiency, increasing energy consumption, and affecting residents' travel experience. Therefore, building an efficient and real-time traffic congestion monitoring system is of great significance for urban traffic management, emergency response, and road network operation assessment. Traditional traffic monitoring methods mainly rely on fixed infrastructure such as geomagnetic coils and fixed-point cameras. However, these devices are not only costly to deploy and maintain, but also limited by fixed installation locations, making it difficult to flexibly cover temporary construction congestion points, areas around large events, or blind spots, and failing to comprehensively capture dynamically changing traffic flow characteristics. In contrast, drones, with their high mobility, wide field of view, and lack of ground road condition limitations, are gradually becoming an ideal mobile aerial traffic observation platform. They can compensate for the shortcomings of fixed monitoring facilities and achieve on-demand, rapid inspection of specific road sections or areas.

[0003] However, despite the immense potential of drones in traffic monitoring, existing congestion estimation schemes based on drone imagery still suffer from significant limitations. Current conventional techniques often employ pixel-based vehicle density heatmap analysis or simple target counting to determine congestion status. These approaches frequently ignore the microscopic spatial distribution characteristics of vehicles on the road, failing to accurately reflect actual queue lengths and congestion patterns. Firstly, due to the influence of drone flight altitude, shooting angle, and lens distortion, a complex nonlinear mapping exists between pixel distances in the image and actual physical distances, making it prone to misjudgment by relying solely on image-level density thresholds. Secondly, existing methods lack a deep understanding of traffic flow topology when dealing with complex road configurations such as curves, ramps, or intersections. They struggle to maintain stable queue order identification even when vehicles are mutually obscured or closely following each other, resulting in an inability to accurately quantify the start and end points of congestion. Consequently, the estimation results lack geographical accuracy and practical value for business applications.

[0004] Therefore, an optimized vehicle congestion estimation scheme based on drone imagery is desired. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a method for estimating vehicle congestion based on unmanned aerial vehicle (UAV) imagery.

[0006] According to one aspect of this application, a method for estimating vehicle congestion based on unmanned aerial vehicle (UAV) imagery is provided, comprising:

[0007] Acquire aerial views of the road from above by the UAV and pose and heading reference system data;

[0008] Extracting the region of interest (ROI) from the UAV's overhead road imagery to obtain a ROI mask;

[0009] Based on region of interest masks, vehicle detection, tracking, and serialization are performed on UAV overhead road images to obtain an ordered sequence of vehicles;

[0010] Based on pose and heading reference system data and camera parameters, the geographic coordinates of the first and last vehicles in the ordered vehicle sequence are mapped to obtain the WGS-84 coordinates of the first vehicle and the WGS-84 coordinates of the last vehicle.

[0011] The queue length is determined based on the WGS-84 coordinates of the first and last vehicles.

[0012] Based on the road level congestion threshold, the congestion level is estimated by measuring the queue length.

[0013] Compared with existing technologies, this application provides a vehicle congestion estimation method based on UAV imagery. First, it utilizes semantic segmentation technology to construct a high-precision road region of interest mask from UAV-viewed road images. Then, it uses the local tangential flow field generated by the mask to replace the global statistical direction, and adaptively spatially sorts vehicles according to the road topology through path integration. Subsequently, combining real-time UAV pose data and a precise photogrammetry model, it inversely maps vehicle pixel coordinates to WGS-84 geodetic coordinates, accurately calculating the actual physical distance between the first and last vehicles on the ground. Finally, by matching road level congestion thresholds, it estimates the congestion level of queue lengths. In this way, it moves from constructing microscopic vehicle topology sequences to quantifying macroscopic geographical queue lengths, completely overcoming estimation errors caused by perspective distortion and curve interference, and achieving a precise and standardized assessment of traffic congestion levels. Attached Figure Description

[0014] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0015] Figure 1 This is a flowchart of a vehicle congestion estimation method based on UAV imagery according to an embodiment of this application;

[0016] Figure 2 This is a data flow diagram illustrating the vehicle congestion estimation method based on UAV imagery according to an embodiment of this application;

[0017] Figure 3 This is a flowchart illustrating the vehicle congestion estimation method based on UAV imagery according to embodiments of this application, which involves detecting, tracking, and serializing vehicles from UAV overhead road images to obtain an ordered sequence of vehicles based on a region of interest mask.

[0018] Figure 4 This is a flowchart illustrating the generation of an ordered sequence of vehicles based on the current frame's tracked vehicle set and a region of interest mask, according to an embodiment of this application's vehicle congestion estimation method based on UAV imagery.

[0019] Figure 5 This is a flowchart illustrating the process of mapping the geographic coordinates of the first and last vehicles in an ordered sequence of vehicles to obtain the WGS-84 coordinates of the first and last vehicles based on pose and heading reference system data and camera parameters, according to an embodiment of the present application's artificial intelligence-based power system fault analysis and diagnosis method. Detailed Implementation

[0020] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0021] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0022] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.

[0023] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0024] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0025] To address the problems of existing UAV traffic monitoring technologies, which neglect complex road geometry and lack pixel-to-physical space mapping, leading to disordered vehicle sequencing, distorted queue length estimation, and a lack of quantitative standards for congestion level assessment on nonlinear road sections such as curves, this application proposes a vehicle congestion estimation method based on UAV imagery. This method first uses a deep learning model to perform semantic segmentation on UAV overhead images to obtain a high-precision road region of interest mask. Further, it mines the geometric features of the mask to construct a tangential flow field that conforms to the local road orientation. Based on this, it performs adaptive topological sorting of detected and tracked vehicles based on path integrals, thereby generating an ordered vehicle sequence that strictly conforms to actual driving logic. Subsequently, using collinearity equations combined with real-time high-precision UAV pose data and camera intrinsic parameters, it performs an inverse mapping from the image pixel plane to the WGS-84 geodetic coordinate system to accurately calculate the geographical locations of the first and last vehicles. Finally, by calculating the actual physical distance on the ground and matching it with road level thresholds, it achieves a scientific and accurate assessment of traffic congestion levels.

[0026] Figure 1 This is a flowchart of a vehicle congestion estimation method based on UAV imagery according to an embodiment of this application. Figure 2 A schematic diagram of the data flow for the vehicle congestion estimation method based on UAV imagery according to an embodiment of this application is shown below. Figure 1 and Figure 2 As shown, the vehicle congestion estimation method based on UAV imagery according to an embodiment of this application includes: S100, acquiring UAV-viewed road images and pose and heading reference system data; S200, extracting the region of interest (ROI) from the UAV-viewed road images to obtain a ROI mask; S300, performing vehicle detection, tracking, and serialization on the UAV-viewed road images based on the ROI mask to obtain an ordered vehicle sequence; S400, mapping the geographic coordinates of the first and last vehicles in the ordered vehicle sequence based on the pose and heading reference system data and camera parameters to obtain the WGS-84 coordinates of the first and last vehicles; S500, determining the queue length based on the WGS-84 coordinates of the first and last vehicles; and S600, estimating the congestion level based on the queue length using a road level congestion threshold to obtain the congestion level.

[0027] Specifically, in step S100, the UAV-viewed road image and pose and heading reference system data are acquired. It should be understood that since a single UAV-viewed two-dimensional road image only contains pixel grayscale and color information, it lacks an absolute geographic reference for measurement mapping to the physical real world. Furthermore, the UAV is in dynamic flight, and its position drift and attitude tilt in real time directly lead to nonlinear changes in the imaging geometry. Without accurate external orientation parameters at the time of imaging, subsequent algorithms will be unable to accurately reconstruct the pixel coordinates of the image plane into geographic coordinates on the ground. Therefore, in the technical solution of this application, by acquiring the UAV-viewed road image and pose and heading reference system data, necessary visual feature inputs are simultaneously provided for subsequent deep learning-based vehicle target recognition, and key external geometric constraint parameters are provided for constructing an inverse projection model from image space to geographic space. This ensures that subsequent steps have accurate time synchronization and spatial reference benchmarks when estimating vehicle congestion queue lengths, thereby effectively eliminating geographic positioning deviations and ranging errors caused by flight attitude fluctuations.

[0028] More specifically, in a concrete example of this application, a drone flight platform equipped with a high-resolution industrial camera performs a vertical overhead imaging task on the target monitoring road segment. Simultaneously, a positioning and attitude determination system, composed of a global navigation satellite system and an inertial measurement unit integrated within the flight platform, performs high-frequency sampling of the drone's motion state along its flight trajectory. At each trigger time t of image acquisition, the onboard flight control system synchronously locks and records the external orientation elements of the camera center in the world coordinate system at that moment. This includes longitude, latitude, and altitude data representing the three-dimensional spatial position, as well as three-axis angles representing the sensor's attitude: pitch, yaw, and roll angles. Subsequently, the system labels the acquired visual frame data as drone-viewed road images and encapsulates a set of synchronously recorded spatial position and attitude parameters into pose and heading reference system data. By adding millisecond-accurate timestamps or establishing index associations to each set of data, the visual images and spatial telemetry data are strictly bound and aligned at the frame level and transmitted to the computing unit as the initial input to the algorithm.

[0029] Specifically, in step S200, the region of interest (ROI) is extracted from the UAV-viewed road image to obtain a ROI mask. It should be understood that because UAV-viewed images have a wide coverage area and complex background information, containing numerous non-traffic interference factors such as roadside buildings, vegetation, non-motorized vehicle lanes, and pedestrian areas, directly performing vehicle detection on the full-frame original image would not only lead to significant redundancy and waste of computational resources but also generate numerous false detections due to identifying stationary vehicles parked in parking lots or off-road areas, severely affecting the accuracy of judging the real-time traffic status of the road. Therefore, in the technical solution of this application, the ROI is further extracted from the UAV-viewed road image to obtain a ROI mask. This allows for the precise pixel-level definition of the effective road surface boundary using deep semantic features, and morphological optimization methods are used to remove edge noise and internal holes, shielding all visual interference from non-road background areas. This strictly limits the subsequent vehicle detection and tracking computation space to the actual physical range of the road, significantly improving the signal-to-noise ratio of the algorithm and the reliability of traffic congestion assessment.

[0030] More specifically, in this embodiment of the application, extracting the region of interest (ROI) from a UAV-viewed road image to obtain a ROI mask includes: inputting the UAV-viewed road image into a pre-trained deep learning semantic segmentation model to obtain a logical output map; probabilizing the logical output map by road category to obtain a probability map; binarizing the probability map based on a confidence threshold to obtain a binary mask; and performing morphological post-processing on the binary mask to obtain the ROI mask, wherein the morphological post-processing includes, but is not limited to, erosion, dilation, opening, and closing operations.

[0031] Accordingly, the UAV-viewed road imagery is input into a pre-trained deep learning semantic segmentation model to obtain a logical output image. It should be understood that since the original road imagery captured by the UAV only represents a discrete pixel matrix containing RGB values, it lacks explicit semantic understanding of the scene content. Furthermore, the actual road environment is often affected by complex factors such as uneven lighting, tree shadows, and worn road markings. Traditional image processing methods relying solely on color thresholds or edge gradients are insufficient to robustly segment road areas in varying environments. Therefore, in the technical solution of this application, the UAV-viewed road imagery is further input into a pre-trained deep learning semantic segmentation model to obtain a logical output image. This leverages the powerful feature extraction and nonlinear mapping capabilities of deep neural networks to perform pixel-level contextual semantic analysis on the input image and calculate the original discrimination score for each pixel belonging to different land cover categories. This provides a data foundation with rich high-level semantic features and strong anti-interference capabilities for subsequent probability transformation and road area extraction, ensuring the consistency and accuracy of road recognition under different environmental conditions.

[0032] In a specific example of this application, the acquired UAV-viewed road imagery is first loaded into a high-performance computing unit and preprocessed, including bilinear interpolation scaling, pixel value normalization, and standardization, to construct an input tensor that meets the input dimension requirements of the neural network. Subsequently, the input tensor is fed into a deep learning semantic segmentation model pre-trained on a large-scale road dataset. This model employs an encoder-decoder architecture, using the encoder to extract multi-scale abstract features of the image layer by layer, and the decoder to perform feature fusion and spatial resolution restoration. Finally, the model completes forward propagation computation through the convolutional layer at the end, outputting a multi-channel logical output graph, where each channel corresponds to a preset semantic category, and the value at a specific location within a channel represents the non-normalized prediction score of the pixel at that location belonging to the corresponding category.

[0033] Specifically, because the internal parameters of deep neural networks exhibit a disordered distribution due to random initialization in their initial state, they lack the ability to recognize and generalize specific visual features. Furthermore, road scenes possess high diversity and complexity in texture, edge, and geometric shape. Without supervised learning based on a large number of real samples, the model cannot establish the mapping relationship between image pixels and semantic categories. Therefore, in the technical solution of this application, a labeled dataset containing rich scenes is constructed, and an iterative training process is performed on the deep learning semantic segmentation model. This minimizes the difference between the predicted results and the real labels, driving the network weights to converge towards the optimal solution, thus solidifying the feature representation pattern of roads into the model parameters. This enables the model to maintain high generalization ability and high-precision segmentation performance even when faced with unseen drone overhead images during the inference stage.

[0034] Specifically, the training process of the deep learning semantic segmentation model includes: first, constructing a sample set of drone aerial images covering different lighting conditions, flight altitudes, road grades, and road surface materials; and performing detailed pixel-level manual annotation on each frame of the sample set to generate ground truth labels that are strictly aligned with the original images, clearly defining the road pixel regions and background regions; then, configuring the training hyperparameters of the deep learning semantic segmentation model, including the learning rate, batch size, and number of iterations, and inputting batches of data-augmented image samples into the network for forward propagation calculation to obtain the current predicted segmentation map; then... Next, the difference between the predicted segmentation map and the ground truth label is calculated using a preset loss function (such as cross-entropy loss function or Dice coefficient loss function) to quantify the current segmentation error. Then, based on this error value, the backpropagation algorithm is executed, and the gradient of the loss function with respect to the weights of each layer of the network is calculated using the chain rule, driving the optimizer (such as SGD or Adam) to update the network parameters along the gradient descent direction. Finally, the above iterative closed loop of forward inference, error calculation and parameter update is repeated until the intersection-union ratio of the model on the validation set reaches the preset convergence criterion, training is stopped and the final model weight file is saved.

[0035] Accordingly, the logical output graph is probabilistically transformed into a road category to obtain a probability graph. It should be understood that since the logical output graph of a deep learning model contains unnormalized raw values, the range of which is unbounded, it cannot directly and intuitively reflect the model's confidence in the category of a pixel. Furthermore, the logical output graph is usually a multi-channel high-dimensional matrix containing all categories, making it inconvenient for direct analysis of a specific target category. Therefore, in the technical solution of this application, the logical output graph is further probabilistically transformed into a road category to obtain a probability graph. This utilizes mathematical transformations to map discrete and unbounded raw scores into normalized numerical intervals that conform to probability axioms, accurately quantifying the probability that each pixel belongs to a road category. In this way, multi-dimensional mixed feature data can be reduced in dimensionality and transformed into a single-channel confidence distribution with clear physical meaning, providing standardized data support for subsequent threshold segmentation and binary mask generation based on unified standards.

[0036] In a specific example of this application, the Softmax activation function is first applied to the logical output image on a specific channel dimension for pixel-by-pixel computation. The monotonically increasing characteristic of the exponential function is used to amplify the differences in high-scoring values, and the proportion of each channel value to the total is calculated, thereby generating a normalized probability distribution tensor for all categories. Each value is strictly limited to a closed interval between 0 and 1, and the sum of the probabilities of all channels is 1. Subsequently, based on a pre-defined semantic category index mapping relationship, a specific channel index corresponding to the road category is locked. Finally, based on this index, a two-dimensional matrix data containing only the road prediction probability is accurately extracted from the probability distribution tensor for all categories, and it is constructed into a single-channel probability map with the same resolution as the original image. The value of each point in the map represents the confidence level of the corresponding pixel being identified as a road surface.

[0037] Accordingly, based on the confidence threshold, the probability map is binarized to obtain a binary mask. It should be understood that since the pixel values ​​in the probability map are continuous floating-point distributions between 0 and 1, this ambiguous soft classification state cannot be directly used as a clear logical mask to select the operation area for subsequent image processing. Furthermore, the model output often contains some low-confidence edge noise or misclassified regions; these low-probability values ​​need to be effectively truncated to ensure the purity of the extraction results. Therefore, in the technical solution of this application, the probability map is further binarized based on the confidence threshold to obtain a binary mask. This allows for the use of clear decision boundaries to forcibly map the continuous probability distribution into discrete binary logical states, achieving rigid segmentation of the road foreground and non-road background. In this way, the soft prediction results of semantic segmentation can be transformed into a hard mask matrix that can be directly manipulated by computer vision algorithms, effectively eliminating prediction noise with insufficient confidence and establishing a preliminary road geometry.

[0038] In a specific example of this application, firstly, based on the model's validation set performance and the application scenario's need to balance recall and precision, a floating-point value between 0 and 1 is set as the decision criterion, i.e., the confidence threshold. Then, a pixel-by-pixel traversal scan and conditional judgment operation is performed on the input single-channel probability map. For each pixel position in the image, its corresponding probability value is read and compared with the confidence threshold. Next, if the probability value of the current pixel is greater than or equal to the confidence threshold, the pixel at the corresponding position in the output matrix is ​​assigned a logical 1 or the maximum grayscale value, marking it as a road candidate point; conversely, if the probability value is less than the confidence threshold, it is assigned a logical 0, classifying it as a background area. Finally, the judgment results of all pixels are integrated to generate and output a binary mask with the same resolution as the original image. The highlighted areas in this mask represent the road surface range initially determined after threshold filtering.

[0039] Accordingly, morphological post-processing is performed on the binary mask to obtain the region of interest mask. This morphological post-processing includes, but is not limited to, erosion, dilation, opening, and closing operations. It should be understood that due to the local discontinuity in pixel-level predictions of deep learning models, the generated binary mask often contains isolated noise points, burrs on road edges, and small holes and breaks within the road area caused by occlusion or texture differences. If these defects are not corrected, they will seriously interfere with the subsequent skeleton extraction and vehicle localization accuracy. Therefore, in the technical solution of this application, morphological post-processing is further performed on the binary mask to obtain the region of interest mask. This utilizes morphological operators in set theory to optimize and shape the topology of the mask, filling internal gaps and filtering out irrelevant minor noise. This significantly improves the connectivity and edge smoothness of the mask, resulting in a more complete, compact region of interest mask that conforms to the actual road geometry, providing high-quality analysis region definition for subsequent steps.

[0040] More specifically, in a concrete example of this application, a structuring element with a specific shape (such as a rectangle or circle) and size is first defined as a probe for morphological operations. The size of the structuring element is selected based on the image resolution and the estimated size of the holes to be filled. Subsequently, the structuring element is used to perform a closing operation on the input binary mask, i.e., first performing dilation to merge adjacent broken areas and fill small voids inside the road surface, followed by erosion to restore the main boundary contours, thereby enhancing the connectivity of the region. Then, for the remaining isolated discrete noise points, an opening operation is performed using the same or adjusted structuring element, i.e., first performing erosion to peel off and remove free pixel blocks and edge burrs smaller than the structuring element, and then performing dilation to restore the original shape of the main road. Finally, after iterative optimization of the above combined operations, the final shaped region of interest mask is output, which accurately covers the connected road surface and eliminates the small noise points caused by false detections.

[0041] Specifically, in step S300, based on the region of interest mask, vehicle detection, tracking, and serialization are performed on the UAV-viewed road image to obtain an ordered vehicle sequence. It should be understood that since traffic congestion is essentially the disordered accumulation and queuing behavior of vehicles in the spatiotemporal dimension, simple static detection can only obtain the instantaneous position of vehicles, failing to capture their movement trajectory and relative order. Furthermore, under complex road conditions, occlusion and dense following between vehicles can easily lead to changes and loss of target IDs. Without temporal tracking and spatial logical constraints, it is difficult to construct a robust vehicle queue relationship. Therefore, in the technical solution of this application, vehicle detection, tracking, and serialization are further performed on the UAV-viewed road image based on the region of interest mask to obtain an ordered vehicle sequence. This allows for the maintenance of vehicle identity continuity within a strictly defined road area using a spatiotemporal correlation algorithm, and the reordering of the chaotic vehicle set based on the road topology. In this way, discrete vehicle targets in the video stream can be integrated into an ordered queue with a clear logical relationship between them, effectively solving the problems of target loss and disordered order in congested scenarios, and laying a topological foundation for subsequent accurate location of the first and last vehicles and calculation of queue length.

[0042] Figure 3 This is a flowchart illustrating the process of vehicle detection, tracking, and serialization of UAV-viewed road images to obtain an ordered sequence of vehicles, based on a region-of-interest mask, according to an embodiment of this application's vehicle congestion estimation method using UAV imagery. Figure 3 As shown, step S300 includes: S310, performing image masking preprocessing on the UAV-viewed road image based on the region of interest mask to obtain the region of interest image; S320, inputting the region of interest image into a pre-trained vehicle target detection model to obtain the current frame detection set, where each element in the current frame detection set includes the bounding box of the detected vehicle and its corresponding confidence score; S330, performing multi-target tracking and ID allocation on the current frame detection set based on the tracking trajectory set of the previous frame to obtain the current frame tracked vehicle set; S340, generating an ordered sequence of vehicles based on the current frame tracked vehicle set and the region of interest mask.

[0043] Accordingly, in step S310, the UAV-viewed road image is preprocessed using image masking based on the region of interest (ROI) mask to obtain the ROI image. It should be understood that the UAV's top-down view contains a large amount of high-frequency texture information from non-road areas such as building roofs, tree canopies, and roadside billboards. These background features may be extremely similar to vehicles in color and shape. Directly inputting the full-frame image into the detection model could easily induce false detections of non-target areas. Furthermore, redundant calculations of background areas would significantly increase inference latency and reduce the system's real-time performance. Therefore, in the technical solution of this application, the UAV-viewed road image is preprocessed using image masking based on the ROI mask to obtain the ROI image. This utilizes a binary mask as a hard spatial constraint, physically shielding all pixel inputs from non-road areas in the image and retaining only valid road surface information. This significantly purifies the model's input data source, significantly reduces the interference of background noise on vehicle feature extraction, effectively compresses the amount of image processing data while improving detection accuracy, and improves the overall operating efficiency of the algorithm.

[0044] Specifically, in a specific example of this application, the original UAV-viewed road image and the region of interest mask obtained through semantic segmentation are first loaded into the graphics processing unit's memory to ensure that they have the same spatial resolution and coordinate system reference. Then, a pixel-by-pixel bitwise AND logical operation is performed, using the region of interest mask as a gating signal. For road locations with a pixel value of logic 1 in the mask, the RGB color data of the original UAV-viewed road image is completely preserved. For background locations with a mask value of logic 0, the RGB value of the corresponding pixel in the original image is forcibly set to zero or filled with a specific mean background color. Finally, the region of interest image is synthesized and output through the above masking operation. In this image, only the road area is presented as a visible texture, while the rest is a solid color background, which is directly used as input for the subsequent object detection network.

[0045] Accordingly, in step S320, the region of interest (ROI) image is input into a pre-trained vehicle target detection model to obtain the current frame detection set. Each element in the current frame detection set includes the bounding boxes of the detected vehicles and their corresponding confidence scores. It should be understood that although the ROI image removes background interference, its content is still an unstructured pixel matrix. The computer cannot directly obtain the specific location, number, and size information of the vehicles from it. Furthermore, vehicle targets exhibit complex shapes with varying scales, rotations, and mutual occlusions from a top-down perspective, requiring algorithms with strong feature representation capabilities to achieve accurate target localization and classification. Therefore, in the technical solution of this application, the ROI image is further input into a pre-trained vehicle target detection model to obtain the current frame detection set. This allows for deep feature pyramid extraction of the image using a deep convolutional neural network, and regression of the rectangular bounding boxes surrounding the vehicle contours and their probabilities of belonging to the vehicle category at multiple scales. This enables the unordered image data to be parsed into a structured target detection list, providing accurate spatial coordinates and confidence scores for subsequent multi-target tracking and queue sorting.

[0046] Specifically, in a concrete example of this application, the preprocessed region of interest image is first adjusted to a fixed size according to the model input requirements and fed into a single-stage object detection network such as YOLOv8. The backbone network of the model extracts the semantic features of the image through multi-layer convolution and downsampling operations, and fuses the deep strong semantics and shallow high-resolution information through a feature pyramid network. Subsequently, the detection head predicts the coordinate offsets (including center point coordinates, width and height) and corresponding confidence scores of multiple candidate bounding boxes at each grid position of the feature map. Next, non-maximum suppression is performed on thousands of candidate boxes, and redundant overlapping boxes targeting the same target are eliminated based on the intersection-union ratio threshold, retaining only the best predicted box with the highest confidence. Finally, the current frame detection set containing information of all retained boxes is output, where each detection item explicitly includes the bounding box parameters of the vehicle in the image coordinate system and its discrimination confidence.

[0047] In particular, it is worth mentioning that the vehicle target detection model adopts a deep convolutional neural network based on a single-stage detection architecture. The specific structure mainly consists of three core parts: a backbone network, a neck network, and a head network. The backbone network uses efficient feature extraction backbones such as CSPDarknet, responsible for extracting multi-scale feature maps from the input image layer by layer, from edge texture to high-level semantics. The neck network adopts a path aggregation network (PANet) or feature pyramid network (FPN) structure, responsible for fusing feature information from different levels of the backbone network to enhance the model's ability to perceive multi-scale targets. The head network is a decoupled detection head that independently predicts the target's class probability and bounding box regression parameters through parallel convolutional branches, ultimately outputting the detection results.

[0048] It is understandable that, given that deep detection models contain millions of learnable parameters, they initially lack any vehicle recognition capabilities. Furthermore, the appearance of vehicles from a drone's top-down view is significantly affected by lighting, occlusion, and rotation angle. Therefore, robust generalization performance must be achieved through supervised learning driven by a large amount of labeled data. Thus, in this application's technical solution, a large-scale dedicated dataset is constructed and a training process is executed. This allows the network parameters to be continuously adjusted via a backpropagation mechanism using the loss function, enabling the model to learn the feature distribution patterns of vehicle targets. This ensures that the model can still accurately and quickly output vehicle location information when facing unknown and complex road scenarios, meeting the needs of real-time traffic monitoring.

[0049] Specifically, the training process of the vehicle target detection model is as follows: First, a massive amount of drone aerial images containing various vehicle types, different weather conditions, and lighting conditions are collected, and the true bounding box coordinates and category labels of each vehicle in the images are manually labeled to construct a training sample set. Then, the model weights are initialized (pre-trained weights can be loaded onto general datasets such as COCO to accelerate convergence), and hyperparameters including learning rate, momentum factor, and batch size are set. During the training phase, Mosaic data augmentation technology is used to randomly stitch, scale, and color-dither the samples to generate augmented input tensors, which are then fed into the network for forward propagation. After the network outputs predicted bounding boxes, the coordinate deviation is calculated using bounding box regression loss functions such as CIoU or DIoU, while the classification error and confidence error are calculated using the binary cross-entropy loss function, and the weighted sum is obtained to obtain the total loss value. Then, the gradient of the total loss with respect to all network parameters is calculated based on the backpropagation algorithm, and the weights are updated using the Adam or SGD optimizer. Finally, after iterative training for tens to hundreds of epochs, training stops when the average accuracy of the model on the validation set no longer improves, and the optimal weight file is saved for inference.

[0050] Accordingly, in step S330, based on the tracking trajectory set of the previous frame, multi-target tracking and ID allocation are performed on the detection set of the current frame to obtain the set of vehicles being tracked in the current frame. It should be understood that since the target detection model can only provide the instantaneous position of vehicles within a single frame image, it cannot identify the identity of vehicles in a time series. Furthermore, in congested traffic, vehicles often occlude each other or temporarily disappear. Without temporal identity association, vehicle IDs will frequently change, making it impossible to accurately count queue lengths and vehicle order. Therefore, in the technical solution of this application, multi-target tracking and ID allocation are further performed on the detection set of the current frame based on the tracking trajectory set of the previous frame to obtain the set of vehicles being tracked in the current frame. This utilizes motion model prediction and feature similarity matching algorithms to map the detection boxes of the current frame to historical trajectories one-to-one, and to retain and re-identify targets temporarily lost due to occlusion. This ensures that each vehicle maintains a unique identity throughout the entire process of entering the field of view, constructing a continuous and stable vehicle spatiotemporal trajectory, effectively supporting subsequent queue sorting and congestion analysis.

[0051] Specifically, in a specific example of this application, an algorithm framework based on the detection and tracking paradigm (such as ByteTrack) is adopted. First, the Kalman filter algorithm is used to predict the prior position of each trajectory in the current frame based on the historical state (including position and velocity) of the tracked trajectory set in the previous frame. Then, the detection boxes in the detection set of the current frame are divided into two groups, high-scoring detection boxes and low-scoring detection boxes, according to the confidence level, and an association cost matrix is ​​constructed. Next, the high-scoring detection boxes are preferentially used to perform the first round of Hungarian algorithm matching based on the intersection-union ratio (IU). For the successfully matched pairs, the trajectory state is updated using the current detection box and its ID is inherited. Then, for the unmatched trajectories, a second round of association matching is attempted using the low-scoring detection boxes to find partially occluded or blurred targets. Finally, the high-scoring detection boxes that are still not associated after two rounds of matching are initialized as new trajectories and assigned a new ID after confirmation in multiple consecutive frames. Trajectories that have not been matched for a long time are judged as targets that have disappeared. Finally, the current frame's tracked vehicle set containing the identity and location information of all active targets is output.

[0052] Accordingly, in step S340, an ordered sequence of vehicles is generated based on the current frame's tracked vehicle set and the region of interest mask. It should be understood that although the vehicles in the tracked vehicle set possess unique IDs, their storage structure remains an unordered list. Furthermore, roads in images may exhibit arbitrary curves or inclinations. Simply sorting based on the image's XY coordinates cannot accurately reflect the actual topological relationships of the vehicles along their travel paths, directly leading to errors in queue order judgment, such as in curved road scenarios. Therefore, in this application's technical solution, an ordered sequence of vehicles is further generated based on the current frame's tracked vehicle set and the region of interest mask. This allows for in-depth mining of the geometric orientation information hidden in the road mask, and based on this, a spatial sorting benchmark conforming to physical traffic logic is constructed for the vehicles. This reorganizes the chaotic vehicle set into a linear queue strictly arranged according to the travel direction, ensuring accurate identification of the first and last vehicles and providing reliable topological support for subsequent calculations of the actual congestion queue length.

[0053] Figure 4 This is a flowchart illustrating the generation of an ordered sequence of vehicles based on the current frame's tracked vehicle set and a region of interest mask, according to an embodiment of this application. (See attached flowchart.) Figure 4 As shown, step S340 includes: S341, determining the local orientation set of the vehicle based on the region of interest mask; S342, performing vehicle spatial sorting on the tracked vehicle set in the current frame based on the local orientation set of the vehicle to obtain an ordered sequence of vehicles.

[0054] Specifically, in step S341, the local orientation set of the vehicle is determined based on the region of interest mask. It should be understood that traditional vehicle sorting methods based on UAV imagery typically employ statistical methods such as principal component analysis to process the pixel set of the road region when calculating the vehicle's driving direction, in order to extract a global principal orientation vector. The core principle of this technique is to find the axis with the most dispersed point distribution and consider it as the overall direction of the road. This method is effective to some extent when dealing with regular, straight road scenes. However, this mechanism has an inherent technical flaw: it relies entirely on the macroscopic statistical distribution characteristics of the point set, completely ignoring the geometric connectivity and local orientation continuity inherent in roads as a special type of linear topology. This flaw is particularly prominent in complex traffic scenarios. For example, when dealing with L-shaped or S-shaped curves or roundabouts, the single principal direction vector calculated by PCA is essentially the statistical average direction of all road pixel coordinates. This direction often crosses the road entity and cuts diagonally through the curve's center, failing to accurately reflect the actual driving direction of vehicles in any local road segment. If vehicle projection sorting is based on this, it will inevitably lead to serious errors in the sorting results. Similarly, in intersections or T-junctions where multiple driving directions converge, the PCA method can only provide a comprehensive and highly fuzzy direction, unable to effectively distinguish and sort congestion queues in specific directions, significantly reducing its application value. The fundamental reason is that this mechanism attempts to describe a dynamic and locally changing driving behavior with a static, global vector. This forced approach makes it unsuitable for the diverse road morphologies in the real world, resulting in a lack of necessary local adaptability.

[0055] To address the aforementioned technical bottlenecks, a novel vehicle adaptive local direction generation method based on road topology perception is proposed in a preferred embodiment of this technical solution. This preferred mechanism no longer calculates a unified global direction, but instead dynamically and accurately generates a localized driving direction vector for each tracked vehicle within the field of view, matching its current road position. This method achieves a paradigm shift from macroscopic statistical averaging to microscopic local tangents by deeply mining the geometric information contained in the road mask image. In the technical solution of this application, a set of local vehicle directions is further determined based on the region of interest mask, thereby abandoning the unified global direction reference and instead dynamically generating a localized driving direction vector that accurately matches the current spatial position of each tracked vehicle within the field of view by deeply mining the geometric connectivity information of the mask. This enables a paradigm shift from macroscopic statistical averaging to microscopic local tangents, ensuring that even under rapidly changing road conditions, each vehicle obtains a correct sorting benchmark that conforms to its physical motion trend, thus fundamentally solving the technical problem of vehicle queue construction failure under complex road conditions.

[0056] More specifically, in the embodiments of this application, determining the local orientation set of vehicles based on the region of interest mask includes: performing road mask distance transformation calculation on the region of interest mask to obtain a distance transformation map; determining the gradient orientation field and tangent orientation field based on the distance transformation map; and mapping and assigning vehicle local orientation vectors to the tracked vehicle set in the current frame based on the tangent orientation field to obtain the vehicle local orientation set.

[0057] Accordingly, a road mask distance transformation is calculated on the region of interest mask to obtain a distance transformation map. It should be understood that the original binary road mask only provides pixel-level region attribution information (i.e., a pixel value of 1 represents a road, and 0 represents a non-road), lacking a deep-level structured topological description reflecting road geometry, skeleton orientation, and boundary distances. Direction extraction based directly on this binary image would be insufficient to obtain continuous and robust flow field information. Therefore, in this application's technical solution, a road mask distance transformation is further calculated on the region of interest mask to obtain a distance transformation map. This implements a dimensionality conversion from a discrete binary domain to a continuous scalar field, explicitly quantizing the spatial relationship between each pixel within the mask and the nearest road boundary into a continuously changing metric field, thereby extracting the implicit road skeleton. This upgrades a single region mask into a distance field image with higher information entropy and richer topological features, causing the road centerline region to exhibit a highlighted feature due to its furthest distance from the boundary, laying a solid mathematical foundation for subsequent accurate calculation of tangent directions that conform to the local geometric trend of the road.

[0058] In a specific example of this application, the region of interest mask is input into the distance transformation algorithm module, which iterates through each pixel within the mask that is marked as the road foreground. And search the entire image for the nearest background pixel to that point. (i.e., road boundary points), then calculate the Euclidean distance between them, and directly assign this distance value to the corresponding pixel position in the newly generated image. This process is expressed as follows:

[0059]

[0060] in, The pixels in the distance transformation graph representing the output The grayscale intensity value or scalar distance value, It is any road pixel within the region of interest mask. This represents the set of boundaries between the road region and the non-road background region in the region of interest mask, i.e., the road boundary. This is the mask boundary. The pixels on the surface, and This represents the Euclidean distance between the two points. This step, through a minimization search mechanism, assigns a physical property to each internal point within the mask—its distance from the boundary—thus transforming a simple binary image into a distance transformation map with higher information entropy. In the generated distance transformation map, pixels near the road edge have lower values ​​(darker) due to their proximity to the boundary, while pixels in the center of the road have the highest values ​​(brightest) because they are furthest from both sides of the boundary. This brightness distribution naturally forms a bright ridge extending along the road's direction, explicitly defining the road's central axis skeleton. This transforms the directionless binary region into a potential energy field with a clear gradient trend, providing a direct geometric basis for subsequently using gradient operators to extract the normal direction perpendicular to the distance contour lines.

[0061] Accordingly, the gradient direction field and tangent direction field are determined based on the distance transformation map. It should be understood that although the generated distance transformation map visually presents the ridge structure of the road through changes in brightness, it is essentially a static scalar field, in which the trajectory of the road centerline is implicit and cannot directly provide specific vectorized guidance for vehicles. According to the principles of differential geometry, the gradient vector at any point in a scalar field is always perpendicular to the tangent direction of the contour lines passing through that point. In the specific scenario of the distance transformation map, the envelope of the distance contour lines is the road centerline; therefore, the gradient inverse of this map must be perpendicular to the local extension direction of the road. Therefore, in the technical solution of this application, the gradient direction field and tangent direction field are further determined based on the distance transformation map, thereby explicitly transforming the implicit distance distribution information into usable vector data. By cleverly utilizing the geometrical orthogonal duality between the distance field and the road direction, and by first solving for the gradient perpendicular to the road and then rotating it, the precise tangent direction of each road pixel in the image is efficiently calculated. In this way, without the need for complex curve fitting or explicit skeleton extraction, a refined direction guidance network covering the entire map can be constructed, achieving comprehensive coverage and accurate description of the local trends of roads of any shape, such as curves and ramps.

[0062] In a specific example of this application, the road centerline direction field is first generated and calculated for the distance transformation map, for each pixel position in the map. The gradient vector at that location can be calculated by taking the partial derivatives in the horizontal and vertical directions using the Sobel operator or the central difference method. It is represented as:

[0063]

[0064] The result obtained here That is, the distance transformation graph at point The gradient is the gradient pointing in the direction of the fastest increase in distance. Normalizing this gradient vector yields a gradient direction field pointing towards the road boundary. Subsequently, this gradient vector is normalized to obtain a unit gradient vector. Then, a 90-degree orthogonal rotation operation is performed on it to calculate the unit tangent vector representing the local extension direction of the road. The transformation process can be represented as:

[0065]

[0066] in, It is a point The unit gradient vector at that point, , This indicates a 90-degree rotation operation, while This is the desired local unit tangent vector. If Represented as ,but Desirable or This process simply requires maintaining consistency in direction selection across the entire image. It cleverly leverages the geometric duality between the distance field and road orientation, efficiently calculating a precise tangent direction for each road pixel in the image that aligns with the local road trend, without requiring complex curve fitting or skeleton extraction. In other words, the gradient vector essentially indicates the pixel's trend of moving away from the road boundary and its centerline (i.e., the direction laterally across the road), while the tangent vector obtained through orthogonal rotation precisely points to the tangent direction of the road's equidistant lines (i.e., the longitudinal direction of travel along the road). This process traverses the entire distance transformation image, ultimately generating a tangent direction field covering the entire road area. This mathematical processing transforms the static distance scalar field into a dynamic vector flow field, giving each pixel in the image a driving direction guide that strictly matches its local geometric features. This provides comprehensive underlying data support for subsequent vehicles to adaptively obtain accurate sorting directions based on their own positions.

[0067] Accordingly, based on the tangent direction field, the local direction vectors of the tracked vehicle set in the current frame are mapped and assigned to obtain the local direction set of vehicles. It should be understood that since the tangent direction field constructed in the preceding steps is a dense vector matrix covering all road pixels in the entire map, while the actual target is a set of discretely distributed and constantly moving vehicle targets, if this environmental topology information cannot be accurately associated with specific traffic participants, it is impossible to achieve the leap from environmental perception to target behavior prediction. Therefore, in the technical solution of this application, the local direction vectors of the tracked vehicle set in the current frame are further mapped and assigned based on the tangent direction field to obtain the local direction set of vehicles. This is used to perform a spatial index query operation from the microscopic environmental flow field to the macroscopic target individual, directly applying the pre-computed high-resolution direction map to each existing vehicle instance, completely abandoning the one-size-fits-all approach of all vehicles sharing a single global average direction in the original mechanism, and realizing personalized customization and adaptive matching of vehicle driving directions. This ensures that regardless of whether a vehicle is on a straight road, at the apex of a curve, or at the entrance of a ramp, the assigned direction vector strictly matches the road geometry at the current location, providing accurate vector input for subsequent high-precision path integration sorting.

[0068] In a specific example of this application, each vehicle in the tracked vehicle set in the current frame is traversed, its detection bounding box information is parsed, and the coordinates of the vehicle's geometric center point are calculated. Subsequently, using this coordinate... As a two-dimensional spatial index, it directly addresses and accesses the pre-calculated tangent direction field matrix, and extracts the unit local tangent vector stored at that location. Next, the extracted vector It is formally assigned to vehicle k as its personalized, adaptive driving direction in the current frame, that is, its exclusive local driving direction vector at the current moment. Furthermore, all vehicles and their corresponding direction vectors are encapsulated into a set, namely the vehicle local direction set. In this way, the one-size-fits-all global direction vector in the original mechanism is completely abandoned, and each vehicle is equipped with a tailor-made direction reference.

[0069] Through the implementation of the above-described preferred embodiments, this solution achieves a fundamental shift from relying on macroscopic statistics to mining microscopic topology. This method endows UAV vision systems with the ability to accurately perceive complex road geometry. In the subsequent vehicle spatial sorting process, the sorting reference for each vehicle will be its own dynamically acquired local driving direction, rather than a coarse global average direction. This fundamentally guarantees the accuracy of vehicle sorting in complex scenarios where traditional methods fail, such as curves, ramps, roundabouts, and even intersections, ensuring that the order of vehicles in the queue can be stably and reliably resolved even when road conditions change drastically. Ultimately, this improvement significantly enhances the robustness, accuracy, and scenario universality of the entire vehicle congestion estimation scheme, laying a solid technical foundation for achieving intelligent and refined traffic situational awareness.

[0070] Specifically, in step S342, the vehicle spatial sorting of the current frame's tracked vehicle set is performed based on the vehicle's local orientation set to obtain an ordered vehicle sequence. It should be understood that since the physical travel of vehicles on a road is not along a straight line, but rather along the geometric centerline of the road, especially in S-curves, roundabouts, or ramp areas, traditional sorting methods based on straight-line projection (such as simple coordinate axis projection or PCA principal direction projection) can lead to serious sequence logic errors because they ignore the curvature changes of the path (for example, on a U-curve, straight-line projection may incorrectly classify vehicles located at opposite ends of the curve but spatially close as adjacent). Therefore, a new sorting criterion is needed to measure the order of vehicles on a curved path. In the technical solution of this application, the vehicle spatial sorting of the current frame's tracked vehicle set is further performed based on the vehicle's local orientation set to obtain an ordered vehicle sequence, thereby introducing an adaptive topological sorting mechanism based on local tangent flow. This mechanism uses the idea of ​​path integral to quantify the actual fluid distance traveled by each vehicle along the curved road, rather than the Euclidean straight-line distance. In this way, the complex nonlinear spatial sorting problem can be transformed into a one-dimensional integral metric problem along the flow field path, ensuring that the generated vehicle sequence strictly conforms to the human driver's cognitive logic of the distance between vehicles, thus enabling the correct queue order to be output robustly under any complex road conditions.

[0071] More specifically, in this embodiment of the application, the vehicle spatial sorting of the current frame's tracked vehicle set based on the vehicle local orientation set to obtain an ordered vehicle sequence includes: dynamically anchoring the queue starting point of the current frame's tracked vehicle set based on the vehicle local orientation set to obtain anchor point vehicles and flow field vector sets; performing path integration on the current frame's tracked vehicle set, anchor point vehicles, and flow field vector sets to obtain a vehicle topology order value set; and arranging the current frame's tracked vehicle set in ascending order based on the vehicle topology order value set to obtain an ordered vehicle sequence.

[0072] Specifically, in a concrete example of this application, dynamic anchoring of the queue starting point is first performed. A consistency calibration is then performed on all unit tangent vectors in the vehicle's local orientation set to construct a flow field vector set. A vehicle with specific spatial characteristics (such as a projected extreme point on a coordinate axis) is selected from the current frame's tracked vehicle set as the logical reference zero, i.e., the anchor vehicle. Subsequently, for each target vehicle k in the queue, its topological order value relative to the anchor point is calculated using its own local orientation vector and the anchor vehicle's local orientation vector, through a first-order path approximation integral formula. This step estimates the average direction of the road segment between the target vehicle and the anchor vehicle by jointly using the local direction vectors at both locations. The linear displacement between the two vehicles is then projected onto this average direction to obtain a scalar value reflecting the distance traveled along the curved road. The calculation process is as follows:

[0073]

[0074] in, This represents the approximate cumulative path distance of vehicle k along the road direction relative to the anchor point, i.e., the topological order value. and Let these represent the center point coordinate vectors of the target vehicle k and the anchor vehicle, respectively. That is, the linear displacement vector between the two; and These refer to the unit local tangent vectors extracted from the flow field vector set, corresponding to the vehicle k and the anchor point vehicle positions, respectively. This represents the dot product operation of vectors. By jointly using the average of the local tangent directions at the starting point (anchor point) and the ending point (target vehicle), it constructs a virtual direction vector representing the average direction of the curved road segment, and then projects the linear displacements of the two vehicles onto this average flow direction. This processing method is essentially a trapezoidal approximation integral over the length of the curved path, which greatly corrects the directional deviation caused by road curvature compared to single-direction projection. Finally, based on the calculated vehicle topology order set, all vehicles are numerically sorted in ascending order to generate the final ordered vehicle sequence, thus perfectly solving the problem of vehicle topology sorting in curved road scenarios.

[0075] This improved mechanism, by introducing distance transformation and local tangent fields, achieves precise perception of complex road topology, enhancing the accuracy and robustness of vehicle spatial sequencing. Thus, no longer limited by road geometry, whether it's an S-curve, a roundabout, or a complex ramp, it can dynamically assign each vehicle a local driving direction appropriate to its current position and perform topological sequencing along that direction field, resulting in a reliable vehicle sequence consistent with human driver perception. This adaptive sequencing capability directly overcomes the technical shortcomings of the original global projection method in complex road conditions, ensuring the correctness of subsequent calculations of the head and tail positions of the vehicle queue. This provides a solid data foundation for accurately estimating congestion queue length and outputting reliable traffic congestion levels, ultimately achieving the goal of intelligent and high-precision estimation of vehicle congestion levels in various complex real-world road scenarios.

[0076] Specifically, in step S400, based on pose and heading reference system data and camera parameters, the ordered vehicle sequence is mapped to the geographic coordinates of the first and last vehicles to obtain the WGS-84 coordinates of the first and last vehicles. It should be understood that since the ordered vehicle sequence obtained in the preceding steps is essentially defined only in the two-dimensional pixel coordinate system of the image, the pixel Euclidean distance it represents is limited by the perspective effect and distortion characteristics of the airborne camera lens, as well as the dynamic tilt of the UAV's flight attitude, and cannot directly represent the true physical spatial scale. Without rigorous spatial geometric reconstruction, the calculation result of the congestion length will produce a huge nonlinear error. Therefore, in the technical solution of this application, the ordered vehicle sequence is further mapped to the geographic coordinates of the first and last vehicles based on pose and heading reference system data and camera parameters to obtain the WGS-84 coordinates of the first and last vehicles. This allows for the reconstruction of the imaging ray path from the camera optical center to the ground target based on the collinearity equation principle of photogrammetry, thus converting the relative image pixel positions into geodetic coordinates with absolute geographic reference value. This completely eliminates measurement deviations caused by fluctuations in flight altitude and lens distortion, providing a precise geographical reference for the beginning and end of the queue based on the formula for calculating the accurate physical queue length.

[0077] Figure 5 This is a flowchart illustrating the process of mapping the geographic coordinates of the first and last vehicles in an ordered sequence of vehicles to obtain the WGS-84 coordinates of the first and last vehicles, based on pose and heading reference system data and camera parameters, according to an embodiment of this application's vehicle congestion estimation method based on UAV imagery. Figure 5As shown, step S400 includes: S410, extracting target pixel coordinates and correcting distortion for the ordered sequence of vehicles based on camera parameters to obtain a set of distortion-free pixel coordinates; S420, constructing a rotation matrix and camera center coordinates based on pose and heading reference system data and camera parameters; S430, solving the collinearity equation for each distortion-free pixel coordinate in the set of distortion-free pixel coordinates based on the rotation matrix, camera center coordinates, target point elevation, and camera focal length to obtain a set of ground coordinates; S440, extracting the ground coordinates of the first and last vehicles from the set of ground coordinates and performing an inverse coordinate system transformation on them to obtain the WGS-84 coordinates of the first and last vehicles.

[0078] Accordingly, in step S410, based on camera parameters, target pixel coordinates are extracted and distortion correction is performed on the ordered sequence of vehicles to obtain a distortion-free pixel coordinate set. It should be understood that since most cameras mounted on UAVs use wide-angle or zoom lenses, the imaging process is inevitably affected by the physical curvature of the lens and assembly process errors, resulting in significant barrel or pincushion radial distortion at the edges of the original image and tangential distortion such as image point center offset. This means that there is a geometric positional deviation between the pixel coordinates directly extracted from the image and the actual physical projection point. Without correction, subsequent coordinate mapping will produce significant positioning drift based on incorrect input. Therefore, in the technical solution of this application, target pixel coordinates are further extracted and distortion correction is performed on the ordered sequence of vehicles based on camera parameters to obtain a distortion-free pixel coordinate set. This allows for mathematical compensation and restoration of this nonlinear systematic error at the data source, based on classic optical distortion models such as Brown-Conrady. In this way, the distorted imaging plane coordinates can be restored to the ideal geometric coordinates that conform to the linear pinhole camera model, ensuring that the prerequisite of straight-line propagation is met when applying the collinear equation for back projection, thereby greatly improving the accuracy of geolocation calculation.

[0079] In a specific example of this application, the ordered sequence of vehicles is first parsed, and the geometric center points of the bounding boxes of the first and last vehicles in the queue are located and extracted to obtain their pixel coordinates on the original distorted image. , Subsequently, using the pre-calibrated intrinsic parameter matrix information in the camera parameters, including focal length... , and principal point coordinates , Convert the original pixel coordinates to coordinates on the normalized image plane. , The conversion process is represented as follows:

[0080]

[0081]

[0082] The above formula transforms the image from a pixel-based measurement system to a normalized planar system based on physical focal length. and This represents the projection position of the uncorrected ray onto the normalized plane, providing standardized input data for subsequent applications of physical distortion models.

[0083] Next, we introduce a set of factors including focal length, principal point offset, and polynomial distortion coefficients (such as radial distortion coefficients). , , and tangential distortion coefficient , The camera intrinsic data is used to calculate the squared radial distance from the normalized point to the distortion center. By applying the inverse process or correction formula of the Brown-Conrady distortion model, the normalized ideal coordinates after distortion removal are calculated. , The calculation formula is as follows:

[0084]

[0085]

[0086] This step performs mathematical compensation for the imaging optical path from the perspective of physical optics. The first part of the formula... It is mainly used to correct radial extrusion deformation from the inside out caused by the lens shape (such as barrel distortion), while the latter part involves , The term is used to correct the tangential tilt error caused by the lens not being strictly parallel to the imaging sensor plane. Through this calculation, the bent optical path is mathematically straightened, restoring the accurate position of the vehicle target under the ideal pinhole imaging model.

[0087] Finally, the corrected normalized coordinates ( , ') Remap back to the pixel coordinate system to obtain the final distortion-free pixel coordinates. , ), and store it in the distortion-free pixel coordinate set, the calculation formula is:

[0088]

[0089]

[0090] This formula completes the final regression from the physical model space to the digital image space. After this series of processes, the obtained ( , The points are no longer the distorted points visible to the naked eye in the original image, but rather the true geometric projection points after optical correction. These precise coordinate points will be directly used as the input reference for the subsequent inverse solution of the collinearity equation to obtain the ground coordinates.

[0091] Accordingly, in step S420, a rotation matrix and camera center coordinates are constructed based on the pose and heading reference system data and camera parameters. It should be understood that since image data is acquired in the unsteady environment of dynamic UAV flight, the camera's spatial position (latitude, longitude, and altitude) is constantly changing, and the camera's optical axis pointing (i.e., shooting attitude) is directly affected by the aircraft's pitch, roll, and yaw movements. This results in an extremely complex and nonlinear spatial geometric transformation relationship between the imaging plane and the ground coordinate system. Without a precise mathematical description of these external orientation elements, it is impossible to establish a projection mapping from a two-dimensional image to three-dimensional geographic space. Therefore, in the technical solution of this application, a rotation matrix and camera center coordinates are further constructed based on the pose and heading reference system data and camera parameters. This converts the original latitude, longitude, and Euler angle data recorded by the POS system into rigorous mathematical model parameters in photogrammetry, clearly defining the absolute position of the camera's optical center in the spatial rectangular coordinate system and the rotation attitude of the camera coordinate system relative to the ground coordinate system. This provides complete geometric constraints for establishing the collinearity equations, ensuring that the projection deviation caused by the UAV's flight attitude can be accurately compensated when calculating the ground coordinates in the subsequent back-calculation.

[0092] In a specific example of this application, the longitude, latitude, and geodetic height of the UAV recorded in the data are first extracted in the world geodetic coordinate system, and then converted into three-dimensional coordinates in the local tangent plane spatial rectangular coordinate system using the Gauss-Kruger projection or the general transverse Mercator projection algorithm. , , This coordinate point was identified as the center of the imaging process, representing the physical source of the light rays at the moment of imaging. Subsequently, the three-axis attitude angle data, namely the pitch angle around the X-axis, was extracted. Roll angle about the Y-axis and yaw angle about the Z-axis Three fundamental rotation matrices are constructed based on Euler's angle rotation theorem. , , And perform matrix multiplication operations according to a specific rotation order (such as ZYX order). The final 3×3 rotation matrix is ​​calculated. The 9 elements in this matrix ( , , , , , , , , It precisely quantifies the rigid body rotation transformation relationship from the ground space coordinate system to the image space auxiliary coordinate system.

[0093] Accordingly, in step S430, based on the rotation matrix, the coordinates of the camera center, the elevation of the target point, and the camera focal length, the collinearity equations of each distortion-free pixel coordinate in the distortion-free pixel coordinate set are solved to obtain the ground coordinate set. It should be understood that the core geometric principle of photogrammetry lies in the collinearity condition, namely, the camera center, the image point, and the corresponding object point on the ground must be strictly located on the same straight line, and this straight line is mathematically constrained by both external orientation elements (position and attitude) and internal orientation elements (focal length and principal point). Only by solving the set of equations describing this strict geometric relationship can the pixels on the two-dimensional plane be projected back into three-dimensional space, thereby determining their true position on the ground. Therefore, in the technical solution of this application, the collinearity equations of each distortion-free pixel coordinate in the distortion-free pixel coordinate set are further solved based on the rotation matrix, the coordinates of the camera center, the elevation of the target point, and the camera focal length to obtain the ground coordinate set. This allows the unknown ground plane coordinates in the collinearity equations to be solved in reverse using known image coordinates, camera intrinsic and extrinsic parameters, and estimated or known elevation information. In this way, we can achieve the leap from image to object, converting geographically unrelated pixel data into ground projection coordinates with practical measurement significance, laying the coordinate foundation for subsequent calculation of the actual congestion queue length.

[0094] In a specific example of this application, firstly, for each coordinate point representing the first or last vehicle in the distortion-free pixel coordinate set ( , Establish a standard set of collinearity equations for photogrammetry, which will define the image point coordinates ( , ) is expressed as the coordinates of the center of the photograph ( , , ), ground point coordinates ( , , ) and the nonlinear function of the rotation matrix parameters; subsequently, the known camera focal length The calculated rotation matrix coefficients, the coordinates of the photography center, and the elevation of the target point obtained through a digital elevation model or the mean ground elevation assumption. Substitute into the equation; then, since only the ground plane coordinates remain in the equation ( , Two unknowns, and the system of equations contains information about... and The two independent equations can be solved directly using algebraic transformations or Gaussian elimination to calculate the corresponding plane coordinates of the ground points. , Finally, the above solution process is completed for all vehicle targets of interest, and the results are aggregated and output as a ground coordinate set. The photogrammetric collinearity equations are expressed as:

[0095]

[0096] Accordingly, in step S440, the ground coordinates of the first and last vehicles are extracted from the ground coordinate set, and an inverse coordinate transformation is performed on them to obtain the WGS-84 coordinates of the first and last vehicles. It should be understood that since the ground coordinate set data obtained from solving the collinearity equation is defined in Euclidean distance values ​​(usually in meters) under local projected plane coordinate systems such as Gauss-Kruger or UTM, and geographic information systems and globally accepted location service standards all use the WGS-84 ellipsoid as a unified reference benchmark, in order to ensure that the calculated vehicle positions can be directly verified by other universal devices such as GPS, accurately located on standard electronic maps, and the length across regions can be calculated using the standard great circle distance formula, the technical solution of this application further extracts the ground coordinates of the first and last vehicles from the ground coordinate set and performs an inverse coordinate transformation on them to obtain the WGS-84 coordinates of the first and last vehicles. This performs an inverse mapping transformation of the projected coordinate system, restoring the plane rectangular coordinates to globally unified latitude and longitude spherical coordinates. This approach breaks down the barriers between local measurement data and the global geographic information system, outputting geographic location information with universal interoperability, and providing standardized input for subsequent calculations of accurate physical queue lengths on a large scale.

[0097] In a specific example of this application, the planar coordinates corresponding to the starting vehicles of the congestion queue are first retrieved and extracted from the ground coordinate set. , ) and the planar coordinates corresponding to the last vehicle ( , Subsequently, using a map projection inverse algorithm (such as the Gaussian projection inverse calculation formula) that matches the previous coordinate transformation steps, combined with the longitude of the central meridian of the projection zone, the projection scale factor, and the parameters of the reference ellipsoid (such as the semi-major axis and flattening of the WGS-84 ellipsoid), the above-mentioned plane coordinates are reverse-calculated. Next, the calculation results, i.e., the X and Y values ​​on the projection plane, are converted into longitude and latitude values ​​in the geodetic coordinate system. Finally, the obtained transformation results are packaged and output as the WGS-84 coordinates of the first vehicle and the WGS-84 coordinates of the last vehicle, usually in decimal degree form.

[0098] Specifically, in step S500, the queue length is determined based on the WGS-84 coordinates of the first and last vehicles. It should be understood that since the essential definition of congestion queue length is the actual spatial span occupied by the traffic flow on the physical road, and distances calculated directly from image pixels will have significant errors due to changes in drone flight altitude and perspective effects, and simple Euclidean plane geometry calculations cannot accurately account for the subtle influence of the Earth's curvature on large-scale geographical measurements, in order to obtain highly accurate and uniform physical distance data (such as meters), calculations must be performed using a spherical geometry system. Therefore, in the technical solution of this application, the queue length is further determined based on the WGS-84 coordinates of the first and last vehicles. This allows the use of spherical trigonometry principles to directly calculate the shortest surface path length connecting the start and end points of the congestion queue on the Earth's ellipsoid, i.e., the great circle distance. This overcomes the deformation errors of two-dimensional plane projection, outputting a precise meter-level queue length that strictly corresponds to the actual physical world, providing the most crucial and reliable quantitative indicator for subsequent traffic congestion level determination.

[0099] More specifically, in this embodiment of the application, determining the queue length based on the WGS-84 coordinates of the first vehicle and the last vehicle includes: extracting and preprocessing coordinate data from the WGS-84 coordinates of the first and last vehicles to obtain the radian latitude and longitude of the first vehicle's position, the radian latitude and longitude of the last vehicle's position, and the latitude and longitude difference between the two points; determining the central angle based on the radian latitude of the first vehicle's position, the radian latitude of the last vehicle's position, and the latitude and longitude difference between the two points; and determining the queue length based on the central angle and the Earth's radius. Specifically, in a concrete example of this application, the input WGS-84 coordinates of the first and last vehicles are first preprocessed. The latitude and longitude values, originally in degrees, are multiplied by a coefficient of π / 180 to convert them into radian latitude and longitude for mathematical calculation. The latitude and longitude differences between the first and last points in radian are then calculated. Subsequently, the core algorithm of the semi-sine formula is applied, using the sine and cosine function values ​​of the latitude and latitude and longitude differences of the two points to calculate the central angle (i.e., the central angle) corresponding to the spherical path between the two points. This angle represents the size of the angle subtended by the two points on the sphere relative to the Earth's center. Finally, the calculated central angle is multiplied by the average radius of the Earth (approximately 6371 kilometers) to calculate the arc length along the Earth's surface between the two points, which is the final determined queue length (in meters).

[0100] Specifically, in step S600, the congestion level is obtained by estimating the congestion degree of the queue length based on the road level congestion threshold. It should be understood that since queue length is only an absolute physical metric, its indicative significance for traffic operation is highly dependent on the specific road type and capacity design. For example, the same physical queue distance may only represent slight slow traffic on a highway with a large design capacity, while it may mean severe traffic paralysis on a city branch road with limited capacity. Using a single, fixed evaluation standard will lead to a significant disconnect between the assessment results and actual road conditions. Therefore, in the technical solution of this application, the congestion degree of the queue length is further estimated based on the road level congestion threshold to obtain the congestion level, thereby constructing an adaptive evaluation system that matches the road function classification, transforming absolute physical distance into a relative description of congestion status. This significantly improves the rationality and operational applicability of traffic situation assessment, accurately reflects the actual congestion severity under different road scenarios, and provides traffic management departments with targeted graded early warning information.

[0101] More specifically, in a concrete example of this application, the system first parses the geographic attribute information or configuration parameters of the current UAV-monitored road segment to determine the legal road level to which the road segment belongs, which is not limited to categories such as urban arterial roads, urban secondary arterial roads, branch roads, or expressways. Then, the system queries a pre-set road level congestion threshold database and retrieves the corresponding classification criteria based on the determined road level index. This criterion typically includes a series of distance thresholds corresponding to different congestion states. Next, the system performs a step-by-step numerical comparison logic operation between the precise queue length value calculated in the preceding steps and the retrieved thresholds. Finally, based on the range in which the value falls, the current traffic flow state is mapped and output as a specific congestion level label, such as smooth flow, congested, light congestion, moderate congestion, or severe congestion, completing the final quantitative classification of the traffic situation.

[0102] In summary, the vehicle congestion estimation method based on UAV imagery according to the embodiments of this application is explained. First, it utilizes semantic segmentation technology to construct a high-precision road region of interest mask from UAV-viewed road images. Then, it uses the local tangential flow field generated by the mask to replace the global statistical direction, and performs adaptive spatial sorting of vehicles according to the road topology through path integration. Subsequently, combining real-time UAV pose data and a precise photogrammetry model, the vehicle pixel coordinates are inversely mapped to WGS-84 geodetic coordinates to accurately calculate the actual physical distance between the first and last vehicles on the ground. Finally, by matching road level congestion thresholds, the congestion level of the queue length is estimated. In this way, it moves from constructing a microscopic vehicle topology sequence to quantifying a macroscopic geographical queue length, completely overcoming estimation errors caused by perspective distortion and curve interference, and achieving a precise and standardized assessment of traffic congestion.

[0103] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for estimating vehicle congestion based on UAV imagery, characterized in that, include: Acquire aerial views of the road from above by the UAV and pose and heading reference system data; Extracting the region of interest (ROI) from the UAV's overhead road imagery to obtain a ROI mask; Based on region of interest masks, vehicle detection, tracking, and serialization are performed on UAV overhead road images to obtain an ordered sequence of vehicles; Based on pose and heading reference system data and camera parameters, the geographic coordinates of the first and last vehicles in the ordered vehicle sequence are mapped to obtain the WGS-84 coordinates of the first and last vehicles. This includes: extracting target pixel coordinates and correcting distortion in the ordered vehicle sequence based on camera parameters to obtain a set of distortion-free pixel coordinates; constructing a rotation matrix and camera center coordinates based on pose and heading reference system data and camera parameters; solving collinearity equations for each distortion-free pixel coordinate in the set of distortion-free pixel coordinates based on the rotation matrix, camera center coordinates, target point elevation, and camera focal length to obtain a ground coordinate set; extracting the ground coordinates of the first and last vehicles from the ground coordinate set and performing an inverse coordinate system transformation to obtain the WGS-84 coordinates of the first and last vehicles. Based on the WGS-84 coordinates of the first and last vehicles, the queue length is determined, including: extracting and preprocessing coordinate data from the WGS-84 coordinates of the first and last vehicles to obtain the radian latitude and longitude of the first vehicle's position, the radian latitude and longitude of the last vehicle's position, and the latitude and longitude differences between the two points; determining the central angle based on the radian latitude of the first and last vehicles' positions and the latitude and longitude differences between the two points; and determining the queue length based on the central angle and the Earth's radius. Based on the road level congestion threshold, the congestion level is estimated by measuring the queue length.

2. The method for estimating vehicle congestion based on UAV imagery according to claim 1, characterized in that, Extracting the region of interest (ROI) from the UAV's overhead road imagery to obtain a ROI mask includes: The image of the road from above by the drone is input into a pre-trained deep learning semantic segmentation model to obtain a logical output image; The logic output graph is probabilistically classified by road category to obtain a probability graph; Based on the confidence threshold, the probability map is binarized to obtain a binary mask; A binary mask is subjected to morphological post-processing to obtain a region of interest mask. The morphological post-processing includes, but is not limited to, erosion, dilation, opening, and closing operations.

3. The method for estimating vehicle congestion based on UAV imagery according to claim 1, characterized in that, Based on region-of-interest masks, vehicle detection, tracking, and serialization are performed on UAV-viewed road images to obtain an ordered sequence of vehicles, including: Based on the region of interest mask, image masking preprocessing is performed on the UAV overhead road image to obtain the region of interest image; The region of interest image is input into a pre-trained vehicle target detection model to obtain the current frame detection set. Each element in the current frame detection set includes the bounding box of the detected vehicle and its corresponding confidence score. Based on the tracking trajectory set of the previous frame, multi-target tracking and ID allocation are performed on the detection set of the current frame to obtain the set of tracked vehicles in the current frame. Based on the set of tracked vehicles in the current frame and the region of interest mask, an ordered sequence of vehicles is generated.

4. The method for estimating vehicle congestion based on UAV imagery according to claim 3, characterized in that, Based on the set of tracked vehicles in the current frame and the region of interest mask, an ordered sequence of vehicles is generated, including: Based on the region of interest mask, determine the local orientation set of the vehicle; Based on the local orientation set of the vehicles, the tracked vehicle set in the current frame is spatially sorted to obtain an ordered sequence of vehicles.

5. The method for estimating vehicle congestion based on UAV imagery according to claim 4, characterized in that, Based on the region of interest mask, the local orientation set of the vehicle is determined, including: The distance transformation of the road mask is calculated for the region of interest mask to obtain the distance transformation map; Based on the distance transformation diagram, determine the gradient direction field and the tangent direction field; Based on the tangent direction field, the local direction vectors of the tracked vehicle set in the current frame are mapped and assigned to obtain the local direction set of the vehicles.

6. The method for estimating vehicle congestion based on UAV imagery according to claim 4, characterized in that, Based on the vehicle local orientation set, the tracked vehicle set in the current frame is spatially sorted to obtain an ordered vehicle sequence, including: Based on the local orientation set of the vehicle, the starting point of the queue is dynamically anchored for the tracked vehicle set in the current frame to obtain the anchor vehicle and the flow field vector set; Path integration is performed on the current frame's tracked vehicle set, anchor point vehicles, and flow field vector set to obtain the vehicle topology order set; Based on the vehicle topology order set, the tracked vehicle sets in the current frame are sorted in ascending order to obtain an ordered sequence of vehicles.

Images