Road congestion detection method and system based on unmanned aerial vehicle

Abstract

The invention relates to a road congestion detection method based on an unmanned aerial vehicle, and the method comprises the steps: carrying out the analysis of a target region road image obtained by the unmanned aerial vehicle, obtaining a vehicle target frame and an arterial road straight line, determining the central point of each vehicle according to the vehicle target frame, obtaining a vehicle fitting straight line according to the central point, and analyzing the angle, distance and overlapping degree of the trunk road straight line and the vehicle fitting straight line to determine similarity, tracking each target vehicle when the similarity is greater than a preset similarity threshold, and determining the speed of each target vehicle based on a tracking result, the unmanned aerial vehicle position information and the unmanned aerial vehicle camera operation parameters. And determining a reference speed based on the speeds and the number of all the target vehicles, and judging whether congestion occurs according to the reference speed and the number of the target vehicles. Through the method and the device, the problem of poor flexibility and expandability of road congestion detection is solved, the image is acquired by depending on the unmanned aerial vehicle, the target vehicle is screened through the similarity threshold, and the method and the device can be adapted to scenes with different lane widths and road directions.

Description

Technical Field

[0001] This application relates to the field of traffic detection, and in particular to a method and system for detecting road congestion based on unmanned aerial vehicles (UAVs). Background Technology

[0002] In existing technologies, road congestion detection solutions mostly rely on road information obtained from fixed surveillance cameras, vehicle-mounted terminal cameras, and related platforms, comprehensively considering the occurrence of road congestion. Typically, vehicle detection models are used to identify vehicles appearing in video footage, multi-frame joint tracking is used to calculate target speed, and the number of vehicles at the actual speed and preset speed is used to determine the road congestion situation.

[0003] However, these methods for detecting traffic congestion all require the road direction to be determined in advance. For example, fixed cameras need to confirm the camera orientation and road conditions in advance, while vehicle-mounted cameras collect traffic density information based on the direction of vehicle travel, resulting in poor flexibility and scalability. Summary of the Invention

[0004] This application provides a road congestion detection method, system, electronic device, and storage medium based on unmanned aerial vehicles (UAVs), which at least solves the problem of poor flexibility and scalability of road congestion detection methods in related technologies.

[0005] In a first aspect, embodiments of this application provide a road congestion detection method based on unmanned aerial vehicles (UAVs), the method comprising: By acquiring road images of the target area using drones, and analyzing the road images, vehicle target boxes and main road straight lines are obtained; The center point of each vehicle is determined based on the vehicle target box. The vehicle fitting line is obtained based on the center point using the RANSAC algorithm. The angle, distance and overlap between the main road line and the vehicle fitting line are analyzed. The similarity between the main road line and the vehicle fitting line is determined based on the analysis results. If the similarity is greater than a preset similarity threshold, each target vehicle is tracked. Based on the tracking results, UAV location information, and UAV camera operating parameters, the speed of each target vehicle is determined, wherein the target vehicle is determined based on a straight line fitted to the vehicle. The number of target vehicles is counted, a reference speed is determined based on the speed and number of all target vehicles, and congestion in the target area is determined based on the reference speed and the number of target vehicles.

[0006] In some embodiments, analyzing the road image to obtain the straight line of the main road includes: Based on the YOLO11 model, the road image is segmented to obtain a mask image containing road classification information; Based on the road classification information, the mask image is converted into a binary image; Based on principal component analysis, the straight line of the main road is determined according to the mask image and the binarized image.

[0007] In some embodiments, determining the straight line of the main road based on the mask image and the binarized image using principal component analysis includes: Center all pixels in the mask image and calculate the covariance matrix of the mask image after centering. Calculate the eigenvalues ​​of the covariance matrix and the eigenvectors corresponding to the eigenvalues, and take the eigenvector with the largest eigenvalue as the principal component direction; The target centroid is determined based on the binarized image, and the main road straight line is determined based on the target centroid and the principal component direction.

[0008] In some embodiments, obtaining the vehicle fitting line based on the center point using the RANSAC algorithm includes: A preset number of center point subsets are generated by randomly sampling from all center points, and line fitting is performed based on each of the center point subsets to obtain multiple sets of candidate parameters for the line. For each set of candidate parameters, obtain the error from all center points to the line under the candidate parameter, mark the center points whose error is less than or equal to a preset error threshold as interior points of the line under the candidate parameter, and count the number of interior points. The line with the largest number of interior points is used as the vehicle fitting line.

[0009] In some embodiments, the analysis of the angle, distance, and overlap between the straight road and the vehicle-fitted straight line, and the determination of the similarity between the straight road and the vehicle-fitted straight line based on the analysis results, includes: Determine the angle difference between the straight line of the main road and the fitted line of the vehicle, and obtain the angle score based on the angle difference and the preset angle scoring model; The endpoints of the arterial road straight line and the vehicle fitted line are determined in the road image to obtain the arterial road line segment and the vehicle fitted line segment. The shortest distance from the vehicle fitted line segment to the arterial road line segment is calculated. Based on the shortest distance and a preset distance scoring model, a distance score is obtained. Project the vehicle-fitted line segment onto the main road line segment, and use the intersection length of the projection intervals as the overlap score; The similarity between the straight line of the main road and the fitted line of the vehicle is obtained by weighting the angle score, the distance score and the overlap score.

[0010] In some embodiments, the UAV location information includes UAV latitude and longitude information, altitude information, and gimbal attitude information; determining the speed of each target vehicle based on the tracking results, UAV location information, and UAV camera operating parameters includes: The tracking results of multiple consecutive frames are obtained as target frames, and the pixel coordinates of the target vehicle in each target frame are determined. Based on the ground clearance information, the gimbal attitude information, and the UAV camera operating parameters, the pixel coordinates of the target vehicle in each target frame are normalized to the camera coordinate system to obtain the relative coordinates of the target vehicle in each target frame. Based on the latitude and longitude information of the UAV and the relative coordinates, the world coordinates of the target vehicle in each target frame are obtained; The time parameters corresponding to the target frame are determined based on the operating parameters of the UAV camera, and the speed of each target vehicle is determined based on the time parameters and the world coordinates of the target vehicle.

[0011] In some embodiments, determining the reference speed based on the speed of all the target vehicles includes: Based on the number of target vehicles and the speed of all target vehicles, determine the median vehicle speed, and use the median as the reference speed; or Based on the speeds of all the target vehicles, the average speed of the vehicles is determined, and the average speed is used as the reference speed.

[0012] Secondly, embodiments of this application provide a road congestion detection system based on unmanned aerial vehicles (UAVs), the system comprising: The image acquisition module is used to acquire road images of the target area via a drone, analyze the road images, and obtain vehicle target boxes and main road straight lines; The similarity calculation module is used to determine the center point of each vehicle based on the vehicle target box, obtain the vehicle fitting line based on the center point using the RANSAC algorithm, analyze the angle, distance and overlap between the main road line and the vehicle fitting line, and determine the similarity between the main road line and the vehicle fitting line based on the analysis results. The speed determination module is used to track each target vehicle when the similarity is greater than a preset similarity threshold, and determine the speed of each target vehicle based on the tracking results, UAV location information and UAV camera operating parameters, wherein the target vehicle is determined based on a straight line fitted to the vehicle. The judgment module is used to count the number of target vehicles, determine a reference speed based on the speed and number of all target vehicles, and determine whether the target area is congested based on the reference speed and the number of target vehicles.

[0013] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the road congestion detection method based on unmanned aerial vehicles as described in the first aspect above.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the road congestion detection method based on unmanned aerial vehicles as described in the first aspect above.

[0015] Compared to related technologies, the road congestion detection method based on UAVs provided in this application significantly reduces false positives by using dual calibration of vehicle target boxes and straight lines on main roads, combined with multi-dimensional similarity analysis of angle, distance, and overlap. Relying on UAVs to acquire images, it is not limited by the location of fixed ground equipment and can cover various target areas such as urban main roads and suburban roads. By filtering target vehicles through similarity thresholds, it can adapt to scenarios with different lane widths and road orientations, demonstrating strong versatility and solving the problem of poor flexibility and scalability in traditional road congestion detection methods. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a road congestion detection method based on unmanned aerial vehicles (UAVs) according to an embodiment of this application; Figure 2 This is a flowchart of a road congestion early warning method according to an embodiment of this application; Figure 3 This is a structural block diagram of a road congestion detection system based on an unmanned aerial vehicle (UAV) according to an embodiment of this application; Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0018] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0019] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0020] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0021] This embodiment provides a road congestion detection method based on unmanned aerial vehicles (UAVs). Figure 1 This is a flowchart of a road congestion detection method based on unmanned aerial vehicles (UAVs) according to an embodiment of this application, such as... Figure 1As shown, the process includes the following steps: Step S101: Obtain road images of the target area using a drone, analyze the road images to obtain vehicle target boxes and main road straight lines.

[0022] The spatial location of all vehicles in the image is located using object detection algorithms (such as YOLO, Faster R-CNN, etc.), and the boundaries of the vehicles are marked with rectangular boxes. Road images may contain interfering elements such as pedestrians, non-motorized vehicles, and roadside buildings. The extraction process of "vehicle target boxes" filters out non-vehicle targets through algorithms to ensure that subsequent analysis focuses only on motor vehicles on the road and reduces interference from irrelevant information.

[0023] In some embodiments, step S101 involves analyzing the road image to obtain the straight line of the main road, including: Step S1011: Based on the YOLO11 model, segment the road image to obtain a mask image containing road classification information.

[0024] Step S1012: Based on the road classification information, convert the mask image into a binary image.

[0025] Step S1013: Based on principal component analysis, determine the straight line of the main road according to the mask image and the binarized image.

[0026] The YOLO11 model is used to segment the road image, obtaining a mask containing road classification information. The mask belonging to the road category is extracted from the mask information and processed into a binary image. Optionally, the target road is white and the rest of the background is black; or, the road area is 1 and the non-road area is 0.

[0027] The YOLO11 model is used to segment road images. Through the semantic segmentation capabilities of deep learning models, the specific category of "road" in the image can be accurately identified, and a mask image containing road classification information can be generated. Compared with traditional edge detection, this method can more intelligently filter out non-road areas such as sky, buildings, and vegetation, narrowing the scope of subsequent analysis from the source and ensuring that only the real road areas are focused on.

[0028] Converting the mask image into a binary image, through this pixel-level processing that clearly distinguishes black and white, further simplifies the morphological features of the road, highlighting its boundaries and overall outline. The binarized image reduces the impact of grayscale variations and lighting interference (such as tree shade and backlighting) on ​​subsequent line extraction, making the road's geometric features easier for the algorithm to capture.

[0029] Principal Component Analysis (PCA) is used to determine the straightness of arterial roads. The core of PCA is to find the direction with the largest variance in the data, which corresponds to the main direction in the binary image where the pixel distribution in the road area is most concentrated. This direction can precisely reflect the extension trend of the arterial road (such as the direction of a straight road or the approximate straight direction of a gentle curve). Compared with traditional straight-line detection algorithms such as Hough Transform, PCA can more robustly ignore local fluctuations at the road edge (such as lane spacing and roadside gravel), and directly fit a baseline straight line representing the overall direction of the arterial road. It is especially suitable for complex scenarios such as multi-lane and wide roads.

[0030] In some embodiments, step S1013 specifically includes: Step S201: Center all pixels in the mask image and calculate the covariance matrix of the mask image after centering.

[0031] Step S202: Calculate the eigenvalues ​​of the covariance matrix and the corresponding eigenvectors, and take the eigenvector with the largest eigenvalue as the principal component direction.

[0032] Step S203: Determine the target centroid based on the binarized image, and determine the main road straight line based on the target centroid and the principal component direction.

[0033] Assume there are n points in the mask. Center these points and remove the mean:

[0034] Centering is performed on all pixels in the mask image (subtracting the mean of the pixel coordinates) to eliminate the influence of the origin position on the data distribution, allowing the analysis to focus on the relative distribution trend of the pixels.

[0035] Put all X i Organized by rows / columns (common matrix D is n×2), the covariance matrix is:

[0036] Where C is a 2×2 symmetric matrix:

[0037] The centered covariance matrix is ​​calculated to quantify the distribution correlation of pixels in the road area along the x and y directions (e.g., whether they are densely arranged along a certain direction), providing a mathematical basis for subsequent extraction of the main direction. This step avoids the limitation of relying solely on edge pixels, but instead utilizes the global distribution characteristics of all pixels within the road area, enhancing adaptability to complex road morphologies (such as wide roads and slightly deformed roads).

[0038] calculate The eigenvalues ​​λ1≥λ2 and the corresponding eigenvectors v1, v2 are used to determine the direction of maximum variance. As the principal component direction, that is, the main road direction vector.

[0039] The eigenvalues ​​of the covariance matrix reflect the degree of dispersion (variance) of the data in the direction of the corresponding eigenvector. The eigenvector with the largest eigenvalue (principal component direction) corresponds exactly to the direction in which the road pixel distribution is most "extensive" - ​​this direction is the core trend of the extension of the main road (such as the direction of a straight road or the approximate straight direction of a gentle curve).

[0040] If the first principal component vector is v1=(v x ,v y The commonly used angle is defined as the direction relative to the x-axis of the image (horizontally to the right):

[0041] For a set S of white pixels (representing roads) in a binary image, the centroid (C0) is... x C y It can be calculated from the average of the pixel coordinates: Let the set of white pixel coordinates be ( For column indexes, horizontally to the right; (If it is a row index, vertically downwards), then

[0042] If the pixel value of a binary image is I(x,y)∈{0,1}, then the zeroth moment Q 00 =∑I(x,y)=M, first moment Q 10 =∑xI(x,y),Q 01 =∑yI(x,y), then the centroid is

[0043] The main road line can be obtained based on the centroid and the first principal component vector: passing through point (C). x C y The direction is θ. Therefore, the direction vector of the main road is:

[0044] The parametric formula for a straight main road is:

[0045] Determining the target centroid (geometric center of the road area) by binarizing the image essentially provides an "anchor point" for the arterial road straight line. Combined with the principal component direction (the main orientation of the road), the spatial location of the arterial road straight line (a straight line passing through the centroid and along the principal component direction) can be uniquely determined. This dual constraint of "direction + location" ensures that the extracted arterial road straight line not only conforms to the overall orientation of the road but also accurately falls near the center of the road area, preventing the straight line from deviating from the actual road range and providing a precise spatial benchmark for subsequent vehicle-road matching analysis.

[0046] Continue to refer to Figure 1 After obtaining the vehicle target box and the main road straight line, step S102 is executed.

[0047] Step S102: Determine the center point of each vehicle based on the vehicle target box, obtain the vehicle fitting line based on the center point using the RANSAC algorithm, analyze the angle, distance and overlap between the main road line and the vehicle fitting line, and determine the similarity between the main road line and the vehicle fitting line based on the analysis results.

[0048] Optionally, the formula for calculating the vehicle center coordinates is:

[0049] In some embodiments, step S102, obtaining the vehicle fitting line based on the center point using the RANSAC algorithm, includes: Step S1021: Randomly sample from all center points to generate a preset number of center point subsets, and perform line fitting based on each center point subset to obtain multiple sets of candidate parameters for the line.

[0050] Step S1022: For each set of candidate parameters, obtain the error from all center points to the line under the candidate parameter, mark the center points whose error is less than or equal to the preset error threshold as the interior points of the line under the candidate parameter, and count the number of interior points.

[0051] Step S1023: Select the line with the largest number of interior points as the vehicle fitting line.

[0052] Assume the vehicle's fitted linear model is as follows:

[0053] The parameters of the line are:

[0054] For any point (x) i , y i The error of the line is defined as:

[0055] If the center point error is less than the preset error threshold t, then the point is considered an interior point of the current line.

[0056] Assuming the number of center points is N1, the objective of the fitting parameters is to maximize the number of interior points:

[0057] Random sampling point set Perform fitting model

[0058] Calculate the error from all points to the model:

[0059] Based on the magnitude of the error, determine how to find the optimal model within a finite number of iterations K:

[0060] If the error exceeds expectations, the center point coordinates are recalculated and the analysis is repeated.

[0061] First, the center point (such as the geometric center of the box) is determined by the vehicle target box, simplifying the vehicle from the "box" to "point", focusing on the spatial distribution of the vehicle; then, the RANSAC algorithm is used to fit the vehicle straight line - the core of RANSAC is to filter out the "interior points" (vehicle points that conform to the mainstream distribution) from the noisy data and fit a straight line, which can effectively eliminate the interference of abnormal vehicles (such as parked vehicles, vehicles crossing the road), and the final vehicle fitted straight line can truly reflect the overall distribution trend of vehicles on the road.

[0062] In some embodiments, step S102 involves analyzing the angle, distance, and overlap between the straight road and the vehicle-fitted straight line, and determining the similarity between the straight road and the vehicle-fitted straight line based on the analysis results, including: Step S1024: Determine the angle difference between the main road straight line and the vehicle fitted straight line, and obtain the angle score based on the angle difference and the preset angle scoring model.

[0063] Step S1025: Determine the endpoints of the main road straight line and the vehicle fitted line in the road image to obtain the main road line segment and the vehicle fitted line segment. Calculate the shortest distance from the vehicle fitted line segment to the main road line segment. Based on the shortest distance and the preset distance scoring model, obtain the distance score.

[0064] Step S1026: Project the vehicle fitted line segment onto the main road line segment, and use the intersection length of the projection interval as the overlap score.

[0065] Step S1027: The angle score, distance score, and overlap score are weighted and calculated to obtain the similarity between the main road straight line and the vehicle fitted straight line.

[0066] Calculate the angle difference between the two lines: Two-line direction vectors Angular difference (taken in [0, 90°] or [0, 180°]), vector angle (radians):

[0067] The endpoints of the arterial road segment and the vehicle-fitted line segment are determined based on the road image. The shortest distance from the vehicle-fitted line segment to the arterial road segment is calculated; if they intersect, the distance is set to 0.

[0068] Calculate the projected length of the vehicle-fitted line segment onto the main road segment: Project the two line segments onto a number line (taking a unit vector in a common direction) along a shared principal direction. ), and perform scalar projection on the endpoints, with the overlap length being the intersection length of the projected intervals. Define the overlap ratio as:

[0069] Where overlap_ratio is the overlap ratio and overlap_len is the overlap length. l 1 and l 2 represents the lengths of the vehicle-fitted line segment and the main road line segment, respectively. The value of overlap_ratio is... .

[0070] Angle Score S θ The calculation formula is as follows:

[0071] in, θ max Choose 90° or 180° depending on the actual situation.

[0072] Distance Score d The calculation formula is as follows:

[0073] in, d th This is the normalized distance threshold.

[0074] Overlap score S o The calculation formula is as follows:

[0075] Provides a practical combined metric that combines angle, distance, and overlap into a single score. .

[0076] Calculate the similarity S:

[0077] Among them, w θ w d w o All are parameter weights, w θ +w d +w o =1. Optionally, w θ =0.4, w d =0.3, w o =0.3.

[0078] Continue to refer to Figure 1 After obtaining the similarity between the main road straight line and the vehicle fitted straight line, step S103 is executed.

[0079] Step S103: If the similarity is greater than a preset similarity threshold, track each target vehicle. Based on the tracking results, UAV location information and UAV camera operating parameters, determine the speed of each target vehicle. The target vehicle is determined based on a vehicle fitted straight line.

[0080] A threshold for line segment similarity is set. Only when the similarity reaches the threshold is the vehicle's fitted straight line considered valid, and the vehicle is allowed to travel on the main road.

[0081] If the center point of a vehicle deviates too far from the fitted straight line (exceeding a certain distance), these vehicles will be identified as interfering vehicles, and all other vehicles will be set as target vehicles. The fitted straight line will then be used to filter out truly valid vehicles traveling along the main road, eliminating interfering vehicles (such as parked vehicles, vehicles traveling in the wrong direction, and vehicles not traveling on the main road).

[0082] In some embodiments, the UAV location information includes the UAV's latitude and longitude information, altitude information, and gimbal attitude information; step S103, based on the tracking results, UAV location information, and UAV camera operating parameters, determines the speed of each target vehicle, including: Step S1031: Obtain the tracking results of multiple consecutive frames as target frames, and determine the pixel coordinates of the target vehicle in each target frame.

[0083] Step S1032: Based on the ground altitude information, gimbal attitude information, and UAV camera operating parameters, normalize the pixel coordinates of the target vehicle in each target frame to the camera coordinate system to obtain the relative coordinates of the target vehicle in each target frame.

[0084] Step S1033: Based on the latitude and longitude information and relative coordinates of the UAV, obtain the world coordinates of the target vehicle in each target frame.

[0085] Step S1034: Determine the time parameters corresponding to the target frame based on the UAV camera's operating parameters, and determine the speed of each target vehicle based on the time parameters and the world coordinates of the target vehicles.

[0086] Acquire the UAV's WGS84 latitude and longitude location information, gimbal attitude information (pitch, yaw), altitude information, and camera intrinsic parameters (f). x , f y , c x , c y The system calculates the vehicle's latitude and longitude using the video resolution and the vehicle's pixel location (pixel_x, pixel_y).

[0087] First, normalize the coordinate system by normalizing the pixel coordinate system to the camera coordinate system:

[0088] At this time, (X) c Y c Z c This indicates the viewing direction vector of the pixel in the camera coordinate system.

[0089] Calculate the scale factor K for the intersection with the ground plane based on the camera height and pitch angle. Assume the camera height is A, the camera pitch angle is θ = pitch, and the current pixel's line-of-sight tilt angle (the angle relative to the camera's optical axis) is Y. c The intersection of the line of sight and the ground plane satisfies:

[0090] Right now:

[0091] Calculate the spatial distance from the camera's line of sight to the ground: The actual spatial projection distance (R) from the camera's line of sight to the ground and its lateral component (W) are as follows:

[0092] In the UAV coordinate system, the two-dimensional offset of a ground point relative to the camera includes: forward distance K and lateral distance W.

[0093] Convert the camera's GPS coordinates to UTM coordinates (CamX, CamY). Assuming the camera's yaw angle (camera facing angle) is yaw, the world coordinates of the ground point are:

[0094] The formula for converting UTM coordinates back to GPS coordinates is as follows:

[0095] When normalizing pixel coordinates to the camera coordinate system, the effects of camera tilt and altitude changes on pixel position are eliminated by combining the drone's altitude, gimbal attitude, and camera parameters (such as focal length and intrinsic matrix). This transforms the two-dimensional image coordinates into a positional relationship relative to the camera in three-dimensional space. Furthermore, by combining the drone's own latitude and longitude information, the relative coordinates are mapped to world coordinates in the actual geographic space (such as latitude, longitude, and altitude), achieving a precise conversion from image pixels to physical location.

[0096] The KCF algorithm is used to track multiple target vehicles, assigning each target a different ID. Tracking results for N consecutive frames are taken, and the above steps are repeated to calculate the world coordinates of the target vehicle in each frame. Based on the calculation results, the latitude and longitude displacement dx of the vehicle for N consecutive frames is obtained. Simultaneously, the real-time T corresponding to the N frames is calculated.

[0097] The vehicle speed is calculated using displacement and time. v 0:

[0098] The time interval (time parameter) of the target frame is determined by the camera's operating parameters (such as frame rate), and the speed is calculated by combining the displacement difference of world coordinates. This directly correlates the movement distance and time in physical space, and yields the vehicle's actual driving speed on the actual road.

[0099] Step S104: Count the number of target vehicles, determine the reference speed based on the speed and number of all target vehicles, and determine whether congestion has occurred in the target area based on the reference speed and the number of target vehicles.

[0100] When the number of target vehicles exceeds a set threshold and the reference speed is less than a set speed threshold, the road is considered congested, triggering a congestion warning. Figure 2 This is a flowchart of a road congestion early warning method according to an embodiment of this application.

[0101] In some embodiments, determining the reference speed based on the speeds of all target vehicles in step S104 includes: Step S1041: Based on the number of target vehicles and the speed of all target vehicles, determine the median vehicle speed and use the median as the reference speed.

[0102] Step S1042: Determine the average speed of the vehicles based on the speeds of all target vehicles, and use the average speed as the reference speed.

[0103] In this embodiment, the median speed of all target vehicles can be used as the reference distance. The median is not affected by a few extreme speed values ​​and can accurately reflect the actual driving status of most target vehicles. In traffic flow, the driving speed of most vehicles tends to be concentrated, and the median can directly anchor to this "mainstream speed," avoiding the distortion of reference speed caused by extreme values ​​and making congestion judgment more in line with actual road conditions.

[0104] The average speed of all target vehicles can also be used as a reference distance. When the vehicle speed distribution on the road is relatively uniform (without obvious extreme values), the average speed can comprehensively reflect the speed level of the overall traffic flow, providing a balanced quantitative basis for congestion judgment.

[0105] The two methods can be switched according to the actual scenario. For example, the median should be used for congested road sections (where low speeds are likely to occur), while the average value can be used for unobstructed road sections (where speeds are evenly distributed).

[0106] Through the above steps, the dual calibration of the vehicle target bounding box and the straight line of the main road, combined with multi-dimensional similarity analysis of angle, distance, and overlap, significantly reduces false positives. Relying on drones to acquire images, it is not limited by the location of fixed ground equipment and can cover various target areas such as urban main roads and suburban roads. By filtering target vehicles through similarity thresholds, it can adapt to scenarios with different lane widths and road directions, demonstrating strong versatility and solving the problem of poor flexibility and scalability of road congestion detection methods.

[0107] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0108] This embodiment also provides a road congestion detection system based on unmanned aerial vehicles (UAVs). This system is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0109] Figure 3 This is a structural block diagram of a road congestion detection system based on an unmanned aerial vehicle (UAV) according to an embodiment of this application, such as... Figure 3 As shown, the system includes: The image acquisition module 31 is used to acquire road images of the target area through the UAV, analyze the road images, and obtain vehicle target boxes and main road straight lines.

[0110] The similarity calculation module 32 is used to determine the center point of each vehicle based on the vehicle target box, obtain the vehicle fitting line based on the center point using the RANSAC algorithm, analyze the angle, distance and overlap between the main road line and the vehicle fitting line, and determine the similarity between the main road line and the vehicle fitting line based on the analysis results.

[0111] The speed determination module 33 is used to track each target vehicle when the similarity is greater than a preset similarity threshold. Based on the tracking results, UAV location information and UAV camera operating parameters, the speed of each target vehicle is determined, wherein the target vehicle is determined based on the vehicle fitted straight line.

[0112] The judgment module 34 is used to count the number of target vehicles, determine the reference speed based on the speed and number of all target vehicles, and determine whether congestion has occurred in the target area based on the reference speed and the number of target vehicles.

[0113] In some embodiments, the image acquisition module 31 includes: The segmentation module is used to segment road images based on the YOLO11 model to obtain mask images containing road classification information.

[0114] The binarization module is used to convert the mask image into a binarized image based on road classification information.

[0115] The trunk road determination module is used to determine the straightness of trunk roads based on principal component analysis, using mask images and binarized images.

[0116] In some embodiments, the trunk road determination module includes: The matrix generation module is used to center all pixels in the mask image and calculate the covariance matrix of the mask image after centering.

[0117] The direction determination module is used to calculate the eigenvalues ​​of the covariance matrix and the corresponding eigenvectors, and to take the eigenvector with the largest eigenvalue as the principal component direction.

[0118] The straight line determination module is used to determine the target centroid based on the binarized image, and to determine the main road straight line based on the target centroid and the principal component direction.

[0119] In some embodiments, the similarity calculation module 32 includes: The parameter generation module is used to randomly sample from all center points to generate a preset number of center point subsets, and perform line fitting based on each center point subset to obtain multiple sets of candidate parameters for the line.

[0120] The quantity statistics module is used to obtain the error of all center points to the line under the candidate parameter for each set of candidate parameters, mark the center points with errors less than or equal to the preset error threshold as the interior points of the line under the candidate parameter, and count the number of interior points.

[0121] The fitting line determination module is used to select the line with the largest number of interior points as the vehicle fitting line.

[0122] In some embodiments, the similarity calculation module 32 includes: The angle analysis module is used to determine the angle difference between the straight road and the fitted straight line of the vehicle, and to obtain the angle score based on the angle difference and the preset angle scoring model.

[0123] The distance analysis module is used to calculate the shortest distance from the fitted straight line of the vehicle to the straight line of the main road. Based on the shortest distance and the preset distance scoring model, a distance score is obtained.

[0124] The overlap analysis module is used to project the vehicle's fitted straight line onto the main road straight line, and the intersection length of the projected intervals is used as the overlap score.

[0125] The similarity analysis module is used to perform weighted calculations on angle scores, distance scores, and overlap scores to obtain the similarity between the main road straight line and the vehicle fitted straight line.

[0126] In some embodiments, the UAV location information includes the UAV's latitude and longitude information, altitude information, and gimbal attitude information; the speed determination module 33 includes: The tracking module is used to acquire the tracking results of multiple consecutive frames as target frames and determine the pixel coordinates of the target vehicle in each target frame.

[0127] The coordinate normalization module is used to normalize the pixel coordinates of the target vehicle in each target frame to the camera coordinate system based on the ground altitude information, gimbal attitude information and UAV camera operating parameters, so as to obtain the relative coordinates of the target vehicle in each target frame.

[0128] The coordinate transformation module is used to obtain the world coordinates of the target vehicle in each target frame based on the latitude and longitude information and relative coordinates of the UAV.

[0129] The speed analysis module is used to determine the time parameters corresponding to the target frame based on the operating parameters of the UAV camera, and to determine the speed of each target vehicle based on the time parameters and the world coordinates of the target vehicle.

[0130] In some embodiments, the determination module 34 includes: The first speed calculation module is used to determine the median speed of the vehicles based on the number of target vehicles and the speed of all target vehicles, and uses the median speed as a reference speed.

[0131] The second speed calculation module is used to determine the average speed of the vehicles based on the speeds of all target vehicles, and uses the average speed as a reference speed.

[0132] By employing dual calibration of vehicle target boxes and straight lines on main roads, combined with multi-dimensional similarity analysis of angle, distance, and overlap, false positives are significantly reduced. Relying on drones for image acquisition, it is not limited by the location of fixed ground equipment and can cover various target areas such as urban main roads and suburban roads. Target vehicles are filtered through similarity thresholds, making it adaptable to scenarios with different lane widths and road orientations. Its strong versatility solves the problem of poor flexibility and scalability in road congestion detection methods.

[0133] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0134] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0135] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0136] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program: S1: Obtain road images of the target area using a drone, analyze the road images to obtain vehicle target boxes and main road straight lines.

[0137] S2. Determine the center point of each vehicle based on the vehicle target box. Using the RANSAC algorithm, obtain the vehicle fitting line based on the center point. Analyze the angle, distance and overlap between the main road line and the vehicle fitting line. Determine the similarity between the main road line and the vehicle fitting line based on the analysis results.

[0138] S3. When the similarity is greater than the preset similarity threshold, track each target vehicle. Based on the tracking results, UAV location information and UAV camera operating parameters, determine the speed of each target vehicle. The target vehicle is determined based on the vehicle fitted straight line.

[0139] S4: Count the number of target vehicles, determine the reference speed based on the speed and number of all target vehicles, and determine whether congestion has occurred in the target area based on the reference speed and the number of target vehicles.

[0140] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0141] In one embodiment, Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 4 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 4 As shown, this electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a UAV-based road congestion detection method.

[0142] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0144] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A road congestion detection method based on unmanned aerial vehicles (UAVs), characterized in that, The method includes: By acquiring road images of the target area using drones, and analyzing the road images, vehicle target boxes and main road straight lines are obtained; The center point of each vehicle is determined based on the vehicle target box. The vehicle fitting line is obtained based on the center point using the RANSAC algorithm. The angle, distance and overlap between the main road line and the vehicle fitting line are analyzed. The similarity between the main road line and the vehicle fitting line is determined based on the analysis results. If the similarity is greater than a preset similarity threshold, each target vehicle is tracked. Based on the tracking results, UAV location information, and UAV camera operating parameters, the speed of each target vehicle is determined, wherein the target vehicle is determined based on a straight line fitted to the vehicle. The number of target vehicles is counted, a reference speed is determined based on the speed and number of all target vehicles, and congestion in the target area is determined based on the reference speed and the number of target vehicles.

2. The method according to claim 1, characterized in that, The analysis of the road image to obtain the straight lines of the main road includes: Based on the YOLO11 model, the road image is segmented to obtain a mask image containing road classification information; Based on the road classification information, the mask image is converted into a binary image; Based on principal component analysis, the straight line of the main road is determined according to the mask image and the binarized image.

3. The method according to claim 2, characterized in that, The method based on principal component analysis, determining the straight line of the main road according to the mask image and the binarized image, includes: Center all pixels in the mask image and calculate the covariance matrix of the mask image after centering. Calculate the eigenvalues ​​of the covariance matrix and the eigenvectors corresponding to the eigenvalues, and take the eigenvector with the largest eigenvalue as the principal component direction; The target centroid is determined based on the binarized image, and the main road straight line is determined based on the target centroid and the principal component direction.

4. The method according to claim 1, characterized in that, The process of obtaining the vehicle fitting line based on the center point using the RANSAC algorithm includes: A preset number of center point subsets are generated by randomly sampling from all center points, and line fitting is performed based on each of the center point subsets to obtain multiple sets of candidate parameters for the line. For each set of candidate parameters, obtain the error from all center points to the line under the candidate parameter, mark the center points whose error is less than or equal to a preset error threshold as interior points of the line under the candidate parameter, and count the number of interior points. The line with the largest number of interior points is used as the vehicle fitting line.

5. The method according to claim 1, characterized in that, The analysis of the angle, distance, and overlap between the straight line of the main road and the fitted line of the vehicle, and the determination of the similarity between the straight line of the main road and the fitted line of the vehicle based on the analysis results, includes: Determine the angle difference between the straight line of the main road and the fitted line of the vehicle, and obtain the angle score based on the angle difference and the preset angle scoring model; The endpoints of the arterial road straight line and the vehicle fitted line are determined in the road image to obtain the arterial road line segment and the vehicle fitted line segment. The shortest distance from the vehicle fitted line segment to the arterial road line segment is calculated. Based on the shortest distance and a preset distance scoring model, a distance score is obtained. Project the vehicle-fitted line segment onto the main road line segment, and use the intersection length of the projection intervals as the overlap score; The similarity between the straight line of the main road and the fitted line of the vehicle is obtained by weighting the angle score, the distance score and the overlap score.

6. The method according to claim 1, characterized in that, The UAV location information includes UAV latitude and longitude information, ground clearance information, and gimbal attitude information; determining the speed of each target vehicle based on the tracking results, UAV location information, and UAV camera operating parameters includes: The tracking results of multiple consecutive frames are obtained as target frames, and the pixel coordinates of the target vehicle in each target frame are determined. Based on the ground clearance information, the gimbal attitude information, and the UAV camera operating parameters, the pixel coordinates of the target vehicle in each target frame are normalized to the camera coordinate system to obtain the relative coordinates of the target vehicle in each target frame. Based on the latitude and longitude information of the UAV and the relative coordinates, the world coordinates of the target vehicle in each target frame are obtained; The time parameters corresponding to the target frame are determined based on the operating parameters of the UAV camera, and the speed of each target vehicle is determined based on the time parameters and the world coordinates of the target vehicle.

7. The method according to claim 1, characterized in that, The determination of the reference speed based on the speed of all the target vehicles includes: Based on the number of target vehicles and the speed of all target vehicles, determine the median vehicle speed, and use the median as the reference speed; or Based on the speeds of all the target vehicles, the average speed of the vehicles is determined, and the average speed is used as the reference speed.

8. A road congestion detection system based on unmanned aerial vehicles (UAVs), characterized in that, The system includes: The image acquisition module is used to acquire road images of the target area through a drone, analyze the road images, and obtain vehicle target boxes and main road straight lines; The similarity calculation module is used to determine the center point of each vehicle based on the vehicle target box, obtain the vehicle fitting line based on the center point using the RANSAC algorithm, analyze the angle, distance and overlap between the main road line and the vehicle fitting line, and determine the similarity between the main road line and the vehicle fitting line based on the analysis results. The speed determination module is used to track each target vehicle when the similarity is greater than a preset similarity threshold, and determine the speed of each target vehicle based on the tracking results, UAV location information and UAV camera operating parameters, wherein the target vehicle is determined based on a straight line fitted to the vehicle. The judgment module is used to count the number of target vehicles, determine a reference speed based on the speed and number of all target vehicles, and determine whether the target area is congested based on the reference speed and the number of target vehicles.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the road congestion detection method based on unmanned aerial vehicles as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the UAV-based road congestion detection method as described in any one of claims 1 to 7.

Images