Highway vehicle speed measurement method and device based on unmanned aerial vehicle aerial photography

Abstract

The invention provides an expressway vehicle speed measurement method and device based on unmanned aerial vehicle aerial photography, and the method comprises the steps: recognizing an expressway pavement region, a green belt region, a lane line position and a first vehicle target according to a video image obtained through the aerial photography of an unmanned aerial vehicle on an expressway; screening the first vehicle target according to the highway pavement area to obtain a second vehicle target; performing tracking and camera motion compensation on the second vehicle target to obtain an actual pixel displacement of the second vehicle target between adjacent image frames; fitting according to a green belt area to obtain a highway main direction curve, and calculating the pixel width of an adjacent lane in combination with a lane line position; establishing a mapping relation between the image pixel and the actual space distance; and calculating the actual driving speed of the second vehicle target according to the actual pixel displacement of the second vehicle target between the adjacent image frames and the mapping relationship between the image pixels and the actual spatial distance. Therefore, the accurate measurement of the vehicle running speed in the dynamic aerial photography scene is realized.

Description

Technical Field

[0001] This invention relates to the field of vehicle speed measurement technology, and in particular to a method and device for measuring vehicle speed on highways based on drone aerial photography. Background Technology

[0002] With the development of intelligent transportation technology, drones, with their advantages of flexibility, mobility, and wide field of view, have gradually replaced or assisted manual labor as an important means of daily highway inspection. During inspection operations, vehicle speed is the core basis for judging key events such as traffic congestion, abnormal parking, traffic accidents, and obstruction of emergency lanes. Therefore, obtaining accurate vehicle speed information during drone mobile inspections is crucial for improving the level of intelligent highway management.

[0003] Currently, vehicle speed measurement on highways mainly relies on fixed camera monitoring systems. This fixed-point monitoring technology is well-suited for applications in scenarios where equipment installation locations are fixed and the background environment is relatively stable.

[0004] However, directly applying the aforementioned fixed-viewpoint-based speed measurement scheme to drone aerial photography scenarios faces significant technical obstacles. When performing inspection missions, drones are in continuous flight, and their pitch, yaw, and translational movements cause overall dynamic shifts in the captured images. Traditional methods struggle to distinguish between the global background displacement caused by camera movement and the local displacement of the vehicle itself, leading to target positioning drift and displacement calculation failures. This fails to meet the requirements for real-time, accurate vehicle speed measurement in mobile inspection scenarios. Summary of the Invention

[0005] This invention provides a method and device for measuring vehicle speed on highways based on drone aerial photography, which solves the problem that existing speed measurement schemes cannot accurately measure speed when directly applied to drone aerial images, and achieves accurate measurement of vehicle speed in dynamic aerial photography scenarios.

[0006] This invention provides a method for measuring vehicle speed on highways based on drone aerial photography, comprising the following steps: Based on video images obtained by drones from aerial photography of highways, identify the highway surface area, green belt area, lane line position, and the first vehicle target; The first vehicle target is filtered according to the highway road surface area to obtain the second vehicle target; and the second vehicle target is tracked and camera motion compensation is performed to obtain the actual pixel displacement of the second vehicle target between adjacent image frames. The main direction curve of the highway is obtained by fitting the green belt area; the pixel width of the adjacent lane is calculated based on the main direction curve of the highway and the position of the lane line; and a mapping relationship between image pixels and actual spatial distance is established based on the standard physical width of the highway and the pixel width of the adjacent lane. The actual speed of the second vehicle target is calculated based on the actual pixel displacement of the second vehicle target between adjacent image frames and the mapping relationship between the image pixels and the actual spatial distance.

[0007] According to the present invention, a method for measuring vehicle speed on highways based on unmanned aerial vehicle (UAV) aerial photography is provided. This method identifies the highway surface area, green belt area, lane line positions, and a first vehicle target based on video images obtained from UAV aerial photography of the highway. The method includes: Based on the trained semantic segmentation model, masks for the highway road surface area and the green belt area are obtained from the video image; the semantic segmentation model is trained based on a dataset of aerial images of highways with labels for the highway road surface area and the green belt area. The lane line recognition model is trained to extract lane line positions from the video image; the lane line recognition model is trained on a dataset of lane line image samples with lane line position labels. Based on the trained target detection model, the target detection result of the first vehicle target is obtained from the video image. The target detection result of the first vehicle target includes the bounding box coordinates, orientation angle and vehicle category of the first vehicle target. The target detection model is trained on a highway vehicle image sample dataset with bounding box coordinate labels, orientation angle labels and vehicle category labels.

[0008] According to the present invention, a method for measuring vehicle speed on highways based on UAV aerial photography is provided, which tracks the second vehicle target and performs camera motion compensation to obtain the actual pixel displacement of the second vehicle target between adjacent image frames, including: Background feature points other than the second vehicle target are extracted from the video image, and the projection transformation relationship between adjacent image frames is calculated by tracking the position of the background feature points in adjacent image frames. The first position of the second vehicle target in the previous image frame is obtained; and the second position of the second vehicle target in the current image frame is determined according to the optical flow tracing algorithm. Motion compensation is performed on the first position according to the projection transformation relationship between adjacent image frames, and the first position is transformed into the coordinate system of the current frame image to obtain the compensated first position; Based on the compensated first position and second position, the actual pixel displacement of the second vehicle target between adjacent image frames is taken.

[0009] According to the present invention, a method for measuring vehicle speed on highways based on UAV aerial photography, determining the second position of the second vehicle target in the current image frame according to an optical flow tracking algorithm, includes: A Kalman filter is established for the second vehicle target, and the predicted state of the second vehicle target in the current frame is predicted according to the state transition equation; The target detection result of the second vehicle target is fused with the target position and velocity information obtained by optical flow tracking to obtain the observation position of the second vehicle target in the current frame; The predicted state of the second vehicle target in the current frame is motion compensated using the projection transformation relationship to obtain the compensated predicted position. The compensated predicted position is optimally matched with the observed position, and the observed position after successful matching is determined as the second position of the second vehicle target in the current frame image.

[0010] According to the present invention, a method for measuring vehicle speed on highways based on drone aerial photography is provided, wherein the main direction curve of the highway is obtained by fitting the green belt area, including: Based on the green belt area, extract the set of contour pixel coordinates of the green belt area; Based on the set of contour pixel coordinates, a polynomial curve is fitted using the least squares method, and outliers are removed according to a pre-set distance threshold to obtain the main direction curve of the highway.

[0011] According to the present invention, a method for measuring vehicle speed on a highway based on drone aerial photography calculates the pixel width of adjacent lanes based on the main direction curve of the highway and the position of the lane lines, including: Using the center coordinates of the target detection box of the second vehicle target as a reference point, draw a normal line in the direction of the main direction curve of the highway. Calculate the intersection points of the normal line and the two lane lines adjacent to the second vehicle target in the lane line position, and take the pixel distance between the intersection points as the pixel width of the adjacent lane.

[0012] The present invention also provides a highway vehicle speed measurement device based on drone aerial photography, comprising the following modules: The aerial photography recognition module is used to identify the highway surface area, green belt area, lane line position, and first vehicle target based on video images obtained by drones from aerial photography of the highway. The tracking compensation module is used to filter the first vehicle target based on the highway road surface area to obtain the second vehicle target; and to track and compensate the second vehicle target for camera motion to obtain the actual pixel displacement of the second vehicle target between adjacent image frames. The reference mapping module is used to fit the main direction curve of the highway based on the green belt area, calculate the pixel width of the adjacent lane based on the main direction curve of the highway and the lane line position, and establish a mapping relationship between image pixels and actual spatial distance based on the standard physical width of the highway and the pixel width of the adjacent lane. The calculation output module is used to calculate the actual driving speed of the second vehicle target based on the actual pixel displacement of the second vehicle target between adjacent image frames and the mapping relationship between the image pixels and the actual spatial distance.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the highway vehicle speed measurement method based on UAV aerial photography as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the highway vehicle speed measurement method based on UAV aerial photography as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the highway vehicle speed measurement method based on UAV aerial photography as described above.

[0016] The present invention provides a method and device for measuring vehicle speed on highways based on drone aerial photography. By identifying road surface elements and vehicle targets in aerial images, the selected vehicles are tracked and their true pixel displacements are obtained using camera motion compensation. At the same time, a mapping relationship between image pixels and actual spatial distance is established based on the main direction curve fitted by the green belt and the standard width of the lane. This effectively compensates for the interference caused by the drone's flight motion and achieves accurate measurement of vehicle speed in dynamic aerial photography scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the method for measuring vehicle speed on highways based on drone aerial photography provided by the present invention.

[0019] Figure 2 This is a schematic diagram of vehicle speed calculation provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the structure of the highway vehicle speed measurement device based on drone aerial photography provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] Figure 1 This is a flowchart illustrating the highway vehicle speed measurement method based on UAV aerial photography provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps: Step 100: Based on the video images obtained by the drone from aerial photography of the highway, identify the highway surface area, green belt area, lane line position, and the first vehicle target.

[0024] Step 101: Filter the first vehicle target according to the highway road surface area to obtain the second vehicle target; and track and compensate the second vehicle target for camera motion to obtain the actual pixel displacement of the second vehicle target between adjacent image frames.

[0025] Step 102: Fit the main direction curve of the highway based on the green belt area, calculate the pixel width of the adjacent lane based on the main direction curve of the highway and the position of the lane lines; and establish the mapping relationship between image pixels and actual spatial distance based on the standard physical width of the highway and the pixel width of the adjacent lanes.

[0026] Step 103: Calculate the actual driving speed of the second vehicle target based on the actual pixel displacement of the second vehicle target between adjacent image frames and the mapping relationship between image pixels and actual spatial distance.

[0027] Specifically, the vehicle speed measurement method for highways based on drone aerial photography provided by the present invention can be implemented by a system consisting of a drone with video recording function and an electronic device with certain computing capabilities.

[0028] In this embodiment of the invention, drones are first used to conduct aerial photography over highways to acquire real-time video image data. Based on these video images, the system can use image recognition algorithms to analyze key elements in the video footage, identifying the highway surface area, the green belt area serving as the median strip, and the lane line positions. Simultaneously, the system can also detect all vehicles present in the image and mark them as the first vehicle target.

[0029] After completing the basic element recognition in the video image, the first vehicle target detected in the preliminary stage can be screened and refined.

[0030] In some implementations, the system can use the identified highway surface area as a mask or boundary condition to spatially filter the first vehicle target, eliminate interfering targets outside the road surface area, and thus obtain the second vehicle target located within the highway driving area.

[0031] The system can then perform cross-frame tracking of these second vehicle targets and conduct camera motion compensation. This compensation process eliminates motion components introduced by changes in the UAV's flight attitude or movement, thereby accurately calculating the actual pixel displacement of the second vehicle target relative to the ground between adjacent image frames.

[0032] In order to convert pixel-level displacement into displacement in the real physical world, the system can establish a geometric mapping relationship between the image and the real space.

[0033] In this process, the system can fit a main direction curve representing the road's orientation to the identified green belt area. Using this curve as a reference, and combining it with lane line positions, the system calculates the pixel width of adjacent lanes, i.e., the pixel width between adjacent lanes. The standard physical width of the highway lanes can be used as a known prior condition. By comparing the standard physical width of the highway lanes with the calculated pixel width of adjacent lanes, a mapping relationship between image pixels and actual spatial distances is established, serving as a scale in the speed calculation process.

[0034] Finally, the speed can be calculated by combining the data obtained from the above steps. The system can convert the pixel displacement of the second vehicle target after motion compensation between adjacent image frames into the actual physical distance by combining the established mapping relationship between image pixels and actual spatial distance. The actual physical distance is then divided by the interval time between image frames to calculate the actual driving speed of the second vehicle target.

[0035] The present invention provides a method for measuring vehicle speed on highways based on drone aerial photography. By identifying road surface elements and vehicle targets in aerial images, the method tracks the selected vehicles and uses camera motion compensation to obtain their true pixel displacement. At the same time, it establishes a mapping relationship between image pixels and actual spatial distance based on the main direction curve fitted by the green belt and the standard width of the lane, thereby effectively compensating for the interference caused by the drone's flight motion and realizing accurate measurement of vehicle speed in dynamic aerial photography scenarios.

[0036] According to the present invention, a method for measuring vehicle speed on highways based on unmanned aerial vehicle (UAV) aerial photography is provided. This method identifies the highway surface area, green belt area, lane line positions, and a first vehicle target based on video images obtained from UAV aerial photography of the highway. The method includes: Based on the trained semantic segmentation model, masks for highway road surface areas and green belt areas are obtained from video images; the semantic segmentation model is trained on a dataset of aerial images of highways with labels for highway road surface areas and green belt areas. Based on the trained lane line recognition model, lane line positions are extracted from video images; the lane line recognition model is trained on a dataset of lane line image samples with lane line position labels. Based on the trained target detection model, the target detection result of the first vehicle target is obtained from the video image. The target detection result of the first vehicle target includes the bounding box coordinates, orientation angle and vehicle category of the first vehicle target. The target detection model is trained on a highway vehicle image sample dataset with bounding box coordinate labels, orientation angle labels and vehicle category labels.

[0037] Specifically, in order to achieve comprehensive perception of highway scenes, embodiments of this application can identify key components in the images one by one using different types of deep learning models based on video images acquired by drone aerial photography.

[0038] First, to distinguish between road and non-road areas, a semantic segmentation model can be constructed using deep learning techniques. The training process for this model can utilize a pre-prepared dataset of aerial images of highways, which may contain precisely labeled information indicating highway surface areas and green belt areas. By training the semantic segmentation model using this dataset, it can acquire pixel-level classification capabilities.

[0039] In practical applications, video images can be input into the trained semantic segmentation model, which can output masks of the corresponding highway road surface area and green belt area, thereby clarifying the region category to which each pixel in the image belongs.

[0040] To clarify the road's geometry, a lane line recognition model can be used to process video images. The training process for this model utilizes a dataset of lane line image samples with lane line location labels, allowing the model to learn the texture and geometric features of the lane lines. Based on the trained lane line recognition model, the system can analyze video images captured by drones, accurately extracting the lane line location information and providing a geometric reference for subsequent lane width calculations.

[0041] For vehicles in video images, object detection models can be used to identify the first vehicle target. During the training phase of the object detection model, a highway vehicle image sample dataset containing rich annotation information can be used. These sample labels can include not only the vehicle's bounding box coordinates but also detailed information such as the vehicle's heading angle and vehicle category. The object detection model trained using this dataset can then perform inference on the video image to obtain the target detection result for the first vehicle target. This detection result can specifically include the bounding box coordinates of the first vehicle target in the image, its heading angle, and its specific vehicle category, thereby achieving accurate localization and attribute recognition of the vehicle target.

[0042] According to the present invention, a method for measuring vehicle speed on highways based on UAV aerial photography is provided, which tracks a second vehicle target and performs camera motion compensation to obtain the actual pixel displacement of the second vehicle target between adjacent image frames, including: Background feature points other than the second vehicle target are extracted from the video image. The projection transformation relationship between adjacent image frames is calculated by tracking the position of the background feature points in adjacent image frames. Obtain the first position of the second vehicle target in the previous image frame; and determine the second position of the second vehicle target in the current image frame according to the optical flow tracing algorithm. Motion compensation is performed on the first position based on the projection transformation relationship between adjacent image frames, and the first position is transformed into the coordinate system of the current frame image to obtain the compensated first position; Based on the compensated first and second positions, the actual pixel displacement of the second vehicle target between adjacent image frames is taken.

[0043] Specifically, in order to accurately calculate the actual motion of the vehicle in a dynamic context, embodiments of the present invention can first eliminate interference caused by the movement of the UAV itself.

[0044] The system can identify and extract the area outside the second vehicle target from video images as the background region, and extract background feature points from it. By continuously tracking the positional changes of these background feature points in adjacent image frames, the overall motion trend of the background can be analyzed, and the projection transformation relationship between adjacent image frames can be calculated. This relationship actually describes how the camera's field of view has undergone translation, rotation, or scale changes between two frames.

[0045] After obtaining the transformation relationship of camera motion, the specific state of the vehicle target at different times can be further determined. The system can obtain the position of each second vehicle target in the previous image frame and record it as the first position.

[0046] To determine the position of each second vehicle target at the current moment, an optical flow tracing algorithm can be used to track the targets, thereby determining the position of each second vehicle target in the current image frame and marking it as the second position. This step ensures that the system can capture the vehicle's position information over a continuous time series.

[0047] To compare the second vehicle target in the previous frame with the second vehicle target in the current frame under the same reference, the coordinates can be standardized. Using the previously calculated projection transformation relationship between adjacent image frames, the system can perform motion compensation on the first position in the previous frame. This transformation maps or converts the first position to the coordinate system of the current frame image, thus obtaining the compensated first position. This compensated position represents the position the vehicle should appear in the current frame if it were stationary, solely due to camera movement.

[0048] Finally, based on the above processing results, the actual motion of the vehicle can be calculated. The system can compare the compensated first position with the second position of the current frame obtained through optical flow tracking, calculate the difference between the two, and use it as the actual pixel displacement of the second vehicle target between adjacent image frames.

[0049] According to the present invention, a method for measuring vehicle speed on highways based on UAV aerial photography, which determines the second position of a second vehicle target in the current image frame using an optical flow tracking algorithm, includes: A Kalman filter is established for the second vehicle target, and the predicted state of the second vehicle target in the current frame is predicted according to the state transition equation. The target detection results of the second vehicle target are fused with the target position and velocity information obtained by optical flow tracking to obtain the observation position of the second vehicle target in the current frame; Motion compensation is performed on the predicted state of the second vehicle target in the current frame using the projection transformation relationship to obtain the compensated predicted position. The compensated predicted position is optimally matched with the observed position, and the observed position after successful matching is determined as the second position of the second vehicle target in the current frame image.

[0050] Specifically, in order to achieve continuous and stable tracking of vehicle targets, embodiments of the present invention can first estimate the motion state of the vehicle through a prediction mechanism.

[0051] The system can establish a Kalman filter for each second vehicle target to be tracked, which internally maintains the target's state vector. Based on the state information of the previous time step, the system can use the state transition equation to deduce the target's state distribution at the current time step, thereby predicting the predicted state of the second vehicle target in the current frame, thus providing a priori position estimation for subsequent tracking.

[0052] While acquiring the predicted state, the system also needs to obtain the actual observation data at the current moment to correct the prediction. To this end, the system can fuse the target detection results obtained through the target detection model with the target position and velocity information obtained through the optical flow tracking algorithm. This fusion mechanism allows for the comprehensive utilization of both the semantic information from the detection and the motion information from the optical flow, thereby obtaining a more accurate observation position of the second vehicle target in the current frame.

[0053] Given the camera movement during drone aerial photography, the predicted state of the previous frame is derived from the coordinate system of that frame, which will lead to discrepancies when directly compared with the observed position in the current frame. Therefore, after obtaining the predicted state, motion compensation for the second vehicle target in the current frame can be performed using pre-calculated projection transformation relationships between adjacent image frames. Through this step, the predicted state based on the coordinate system of the previous frame can be mapped to the coordinate system of the current frame, resulting in a corrected, compensated predicted position aligned with the current view.

[0054] Finally, to determine the vehicle's exact location in the current frame, the system performs a correlation analysis between the compensated predicted location obtained in the previous steps and the observed location obtained through data fusion. By executing an optimal matching algorithm, the system finds the best correspondence between possible predictions and observations. When a match is successful, the system determines the corresponding observed location as the second position of the second vehicle target in the current image frame, thus completing the precise localization of the target in the current frame.

[0055] According to the present invention, a method for measuring vehicle speed on highways based on drone aerial photography is provided, which obtains the main direction curve of the highway by fitting the green belt area, including: Based on the green belt area, extract the set of outline pixel coordinates of the green belt area; Based on the set of contour pixel coordinates, a polynomial curve is fitted using the least squares method, and outliers are removed according to a pre-set distance threshold to obtain the main direction curve of the highway.

[0056] Specifically, in order to construct a geometric reference benchmark for speed measurement, this embodiment can first extract geometric features based on the green belt area identified in the previous steps.

[0057] The system can extract the pixel location data that constitutes the boundary of the green belt area, and collect these discrete pixel location data to obtain the contour pixel coordinate set of the green belt area, realizing the transformation of the semantic region at the image level into a coordinate dataset at the mathematical level.

[0058] After obtaining the above set of contour pixel coordinates, in order to accurately restore the overall trend of the highway, the system can then perform mathematical modeling on these discrete coordinate data.

[0059] In some implementations, a preliminary highway route curve can be constructed by performing polynomial curve fitting based on the set of contour pixel coordinates using the least squares method.

[0060] In some implementations, considering that the original identified contour may contain noise or irregular protrusions, the system can also verify the data points involved in the fitting based on a preset distance threshold, and identify and remove those points that are too far away from the fitted curve as outliers.

[0061] Through the above fitting process, a smooth and representative main direction curve of the highway can be obtained.

[0062] The embodiments of the present invention employ the least squares method for polynomial fitting and an outlier removal mechanism, which can effectively adapt to various complex road shapes such as straight or curved highways and filter out edge noise interference, thereby providing an accurate reference for subsequently establishing a high-precision mapping relationship between image pixels and actual spatial distance.

[0063] According to the present invention, a method for measuring vehicle speed on highways based on unmanned aerial vehicle (UAV) aerial photography calculates the pixel width of adjacent lanes based on the main direction curve and lane line positions of the highway, including: Using the center coordinates of the target detection box of the second vehicle target as the reference point, draw a normal line in the direction of the main curve of the highway. Calculate the intersection of the normal and the two lane lines adjacent to the second vehicle target in the lane line position, and take the pixel distance between the intersection points as the pixel width of the adjacent lane.

[0064] Specifically, in order to obtain a reference quantity in the image pixel space that can accurately reflect the spatial scale of the current vehicle position, embodiments of the present invention may first construct geometric auxiliary lines for measurement.

[0065] The system can use the center coordinates of the target detection box of the second vehicle target as a reference point to determine the vehicle's specific position in the image. Then, based on the fitted highway main direction curve as a reference for the road direction, a normal line is drawn from this reference point towards the direction of the highway main direction curve. This step is to establish a measurement reference line perpendicular to the road's direction of travel, ensuring that subsequent width measurements are based on the vehicle's current lateral cross-section.

[0066] After establishing the aforementioned normal reference, specific pixel distance calculations can be performed based on this normal. The system can perform spatial geometric analysis on the normal and the lane line positions identified in the image, filter out the two lane lines adjacent to the second vehicle target, and calculate the intersection of the normal and these two adjacent lane lines.

[0067] Finally, the system can calculate the pixel distance between the two intersection points using mathematical calculations, and use this distance as the pixel width of the adjacent lane.

[0068] In this embodiment of the invention, by drawing the normal to the main direction curve with the vehicle center as the reference, the lane pixel width at each point can be dynamically calculated based on the specific location of each vehicle and the degree of road curvature. This dynamic measurement method for each vehicle target can accurately adapt to perspective distortion and road geometric changes, thus providing an accurate foundation for subsequently establishing the mapping relationship between image pixels and actual spatial distances.

[0069] The following examples, through specific application scenarios, further illustrate the highway vehicle speed measurement method based on UAV aerial photography provided by the present invention.

[0070] I. System Overview The vehicle speed measurement method in this embodiment is integrated into the intelligent highway inspection system. This system uses a drone as a mobile monitoring platform, equipped with a high-resolution optical camera to conduct dynamic inspections of highways. The real-time video stream collected by the drone can be analyzed through two processing paths: first, relying on the drone's built-in edge computing module, lightweight artificial intelligence (AI) algorithms are used to complete real-time data processing; second, the video data is transmitted back to the cloud server in real time, leveraging the powerful computing resources of the cloud to perform deep analysis. Based on deep learning algorithms, the system analyzes the video stream frame by frame, accurately identifying various abnormal highway conditions such as abnormal parking, low-speed driving, traffic congestion, traffic accidents, and occupation of emergency lanes. After detecting an anomaly, the system pushes the anomaly alarm information to the highway management platform in real time through Message Queuing Telemetry Transport (MQTT) or Hypertext Transfer Protocol (HTTP) interfaces. This platform integrates multi-dimensional data, providing managers with event warnings, trend analysis, and decision support, significantly improving the level of intelligent highway management and emergency response efficiency.

[0071] II. Methods and Steps The vehicle speed measurement method in this embodiment first preprocesses the video frames to optimize image quality and standardize specifications; then, through semantic segmentation, lane line recognition, and target detection, it clarifies the road surface area, lane line position, and vehicle target; next, it tracks the vehicle target while compensating for camera motion to obtain accurate target displacement, and fits the main direction curve of the highway as a reference benchmark; finally, it calculates the vehicle's speed by combining the mapping relationship between the actual lane width and image pixels and the vehicle's inter-frame displacement. The specific steps are as follows: 1. Image preprocessing For video frame images acquired by drone aerial photography, median filtering algorithm is used to denoise the images, and camera intrinsic parameters provided by the manufacturer are used to correct image distortion. To adapt to the input requirements of subsequent deep learning models, the images are uniformly adjusted to a fixed size.

[0072] 2. Image Region and Target Recognition: (1) Semantic segmentation: Based on the semantic segmentation model, a large amount of labeled data (including road surface, green belt and other areas) from the aerial photography scene of the highway was selected for model training. The model performs pixel-level classification on the preprocessed image and outputs the segmentation mask of the highway road surface area and the middle green belt area to clarify the pixel ownership of each area.

[0073] (2) Lane line recognition: The lane line model is trained by a large number of highway lane line images, and key features such as the geometric contour and virtual and real texture of the lane line are extracted. The pixel coordinates of the lane line are output to realize the accurate positioning and depiction of the lane line.

[0074] (3) Vehicle target detection: A target detection model is selected to detect vehicle targets. The model is input with the preprocessed image and outputs the bounding box coordinates (x, y, w, h), vehicle orientation angle, class label and confidence score of all vehicle targets in the image.

[0075] 3. Target tracking and image displacement calculation; (1) Target filtering: Based on the highway surface area mask obtained by semantic segmentation, the vehicle targets detected by the target detection model are spatially filtered to retain only vehicle targets that are completely or partially overlapping with the highway area.

[0076] (2) Optical flow tracking: A sparse optical flow tracking method based on feature points is adopted to extract significant feature points within the vehicle target box; between two adjacent frames, the optical flow algorithm is used to track the feature points, and the displacement of the feature points is solved by minimizing the error function; based on the optical flow vector of the feature points, the centroid displacement of the target box is calculated, and the position of the target box is updated.

[0077] (3) Camera motion compensation. First, extract the background area outside the vehicle target from the image and find the feature points in it; then track the positional correspondence of these background feature points in two adjacent frames, and calculate the projection transformation relationship between the two frames.

[0078] (4) Multi-target tracking. A Kalman filter is established for each tracked target, and the state vector contains information such as the target's position and velocity. The state distribution of the target in the next frame is predicted by the state transition equation. The detection results of the target detection model are fused with the target position and velocity information obtained by optical flow tracking to form the observed vehicle position. Motion compensation is performed on the predicted state using the projection transformation relationship between two frames. Then, the predicted vehicle position and the observed vehicle position are optimally matched so that each prediction result corresponds to the most matched actual vehicle. The positions of the same vehicle in consecutive frames are strung together to achieve continuous tracking of multiple vehicles.

[0079] 4. Fitting the high-speed main direction curve Based on the mask of the intermediate green belt region obtained by semantic segmentation, its contour pixel coordinate set is extracted, and the least squares method is used to fit the contour points of the green belt to a curve. To accommodate the various shapes that may exist on highways, such as straight lines and curves, a polynomial is selected as the fitting model to construct the error function. By randomly selecting a subset of data for fitting, the distance from the data points to the fitted curve is calculated, and outliers are removed by setting a distance threshold. This process is repeated until the fitted curve with the best robustness is obtained.

[0080] 5. Speed ​​Calculation Calculation of pixel width of adjacent lanes. Figure 2 This is a schematic diagram of vehicle speed calculation provided by the present invention, such as... Figure 2 As shown, for each tracked vehicle target, a normal line is drawn towards the main direction curve of the highway, using the center coordinates of its target box as the reference point. Using the lane line recognition results, the intersection point of this normal line with the adjacent lane line is found (e.g., ...). Figure 2 (For points X1 and X2), calculate the pixel distance between adjacent intersection points as the pixel width of the adjacent lane.

[0081] Speed ​​calculation. Given the standard physical width of highway lanes and the pixel width of adjacent lanes, establish a mapping relationship between image pixels and actual spatial distances. Obtain the vehicle's pixel position from the previous frame using multi-target tracking, and perform motion compensation based on the projection transformation relationship between two frames to transform the vehicle's position from the previous frame to its position in the current frame (e.g., ...). Figure 2 C1 represents the pixel position of the vehicle from the previous frame to the current frame, and is then compared with the vehicle's actual position in the current frame (e.g., ...). Figure 2 (C2 is the current frame position of the vehicle) to calculate the actual pixel displacement of the vehicle between adjacent image frames; divide by the image frame interval time to calculate the actual driving speed of the vehicle.

[0082] This embodiment successfully solves the problem of high-speed vehicle speed measurement when the camera and vehicle are moving simultaneously during drone aerial photography, providing vehicle speed information for traffic management based on drone inspection and effectively meeting the actual needs of intelligent inspection of highways.

[0083] Based on the vehicle speed information provided in this embodiment, various traffic event monitoring functions can be realized during drone patrol. Specifically, these include high-speed abnormal parking detection (by judging whether the vehicle speed is below a threshold and remains stationary for an extended period), low-speed driving detection (by setting a lower speed threshold), and road congestion assessment (classifying traffic as smooth, slow, or congested based on average vehicle speed and density). Furthermore, it provides auxiliary data support for traffic accident analysis and emergency lane occupancy detection, thereby improving the intelligence and efficiency of highway traffic management.

[0084] The following describes the highway vehicle speed measurement device based on UAV aerial photography provided by the present invention. The highway vehicle speed measurement device based on UAV aerial photography described below can be referred to in correspondence with the highway vehicle speed measurement method based on UAV aerial photography described above.

[0085] Figure 3 This is a schematic diagram of a highway vehicle speed measurement device based on drone aerial photography provided by the present invention, as shown below. Figure 3 As shown, the device includes the following modules: The aerial photography recognition module 300 is used to identify the highway road surface area, green belt area, lane line position and first vehicle target based on video images obtained by drone aerial photography of the highway. The tracking compensation module 310 is used to filter the first vehicle target according to the highway road surface area to obtain the second vehicle target; and to track and compensate the second vehicle target for camera motion to obtain the actual pixel displacement of the second vehicle target between adjacent image frames. The reference mapping module 320 is used to fit the main direction curve of the highway according to the green belt area, calculate the pixel width of the adjacent lane according to the main direction curve of the highway and the lane line position; and establish a mapping relationship between image pixels and actual spatial distance according to the standard physical width of the highway and the pixel width of the adjacent lane. The calculation output module 330 is used to calculate the actual driving speed of the second vehicle target based on the actual pixel displacement of the second vehicle target between adjacent image frames and the mapping relationship between the image pixels and the actual spatial distance.

[0086] According to the present invention, a highway vehicle speed measurement device based on drone aerial photography identifies the highway road surface area, green belt area, lane line position, and a first vehicle target based on video images obtained by drone aerial photography of the highway, including: Based on the trained semantic segmentation model, masks for highway road surface areas and green belt areas are obtained from video images; the semantic segmentation model is trained on a dataset of aerial images of highways with labels for highway road surface areas and green belt areas. Based on the trained lane line recognition model, lane line positions are extracted from video images; the lane line recognition model is trained on a dataset of lane line image samples with lane line position labels. Based on the trained target detection model, the target detection result of the first vehicle target is obtained from the video image. The target detection result of the first vehicle target includes the bounding box coordinates, orientation angle and vehicle category of the first vehicle target. The target detection model is trained on a highway vehicle image sample dataset with bounding box coordinate labels, orientation angle labels and vehicle category labels.

[0087] According to the present invention, a highway vehicle speed measurement device based on UAV aerial photography tracks a second vehicle target and performs camera motion compensation to obtain the actual pixel displacement of the second vehicle target between adjacent image frames, including: Background feature points other than the second vehicle target are extracted from the video image. The projection transformation relationship between adjacent image frames is calculated by tracking the position of the background feature points in adjacent image frames. Obtain the first position of the second vehicle target in the previous image frame; and determine the second position of the second vehicle target in the current image frame according to the optical flow tracing algorithm. Motion compensation is performed on the first position based on the projection transformation relationship between adjacent image frames, and the first position is transformed into the coordinate system of the current frame image to obtain the compensated first position; Based on the compensated first and second positions, the actual pixel displacement of the second vehicle target between adjacent image frames is taken.

[0088] According to the present invention, a highway vehicle speed measurement device based on UAV aerial photography determines the second position of a second vehicle target in the current image frame based on an optical flow tracking algorithm, comprising: A Kalman filter is established for the second vehicle target, and the predicted state of the second vehicle target in the current frame is predicted according to the state transition equation. The target detection results of the second vehicle target are fused with the target position and velocity information obtained by optical flow tracking to obtain the observation position of the second vehicle target in the current frame; Motion compensation is performed on the predicted state of the second vehicle target in the current frame using the projection transformation relationship to obtain the compensated predicted position. The compensated predicted position is optimally matched with the observed position, and the observed position after successful matching is determined as the second position of the second vehicle target in the current frame image.

[0089] According to the present invention, a highway vehicle speed measurement device based on UAV aerial photography obtains the main direction curve of the highway by fitting a green belt area, comprising: Based on the green belt area, extract the set of outline pixel coordinates of the green belt area; Based on the set of contour pixel coordinates, a polynomial curve is fitted using the least squares method, and outliers are removed according to a pre-set distance threshold to obtain the main direction curve of the highway.

[0090] According to the present invention, a highway vehicle speed measurement device based on UAV aerial photography calculates the pixel width of adjacent lanes based on the main direction curve and lane line positions of the highway, including: Using the center coordinates of the target detection box of the second vehicle target as the reference point, draw a normal line in the direction of the main curve of the highway. Calculate the intersection of the normal and the two lane lines adjacent to the second vehicle target in the lane line position, and take the pixel distance between the intersection points as the pixel width of the adjacent lane.

[0091] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a highway vehicle speed measurement method based on UAV aerial photography, the method including: Based on video images obtained by drones from aerial photography of highways, identify the highway surface area, green belt area, lane line position, and the first vehicle target; The first vehicle target is selected based on the highway road surface area to obtain the second vehicle target; and the second vehicle target is tracked and camera motion compensation is performed to obtain the actual pixel displacement of the second vehicle target between adjacent image frames. The main direction curve of the highway is obtained by fitting the green belt area; the pixel width of the adjacent lane is calculated based on the main direction curve of the highway and the position of the lane lines; and a mapping relationship between image pixels and actual spatial distance is established based on the standard physical width of the highway and the pixel width of the adjacent lanes. The actual speed of the second vehicle target is calculated based on the actual pixel displacement of the second vehicle target between adjacent image frames and the mapping relationship between image pixels and actual spatial distance.

[0092] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the highway vehicle speed measurement method based on UAV aerial photography provided by the above methods, the method comprising: Based on video images obtained by drones from aerial photography of highways, identify the highway surface area, green belt area, lane line position, and the first vehicle target; The first vehicle target is selected based on the highway road surface area to obtain the second vehicle target; and the second vehicle target is tracked and camera motion compensation is performed to obtain the actual pixel displacement of the second vehicle target between adjacent image frames. The main direction curve of the highway is obtained by fitting the green belt area; the pixel width of the adjacent lane is calculated based on the main direction curve of the highway and the position of the lane lines; and a mapping relationship between image pixels and actual spatial distance is established based on the standard physical width of the highway and the pixel width of the adjacent lanes. The actual speed of the second vehicle target is calculated based on the actual pixel displacement of the second vehicle target between adjacent image frames and the mapping relationship between image pixels and actual spatial distance.

[0094] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the highway vehicle speed measurement method based on UAV aerial photography provided by the above methods, the method comprising: Based on video images obtained by drones from aerial photography of highways, identify the highway surface area, green belt area, lane line position, and the first vehicle target; The first vehicle target is selected based on the highway road surface area to obtain the second vehicle target; and the second vehicle target is tracked and camera motion compensation is performed to obtain the actual pixel displacement of the second vehicle target between adjacent image frames. The main direction curve of the highway is obtained by fitting the green belt area; the pixel width of the adjacent lane is calculated based on the main direction curve of the highway and the position of the lane lines; and a mapping relationship between image pixels and actual spatial distance is established based on the standard physical width of the highway and the pixel width of the adjacent lanes. The actual speed of the second vehicle target is calculated based on the actual pixel displacement of the second vehicle target between adjacent image frames and the mapping relationship between image pixels and actual spatial distance.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for measuring vehicle speed on highways based on drone aerial photography, characterized in that, include: Based on video images obtained by drones from aerial photography of highways, identify the highway surface area, green belt area, lane line position, and the first vehicle target; The first vehicle target is filtered based on the highway surface area to obtain the second vehicle target; In addition, the second vehicle target is tracked and camera motion compensation is performed to obtain the actual pixel displacement of the second vehicle target between adjacent image frames; The main direction curve of the highway is obtained by fitting the green belt area, and the pixel width of the adjacent lane is calculated based on the main direction curve of the highway and the position of the lane line. Furthermore, based on the standard physical width of the highway and the pixel width of the adjacent lanes, a mapping relationship between image pixels and actual spatial distance is established; The actual speed of the second vehicle target is calculated based on the actual pixel displacement of the second vehicle target between adjacent image frames and the mapping relationship between the image pixels and the actual spatial distance.

2. The method for highway vehicle speed measurement based on UAV aerial photography according to claim 1, characterized in that, Based on video images obtained by drones from highway aerial photography, the system identifies highway surface areas, green belt areas, lane line positions, and the first vehicle target, including: Based on the trained semantic segmentation model, masks for the highway road surface area and the green belt area are obtained from the video image; the semantic segmentation model is trained based on a dataset of aerial images of highways with labels for the highway road surface area and the green belt area. The lane line recognition model is trained to extract lane line positions from the video image; the lane line recognition model is trained on a dataset of lane line image samples with lane line position labels. Based on the trained target detection model, the target detection result of the first vehicle target is obtained from the video image. The target detection result of the first vehicle target includes the bounding box coordinates, orientation angle and vehicle category of the first vehicle target. The target detection model is trained on a highway vehicle image sample dataset with bounding box coordinate labels, orientation angle labels and vehicle category labels.

3. The method for highway vehicle speed measurement based on UAV aerial photography according to claim 2, characterized in that, Tracking and camera motion compensation are performed on the second vehicle target to obtain the actual pixel displacement of the second vehicle target between adjacent image frames, including: Background feature points other than the second vehicle target are extracted from the video image, and the projection transformation relationship between adjacent image frames is calculated by tracking the position of the background feature points in adjacent image frames. The first position of the second vehicle target in the previous image frame is obtained; and the second position of the second vehicle target in the current image frame is determined according to the optical flow tracing algorithm. Motion compensation is performed on the first position according to the projection transformation relationship between adjacent image frames, and the first position is transformed into the coordinate system of the current frame image to obtain the compensated first position; Based on the compensated first position and second position, the actual pixel displacement of the second vehicle target between adjacent image frames is taken.

4. The method for highway vehicle speed measurement based on UAV aerial photography according to claim 3, characterized in that, Based on the optical flow tracing algorithm, the second position of the second vehicle target in the current image frame is determined, including: A Kalman filter is established for the second vehicle target, and the predicted state of the second vehicle target in the current frame is predicted according to the state transition equation; The target detection result of the second vehicle target is fused with the target position and velocity information obtained by optical flow tracking to obtain the observation position of the second vehicle target in the current frame; The predicted state of the second vehicle target in the current frame is motion compensated using the projection transformation relationship to obtain the compensated predicted position. The compensated predicted position is optimally matched with the observed position, and the observed position after successful matching is determined as the second position of the second vehicle target in the current frame image.

5. The method for highway vehicle speed measurement based on UAV aerial photography according to claim 1, characterized in that, The main direction curve of the highway is obtained by fitting the green belt area, including: Based on the green belt area, extract the set of contour pixel coordinates of the green belt area; Based on the set of contour pixel coordinates, a polynomial curve is fitted using the least squares method, and outliers are removed according to a pre-set distance threshold to obtain the main direction curve of the highway.

6. The method for highway vehicle speed measurement based on UAV aerial photography according to claim 1 or 5, characterized in that, Calculating the pixel width of adjacent lanes based on the main direction curve of the highway and the position of the lane lines includes: Using the center coordinates of the target detection box of the second vehicle target as a reference point, draw a normal line in the direction of the main direction curve of the highway. Calculate the intersection points of the normal line and the two lane lines adjacent to the second vehicle target in the lane line position, and take the pixel distance between the intersection points as the pixel width of the adjacent lane.

7. A highway vehicle speed measurement device based on drone aerial photography, characterized in that, include: The aerial photography recognition module is used to identify the highway surface area, green belt area, lane line position, and first vehicle target based on video images obtained by drones from aerial photography of the highway. The tracking compensation module is used to filter the first vehicle target based on the highway road surface area to obtain the second vehicle target; In addition, the second vehicle target is tracked and camera motion compensation is performed to obtain the actual pixel displacement of the second vehicle target between adjacent image frames; The reference mapping module is used to fit the main direction curve of the highway based on the green belt area, and to calculate the pixel width of the adjacent lane based on the main direction curve of the highway and the lane line position. Furthermore, based on the standard physical width of the highway and the pixel width of the adjacent lanes, a mapping relationship between image pixels and actual spatial distance is established; The calculation output module is used to calculate the actual driving speed of the second vehicle target based on the actual pixel displacement of the second vehicle target between adjacent image frames and the mapping relationship between the image pixels and the actual spatial distance.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the highway vehicle speed measurement method based on UAV aerial photography as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the highway vehicle speed measurement method based on UAV aerial photography as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the highway vehicle speed measurement method based on UAV aerial photography as described in any one of claims 1 to 6.

Images