Vehicle speed estimation method and device based on aerial photography of unmanned aerial vehicle and medium

Abstract

The invention discloses a vehicle speed estimation method and device based on unmanned aerial vehicle aerial photography and a medium, belongs to the technical field of vehicle speed estimation, and is used for solving the technical problems that the vehicle speed can be normally calculated only through tedious positioning and calibration, the cost is high and the calculation error is large in an existing method for estimating the vehicle speed through an unmanned aerial vehicle. The method comprises the following steps: performing region-of-interest extraction and image noise reduction preprocessing on video data acquired by aerial photography of an unmanned aerial vehicle to obtain preprocessed video data; performing enhancement processing on an image pixel value in the preprocessed video data to obtain enhanced video data; carrying out linear fitting processing on a lane boundary line on a lane boundary line region in the enhanced video data to obtain a lane boundary line position feature; performing geometric correction and spatial alignment processing on each frame of image; performing identification processing between the vehicle and the dotted line white segment of the lane separation line on the final video data; and performing dynamic and static frame difference estimation on the vehicle speed to obtain the actual vehicle speed.

Description

Technical Field

[0001] The present application relates to the field of vehicle speed estimation, and in particular to a vehicle speed estimation method, device and medium based on drone aerial photography. Background Art

[0002] The methods for estimating vehicle speed using drones mainly include those based on image processing and those based on GPS / IMU data. The image processing-based method usually uses drone aerial photography to obtain raw traffic images, and then performs preprocessing, uses continuous image subtraction to detect moving vehicles, and extracts lane lines to determine the lane range. The lane width is used as a scale to estimate the speed of the moving vehicle.

[0003] Method based on GPS data: The location information provided by GPS is used to calculate the location change at different time points, and then the speed is obtained by dividing it by the time interval. This method has high accuracy in open areas, but is greatly affected by GPS signals. Environments such as urban canyons or forests may affect accuracy and efficiency.

[0004] Inertial measurement unit (IMU) based method: Use accelerometer and gyroscope data to calculate speed by integration. This method does not rely on external signals and can be used in any environment. It is suitable for scenes with rapid dynamic changes, but the cumulative error is large and error correction is required.

[0005] Existing drones have high requirements for detecting vehicle speed, such as flight altitude and shooting angle. They are also prone to flight jitter, which leads to large errors in the captured images and video screen offset. At the same time, due to the complexity of drone hardware and software, it is difficult to apply to more complex scenarios, and thus it is impossible to accurately calculate the actual speed of the moving vehicle. Summary of the invention

[0006] The embodiments of the present application provide a vehicle speed estimation method, device and medium based on drone aerial photography, which are used to solve the following technical problems: the existing drone estimation method of vehicle speed is relatively dependent on high-performance drones, and requires cumbersome positioning and calibration to normally calculate the vehicle speed, which is costly and has large calculation errors.

[0007] The present application embodiment adopts the following technical solutions:

[0008] On the one hand, an embodiment of the present application provides a vehicle speed estimation method based on drone aerial photography, including: performing interest region extraction and image noise reduction preprocessing on video data collected by drone aerial photography to obtain preprocessed video data; based on the adaptive histogram equalization technology, enhancing the image pixel values ​​in the preprocessed video data to obtain enhanced video data; performing straight line fitting processing on the lane boundary line area in the enhanced video data to obtain lane boundary line position characteristics; based on the lane boundary line position characteristics and based on the perspective transformation matrix, performing geometric correction and spatial alignment processing on each frame image to generate final video data; performing recognition processing between the vehicle and the dashed white segment of the lane separator line on the final video data to determine the vehicle trajectory data; estimating the vehicle speed in the vehicle trajectory data by the dynamic and static frame difference method to obtain the actual vehicle speed.

[0009] The embodiments of the present application allow the UAV to accurately measure speed at a variety of height and angle combinations. Stable speed measurement results can be obtained by simply ensuring that the parameters of a single video acquisition are consistent. This feature greatly broadens its scope of application, and is particularly suitable for traffic monitoring and emergency tasks in rapid deployment, complex scenarios, or resource-constrained environments. Not only does it significantly improve the anti-interference ability of the system, but it also enables the speed measurement process to remain stable in complex environments, fully meeting the application requirements of high real-time, low cost, and high reliability, and opening up a new path for data collection and processing in UAV aerial photography scenarios. In addition, vehicle speed estimation is achieved entirely based on video images, without the need to rely on the flight speed, geographic coordinate system, or other external sensor data support of the UAV, which fundamentally simplifies the complexity of hardware and software integration and significantly reduces implementation costs.

[0010] In a feasible implementation, the video data collected by the drone aerial photography is subjected to interest region extraction and image noise reduction preprocessing to obtain preprocessed video data, specifically comprising: using the drone to perform mobile aerial photography of the speed measurement section to collect and obtain first video data; according to the four point coordinates of the preset interest region, the first video data is subjected to the interest region extraction processing to obtain second video data; according to The image value I′(x) after denoising of the second video data is obtained; wherein I(x) and I(y) are the pixel values ​​at pixel x and pixel y, respectively; Ω is a set of all pixels in the image and represents the area where similarity is calculated in the entire image; h is a smoothing parameter used to control the sensitivity of similarity; ||I(x)-I(y)|| 2 is the Euclidean distance between pixel x and pixel y, which is used to measure their similarity; e is a mathematical symbol; based on the image value after denoising of the second video data, the preprocessed video data is obtained again.

[0011] In a feasible implementation, according to the adaptive histogram equalization technology, the image pixel values ​​in the preprocessed video data are enhanced to obtain enhanced video data, specifically including: converting the color image in the preprocessed video data into a grayscale image, and representing the brightness information of each pixel in grayscale; marking the grayscale value of each pixel in the preprocessed video data with a brightness threshold through a preset binary mask to obtain a mask parameter; according to The image grayscale value G enhanced based on the adaptive histogram equalization technology is obtained. enhanced (x, y); wherein G(x, y) is the grayscale value of the original image in the preprocessed video data; H s (k′) is the histogram frequency of the gray level K of the sth small block; L clip is the clipping limit, which is used to prevent the contrast of the area from being excessively enhanced; according to I final (x,y)=M(x,y)·G enhanced (x,y)+(1-M(x,y))·I(x,y), get the final pixel value I of the synthesized image final (x, y); wherein, M(x, y) is a mask parameter; I(x, y) is an image pixel value in the preprocessed video data; based on the final image pixel value, the enhanced video data is determined.

[0012] In a feasible implementation, the lane boundary line area in the enhanced video data is subjected to a linear fitting process related to the lane boundary line to obtain the lane boundary line position feature, specifically comprising: performing image region cutting process on the lane area in the enhanced video data through a preset deep learning model U-Net and based on an activation function to obtain a lane area map; performing coordinate calculation on a set of middle points in the lane area map to obtain a set of coordinates of middle points in the lane area; and performing a linear fitting process on the lane boundary line area in accordance with the preset deep learning model U-Net and the activation function to obtain a set of coordinates of middle points in the lane area. Get the lane boundary line position feature L fit ; Among them, (x i ,y i ) is the coordinate of the lane area midpoint coordinate set M, and the lane boundary line position feature L fit is the fitting straight line function, m is the slope of the fitting straight line function, and b is the constant of the fitting straight line function.

[0013] In a feasible implementation manner, before performing geometric correction and spatial alignment processing on each frame of image according to the lane boundary line position features and based on the perspective transformation matrix to generate final video data, the method further includes:

[0014] The source coordinates H before perspective transformation of each frame image in the enhanced video data are obtained; wherein, m1 is the slope of the first lane boundary line in the lane boundary line position feature, b1 is the constant of the first lane boundary line, and (x1, y1) are the coordinates of the first lane boundary line; m2 is the slope of the second lane boundary line in the lane boundary line position feature, b2 is the constant of the second lane boundary line, and (x2, y2) are the coordinates of the second lane boundary line; the lane boundary line position feature includes: the first lane boundary line and the second lane boundary line; based on the pixel width of the ROI region of interest in the enhanced video data, the source coordinates are converted into expected coordinates; according to the eight key points preset in the lane boundary line, the corresponding coordinates of the expected coordinates are extracted to obtain eight key coordinate points; according to Get the perspective transformation matrix; where, is the perspective transformation matrix (3×3); homogeneous coordinates (x, y, 1) Τ is the source coordinate of each frame image; homogeneous coordinate (x′, y′, w′) Τ is the expected coordinate of each frame image; w′=h 31 x+h 32 y+h 33 is a normalization factor; the eight key coordinate points are input into the perspective transformation matrix to calculate the perspective transformation coordinates.

[0015] In a feasible implementation, according to the lane boundary line position features and based on the perspective transformation matrix, each frame image is geometrically corrected and spatially aligned to generate final video data, specifically including:

[0016] A perspective transformed ROI region of interest ROI′ is obtained; wherein, m3 is the slope of the first lane boundary line after geometric correction and spatial alignment, b3 is a constant of the first lane boundary line, and (x3, y3) is the perspective transformed coordinates of the first lane boundary line; m4 is the slope of the second lane boundary line after geometric correction and spatial alignment, b4 is a constant of the second lane boundary line, and (x4, y4) is the perspective transformed coordinates of the second lane boundary line; based on the perspective transformed ROI region of interest, each frame of the enhanced video data is subjected to geometric correction and spatial alignment processing to obtain the final video data.

[0017] In a feasible implementation manner, the final video data is processed for identification between vehicles and dashed white segments of lane separators to determine vehicle trajectory data, specifically including: identifying all vehicles and all dashed white segments of lane separators in the final video data through a preset YOLO model, and labeling the vehicles and the dashed white segments of lane separators and generating detection frame information; according to a preset DeepSORT algorithm and based on the labels and the detection frame information, tracking all the vehicles and all the dashed white segments of the lane separators, and generating the vehicle trajectory data.

[0018] In a feasible implementation manner, the vehicle speed in the vehicle trajectory data is estimated by a dynamic and static frame difference method to obtain the actual vehicle speed, specifically including: Get the dynamic relative speed of each vehicle in, is the speed of the i-th vehicle in the k-th frame image based on the vehicle trajectory data in the final video data; is the displacement pixel distance of the i-th vehicle from the k-th frame to the k+1-th frame image based on the vehicle trajectory data; N FPS is the number of frames per second of the final video data; Get the average static relative velocity in, is the average speed of all dashed white segments of lane separators in the kth frame image based on the vehicle trajectory data in the final video data; is the number of dashed white segments of the lane separator in the k-th frame image; is the displacement pixel distance of the jth lane separator dashed line white segment in the kth frame to the k+1th frame image based on the vehicle trajectory data;

[0019] Get pixel estimation speed in, is the estimated speed of the pixel of vehicle i in the kth frame based on the vehicle trajectory data; Drone↑Vehicle means that the vehicle and the drone are traveling in the same direction; Drone↓Vehicle means that the vehicle and the drone are traveling in opposite directions; Get the actual speed of the vehicle in, represents the actual estimated speed of vehicle i in the kth frame based on the vehicle trajectory data; Indicates the actual length of the dashed white segment of the lane divider on the road; represents the pixel distance of lane separation line j detected in the kth frame based on the vehicle trajectory data.

[0020] In a second aspect, an embodiment of the present application also provides a vehicle speed estimation device based on drone aerial photography, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute a vehicle speed estimation method based on drone aerial photography as described in any of the above embodiments.

[0021] In a third aspect, an embodiment of the present application further provides a non-volatile computer storage medium, characterized in that the storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions, and when the instructions are executed by a terminal, the terminal executes a vehicle speed estimation method based on drone aerial photography as described in any of the above-mentioned embodiments.

[0022] The present application provides a vehicle speed estimation method, device and medium based on drone aerial photography. Compared with the prior art, the embodiments of the present application have the following beneficial technical effects:

[0023] (1) In the vehicle speed measurement of drone aerial videos, the present invention has no strict requirements on the flight altitude and shooting angle, but it is necessary to maintain its stability during the acquisition of a single video segment without mid-course changes. This design significantly reduces the dependence on the hardware performance and control accuracy of the drone, and provides higher flexibility for the adaptability of the speed measurement system. Unlike traditional methods that strictly rely on fixed flight parameters, the present invention allows accurate speed measurement under a variety of height and angle combinations. It only needs to ensure that the single video acquisition parameters are consistent to obtain stable speed measurement results. This feature greatly broadens its scope of application, and is particularly suitable for traffic monitoring and emergency tasks in rapid deployment, complex scenarios or resource-constrained environments.

[0024] (2) The present invention uses automated image correction technology to effectively eliminate the problem of video image offset caused by jitter, vibration and wind interference during the flight of the drone, thereby ensuring the accuracy of the speed measurement process and the reliability of the data. Unlike traditional methods that rely on mechanical stabilization equipment or complex post-processing, the present invention uses an image analysis algorithm to automatically extract and correct the offset of the image trajectory to achieve high-precision restoration of the target motion information. This design not only significantly improves the system's anti-interference ability, but also enables the speed measurement process to remain stable in complex environments, fully meeting the application requirements of high real-time, low cost and high reliability, and opening up a new path for data collection and processing in drone aerial photography scenarios.

[0025] (3) In addition, the present invention realizes vehicle speed estimation entirely based on video images, without the need for support from the UAV's flight speed, geographic coordinate system or other external sensor data, which fundamentally simplifies the complexity of hardware and software integration and significantly reduces the implementation cost. The algorithm directly calculates the target movement speed through the information between video frames, which reduces the difficulty of deployment while improving the independence and efficiency of the system. The highly automated design enables the technology to be quickly applied to complex multi-target scenarios and can run efficiently without tedious parameter calibration. It is particularly suitable for scenarios that require flexible deployment and rapid response, such as traffic monitoring and disaster emergency response. The above characteristics not only enhance the practical value of the present invention, but also provide important support for the further development of UAV-assisted traffic monitoring technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0027] Figure 1 A flow chart of a vehicle speed estimation method based on drone aerial photography provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of a dashed white segment of a lane separator provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of the structure of a vehicle speed estimation device based on drone aerial photography provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0031] The present application embodiment provides a vehicle speed estimation method based on drone aerial photography, such as Figure 1 As shown, the vehicle speed estimation method based on drone aerial photography specifically includes steps S101-S106:

[0032] S101, performing interest region extraction and image noise reduction preprocessing on video data collected by drone aerial photography to obtain preprocessed video data.

[0033] Specifically, the drone is used to perform mobile aerial photography of the speed measurement section to collect and obtain the first video data.

[0034] Furthermore, according to the coordinates of four points of the preset area of ​​interest, the first video data is subjected to an extraction process of the area of ​​interest to obtain the second video data.

[0035] In one embodiment, a basic section of a highway is selected, and a drone is used to take aerial photos of the speed measurement section. The collected video is recorded as (First video data). During the shooting process, the flight altitude, angle and route of the drone should remain stable. There is no limit on the initial altitude, angle and route direction. You only need to ensure that the road can be clearly photographed. Extract the region of interest and record the extracted video as (Second video data), the ROI calculation formula is as follows: ROI = {(x1, y1), (x2, y1), (x2, y2), (x1, y2)}; where ROI is the extracted region of interest; (x1, y1) ~ (x1, y2) are the coordinates of the four points in the region of interest.

[0036] Furthermore, the video Perform image noise reduction and record the video after noise reduction as (Preprocessing video data). Use non-local means (NLM) to reduce the noise of each pixel x to obtain the image value I′(x), that is, according to The image value I′(x) after denoising the second video data is obtained. Where I(x) and I(y) are the pixel values ​​at pixel x and pixel y respectively; Ω is the set of all pixels in the image and represents the area where similarity is calculated in the entire image. h is a smoothing parameter used to control the sensitivity of similarity; ||I(x)-I(y)|| 2 is the Euclidean distance between pixel x and pixel y, which is used to measure their similarity; e is a mathematical symbol.

[0037] Furthermore, based on the image value after noise reduction of the second video data, the preprocessed video data is obtained again.

[0038] S102: Perform enhancement processing on the image pixel values ​​in the pre-processed video data according to the adaptive histogram equalization technology to obtain enhanced video data.

[0039] Specifically, the color image in the preprocessed video data is converted into a grayscale image, and the brightness information of each pixel is represented in grayscale.

[0040] In one embodiment, first, the input color image is converted into a grayscale image so that the brightness information of each pixel is represented by a grayscale value. The calculation formula is as follows:

[0041] G(x,y)=0.299·I R (x,y)+0.587·I G (x,y)+0.114·I B (x,y); where I R (x,y),I G (x,y) and I B (x, y) are the values ​​of the red, green, and blue channels of the corresponding pixels in the original image.

[0042] Furthermore, the gray value of each pixel in the preprocessed video data is marked with a brightness threshold through a preset binary mask to obtain a mask parameter.

[0043] In one embodiment, next, a binary mask is defined to mark which pixels have grayscale values ​​within a set brightness range. The mask M(x, y) is defined as follows: Where, L min and L max is the brightness threshold, and the value of the present invention is (200, 255).

[0044] Furthermore, adaptive histogram equalization (CLAHE) is used to divide the image into multiple small blocks, and the grayscale distribution of each small block is equalized to enhance the contrast of the local area. Get the image gray value G after enhancement based on adaptive histogram equalization technology enhanced (x, y). Where G(x, y) is the grayscale value of the original image in the preprocessed video data. s (k′) is the histogram frequency of the gray level K of the sth small block; L clip This is the clipping limit, which is used to prevent areas of excessive contrast enhancement.

[0045] Furthermore, the area close to white is enhanced and merged with the full image to obtain the final image. The final processed video is recorded as (Enhanced Video Data), i.e., according to I final (x,y)=M(x,y)·G enhanced (x,y)+(1-M(x,y))·I(x,y), get the final pixel value I of the synthesized image final(x, y). Where M(x, y) is the mask parameter; I(x, y) is the image pixel value in the preprocessed video data. Then, based on the final image pixel value, the enhanced video data is determined.

[0046] S103: Perform a linear fitting process on the lane boundary line region in the enhanced video data to obtain a lane boundary line position feature.

[0047] Specifically, through the preset deep learning model U-Net and based on the activation function, the lane area in the enhanced video data is subjected to image region cutting processing to obtain a lane area map.

[0048] Furthermore, coordinates of the middle point set in the lane area map are calculated to obtain a coordinate set of the middle points in the lane area.

[0049] In one embodiment, first, the lane area is segmented using the deep learning model U-Net, and the calculation formula is as follows: O = σ(Decoder(Concatenate(Encoder(I), skip connections))); where I is the input video The image size is ; O is the output lane area map; σ is the sigmoid activation function. Then calculate the middle point set M of the lane area, that is, the coordinate set of the middle point in the lane area, and the calculation formula is as follows:

[0050] Furthermore, for the segmented lane area map O, the lane boundary line is fitted in the lane boundary line area through Hough transform to generate the position parameters of the lane boundary line, that is, according to Get the lane boundary line position feature L fit Among them, (x i ,y i ) is the coordinate of the middle point in the lane area coordinate set M, and the lane boundary line position feature L fit To fit a straight line function, for example, L fit =y=mx+b;, where m is the slope of the fitted straight line function, and b is the constant of the fitted straight line function.

[0051] S104: According to the lane boundary line position features and based on the perspective transformation matrix, each frame of the image is geometrically corrected and spatially aligned to generate final video data.

[0052] Specifically, according to the two L fit , we can calculate the corresponding x coordinate when y=0 or y=y2-y1, and get the source coordinate H before the perspective transformation of each frame of the picture, that is, according to The source coordinates H before perspective transformation of each frame image in the enhanced video data are obtained. Among them, m1 is the slope of the first lane boundary line in the lane boundary line position feature, b1 is the constant of the first lane boundary line, and (x1, y1) is the coordinate of the first lane boundary line; m2 is the slope of the second lane boundary line in the lane boundary line position feature, b2 is the constant of the second lane boundary line, and (x2, y2) is the coordinate of the second lane boundary line; the lane boundary line position feature includes: the first lane boundary line and the second lane boundary line.

[0053] Further, the source coordinates are converted into expected coordinates based on the pixel width of the ROI region of interest in the enhanced video data.

[0054] In one embodiment, the transformed expected coordinate H′ is calculated based on the obtained source coordinate. The calculation formula is as follows: Where: y2-y1 is the pixel width of the ROI region of interest.

[0055] Furthermore, according to eight key points preset in the lane boundary line, corresponding coordinates are extracted for the expected coordinates to obtain eight key coordinate points.

[0056] Furthermore, after obtaining eight important key coordinate points of the lane edge, the eight points are substituted into the following equation to solve the perspective transformation moment. Get the perspective transformation matrix. Among them, is the perspective transformation matrix (3×3); homogeneous coordinates (x, y, 1) Τ is the source coordinate of each frame image; homogeneous coordinate (x′, y′, w′) Τ is the expected coordinate of each frame image; w′=h 31 x+h 32 y+h 33 is the normalization factor.

[0057] Furthermore, the eight key coordinate points are input into the perspective transformation matrix to calculate the perspective transformation coordinates.

[0058] Further, according to

[0059] The perspective transformation ROI region of interest ROI′ is obtained. Among them, m3 is the slope of the first lane boundary line after geometric correction and spatial alignment, b3 is the constant of the first lane boundary line, and (x3, y3) is the perspective transformation coordinate of the first lane boundary line; m4 is the slope of the second lane boundary line after geometric correction and spatial alignment, b4 is the constant of the second lane boundary line, and (x4, y4) is the perspective transformation coordinate of the second lane boundary line.

[0060] Furthermore, based on the perspective transformation ROI region of interest, each frame of the enhanced video data is geometrically corrected and spatially aligned, and the pixel size of the extracted image is The final video data is obtained, denoted as

[0061] S105 , performing recognition processing between the vehicle and the dashed white segment of the lane separator line on the final video data to determine vehicle trajectory data.

[0062] Specifically, Figure 2 A schematic diagram of a dashed white segment of a lane separator provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, through the preset YOLO model, all vehicles and all lane separators in the final video data are identified, and labels and detection box information are generated for the vehicles and lane separators.

[0063] Furthermore, according to the preset DeepSORT algorithm, and based on the label and detection box information, all vehicles and all lane separator dashed white segments are tracked, and vehicle trajectory data is generated.

[0064] S106, estimating the vehicle speed in the vehicle trajectory data by using a dynamic and static frame difference method to obtain the actual vehicle speed.

[0065] Specifically, according to Get the dynamic relative speed of each vehicle in, is the speed of the i-th vehicle in the k-th frame image based on the vehicle trajectory data in the final video data; is the displacement pixel distance of the i-th vehicle from the k-th frame to the k+1-th frame image based on the vehicle trajectory data; N FPS The number of frames per second of the final video data.

[0066] Further, according to Get the average static relative velocity in, is the average speed of all lane separator dashed white segments in the kth frame image based on the vehicle trajectory data in the final video data; is the number of dashed white segments of the lane separator in the k-th frame image; is the displacement pixel distance of the jth lane separator dashed line white segment in the image from the kth frame to the k+1th frame based on the vehicle trajectory data.

[0067] Further, according to

[0068] Get pixel estimation speed in, is the estimated speed of the pixels of vehicle i in the kth frame based on the vehicle trajectory data; Drone↑Vehicle means that the vehicle and the drone are moving in the same direction; Drone↓Vehicle means that the vehicle and the drone are moving in opposite directions.

[0069] Further, according to Get the actual speed of the vehicle in, represents the actual estimated speed of vehicle i in the kth frame based on the vehicle trajectory data; Indicates the actual length of the dashed white segment of the lane divider on the road; represents the pixel distance of lane separator line j detected in the kth frame based on the vehicle trajectory data.

[0070] As a feasible implementation method, the technical solution of the present invention can solve the limitations of existing traffic speed measurement technology that relies on fixed equipment, coordinate systems and complex sensor data, and can achieve high-precision vehicle speed estimation through video image processing technology without relying on external coordinate information or dynamic data of drones and vehicles. Its core technologies include: using a group of drones to carry out mobile aerial photography of the high-speed road section to be measured, and collecting video data during vehicle driving; extracting and analyzing lane line features in the video, and detecting and extracting lane lines through deep learning algorithms (such as U-Net) to obtain lane separation information; using perspective transformation technology to correct video images, eliminating the effects caused by drone vibration or flight angle changes, and ensuring accurate alignment of image space; finally, using YOLO and DeepSORT algorithms to detect and track target vehicles in the video, and estimating the actual dynamic speed based on the static relative speed of the vehicle. This method does not need to rely on traditional radar speed measurement equipment or ground sensors, can achieve flexible speed measurement applications, eliminate environmental interference, improve speed measurement accuracy, and has strong adaptability and low hardware cost.

[0071] In addition, the embodiment of the present application also provides a vehicle speed estimation device based on drone aerial photography, such as Figure 3 As shown, the vehicle speed estimation device 300 based on drone aerial photography specifically includes:

[0072] At least one processor 301; and a memory 302 in communication with the at least one processor 301; wherein the memory 302 stores instructions executable by the at least one processor 301, so that the at least one processor 301 can execute:

[0073] Perform interest region extraction and image noise reduction preprocessing on the video data collected by drone aerial photography to obtain preprocessed video data;

[0074] According to the adaptive histogram equalization technology, the image pixel values ​​in the pre-processed video data are enhanced to obtain enhanced video data;

[0075] Performing linear fitting processing on the lane boundary line area in the enhanced video data to obtain the lane boundary line position feature;

[0076] According to the position characteristics of the lane boundary line and based on the perspective transformation matrix, each frame of the image is geometrically corrected and spatially aligned to generate the final video data;

[0077] The final video data is processed to identify the vehicle and the dashed white segment of the lane separator to determine the vehicle trajectory data;

[0078] The vehicle speed in the vehicle trajectory data is estimated by the dynamic and static frame difference method to obtain the actual vehicle speed.

[0079] The embodiments of the present application allow the UAV to accurately measure speed at a variety of height and angle combinations. Stable speed measurement results can be obtained by simply ensuring that the parameters of a single video acquisition are consistent. This feature greatly broadens its scope of application, and is particularly suitable for traffic monitoring and emergency tasks in rapid deployment, complex scenarios, or resource-constrained environments. Not only does it significantly improve the anti-interference ability of the system, but it also enables the speed measurement process to remain stable in complex environments, fully meeting the application requirements of high real-time, low cost, and high reliability, and opening up a new path for data collection and processing in UAV aerial photography scenarios. In addition, vehicle speed estimation is achieved entirely based on video images, without the need to rely on the flight speed, geographic coordinate system, or other external sensor data support of the UAV, which fundamentally simplifies the complexity of hardware and software integration and significantly reduces implementation costs.

[0080] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0081] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0082] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0083] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0084] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0085] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0086] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0087] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present application should be included in the scope of the claims of the present application.

Claims

1. A vehicle speed estimation method based on drone aerial photography, characterized in that: The method comprises: Perform interest region extraction and image noise reduction preprocessing on the video data collected by drone aerial photography to obtain preprocessed video data; According to the adaptive histogram equalization technology, the image pixel values ​​in the pre-processed video data are enhanced to obtain enhanced video data; Performing a linear fitting process on the lane boundary line region in the enhanced video data to obtain a lane boundary line position feature; According to the lane boundary line position features and based on the perspective transformation matrix, each frame of the image is geometrically corrected and spatially aligned to generate final video data; Performing identification processing between the vehicle and the dashed white segment of the lane separator line on the final video data to determine vehicle trajectory data; The vehicle speed in the vehicle trajectory data is estimated by using a dynamic and static frame difference method to obtain the actual vehicle speed.

2. The vehicle speed estimation method based on drone aerial photography according to claim 1 is characterized in that: The video data collected by drone aerial photography is preprocessed by extracting the region of interest and reducing image noise to obtain preprocessed video data, including: Using the drone, the speed measurement section is photographed by mobile aerial photography to collect and obtain first video data; According to the four point coordinates of the preset area of ​​interest, the first video data is subjected to an extraction process of the area of ​​interest to obtain second video data; according to The image value I′(x) after denoising of the second video data is obtained; wherein I(x) and I(y) are the pixel values ​​at pixel x and pixel y, respectively; Ω is a set of all pixels in the image and represents the area where similarity is calculated in the entire image; h is a smoothing parameter used to control the sensitivity of similarity; ||I(x)-I(y)|| 2 is the Euclidean distance between pixel x and pixel y, which is used to measure their similarity; e is a mathematical symbol; The preprocessed video data is obtained again based on the image value after noise reduction of the second video data.

3. The vehicle speed estimation method based on drone aerial photography according to claim 1 is characterized in that: According to the adaptive histogram equalization technology, the image pixel values ​​in the pre-processed video data are enhanced to obtain enhanced video data, specifically including: Converting the color image in the preprocessed video data into a grayscale image and expressing the brightness information of each pixel in grayscale; Through a preset binary mask, a brightness threshold is marked on the gray value of each pixel in the preprocessed video data to obtain a mask parameter; according to The image grayscale value G enhanced based on the adaptive histogram equalization technology is obtained. enhanced (x, y); wherein G(x, y) is the grayscale value of the original image in the preprocessed video data; H s (k′) is the histogram frequency of the gray level K of the sth small block; L clip It is a clipping limit, which is used to prevent the contrast of an area from being over-enhanced; According to I final (x,y)=M(x,y)·G enhanced (x,y)+(1-M(x,y))·I(x,y), get the final pixel value I of the synthesized image final (x, y); wherein M(x, y) is a mask parameter; I(x, y) is an image pixel value in the preprocessed video data; The enhanced video data is determined based on the final image pixel values.

4. The vehicle speed estimation method based on drone aerial photography according to claim 1 is characterized in that: The lane boundary line region in the enhanced video data is subjected to a linear fitting process related to the lane boundary line to obtain the lane boundary line position feature, specifically including: By using a preset deep learning model U-Net and based on an activation function, an image region cutting process is performed on the lane region in the enhanced video data to obtain a lane region map; Calculating the coordinates of the middle point set in the lane area map to obtain the middle point coordinate set of the lane area; according to Get the lane boundary line position feature L fit ; Among them, (x i ,y i ) is the coordinate of the lane area midpoint coordinate set M, and the lane boundary line position feature L fit is the fitting straight line function, m is the slope of the fitting straight line function, and b is the constant of the fitting straight line function.

5. The vehicle speed estimation method based on drone aerial photography according to claim 1 is characterized in that: Before performing geometric correction and spatial alignment processing on each frame of image according to the lane boundary line position features and based on the perspective transformation matrix to generate final video data, the method further includes: according to Obtaining the source coordinates H of each frame image before perspective transformation in the enhanced video data; wherein m1 is the slope of the first lane boundary line in the lane boundary line position feature, b1 is the constant of the first lane boundary line, and (x1, y1) is the coordinate of the first lane boundary line; m2 is the slope of the second lane boundary line in the lane boundary line position feature, b2 is the constant of the second lane boundary line, and (x2, y2) is the coordinate of the second lane boundary line; the lane boundary line position feature includes: the first lane boundary line and the second lane boundary line; Based on the pixel width of the ROI region of interest in the enhanced video data, converting the source coordinates into desired coordinates; According to eight key points preset in the lane boundary line, corresponding coordinates are extracted for the expected coordinates to obtain eight key coordinate points; according to Get the perspective transformation matrix; where, is the perspective transformation matrix (3×3); homogeneous coordinates (x, y, 1) Τ is the source coordinate of each frame image; homogeneous coordinate (x′, y′, w′) Τ is the expected coordinate of each frame image; w′=h 31 x+h 32 y+h 33 is the normalization factor; The eight key coordinate points are input into the perspective transformation matrix to calculate the perspective transformation coordinates.

6. The vehicle speed estimation method based on drone aerial photography according to claim 5 is characterized in that: According to the lane boundary line position features and based on the perspective transformation matrix, each frame of the image is geometrically corrected and spatially aligned to generate the final video data, specifically including: according to Obtain the perspective transformed ROI region of interest ROI′; wherein, m3 is the slope of the first lane boundary line after geometric correction and spatial alignment, b3 is the constant of the first lane boundary line, and (x3, y3) is the perspective transformed coordinate of the first lane boundary line; m4 is the slope of the second lane boundary line after geometric correction and spatial alignment, b4 is the constant of the second lane boundary line, and (x4, y4) is the perspective transformed coordinate of the second lane boundary line; Based on the perspective transformation ROI region of interest, geometric correction and spatial alignment processing are performed on each frame of the enhanced video data to obtain the final video data.

7. The vehicle speed estimation method based on drone aerial photography according to claim 1 is characterized in that: The final video data is processed to identify the vehicle and the dashed white segment of the lane separator line to determine the vehicle trajectory data, specifically including: Using a preset YOLO model, all vehicles and all dashed white segments of lane dividers in the final video data are identified, and labels and detection box information are generated for the vehicles and dashed white segments of lane dividers; According to the preset DeepSORT algorithm, and based on the label and the detection frame information, all the vehicles and all the dashed white segments of the lane separators are tracked, and the vehicle trajectory data is generated.

8. The vehicle speed estimation method based on drone aerial photography according to claim 1 is characterized in that: The vehicle speed in the vehicle trajectory data is estimated by a dynamic and static frame difference method to obtain the actual vehicle speed, specifically including: according to Get the dynamic relative speed of each vehicle in, is the speed of the i-th vehicle in the k-th frame image based on the vehicle trajectory data in the final video data; is the displacement pixel distance of the i-th vehicle from the k-th frame to the k+1-th frame image based on the vehicle trajectory data; N FPS The number of frames per second of the final video data; according to Get the average static relative velocity in, is the average speed of all dashed white segments of lane separators in the kth frame image based on the vehicle trajectory data in the final video data; is the number of dashed white segments of the lane separator in the k-th frame image; is the displacement pixel distance of the jth lane separator dashed line white segment in the kth frame to the k+1th frame image based on the vehicle trajectory data; according to Get pixel estimation speed in, is the estimated speed of the pixels of vehicle i in the kth frame based on the vehicle trajectory data; Drone↑Vehicle means that the vehicle and the drone are traveling in the same direction; Drone↓Vehicle means that the vehicle and the drone are traveling in opposite directions; according to Get the actual speed of the vehicle in, represents the actual estimated speed of vehicle i in the kth frame based on the vehicle trajectory data; Indicates the actual length of the dashed white segment of the lane divider on the road; represents the pixel distance of lane separation line j detected in the kth frame based on the vehicle trajectory data.

9. A vehicle speed estimation device based on drone aerial photography, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute the vehicle speed estimation method based on drone aerial photography according to any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that: The storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions. When the instructions are executed by the terminal, the terminal executes the vehicle speed estimation method based on drone aerial photography according to any one of claims 1-8.

Images