A vehicle speed estimation method, device and medium based on unmanned aerial vehicle aerial photography

Abstract

The application discloses a vehicle speed estimation method and device based on unmanned aerial vehicle aerial photography and a medium, belongs to the technical field of vehicle speed estimation, and is used for solving the technical problems that the existing unmanned aerial vehicle vehicle speed estimation method needs complicated positioning and calibration to normally calculate vehicle speed, cost is high, and calculation error is large. The method comprises the following steps: extracting an interesting region from video data collected by unmanned aerial vehicle aerial photography and performing image noise reduction preprocessing to obtain preprocessed video data; performing enhancement processing on image pixel values in the preprocessed video data to obtain enhanced video data; performing straight line fitting processing on a lane boundary line region in the enhanced video data to obtain lane boundary line position characteristics; performing geometric correction and spatial alignment processing on each frame of image; performing identification processing on the final video data between a vehicle and a dashed white segment of a lane separation line; and performing dynamic and static frame difference method estimation on vehicle speed to obtain actual vehicle speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicle speed estimation, and in particular to a vehicle speed estimation method based on aerial photography by a UAV, a device and a medium. BACKGROUND

[0002] Methods for estimating vehicle speed by a UAV mainly include methods based on image processing and methods based on GPS / IMU data. The method based on image processing usually uses aerial photography by a UAV to obtain original traffic images, then performs preprocessing, detects moving vehicles using the continuous image subtraction method, and extracts lane lines to determine the lane range. By taking the lane width as a scale, the speed of the moving vehicle is estimated.

[0003] The method based on GPS data: through the position information provided by GPS, the position change at different time points is calculated, and then the speed is obtained by dividing the time interval. This method has high accuracy in open areas, but is greatly affected by GPS signals, and the accuracy and efficiency may be affected in environments such as urban canyons or forests.

[0004] The method based on inertial measurement unit (IMU): using accelerometer and gyroscope data, the speed is calculated by integration. This method does not depend on external signals and can be used in any environment, and is suitable for fast dynamic changes, but has large cumulative error and needs to be corrected.

[0005] The existing UAV for detecting vehicle speed has high requirements for the UAV, such as flight height and shooting angle, and is also prone to flight jitter, resulting in large image error and video picture offset problems. At the same time, due to the complexity of the hardware and software of the UAV, it is difficult to apply to relatively complex scenes, so it is difficult to accurately calculate the actual speed of the moving vehicle. SUMMARY

[0006] The embodiments of the present application provide a vehicle speed estimation method based on aerial photography by a UAV, a device and a medium, which are used to solve the following technical problems: the existing method for estimating vehicle speed by a UAV relies on high-performance UAVs, and needs tedious positioning and calibration to normally calculate the vehicle speed, which has high cost and large calculation error.

[0007] The embodiments of the present application adopt the following technical solutions:

[0008] On one hand, this application provides a vehicle speed estimation method based on UAV aerial photography, including: performing region of interest extraction and image noise reduction preprocessing on video data collected by UAV aerial photography to obtain preprocessed video data; enhancing the image pixel values ​​in the preprocessed video data according to adaptive histogram equalization technology to obtain enhanced video data; performing straight line fitting processing on the lane boundary line region in the enhanced video data to obtain lane boundary line position features; performing geometric correction and spatial alignment processing on each frame of the image based on the lane boundary line position features and a perspective transformation matrix to generate final video data; performing recognition processing between the vehicle and the dashed white segment of the lane divider line on the final video data to determine vehicle trajectory data; and estimating the vehicle speed in the vehicle trajectory data using the dynamic and static frame difference method to obtain the actual vehicle speed.

[0009] This application's embodiments allow UAVs to achieve accurate speed measurement under various combinations of altitude and angle. Stable speed measurement results can be obtained simply by ensuring consistent parameters in a single video segment. This feature greatly expands its application scope, making it particularly suitable for traffic monitoring and emergency tasks in rapidly deployable, complex scenarios, or resource-constrained environments. It not only significantly improves the system's anti-interference capability but also ensures stable speed measurement even in complex environments, fully meeting the application requirements of high real-time performance, low cost, and high reliability, thus opening up new pathways for data acquisition and processing in UAV aerial photography scenarios. Furthermore, vehicle speed estimation is entirely based on video images, without relying on UAV flight speed, geographic coordinates, or other external sensor data, fundamentally simplifying the integration complexity of hardware and software and significantly reducing implementation costs.

[0010] In one feasible implementation, the video data collected by drone aerial photography undergoes region of interest extraction and image noise reduction preprocessing to obtain preprocessed video data. Specifically, this includes: using the drone to perform mobile aerial photography of a speed measurement section, collecting and obtaining first video data; extracting the region of interest from the first video data based on the coordinates of four points of a preset region of interest to obtain second video data; and further processing the extraction based on the region of interest. The image value I′(x) after denoising the second video data is obtained; where I(x) and I(y) are the pixel values ​​at pixel x and pixel y, respectively; Ω is the set of all pixels in the image and represents the region for calculating similarity in the entire image; h is a smoothing parameter used to control the sensitivity of similarity; ||I(x)-I(y)|| 2 is the Euclidean distance between pixels x and y, used to measure their similarity; e is a mathematical symbol; based on the image values ​​after denoising the second video data, the preprocessed video data is obtained again.

[0011] In an implementable embodiment, the image pixel values in the pre-processed video data are enhanced according to an adaptive histogram equalization technique to obtain enhanced video data, specifically including: converting the color images in the pre-processed video data into gray-scale images, and representing the brightness information of each pixel in gray scale; marking the gray-scale values of each pixel in the pre-processed video data with a brightness threshold value through a pre-set binary mask to obtain a mask parameter; according to obtaining the gray-scale values of the images enhanced based on the adaptive histogram equalization technique G enhanced (x,y); wherein G(x,y) is the gray-scale value of the original image in the pre-processed video data; H s (k') is the histogram frequency of the gray-scale level K of the s-th small block; L clip is a clipping limit for preventing excessive enhancement of the contrast of the region; according to I final (x,y) = M(x,y)·G enhanced (x,y) + (1-M(x,y))·I(x,y), to obtain the final image pixel value I final (x,y) after synthesis; wherein M(x,y) is the mask parameter; I(x,y) is the image pixel value in the pre-processed video data; based on the final image pixel value, the enhanced video data is determined.

[0012] In an implementable embodiment, the lane boundary line region in the enhanced video data is subjected to a straight line fitting process related to the lane boundary line to obtain lane boundary line position features, specifically including: performing image region cutting on the lane region in the enhanced video data through a pre-set deep learning model U-Net and based on an activation function to obtain a lane region graph; performing coordinate calculation on the intermediate point set in the lane region graph to obtain a lane region intermediate point coordinate set; according to obtaining the lane boundary line position features L fit ; wherein (x i ,y i ) is the coordinate in the lane region intermediate point coordinate set M, and the lane boundary line position features L fit is a fitting straight line function, and m is the slope of the fitting straight line function, and b is the constant of the fitting straight line function.

[0013] In an implementable embodiment, before performing geometric correction and spatial alignment on each frame of image according to the lane boundary line position features and based on a perspective transformation matrix to generate final video data, the method further includes: according to

[0014] obtaining source coordinates H of each frame image in the enhanced video data before perspective transformation; wherein, m1 is a slope of a first lane boundary straight line in the lane boundary line position feature, b1 is a constant of the first lane boundary straight line, and (x1, y1) is a coordinate of the first lane boundary straight line; m2 is a slope of a second lane boundary straight line in the lane boundary line position feature, b2 is a constant of the second lane boundary straight line, and (x2, y2) is a coordinate of the second lane boundary straight line; the lane boundary line position feature comprises the first lane boundary straight line and the second lane boundary straight line; converting the source coordinates into expected coordinates based on a pixel width of a region of interest (ROI) in the enhanced video data; extracting corresponding coordinates of the expected coordinates according to eight preset key points in the lane boundary line to obtain eight key coordinate points; and obtaining the perspective transformation matrix according to the eight key coordinate points. obtaining the perspective transformation matrix; wherein, is the perspective transformation matrix (3x3); and homogeneous coordinates (x, y, 1) Τ is the source coordinates of each frame image; and homogeneous coordinates (x', y', w') Τ is the expected coordinates of each frame image; and w' = h 31 x + h 32 y + h 33 is a normalization factor; and inputting the eight key coordinate points into the perspective transformation matrix to calculate perspective transformation coordinates.

[0015] In a feasible implementation, according to the lane boundary line position feature and based on the perspective transformation matrix, geometric correction and spatial alignment processing are performed on each frame image to generate final video data, specifically comprising: according to

[0016] obtaining a perspective transformation region of interest (ROI) ROI'; wherein, m3 is a slope of a first lane boundary straight line after geometric correction and spatial alignment, b3 is a constant of the first lane boundary straight line, and (x3, y3) is a perspective transformation coordinate of the first lane boundary straight line; m4 is a slope of a second lane boundary straight line after geometric correction and spatial alignment, b4 is a constant of the second lane boundary straight line, and (x4, y4) is a perspective transformation coordinate of the second lane boundary straight line; and performing geometric correction and spatial alignment processing on each frame image in the enhanced video data based on the perspective transformation ROI to obtain the final video data.

[0017] In an implementable embodiment, the final video data is subjected to a recognition process between a vehicle and a white dashed line segment of a lane divider, and vehicle trajectory data is determined, specifically including: all vehicles and all white dashed line segments of lane dividers in the final video data are recognized through a preset YOLO model, and the vehicles and the white dashed line segments of lane dividers are both subjected to label marking and detection box information generation; according to a preset DeepSORT algorithm and based on the labels and the detection box information, all the vehicles and all the white dashed line segments of lane dividers are tracked, and the vehicle trajectory data is generated.

[0018] In an implementable embodiment, the speed of a vehicle in the vehicle trajectory data is estimated by a dynamic and static frame difference method to obtain a vehicle actual speed, specifically including: according to a dynamic relative speed of each vehicle is obtained wherein, is a speed of an i-th vehicle in a k-th image based on the vehicle trajectory data in the final video data; is a displacement pixel distance of the i-th vehicle based on the vehicle trajectory data in a k-th image to a k+1-th image; N FPS is a frame number per second of a video of the final video data; according to an average static relative speed is obtained wherein, is an average speed of all white dashed line segments of lane dividers in a k-th image based on the vehicle trajectory data in the final video data; is a number of white dashed line segments of lane dividers in the k-th image; is a displacement pixel distance of a j-th white dashed line segment of lane divider based on the vehicle trajectory data in the k-th image to the k+1-th image; according to

[0019] a pixel estimation speed is obtained wherein, is a pixel estimation speed of a vehicle i in a k-th image based on the vehicle trajectory data; Drone↑Vehicle represents that a vehicle and a drone have the same driving direction; Drone↓Vehicle represents that a vehicle and a drone have opposite driving directions; according to the vehicle actual speed is obtained wherein, represents an actual estimation speed of a vehicle i in a k-th image based on the vehicle trajectory data; represents an actual length of the white dashed line segment of lane divider on the road; represents a pixel distance of detecting a lane divider j in a k-th image based on the vehicle trajectory data.

[0020] In a second aspect, the embodiments of the present application also provide a vehicle speed estimation device based on aerial photography of a UAV, the device comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions capable of being executed by the at least one processor, so as to enable the at least one processor to execute the vehicle speed estimation method based on aerial photography of a UAV according to any of the embodiments described above.

[0021] In a third aspect, the embodiments of the present application also provide a non-volatile computer storage medium, characterized in that the storage medium is a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores at least one program, each of the programs comprising instructions which, when executed by a terminal, cause the terminal to execute the vehicle speed estimation method based on aerial photography of a UAV according to any of the embodiments described above.

[0022] The present application provides a vehicle speed estimation method, device and medium based on aerial photography of a UAV, compared with the prior art, the embodiments of the present application have the following beneficial technical effects:

[0023] (1) In the vehicle speed measurement of the aerial photography video of the UAV, the present application does not have strict requirements on the flight height and the shooting angle, but needs to maintain the stability during the single video acquisition period and does not change in the middle. This design significantly reduces the dependence on the hardware performance and the control accuracy of the UAV, and provides higher flexibility for the adaptability of the speed measurement system. Unlike the traditional method which strictly depends on fixed flight parameters, the present application allows accurate speed measurement under various height and angle combinations, as long as the single video acquisition parameters are consistent, stable speed measurement results can be obtained. This feature greatly widens the application range, and is especially suitable for traffic monitoring and emergency tasks in fast deployment, complex scenes or resource-limited environments.

[0024] (2) The present application can effectively eliminate the video picture offset problem caused by shaking, vibration and wind interference in the flight process of the UAV through automatic image correction technology, so as to ensure the accuracy and reliability of the speed measurement process. Unlike the traditional method which depends on mechanical stabilization equipment or complex post-processing, the present application automatically extracts and corrects the offset of the picture track by using image analysis algorithm, so as to realize high-precision restoration of target motion information. This design not only significantly improves the anti-interference ability of the system, but also enables the speed measurement process to remain stable in complex environments, fully meets the application requirements of high real-time, low cost and high reliability, and opens up a new path for data acquisition and processing in the aerial photography scene of the UAV.

[0025] (3)In addition, the application realizes vehicle speed estimation based on video images, without the support of flight speed, geographic coordinate system or other external sensor data of the unmanned aerial vehicle, fundamentally simplifies the integration complexity of hardware and software, and significantly reduces the implementation cost. The algorithm directly calculates the target motion speed through the information between video frames, which improves the independence and efficiency of the system while reducing the deployment difficulty. The highly automated design enables the technology to be quickly applied to complex multi-target scenes, and the technology can run efficiently without tedious parameter calibration, which is particularly suitable for traffic monitoring, disaster emergency and other scenes that require flexible deployment and rapid response. The above characteristics not only improve the practical value of the application, but also provide important support for the further development of unmanned aerial vehicle assisted traffic monitoring technology. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0027] Figure 1 A flow chart of a vehicle speed estimation method based on aerial photography of an unmanned aerial vehicle is provided for the embodiments of the present application.

[0028] Figure 2 A dashed white segment schematic diagram of a lane separation line is provided for the embodiments of the present application.

[0029] Figure 3 A structural schematic diagram of a vehicle speed estimation device based on aerial photography of an unmanned aerial vehicle is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0031] The embodiments of the present application provide a vehicle speed estimation method based on aerial photography of an unmanned aerial vehicle, as shown in Figure 1 The vehicle speed estimation method based on aerial photography of an unmanned aerial vehicle specifically includes steps S101-S106:

[0032] S101, interest region extraction and image noise reduction preprocessing are performed on video data collected by the unmanned aerial vehicle, and preprocessed video data is obtained.

[0033] Specifically, the unmanned aerial vehicle moves and takes pictures on the speed measurement section to collect and obtain first video data.

[0034] Further, according to the four point coordinates of the preset region of interest, the first video data is extracted and processed to obtain second video data.

[0035] In one embodiment, a basic road section on a highway is selected, the unmanned aerial vehicle moves and takes pictures on the speed measurement section, and the collected video is denoted as (first video data). During the shooting process, the flight height, angle and route of the unmanned aerial vehicle should be kept stable, and the initial height, angle and route direction are not limited, as long as the road can be clearly shot. The video is extracted for the region of interest, and the extracted video is denoted as (second video data), and the ROI calculation formula is as follows: ROI={(x1,y1),(x2,y1),(x2,y2),(x1,y2)}; in the formula, ROI is the extracted region of interest; (x1,y1)~(x1,y2) are four point coordinates of the region of interest.

[0036] Further, the video is subjected to image noise reduction processing, and the video obtained after noise reduction is denoted as (preprocessed video data). The non-local mean (Non-Local Means, NLM) is used to reduce the noise of each pixel x to obtain the image value I'(x), that is, the image value I'(x) of the second video data after noise reduction is obtained according to . Wherein, I(x) and I(y) are pixel values at pixels x and y respectively; Ω is a set of all pixels in the image, and represents a region for calculating similarity in the entire image. h is a smoothing parameter for controlling the sensitivity of similarity; ||I(x)-I(y)|| 2 is the Euclidean distance between pixels x and y, which is used to measure their similarity; e is a mathematical symbol.

[0037] Further, based on the image value after noise reduction of the second video data, the preprocessed video data is reobtained.

[0038] S102, according to the adaptive histogram equalization technique, the image pixel value in the preprocessed video data is enhanced to obtain enhanced video data.

[0039] Specifically, the color images in the preprocessed video data are converted to grayscale, and the brightness information of each pixel is represented in grayscale.

[0040] In one embodiment, the input color image is first converted to a grayscale image, so that the brightness information of each pixel is represented by a grayscale value. The calculation formula is as follows:

[0041] G(x,y)=0.299·I R (x,y)+0.587·I G (x,y)+0.114·I B (x,y); where I R (x,y), I G (x,y) and I B (x, y) are the values ​​of the red, green, and blue channels of the corresponding pixels in the original image, respectively.

[0042] Furthermore, by using a preset binary mask, the grayscale value of each pixel in the preprocessed video data is marked with a brightness threshold to obtain the mask parameters.

[0043] In one embodiment, a binary mask is then defined to mark which pixels have grayscale values ​​within a defined brightness range. The mask M(x,y) is defined as follows: In the formula, L min and L max The brightness threshold is (200, 255) in this invention.

[0044] Furthermore, adaptive histogram equalization (CLAHE) is used to divide the image into multiple small blocks, and the gray-level distribution of each block is equalized to enhance the contrast of local areas. This means that it is necessary to adjust the gray-level distribution according to the specific conditions of the blocks. The image grayscale value G is obtained after enhancement by adaptive histogram equalization. enhanced (x,y). Where G(x,y) is the grayscale value of the original image in the preprocessed video data. H s (k′) is the histogram frequency of the gray level K of the s-th small block; L clip This is a clipping limit used to prevent excessive contrast enhancement in certain areas.

[0045] Furthermore, areas approaching white are enhanced, and the images are merged based on the entire image to obtain the final image. The final processed video is denoted as... (Enhanced video data), i.e., based on I final (x,y)=M(x,y)·G enhanced (x,y)+(1-M(x,y))·I(x,y) yields the final pixel value I of the synthesized image. final(x,y). Wherein, M(x,y) is a mask parameter; I(x,y) is an image pixel value in the pre-processed video data. Then, based on the final image pixel value, the enhanced video data is determined.

[0046] S103, a straight line fitting process about the lane boundary line is performed on the lane boundary line region in the enhanced video data, to obtain lane boundary line position characteristics.

[0047] Specifically, by a preset deep learning model U-Net, and based on an activation function, an image region segmentation process is performed on the lane region in the enhanced video data, to obtain a lane region graph.

[0048] Further, a coordinate calculation is performed on a middle point set in the lane region graph, to obtain a lane region middle point coordinate set.

[0049] In one embodiment, first, a deep learning model U-Net is used for lane region segmentation, and the calculation formula is as follows: O = σ (Decoder (Concatenate (Encoder (I), skip connections))) ; in the formula, I is an image size of an input video ; O is an output lane region graph; and σ is a sigmoid activation function. Then, a lane region middle point set M, that is, a lane region middle point coordinate set, is calculated, and the calculation formula is as follows:

[0050] Further, for the segmented lane region graph O, a Hough transformation is used to fit the lane boundary line in the lane boundary line region, to generate a position parameter of the lane boundary line, that is, to obtain lane boundary line position characteristics L fit according to i (x i ,y fit ) is a coordinate in the lane region middle point coordinate set M, and the lane boundary line position characteristics L fit is a fitting straight line function, for example, L fit =y Τ mx+b; m is a slope of the fitting straight line function, and b is a constant of the fitting straight line function.

[0051] S104, according to the lane boundary line position characteristics, and based on a perspective transformation matrix, a geometric correction and spatial alignment process is performed on each frame of image, to generate final video data.

[0052] Specifically, according to the two obtained L fit , a corresponding x coordinate when y = 0 or y = y2-y1 can be calculated, to obtain a source coordinate H of each frame of picture before perspective transformation, that is, according to The source coordinates H before perspective transformation of each image in the enhanced video data are obtained. Wherein, m1 is the slope of the first lane boundary straight line in the lane boundary line position feature, b1 is the constant of the first lane boundary straight line, and (x1, y1) is the coordinate of the first lane boundary straight line; m2 is the slope of the second lane boundary straight line in the lane boundary line position feature, b2 is the constant of the second lane boundary straight line, and (x2, y2) is the coordinate of the second lane boundary straight line; the lane boundary line position feature comprises the first lane boundary straight line and the second lane boundary straight line.

[0053] Further, the source coordinates are converted into the expected coordinates based on the pixel width of the ROI region of interest in the enhanced video data.

[0054] In one embodiment, the converted expected coordinates H' are calculated according to the obtained source coordinates. The calculation formula is as follows: In the formula, y2-y1 is the pixel width of the ROI region of interest.

[0055] Further, the corresponding coordinates of the expected coordinates are extracted according to the eight key points preset in the lane boundary line, to obtain eight key coordinate points.

[0056] Further, after the eight important key coordinate points of the lane edge are obtained, the eight points are brought into the following equation to solve the perspective transformation matrix, according to The perspective transformation matrix is obtained. Wherein, is the perspective transformation matrix (3x3); the homogeneous coordinates (x, y, 1) Τ is the source coordinates of each image; the homogeneous coordinates (x', y', w') Τ is the expected coordinates of each image; w'=h 31 x+h 32 y+h 33 is the normalization factor.

[0057] Further, the eight key coordinate points are input into the perspective transformation matrix, to calculate the perspective transformation coordinates.

[0058] Further, according to

[0059] The perspective transformation ROI region of interest ROI' is obtained. Wherein, m3 is the slope of the first lane boundary straight line after the geometric correction and spatial alignment, b3 is the constant of the first lane boundary straight line, and (x3, y3) is the perspective transformation coordinate of the first lane boundary straight line; m4 is the slope of the second lane boundary straight line after the geometric correction and spatial alignment, b4 is the constant of the second lane boundary straight line, and (x4, y4) is the perspective transformation coordinate of the second lane boundary straight line.

[0060] Further, based on the perspective transformation ROI region of interest, the geometric correction and spatial alignment processing are performed on each frame image in the enhanced video data, and the pixel size of the extracted image is The final video data is obtained, denoted as

[0061] S105, the recognition processing between the vehicle and the white segment of the dashed line of the lane separation line is performed on the final video data, and the vehicle trajectory data is determined.

[0062] Specifically, Figure 2 A lane separation line white segment schematic diagram provided by the embodiment of the application is shown in FIG. 6. Figure 2 As shown in the figure, all vehicles and all white segments of the dashed line of the lane separation line in the final video data are recognized by the preset YOLO model, and the vehicles and the white segments of the dashed line of the lane separation line are both labeled and detected frame information is generated.

[0063] Further, according to the preset DeepSORT algorithm, and based on the label and the detection frame information, all vehicles and all white segments of the dashed line of the lane separation line are tracked, and the vehicle trajectory data is generated.

[0064] S106, the dynamic and static frame difference method is used to estimate the vehicle speed in the vehicle trajectory data, and the actual vehicle speed is obtained.

[0065] Specifically, according to The dynamic relative speed of each vehicle is obtained Wherein, is the speed of the i-th vehicle in the k-th image based on the vehicle trajectory data in the final video data; is the displacement pixel distance of the i-th vehicle based on the k-th image to the k+1-th image in the vehicle trajectory data; N FPS is the frame number per second of the video of the final video data.

[0066] Further, according to The average static relative speed is obtained Wherein, is the average speed of all white segments of the dashed line of the lane separation line in the k-th image based on the vehicle trajectory data in the final video data; is the number of white segments of the dashed line of the lane separation line in the k-th image; is the displacement pixel distance of the j-th white segment of the dashed line of the lane separation line based on the k-th image to the k+1-th image in the vehicle trajectory data.

[0067] Further, according to

[0068] The pixel estimated speed is obtained Wherein, estimating the speed of the kth frame of vehicle i based on vehicle trajectory data; Drone↑Vehicle represents that the vehicle and the drone have the same driving direction; Drone↓Vehicle represents that the vehicle and the drone have opposite driving directions.

[0069] Further, according to obtaining the actual speed of the vehicle wherein, represents the actual estimated speed of the kth frame of vehicle i based on vehicle trajectory data; represents the actual length of the white segment of the dashed line of the lane separation line on the road; represents the pixel distance of detecting the lane separation line j based on the kth frame of vehicle trajectory data.

[0070] As a feasible implementation, the technical scheme of the present application can solve the limitations of relying on fixed equipment, coordinate system and complex sensor data in existing traffic speed measurement technology, and can realize high-precision vehicle speed estimation through video image processing technology without relying on external coordinate information or dynamic data of the drone and the vehicle. The core technology includes: a group of drones are used to take aerial photos of the to-be-measured highway section, and video data of the vehicle driving process is collected; lane line features are extracted and analyzed in the video, lane line detection and extraction are performed through a deep learning algorithm (such as U-Net), and lane separation information is obtained; perspective transformation technology is used to correct the video image, eliminate the influence caused by drone vibration or flight angle change, and ensure the accurate alignment of the image space; finally, the target vehicle in the video is detected and tracked through YOLO and DeepSORT algorithms, and the actual dynamic speed is estimated according to the static relative speed of the vehicle. This method does not rely on traditional radar speed measurement equipment or ground sensors, can realize flexible speed measurement application, eliminate environmental interference, improve speed measurement accuracy, has strong adaptability and low hardware cost.

[0071] In addition, the present application also provides a vehicle speed estimation device based on aerial photography of a drone, as shown in Figure 3 The vehicle speed estimation device based on aerial photography of a drone 300 specifically includes:

[0072] at least one processor 301; and a memory 302 communicatively connected with the at least one processor 301; wherein the memory 302 stores instructions executable by the at least one processor 301, so that the at least one processor 301 can perform:

[0073] extracting the region of interest from the video data collected by the aerial photography of the drone and performing image noise reduction preprocessing to obtain preprocessed video data;

[0074] According to the adaptive histogram equalization technology, the image pixel values in the pre-processed video data are enhanced to obtain enhanced video data;

[0075] The lane boundary line region in the enhanced video data is subjected to a straight line fitting process related to the lane boundary line to obtain lane boundary line position features;

[0076] According to the lane boundary line position features and based on a perspective transformation matrix, each frame of image is subjected to geometric correction and spatial alignment to generate final video data;

[0077] The vehicle trajectory data is determined by performing a recognition process between the vehicle and the white dashed section of the lane divider in the final video data;

[0078] The vehicle speed in the vehicle trajectory data is estimated by using a dynamic and static frame difference method to obtain the actual vehicle speed.

[0079] The embodiments of the present application allow the unmanned aerial vehicle to achieve accurate speed measurement at various height and angle combinations. As long as the single video acquisition parameters are consistent, stable speed measurement results can be obtained. This feature greatly widens the application range, and is particularly suitable for traffic monitoring and emergency tasks in fast deployment, complex scenes or resource-limited environments. Not only does it significantly improve the anti-interference ability of the system, but also enables the speed measurement process to remain stable in complex environments, fully meeting the application requirements of high real-time performance, low cost and high reliability, opening up a new path for data acquisition and processing in unmanned aerial vehicle aerial scenes. Moreover, the vehicle speed is estimated based entirely on video images, without the need for support from the flight speed of the unmanned aerial vehicle, the geographic coordinate system or other external sensor data, which fundamentally simplifies the integration complexity of hardware and software and significantly reduces the implementation cost.

[0080] Each of the embodiments in the present application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0081] The device and medium provided by the embodiments of the present application are one-to-one corresponding to the method, and therefore, the device and medium also have similar beneficial technical effects to the method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device and medium will not be described here.

[0082] Those skilled in the art will appreciate that embodiments of the present application can be readily used as software, hardware, or a combination of software and hardware. In one

[0083] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of Figure 1 The functions of the various elements shown in the figures can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Figure 1 The various elements shown in the figures can be described using any number of patents, patent applications, specifications, or other documents.

[0084] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the processor modifies. The memory is an example of computer readable media.

[0085] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0086] The above-described embodiments of the application have several aspects, no single one of which is solely responsible for the application's desirable attributes. Without limiting the scope of this application, other aspects of the application will become apparent from consideration of the drawings and following detailed description, it being understood that such changes in the details are within the scope of this application. It is to be understood that the phraseology or terminology herein is for the purpose of description and not of limitation, such that the terminology or phraseology of the present specification is to be interpreted by those skilled in the art in light of the teachings and

[0087] The embodiments of the application described hereinabove are intended to be merely exemplary and those skilled in the art will recognize variations from the described embodiments that fall within the scope of the application. Those variations are intended to be included within the scope of claims de- fined herein as well as the scope of equivalents to those claims.

Claims

1. A method for estimating vehicle speed based on aerial photography by a UAV, characterized in that, The method comprises: The video data collected by the unmanned aerial vehicle aerial photography is extracted and preprocessed to obtain preprocessed video data; According to the adaptive histogram equalization technology, the image pixel value in the preprocessed video data is enhanced to obtain enhanced video data; The lane boundary line region in the enhanced video data is processed by straight line fitting related to the lane boundary line to obtain the lane boundary line position feature; According to the lane boundary line position feature and based on the perspective transformation matrix, each frame of image is geometrically corrected and spatially aligned to generate final video data; The vehicle trajectory data is determined by identifying the vehicle and the white segment between the lane separation lines in the final video data; The vehicle speed in the vehicle trajectory data is estimated by the dynamic and static frame difference method to obtain the actual vehicle speed, specifically including: According to , the dynamic relative speed of each vehicle is obtained ; wherein, is the speed of the i-th vehicle in the k-th image based on the vehicle trajectory data in the final video data; is the displacement pixel distance of the i-th vehicle from the k-th image to the k+1-th image based on the vehicle trajectory data; is the frame number per second of the video of the final video data; According to , the average static relative speed ; wherein, is the average speed of all lane marking dashed white segments in the kth frame of image based on the vehicle trajectory data in the final video data; is the number of lane marking dashed white segments in the kth frame of image; is the displacement pixel distance of the jth lane marking dashed white segment from the kth frame to the k+1th frame of image based on the vehicle trajectory data; According to , the pixel estimation speed of the vehicle i in the kth frame is obtained; wherein, is the pixel estimation speed of the vehicle i in the kth frame based on the vehicle trajectory data; indicates that the vehicle and the UAV have the same driving direction; indicates that the vehicle and the UAV have opposite driving directions; According to , the actual speed of the vehicle ; wherein, represents the actual estimated speed of the vehicle i in the kth frame based on the vehicle trajectory data; represents the actual length of the white dashed segment of the lane marking on the road; represents the pixel distance of the detected lane marking j based on the vehicle trajectory data in the kth frame. 2.The vehicle speed estimation method based on UAV aerial photography according to claim 1, wherein, The video data collected by the unmanned aerial vehicle aerial photography is extracted and preprocessed to obtain preprocessed video data, specifically including: The first video data is collected by moving aerial photography of the speed measurement section by the unmanned aerial vehicle; According to the four point coordinates of the preset region of interest, the first video data is extracted and processed to obtain the second video data; According to , the image value of the second video data after noise reduction ; wherein, and are pixel values at pixel x and pixel y, respectively; is a set of all pixels in the image, and represents a region for calculating similarity in the entire image; is a smoothing parameter for controlling the sensitivity of similarity; is the Euclidean distance between pixel x and pixel y, used to measure their similarity; is a mathematical symbol; Based on the image value of the second video data after noise reduction, the preprocessed video data is obtained again. 3.The vehicle speed estimation method based on UAV aerial photography according to claim 1, wherein, According to the adaptive histogram equalization technology, the image pixel value in the preprocessed video data is enhanced to obtain enhanced video data, specifically including: The color image in the preprocessed video data is converted into a gray image, and the brightness information of each pixel is represented in gray scale; The gray value of each pixel in the preprocessed video data is marked by a preset binary mask to obtain a mask parameter; According to , the image gray value enhanced based on the adaptive histogram equalization technique ; wherein, is the gray value of the original image in the pre-processed video data; is the histogram frequency of the gray level K of the s-th small block; is the clipping limit, used to prevent the contrast of the region from being excessively enhanced; According to , the final image pixel value after synthesis is obtained ; wherein, is a mask parameter; is an image pixel value in the preprocessed video data; Based on the final image pixel value, the enhanced video data is determined. 4.The vehicle speed estimation method based on UAV aerial photography according to claim 1, wherein, The lane boundary line region in the enhanced video data is processed by straight line fitting related to the lane boundary line to obtain the lane boundary line position feature, specifically including: The lane region in the enhanced video data is cut by a preset deep learning model U-Net based on an activation function to obtain a lane region graph; The coordinates of the middle point set in the lane region graph are calculated to obtain the lane region middle point coordinate set; According to , the lane boundary line position feature is obtained; wherein, is a coordinate in the lane region middle point coordinate set , the lane boundary line position feature is a fitting straight line function, and m is the slope of the fitting straight line function, and b is the constant of the fitting straight line function. 5.The vehicle speed estimation method based on UAV aerial photography according to claim 1, wherein, Before the lane boundary line position feature and based on the perspective transformation matrix, each frame of image is geometrically corrected and spatially aligned to generate final video data, the method further comprises: According to , the source coordinates of each frame image in the enhanced video data before perspective transformation are obtained ; wherein m1 is the slope of the first lane boundary straight line in the lane boundary line position feature, b1 is the constant of the first lane boundary straight line, and (x1, y1) is the coordinate of the first lane boundary straight line; m2 is the slope of the second lane boundary straight line in the lane boundary line position feature, b2 is the constant of the second lane boundary straight line, and (x2, y2) is the coordinate of the second lane boundary straight line; the lane boundary line position feature comprises the first lane boundary straight line and the second lane boundary straight line; Based on the enhanced video data ROI The pixel width of the region of interest is used to convert the source coordinates into desired coordinates; According to the eight key points in the lane boundary line, the corresponding coordinates of the expected coordinates are extracted to obtain eight key coordinate points; According to , the perspective transformation matrix is obtained; wherein, is the perspective transformation matrix ; homogeneous coordinates is the source coordinates of each frame of image; homogeneous coordinates is the desired coordinates of each frame of image; is a normalization factor; The eight key coordinate points are input into the perspective transformation matrix to calculate the perspective transformation coordinates. 6.The vehicle speed estimation method based on UAV aerial photography according to claim 5, wherein, According to the lane boundary line position feature and based on the perspective transformation matrix, each frame of image is geometrically corrected and spatially aligned to generate final video data, specifically including: According to , a perspective transformation ROI region of interest is obtained ; wherein m3 is the slope of the first lane boundary straight line after geometric correction and spatial alignment, b3 is the constant of the first lane boundary straight line, and (x3, y3) is the perspective transformation coordinates of the first lane boundary straight line; m4 is the slope of the second lane boundary straight line after geometric correction and spatial alignment, b4 is the constant of the second lane boundary straight line, and (x4, y4) is the perspective transformation coordinates of the second lane boundary straight line; Based on the perspective transformation ROI region of interest, each frame image in the enhanced video data is geometrically corrected and spatially aligned to obtain the final video data. 7.The vehicle speed estimation method based on UAV aerial photography according to claim 1, wherein, The final video data is subjected to recognition processing between a vehicle and a white dashed line segment of a lane marking line to determine vehicle trajectory data, specifically including: All vehicles and all white dashed line segments of lane marking lines in the final video data are recognized by a preset YOLO model, and the vehicles and the white dashed line segments of lane marking lines are both subjected to label marking and detection box information generation; According to a preset DeepSORT algorithm and based on the labels and the detection box information, all the vehicles and all the white dashed line segments of lane marking lines are tracked, and the vehicle trajectory data is generated.

8. An apparatus for estimating a speed of a vehicle based on aerial photography by an unmanned aerial vehicle, characterized by comprising: The device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein The memory stores instructions executable by the at least one processor to enable the at least one processor to perform a vehicle speed estimation method based on aerial photography by a UAV according to any one of claims 1-7.

9. A non-transitory computer storage medium, comprising: The storage medium is a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores at least one program, and each program includes instructions which, when executed by a terminal, cause the terminal to perform a vehicle speed estimation method based on aerial photography by a UAV according to any one of claims 1-7.

Images