Method, device, electronic device and medium for collecting road images by drone

Abstract

This application provides a method, device, electronic device, and medium for capturing road images using a drone, relating to the field of drones. By analyzing the curbs of roads in images, this application can calculate the actual road width and achieve precise measurement by combining drone flight parameters and camera parameters. Based on the actual road width, a mathematical model is used to adaptively adjust the camera zoom factor to optimize the shooting effect and ensure the optimal proportion of the road in the image and the required accuracy. This process efficiently integrates drone flight, image acquisition and processing, and precise measurement technologies, providing an innovative solution for urban road inspections.

Description

Technical Field

[0001] The present application relates to the field of drones, and more specifically, to a method, device, electronic device, and medium for collecting road images using a drone. Background Art

[0002] With the acceleration of global urbanization, urban road networks are becoming increasingly complex, posing unprecedented challenges to urban management, maintenance, and traffic optimization. Traditional urban road inspection methods, including manual inspections and vehicle-based equipment inspections, are no longer able to meet the needs of modern urban management and traffic planning due to their inefficiency and limited coverage. Unmanned aerial vehicles (UAVs), due to their high maneuverability, flexibility, and freedom from ground-based transportation constraints, are becoming a new option for urban road inspections.

[0003] Existing drone systems face problems in data collection, such as difficulty in adjusting camera shooting angles and focal lengths, insufficient adaptability, and weak ability to handle environmental interference. These problems result in low data collection quality and make it difficult to meet the diverse needs of urban road inspections. Summary of the Invention

[0004] The purpose of the embodiments of the present application is to provide a method, device, electronic device and medium for collecting road images by a drone, so as to solve the above-mentioned problems existing in the prior art, and to adaptively adjust the camera zoom ratio to optimize the shooting effect.

[0005] In a first aspect, a method for collecting road images by a drone is provided, which may include:

[0006] Processing a current image captured by the drone to determine edge images corresponding to each road in the current image; the current image is an image captured by the drone based on current camera parameters of multiple roads in the target area;

[0007] Using a line detection algorithm to perform feature extraction on the edge image, determine position data of different curbs involved in each road in the edge image, and determine coordinate data of the position data of the different curbs in an image coordinate system of the current image;

[0008] For any roadside, determining a coordinate adjustment angle based on the coordinate data corresponding to the roadside and the edge direction of the current image, and adjusting the coordinate data based on the coordinate adjustment angle to obtain target coordinate data;

[0009] Determining, based on the target coordinate data of each roadside, the number of pixels from the roadside center of the corresponding roadside to the image center of the current image;

[0010] Determining a zoom factor corresponding to each roadside based on the number of pixels corresponding to each roadside and the current camera parameters;

[0011] Based on the configured screening mechanism, each zoom factor is screened to determine the target zoom factor, and the camera of the UAV is controlled to zoom according to the target zoom factor to perform image acquisition.

[0012] In one possible implementation, the current camera parameters include original focal length, sensor size, number of sensor pixels, and shooting distance;

[0013] Determine the zoom factor corresponding to the corresponding curb, including:

[0014] For any roadside, a ground sampling distance is determined according to the shooting distance, the sensor size, the original focal length, and the number of sensor pixels; the ground sampling distance represents the ground size corresponding to a single pixel;

[0015] Based on the relationship between the ground sampling distance and the target ground sampling distance corresponding to each roadside after zooming, the zoom factor corresponding to the corresponding roadside is determined.

[0016] In a possible implementation, determining the ground sampling distance according to the shooting distance, the sensor size, the original focal length, and the number of sensor pixels includes:

[0017] Determining a first parameter based on the sensor size and the number of sensor pixels, wherein the first parameter represents the sensor size corresponding to a single pixel;

[0018] The ground sampling distance corresponding to any roadside is determined based on the first parameter, the shooting distance and the original focal length.

[0019] In a possible implementation, after determining the ground sampling distance, the method further includes:

[0020] The horizontal distance of the camera image of the UAV is determined based on the ground sampling distance and the number of image pixels in the horizontal direction of the current image.

[0021] In a possible implementation, the different curbs include: a first curb and a second curb;

[0022] Determining a zoom factor corresponding to each roadside based on a relationship between the ground sampling distance and a target ground sampling distance corresponding to each roadside after zooming includes:

[0023] Converting the relationship between the ground sampling distance and the target ground sampling distance corresponding to each road edge after zooming into the relationship between the horizontal distance and the first target distance and the second target distance, respectively; the first target distance represents the distance from the center of the first road edge to the center of the image, and the second target distance represents the distance from the center of the second road edge to the center of the image;

[0024] Converting the relationships between the horizontal distance and the first target distance and the second target distance, respectively, into relationships between the number of image pixels and the first number of pixels from the center point coordinate of the first road edge to the center of the image and the second number of pixels from the center point coordinate of the second road edge to the center of the image;

[0025] A first zoom factor of a first edge and a second zoom factor of a second edge are determined based on relationships between the number of image pixels and the first number of pixels and the second number of pixels, respectively.

[0026] In a possible implementation, the screening mechanism of the configuration is: determining the minimum value among the zoom factors as the target zoom factor.

[0027] In a possible implementation, after determining the coordinate adjustment angle, the method further includes:

[0028] Based on the coordinate adjustment angle, the camera gimbal of the UAV is adjusted to obtain a target angle, so that the camera of the UAV performs image acquisition according to the target angle.

[0029] In a second aspect, a device for collecting road images using a drone is provided, and the device may include:

[0030] a processing unit configured to process a current image captured by the drone and determine an edge image corresponding to each road in the current image; the current image is an image captured by the drone based on current camera parameters of multiple roads within a target area;

[0031] a determining unit, configured to extract features from the edge image using a line detection algorithm, determine position data of different road edges involved in each road in the edge image, and determine coordinate data of the position data of the different road edges in an image coordinate system of the current image;

[0032] and, for any roadside, determining a coordinate adjustment angle based on the coordinate data corresponding to the roadside and the edge direction of the current image, and adjusting the coordinate data based on the coordinate adjustment angle to obtain target coordinate data;

[0033] and, based on the target coordinate data of each roadside, determining the number of pixels from the roadside center of the corresponding roadside to the image center of the current image;

[0034] and, determining a zoom factor corresponding to each roadside based on the number of pixels corresponding to each roadside and the current camera parameters;

[0035] Furthermore, based on the configured screening mechanism, each zoom factor is screened to determine a target zoom factor, and the camera of the drone is controlled to capture images after zooming according to the target zoom factor.

[0036] In a third aspect, an electronic device is provided, the electronic device including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0037] Memory for storing computer programs;

[0038] The processor is configured to implement any of the method steps described in the first aspect when executing a program stored in the memory.

[0039] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the method steps described in the first aspect is implemented.

[0040] The present application provides a method for collecting road images using a drone, the method comprising: processing a current image collected by a drone to determine an edge image corresponding to each road in the current image; using a line detection algorithm to extract features from the edge image, determining the position data of different roadside edges involved in each road in the edge image, and determining the coordinate data of the position data of the different roadside edges in the image coordinate system of the current image; for any roadside edge, determining a coordinate adjustment angle based on the coordinate data corresponding to the roadside edge and the edge direction of the current image, and adjusting the coordinate data based on the coordinate adjustment angle to obtain target coordinate data; based on the target coordinate data of each roadside edge, determining the number of pixels from the roadside center of the corresponding roadside edge to the image center of the current image; based on the number of pixels corresponding to each roadside edge and current camera parameters, determining a zoom factor corresponding to the corresponding roadside edge; based on a configured screening mechanism, screening each zoom factor to determine a target zoom factor, and controlling the drone's camera to zoom according to the target zoom factor before performing image collection. The present application can calculate the actual road width by analyzing the roadside edges in the image, and achieve accurate measurement by combining drone flight parameters and camera parameters. Based on the actual road width, a mathematical model is used to adaptively adjust the camera zoom factor, optimizing the capture effect and ensuring the optimal proportion of the road within the frame and the required accuracy. This process efficiently integrates drone flight, image acquisition and processing, and precise measurement technologies, providing an innovative solution for urban road inspections. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 A system architecture diagram of a method for collecting road images using a drone, provided in an embodiment of the present application;

[0043] Figure 2 A flowchart of a method for collecting road images using a drone provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the overall process of collecting road images by a drone provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of the function of a drone collecting road images provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the structure of a device for collecting road images using a drone provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0049] The method for collecting road images by a drone provided in the embodiment of the present application can be applied to Figure 1 In the system architecture shown in Figure 1 As shown, the system may include: a drone and a processor.

[0050] The drone is used to collect images of multiple roads in a target area based on current camera parameters, obtain a current image, and send the current image to a processor.

[0051] The processor is used to receive the current image to execute a method for collecting road images using a drone provided in this application.

[0052] Although drones have great potential for application in urban environments, current technological implementation still faces many challenges. Specifically, in terms of data collection, existing drone systems generally have the following problems:

[0053] Difficulty adjusting the camera's shooting angle and focal length: Existing drone cameras are usually designed with fixed settings or limited adjustment capabilities. They cannot dynamically adjust the shooting angle and focal length according to actual road conditions, resulting in blurred images and loss of edge information.

[0054] Lack of adaptability: Different inspection tasks have different requirements for image quality and viewing angles. For example, identifying road cracks and potholes requires high-resolution, close-up images, while overall assessment of large road structures requires a wider perspective. However, current drone systems lack sufficient intelligence to automatically adapt to these changing requirements.

[0055] Weak ability to handle environmental interference factors: Complex lighting conditions, weather changes and other factors in urban environments can also affect the quality of drone data collection. Existing systems often fail to fully consider these issues, resulting in inaccurate and unreliable collection results.

[0056] Therefore, the present application provides a method for collecting road images using a drone, which solves the above-mentioned problems existing in the prior art and can adaptively adjust the camera zoom ratio to optimize the shooting effect.

[0057] Explanation of the terms involved in this application:

[0058] A three-dimensional flight path is a flight path planned for a drone in three-dimensional space (including longitude, latitude, and altitude), consisting of a series of waypoints connected by specific height, speed, and heading parameters. This route not only takes into account the road's horizontal orientation but also factors such as the surrounding terrain and building heights. This ensures that drones can conduct comprehensive and efficient inspections of urban roads from various heights and angles, avoiding blind spots caused by terrain or obstacles. For example, when inspecting mountainous roads, the flight path can adjust its altitude based on the mountain's contours. In urban areas with densely populated buildings, the flight altitude can be adjusted to the height of the buildings to obtain clear road images.

[0059] Machine vision is a technical field that enables machines to understand image or video data. The machine vision system in this application primarily includes a camera, image sensor, image processor, and related image processing algorithms. Through these components and algorithms, the machine vision system can analyze and process road images captured by the camera, such as image preprocessing (denoising, enhancement, etc.), target detection and recognition (identifying roads, curbs, lane markings, etc.), thereby providing key information for subsequent road width measurement, zoom adjustment, and data collection and analysis. For example, after training a large number of annotated road images using a deep learning algorithm, the machine vision system can accurately identify various road elements in the image and extract their feature information for further analysis and decision-making.

[0060] Ground Sampling Distance (GSD): refers to the actual ground size corresponding to a pixel on the image, and the unit is usually centimeters / pixel or meters / pixel, etc. In the drone urban road inspection of this application, GSD is an important indicator to measure the spatial resolution of the image. It is closely related to parameters such as the focal length, shooting distance and sensor size of the camera. For example, a smaller GSD means that the image can distinguish finer ground details, and tiny cracks in the road surface, wear of marking lines, etc. can be more clearly detected during road inspections; while a larger GSD can obtain a wider field of view, but some detailed information may be lost. By adjusting the zoom factor of the camera, the flight altitude of the drone and other operations, the size of the GSD can be changed to adapt to different inspection needs.

[0061] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application can be combined with each other if there is no conflict.

[0062] Figure 2 This is a flow chart of a method for collecting road images using a drone provided in an embodiment of the present application. Figure 2 As shown, the method may include:

[0063] Step S210: Process the current image captured by the drone to determine the edge images corresponding to each road in the current image.

[0064] Among them, the current image is an image collected by the drone based on the current camera parameters of multiple roads in the target area.

[0065] It should be noted that the drone is equipped with a camera and a camera gimbal;

[0066] The camera gimbal enhances shooting stability, maintaining a stable camera orientation even when the drone is swaying in the wind or performing maneuvers. It also expands the camera's shooting angles, adjusting the camera's orientation independently of the drone's body for more flexible viewing angles. The gimbal's anti-shake function reduces blur, making it particularly useful for low-altitude, high-speed flight scenarios.

[0067] Before executing step S210, Figure 3 As shown, the UAV's flight path (3D route) is configured in three-dimensional space based on the specific requirements of the flight mission and the geographical characteristics of the target area. This route will specify flight parameters such as the UAV's flight speed, altitude, the locations of multiple key waypoints, and the actions the camera gimbal should perform at each location. This ensures that the target area to be inspected and the data collection standards are met, thereby achieving the desired mission results.

[0068] Afterwards, at the configured takeoff position, the drone will perform a series of self-checks and communication equipment checks before takeoff, ensuring that all communication data is transmitted normally and that key parameters, such as the drone and its onboard camera, are in normal condition. Furthermore, all installed equipment on the drone must be initialized and configured, including but not limited to adjusting the camera to the configured camera parameters and activating the machine vision system. This ensures that all onboard equipment functions efficiently and accurately during the drone's mission.

[0069] Step S210 specifically includes: the drone takes off according to the flight route, flies to the starting point (target area) of the key waypoint, and performs data collection. The camera adjusts to shoot directly below according to the command action. The image collected by the drone is transmitted back to the processor in real time via the communication transmission module. The machine vision system configured in the processor performs real-time analysis of the current image collected. The analysis process includes:

[0070] It should be noted that the current image may include one or more roads in the target area.

[0071] 1. Process the current image to obtain a grayscale image: Use the cv.cvtColor function to assign different weights to the red, green, and blue channels through the weighted average method to convert the current image into a grayscale image, and convert the three-channel RGB image into a single-channel image. The generally used weight formula is:

[0072]

[0073] Among them, Y is the grayscale value, R, G, and B are the values ​​corresponding to the red, green, and blue channels respectively.

[0074] 2. Filter the grayscale image to obtain a filtered image: perform weighted averaging on the grayscale values ​​of the pixel to be filtered and its neighboring points in the grayscale image according to the parameter rule generated by the Gaussian formula:

[0075]

[0076] in, is convolution, is the filtered image after filtering, The standard deviation is Gaussian kernel, It can be expressed as: ; where x and y are pixel points The position offset relative to the center point of the Gaussian kernel.

[0077] 3. Extract edge information from the filtered image: Use the Sobel filter to perform convolution on the filtered image and calculate the gradient components in the horizontal and vertical directions respectively. Specifically, the Sobel operator consists of two 3x3 convolution kernels:

[0078] A convolution kernel is used to calculate the horizontal gradient component in the horizontal direction ; ;

[0079] Another convolution kernel is used to calculate the vertical gradient component in the vertical direction ; ;

[0080] Based on the horizontal gradient component and the vertical gradient component , determine the gradient amplitude G and gradient direction θ; the process can be expressed as: ; ;

[0081] After obtaining the gradient magnitude and direction, non-maximum suppression is used to find the local maximum in the gradient direction of each pixel, retaining only these maximum points to refine edge information. In this process, sharp edges are directly identified and retained; less obvious edges must be attached to strong edges, meaning they are only retained if they are continuously connected to strong edges. This effectively filters out more accurate and important edge details, enhancing the effectiveness of subsequent processing.

[0082] In some embodiments, edge regions are identified based on a grayscale threshold. These edge regions can be considered regions of interest (ROIs). Specifically, if the grayscale value of a pixel in the filtered image is above (or below) the set threshold, the pixel's position is marked as 1 in the corresponding mask, indicating that it is part of the ROI (i.e., the area to be analyzed or processed, typically displayed as white). Conversely, if the pixel's grayscale value does not meet the threshold, the pixel's position is marked as 0 in the mask, indicating that the pixel is not part of the ROI (i.e., considered part of the background, typically displayed as black). For locations marked as 1 in the mask, the corresponding pixel values ​​in the current image are retained, indicating that these pixels are part of the ROI. For locations marked as 0 in the mask, the corresponding pixel values ​​in the current image are set to 0 (or another background value), thereby removing non-critical areas. In this way, specific regions of interest can be accurately extracted from the current image for further analysis or processing.

[0083] Furthermore, the grayscale threshold determination process may include:

[0084] The grayscale threshold is determined based on the distribution of gradient magnitudes and the grayscale histogram corresponding to the filtered image. Specifically, the grayscale histogram shows the distribution of pixels at different grayscale values ​​in the image. Typically, the valley between two peaks in the grayscale histogram is selected as the threshold. If the grayscale distribution is relatively continuous or has a lot of overlap, the distribution of gradient magnitudes is statistically analyzed to identify regions with high gradient magnitudes. If certain grayscale value regions have high gradient magnitudes simultaneously, the regions corresponding to these grayscale values ​​are considered key regions.

[0085] Step S220: Use a line detection algorithm to extract features from the edge image, determine the position data of different curbs involved in each road in the edge image, and determine the coordinate data of the position data of different curbs in the image coordinate system of the current image.

[0086] Specifically, a line detection algorithm, such as the Hough transform, is used to identify potential straight lines in the edge image. Since road edges often appear as straight or nearly straight lines in a top-down view, the Hough transform can convert the lines in the edge image into a parameterized form (e.g., defining the slope and intercept of the line), allowing these lines to be calibrated in the parameter space. Next, by identifying peak regions in the parameter space, the specific parameters of these lines are determined, successfully extracting the lines from the image.

[0087] Since the Hough transform may detect some pseudo straight lines caused by noise or interference, it is necessary to screen and verify the extracted straight lines based on the road geometry and prior knowledge.

[0088] Specifically, short line segments (which may be noise or the edges of irrelevant objects) are excluded based on the configured length threshold. That is, line segments that do not meet the length threshold are excluded. The length threshold configuration must take into account the actual application scenario (such as road width, camera viewing angle, etc.) and the impact of noise.

[0089] Since curbs are relatively fixed, we can determine whether a straight line meets the characteristics of a curb based on the road geometry and prior knowledge. For example, curbs are usually located on both sides of the road and have a certain inclination angle or height difference.

[0090] You can verify whether a line is a curb by analyzing its position, direction, and relationship to other lines.

[0091] In this method, the Hough transform is used to extract line features from the filtered image. This effectively maps lines in image space to parameter space and determines the line parameters by detecting peaks. Feature screening and verification eliminates the influence of noise and interference by using a length threshold and combining prior knowledge of the road (such as the location and shape of curbs). Ultimately, the line segments belonging to the curbs are determined, thereby determining the location data of different curbs. It is understandable that since curbs are on both sides of the road, any road should have two curbs: a primary curb and a secondary curb.

[0092] Afterwards, based on the position data of the line segments corresponding to different road edges, the coordinate data of the corresponding line segments in the current image are determined: the coordinate data of different road edges in the current image can be determined by determining the coordinates of the two end points corresponding to different road edges. The specific method is to mark the coordinate data of the line segments corresponding to different road edges in the image coordinate system. The coordinate data includes: the starting point coordinates ( ) and the end point coordinates , thereby defining the specific range of the line segments corresponding to different road edges on the current image. This positioning method provides accurate position data for subsequent operations.

[0093] The image coordinate system can take the upper left corner of the camera's image screen as the origin (0,0), the horizontal right direction of the screen is the x-axis direction, and the vertical downward direction is the y-axis direction. The coordinate value is in pixels, and the image resolution is W×H.

[0094] In some embodiments, after determining the position data of the roadside, a tracking model is constructed to achieve real-time tracking. Assuming that the roadside maintains uniform linear motion over a short period of time (assuming the drone's flight speed is constant and the road conditions do not change suddenly, the position of the roadside relative to the drone remains unchanged), the roadside's position at the next moment can be predicted based on this model.

[0095] Step S230: For any roadside, determine a coordinate adjustment angle based on the coordinate data corresponding to the roadside and the edge direction of the current image, and adjust the coordinate data based on the coordinate adjustment angle to obtain target coordinate data.

[0096] Specifically, for any road edge, calculate the slope of the line segment corresponding to the road edge :

[0097]

[0098] Since, in an ideal state, the line segment corresponding to the road edge is parallel to the Y axis and has a slope of 0, in practice, an allowable error range is set. ,set up =0.05, if , it means that the line segment corresponding to the curb is not parallel to the Y axis and needs to be adjusted. , no adjustment is required.

[0099] The adjustment process may include:

[0100] Determine the coordinate adjustment angle based on the starting and ending coordinates of the roadside ;

[0101]

[0102] Based on the coordinate adjustment angle, the starting point coordinates and the end point coordinates are adjusted to obtain target coordinate data, which includes: target starting point coordinates and target end point coordinates.

[0103] In some embodiments, since the road image is collected by adjusting the camera's shooting angle by rotating the camera's pan-tilt platform on the X axis, the adjustment angle of the camera's pan-tilt platform on the X axis is It is approximately equal to the coordinate adjustment angle (Pay attention to direction judgment, if 0, indicating that the road edge has a rising trend from left to right, and the camera gimbal should rotate counterclockwise around the X axis; if 0, then it should rotate clockwise around the X axis).

[0104] Furthermore, if the line segment corresponding to the first road is in a continuous line segment state, the target starting point coordinates of the line segment corresponding to the first road are recorded. and the target endpoint coordinates ;

[0105] If the curve shape is composed of multiple discrete points, select the key pixel coordinate set to indicate its approximate location.

[0106] Similarly, if the line segment corresponding to the second road is a continuous line segment, the target starting point coordinates of the line segment corresponding to the second road are recorded. and the target endpoint coordinates ;

[0107] If the curve shape is composed of multiple discrete points, select the key pixel coordinate set to indicate its approximate location.

[0108] Step S240: Based on the target coordinate data of each roadside, determine the number of pixels from the roadside center of the corresponding roadside to the image center of the current image, and based on the number of pixels corresponding to each roadside and the current camera parameters, determine the zoom factor corresponding to the corresponding roadside.

[0109] Specifically, 1. Based on the target coordinate data of each roadside, the horizontal center coordinates of each roadside are determined:

[0110]

[0111]

[0112] in, is the center coordinate of the first edge of the first edge in the horizontal direction; is the center coordinate of the second edge in the horizontal direction.

[0113] 2. Based on the number of image pixels in the horizontal direction of the current image , the first roadside center coordinates and the second roadside center coordinates, determine the number of pixels from the roadside center coordinates of each roadside to the image center of the current image, the number of pixels includes: the first pixel number corresponding to the first roadside ; The second number of pixels corresponding to the second road edge .

[0114]

[0115]

[0116] 3. Current camera parameters include original focal length (unit: mm), sensor size d (unit: mm), number of sensor pixels n (unit: pixels) and shooting distance H (unit: meters);

[0117] The first parameter s is determined based on the sensor size and the number of sensor pixels. The first parameter represents the sensor size corresponding to a single pixel:

[0118] (Unit: mm / pixel)

[0119] 4. According to the principle of similar triangles, when the camera does not change the focal length, determine the ground sampling distance corresponding to any road edge based on the first parameter, shooting distance and original focal length .

[0120]

[0121] Assuming the zoom factor is Z, the target focal length after zooming is When the first path along the center coordinate When adjusted to the leftmost side of the screen (horizontally 0), the width of the curb in the screen becomes pixels (assuming the curb width remains unchanged in the screen). At this time, the target ground sampling distance It can be expressed as:

[0122]

[0123] Among them, the target ground sampling distance Includes the first target ground sampling distance and the second target sampling distance ;

[0124]

[0125] 5. Based on ground sampling distance and the number of image pixels in the horizontal direction of the current image , determine the horizontal distance of the drone's camera image , the process can be expressed as:

[0126]

[0127] And, it can also be based on the first target ground sampling distance assuming that the zoom factor is Z , second target sampling distance and the number of image pixels , respectively determine the first target distance and the second target distance , the first target distance represents the distance from the center of the first roadside to the center of the image; the second target distance represents the distance from the center of the second roadside to the center of the image; the expression of this process is:

[0128]

[0129]

[0130] 6. Determine the zoom factor corresponding to the corresponding roadside based on the relationship between the ground sampling distance and the sub-ground sampling distance corresponding to each roadside after zooming;

[0131] Specifically, the first zoom factor corresponding to the first edge ;

[0132] The second zoom factor corresponding to the second edge: ;

[0133] because , , ; Therefore, the ratio of the target focal length to the original focal length can be equivalent to the ratio of the ground sampling distance to the target ground sampling distance corresponding to the corresponding road edge:

[0134] Right now: ;

[0135] ;

[0136] because , , Therefore, the ratio of the ground sampling distance to the target ground sampling distance corresponding to the corresponding roadside is equivalent to the ratio of the horizontal distance to the corresponding target distance (the first target distance or the second target distance);

[0137] Right now:

[0138]

[0139] Then, the ratio of the horizontal distance to the corresponding target distance (the first target distance or the second target distance) is converted into a ratio of the number of image pixels to a first number of pixels from the center point coordinate of the first road edge to the center of the image and a second number of pixels from the center point coordinate of the second road edge to the center of the image;

[0140] Right now:

[0141]

[0142] In summary, through , , the first zoom factor and the second zoom factor can be obtained.

[0143] Step S250: Based on the configured screening mechanism, each zoom factor is screened to determine a target zoom factor, and the camera of the drone is controlled to zoom according to the target zoom factor to perform image capture.

[0144] Specifically, the filtering mechanism can be to use the minimum value among all zoom factors as the target zoom factor. This means that the widest angle (i.e., lowest magnification) setting is selected as the standard among a range of possible zoom settings. This selection method ensures maximum image acquisition coverage, reduces image jitter or blur caused by high-magnification zoom, and optimizes resource utilization and data processing efficiency.

[0145] Combine Figure 4 As shown, the drone's camera is controlled to adjust to the selected target zoom factor and perform image acquisition. After zoom adjustment is complete, real-time image or video streaming begins. The wider viewing angle (lower zoom factor) allows for greater information capture, facilitating subsequent analysis and processing. This setting also reduces the risk of image distortion caused by device movement or external environmental influences. The captured image or video quality is then checked to ensure it meets expectations, and any adjustments are made as needed. If the existing settings no longer meet your needs under specific conditions, it may be necessary to reevaluate and select a new target zoom factor.

[0146] In this approach, after adaptive zoom adjustment, the images captured by the drone's camera are instantly transmitted to the drone management platform via 5G network equipment, allowing operators to monitor the collected image data in real time. Furthermore, after data collection is completed, the videos or images recorded during the inspection are automatically synchronized and archived in the database. This collected data can serve as the basis for tracing inspection events.

[0147] The present application provides a method for collecting road images using a drone, the method comprising: processing a current image collected by a drone to determine an edge image corresponding to each road in the current image; using a line detection algorithm to extract features from the edge image, determining the position data of different roadside edges involved in each road in the edge image, and determining the coordinate data of the position data of the different roadside edges in the image coordinate system of the current image; for any roadside edge, determining a coordinate adjustment angle based on the coordinate data corresponding to the roadside edge and the edge direction of the current image, and adjusting the coordinate data based on the coordinate adjustment angle to obtain target coordinate data; based on the target coordinate data of each roadside edge, determining the number of pixels from the roadside center of the corresponding roadside edge to the image center of the current image; based on the number of pixels corresponding to each roadside edge and current camera parameters, determining a zoom factor corresponding to the corresponding roadside edge; based on a configured screening mechanism, screening each zoom factor to determine a target zoom factor, and controlling the drone's camera to zoom according to the target zoom factor before performing image collection. The present application can calculate the actual road width by analyzing the roadside edges in the image, and achieve accurate measurement by combining drone flight parameters and camera parameters. Based on the actual road width, a mathematical model is used to adaptively adjust the camera zoom factor, optimizing the capture effect and ensuring the optimal proportion of the road within the frame and the required accuracy. This process efficiently integrates drone flight, image acquisition and processing, and precise measurement technology, providing an innovative solution for urban road inspections.

[0148] Corresponding to the above method, the embodiment of the present application also provides a device for collecting road images by a drone, such as Figure 5 As shown, the device includes:

[0149] A processing unit 510 is configured to process a current image captured by the drone to determine edge images corresponding to each road in the current image; the current image is an image captured by the drone based on current camera parameters of multiple roads within a target area;

[0150] a determining unit 520 configured to extract features from the edge image using a line detection algorithm, determine position data of different road edges involved in each road in the edge image, and determine coordinate data of the position data of the different road edges in an image coordinate system of the current image;

[0151] and, for any roadside, determining a coordinate adjustment angle based on the coordinate data corresponding to the roadside and the edge direction of the current image, and adjusting the coordinate data based on the coordinate adjustment angle to obtain target coordinate data;

[0152] and, based on the target coordinate data of each roadside, determining the number of pixels from the roadside center of the corresponding roadside to the image center of the current image;

[0153] and, determining a zoom factor corresponding to each roadside based on the number of pixels corresponding to each roadside and the current camera parameters;

[0154] Furthermore, based on the configured screening mechanism, each zoom factor is screened to determine a target zoom factor, and the camera of the drone is controlled to capture images after zooming according to the target zoom factor.

[0155] The functions of the various functional units of the device for collecting road images by a drone provided in the above-mentioned embodiment of the present application can be realized through the above-mentioned method steps. Therefore, the specific working process and beneficial effects of the various units in the device for collecting road images by a drone provided in the embodiment of the present application will not be repeated here.

[0156] The present application also provides an electronic device, such as Figure 6 As shown, it includes a processor 610 , a communication interface 620 , a memory 630 and a communication bus 640 , wherein the processor 610 , the communication interface 620 , and the memory 630 communicate with each other via the communication bus 640 .

[0157] Memory 630, for storing computer programs;

[0158] The processor 610 is configured to execute the program stored in the memory 630 by performing the following steps:

[0159] Processing a current image captured by the drone to determine edge images corresponding to each road in the current image; the current image is an image captured by the drone based on current camera parameters of multiple roads in the target area;

[0160] Using a line detection algorithm to perform feature extraction on the edge image, determine position data of different curbs involved in each road in the edge image, and determine coordinate data of the position data of the different curbs in an image coordinate system of the current image;

[0161] For any roadside, determining a coordinate adjustment angle based on the coordinate data corresponding to the roadside and the edge direction of the current image, and adjusting the coordinate data based on the coordinate adjustment angle to obtain target coordinate data;

[0162] Determining, based on the target coordinate data of each roadside, the number of pixels from the roadside center of the corresponding roadside to the image center of the current image;

[0163] Determining a zoom factor corresponding to each roadside based on the number of pixels corresponding to each roadside and the current camera parameters;

[0164] Based on the configured screening mechanism, each zoom factor is screened to determine the target zoom factor, and the camera of the UAV is controlled to zoom according to the target zoom factor to perform image acquisition.

[0165] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into address buses, data buses, and control buses. For ease of illustration, the figure uses only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0166] The communication interface is used for communication between the above electronic device and other devices.

[0167] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0168] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0169] The implementation methods and beneficial effects of the various components of the electronic device in the above embodiments to solve the problems can be found in Figure 2 The various steps in the embodiment shown are implemented, therefore, the specific working process and beneficial effects of the electronic device provided by the embodiment of the present application are not repeated here.

[0170] In another embodiment provided in the present application, a computer-readable storage medium is also provided, which stores instructions. When the computer-readable storage medium is executed on a computer, the computer executes the method for collecting road images by a drone as described in any of the above embodiments.

[0171] In another embodiment provided by the present application, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer executes the method for collecting road images using a drone as described in any of the above embodiments.

[0172] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware. Furthermore, the embodiments of the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0173] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0174] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0175] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0176] Unless otherwise defined, the technical or scientific terms used in this application should have the usual meanings understood by persons of ordinary skill in the field to which the invention belongs. The words "first", "second" and similar terms used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect", "couple" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0177] Although preferred embodiments have been described in the present application, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the present application is intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0178] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the embodiments of the present application and their equivalents, the embodiments of the present application are also intended to include these modifications and variations.

Claims

1. A method for collecting road images using a drone, characterized in that: The method comprises: Processing a current image captured by the drone to determine edge images corresponding to each road in the current image; the current image is an image captured by the drone based on current camera parameters of multiple roads in the target area; Using a line detection algorithm to perform feature extraction on the edge image, determine position data of different curbs involved in each road in the edge image, and determine coordinate data of the position data of the different curbs in an image coordinate system of the current image; For any roadside, determining a coordinate adjustment angle based on the coordinate data corresponding to the roadside and the edge direction of the current image, and adjusting the coordinate data based on the coordinate adjustment angle to obtain target coordinate data; Determining, based on the target coordinate data of each roadside, the number of pixels from the roadside center of the corresponding roadside to the image center of the current image; Determining a zoom factor corresponding to each roadside based on the number of pixels corresponding to each roadside and the current camera parameters; Based on the configured screening mechanism, each zoom factor is screened to determine a target zoom factor, and the camera of the drone is controlled to zoom according to the target zoom factor to perform image acquisition; Wherein, different curbs include: a first curb and a second curb; Determining the zoom factor corresponding to the corresponding roadside includes: converting the relationship between the ground sampling distance and the target ground sampling distance corresponding to each roadside after zooming into the relationship between the horizontal distance of the camera image of the drone and the first target distance and the second target distance; the first target distance represents the distance from the center of the first roadside of the first roadside to the center of the image, and the second target distance represents the distance from the center of the second roadside of the second roadside to the center of the image; the ground sampling distance represents the ground size corresponding to a single pixel; Converting the relationship between the horizontal distance of the camera image of the drone and the first target distance and the second target distance into the relationship between the number of image pixels and the first number of pixels from the center point coordinates of the first road edge to the center of the image and the second number of pixels from the center point coordinates of the second road edge to the center of the image; the number of image pixels is the ground sampling distance and the number of pixels in the horizontal direction of the current image; A first zoom factor of a first edge and a second zoom factor of a second edge are determined based on relationships between the number of image pixels and the first number of pixels and the second number of pixels, respectively.

2. The method according to claim 1, wherein The current camera parameters include original focal length, sensor size, number of sensor pixels and shooting distance; Determine the zoom factor corresponding to the corresponding curb, including: For any roadside, determining a ground sampling distance according to the shooting distance, the sensor size, the original focal length, and the number of sensor pixels; Based on the relationship between the ground sampling distance and the target ground sampling distance corresponding to each roadside after zooming, the zoom factor corresponding to the corresponding roadside is determined.

3. The method according to claim 2, wherein Determining a ground sampling distance according to the shooting distance, the sensor size, the original focal length, and the number of sensor pixels includes: Determining a first parameter based on the sensor size and the number of sensor pixels, wherein the first parameter represents the sensor size corresponding to a single pixel; The ground sampling distance corresponding to any roadside is determined based on the first parameter, the shooting distance and the original focal length.

4. The method according to claim 3, wherein After determining the ground sampling distance, the method further includes: The horizontal distance of the camera image of the UAV is determined based on the ground sampling distance and the number of image pixels in the horizontal direction of the current image.

5. The method according to claim 1, wherein The screening mechanism of the configuration is: the minimum value among the zoom factors is determined as the target zoom factor.

6. The method according to claim 1, wherein After determining the coordinate adjustment angle, the method further includes: Based on the coordinate adjustment angle, the camera gimbal of the UAV is adjusted to obtain a target angle, so that the camera of the UAV performs image acquisition according to the target angle.

7. A device for collecting road images by a drone, characterized in that: The device comprises: a processing unit configured to process a current image captured by the drone and determine an edge image corresponding to each road in the current image; the current image is an image captured by the drone based on current camera parameters of multiple roads within a target area; a determining unit, configured to extract features from the edge image using a line detection algorithm, determine position data of different road edges involved in each road in the edge image, and determine coordinate data of the position data of the different road edges in an image coordinate system of the current image; and, for any roadside, determining a coordinate adjustment angle based on the coordinate data corresponding to the roadside and the edge direction of the current image, and adjusting the coordinate data based on the coordinate adjustment angle to obtain target coordinate data; and, based on the target coordinate data of each roadside, determining the number of pixels from the roadside center of the corresponding roadside to the image center of the current image; and, determining a zoom factor corresponding to each roadside based on the number of pixels corresponding to each roadside and the current camera parameters; and, based on a configured screening mechanism, screening each zoom factor, determining a target zoom factor, and controlling the camera of the drone to zoom according to the target zoom factor to perform image capture; Wherein, different curbs include: a first curb and a second curb; Determine the zoom factor corresponding to the corresponding curb, including: Converting the relationship between the ground sampling distance and the target ground sampling distance corresponding to each road edge after zooming into the relationship between the horizontal distance of the camera image of the drone and the first target distance and the second target distance; the first target distance represents the distance from the center of the first road edge to the center of the image, and the second target distance represents the distance from the center of the second road edge to the center of the image; the ground sampling distance represents the ground size corresponding to a single pixel; Converting the relationship between the horizontal distance of the camera image of the drone and the first target distance and the second target distance into the relationship between the number of image pixels and the first number of pixels from the center point coordinates of the first road edge to the center of the image and the second number of pixels from the center point coordinates of the second road edge to the center of the image; the number of image pixels is the ground sampling distance and the number of pixels in the horizontal direction of the current image; A first zoom factor of a first edge and a second zoom factor of a second edge are determined based on relationships between the number of image pixels and the first number of pixels and the second number of pixels, respectively.

8. An electronic device, characterized in that: The electronic device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 6 when executing a program stored in a memory.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 6 are implemented.

Images