A method, device and medium for unmanned aerial vehicle (UAV) flight path planning

Abstract

This invention discloses a method, device, and medium for drone flight path planning, relating to the field of photovoltaic cleaning. The method includes: first, acquiring a single image of the area of ​​the component to be cleaned and a LiDAR point cloud, then stitching them together to output an orthophoto; rasterizing the point cloud to generate a two-dimensional elevation mean model, and extracting component edges and identifying corner points from the image; using geometric fusion to project the corner points onto the model to obtain preliminary three-dimensional coordinates of the corner points, which are then optimized through principal component analysis; next, clustering and fusing the corner points from multiple images, overlaying them with the optimized coordinates onto the orthophoto for verification, forming a set of regional corner point coordinates and grouping them; finally, generating a two-dimensional flight path based on this and the flight path spacing, explicitly specifying the absolute elevation to obtain the target cleaning flight path. This method integrates multi-source data, utilizes single images for precise corner point identification, employs LiDAR point clouds to improve positioning accuracy, and relies on orthophotos to ensure complete global analysis. The generated flight path has high accuracy and strong adaptability, accurately guiding drone cleaning and improving efficiency and quality.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic cleaning, and in particular to a method, equipment and medium for unmanned aerial vehicle (UAV) flight path planning. Background Technology

[0002] Drone cleaning is gradually being promoted in the photovoltaic cleaning field. It has advantages such as not directly contacting the components, high cleaning efficiency, high degree of automation and low labor cost. However, at present, most drones are still operated by humans visually, and there are great difficulties in flight path planning.

[0003] Traditional drone flight path planning for photovoltaic power plants relies solely on parallel arc-shaped flight paths based on the polygonal boundaries of the entire plant. This approach is suitable for aerial photography of the entire plant or power plant inspections. However, due to the diverse arrangement of photovoltaic modules, this method cannot guarantee that the cleaning drone will complete the cleaning task above the modules, making it unsuitable for photovoltaic module drone cleaning applications. To create a cleaning flight path above the modules, cleaning drones or markers are typically used to mark key points along the flight path at various points on-site. However, this method is time-consuming and costly. Alternatively, relying on images and surface models, key flight path points are manually selected, or algorithms are used to identify module boundary points. While image recognition is mature, model interpolation is prone to edge elevation deviations, and issues such as splicing misalignment and registration errors also reduce positioning accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a method, device, and medium for planning unmanned aerial vehicle (UAV) flight paths, which can achieve fine identification, accurate positioning, and global layout analysis of component corner points, and generate high-precision target cleaning flight paths.

[0005] To address the aforementioned technical problems, this invention provides a method for unmanned aerial vehicle (UAV) route planning, comprising:

[0006] Acquire single images of the boundary of the component area to be cleaned and LiDAR point cloud data, stitch them together and output an orthophoto file;

[0007] The lidar point cloud data is rasterized, and the average elevation of each raster unit is calculated to generate a two-dimensional average elevation model; component edge extraction and corner point recognition are performed on the single image to obtain the corner point recognition result;

[0008] The corner point identification results are projected onto the two-dimensional elevation mean model using a geometric fusion method to obtain the preliminary three-dimensional coordinates of the corner points. Principal component analysis is then performed on the lidar point cloud surrounding the preliminary three-dimensional coordinates of the corner points to obtain the optimized three-dimensional coordinates of the corner points.

[0009] Multiple images of the boundary of the component area to be cleaned are acquired, and the corner points of the multiple images are clustered and fused to obtain the optimal set of corner point coordinates after fusion. The optimal set of corner point coordinates and the optimized set of three-dimensional coordinates are superimposed on the orthophoto file. After verification, the set of corner point coordinates of the region is formed and sorted by group.

[0010] Based on the coordinates of the corner points of the grouped and sorted areas and the spacing between the routes, a two-dimensional route file is generated, and the absolute elevation is explicitly specified in the two-dimensional route file to obtain the target cleaning route file.

[0011] To address the aforementioned technical problems, the present invention also provides an electronic device, comprising:

[0012] Memory, used to store computer programs;

[0013] A processor is used to implement the steps of the above-described UAV route planning method when executing the computer program.

[0014] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned UAV route planning method.

[0015] As can be seen from the above technical solution, the UAV route planning method provided by the present invention includes: first, acquiring a single image of the boundary of the component area to be cleaned and lidar point cloud data, stitching them together and outputting an orthophoto file; then, rasterizing the lidar point cloud data, calculating the average elevation of each raster unit to generate a two-dimensional average elevation model; and performing component edge extraction and corner point recognition on the single image to obtain the corner point recognition result. Then, a geometric fusion method was used to project the corner recognition results onto a two-dimensional elevation mean model to obtain the preliminary three-dimensional coordinates of the corners. Principal component analysis was then performed on the lidar point cloud surrounding the preliminary three-dimensional coordinates of the corners to obtain the optimized three-dimensional coordinates of the corners. Subsequently, multiple images of the boundary of the component area to be cleaned were acquired, and the corners of the multiple images were clustered and fused to obtain the optimal set of fused corner coordinates. The optimal set of corner coordinates and the set of optimized three-dimensional coordinates were superimposed on the orthophoto file. After verification, the set of regional corner coordinates was formed and grouped and sorted. Finally, based on the grouped and sorted regional corner coordinates and the flight path spacing, a two-dimensional flight path file was generated, and the absolute elevation was explicitly specified in the two-dimensional flight path file to obtain the target cleaning flight path file.

[0016] The beneficial effects of this invention are as follows: The above-mentioned UAV flight path planning method provided by this invention performs component edge and corner point recognition based on a single image, integrates LiDAR point cloud data for component area corner point positioning, and performs global layout analysis of component corner points through orthophotos of the component area to generate a high-precision flight path for the cleaning UAV. Through multi-source data fusion and precise processing, it not only achieves fine recognition of component edges and corner points using a single image, but also improves the accuracy of corner point positioning through LiDAR point cloud data. At the same time, it relies on orthophotos to ensure the completeness of the global layout analysis of corner points. The final generated target cleaning flight path file containing explicit absolute elevation has higher accuracy and stronger adaptability, and can provide precise guidance for UAV cleaning operations, effectively improving cleaning efficiency and quality, and reducing operational deviations and errors.

[0017] In addition, the present invention also provides a corresponding UAV route planning electronic device and computer-readable storage medium for the UAV route planning method, which have the same or corresponding technical features as the UAV route planning method mentioned above, and have the same effect. Attached Figure Description

[0018] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart of the UAV route planning method provided in an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of the flight path of a surveying drone collecting data at the boundary of the component area to be cleaned, provided in an embodiment of the present invention.

[0021] Figure 3 A schematic diagram illustrating the framework of the UAV route planning method provided in an embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of vertical viewing angle correction for a single image provided in an embodiment of the present invention;

[0023] Figure 5 A schematic diagram of a framework for component edge and corner point recognition based on a single image provided in an embodiment of the present invention;

[0024] Figure 6 This is a schematic diagram of corner point geometric fusion positioning provided in an embodiment of the present invention;

[0025] Figure 7 This is a schematic diagram illustrating the precise positioning of corner points provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0027] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0028] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] The specific application environment architecture or specific hardware architecture on which the execution of the UAV flight path planning method depends is described here.

[0030] The embodiments of the present invention provide a method for planning unmanned aerial vehicle (UAV) flight paths, and the method is described in detail in conjunction with the execution flow of the UAV flight path planning method. Figure 1 A flowchart of the UAV route planning method provided in the embodiments of the present invention is shown below. Figure 1 As shown, the method includes:

[0031] S101. Acquire a single image of the boundary of the component area to be cleaned and LiDAR point cloud data, stitch them together and output an orthophoto file.

[0032] It should be noted that the original resolution of a single image is high, uncompressed or unresampled, and the edge details are clear. Component edge and corner point recognition based on a single image is the fastest and most accurate, with the planar positioning accuracy of corner points being closest to that of Real-Time Kinematic (RTK) positioning.

[0033] In implementation, surveying drones can be used to collect image data and LiDAR point cloud data of the boundaries of the component area to be cleaned, reducing the workload of overall site aerial modeling. Then, the original single high-resolution RGB image with Position and Orientation System (POS) data from the surveying drone is stored.

[0034] Figure 2 This is a schematic diagram of the flight path of a surveying drone collecting data at the boundary of the component to be cleaned, provided in an embodiment of the present invention. Figure 2 As shown, the surveying drone can perform surveying along the route from 1 to 2, 2 to 3, 3 to 4, 4 to 1 and 5, 5 to 6, 6 to 7, 7 to 8, 8 to 9. Figure 2 The black dots in the image represent corner points of the component area. This invention can use software tools (such as Pix4D, ContextCapture, etc.) to import aerial photographs with POS data, perform aerial triangulation calculations and orthorectification, stitch together orthophotos, and output a Georeferenced Tagged Image File Format (GeoTIFF) file. The GeoTIFF orthophoto file can then be imported into a Geographic Information System (ArcGIS).

[0035] S102. Rasterize the lidar point cloud data, calculate the average elevation of each grid cell to generate a two-dimensional average elevation model; perform component edge extraction and corner point recognition on a single image to obtain the corner point recognition result.

[0036] In practice, the above steps involve targeted processing of two types of core data: lidar point clouds and single images, laying the foundation for subsequent multi-source data fusion.

[0037] Figure 3 This is a schematic diagram illustrating the framework of the UAV route planning method provided in an embodiment of the present invention. Figure 3 As shown, after acquiring images and radar data, the lidar point cloud data can be processed into a two-dimensional raster, and then the elevation mean of all point clouds in each raster unit (i.e., grid unit) can be calculated to smooth the noise in the point cloud data, and finally generate a two-dimensional elevation mean model; for a single image, component edge extraction and corner point recognition can be performed. This is achieved by using image processing algorithms to capture the contour boundary and key corner points of the component to be cleaned, and finally obtain the corner point recognition result.

[0038] S103. The corner point identification results are projected onto the two-dimensional elevation mean model using a geometric fusion method to obtain the preliminary three-dimensional coordinates of the corner points. Principal component analysis is then performed on the lidar point cloud surrounding the preliminary three-dimensional coordinates of the corner points to obtain the optimized three-dimensional coordinates of the corner points.

[0039] In implementation, the above steps are the key transformation process for realizing the 3D coordinates of corner points from two-dimensional to three-dimensional, and from preliminary to precise. First, through geometric fusion, the two-dimensional corner points identified in a single image are projected onto a two-dimensional elevation mean model generated from the LiDAR point cloud. Using the elevation data contained in the model, the two-dimensional corner points are assigned height information, thus obtaining the preliminary 3D coordinates of the corner points, completing the mapping from the image plane to 3D space. Subsequently, principal component analysis is performed on the LiDAR point cloud data surrounding the preliminary 3D coordinates. This analysis can effectively extract the main distribution features and spatial structure of the point cloud in this area. By eliminating noise points and fitting the optimal spatial position, possible deviations in the preliminary 3D coordinates are corrected, ultimately obtaining optimized 3D coordinates with higher accuracy and closer to the actual physical location. This retains the feature accuracy of corner point recognition in the image while leveraging the advantages of LiDAR point clouds in 3D positioning.

[0040] S104. Acquire multiple images of the boundary of the component area to be cleaned, and cluster and fuse the corner points of the multiple images to obtain the optimal set of corner point coordinates after fusion. Then, overlay the optimal set of corner point coordinates and the optimized set of three-dimensional coordinates in the orthophoto file. After verification, form the set of corner point coordinates of the region and sort them by group.

[0041] In implementation, the above steps are an integration process from local corner data to a globally unified corner set. This is achieved through multi-image data complementarity and global verification to ensure the integrity, consistency, and usability of corner coordinates. First, multiple images of the boundary of the component area to be cleaned are acquired to compensate for corner omissions / misidentifications caused by the limited field of view, viewing angle deviation, or local occlusion of a single image. Corner data from different perspectives are collected to form multi-source local samples. The corners of these multiple images are clustered and fused. Algorithms are used to group the recognition results of different images with similar features and corresponding to the same physical corner into one category, eliminating duplicates or outliers with excessive errors, and finally generating the optimal set of corner coordinates to ensure the uniformity of corner positions. Next, the optimal set is superimposed on the optimized 3D coordinate set obtained in step S103 onto the orthophoto file. This is to visually verify whether the spatial distribution of corner points conforms to the actual structure of the component through the global view and geographic reference of the orthophoto, and to verify and eliminate contradictory data. After verification, the effective corner points are integrated into a set of regional corner point coordinates and grouped and sorted. This is to logically classify and arrange each corner point according to the physical partition of the component, laying the foundation for the subsequent generation of orderly and accurate cleaning routes based on corner points, and ensuring that the routes can cover the entire area and fit the component layout.

[0042] S105. Based on the coordinates of the corner points of the grouped and sorted areas and the spacing between the routes, generate a two-dimensional route file, and explicitly specify the absolute elevation in the two-dimensional route file to obtain the target cleaning route file.

[0043] In implementation, the above steps are the process of converting corner data into executable UAV operation commands. First, the grouped and sorted corner coordinates provide path anchors for flight path planning. Corner points grouped and sorted according to the physical layout of components define the coverage area and path direction of the UAV cleaning operation. The flight path spacing is a key parameter for controlling the cleaning density, ensuring that there is neither duplication nor redundancy between adjacent flight paths, nor cleaning blind spots. Based on these two criteria, a two-dimensional flight path file containing the flight path can be generated. Subsequently, the absolute elevation is explicitly specified in the two-dimensional flight path file to compensate for the insufficiency of the two-dimensional flight path containing only planar coordinates. This avoids problems such as UAV collisions due to missing elevation information and improper altitude affecting the cleaning effect. The final target cleaning flight path file contains both planar path and vertical height information and can be directly imported into the UAV control system to achieve automated and high-precision cleaning operations.

[0044] In the UAV flight path planning method provided by the embodiments of the present invention, component edge and corner point recognition is performed based on a single image, and corner point positioning of the component area is performed by fusing LiDAR point cloud data. Global layout analysis of component corner points is performed through orthophotos of the component area to generate a high-precision flight path for the cleaning UAV. Through multi-source data fusion and precise processing, it not only achieves fine recognition of component edges and corner points using a single image, but also improves the accuracy of corner point positioning through LiDAR point cloud data. At the same time, it relies on orthophotos to ensure the completeness of the global layout analysis of corner points. The final generated target cleaning flight path file containing explicit absolute elevation has higher accuracy and stronger adaptability, and can provide precise guidance for UAV cleaning operations, effectively improving cleaning efficiency and quality, and reducing operational deviations and errors.

[0045] Furthermore, in a specific implementation, in the above-mentioned UAV route planning method provided in the embodiments of the present invention, step S102 of rasterizing the lidar point cloud data may specifically include: calculating the column index and row index of the corresponding two-dimensional raster by taking the original Cartesian coordinates of each point in the lidar point cloud data, and storing them in the corresponding location; wherein the column index is obtained by subtracting the minimum boundary value in the x direction from the x coordinate of the point and then dividing by the width of the raster cell, and the row index is obtained by subtracting the y coordinate of the point from the maximum boundary value in the y direction and then dividing by the height of the raster cell.

[0046] In practice, the original airborne lidar point cloud data is converted into a format more suitable for real-time processing and analysis through two-dimensional rasterization.

[0047] Each point in the airborne lidar point cloud can be stored in a two-dimensional grid based on its row and column coordinates. The row and column coordinates of each point in the two-dimensional grid are determined by its original Cartesian coordinates, since the row coordinates of the grid... The coordinates are usually increased from top to bottom (the origin of the raster coordinate system is usually at the top left corner, opposite to the Y-axis direction of the Cartesian coordinate system), so subtraction is used to achieve coordinate flipping, as shown in the following formula:

[0048] ;

[0049] in, This represents the column index in the converted point cloud raster; This represents the row index in the converted point cloud raster; The original Cartesian coordinates of the i-th point in the airborne lidar point cloud are represented; n represents the total number of points in the airborne lidar point cloud. , This represents the width (column resolution) and height (row resolution) of a raster cell, i.e., the actual physical size of each raster cell; , These are the boundary values ​​of the point cloud raster, representing the initial minimum coordinate of x and the maximum coordinate of y, respectively.

[0050] Furthermore, in a specific implementation, in the above-mentioned UAV route planning method provided in the embodiments of the present invention, step S102, which calculates the average elevation of each grid cell to generate a two-dimensional average elevation model, may specifically include: dividing the grid cell into empty cells without points, sparse cells containing one or two points, and dense cells containing three or more points, based on the number of points contained within the grid cell; performing multi-directional interpolation on the empty cells with the empty cells as the center to calculate the average elevation of the empty cells; fitting the sparse cells with the sparse cells as the center using a distance-weighted method to calculate the average elevation of the sparse cells; fitting the dense cells using a median filtering method to obtain the average elevation of the dense cells; and after calculating the average elevation of all grid cells, using the average elevation of each grid cell as the elevation value of its own covered local area, and splicing them together to form a two-dimensional average elevation model.

[0051] It should be noted that lidar point clouds are susceptible to interference from adverse environmental factors, such as the absence of data in shadowed areas and reflections from water surfaces, which can cause missing data. Therefore, the rasterized airborne lidar point cloud still exhibits spatial inhomogeneity and contains outliers. This invention calculates the elevation mean of each raster cell. Considering the different noise reduction and fitting requirements corresponding to the density differences in the rasterized lidar point cloud of each raster cell, the raster cells are divided into three categories:

[0052] For empty cells that do not contain points, multi-directional interpolation is performed to calculate the elevation of the central empty raster cell. The steps are as follows:

[0053] Step 1: Using the empty grid cell to be calculated as the center, calculate the angles in eight directions sequentially. The search is performed on adjacent graticles. Only non-empty and valid neighboring graticles are considered, i.e., graticle cells containing two or more points in a certain direction. If a graticle cell contains two or more points in a certain direction, the search in that direction is stopped, and all points found in that graticle cell are considered points found in that direction, making that direction a valid direction. If no graticle cell containing two or more points is found in a certain direction, then that search direction is determined to not contain any points, and the direction is considered invalid.

[0054] Step 2: Determine the number of search directions with non-empty and valid neighboring grid cells. If all eight directions are invalid, the central empty grid cell is considered a superboundary cell and is skipped. If valid directions exist, the number of valid directions is recorded as follows: Continue with step three.

[0055] Step 3: Calculate the average elevation of the central grid cell, as shown below:

[0056] ;

[0057] ;

[0058] ;

[0059] ;

[0060] in, The average elevation of the central grid cell; For effective search direction, ; For direction The number of points searched; In direction The elevation of the k-th point among the points searched; In direction The weight of the points searched; In direction The directional weights of the points searched are the reciprocal of the Euclidean distance; In direction The coordinates of the grid cells on the grid; The coordinates of the central grid cell; In direction The number of points searched and their weights, in terms of direction. The logarithm of the number of points searched above suppresses the strong influence of high-density or sparse regions, preventing a few dense points from dominating the weights. Dividing by the number of points further balances the contributions of regions with different densities. Since the search in that direction only stops when there are two or more grid cells, there is no... , The situation.

[0061] Step 4: Calculate the standard deviation :

[0062] .

[0063] Step 5: Calculate the average elevation of the central grid cell:

[0064] ;

[0065] in, The average elevation of the local area covered by the central grid cell; As The standard deviation multiple constraint can screen out efficient points that conform to a normal distribution and eliminate outliers. It is robust, and only those points that meet the conditions are included. Only then does it participate in the calculation of the average elevation of the central grid cell.

[0066] For sparse cells containing one or two points, a distance-weighted method is used to fit the elevation of the raster cells:

[0067] First, calculate the search area. Centered on the grid cell to be calculated, within... The central grid cell is searched within its neighborhood. If the number of valid points found in the surrounding grid cells is less than the threshold Va, the central grid cell is determined to be an empty cell containing no points, and the eight-axis imaging method for empty cells is used for calculation. If the number of valid points is greater than or equal to the threshold Va, the subsequent steps are continued.

[0068] ;

[0069] ;

[0070] in, The average nearest neighbor distance during the rasterization process is denoted as the point set, which consists of a sufficient number of N points randomly sampled from the point cloud. For each point Calculate its nearest neighbor. The distance is calculated by averaging the distances of all nearest neighbors. Let be the Euclidean norm, representing the i-th point randomly sampled from the point cloud. To its nearest neighbor The distance; is the raster resolution of the rasterization process, i.e., the physical size of each raster cell after rasterization; Va is the point threshold, taking 3-5 points in near-field high-density point clouds and 1-2 points in far-field low-density point clouds.

[0071] Then, calculate the isotropic simplified Gaussian weights:

[0072] ;

[0073] in, The coordinates of the raster cells within the search range; The coordinates of the central grid cell; These are the normalization coefficients; These are the Gaussian weights of the raster cells within the search range, used for calculation. coordinates to the center grid cell The weights decrease with the square of the Euclidean distance; This is the variance of the Gaussian weight function. Since the smoothing requirements are the same in the x and y directions, to reduce parameters and improve computational efficiency, isotropic simplification can be performed. It is assumed that the weight decay rates in both directions are the same, i.e., the standard deviations in both directions are the same. , Calculation based on data point density:

[0074] .

[0075] Next, calculate the elevation-weighted mean of the central grid cell:

[0076] ;

[0077] in, is the elevation-weighted average of the central grid cell; z represents the search neighborhood range, which must be an odd number to ensure central symmetry, and is calculated based on the average density of the point cloud. For the coordinates are The number of points to search within the grid cells; In grid cells The elevation of the k-th point in the search.

[0078] Calculate the standard deviation :

[0079] ;

[0080] Calculate the average elevation of the central grid cell:

[0081] ;

[0082] ;

[0083] in, The average elevation of the local area covered by the central grid cell; As The standard deviation multiple constraint can screen out efficient points that conform to a normal distribution and eliminate outliers. It is robust, and only those points that meet the conditions are included. Only then does it participate in the calculation of the average elevation of the central grid cell.

[0084] For dense cells containing three or more points, median filtering is used to fit the mean elevation of the raster cells. All points within the raster cell are sorted by elevation, and the median value of the sorted results is taken as the elevation of the raster cell, as shown in the following formula:

[0085] ;

[0086] in, The average elevation of the local area covered by the central grid cell; This represents the median filtering function, which can be used to obtain the median elevation of all points in a grid cell; Represents the set of elevations of points within a raster cell.

[0087] Therefore, the average elevation of all grid cells can be calculated using the three methods described above. This average elevation is then used as the elevation of the local area covered by that pixel, forming a two-dimensional average elevation model.

[0088] Furthermore, in a specific implementation, in the above-mentioned UAV route planning method provided in the embodiments of the present invention, step S102 performs component edge extraction and corner point recognition on a single image to obtain corner point recognition results. Specifically, it may include: performing vertical viewpoint correction on a single image to generate a corresponding horizontal image; using a multi-scale filter bank and phase-consistent edge detection method to perform component edge extraction and corner point recognition on the horizontal image, and performing corner point sub-pixel optimization to obtain corner point recognition results.

[0089] In practice, correction is performed on a single image first. Figure 4 This is a schematic diagram of vertical viewing angle correction for a single image provided in an embodiment of the present invention. Figure 4 As shown, It is the projection center of the camera; These are raw single images taken by a surveying drone, which contain tilt angles, perspective distortion, and occlusion. yes A horizontal image after vertical perspective correction to eliminate perspective distortion; yes The main point, and , yes The main point, and ; yes Any image point on; yes Image points on yes Homologous image points.

[0090] exist Choose any point ,according to and Determine the geometric relationship between them The two-dimensional image coordinates are projected using the following equation:

[0091] ;

[0092] The product of three rotation matrices expands into one A matrix whose elements correspond to coefficients. The formula for the rotation matrix is ​​as follows:

[0093] ;

[0094] in, for In horizontal image Two-dimensional image coordinates; for In the original image Two-dimensional image coordinates; To measure the focal length of the drone camera; The coefficients of the rotation matrix are given by... Decide; These represent the pitch angle, roll angle, and heading angle of the surveying drone, respectively, and are obtained from the POS data of the original single aerial image taken by the surveying drone. For rotation around the X-axis, Rotate around the Y-axis Rotate around the Z-axis.

[0095] Apply the above steps For each image point on the image, the following can be calculated: Horizontal image after vertical perspective correction to eliminate perspective distortion .

[0096] Furthermore, in specific implementation, in the above steps, a multi-scale filter bank and phase-consistency edge detection method are used to extract component edges and identify corner points in the horizontal image, and corner point sub-pixel optimization is performed to obtain corner point identification results. Specifically, this may include: converting the horizontal image into a grayscale image and performing normalization processing; transforming the normalized image to the frequency domain through Fourier transform, filtering it through a multi-scale Log-Gabor filter bank and performing inverse discrete Fourier transform to obtain the spatial domain complex response; calculating the phase consistency based on the spatial domain complex response, extracting the edge feature map after non-maximum suppression and threshold adjustment, and outputting the horizontal image coordinates of the corner points of the integer-level component region; using the horizontal image coordinates of the corner points of the integer-level component region as the center, calculating the horizontal and vertical gradients using a window of a set size to obtain the gradient vector and Hessian matrix, and iteratively solving the offset through a second-order Taylor expansion of grayscale changes to update the horizontal image coordinates of the corner points until the offset is less than a preset threshold or the maximum number of iterations is reached; and using random sampling consistency (RandomSample)... The Consensus (RANSAC) algorithm verifies the consistency of the gradient direction of corner points, retains the maximum set of interior points, and obtains the corner point recognition results.

[0097] It should be added that, before performing component edge extraction and corner point recognition on the horizontal image, the present invention can acquire the original training image (including a single image) as training data through the visible light camera of the UAV. Figure 5 This is a schematic diagram of a framework for component edge and corner recognition based on a single image, provided as an embodiment of the present invention. Figure 5 As shown, the original training image can be preprocessed, including: adding a reflective sensor to the Squeeze-Excitation (SE) network to detect reflective areas and suppress the feature responses of reflective areas; using the Fast Marching method to fill the edges of reflective areas with textures from adjacent non-reflective areas to achieve edge repair; and performing automatic exposure correction on the visible light feature map after suppressing reflections, with the target median brightness set to 100-120 and the brightness percentile set to 50%. Then, the preprocessed image can be corrected for vertical viewing angle to obtain a horizontal image.

[0098] In implementation, because the feature points at the image edges have consistent phase positions in different frequency components, such as Figure 5As shown, this invention constructs a multi-scale filter bank and uses a Log-Gabor filter and a phase-consistent edge detection method for edge extraction. It detects low-contrast borders, unaffected by reflections from the photovoltaic module glass surface, thus solving the edge breakage problem caused by reflections. Furthermore, based on the approximately parallel gradient directions of corner points within the same row of modules, this invention uses the RANSAC algorithm to randomly select two corner points and calculate the gradient direction of the connecting line as a local edge model. It verifies whether the gradient direction deviation of other corner points is less than a threshold. After multiple sampling iterations, it retains the model with the most interior points and its corresponding interior point set, while removing outliers to obtain the corner point recognition result for a single image. The specific steps are as follows:

[0099] First, set the horizontal image coordinates Convert to grayscale and normalize to The range is used to perform a Fast Fourier Transform (FFT) to obtain the frequency space coordinates. and frequency domain representation .

[0100] Then, construct a multi-scale Log-Gabor filter bank. Divide the filter into 4-5 center frequencies according to octaves (e.g., 4 scales). =[0.05,0.1,0.2,0.4]). Set 6 directional angles. It covers anisotropic edges.

[0101] Generative filter formula:

[0102] ;

[0103] in, In order to be in Filter response value at frequency domain coordinates; Frequency domain coordinates; The center frequency of the filter; For bandwidth parameters, ; This is the current direction angle; The direction angle at the current frequency point. ; This refers to directional bandwidth, which controls the selectivity of directions. .

[0104] Next, frequency domain filtering and phase calculations are performed. For the Log-Gabor filter at each scale and direction, the filter response value is calculated to obtain the frequency domain filtering results. The spatial complex response is calculated using the inverse discrete Fourier transform (IDFT). :

[0105] ;

[0106] ;

[0107] Subsequently, phase consistency calculations are performed using multi-scale and directional accumulation:

[0108] ;

[0109] in, To be at the image location The phase consistency value at the location is within the range of The closer the value is to 1, the more pronounced the edge. To prevent the denominator from being zero, it is usually a very small constant. .

[0110] Next, edge feature maps are extracted. Non-maximum points are suppressed along the gradient direction (determined by the phase consistency direction), and the mean / standard deviation is calculated in blocks to adjust the threshold. The edge feature maps are then output. The horizontal image coordinates of the corner points of the i integer-level component regions are also output. .

[0111] Finally, subpixel optimization is performed on the corner points of the Log-Gabor output.

[0112] The i-th integer-level corner point in the Log-Gabor output Nearby, the image gray-level gradient distribution can be approximated by a second-order Taylor expansion. The Taylor expansion is used to fit the gradient distribution within the neighborhood of the integer-order corner horizontal image coordinates. Taking the integer-order corner horizontal image coordinates as the center... The window for calculating levels and vertical gradient Then calculate the gradient vector. And the Hessian matrix. A second-order Taylor expansion of grayscale variations within the window. Since the first derivative of the function at the extreme point is zero, we set the derivative of the second Taylor expansion to zero, and substitute the gradient vector and Hessian matrix to obtain the offset and update the horizontal image coordinates of the corner points. This iterative optimization continues until... (like If the maximum number of iterations (in pixels) is reached, the iteration will terminate; otherwise... Repeat the calculation for the new initial point until the final precise extreme point horizontal image coordinates of that corner are obtained. The formula is shown below:

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] ;

[0118] ;

[0119] ;

[0120] Since the gradient directions at the corner points of components in the same row should be approximately parallel, robustness verification is performed using RANSAC, with two corner points randomly selected. As model samples, the gradient direction (unit vector) of the line connecting the two corner points is calculated as a local edge model. To verify the gradient consistency of other corner points (e.g., gradient direction deviation is less than a preset angle tolerance threshold), two corner points are randomly sampled multiple times, and the process is iteratively updated until the largest set of interior points is found. That is, the model with the most interior points and its corresponding set of interior points are retained, while abnormal gradient corner points are removed.

[0121] ;

[0122] ;

[0123] in, For two random corner points A local edge model formed by connecting gradient directions (unit vectors); To allow for angular deviation; Let be the horizontal image coordinates of the i-th precise extremum point after subpixel optimization.

[0124] After performing the above image recognition processing on a single image, the largest set of interior points obtained is the result of corner point recognition for that single image. The recognized corner points are denoted as... The horizontal image coordinates of the corner points are as follows .

[0125] Furthermore, in specific implementation, in the above-mentioned UAV flight path planning method provided in the embodiments of the present invention, step S103 uses a geometric fusion method to project the corner point recognition result onto a two-dimensional elevation mean model to obtain the preliminary three-dimensional coordinates of the corner point. Specifically, it may include: converting a single image and the two-dimensional elevation mean model to the same coordinate system; obtaining the three-dimensional coordinates of the camera's projection center; extending the line connecting the projection center and the principal point of the horizontal image to the ground to obtain the first intersection point; retrieving the first elevation of the grid cell where the first intersection point is located from the two-dimensional elevation mean model; and calculating the vertical distance from the projection center to the first intersection point; drawing a straight line from the projection center to the corner point on the horizontal image and extending it to the ground; and drawing a line from the first intersection point to the corner point. The horizontal line and the intersection of the horizontal line and the straight line are the second intersection point. Based on the vertical distance from the projection center to the first intersection point, the horizontal image coordinates of the corner point, and the planar coordinates of the projection center, the three-dimensional coordinates of the second intersection point are calculated to obtain the projection point coordinates of the second intersection point relative to the ground. The second elevation of the grid cell containing the projection point corresponding to the second intersection point is retrieved from the two-dimensional elevation mean model, and the difference between the first elevation and the second elevation is calculated. Using the difference between the first elevation and the second elevation, the coordinates of the next projection point and the difference in elevation between two adjacent projection points are calculated. The steps of calculating the projection point coordinates and the difference are repeated until the difference in elevation between two adjacent projection points is less than a set threshold, thus obtaining the preliminary three-dimensional coordinates of the corner point.

[0126] In practice, the geometric fusion method is used to find the three-dimensional projection point on the ground of the horizontal image coordinates of the corner point of the original single image taken by the surveying drone, and the three-dimensional coordinates of the projection point are calculated to realize the positioning of the horizontal image coordinates of the corner point of the original single image taken by the surveying drone.

[0127] Figure 6 This is a schematic diagram of corner point geometric fusion positioning provided in an embodiment of the present invention. Figure 6 As shown, the two-dimensional mean elevation model is generated from the airborne radar point cloud using the method described above; It is the projection center of the camera; yes A horizontal image after vertical perspective correction to eliminate perspective distortion; yes The main point, and , yes The main point, and ; for The horizontal image coordinates of any corner point in the image, superior; yes The projection point on the ground; , and On a line.

[0128] Geometric fusion methods may include the following steps:

[0129] The first step is to convert a single aerial image taken by a surveying drone and a two-dimensional elevation mean model obtained from lidar point clouds to the same coordinate system, with the same direction and the origin corresponding to the same position in the physical world.

[0130] The second step is to obtain the projection center from the POS data of the surveying drone. 3D coordinates .extend Intersecting with the ground at ,but The plane coordinates of the point are Search Determine the elevation of the ground grid cell where the point is located, and calculate it. The length is:

[0131] ;

[0132] in, for length; for Point elevation; , This is an elevation retrieval function used to retrieve the elevation of the ground grid cell where the coordinates are located in a two-dimensional elevation mean model, based on a given plane rectangular coordinate. for Point elevation is obtained from the camera projection center elevation (ellipsoidal height) in the POS data of a single aerial image taken by the surveying drone. If the POS data records the elevation of the surveying drone's GNSS antenna, the vertical offset of the camera installation needs to be added. .

[0133] Step 3: Draw a straight line Intersecting with the ground at .from Draw a horizontal line from the point and Intersect at ,triangle and Similar, then The three-dimensional coordinates are:

[0134] ;

[0135] in, for 3D coordinates; for The horizontal image coordinates of any corner point in the image, superior; for Cartesian coordinates.

[0136] Step 4 Draw a perpendicular line that intersects the ground at... . Cartesian coordinates and Their Cartesian coordinates are the same. From Draw a horizontal line from the point and Intersect at ,and Intersect at Retrieving values ​​from a two-dimensional mean elevation model The elevation of the grid cell in question is calculated. The length of is given by the following formula:

[0137] ;

[0138] in, For line segments Length; for Elevation; for Elevation; for Cartesian coordinates in the plane .

[0139] Step 5 and resemblance, 3D coordinates The calculation formula is as follows:

[0140] .

[0141] Step 6: Repeat steps 4 and 5 above until... and If the height is less than a specific threshold, select... As a three-dimensional corner point, apply the above steps By using the horizontal image coordinates of any corner point, the three-dimensional coordinates of each corner point can be calculated, enabling the preliminary positioning of corner points in the photovoltaic area from a single aerial image taken by a surveying drone. .

[0142] Furthermore, in a specific implementation, in the UAV flight path planning method provided in the embodiments of the present invention, step S103 performs principal component analysis on the lidar point cloud surrounding the initial three-dimensional coordinates of the corner point to obtain the optimized three-dimensional coordinates of the corner point. Specifically, this may include: searching for lidar point clouds in multiple surrounding grid cells centered on the grid cell containing the projection point of the corner point on the ground; determining the vertical distance from the searched lidar point cloud to the target line; the target line is the line between the projection center and the projection point of the corner point on the ground; filtering out the lidar point clouds corresponding to those with a vertical distance less than a set distance value, and calculating the mean coordinates of the filtered lidar point clouds; obtaining the covariance matrix of the filtered lidar point clouds based on the mean coordinates; calculating the eigenvalues ​​and eigenvectors of the covariance matrix using principal component analysis to construct new three-dimensional coordinates; and determining the coordinates of the corner point in the new three-dimensional coordinates using geometric constraints combined with displacement parameters to obtain the optimized three-dimensional coordinates of the corner point.

[0143] In practice, the two-dimensional elevation mean model generated from the original airborne radar point cloud is a fixed-resolution raster image, with each pixel storing a single elevation value, which cannot reflect the terrain changes within a pixel. When projecting the UAV image, each pixel of the two-dimensional elevation mean model is treated as a horizontal plane by default, resulting in texture distortion. This affects the positioning accuracy of the corner points of the selected component area in the original single image taken by the UAV. If the raster size is reduced to approximate the real terrain, the amount of data will surge, and the computational efficiency will decrease. Therefore, this invention uses principal component analysis of the original airborne lidar point cloud around the corner points of the selected component area to achieve accurate positioning of the corner points of the selected component area.

[0144] Figure 7 This is a schematic diagram illustrating precise corner point positioning provided in an embodiment of the present invention. Figure 7 As shown, It is the raster cell in the i-th row and j-th column of the two-dimensional mean elevation model; It is obtained by the geometric fusion positioning method described above. ground projection point, yes The precise positioning point is determined by the following steps:

[0145] First, with the inclusion point Centered on the grid cell, search for all point clouds in the surrounding 9 grid cells.

[0146] Then, set the threshold. And filter from the search points that match the line The vertical distance is less than If the number of selected points is less than 3, precise positioning is not required; if it is 3 or more, calculate the average coordinates of the selected points using the following formula:

[0147] .

[0148] Next, the covariance matrix of the selected points is calculated using the following formula:

[0149] ;

[0150] because: We can obtain:

[0151] ;

[0152] in, The mean coordinates of the corner points of the selected component area; Let be the three-dimensional coordinates of the i-th point among the selected points. , where n is the number of points being selected; It is the covariance matrix of the selected points.

[0153] Subsequently, principal component analysis was used to calculate the covariance matrix. eigenvalues and eigenvectors eigenvalues The characteristic equation is as follows:

[0154] ;

[0155] Solve for eigenvalues , .

[0156] For eigenvalues Solve the system of linear equations:

[0157] ;

[0158] The solution is obtained by Gaussian elimination and normalization to obtain the unit eigenvector. The principal component directions that constitute PCA.

[0159] Calculate the correction parameters. Figure 7 middle, It is a three-dimensional coordinate system. The origin is the coordinate system of the above-mentioned mean coordinates. ; It is the coordinate axis where the feature vector is located.

[0160] Located in the plane Principal plane and line superior; , and For the parameters to be solved, the projection points are... Relative to the above mean coordinates The displacement vector is transformed to In the coordinate system formed, The coordinates of the displacement vector under the mixed basis matrix are calculated using the following formula:

[0161] ;

[0162] in, The geometric fusion positioning method obtained ground projection point The coordinates; For feature vectors With line constraints The mixed basis matrix is ​​formed; It is the unit direction vector in the CS direction.

[0163] Finally, the coordinates are solved using geometric constraints. Because Using the above mean coordinates As a reference point, located on the plane Main plane, , As parameters; with projection points As a reference point, it lies on the straight line. superior, The parameters are respectively satisfied by the following equations:

[0164] ;

[0165] Substitution It can be calculated The three-dimensional coordinates enable precise positioning of corner points of selected component areas on a single original aerial image taken by a surveying drone. .

[0166] Furthermore, in a specific implementation, in the above-mentioned UAV route planning method provided in this embodiment of the invention, step S104 clusters and fuses the corner points of multiple images to obtain a fused optimal set of corner point coordinates. The optimal set of corner point coordinates and the optimized set of three-dimensional coordinates are then superimposed on the orthophoto file. After verification, a set of regional corner point coordinates is formed and grouped and sorted. Specifically, this may include: performing a density-based spatial clustering algorithm with noise on multiple images. Clustering using Noise (DBSCAN) is performed. Successfully clustered corner point clusters are labeled, with corner points within the same cluster considered as observations of the same physical corner point. The cluster coordinate set is output, while the coordinates of unclustered corner points are retained. Based on the cluster coordinate set, inliers are selected from the clusters, and the optimal corner point coordinate set for each corner point cluster after robust fusion is obtained using inliers. ArcGIS is used to overlay the optimal corner point coordinate set, the optimized 3D coordinate set, and the coordinates of unclustered corner points in the orthophoto file, and it is determined whether the optimal corner point coordinate set contains all component area corner points. If not, the missing point coordinates are formed. The optimal corner point coordinate set, the unclustered corner points to be retained, and the missing point coordinates are combined and output as the area corner point coordinate set. ArcGIS is used to overlay the area corner point coordinate set in the orthophoto file. Based on the global component layout and the cleaning drone operation sequence, the area corner point coordinate set is grouped and numbered. The grouped area corner point coordinates are uniformly sorted in clockwise or counterclockwise order.

[0167] In practice, this invention can pre-position the precise corner points of component areas in all individual images of this aerial photography. Transform the image to the same coordinate system as the orthophoto, ensuring consistent units. The following steps can then be performed:

[0168] First, corner point association matching is performed: spatial proximity. Corner points from different images are clustered using DBSCAN, with the coordinate projection error of the same physical corner point set to be less than the neighborhood radius (based on RTK accuracy). Successfully clustered corner point clusters are labeled, and corner points within the same cluster are considered observations of the same physical corner point. The set of coordinates within each cluster is output. The coordinates of corner points that are not successfully clustered are retained and verified as noise in post-processing and validation steps.

[0169] Then, the optimal coordinate fusion algorithm is executed. Two points are randomly selected from the cluster, and the Euclidean distance from all points to the mean of these two points is calculated. If the Euclidean distance residual is less than the inlier residual threshold (set according to RTK accuracy), the observation point is marked as an inlier; otherwise, it is discarded. This process is repeated, randomly selecting two points from the cluster and performing the above calculation. If the current number of inliers is greater than the optimal number of inliers, the two randomly selected points are updated. The process stops when the maximum number of iterations is reached or the proportion of inliers exceeds the threshold. Finally, the optimal coordinates are re-estimated using all inliers. The optimal corner coordinate set for each corner cluster after robust fusion is output. .

[0170] Next, post-processing and visualization verification are performed. This involves overlaying a precise set of corner locations for the component regions of all individual images onto the orthophoto using ArcGIS. , which is the set of intersection coordinates of successfully clustered points and the optimal set of corner coordinates for each fused corner cluster. The data is represented by layers of different colors. The optimal corner coordinate set is checked to see if it includes all corners of the component area, and it is determined whether corners that did not successfully cluster should be retained. If a photovoltaic module area on the orthophoto only has corners that did not successfully cluster, then those corners are retained, and the rest are removed. If a photovoltaic module area on the orthophoto does not have an optimal corner, then the corner is manually selected in the orthophoto file, its latitude and longitude are obtained, and the elevation value at that latitude and longitude is found in the radar point cloud file to form the coordinates of the missing point.

[0171] Then, the optimal corner coordinates, the corners that were not successfully clustered and the missing points that were added are output, which together form the set of corner coordinates of the region.

[0172] Finally, the area corner point coordinate sets are grouped. Using ArcGIS, the area corner point coordinate sets are overlaid on the orthophoto file. Based on the global component layout and the order of the cleaning drone operations, the area corner point coordinate sets are grouped and numbered. The grouped area corner point coordinates are then uniformly sorted in either clockwise or counterclockwise order.

[0173] Furthermore, in a specific implementation, in the above-mentioned UAV route planning method provided in the embodiments of the present invention, step S105 generates a two-dimensional route file based on the coordinates of the corner points of the grouped and sorted regions and the route spacing, and explicitly specifies the absolute elevation in the two-dimensional route file to obtain the target cleaning route file. Specifically, it may include: generating a grouped closed polygon KML file based on the coordinates of the corner points of the grouped and sorted regions, calculating the main direction angle, and generating parallel routes within the grouped regions in combination with the route spacing, generating a two-dimensional route file after connecting the first and last regions; explicitly specifying the absolute elevation in the two-dimensional route file to generate a three-dimensional waypoint file, importing radar point clouds and performing inter-waypoint line-of-sight analysis to generate the target cleaning route file.

[0174] In implementation, the PythonSimplekml library is used to import the corner coordinates of the grouped regions and generate a KML file of the grouped closed polygons.

[0175] The Shapely library is used to calculate the direction vector of the longest side of each group of polygons, which is then used as the principal direction angle. The input route spacing generates parallel routes within the grouped regions. The endpoint of a parallel route within a certain group is connected to the starting point of a parallel route in the next group. Connecting multiple regions end-to-end generates a two-dimensional route file.

[0176] Save the flight path KML file and overlay the KML file with orthophotos to check the rationality of the flight path spacing. Because the relative flight altitude between the cleaning drone and its components is relatively low during cleaning drone operations to ensure cleaning effectiveness, and to balance operational safety, the absolute elevation of the waypoints is the sum of the maximum elevation of the corner coordinates of each group of areas and the required relative flight altitude between the cleaning drone and its components.

[0177] Explicitly specify the absolute elevation in the route KML file, generate a 3D waypoint file, import the radar point cloud and perform a line-of-sight analysis between waypoints, check the safety of the flight path, and the 3D route file that passes the line-of-sight analysis can be used to generate the final route file.

[0178] Therefore, the UAV flight path planning method provided by the present invention may include: after acquiring boundary images and radar data of the component area to be cleaned by aerial photography of the surveying UAV, stitching them together and outputting an orthophoto file; rasterizing the airborne lidar point cloud and calculating the elevation mean of each grid cell to generate an accurate two-dimensional elevation mean model; correcting the tilted original single aerial images of the surveying UAV to generate corresponding horizontal original single aerial images; using a Log-Gabor filter and phase consistency edge detection method based on the single image to identify component edges and corners, and optimizing the sub-pixel accuracy of the corners; projecting the identified corner horizontal image coordinates onto the two-dimensional elevation mean model using geometric fusion to obtain the preliminary three-dimensional positioning of the corners; and performing principal component analysis on the lidar point cloud around the three-dimensional coordinates of the corners to achieve accurate three-dimensional positioning of the corners in the component area. Based on DBSCAN clustering and robust fusion, and leveraging the holistic nature of orthophotos, the optimal corner coordinates of multiple images are fused, verified, and supplemented. The optimal corner coordinates, the corners that were not successfully clustered and need to be retained, and the coordinates of the supplemented missing points are combined to form and output a set of regional corner coordinates. ArcGIS is used to overlay this set of regional corner coordinates onto the orthophoto file. Based on the global component layout and the cleaning UAV operation sequence, the regional corner coordinates are grouped and numbered, with the grouped regional corner coordinates uniformly sorted clockwise or counterclockwise. The grouped regional corner coordinates are imported to generate grouped closed polygon KML files. The principal direction angle is calculated, the flight path spacing is input, and parallel flight paths are generated within the grouped regions. Multiple regions are connected end-to-end to generate a 2D flight path file. The flight path KML files are saved, and the orthophotos are overlaid onto the KML files to check the rationality of the flight path spacing. The absolute elevation is explicitly specified in the flight path KML files to generate 3D waypoint files. Radar point clouds are imported, and inter-waypoint visibility analysis is performed to generate the final target cleaning flight path file.

[0179] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0180] Embodiments of the present invention also provide a drone flight path planning device. Based on functional modules, this device includes:

[0181] The data acquisition module is used to acquire single images of the boundary of the component area to be cleaned and LiDAR point cloud data, stitch them together and output orthophoto files;

[0182] The data processing module is used to rasterize the lidar point cloud data, calculate the average elevation of each raster unit to generate a two-dimensional average elevation model, and perform component edge extraction and corner point recognition on a single image to obtain the corner point recognition results.

[0183] The geometric fusion analysis module is used to project the corner recognition results onto a two-dimensional elevation mean model using a geometric fusion method, obtain the preliminary three-dimensional coordinates of the corner, and perform principal component analysis on the lidar point cloud around the preliminary three-dimensional coordinates of the corner to obtain the optimized three-dimensional coordinates of the corner.

[0184] The clustering fusion verification module is used to acquire multiple images of the boundary of the component region to be cleaned and to cluster and fuse the corner points of the multiple images to obtain the optimal set of corner point coordinates after fusion. The optimal set of corner point coordinates and the optimized set of three-dimensional coordinates are superimposed on the orthophoto file. After verification, the set of corner point coordinates of the region is formed and sorted by group.

[0185] The route file generation module is used to generate a two-dimensional route file based on the coordinates of the corner points of the grouped and sorted areas and the route spacing. The absolute elevation is explicitly specified in the two-dimensional route file to obtain the target cleaning route file.

[0186] In the UAV route planning device provided in this embodiment of the invention, the interaction of the above five modules enables component edge and corner point recognition based on a single image, component area corner point positioning by fusing LiDAR point cloud data, and global layout analysis of component corner points by orthophotos of the component area, generating a high-precision route for cleaning UAVs. This achieves fine recognition of component edges and corner points using a single image, improves the accuracy of corner point positioning by using LiDAR point cloud data, and ensures the completeness of global corner point layout analysis by relying on orthophotos. The final generated target cleaning route file containing explicit absolute elevation has higher accuracy and stronger adaptability, providing precise guidance for UAV cleaning operations, effectively improving cleaning efficiency and quality, and reducing operational deviations and errors.

[0187] Since the embodiments of the UAV flight path planning device and the UAV flight path planning method correspond to each other, the descriptions of the features in the embodiments corresponding to the UAV flight path planning device can be found in the relevant descriptions of the embodiments corresponding to the UAV flight path planning method, and will not be repeated here. Furthermore, it has the same beneficial effects as the aforementioned UAV flight path planning method.

[0188] Embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the UAV route planning method.

[0189] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program configured to execute the steps in any of the above embodiments of the UAV route planning method at runtime.

[0190] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0191] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the UAV route planning method.

[0192] Embodiments of the present invention also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above embodiments of the UAV route planning method.

[0193] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0194] The present invention has provided a detailed description of a method, device, and medium for unmanned aerial vehicle (UAV) route planning. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of these embodiments are only intended to aid in understanding the method and core ideas of the invention. It should be noted that those skilled in the art can make various improvements and modifications to the invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A method for planning flight routes for unmanned aerial vehicles (UAVs), characterized in that, The method comprises the following steps: Obtain a single image and laser radar point cloud data of the region boundary of the component to be cleaned, splice and output an orthographic image file; Grid the laser radar point cloud data, calculate the elevation mean value of each grid unit to generate a two-dimensional elevation mean value model, extract the component edge and identify the corner point of the single image to obtain a corner point identification result; Project the corner point identification result onto the two-dimensional elevation mean value model by using a geometric fusion method to obtain the preliminary three-dimensional coordinates of the corner point, and perform principal component analysis on the laser radar point cloud around the preliminary three-dimensional coordinates of the corner point to obtain the optimized three-dimensional coordinates of the corner point; Obtain multiple images of the region boundary of the component to be cleaned, cluster and fuse the corner points of the multiple images to obtain a fused optimal corner point coordinate set, superimpose the optimal corner point coordinate set and the set of optimized three-dimensional coordinates in the orthographic image file, and after verification, form a region corner point coordinate set and perform grouping and sorting; Generate a two-dimensional flight line file according to the region corner point coordinates after grouping and sorting and the flight line spacing, and explicitly specify the absolute elevation in the two-dimensional flight line file to obtain a target cleaning flight line file. 2.The method of claim 1, wherein, The method for calculating the elevation mean value of each grid unit to generate a two-dimensional elevation mean value model comprises the following steps: According to the number of points contained in the grid unit, the grid unit is divided into an empty unit containing no points, a sparse unit containing one to two points, and a dense unit containing more than three points; Centering on the empty unit, the elevation mean value of the empty unit is calculated by multi-directional interpolation; Centering on the sparse unit, the elevation mean value of the sparse unit is calculated by using distance weighting method; The dense unit is fitted by using median filtering method to obtain the elevation mean value of the dense unit; After calculating the elevation mean value of all grid units, the elevation mean value of each grid unit is taken as the elevation value of the local region covered by itself to splice and form a two-dimensional elevation mean value model. 3.The method of claim 1, wherein, The method for extracting the component edge and identifying the corner point of the single image to obtain a corner point identification result comprises the following steps: Perform vertical angle correction on the single image to generate a corresponding horizontal image; Use a multi-scale filter set and a phase consistency edge detection method to extract the component edge and identify the corner point of the horizontal image, and perform corner point sub-pixel optimization to obtain a corner point identification result. 4.The method of claim 3, wherein, The method for extracting the component edge and identifying the corner point of the horizontal image by using a multi-scale filter set and a phase consistency edge detection method and performing corner point sub-pixel optimization to obtain a corner point identification result comprises the following steps: Convert the horizontal image to a grayscale image and perform normalization processing; Convert the normalized image to the frequency domain by Fourier transform, filter by a multi-scale Log-Gabor filter set, and inverse discrete Fourier transform to obtain an empty domain complex response; According to the empty domain complex response, accumulate the phase consistency, extract the edge feature map after non-maximum suppression and threshold adjustment, and output the horizontal image coordinates of the integer-level component region corner point; The horizontal image coordinates of the integer-level component region corner points are taken as the center, a window of a set size is set to calculate the horizontal gradient and the vertical gradient to obtain the gradient vector and the Hessian matrix, and the horizontal image coordinates of the corner points are iteratively solved by the second-order Taylor expansion of the gray change to update the offset until the offset is less than a preset threshold or the maximum number of iterations is reached; The RANSAC algorithm is used to verify the consistency of the gradient direction of the corner points, and the maximum inlier set is reserved to obtain the corner point recognition result. 5.The method of claim 3, wherein, The corner point recognition result is projected onto the two-dimensional elevation mean model in a geometric fusion manner to obtain the preliminary three-dimensional coordinates of the corner points, including: Converting the single image and the two-dimensional elevation mean model to the same coordinate system; Obtaining the three-dimensional coordinates of the projection center of the camera, extending the line between the projection center and the principal point of the horizontal image to the ground to obtain a first intersection point, retrieving a first elevation of the grid cell where the first intersection point is located from the two-dimensional elevation mean model, and calculating the vertical distance from the projection center to the first intersection point; Drawing a straight line from the projection center to the corner point on the horizontal image and extending it to the ground, and drawing a horizontal line from the first intersection point, and the intersection point of the straight line and the horizontal line is a second intersection point; According to the vertical distance from the projection center to the first intersection point, the horizontal image coordinates of the corner point and the plane coordinates of the projection center, the three-dimensional coordinates of the second intersection point are calculated to obtain the projection point coordinates of the second intersection point on the ground; Retrieving the second elevation of the grid cell where the projection point corresponding to the second intersection point is located from the two-dimensional elevation mean model, and calculating the difference between the first elevation and the second elevation; Using the difference between the first elevation and the second elevation, the next projection point coordinates and the difference in elevation between the adjacent two projection points are calculated; The steps of projection point coordinate calculation and difference calculation are repeatedly performed until the difference in elevation between the adjacent two projection points is less than a set threshold to obtain the preliminary three-dimensional coordinates of the corner points. 6.The method of claim 1, wherein, Principal component analysis is performed on the laser radar point cloud around the preliminary three-dimensional coordinates of the corner points to obtain the optimized three-dimensional coordinates of the corner points, including: Taking the grid cell containing the projection point of the corner point on the ground as the center, searching for the laser radar point cloud in the surrounding multiple grid cells; Determining the vertical distance from the searched laser radar point cloud to the target straight line; the target straight line is the straight line between the projection center and the projection point of the corner point on the ground; Screening out the laser radar point cloud corresponding to the determined vertical distance less than a set distance value, and calculating the mean coordinates of the screened laser radar point cloud; According to the mean coordinates, the covariance matrix of the screened laser radar point cloud is obtained; The eigenvalues and eigenvectors of the covariance matrix are calculated by the principal component analysis method to construct new three-dimensional coordinates; Using geometric constraints combined with displacement parameters, the coordinates of the corner point in the new three-dimensional coordinates are determined to obtain the optimized three-dimensional coordinates of the corner point. 7.The method of claim 1, wherein, The corner points of the multiple images are clustered and fused to obtain a fused optimal corner point coordinate set, and the optimal corner point coordinate set and the set of optimized three-dimensional coordinates are superimposed in the orthophoto file, and after verification, a region corner point coordinate set is formed and grouped and sorted, including: DBSCAN clustering is performed on the plurality of images, and a label is given to a corner point cluster that is successfully clustered. Corner points in the same cluster are regarded as observation values of the same physical corner point, and a coordinate set in the cluster is output. Corner points that are not successfully clustered are retained; According to the coordinate set in the cluster, an in-point is selected from the cluster, and an optimal corner point coordinate set of each corner point cluster after robust fusion is obtained using the in-point; The optimal corner point coordinate set, the set of optimized three-dimensional coordinates, and the coordinates of corner points that are not successfully clustered are superimposed in the orthographic image file using ArcGIS, and it is determined whether the optimal corner point coordinate set contains all component area corner points. If not, missing point coordinates are formed; The optimal corner point coordinate set, the corner points that need to be retained and the missing point coordinates are collectively formed and output as an area corner point coordinate set; The area corner point coordinate set is superimposed in the orthographic image file using ArcGIS, the area corner point coordinate set is grouped and numbered according to the global component arrangement form and the cleaning unmanned aerial vehicle operation sequence, and the grouped area corner point coordinates are sorted in a clockwise or counterclockwise direction. 8.The method of claim 1, wherein, According to the grouped and sorted area corner point coordinates and the flight line spacing, a two-dimensional flight line file is generated, and an absolute elevation is explicitly specified in the two-dimensional flight line file to obtain a target cleaning flight line file, including: A grouped closed polygon KML file is generated according to the grouped and sorted area corner point coordinates, a main direction angle is calculated, and parallel flight lines in the grouped area are generated in combination with the flight line spacing. After connecting the first and last areas, a two-dimensional flight line file is generated; An absolute elevation is explicitly specified in the two-dimensional flight line file to generate a three-dimensional waypoint file, radar point clouds are imported and visibility analysis between waypoints is performed to generate a target cleaning flight line file.

9. An electronic device, comprising: The computer readable storage medium stores a computer program, and the processor executes the computer program to implement the steps of the unmanned aerial vehicle flight path planning method according to any one of claims 1 to 8. The computer readable storage medium stores a computer program, and the processor executes the computer program to implement the steps of the unmanned aerial vehicle flight path planning method according to any one of claims 1 to 8. The computer readable storage medium stores a computer program, and the processor executes the computer program to implement the steps of the unmanned aerial vehicle flight path planning method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, ​

Images