Unmanned aerial vehicle route planning method, equipment and medium

Abstract

The invention discloses an unmanned aerial vehicle route planning method and device and a medium, and relates to the field of photovoltaic cleaning, and the method comprises the steps: firstly obtaining a single image of a to-be-cleaned assembly region and a laser radar point cloud, and splicing the single image and the laser radar point cloud to output an orthoimage; a two-dimensional elevation mean value model is generated after point cloud rasterization, and component edges and recognition angular points are extracted from an image; angular points are projected to the model in a geometric fusion mode to obtain initial three-dimensional coordinates of the angular points, and the initial three-dimensional coordinates are optimized through principal component analysis; clustering and fusing angular points of a plurality of images, overlapping the angular points and the optimized coordinates to an orthoimage for verification, forming a regional angular point coordinate set, and grouping and sequencing the regional angular point coordinate set; and finally, generating a two-dimensional route according to the route spacing, and explicitly assigning the absolute elevation to obtain a target cleaning route. In this way, multi-source data are fused, a single image is used for finely recognizing angular points, the laser radar point cloud is used for improving the positioning precision, overall analysis completeness is guaranteed through orthoimages, the generated route is high in precision and high in adaptability, the unmanned aerial vehicle can be accurately guided to clean, and efficiency and quality are improved.

Description

TECHNICAL FIELD

[0001] The application relates to the field of photovoltaic cleaning, in particular to a method, device and medium for planning a flight path of an unmanned aerial vehicle. BACKGROUND

[0002] Unmanned aerial vehicle cleaning is gradually popularized in the field of photovoltaic cleaning, and has the advantages of not directly contacting components, high cleaning efficiency, high automation degree and low labor cost, but current manual visual operation of unmanned aerial vehicles is still the main method, and there are great difficulties in flight path planning.

[0003] Traditional flight path planning of unmanned aerial vehicles for photovoltaic power stations only plans parallel arc flight paths according to the polygon boundary of the entire station, and is suitable for whole-station aerial photography or power station inspection. Since photovoltaic components have various arrangement forms, the flight path planned in this way cannot guarantee that the cleaning unmanned aerial vehicle flies above the components to complete the cleaning task, and is not suitable for application scenarios of photovoltaic component cleaning by unmanned aerial vehicles. In order to form a cleaning flight path above the components, a cleaning unmanned aerial vehicle or a dotter is generally used to mark points at key points of each flight path on site of the photovoltaic power station, but this method is time-consuming and costly. Alternatively, key flight path points are manually selected or component boundary points are identified by an algorithm relying on images and surface models. Although image recognition is mature, model interpolation is prone to edge elevation deviation, and problems such as misalignment and registration error also reduce positioning accuracy. SUMMARY

[0004] The purpose of the present application is to provide a method, device and medium for planning a flight path of an unmanned aerial vehicle, which can realize fine identification, accurate positioning and global layout analysis of component corner points, and generate a high-precision target cleaning flight path.

[0005] To solve the above technical problems, the present application provides a method for planning a flight path of an unmanned aerial vehicle, comprising: acquiring a single image and laser radar point cloud data of a component region boundary to be cleaned, splicing and outputting an orthographic image file; performing rasterization on the laser radar point cloud data, calculating the elevation mean value of each raster unit to generate a two-dimensional elevation mean value model, performing component edge extraction and corner point identification on the single image to obtain a corner point identification result; projecting the corner point identification result onto the two-dimensional elevation mean value model in a geometric fusion manner to obtain preliminary three-dimensional coordinates of the corner points, and performing principal component analysis on laser radar point clouds around the preliminary three-dimensional coordinates of the corner points to obtain optimized three-dimensional coordinates of the corner points; acquiring multiple images of the component region boundary to be cleaned, clustering and fusing the corner points of the multiple images to obtain a fused optimal corner point coordinate set, and superimposing the optimal corner point coordinate set and the set of optimized three-dimensional coordinates in the orthographic image file, to form a regional corner point coordinate set after verification and grouping and sorting; According to the region corner point coordinates after grouping and sorting and the flight line interval, a two-dimensional flight line file is generated, and an absolute elevation is explicitly specified in the two-dimensional flight line file, so as to obtain a target cleaning flight line file.

[0006] To solve the above technical problems, the application further provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the unmanned aerial vehicle flight path planning method.

[0007] To solve the above technical problems, the application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the unmanned aerial vehicle flight path planning method.

[0008] As can be seen from the above technical solutions, the unmanned aerial vehicle flight path planning method provided by the application comprises the following steps: firstly, a single image and laser radar point cloud data of a region boundary of a component to be cleaned are acquired, and a orthographic image file is spliced and output; then, the laser radar point cloud data is rasterized, the elevation average of each raster unit is calculated to generate a two-dimensional elevation average model; the component edge of the single image is extracted and the corner points are identified to obtain a corner point identification result. Then, the corner point identification result is projected onto the two-dimensional elevation average model in a geometric fusion manner to obtain a preliminary three-dimensional coordinate of the corner point, and principal component analysis is performed on the laser radar point cloud around the preliminary three-dimensional coordinate of the corner point to obtain an optimized three-dimensional coordinate of the corner point. Subsequently, multiple images of the region boundary of the component to be cleaned are acquired, and the corner points of the multiple images are clustered and fused to obtain a fused optimal corner point coordinate set. The optimal corner point coordinate set and the set of optimized three-dimensional coordinates are superimposed in the orthographic image file, and after verification, a region corner point coordinate set is formed and grouped and sorted. Finally, according to the region corner point coordinates after grouping and sorting and the flight line interval, a two-dimensional flight line file is generated, and an absolute elevation is explicitly specified in the two-dimensional flight line file, so as to obtain a target cleaning flight line file.

[0009] The unmanned aerial vehicle flight path planning method provided by the application has the advantages that the component edge and corner point identification are carried out based on a single image, the component region corner point positioning is carried out by fusing laser radar point cloud data, the global layout analysis of the component corner point is carried out through the component region orthographic image, a high-precision cleaning unmanned aerial vehicle flight path is generated, and through multi-source data fusion and accurate processing, the component edge and corner point are finely identified by using a single image, the accuracy of the corner point positioning is improved through laser radar point cloud data, and the integrity of the global layout analysis of the corner point is ensured by relying on the orthographic image. Finally, the target cleaning flight line file containing an explicit absolute elevation has higher precision and stronger adaptability, can provide accurate guidance for the unmanned aerial vehicle cleaning operation, effectively improves the cleaning efficiency and quality, and reduces operation deviation and errors.

[0010] In addition, the present application also provides a corresponding unmanned aerial vehicle route planning electronic device and a computer readable storage medium for the unmanned aerial vehicle route planning method, which have the same or corresponding technical features as the above-mentioned unmanned aerial vehicle route planning method and the same effects. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0012] Figure 1 The flowchart of the unmanned aerial vehicle route planning method provided by the embodiments of the present application is shown in the figure. Figure 2 The route schematic diagram of the surveying and mapping unmanned aerial vehicle collecting data in the boundary of the component area to be cleaned is shown in the figure. Figure 3 The frame schematic diagram of the unmanned aerial vehicle route planning method provided by the embodiments of the present application is shown in the figure. Figure 4 The vertical angle correction schematic diagram of a single image provided by the embodiments of the present application is shown in the figure. Figure 5 The frame schematic diagram of the component edge and corner point identification based on a single image provided by the embodiments of the present application is shown in the figure. Figure 6 The corner point geometric fusion positioning schematic diagram provided by the embodiments of the present application is shown in the figure. Figure 7 The corner point accurate positioning schematic diagram provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0013] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0014] It should be noted that in the description of the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or equipment. The terms "first", "second" and the like in the present application are used to distinguish similar objects, not to describe a specific order or sequence.

[0015] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below in conjunction with the drawings and specific embodiments.

[0016] In conjunction with the specific application environment architecture or specific hardware architecture on which the UAV route planning method is executed, the specific application environment architecture or specific hardware architecture is described here.

[0017] The embodiment of the present application provides a UAV route planning method, and the method is described in detail in conjunction with the execution process of the UAV route planning method. Figure 1 The flowchart of the UAV route planning method provided by the embodiment of the present application is shown in Figure 1 The method comprises the following steps. S101, acquiring a single image of a to-be-cleaned component region boundary and laser radar point cloud data, splicing and outputting an orthographic image file.

[0018] It should be noted that the original resolution of the single image is high, and the edge details are clear without compression or resampling. Based on the component edge and corner point recognition of the single image, the speed is the fastest, the accuracy is the highest, and the plane positioning accuracy of the corner point is closest to the positioning accuracy of the Real-Time Kinematic (RTK) positioning mode.

[0019] In implementation, a surveying and mapping UAV can be used to collect picture data and laser radar point cloud data of the to-be-cleaned component region boundary, thereby reducing the workload of whole-station aerial modeling. Then, the surveying and mapping UAV aerial original single high-resolution RGB image with Position and Orientation System (POS) data is stored.

[0020] Figure 2 The flight route schematic diagram of the surveying and mapping UAV for collecting data at the to-be-cleaned component region boundary is provided by the embodiment of the present application. As shown in Figure 2 The surveying and mapping UAV can survey according to the routes from 1 to 2, 2 to 3, 3 to 4, 4 to 1 and 5, 5 to 6, 6 to 7, 7 to 8, and 8 to 9. Figure 2The black dots in the figure represent the corner points of the component area. The application can use software tools such as Pix4D, ContextCapture, etc. to import aerial photos with POS data, perform aerial triangulation and orthorectification, splice orthographic images and output Georeferenced Tagged Image File Format (GeoTIFF) files, and import the orthographic image GeoTIFF file into an Arc Geographic Information System (ArcGIS).

[0021] S102, rasterize the laser radar point cloud data, calculate the elevation mean value of each grid unit to generate a two-dimensional elevation mean value model; perform component edge extraction and corner point recognition on a single image to obtain a corner point recognition result.

[0022] In implementation, the above steps are respectively targeted processing of laser radar point cloud and single image two types of core data, laying a foundation for subsequent multi-source data fusion.

[0023] Figure 3 The framework diagram of the unmanned aerial vehicle route planning method provided by the embodiment of the application is shown in FIG. 1. Figure 3 As shown in FIG. 1, after collecting images and radar data, the laser radar point cloud data can be processed by two-dimensional rasterization, and then the elevation mean value of all point clouds in each grid unit (i.e. grid unit) is calculated, which can smooth the noise in the point cloud data, and finally a two-dimensional elevation mean value model is generated. The single image can be subjected to component edge extraction and corner point recognition, which is to capture the contour boundary and key corner points of the component to be cleaned by using image processing algorithms, and finally obtain a corner point recognition result.

[0024] S103, project the corner point recognition result onto the two-dimensional elevation mean value model in a geometric fusion manner to obtain the preliminary three-dimensional coordinates of the corner point, and perform principal component analysis on the laser radar point cloud around the preliminary three-dimensional coordinates of the corner point to obtain the optimized three-dimensional coordinates of the corner point.

[0025] In implementation, the above steps are the key conversion process of realizing the three-dimensional coordinates of the corner points from two-dimensional to three-dimensional, from preliminary to accurate. First, through geometric fusion, the two-dimensional corner points identified in the single image are projected onto the two-dimensional elevation mean model generated by the laser radar point cloud, and with the aid of the elevation data contained in the model, the height information is given to the two-dimensional corner points, thereby obtaining the preliminary three-dimensional coordinates of the corner points, and completing the mapping from the image plane to the three-dimensional space. Subsequently, principal component analysis is performed on the laser radar point cloud data around the preliminary three-dimensional coordinates, which can effectively extract the main distribution characteristics and spatial structure of the point cloud in this region. By excluding noise points, fitting the optimal spatial position, etc., the possible deviation of the preliminary three-dimensional coordinates is corrected, and finally the optimized three-dimensional coordinates with higher precision and more in line with the actual physical position are obtained, which not only retains the feature accuracy of the image recognition corner points, but also takes advantage of the laser radar point cloud in three-dimensional positioning.

[0026] S104, a plurality of images of the region boundary of the component to be cleaned are acquired, and the corner points of the plurality of images are clustered and fused to obtain a set of optimal corner point coordinates after fusion, and the set of optimal corner point coordinates and the set of optimized three-dimensional coordinates are superimposed in the orthographic image file, and after verification, a set of region corner point coordinates is formed and grouped and sorted.

[0027] In implementation, the above steps are the integration process from local corner point data to global unified corner point set, which is through complementary of multi-image data and global verification to ensure the integrity, consistency and availability of the corner point coordinates. First, a plurality of images of the region boundary of the component to be cleaned are acquired, which is to make up for the problem of missed / misidentified corner points caused by limited field of view, angle deviation or local occlusion of a single image, and to collect corner point data under different angles to form multi-source local samples; the corner points of these multiple images are clustered and fused, which is to use algorithms to classify different image recognition results of the same physical corner point with similar features into one category, and to eliminate abnormal values with repeated or excessive errors, and finally to generate a set of optimal corner point coordinates to ensure the unity of the corner point position. Then, the optimal set and the set of optimized three-dimensional coordinates obtained in step S103 are superimposed into the orthographic image file, which is to verify whether the spatial distribution of the corner points conforms to the actual structure of the component through the global field of view and geographical reference of the orthographic image, and to verify and exclude contradictory data; after verification, the effective corner points are integrated into a set of region corner point coordinates and grouped and sorted, which is to logically classify and arrange the corner points according to the physical partition of the component, to lay a foundation for generating ordered and accurate cleaning routes based on the corner points, and to ensure that the routes can cover the whole region and conform to the component layout.

[0028] S105, according to the grouped and sorted region corner point coordinates and the route spacing, a two-dimensional route file is generated, and an absolute elevation is explicitly specified in the two-dimensional route file to obtain a target cleaning route file.

[0029] In implementation, the above steps are the link from the corner point data to the actual executable unmanned aerial vehicle operation instruction. First, the group-sequenced region corner point coordinates provide path anchor points for route planning, and the group-sequenced corner points according to the physical layout of the component define the coverage range and path direction of the unmanned aerial vehicle cleaning operation; and the route spacing is a key parameter for controlling the cleaning density, ensuring that there is neither redundant repetition nor cleaning blind area between adjacent routes. Based on the two bases, a two-dimensional route file containing the route path can be generated. Subsequently, the absolute elevation is explicitly specified in the two-dimensional route file, in order to make up for the deficiency of the two-dimensional route containing only plane coordinates, avoid the problems of unmanned aerial vehicle collision and improper height affecting cleaning effect due to the lack of elevation information, and finally form a target cleaning route file containing both plane path and vertical height information, which can be directly imported into the unmanned aerial vehicle control system to realize automatic and high-precision cleaning operation.

[0030] In the above unmanned aerial vehicle route planning method provided by the embodiment of the application, the component edge and corner point recognition is carried out based on a single image, the component region corner point positioning is carried out by fusing laser radar point cloud data, the global layout analysis of the component corner point is carried out through the component region orthographic image, and the high-precision cleaning unmanned aerial vehicle route is generated. Through multi-source data fusion and accurate processing, the component edge and corner point can be finely recognized by using a single image, the accuracy of corner point positioning is improved through laser radar point cloud data, and the integrity of global layout analysis of the corner point is ensured by relying on the orthographic image. Finally, the target cleaning route file containing the explicit absolute elevation is generated, which is more accurate and has stronger adaptability, can provide accurate guidance for unmanned aerial vehicle cleaning operation, effectively improve the cleaning efficiency and quality, and reduce operation deviation and errors.

[0031] Further, in the above unmanned aerial vehicle route planning method provided by the embodiment of the application, the step S102 rasterizes the laser radar point cloud data, which can specifically include: the original plane rectangular coordinates of each point in the laser radar point cloud data are stored in the corresponding position after the column index and the row index of the corresponding two-dimensional grid are obtained by calculation; wherein the column index is obtained by subtracting the x-direction minimum boundary value from the x-coordinate of the point and then dividing by the grid cell width, and the row index is obtained by subtracting the y-coordinate of the point from the y-direction maximum boundary value and then dividing by the grid cell height.

[0032] In implementation, the original airborne laser radar point cloud data is converted into a format more suitable for real-time processing and analysis through two-dimensional rasterization.

[0033] Each point of the airborne laser radar point cloud can be stored in a two-dimensional grid according to its row and column coordinates. The row and column coordinates of each point in the two-dimensional grid are determined by its original plane rectangular coordinates. Since the row coordinates of the grid are determined by the x-coordinate of the point, and the column coordinates of the grid are determined by the y-coordinate of the point, the row and column coordinates of each point in the two-dimensional grid are determined by the original plane rectangular coordinates of the point. Generally, it is increasing from top to bottom (the origin of the grid coordinate system is usually in the upper left corner, opposite to the Y-axis direction of the plane rectangular coordinate), so the coordinate flipping is realized by using subtraction, and the formula is as shown below: ; wherein, represents the column index in the converted point cloud grid; represents the row index in the converted point cloud grid; represents the original plane rectangular coordinates of the i-th point in the airborne laser radar point cloud; n represents the total number of points in the airborne laser radar point cloud; 、 represents the width (column direction resolution) and height (row direction resolution) of the grid cell, that is, the actual physical size of each grid cell; 、 is the boundary value of the point cloud grid, which is the starting minimum coordinate of x and the maximum coordinate of y, respectively.

[0034] Further, in the above unmanned aerial vehicle route planning method provided by the embodiment of the application, the step S102 of calculating the elevation mean value of each grid cell to generate a two-dimensional elevation mean value model can specifically include: dividing the grid cell into an empty cell containing no points, a sparse cell containing one to two points, and a dense cell containing more than three points according to the number of points contained in the grid cell; performing multi-directional interpolation on the empty cell with the empty cell as the center to calculate the elevation mean value of the empty cell; fitting the sparse cell using a distance weighting method with the sparse cell as the center to calculate the elevation mean value of the sparse cell; fitting the dense cell using a median filtering method to obtain the elevation mean value of the dense cell; after calculating the elevation mean values of all grid cells, the elevation mean value of each grid cell is taken as the elevation value of the local region covered by itself to form a two-dimensional elevation mean value model.

[0035] It should be noted that the laser radar point cloud is disturbed by adverse environmental factors, such as no data in the shadow area point cloud blocked, and missing point cloud caused by reflection on the water surface. Therefore, the gridded airborne laser radar point cloud still has spatial distribution unevenness and contains outliers. The elevation mean value of each grid cell is calculated, the density difference of the gridded grid cell radar point cloud is considered, different noise reduction and fitting requirements are considered, and the grid cells are divided into three categories: For the empty cell containing no points, multi-directional interpolation is performed to calculate the elevation of the center empty grid cell, and the steps are as follows: Step one, taking the empty grid cell to be calculated as the center, sequentially to eight direction angles neighborhood search is performed. Only non-empty and valid neighborhood grids are considered, i.e. grid cells containing two or more points in a certain direction, then the search in this direction is stopped, all points found in this grid cell are considered as points found in this direction, which is a valid direction. If no grid cell containing two or more points is found in a certain direction, it is judged that this search direction contains no points, which is an invalid direction.

[0036] Step two, the number of search directions with non-empty and valid neighborhood grids is judged, if all eight directions are invalid, the center empty grid cell is considered as an out-of-bound cell, and the empty grid cell is skipped. If there are valid directions, the number of valid directions is denoted as , and step three is continued.

[0037] Step three, the elevation mean value of the center grid cell is calculated, as follows: ; ; ; ; wherein, is the elevation mean value of the center grid cell; is the valid search direction, ; is the number of points searched in direction ; is the elevation of the kth point searched in direction ; is the weight of the points searched in direction ; is the direction weight of the points searched in direction , which is the inverse of the Euclidean distance; is the coordinate of the grid cell in direction ; is the coordinate of the center grid cell; is the number weight of the points searched in direction , which is the logarithm of the number of points searched in direction , to suppress the strong influence of high-density or sparse areas, to avoid the domination of a few dense points, and to further balance the contributions of different density areas. Since the search in the direction is stopped only when a grid cell containing two or more points is found in the above steps, there is no , case.

[0038] Step four, the standard deviation is calculated: .

[0039] Step five, calculate the elevation mean value of the central grid cell: ; wherein, is the elevation mean value of the local area covered by the central grid cell; as a multiple of the standard deviation constraint, the effective points conforming to the normal distribution can be screened out, and outliers can be removed, which is robust, and only those satisfying the condition participate in the calculation of the elevation mean value of the central grid cell.

[0040] For sparse cells containing one to two points, a distance weighting method is used to fit the elevation of the grid cell: First, calculate the search field range. Take the grid cell to be calculated as the center, search for its surrounding grid cells in the neighborhood range, if the number of effective points searched in the surrounding grid cells is less than the threshold value Va, it is judged that the central grid cell is an empty cell containing no points, and the eight-direction imaging method for the empty cell is used for calculation. If the number of effective points is greater than or equal to the threshold value Va, continue to execute the following steps: ; ; wherein, is the average nearest neighbor distance of the gridding process, and a sufficient number of N points in the random sampling point cloud are recorded as a point set , for each point , calculate the distance from the nearest neighbor point , and take the average of all nearest neighbor distances; is the Euclidean norm, which represents the distance from the i-th point in the random sampling point cloud to its nearest neighbor point ; is the grid resolution of the gridding process, that is, the physical size of each grid cell after gridding; Va is the point threshold, which is 3-5 points in the near-range high-density point cloud, and 1-2 points in the long-range low-density point cloud.

[0041] Then, calculate the isotropic simplified Gaussian weight: ; wherein, is the coordinate of the grid cell in the search range; is the coordinate of the central grid cell; is the normalization coefficient; is the Gaussian weight of the grid cell in the search range, which is used to calculate to the central grid cell coordinate The weights decrease with the square of the Euclidean distance; This is the variance of the Gaussian weight function. Since the smoothing requirements are the same in the x and y directions, to reduce parameters and improve computational efficiency, isotropic simplification can be performed. It is assumed that the weight decay rates in both directions are the same, i.e., the standard deviations in both directions are the same. , Calculation based on data point density: .

[0042] Next, calculate the elevation-weighted mean of the central grid cell: ; in, is the elevation-weighted average of the central grid cell; z represents the search neighborhood range, which must be an odd number to ensure central symmetry, and is calculated based on the average density of the point cloud. For the coordinates are The number of points to search within the grid cells; For grid cells The elevation of the k-th point in the search.

[0043] Calculate the standard deviation : ; Calculate the average elevation of the central grid cell: ; ; in, The average elevation of the local area covered by the central grid cell; As The standard deviation multiple constraint can screen out efficient points that conform to a normal distribution and eliminate outliers. It is robust, and only those points that meet the conditions are included. Only then does it participate in the calculation of the average elevation of the central grid cell.

[0044] For dense cells containing three or more points, median filtering is used to fit the mean elevation of the raster cells. All points within the raster cell are sorted by elevation, and the median value of the sorted results is taken as the elevation of the raster cell, as shown in the following formula: ; in, The average elevation of the local area covered by the central grid cell; This represents the median filtering function, which can be used to obtain the median elevation of all points in a grid cell; This represents the set of elevations of points within a raster cell.

[0045] Therefore, the average elevation of all grid cells can be calculated using the three methods described above. This average elevation is then used as the elevation of the local area covered by that pixel, forming a two-dimensional average elevation model.

[0046] Furthermore, in a specific implementation, in the above-mentioned UAV route planning method provided in the embodiments of the present invention, step S102 performs component edge extraction and corner point recognition on a single image to obtain corner point recognition results. Specifically, it may include: performing vertical viewpoint correction on a single image to generate a corresponding horizontal image; using a multi-scale filter bank and phase-consistent edge detection method to perform component edge extraction and corner point recognition on the horizontal image, and performing corner point sub-pixel optimization to obtain corner point recognition results.

[0047] In practice, correction is performed on a single image first. Figure 4 This is a schematic diagram illustrating vertical viewing angle correction for a single image provided in an embodiment of the present invention. Figure 4 As shown, It is the projection center of the camera; These are raw single images taken by a surveying drone, which contain tilt angles, perspective distortion, and occlusion. yes A horizontal image after vertical perspective correction to eliminate perspective distortion; yes The main point, and , yes The main point, and ; yes Any image point on; yes Image points on yes Homologous image points.

[0048] exist Choose any point ,according to and Determine the geometric relationship between them The two-dimensional image coordinates are projected using the following equation: ; The product of three rotation matrices expands into one A matrix whose elements correspond to coefficients. The formula for the rotation matrix is ​​as follows: ; in, for In horizontal image Two-dimensional image coordinates; for In the original image Two-dimensional image coordinates; To measure the focal length of the drone camera; The coefficients of the rotation matrix are given by... Decide; These represent the pitch angle, roll angle, and heading angle of the surveying drone, respectively, and are obtained from the POS data of the original single aerial image taken by the surveying drone. For rotation around the X-axis, Rotation around the Y-axis Rotate around the Z-axis.

[0049] Apply the above steps For each image point on the image, the following can be calculated: Horizontal image after vertical perspective correction to eliminate perspective distortion .

[0050] Furthermore, in specific implementation, in the above steps, a multi-scale filter bank and phase-consistency edge detection method are used to extract component edges and identify corner points in the horizontal image, and corner point sub-pixel optimization is performed to obtain corner point identification results. Specifically, this may include: converting the horizontal image into a grayscale image and performing normalization processing; transforming the normalized image to the frequency domain through Fourier transform, filtering it through a multi-scale Log-Gabor filter bank and performing inverse discrete Fourier transform to obtain the spatial domain complex response; calculating the phase consistency based on the spatial domain complex response, extracting the edge feature map after non-maximum suppression and threshold adjustment, and outputting the horizontal image coordinates of the corner points of the integer-level component region; using the horizontal image coordinates of the corner points of the integer-level component region as the center, calculating the horizontal and vertical gradients using a window of a set size to obtain the gradient vector and Hessian matrix, and iteratively solving the offset through a second-order Taylor expansion of grayscale changes to update the horizontal image coordinates of the corner points until the offset is less than a preset threshold or the maximum number of iterations is reached; and using random sampling consistency (RandomSample)... The Consensus (RANSAC) algorithm verifies the consistency of the gradient direction of corner points, retains the maximum set of interior points, and obtains the corner point recognition results.

[0051] It should be added that, before performing component edge extraction and corner point recognition on the horizontal image, the present invention can acquire the original training image (including a single image) as training data through the visible light camera of the UAV. Figure 5 This is a schematic diagram of a framework for component edge and corner recognition based on a single image, provided as an embodiment of the present invention. Figure 5As shown, the original training image can be preprocessed, which can include: adding a light reflection perception head in the Squeeze-Excitation (SE) network, detecting the light reflection area, and suppressing the feature response of the light reflection area; using the Fast Marching method to fill the edge of the light reflection area with the texture of the adjacent non-light reflection area to realize edge repair; performing automatic exposure correction on the visible light feature map after suppressing the light reflection, and taking 100-120 as the target median brightness and 50% as the brightness percentile. Then, the preprocessed image can be corrected for vertical viewing angle to obtain a horizontal image.

[0052] In implementation, the feature points of the image edge have consistent phase positions in different frequency components, such as Figure 5 As shown, the present application constructs a multi-scale filter bank, uses a Log-Gabor filter and a phase-consistent edge detection method to perform edge extraction. The low-contrast edge frame is detected, which is not affected by the reflection of the glass surface of the photovoltaic module, and the edge fracture problem caused by light reflection is solved. In addition, based on the characteristic that the gradient directions of the corner points in the same row of modules are approximately parallel, the present application uses the RANSAC algorithm to randomly select two corner points to calculate the gradient direction of the connecting line as a local edge model, verifies whether the gradient direction deviation of other corner points is less than a threshold value, retains the model with the largest number of inliers and the corresponding inlier set through multiple sampling iterations, and removes abnormal points to obtain the corner point recognition result of a single image. The specific steps are as follows: First, the horizontal image coordinates are converted to a gray image and normalized to the range, and Fast Fourier Transform (FFT) is performed to obtain frequency space coordinates and frequency domain representation .

[0053] Then, a multi-scale Log-Gabor filter bank is constructed. The center frequencies of 4-5 filters are divided according to the octave (such as 4 scales: =[0.05, 0.1, 0.2, 0.4]). 6 direction angles are set to cover anisotropic edges.

[0054] The filter formula is generated as follows: ; wherein, is the filter response value at the frequency domain coordinate; is the frequency domain coordinate; is the center frequency of the filter; is a bandwidth parameter, ; is the current direction angle; is the direction angle of the current frequency point, ; is the direction bandwidth, controlling the selectivity of the direction, .

[0055] Next, frequency domain filtering and phase calculation are performed. For each scale and direction of the Log-Gabor filter, the filtering response value is calculated to obtain the frequency domain filtering result , and the spatial domain complex response is calculated by inverse discrete Fourier transform (IDFT) : ; ; Subsequently, multi-scale and direction accumulation are performed for phase consistency calculation: ; wherein, is the phase consistency value at the image position , ranging from . The closer the value is to 1, the more significant the edge is. is a very small constant to prevent the denominator from being zero, usually .

[0056] After that, the edge feature map is extracted. Non-maximum points are suppressed along the gradient direction (determined by the phase consistency direction), the mean / standard deviation is calculated to adjust the threshold, and the edge feature map is output. The horizontal image coordinates of the i integer level component region corner points are also output .

[0057] Finally, the sub-pixel optimization of the corner points output by the Log-Gabor is performed.

[0058] In the vicinity of the i integer level corner point output by the Log-Gabor , the image gray gradient distribution can be approximated by a second-order Taylor expansion, and the gradient distribution within the integer level corner point horizontal image coordinate field is fitted by Taylor expansion. A window of is taken with the integer level corner point horizontal image coordinate as the center, and the horizontal and vertical gradient are calculated, and then the gradient vector and Hessian matrix are calculated. The second-order Taylor expansion of the gray change within the window is performed , and since the first derivative of the function at the extreme point is zero, the derivative of the second-order Taylor expansion is set to zero, and the offset is obtained by substituting the gradient vector and Hessian matrix, and the corner point horizontal image coordinate is updated, and the iteration optimization is performed until , such as If the maximum number of iterations is reached or the maximum number of iterations is reached, the iteration is terminated, otherwise The calculation is repeated with a new initial point until the final accurate extreme value point horizontal image coordinate of the corner point is obtained , as shown in the following formula: ; ; ; ; ; ; ; Since the gradient direction of the same row of component corner points should be approximately parallel, RANSAC is used for robust verification, and two corner points are randomly selected as model samples, and the gradient direction (unit vector) of the line connecting the two corner points is calculated as the local edge model , verify the gradient consistency of other corner points (such as the gradient direction deviation is less than the preset angle tolerance threshold), randomly sample two corner points multiple times, and iteratively update until the maximum inlier set is found. That is, the model with the most inliers and its corresponding inlier set are retained, and the abnormal gradient corner points are removed.

[0059] ; ; wherein, is the local edge model composed of the gradient direction (unit vector) of the line connecting the two randomly selected corner points ; is the allowed angle deviation; is the i-th sub-pixel optimized accurate extreme value point horizontal image coordinate.

[0060] After the above image recognition processing is performed on a single image, the maximum inlier set obtained is the result of the corner point recognition of the single image, and the recognized corner point is denoted as , and the corner point horizontal image coordinate is denoted as .

[0061] Further, in the specific implementation, in the unmanned aerial vehicle route planning method provided by the embodiment of the application, the step S103 projects the corner point recognition result onto the two-dimensional elevation mean model in a geometric fusion manner to obtain the preliminary three-dimensional coordinates of the corner point, and specifically can include: converting the single image and the two-dimensional elevation mean model to the same coordinate system; obtaining the three-dimensional coordinates of the projection center of the camera, extending the line connecting the projection center and the principal point of the horizontal image to the ground to obtain a first intersection point, searching for a first elevation of the grid unit where the first intersection point is located from the two-dimensional elevation mean model, and calculating the vertical distance from the projection center to the first intersection point; drawing a straight line from the projection center to the corner point on the horizontal image and extending to the ground; drawing a horizontal line from the first intersection point, and the intersection point of the horizontal line and the straight line is a second intersection point; calculating the three-dimensional coordinates of the second intersection point according to the vertical distance from the projection center to the first intersection point, the horizontal image coordinates of the corner point and the plane coordinates of the projection center, to obtain the projection point coordinates of the second intersection point on the ground; searching for a second elevation of the grid unit where the projection point corresponding to the second intersection point is located from the two-dimensional elevation mean model, and calculating the difference between the first elevation and the second elevation; calculating the next projection point coordinates and the difference in elevation between the adjacent two projection points by using the difference between the first elevation and the second elevation; and repeatedly executing the steps of calculating the projection point coordinates and the difference, until the difference in elevation between the adjacent two projection points is less than a set threshold, to obtain the preliminary three-dimensional coordinates of the corner point.

[0062] In the implementation, the three-dimensional projection point of the horizontal image coordinates of the corner point of the original single image taken by the surveying and mapping unmanned aerial vehicle on the ground is found by the geometric fusion method, and the three-dimensional coordinates of the projection point are calculated to realize the positioning of the horizontal image coordinates of the corner point of the original single image taken by the surveying and mapping unmanned aerial vehicle.

[0063] Figure 6 The schematic diagram for the geometric fusion positioning of the corner point provided by the embodiment of the application is shown in FIG. 1. Figure 6 As shown in FIG. 1, the two-dimensional elevation mean model is generated by the airborne radar point cloud through the above method; is the projection center of the camera; is the horizontal image after the perspective distortion is eliminated after the vertical angle correction; is the horizontal image after the perspective distortion is eliminated after the vertical angle correction; is the principal point of the horizontal image, and , , is the principal point of the horizontal image, and ; is the horizontal image coordinates of any corner point in the horizontal image, and is the horizontal image coordinates of any corner point in the horizontal image, and is the projection point of the corner point on the ground; is the projection point of the corner point on the ground; and are on a line. , and are on a line.

[0064] Geometric fusion methods may include the following steps: The first step is to convert a single aerial image taken by a surveying drone and a two-dimensional elevation mean model obtained from lidar point clouds to the same coordinate system, with the same direction and the origin corresponding to the same position in the physical world.

[0065] The second step is to obtain the projection center from the POS data of the surveying drone. 3D coordinates .extend Intersecting with the ground at ,but The plane coordinates of the point are Search Determine the elevation of the ground grid cell where the point is located, and calculate it. The length is: ; in, for length; for Point elevation; , This is an elevation retrieval function used to retrieve the elevation of the ground grid cell where the coordinates are located in a two-dimensional elevation mean model, based on a given plane rectangular coordinate. for Point elevation is obtained from the camera projection center elevation (ellipsoidal height) in the POS data of a single aerial image taken by the surveying drone. If the POS data records the elevation of the surveying drone's GNSS antenna, the vertical offset of the camera installation needs to be added. .

[0066] Step 3: Draw a straight line Intersecting with the ground at .from Draw a horizontal line from the point and Intersect at ,triangle and Similar, then The three-dimensional coordinates are: ; in, for 3D coordinates; for The horizontal image coordinates of any corner point in the image, superior; for Cartesian coordinates in the plane.

[0067] Step 4 Draw a perpendicular line that intersects the ground at... . the planar rectangular coordinates of are the same as the planar rectangular coordinates of Draw a horizontal line from to intersect with at , and draw a horizontal line from to intersect with at . Retrieve the elevation of the grid cell where is located in the two-dimensional elevation mean model, and calculate the length of , as follows: where is the length of line segment ; is the elevation of ; is the elevation of ; is the planar rectangular coordinates of .

[0068] Step 5, and are similar, the three-dimensional coordinates of are calculated as follows: .

[0069] Step 6, repeat the above fourth and fifth steps until the height of and is less than a specific threshold value, and select as the three-dimensional corner point. Apply the above steps to the horizontal image coordinates of any corner point in to calculate the three-dimensional coordinates of each corner point, achieving the preliminary positioning of the corner points of the photovoltaic area in the original single image taken by the surveying and mapping unmanned aerial vehicle .

[0070] Furthermore, in a specific implementation, in the UAV flight path planning method provided in the embodiments of the present invention, step S103 performs principal component analysis on the lidar point cloud surrounding the initial three-dimensional coordinates of the corner point to obtain the optimized three-dimensional coordinates of the corner point. Specifically, this may include: searching for lidar point clouds in multiple surrounding grid cells centered on the grid cell containing the projection point of the corner point on the ground; determining the vertical distance from the searched lidar point cloud to the target line; the target line is the line between the projection center and the projection point of the corner point on the ground; filtering out the lidar point clouds corresponding to those with a vertical distance less than a set distance value, and calculating the mean coordinates of the filtered lidar point clouds; obtaining the covariance matrix of the filtered lidar point clouds based on the mean coordinates; calculating the eigenvalues ​​and eigenvectors of the covariance matrix using principal component analysis to construct new three-dimensional coordinates; and determining the coordinates of the corner point in the new three-dimensional coordinates using geometric constraints combined with displacement parameters to obtain the optimized three-dimensional coordinates of the corner point.

[0071] In practice, the two-dimensional elevation mean model generated from the original airborne radar point cloud is a fixed-resolution raster image, with each pixel storing a single elevation value, which cannot reflect the terrain changes within a pixel. When projecting the UAV image, each pixel of the two-dimensional elevation mean model is treated as a horizontal plane by default, resulting in texture distortion. This affects the positioning accuracy of the corner points of the selected component area in the original single image taken by the UAV. If the raster size is reduced to approximate the real terrain, the amount of data will surge, and the computational efficiency will decrease. Therefore, this invention uses principal component analysis of the original airborne lidar point cloud around the corner points of the selected component area to achieve accurate positioning of the corner points of the selected component area.

[0072] Figure 7 This is a schematic diagram illustrating precise corner point positioning provided in an embodiment of the present invention. Figure 7 As shown, It is the raster cell in the i-th row and j-th column of the two-dimensional mean elevation model; It is obtained by the geometric fusion positioning method described above. ground projection point, yes The precise positioning point is determined by the following steps: First, with the inclusion point Centered on the grid cell, search for all point clouds in the surrounding 9 grid cells.

[0073] Then, set the threshold. And filter from the search points that match the line The vertical distance is less than If the number of selected points is less than 3, precise positioning is not required; if it is 3 or more, calculate the average coordinates of the selected points using the following formula: .

[0074] Next, the covariance matrix of the selected points is calculated using the following formula: ; because: We can obtain: ; in, The mean coordinates of the corner points of the selected component area; Let be the three-dimensional coordinates of the i-th point among the selected points. , where n is the number of points being selected; It is the covariance matrix of the selected points.

[0075] Subsequently, principal component analysis was used to calculate the covariance matrix. eigenvalues and eigenvectors eigenvalues The characteristic equation is as follows: ; Solve for eigenvalues , .

[0076] For eigenvalues Solve the system of linear equations: ; The solution is obtained by Gaussian elimination and normalization to obtain the unit eigenvector. The principal component directions that constitute PCA.

[0077] Calculate the correction parameters. Figure 7 middle, It is a three-dimensional coordinate system. The origin is the coordinate system of the above-mentioned mean coordinates. ; It is the coordinate axis where the feature vector is located.

[0078] Located in the plane Principal plane and line superior; , and For the parameters to be solved, the projection points are... Relative to the above mean coordinates displacement vector transformation to In the coordinate system formed, The coordinates of the displacement vector under the mixed basis matrix are calculated using the following formula: ; wherein, is the coordinate of the ground projection point of the geometric fusion positioning method ; is the coordinate of the ground projection point of the geometric fusion positioning method ; is the feature vector ; is the unit direction vector of the CS direction.

[0079] Finally, the coordinates are solved by using geometric constraints. Since , the above mean coordinates are taken as the reference points, the selected component region corner points are located on the plane main plane, , are parameters; the projection points are taken as the reference points, the selected component region corner points are located on the line , are parameters, that is, the following formulas are respectively satisfied: ; Substituting , the three-dimensional coordinates of can be calculated, and the accurate positioning of the selected component region corner points on the original single image of the surveying and mapping unmanned aerial vehicle aerial photography is realized .

[0080] Further, in the above unmanned aerial vehicle route planning method provided by the embodiments of the application, in specific implementation, the step S104 clusters and fuses the corner points of the multiple images to obtain a fused optimal corner point coordinate set, and superimposes the optimal corner point coordinate set and the set of optimized three-dimensional coordinates in the orthographic image file, and after verification, forms a regional corner point coordinate set and performs grouping and sorting, which can specifically include: performing clustering on the multiple images based on a density-based spatial clustering of applications with noise (DBSCAN) algorithm, labeling the successfully clustered corner point clusters, regarding the corner points in the same cluster as observation values of the same physical corner point, outputting the coordinate set in the cluster, and reserving the unclustered corner point coordinates; selecting an inner point from the cluster according to the coordinate set in the cluster, and obtaining the optimal corner point coordinate set of each corner point cluster after robust fusion by using the inner point; superimposing the optimal corner point coordinate set, the set of optimized three-dimensional coordinates, and the unclustered corner point coordinates in the orthographic image file by using ArcGIS, and determining whether the optimal corner point coordinate set contains all component regional corner points; if not, forming a missing point coordinate; collectively forming and outputting the regional corner point coordinate set by using the optimal corner point coordinate set, the unclustered corner points to be reserved, and the missing point coordinate; superimposing the regional corner point coordinate set in the orthographic image file by using ArcGIS, grouping and numbering the regional corner point coordinate set according to the global component arrangement form and the cleaning unmanned aerial vehicle operation sequence, and uniformly sorting the grouped regional corner point coordinates in a clockwise or counterclockwise direction.

[0081] In implementation, the present application can pre-locate the accurate positions of the component regional corner points of all single images of this aerial photography to the same coordinate system, and ensure the consistency of units. The following steps can be performed: First, perform corner point association and matching: spatial proximity. Perform DBSCAN clustering on the corner points from different pictures, set the coordinate projection error of the same physical corner point to be less than the neighborhood radius (set according to the RTK accuracy), label the successfully clustered corner point clusters, regard the corner points in the same cluster as observation values of the same physical corner point, and output the coordinate set in the cluster. The unclustered corner point coordinates are reserved for verification in the post-processing and verification steps.

[0082] Then the optimal coordinate fusion algorithm is executed. Two points are randomly selected from the cluster, the Euclidean distance of all points to the mean point of the two points is calculated, if the Euclidean distance residual is less than the inner point residual threshold (set according to the RTK accuracy), the observation point is marked as an inner point, otherwise the observation point is excluded. Repeat the above calculation by randomly selecting two points from the cluster, if the current number of inner points is greater than the optimal number of inner points, update the two randomly selected points. Stop when the maximum number of iterations is reached or the inner point ratio exceeds the threshold, and finally re-estimate the optimal coordinates of all inner points. Output the optimal corner point coordinate set of each corner point cluster after robust fusion .

[0083] Next, post-processing and visualization verification are performed. Using ArcGIS, the accurate positioning set of the component area corner points of all single images is superimposed on the orthographic image , the successfully clustered intersection coordinates and the optimal corner point coordinate set of each corner point cluster after fusion , and different color layers are used to represent them, to check whether the optimal corner point coordinate set contains all component area corner points and to determine whether the unclustered corner points need to be retained. If there is only an unclustered corner point in a certain photovoltaic component area on the orthographic image, the unclustered corner point is retained, and the remaining unclustered corner points are excluded. If there is no optimal corner point in a certain photovoltaic component area on the orthographic image, the latitude and longitude of the corner point are obtained by manually selecting the corner point in the orthographic image file, and the elevation value at the latitude and longitude is found in the radar point cloud file to form the missing point coordinates.

[0084] Then the optimal corner point coordinates, the unclustered corner points that need to be retained, and the supplemented missing point coordinates are output to form the regional corner point coordinate set.

[0085] Finally, the regional corner point coordinate set grouping is performed. Using ArcGIS, the regional corner point coordinate set is superimposed on the orthographic image file, and the regional corner point coordinate set is grouped and numbered according to the global component arrangement form and the cleaning unmanned aerial vehicle operation sequence. The grouped regional corner point coordinates are sorted in a clockwise or counterclockwise direction.

[0086] Further, in the specific implementation, in the unmanned aerial vehicle route planning method provided in the embodiments of the present application, step S105 generates a two-dimensional route file according to the grouped and sorted regional corner point coordinates and the route spacing, and explicitly specifies the absolute elevation in the two-dimensional route file to obtain a target cleaning route file, which can specifically include: generating a grouped closed polygon KML file according to the grouped and sorted regional corner point coordinates, calculating the main direction angle, and generating parallel routes in the grouped region in combination with the route spacing, to generate a two-dimensional route file after connecting the first and last regions of multiple regions; generating a three-dimensional waypoint file by explicitly specifying the absolute elevation in the two-dimensional route file, importing the radar point cloud and performing visibility analysis between waypoints, and generating a target cleaning route file.

[0087] In the implementation, using the Python Simplekml library, import the group area corner point coordinates to generate a group closed polygon KML file.

[0088] Using the Shapely library to calculate the longest side direction vector of each group polygon as the main direction angle, and input the flight line spacing to generate parallel flight lines in the group area. The end point of the parallel flight lines in a group area is connected to the start point of the parallel flight lines in the next group area, and the head and tail of multiple areas are connected to generate a two-dimensional flight line file.

[0089] Save the flight line KML, and superimpose the flight line KML file on the orthophoto to check the rationality of the flight line spacing. During the cleaning of the unmanned aerial vehicle, in order to ensure the cleaning effect, the relative height of the cleaning unmanned aerial vehicle and the component is low, and in order to consider the safety of the operation, the absolute height of the flight point is the sum of the maximum height of the corner point coordinates of each group area and the relative height of the cleaning unmanned aerial vehicle and the component required.

[0090] Explicitly specify the absolute height in the flight line KML file, generate a three-dimensional flight point file, import the radar point cloud and perform visibility analysis between flight points to check the safety of the flight path, and the three-dimensional flight line file that passes the visibility analysis can generate the final flight line file.

[0091] From the above, the unmanned aerial vehicle route planning method provided by the present application can include: after collecting the image of the region boundary of the component to be cleaned by the unmanned aerial vehicle aerial photography and the radar data, splicing and outputting the orthographic image file. The laser radar point cloud is rasterized, and the elevation average of each grid unit is calculated to generate an accurate two-dimensional elevation average model. The original single image of the tilted surveying and mapping unmanned aerial vehicle aerial photography is corrected to generate the corresponding horizontal aerial photography original single image. Based on the single image, the Log-Gabor filter and the edge detection method based on phase consistency are used to identify the component edge and corner point, and the corner point sub-pixel precision optimization is performed. The identified corner point horizontal image coordinates are projected onto the two-dimensional elevation average model to obtain the preliminary three-dimensional positioning of the corner point. The laser radar point cloud around the corner point three-dimensional coordinates is subjected to principal component analysis, thereby realizing the accurate three-dimensional positioning of the component region corner point. Based on DBSCAN clustering and robust fusion, the integrity of the orthographic image is used to fuse, verify and supplement the optimal coordinates of the corner points of multiple images, and the optimal corner point coordinates, the corner points that need to be retained and the supplemented missing point coordinates are combined to form and output the region corner point coordinate set. The region corner point coordinate set is superimposed in the orthographic image file using ArcGIS, the region corner point coordinate set is grouped and numbered according to the global component arrangement form and the cleaning unmanned aerial vehicle operation sequence, and the grouped region corner point coordinates are sorted in clockwise or counterclockwise order. The grouped closed polygon KML file is generated by importing the grouped region corner point coordinates, the main direction angle is calculated, the route interval is input, the parallel routes in the grouped region are generated, and the two-dimensional route file is generated by connecting the first and last regions. Save the route KML, and superimpose the route KML file on the orthographic image to check the rationality of the route interval. The absolute elevation is specified in the route KML file to generate a three-dimensional waypoint file, the radar point cloud is imported and the visibility analysis between the waypoints is performed to generate the final target cleaning route file.

[0092] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0093] The embodiments of the present application also provide an unmanned aerial vehicle route planning device. The present embodiment is based on the angle of functional modules, and the device comprises: A data acquisition module is configured to acquire single image and laser radar point cloud data of a region boundary of a component to be cleaned, splice and output an orthographic image file; A data processing module is configured to rasterize the laser radar point cloud data, calculate the elevation average of each grid unit to generate a two-dimensional elevation average model, extract the component edge and identify the corner point of the single image, and obtain a corner point identification result; a geometric fusion analysis module, configured to project the corner point recognition result onto a two-dimensional elevation mean model in a geometric fusion manner to obtain preliminary three-dimensional coordinates of the corner point, and perform principal component analysis on laser radar point clouds around the preliminary three-dimensional coordinates of the corner point to obtain optimized three-dimensional coordinates of the corner point; a clustering fusion verification module, configured to obtain multiple images of a region boundary of a component to be cleaned, and perform clustering and fusion on corner points of the multiple images to obtain a fused optimal corner point coordinate set, and superimpose the optimal corner point coordinate set and the set of optimized three-dimensional coordinates in the orthographic image file, and form a region corner point coordinate set after verification and grouping and sorting; a flight path file generation module, configured to generate a two-dimensional flight path file according to the region corner point coordinates after grouping and sorting and a flight path interval, and explicitly specify an absolute elevation in the two-dimensional flight path file to obtain a target cleaning flight path file.

[0094] In the unmanned aerial vehicle flight path planning device provided in the embodiment of the present application, the component edge and corner point recognition can be carried out based on a single image, the component region corner point positioning can be carried out by fusing laser radar point cloud data, the global layout analysis of the component corner point can be carried out by using the component region orthographic image, and the cleaning unmanned aerial vehicle high-precision flight path can be generated, so that the multi-source data fusion and accurate processing are realized, the component edge and corner point are finely recognized by using a single image, the accuracy of the corner point positioning is improved by using the laser radar point cloud data, and the integrity of the global layout analysis of the corner point is ensured by relying on the orthographic image. The target cleaning flight path file containing the explicit absolute elevation has higher precision and stronger adaptability, can provide accurate guidance for the unmanned aerial vehicle cleaning operation, effectively improves the cleaning efficiency and quality, and reduces the operation deviation and failure.

[0095] Since the embodiments of the unmanned aerial vehicle flight path planning device part correspond to the embodiments of the unmanned aerial vehicle flight path planning method part, the description of the features in the embodiments corresponding to the unmanned aerial vehicle flight path planning device can be referred to the related description of the embodiments corresponding to the unmanned aerial vehicle flight path planning method, which will not be repeated here. And has the same beneficial effects as the above-mentioned unmanned aerial vehicle flight path planning method.

[0096] The embodiments of the present application also provide an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned unmanned aerial vehicle flight path planning method embodiments.

[0097] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above-mentioned unmanned aerial vehicle flight path planning method embodiments when running.

[0098] In an example embodiment, the computer readable storage medium described above can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0099] Embodiments of the present application also provide a computer program product, which comprises a computer program, and the computer program, when executed by a processor, implements the steps in any of the above unmanned aerial vehicle route planning method embodiments.

[0100] Embodiments of the present application also provide another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps in any of the above unmanned aerial vehicle route planning method embodiments.

[0101] The skilled person can further realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0102] The above provides a detailed description of the unmanned aerial vehicle route planning method, device and medium provided by the present application. The principles and implementation modes of the present application are described by applying specific examples in this paper. The above description of the examples is only applicable to help understand the method of the present application and its core idea. It should be noted that for ordinary skilled persons in the technical field, without departing from the principles of the present application, the present application can be improved and modified in several ways, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A method for planning flight routes for unmanned aerial vehicles (UAVs), characterized in that, The method comprises the following steps: Obtain a single image and laser radar point cloud data of the region boundary of the component to be cleaned, splice and output an orthographic image file; Grid the laser radar point cloud data, calculate the elevation mean value of each grid unit to generate a two-dimensional elevation mean value model, extract the component edge and identify the corner point of the single image to obtain a corner point identification result; Project the corner point identification result onto the two-dimensional elevation mean value model by using a geometric fusion method to obtain the preliminary three-dimensional coordinates of the corner point, and perform principal component analysis on the laser radar point cloud around the preliminary three-dimensional coordinates of the corner point to obtain the optimized three-dimensional coordinates of the corner point; Obtain multiple images of the region boundary of the component to be cleaned, cluster and fuse the corner points of the multiple images to obtain a fused optimal corner point coordinate set, superimpose the optimal corner point coordinate set and the set of optimized three-dimensional coordinates in the orthographic image file, and after verification, form a region corner point coordinate set and perform grouping and sorting; Generate a two-dimensional flight line file according to the region corner point coordinates after grouping and sorting and the flight line spacing, and explicitly specify the absolute elevation in the two-dimensional flight line file to obtain a target cleaning flight line file. 2.The method of claim 1, wherein, The method for calculating the elevation mean value of each grid unit to generate a two-dimensional elevation mean value model comprises the following steps: According to the number of points contained in the grid unit, the grid unit is divided into an empty unit containing no points, a sparse unit containing one to two points, and a dense unit containing more than three points; Centering on the empty unit, the elevation mean value of the empty unit is calculated by multi-directional interpolation; Centering on the sparse unit, the elevation mean value of the sparse unit is calculated by using distance weighting method; The dense unit is fitted by using median filtering method to obtain the elevation mean value of the dense unit; After calculating the elevation mean value of all grid units, the elevation mean value of each grid unit is taken as the elevation value of the local region covered by itself to splice and form a two-dimensional elevation mean value model. 3.The method of claim 1, wherein, The method for extracting the component edge and identifying the corner point of the single image to obtain a corner point identification result comprises the following steps: Perform vertical angle correction on the single image to generate a corresponding horizontal image; Use a multi-scale filter set and a phase consistency edge detection method to extract the component edge and identify the corner point of the horizontal image, and perform corner point sub-pixel optimization to obtain a corner point identification result. 4.The method of claim 3, wherein, The method for extracting the component edge and identifying the corner point of the horizontal image by using a multi-scale filter set and a phase consistency edge detection method and performing corner point sub-pixel optimization to obtain a corner point identification result comprises the following steps: Convert the horizontal image to a grayscale image and perform normalization processing; Convert the normalized image to the frequency domain by Fourier transform, filter by a multi-scale Log-Gabor filter set, and inverse discrete Fourier transform to obtain an empty domain complex response; According to the empty domain complex response, accumulate the phase consistency, extract the edge feature map after non-maximum suppression and threshold adjustment, and output the horizontal image coordinates of the integer-level component region corner point; The horizontal image coordinates of the integer-level component region corner points are taken as the center, a window of a set size is set to calculate the horizontal gradient and the vertical gradient to obtain the gradient vector and the Hessian matrix, and the horizontal image coordinates of the corner points are iteratively solved by the second-order Taylor expansion of the gray change to update the offset until the offset is less than a preset threshold or the maximum number of iterations is reached; The RANSAC algorithm is used to verify the consistency of the gradient direction of the corner points, and the maximum inlier set is reserved to obtain the corner point recognition result. 5.The method of claim 3, wherein, The corner point recognition result is projected onto the two-dimensional elevation mean model in a geometric fusion manner to obtain the preliminary three-dimensional coordinates of the corner points, including: Converting the single image and the two-dimensional elevation mean model to the same coordinate system; Obtaining the three-dimensional coordinates of the projection center of the camera, extending the connecting line between the projection center and the principal point of the horizontal image to the ground to obtain a first intersection point, retrieving a first elevation of the grid cell where the first intersection point is located from the two-dimensional elevation mean model, and calculating the vertical distance from the projection center to the first intersection point; Drawing a straight line from the projection center to the corner point on the horizontal image and extending it to the ground, and drawing a horizontal line from the first intersection point, and the intersection point of the straight line and the horizontal line is a second intersection point; According to the vertical distance from the projection center to the first intersection point, the horizontal image coordinates of the corner point and the plane coordinates of the projection center, the three-dimensional coordinates of the second intersection point are calculated to obtain the projection point coordinates of the second intersection point on the ground; Retrieving the second elevation of the grid cell where the projection point corresponding to the second intersection point is located from the two-dimensional elevation mean model, and calculating the difference between the first elevation and the second elevation; Using the difference between the first elevation and the second elevation, the next projection point coordinates and the difference in elevation between the adjacent two projection points are calculated; The steps of projection point coordinate calculation and difference calculation are repeatedly performed until the difference in elevation between the adjacent two projection points is less than a set threshold, and the preliminary three-dimensional coordinates of the corner points are obtained. 6.The method of claim 1, wherein, Principal component analysis is performed on the laser radar point cloud around the preliminary three-dimensional coordinates of the corner points to obtain the optimized three-dimensional coordinates of the corner points, including: Taking the grid cell containing the projection point of the corner point on the ground as the center, searching for the laser radar point cloud in the surrounding multiple grid cells; Determining the vertical distance from the searched laser radar point cloud to the target straight line; the target straight line is the straight line between the projection center and the projection point of the corner point on the ground; When the determined vertical distance is less than a set distance value, the corresponding laser radar point cloud is screened out, and the mean coordinates of the screened laser radar point cloud are calculated; According to the mean coordinates, the covariance matrix of the screened laser radar point cloud is obtained; The eigenvalues and eigenvectors of the covariance matrix are calculated by using the principal component analysis method to construct new three-dimensional coordinates; Using geometric constraints combined with displacement parameters, the coordinates of the corner point in the new three-dimensional coordinates are determined to obtain the optimized three-dimensional coordinates of the corner point. 7.The method of claim 1, wherein, The corner points of the multiple images are clustered and fused to obtain a fused optimal corner point coordinate set, and the optimal corner point coordinate set and the set of optimized three-dimensional coordinates are superimposed in the orthophoto file, and after verification, a region corner point coordinate set is formed and grouped and sorted, including: DBSCAN clustering is performed on the plurality of images, and a label is given to a corner point cluster that is successfully clustered. Corner points in the same cluster are regarded as observation values of the same physical corner point, and a coordinate set in the cluster is output. Corner points that are not successfully clustered are retained; According to the coordinate set in the cluster, an in-point is selected from the cluster, and an optimal corner point coordinate set of each corner point cluster after robust fusion is obtained using the in-point; The optimal corner point coordinate set, the set of optimized three-dimensional coordinates, and the coordinates of corner points that are not successfully clustered are superimposed in the orthographic image file using ArcGIS, and it is determined whether the optimal corner point coordinate set contains all component area corner points. If not, missing point coordinates are formed; The optimal corner point coordinate set, the corner points that need to be retained and the missing point coordinates are collectively formed and output as an area corner point coordinate set; The area corner point coordinate set is superimposed in the orthographic image file using ArcGIS, the area corner point coordinate set is grouped and numbered according to the global component arrangement form and the cleaning unmanned aerial vehicle operation sequence, and the grouped area corner point coordinates are sorted in a clockwise or counterclockwise direction. 8.The method of claim 1, wherein, According to the grouped and sorted area corner point coordinates and the flight line spacing, a two-dimensional flight line file is generated, and an absolute elevation is explicitly specified in the two-dimensional flight line file to obtain a target cleaning flight line file, including: A grouped closed polygon KML file is generated according to the grouped and sorted area corner point coordinates, a main direction angle is calculated, and parallel flight lines in the grouped area are generated in combination with the flight line spacing. After connecting the first and last areas, a two-dimensional flight line file is generated; An absolute elevation is explicitly specified in the two-dimensional flight line file to generate a three-dimensional waypoint file, radar point clouds are imported and visibility analysis between waypoints is performed to generate a target cleaning flight line file.

9. An electronic device, comprising: The computer readable storage medium stores a computer program, and the processor executes the computer program to implement the steps of the unmanned aerial vehicle flight path planning method according to any one of claims 1 to 8. The computer readable storage medium stores a computer program, and the processor executes the computer program to implement the steps of the unmanned aerial vehicle flight path planning method according to any one of claims 1 to 8. The computer readable storage medium stores a computer program, and the processor executes the computer program to implement the steps of the unmanned aerial vehicle flight path planning method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, ​

Images