Spraying route planning method for realizing automatic spraying of building facade by unmanned aerial vehicle

Abstract

The invention discloses a spraying route planning method for realizing automatic spraying of a building facade by an unmanned aerial vehicle, and the method specifically comprises the steps: S1, obtaining an oblique photography three-dimensional model of a sprayed building object, and rendering the oblique photography three-dimensional model into a three-dimensional visual platform; s2, performing coordinate system data conversion on the building three-dimensional model in the three-dimensional visualization platform to obtain coordinate data used for positioning the unmanned aerial vehicle; s3, a 3D model editing tool of the three-dimensional visualization platform is used for drawing a to-be-sprayed two-dimensional / three-dimensional grid graph representing a plane / curved surface spraying working area of the building on the three-dimensional model of the building; s4, a pixel-level route path planning algorithm is used for the two-dimensional grid graph drawn in the building three-dimensional model, an improved A * algorithm for full-coverage path planning is used for the three-dimensional grid graph drawn in the building three-dimensional model, and coordinate data used for unmanned aerial vehicle positioning are combined. And a spraying route of the unmanned aerial vehicle for automatic spraying operation of the building external facade is planned.

Description

Technical Field

[0001] The present invention relates to the field of aerial operation robots, and more particularly, to a spraying route planning method for realizing automatic spraying of the building exterior wall by an unmanned aerial vehicle (UAV). Background Art

[0002] With the development of building technology, the existing building heights are also increasing. However, for high-rise buildings that need to be sprayed, spraying is extremely inconvenient. The traditional spraying method is carried out manually. However, spraying high-rise buildings manually not only has high costs and low efficiency, but also the workers have large movement amplitudes during the spraying process, posing a risk of falling. To improve efficiency, reduce costs and safety risks, more and more enterprises have begun to consider using spraying UAVs to replace workers for building exterior wall painting operations, so as to save production costs and avoid personnel safety problems. Enterprises choose spraying UAVs for applications in fields such as building exterior wall painting, maintenance, and cleaning. Spraying UAVs are much cheaper than hiring workers, more efficient, and have shorter construction periods.

[0003] At present, there is no mature route planning solution for the autonomous spraying operation of spraying UAVs. It is necessary to manually control the UAV for spraying operations, and it is difficult to achieve precise coverage of the spraying area, which urgently needs to be improved. Summary of the Invention

[0004] One technical problem to be solved by the present application is to overcome the defects of the above-related technologies, and provide a spraying route planning method for realizing automatic spraying of the building exterior wall by an unmanned aerial vehicle, planning a spraying work area for a building three-dimensional model and generating a spraying route, so that the UAV can perform autonomous spraying operations on the building facade according to the spraying route, adapt to various building exterior wall spraying scenarios, so as to improve the efficiency and quality of UAV spraying operations and achieve the purpose of reducing costs and increasing efficiency.

[0005] The technical solution adopted by the present invention to solve the technical problem is as follows: A spraying route planning method for realizing automatic spraying of the building exterior wall by an unmanned aerial vehicle, specifically including the following steps:

[0006] S1: Obtain the oblique photography three-dimensional model of the spraying building object, and render the oblique photography three-dimensional model into a three-dimensional visualization platform;

[0007] S2: Perform coordinate system data conversion on the building three-dimensional model in the three-dimensional visualization platform to obtain the coordinate data used for UAV positioning. The coordinate data includes longitude, latitude, and elevation;

[0008] S3: Use the 3D model editing tool of the three-dimensional visualization platform to draw a two-dimensional / three-dimensional grid graph to be sprayed on the building three-dimensional model. The two-dimensional grid graph to be sprayed is the building plane spraying work area to be sprayed, and the three-dimensional grid graph to be sprayed is the building curved surface spraying work area to be sprayed;

[0009] S4: Use a pixel-level flight path planning algorithm for the two-dimensional grid graph to be sprayed, and use an improved A* algorithm for the three-dimensional grid graph to be sprayed under the full coverage path planning. Combine the coordinate data used for UAV positioning to plan the spraying route for the UAV to perform automated spraying operations on the building facade.

[0010] Compared with the related technologies, the present invention has the following advantages:

[0011] 1. It enables the spraying UAV to autonomously complete the spraying task without manual intervention and assistance, improves the automation level of the UAV spraying operation, can ensure that the UAV reaches the specified position during spraying, realizes precise spraying, and avoids the need for manual control of the traditional spraying UAV to spray the building facade.

[0012] 2. The three-dimensional visualization technology is applied to the route planning, so that the route plan planned by this method is generated based on the building real-scene three-dimensional model. Combining with the UAV parameters, it can accurately control the spraying range of the building facade to achieve high-precision positioning and spraying.

[0013] 3. The spraying work area is drawn based on the 3D model editing tool of the three-dimensional visualization platform, which can adapt to the building facades of various complex shapes and structures. At the same time, it can realize the spraying of complex patterns and textures, improving the aesthetic degree of the building facade.

[0014] 4. The improved A* algorithm under the full coverage path planning can find a feasible path from the starting point to the ending point and covering all key points in the given area when the starting point and the ending point are given, realizing the full coverage of the spraying work area, ensuring that the spraying UAV can access and cover each area in the environment, and avoiding the problem of missed spraying.

[0015] Specifically, the specific steps of the UAV flight path planning for the three-dimensional grid graph to be sprayed in step S4 are as follows:

[0016] S41: Calculate the normal vector of the grid vertices according to the three-dimensional grid graph to be sprayed: After loading the three-dimensional grid graph to be sprayed for preprocessing, convert the grid data into point cloud data, filter the point cloud data to remove noise, and then solve the normal vector according to the adjacency relationship of the grid vertices.

[0017] S42: UAV fuselage positioning: Translate the preprocessed point cloud in step S41 outward along the direction of the normal vector by a set distance to obtain the target position of the UAV fuselage; where the set distance is the distance from the nozzle to the UAV fuselage positioning module plus the optimal spraying distance of the nozzle.

[0018] S43: Optimize the path to achieve full-coverage spraying: Use the improved A* algorithm for full-coverage path planning to connect the discrete UAV poses and generate a complete segmented flight path;

[0019] S44: Perform post-processing on the flight path, calculate the heading angle of the waypoints and the relative heading angle.

[0020] Furthermore, the specific steps for filtering and removing noise from the point cloud data in step S41 are as follows:

[0021] S411: Convert the three-dimensional mesh graph to be sprayed into mesh data in off format, and use the Open3D library to create a triangular mesh object from the mesh data;

[0022] S412: Sample the mesh data into point cloud data, and use the poisson_disk algorithm in the Open3D library for point cloud resampling;

[0023] S413: For all resampled point cloud data, perform smoothing and denoising: Use the angle of the normal vector and the distance between the point clouds for clustering, filter out the points that do not meet the distance requirements and the normal vector angle requirements, perform region growing filtering on the point cloud, and extract the point cloud clusters that meet the normal vector angle and distance conditions.

[0024] Specifically, step S43 is as follows:

[0025] S431: Slice the point cloud after the outward shift described in S42 along the Z axis using the set spraying spacing, and perform voxel downsampling on each slice to generate a single strip of sheet-like point cloud;

[0026] S432: For each sheet-like point cloud:

[0027] Project the point cloud onto the contour plane to form two-dimensional points in XY coordinates;

[0028] On a series of two-dimensional points, use the Douglas-Peucker algorithm to fit a two-dimensional flight path;

[0029] Add the Z dimension to the two-dimensional flight path to return to the three-dimensional flight path;

[0030] S433: Connect the fitted flight paths of each sheet-like point cloud in the order of the three-dimensional flight path to make it a continuous flight path within the mesh graph; among them, the heading of the flight path is the normal vector of the current flight path facing the wall; the fitted flight path of the sheet-like point cloud is the spraying flight path, and the line segment connecting the fitted flight paths of each sheet-like point cloud is the non-spraying flight path;

[0031] S434: Traverse each segment in the flight path, and add the opening and closing action points of the nozzle within the flight path segments that need to be sprayed.

[0032] Specifically, the specific steps for the UAV flight path planning of the two-dimensional grid pattern to be sprayed in step S4 are as follows:

[0033] S45: Convert the two-dimensional grid pattern to be sprayed into a two-dimensional picture;

[0034] S46: Based on the two-dimensional picture described in S45, achieve flight path planning at the pixel level and generate a flight path in the pixel coordinate system;

[0035] S47: Post-process the pixel flight path and convert the flight path in the pixel coordinate system described in S46 into a flight path in the three-dimensional coordinate system required by the UAV.

[0036] Furthermore, the specific steps for converting the two-dimensional grid pattern to be sprayed into a two-dimensional picture in step S45 are as follows:

[0037] S451: Calculate the size of the two-dimensional picture and prepare the base map: Calculate the minimum bounding rectangle according to the coordinates of the polygon vertices of the two-dimensional grid pattern to be sprayed;

[0038] S452: Determine the width and height of the picture according to the physical width and height of the calculated minimum bounding rectangle, and in combination with the pixel accuracy set in the configuration file. Initialize a base map filled with all 0s according to the width and height of the picture;

[0039] S453: Convert the physical coordinates of the polygon vertices of the two-dimensional grid pattern to be sprayed into pixel coordinates;

[0040] S454: Use color levels for differentiation and draw the normal spraying work area and the non-sprayable area within the spraying work area on the base map respectively.

[0041] Furthermore, the specific steps for achieving flight path planning at the pixel level and generating a flight path in the pixel coordinate system in step S46 are as follows:

[0042] S461: Expand the non-sprayable area: Determine the Kernel for the erosion operation according to the width and height of the nozzle spraying shape, and expand the non-sprayable area by eroding the picture according to the Kernel;

[0043] S462: Conduct flight path planning for the original spraying work area:

[0044] Calculate the spraying width and height of the spray gun according to the set spraying width and height parameters;

[0045] Set the spraying interval according to the path priority;

[0046] Adjust the non-divisible part by calculating the offset between the image size and the spraying interval;

[0047] Generate the grid coordinates of the flight path waypoints;

[0048] Generate a flight path index and store the actual coordinates of the waypoints.

[0049] S463: If there are non-sprayable areas in the spraying work area, use the non-sprayable areas in the 2D image as a mask to mask the initial flight path, regenerate a new spraying flight path segment, and generate a map for the A* algorithm at the same time:

[0050] Create an A* map for marking passable areas;

[0051] Traverse each waypoint to determine whether the waypoint is located in a sprayable area:

[0052] If the path is horizontal-axis first, reverse the flight path to ensure that the starting point of the spraying flight path is at the position with a higher plane in the spraying work area;

[0053] Return the filtered obstacle-free waypoints and the map for the A* algorithm;

[0054] S464: Use the nearest distance rule to pair the flight paths to be sprayed and obtain the flight path spraying order;

[0055] S465: According to the pairing result, use the A* algorithm to connect each segmented flight path to be sprayed to form a complete pixel flight path;

[0056] S466: Traverse each segment in the flight path, and add the opening and closing action points of the nozzle inside the flight path segment to be sprayed. Protect the non-sprayable areas to avoid spraying paint onto non-sprayable areas during spraying operations.

[0057] Specifically, the specific steps of using the 3D model editing tool of the 3D visualization platform to draw the 3D mesh graph to be sprayed on the building 3D model in step S3 are as follows:

[0058] S31: Use the line segment tool of the 3D visualization platform to enclose a closed 3D mesh graph with line segments on the building surface of the 3D model;

[0059] S32: Based on the 3D network graph in step S31, calculate the triangles located inside the 3D mesh graph, outside the area, and on the area boundary respectively;

[0060] S33: For several triangles located on the area boundary, use the area boundary to cut these triangles, and the triangles on the area boundary are divided into two parts, one inside and one outside;

[0061] S34: Combine the part of the triangles on the area boundary that is inside with the triangles originally inside the area to form a new 3D mesh graph, and this new 3D mesh graph is the 3D mesh graph to be sprayed;

[0062] The specific steps of using the 3D model editing tool of the 3D visualization platform to draw the two-dimensional grid graph to be sprayed on the building 3D model in step S3 are as follows:

[0063] S35: Use the point selection tool of the 3D visualization platform to select at least 6 vertices on the building surface of the 3D model, and use a geometric plane to fit these 3D vertices through the least squares method and matrix singular value decomposition; among them, the geometric plane is parallel to the outer wall to be sprayed of the building model, denoted as the auxiliary plane;

[0064] S36: Use the drawing tool of the 3D visualization platform to draw the two-dimensional grid graph to be sprayed on the auxiliary plane described in S35; the two-dimensional grid graph to be sprayed drawn on the auxiliary plane is parallel and fitted to the building surface; if there are non-sprayable areas on the building surface, use the non-sprayable area marking tool to mark the non-sprayable two-dimensional grid graph.

[0065] Specifically, the specific process of coordinate system data conversion of the building 3D model in step S2 is as follows:

[0066] S21: There is a matrix composed of 16 floating-point numbers in the metadata of the building 3D model. The local tangent plane coordinate system of this matrix is the east-north-up coordinate system ENU. Multiply the three-dimensional coordinates in the building 3D model coordinate system by the matrix and convert them into three-dimensional coordinates in the earth rectangular coordinate system;

[0067] S22: Use the conversion method in the cesium open source library to convert the three-dimensional coordinates in the earth rectangular coordinate system into positioning coordinates that can be used by the drone. The positioning coordinates include longitude, latitude, and elevation.

[0068] Specifically, the specific process of step S1 is as follows:

[0069] S11: Conduct oblique photography through an oblique photography drone, and use a third-party modeling platform for 3D modeling to obtain the oblique photography 3D model of the building to be sprayed;

[0070] S12: Based on the three.js open source library, establish a 3D visualization platform that supports 3D editing tools;

[0071] S13: Render the oblique photography 3D model into the 3D visualization platform using the rendering engine of three.js. Description of the Drawings

[0072] Figure 1 is the overall flowchart of the present invention;

[0073] Figure 2 is the schematic diagram of the spraying work area drawing process provided by the present invention;

[0074] Figure 3It is the flowchart of the route planning algorithm for the building plane spraying work area provided by the present invention;

[0075] Figure 4 It is the schematic diagram of the graphic flow of the route planning algorithm for the building plane spraying work area provided by the present invention;

[0076] Figure 5 It is the flowchart of the route planning algorithm for the building curved surface spraying work area provided by the present invention;

[0077] Figure 6 It is the schematic diagram of the graphic flow of the route planning algorithm for the building curved surface spraying work area provided by the present invention;

[0078] Figure 7 It is the schematic diagram of the route planning result for the building plane spraying work area provided by the present invention;

[0079] Figure 8 It is the schematic diagram of the route planning result for the building curved surface spraying work area provided by the present invention. Detailed implementation manners

[0080] First of all, those skilled in the art should understand that these implementation manners are only used to explain the technical principles of the embodiments of the present application, and are not intended to limit the protection scope of the embodiments of the present application. Those skilled in the art can adjust it as needed to adapt to specific application scenarios.

[0081] The following further elaborates on the present invention in detail with reference to the drawings and specific embodiments.

[0082] The spraying route planning method is the key to realizing the automatic spraying of the building exterior wall by the spraying drone. The spraying drone performs the spraying task on the building exterior wall according to the preset route. The route planning can ensure that the drone maintains a stable spraying distance against the wall during the spraying process, reaches the specified spraying position, and realizes an accurate spraying effect.

[0083] Based on the above tenet, a spraying route planning method for realizing the automatic spraying of the building exterior wall by the drone in this preferred embodiment is as Figure 1 shown. This spraying route planning method draws the spraying work area and plans the spraying work area route for the building three-dimensional model, specifically including the following steps:

[0084] S1: Obtain the oblique photography three-dimensional model of the spraying building object and render the oblique photography three-dimensional model into the three-dimensional visualization platform.

[0085] Among them, the specific process of obtaining and rendering the three-dimensional model is as follows:

[0086] S11: Use an oblique photography drone to perform oblique photography on the building object to be sprayed, and use a third-party modeling platform for 3D modeling to obtain an oblique photography 3D model of the sprayed building;

[0087] S12: Based on the three.js open source library, establish a 3D visualization platform that supports 3D editing tools;

[0088] S13: Render the oblique photography 3D model into the 3D visualization platform by using the rendering engine of three.js.

[0089] S2: Convert the coordinate system data of the building 3D model in the 3D visualization platform into the coordinate data used for drone positioning. The coordinate data includes longitude, latitude, and elevation.

[0090] Among them, the specific process of coordinate system data conversion is as follows:

[0091] S21: There is a matrix composed of 16 floating-point numbers in the metadata of the building 3D model. The body-centered coordinate system of this matrix is the east-north-up coordinate system ENU. Multiply the 3D coordinates in the building 3D model coordinate system by the matrix to convert them into 3D coordinates in the earth rectangular coordinate system;

[0092] S22: Use the conversion method (Cartographic.fromCartesian) in the cesium open source library to convert the 3D coordinates in the earth rectangular coordinate system into the positioning coordinates that can be used by the drone, including longitude, latitude, and elevation (WGS84 coordinate system).

[0093] S3: Based on the 3D model editing tool in the 3D visualization platform, draw a 2D / 3D grid graph to be sprayed on the building 3D model, which is the plane / curved surface spraying work area of the building to be sprayed.

[0094] That is, the building surface spraying work area is divided into a curved surface spraying work area and a plane spraying work area. The drawing steps for the two spraying work areas are different, as Figure 2 shown.

[0095] For the curved surface spraying work area, that is, when the target work area to be sprayed on the building surface is a curved surface, the specific process is as follows:

[0096] S31: Use the line segment tool of the 3D visualization platform to enclose a closed 3D grid graph (curved surface spraying work area) on the building surface of the building 3D model;

[0097] S32: Based on the 3D network graph in step S31, calculate the triangles located inside the area, outside the area, and on the area boundary in the 3D grid graph;

[0098] S33: For several triangles located at the region boundary, use the region boundary to cut these triangles. The triangles on the region boundary are divided into two parts, one inside and the other outside.

[0099] S34: Combine the part of the triangles on the region boundary that is inside with the triangles originally inside the region to form a new three-dimensional mesh graphic, which is the surface spraying work area.

[0100] For the planar spraying work area, that is, when the target area to be sprayed on the building surface is planar, the specific steps are as follows:

[0101] S35: Use the point selection tool of the three-dimensional visualization platform to select at least 6 vertices on the building surface of the building three-dimensional model. Through the least squares method and matrix singular value decomposition (SVD), use a geometric plane to fit these three-dimensional vertices; this geometric plane is almost completely parallel to the exterior wall to be sprayed of the building model, denoted as the auxiliary plane.

[0102] S36: Use the drawing tool of the three-dimensional visualization platform to draw a two-dimensional mesh graphic on the auxiliary plane described in S35. This two-dimensional mesh graphic is the planar spraying work area; the spraying work area drawn on the auxiliary plane will be parallel and conform to the building surface; if there are non-sprayable areas on the building surface, the non-sprayable two-dimensional mesh graphic can be marked using the non-sprayable area marking tool, and such areas will be avoided in the flight path generation process.

[0103] Furthermore, when drawing a two-dimensional mesh graphic in the three-dimensional visualization platform, the following functions are supported:

[0104] On the one hand, the planar spraying work area is default to be rectangular, and it supports adding and deleting vertices on the basis of the rectangular two-dimensional mesh graphic, dragging the positions of existing vertices, so as to modify the shape of the two-dimensional mesh graphic, and various two-dimensional graphics such as triangles, trapezoids, stars, and perforated graphics can be drawn.

[0105] On the other hand, perform two-dimensional Boolean operations on the drawn two-dimensional mesh graphic, and support union operations and difference operations between multiple two-dimensional mesh graphics. Draw two two-dimensional mesh rectangular graphics (rectangular planar spraying areas) on the same auxiliary plane. When the two two-dimensional mesh rectangular graphics partially overlap, use the color adjustment tool of the three-dimensional visualization platform to set the two two-dimensional mesh rectangular graphics to the same color, that is, the spraying work area will spray paint of the same color, then the two two-dimensional mesh rectangular graphics will be combined into 1 rectangular planar spraying work area. When the colors of these two two-dimensional mesh rectangular graphics are different, the covered part of the first drawn two-dimensional mesh rectangular graphic will be cut out from the original area and become a part of the later drawn two-dimensional mesh rectangular graphic.

[0106] S4: Apply the pixel-level flight path planning algorithm to the two-dimensional grid graphics to be sprayed in the 3D building model; apply the improved A* algorithm (also known as the AStar algorithm) for full coverage path planning to the three-dimensional grid graphics to be sprayed in the 3D building model; finally, plan the spraying flight path for the UAV to perform automated spraying operations on the building facade.

[0107] Further, the specific process of the path planning algorithm for the grid graphics (spraying work area) in step S4 is as follows:

[0108] For the curved surface spraying work area, that is, when the target spraying work area on the building surface is a curved surface, as Figure 5 、 6 and shown in Figure 8, the specific steps of the flight path planning are as follows:

[0109] S41: Obtain the 3D grid graphics of the building wall (curved surface spraying work area) and calculate the vertex normal vector. After loading the 3D grid graphics data, perform preprocessing, convert the grid data into point cloud data, filter the point cloud data to remove noise, and then solve the normal vector according to the adjacency relationship of the grid vertices.

[0110] The specific steps are as follows:

[0111] S411: Convert the grid data of the spraying work area drawn in the 3D building model into the off format, and use the Open3D library to create a triangular mesh object from the grid data;

[0112] S412: Sample the grid data into point cloud data. First, calculate the number of resampled point clouds, and use the poisson_disk algorithm in the Open3D library for point cloud resampling;

[0113] S413: Smooth and denoise all the resampled point cloud data; use the angle of the normal vector and the distance between the point clouds for clustering, filter out the points that do not meet the distance requirements (outlier positions) and the normal vector angle requirements (outlier normal vectors), perform region growing filtering on the point cloud, and extract the point cloud clusters that meet the normal vector angle and distance conditions.

[0114] S42: UAV body positioning. Translate the preprocessed point cloud in step S41 outward along the direction of the normal vector by a set distance (the distance from the building surface grid graphics). This set distance is generally the distance from the nozzle to the UAV body positioning module plus the optimal spraying distance of the nozzle to obtain the target position of the UAV body.

[0115] S43: Optimize the path to achieve full coverage spraying. Apply the full coverage path planning algorithm to connect the discrete UAV poses and generate a complete segmented flight path.

[0116] The specific steps are as follows:

[0117] S431: Slice the point cloud after the outward movement described in S42 according to the Z axis using the set spraying spacing, and perform voxel downsampling on each slice to generate a single sheet point cloud;

[0118] S432: Perform the following processing on each sheet point cloud:

[0119] S4321: Project the point cloud onto the plane of equal height to form two-dimensional points (this step only retains the XY coordinates of the point cloud for subsequent route fitting);

[0120] S4322: Fitting routes using the Douglas-Peucker algorithm on a series of two-dimensional points (reducing unnecessary waypoints);

[0121] S4323: Return the two-dimensional route to the three-dimensional route (add the Z dimension, the Z value of each route generated by the sheet point cloud is the same, which is the Z center point of the sheet point cloud);

[0122] S433: connecting each fitted route of the sheet point cloud in the Z direction of the three-dimensional route to form a continuous route in the grid graph, the route direction is the normal vector of the current route facing the wall, the fitted route of the sheet point cloud is the spraying route, and the line segment connecting each fitted route of the sheet point cloud is the non-spraying route;

[0123] S434: traverse each section of the route, add opening and closing action points of the nozzle in the route segment that needs to be sprayed, and ensure that the nozzle is opened and closed within an appropriate distance;

[0124] S4341: If the current route segment needs to be sprayed, the spraying direction and distance of the current segment are calculated according to the path priority (horizontal axis priority or vertical axis priority);

[0125] S4342: If the spraying distance of the current segment is greater than or equal to the sum of the distances of nozzle opening and closing, add nozzle opening and closing points;

[0126] (1) Calculate the distance between the spray gun start point and the starting point and set it to a non-spraying state;

[0127] (2) Calculate the distance between the closing point of the spray gun and the end point, and set it to spraying state;

[0128] (3) The spraying status of the start point and the end point are both set to non-spraying and returned to the route planning result.

[0129] S44: Perform route post-processing to calculate the waypoint heading angle and relative heading angle (the angle at which the drone needs to turn after reaching the specified position, the same below).

[0130] For the planar spraying work area, that is, when the target spraying work area on the building surface is planar, as Figure 3 , 4 and as shown in 7, the specific steps for flight path planning are as follows:

[0131] S45: Obtain the two-dimensional grid graph of the building facade (planar spraying work area), and convert the data of the planar spraying work area into a two-dimensional picture.

[0132] The specific steps are as follows:

[0133] S451: Calculate the size of the two-dimensional picture and prepare the base map: Calculate the minimum bounding rectangle according to the coordinates of the polygon vertices of the two-dimensional grid graph.

[0134] S452: Determine the width and height of the picture according to the physical width and height of the calculated minimum bounding rectangle, combined with the pixel accuracy set in the configuration file. Initialize a base map filled with 0s according to the width and height of the picture.

[0135] S453: Convert the physical coordinates of the polygon vertices of the two-dimensional grid graph to be sprayed into pixel coordinates.

[0136] S454: Use color levels for differentiation, and draw the normal spraying work area to be sprayed and the non-sprayable areas within the spraying work area on the base map respectively.

[0137] S46: Based on the two-dimensional picture described in S45, implement flight path planning at the pixel level and generate a flight path in the pixel coordinate system.

[0138] The specific steps are as follows:

[0139] S461: Expand the non-sprayable area. Determine the Kernel for the erosion operation according to the width and height of the spraying shape of the nozzle. Erode the picture according to the Kernel to expand the non-sprayable area and protect the non-sprayable area to avoid paint being sprayed onto the non-sprayable area during spraying operations.

[0140] S462: Perform flight path planning on the original spraying work area.

[0141] S4621: Calculate the spraying width and spraying height of the spray gun according to the set spraying width and height parameters.

[0142] S4622: Set the spraying interval according to the path priority (vertical axis first or horizontal axis first).

[0143] S4623: Make appropriate adjustments to the non-divisible part by calculating the offset between the image size and the spraying interval to ensure that the spraying work area evenly covers the image.

[0144] S4624: Generate the grid coordinates of the flight path waypoints.

[0145] (1) If the vertical axis is prioritized, generate a route grid with the vertical axis prioritized;

[0146] (2) If the horizontal axis is prioritized, generate a route grid with the horizontal axis prioritized;

[0147] S4625: Generate a flight index and store the actual coordinates of the waypoints;

[0148] S463: If there are non-sprayable areas in the spraying work area, use the non-sprayable areas in the two-dimensional image as a mask (mosaic) to mask the initial route, regenerate the new spraying route segments, and generate the map used by the A* algorithm simultaneously:

[0149] S4631: Create an A* map for marking passable areas;

[0150] S4632: Traverse each waypoint and determine whether the waypoint is located in a sprayable area:

[0151] (1) According to the path priority (horizontal axis priority or vertical axis priority), determine whether the waypoint is outside the image edge (i.e., whether it is a valid waypoint);

[0152] (2) If the waypoint is located within the sprayable work area and not on the edge, mark the point as a passable area on the A* map;

[0153] S4633: If the path is horizontal axis prioritized, reverse the route path to ensure that the starting point of the spraying route is at the position with a higher plane in the spraying work area;

[0154] S4634: Return the filtered obstacle-free waypoints and the map used by the A* algorithm;

[0155] S464: Use the nearest distance rule to pair the routes to be sprayed and obtain the route spraying order;

[0156] S465: According to the pairing result, use the A* algorithm to connect each segmented route to be sprayed to form a complete pixel route;

[0157] S466: Traverse each segment in the route and add the opening and closing action points of the nozzle within the route segment to be sprayed to ensure that the nozzle is opened and closed at an appropriate distance:

[0158] S4661: If the current route segment needs to be sprayed, calculate the spraying direction and distance of the current segment according to the path priority (horizontal axis priority or vertical axis priority);

[0159] S4662: If the spraying distance of the current segment is greater than or equal to the sum of the opening and closing distances of the nozzle, add the nozzle opening and closing points;

[0160] (1) Calculate the distance of the position of the spray gun starting point from the starting point and set it to the non-spraying state;

[0161] (2) Calculate the distance of the position of the spray gun closing point from the ending point and set it to the spraying state;

[0162] (3) Set the spraying states of both the starting point and the ending point to non-spraying and return them to the route planning result.

[0163] S47: Post-process the pixel route, and convert the route in the pixel coordinate system described in S46 into the route in the three-dimensional coordinate system required by the drone.

[0164] This spraying route planning method enables the spraying drone to autonomously complete the spraying task without manual intervention and assistance, improves the automation level of the drone spraying operation, can ensure that the drone reaches the specified position during spraying, realizes precise spraying, and avoids the need for manual control of the traditional spraying drone to spray the building exterior wall.

[0165] Due to the complex shape of the building exterior wall, more precise route planning is required to ensure the spraying effect. Three-dimensional visualization technology is applied to the route planning. The route planning scheme described in this method is generated based on the three-dimensional real model of the building, and combined with the parameters of the spraying drone, it can accurately control the spraying range of the building exterior wall to achieve high-precision positioning and spraying.

[0166] Based on the three-dimensional visualization platform, the spraying work area is drawn, which can adapt to the building exterior walls of various complex shapes and structures. At the same time, it can realize the spraying of complex patterns and textures, improving the aesthetic degree of the building exterior wall.

[0167] The improved A* algorithm under the full-coverage path planning described in this method can find a feasible path from the starting point to the ending point and covering all key points in the given area when the starting point and the ending point are given, realizing the full coverage of the spraying work area, ensuring that the spraying drone can access and cover each area in the environment, and avoiding the problem of missed spraying.

[0168] As described above, it is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in this application should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A spraying route planning method for realizing automatic spraying of the building facade by a drone, characterized in that, Specifically, it includes the following steps: S1: Obtain the three-dimensional model of the building to be sprayed by oblique photography, and render the three-dimensional model of oblique photography into the three-dimensional visualization platform; S2: Convert the coordinate system data of the building three-dimensional model in the three-dimensional visualization platform to obtain the coordinate data used for UAV positioning. The coordinate data includes longitude, latitude, and elevation; S3: Use the 3D model editing tool of the three-dimensional visualization platform to draw the two-dimensional / three-dimensional grid graphics to be sprayed on the building three-dimensional model. The two-dimensional grid graphics to be sprayed are the plane spraying work area of the building to be sprayed, and the three-dimensional grid graphics to be sprayed are the curved surface spraying work area of the building to be sprayed; S4: Use the pixel-level flight path planning algorithm for the two-dimensional grid graphics to be sprayed, and use the improved A* algorithm for the three-dimensional grid graphics to be sprayed under the full coverage path planning. Combine the coordinate data used for UAV positioning to plan the spraying route for the UAV to perform automatic spraying operations on the building facade.

2. The spraying route planning method for realizing automatic spraying of the building exterior wall by a drone according to claim 1, wherein The specific steps of the UAV flight path planning for the three-dimensional grid graphics to be sprayed in step S4 are as follows: S41: Calculate the normal vector of the grid vertices according to the three-dimensional grid graphics to be sprayed: After loading the three-dimensional grid graphics to be sprayed, perform preprocessing, convert the grid data into point cloud data, and solve the normal vector according to the adjacency relationship of the grid vertices after filtering the noise points from the point cloud data; S42: UAV fuselage positioning: Translate the preprocessed point cloud in step S41 outward along the direction of the normal vector by a set distance to obtain the target position of the UAV fuselage; among them, the set distance is the distance from the nozzle to the UAV fuselage positioning module plus the optimal spraying distance of the nozzle; S43: Optimize the path to achieve full coverage spraying: Use the improved A* algorithm for full coverage path planning to connect the discrete UAV poses to generate a complete segmented flight path; S44: Perform post-processing on the flight path to calculate the heading angle and relative heading angle of the waypoints.

3. The spraying route planning method for automatically spraying the building facades by an unmanned aerial vehicle according to claim 2, wherein, The specific steps of filtering and removing noise points from the point cloud data in step S41 are as follows: S411: Convert the three-dimensional grid graphics to be sprayed into off-format grid data, and use the Open3D library to create a triangular mesh object from the grid data; S412: Sample the grid data as point cloud data, and use the poisson_disk algorithm in the Open3D library for point cloud resampling; S413: Smooth and denoise all resampled point cloud data: Use the angle of the normal vector and the distance between point clouds to perform clustering, filter out points that do not meet the distance requirements and normal vector angle requirements, perform region growing filtering on the point cloud, and extract point cloud clusters that meet the normal vector angle and distance conditions.

4. The spraying route planning method for realizing automatic spraying of the building exterior wall by an unmanned aerial vehicle according to claim 3, characterized in that, The specific steps of step S43 are as follows: S431: Slice the point cloud after the outward translation described in S42 along the Z axis using a set spraying spacing, and perform voxel downsampling on each slice to generate a single strip of sheet-like point cloud; S432: Perform the following operations on each sheet-like point cloud: Project the point cloud onto the contour plane to form two-dimensional points of XY coordinates; On a series of two-dimensional points, use the Douglas-Peucker algorithm to fit the two-dimensional flight path; Add the Z dimension to the two-dimensional flight path to return to the three-dimensional flight path; S433: Connect the fitted flight paths of each sheet of point cloud in the order of the three-dimensional flight path to form a continuous flight path within the grid pattern; wherein, the course of the flight path is the normal vector of the current flight path facing the wall; the fitted flight path of the sheet of point cloud is the spraying flight path, and the line segment connecting the fitted flight paths of each sheet of point cloud is the non-spraying flight path; S434: Traverse each segment in the flight path and add the opening and closing action points of the nozzle within the flight path segments that need to be sprayed.

5. A spraying route planning method for realizing automatic spraying of the building facade by a drone according to any one of claims 1 to 4, characterized in that, The specific steps of the UAV flight path planning for the two-dimensional grid pattern to be sprayed in step S4 are as follows: S45: Convert the two-dimensional grid pattern to be sprayed into a two-dimensional picture; S46: Based on the two-dimensional picture described in S45, implement flight path planning at the pixel level to generate a flight path in the pixel coordinate system; S47: Post-process the pixel flight path and convert the flight path in the pixel coordinate system described in S46 into a flight path in the three-dimensional coordinate system required by the UAV.

6. The spraying route planning method for automatically spraying the exterior facade of a building by a drone according to claim 5, characterized in that, The specific steps of converting the two-dimensional grid pattern to be sprayed into a two-dimensional picture in step S45 are as follows: S451: Calculate the size of the two-dimensional picture and prepare the base map: Calculate the minimum bounding rectangle according to the coordinates of the polygon vertices of the two-dimensional grid pattern to be sprayed; S452: Determine the width and height of the picture based on the physical width and height of the calculated minimum bounding rectangle and the pixel accuracy set in the configuration file, and initialize a base map full of 0s according to the width and height of the picture; S453: Convert the physical coordinates of the polygon vertices of the two-dimensional grid pattern to be sprayed into pixel coordinates; S454: Use color levels for distinction and draw the normal spraying work area and the non-sprayable area within the spraying work area on the base map respectively.

7. According to a spraying flight path planning method for realizing automatic UAV spraying of building facades according to claim 6, characterized in that The specific steps of implementing flight path planning at the pixel level in step S46 to generate a flight path in the pixel coordinate system are as follows: S461: Expand the non-sprayable area: Determine the Kernel for the erosion operation according to the width and height of the nozzle spraying shape, and expand the non-sprayable area by eroding the picture according to the Kernel; S462: Conduct flight path planning for the original spraying work area: Calculate the spraying width and height of the spray gun according to the set spraying width and height parameters; Set the spraying interval according to the path priority; Adjust the non-divisible part by calculating the offset between the image size and the spraying interval; Generate the grid coordinates of the flight path points; Generate a flight path index and store the actual coordinates of the waypoints; S463: If there is a non-sprayable area in the spraying work area, use the non-sprayable area in the two-dimensional picture as a mask to mask the initial flight path, regenerate a new spraying flight path segment, and generate the map used by the A* algorithm at the same time: Create an A* map for marking the passable area; Traverse each waypoint and determine whether the waypoint is located in the sprayable area: If the path is horizontal axis first, reverse the flight path to ensure that the starting point of the spraying flight path is at the position with a higher plane in the spraying work area; Return the filtered obstacle-free waypoints and the map for the A* algorithm; S464: Use the nearest distance rule to pair the flight paths to be sprayed and obtain the spraying order of the flight paths; S465: According to the pairing result, use the A* algorithm to connect each segmented spraying route to form a complete pixel route; S466: Traverse each segment of the route and add the opening and closing action points of the nozzle within the route segment to be sprayed.

8. A spraying route planning method for realizing automatic spraying of the exterior facade of a building by an unmanned aerial vehicle according to claim 7, characterized in that The specific steps of using the 3D model editing tool of the three-dimensional visualization platform in step S3 to draw the three-dimensional grid graph to be sprayed on the three-dimensional building model are as follows: S31: Use the line segment tool of the three-dimensional visualization platform to enclose a closed three-dimensional grid graph with line segments on the building surface of the three-dimensional model; S32: Based on the three-dimensional network graph in step S31, calculate the triangles located inside the area, outside the area, and on the area boundary in the three-dimensional grid graph; S33: For several triangles located on the area boundary, cut these triangles with the area boundary, and the triangles on the area boundary are divided into two parts, one inside and one outside; S34: Combine the part of the triangle on the area boundary that is inside with the triangles originally inside the area to form a new three-dimensional grid graph, and this new three-dimensional grid graph is the three-dimensional grid graph to be sprayed; The specific steps of using the 3D model editing tool of the three-dimensional visualization platform in step S3 to draw the two-dimensional grid graph to be sprayed on the three-dimensional building model are as follows: S35: Use the point selection tool of the three-dimensional visualization platform to select at least 6 vertices on the building surface of the three-dimensional model, and fit these three-dimensional vertices with a geometric plane through the least squares method and matrix singular value decomposition; among them, the geometric plane is parallel to the exterior wall to be sprayed of the building model, denoted as the auxiliary plane; S36: Use the drawing tool of the three-dimensional visualization platform to draw the two-dimensional grid graph to be sprayed on the auxiliary plane in step S35; the two-dimensional grid graph to be sprayed drawn on the auxiliary plane is parallel and fitted to the building surface; if there are non-sprayable areas on the building surface, use the non-sprayable area marking tool to mark the non-sprayable two-dimensional grid graph.

9. The spraying route planning method for realizing automatic spraying of the building exterior wall by an unmanned aerial vehicle according to claim 7, wherein, The specific process of the coordinate system data conversion of the three-dimensional building model in step S2 is as follows: S21: There is a matrix composed of 16 floating-point numbers in the metadata of the three-dimensional building model, and the earth-centered coordinate system of this matrix is the east-north-up coordinate system ENU. Multiply the three-dimensional coordinates in the three-dimensional building model coordinate system by the matrix and convert them into three-dimensional coordinates in the earth rectangular coordinate system; S22: Use the conversion method in the cesium open source library to convert the three-dimensional coordinates in the earth rectangular coordinate system into the positioning coordinates that can be used by the unmanned aerial vehicle, and the positioning coordinates include longitude, latitude, and elevation.

10. A spraying route planning method for realizing automatic spraying of the building exterior wall by an unmanned aerial vehicle according to claim 7, characterized in that, The specific process of step S1 is as follows: S11: Conduct oblique photography by an oblique photography unmanned aerial vehicle, and use a third-party modeling platform to conduct three-dimensional modeling to obtain the oblique photography three-dimensional model of the building to be sprayed; S12: Based on the three.js open source library, establish a three-dimensional visualization platform that supports 3D editing tools; S13: Render the oblique photography three-dimensional model into the three-dimensional visualization platform using the rendering engine of three.js.

Images