A spraying route planning method for realizing automatic spraying of an unmanned aerial vehicle on a building facade

Abstract

The application discloses a kind of to realize unmanned plane automatic spraying building facade spraying flight path planning method, specifically, S1: obtain the oblique photography three-dimensional model of spraying building object, and render to three-dimensional visualization platform;S2: the coordinate system data conversion of building three-dimensional model in three-dimensional visualization platform is obtained, and the coordinate data used for unmanned plane positioning;S3: using the 3D model editing tool of three-dimensional visualization platform draws out the two-dimensional / three-dimensional grid figure to be sprayed indicating building plane / curved surface spraying work area on building three-dimensional model;S4: two-dimensional grid figure drawn in building three-dimensional model is using pixel level flight path planning algorithm, three-dimensional grid figure drawn in building three-dimensional model is using the improved A* algorithm for full coverage path planning, and combined with the coordinate data used for unmanned plane positioning, the spraying flight path of unmanned plane for building facade automatic spraying operation is planned.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of aerial operation robots, in particular to a spraying flight path planning method for realizing automatic spraying of building facade by unmanned aerial vehicle. BACKGROUND

[0002] With the development of building technology, the existing building height also increases, but for the high-rise building that needs to be sprayed, spraying is extremely inconvenient. The traditional spraying method is to spray by artificial, but spraying by artificial high-rise building not only has high cost and low efficiency, but also has the risk of falling in the spraying process of workers. In order to improve efficiency, reduce cost and safety risk, more and more enterprises begin to consider using spraying unmanned aerial vehicle to replace workers to paint the building facade, so as to save production cost and avoid personnel safety problems. Enterprises choose spraying unmanned aerial vehicle to be applied to building facade painting, maintenance, cleaning and other fields, spraying unmanned aerial vehicle is much cheaper than hiring workers, has higher efficiency and shorter construction period.

[0003] At present, there is no mature flight path planning solution for autonomous spraying operation of spraying unmanned aerial vehicle, manual control of unmanned aerial vehicle is needed for spraying operation, it is difficult to realize accurate coverage of the spraying area, and improvement is needed. SUMMARY

[0004] One of the technical problems to be solved by the present application is to overcome the defects of the above related technologies, and to provide a spraying flight path planning method for realizing automatic spraying of building facade by unmanned aerial vehicle, which plans spraying work area and generates spraying flight path for building three-dimensional model, realizes autonomous spraying operation of building facade by unmanned aerial vehicle according to spraying flight path, adapts to various building facade spraying scenes, improves spraying operation efficiency and quality of unmanned aerial vehicle, and achieves the purpose of reducing cost and increasing efficiency.

[0005] The technical solution adopted by the present application to solve the technical problem is: a spraying flight path planning method for realizing automatic spraying of building facade by unmanned aerial vehicle, specifically comprising the following steps:

[0006] S1: obtaining an oblique photography three-dimensional model of a spraying building object, and rendering the oblique photography three-dimensional model to a three-dimensional visualization platform;

[0007] S2: performing coordinate system data conversion on the building three-dimensional model in the three-dimensional visualization platform to obtain coordinate data used for positioning of the unmanned aerial vehicle, the coordinate data containing longitude, latitude and elevation;

[0008] S3: using a 3D model editing tool of the three-dimensional visualization platform to draw a two-dimensional / three-dimensional grid pattern to be sprayed on the building three-dimensional model, the two-dimensional grid pattern to be sprayed being a building plane spraying work area to be sprayed, and the three-dimensional grid pattern to be sprayed being a building curved surface spraying work area to be sprayed;

[0009] S4: using a pixel-level route planning algorithm for the two-dimensional grid pattern to be sprayed, using an improved A* algorithm for full coverage route planning for the three-dimensional grid pattern to be sprayed, and combining the coordinate data used for positioning of the unmanned aerial vehicle to plan a spraying route for the unmanned aerial vehicle to perform automatic spraying of the building facade.

[0010] Compared with the related art, the present application has the following advantages:

[0011] 1. The spraying unmanned aerial vehicle can autonomously complete the spraying task without manual intervention and assistance, thereby improving the automation level of the spraying operation of the unmanned aerial vehicle and ensuring that the unmanned aerial vehicle reaches the specified position during spraying to achieve precise spraying, thereby avoiding the need for manual control of the spraying operation of the building facade by the traditional spraying unmanned aerial vehicle.

[0012] 2. The three-dimensional visualization technology is applied to route planning, so that the route scheme planned by the present method is generated based on a three-dimensional model of the actual building scene and can precisely control the spraying range of the building facade in combination with the parameters of the spraying unmanned aerial vehicle to achieve high-precision positioning and spraying.

[0013] 3. The 3D model editing tool based on the three-dimensional visualization platform completes the drawing of the spraying work area, can adapt to building facades of various complex shapes and structures, and can also achieve spraying of complex patterns and textures to improve the aesthetic level of the building facade.

[0014] 4. The improved A* algorithm under full coverage route planning finds a feasible route from the starting point to the ending point that covers all key points in the given area under the condition that the starting point and the ending point are given, thereby achieving comprehensive coverage of the spraying work area and ensuring that the spraying unmanned aerial vehicle can access and cover every area in the environment to avoid the problem of missed spraying.

[0015] Specifically, the specific steps of the unmanned aerial vehicle route planning for the three-dimensional grid pattern to be sprayed in step S4 are as follows:

[0016] S41: calculating the grid vertex normal vector according to the three-dimensional grid pattern to be sprayed: after loading the three-dimensional grid pattern to be sprayed, pre-processing is performed, the grid data is converted into point cloud data, the point cloud data is filtered to remove noise points, and then the normal vector is solved according to the grid vertex adjacency relationship;

[0017] S42: positioning of the unmanned aerial vehicle body: the pre-processed point cloud described in S41 is translated outward along the direction of the normal vector by a set distance to obtain the target position of the unmanned aerial vehicle body; wherein the set distance is the distance from the spray head to the unmanned aerial vehicle body positioning module plus the optimal spraying distance of the spray head.

[0018] S43: Optimize the path to achieve full-coverage spraying: use the improved A* algorithm for full-coverage path planning to connect discrete UAV poses and generate a complete segmented flight path.

[0019] S44: Perform flight path post-processing to calculate waypoint heading angles and relative heading angles.

[0020] Further, the specific steps of filtering and removing noise points in the point cloud data in S41 are as follows:

[0021] S411: Convert the three-dimensional grid pattern to be sprayed into off format grid data, and use Open3D library to create a triangular mesh object from the grid data;

[0022] S412: Use grid data sampling for point cloud data, and use poisson_disk algorithm in Open3D library for point cloud resampling;

[0023] S413: Smooth and denoise all resampled point cloud data: use the angle of the normal vector and the distance between the point cloud to cluster and filter out points that do not meet the distance and normal vector angle requirements, and perform region growing filtering on the point cloud to extract point cloud clusters that meet the normal vector angle and distance conditions.

[0024] Specifically, S43 is as follows:

[0025] S431: Slice the point cloud after moving outward based on S42 according to the Z-axis using a set spraying interval, and perform voxel downsampling on each slice to generate a single slice-shaped point cloud;

[0026] S432: For each slice-shaped point cloud:

[0027] Project the point cloud onto the contour plane to form a two-dimensional point in XY coordinates;

[0028] Fit a two-dimensional flight path on a series of two-dimensional points using the Douglas-Peucker algorithm;

[0029] Add Z dimension to the two-dimensional flight path to return to a three-dimensional flight path;

[0030] S433: Connect the fitted flight path of each slice-shaped point cloud in the order of the three-dimensional flight path to make it a continuous flight path within the grid pattern; wherein the heading of the flight path is the normal vector of the wall surface facing the current flight path; the fitted flight path of the slice-shaped point cloud is the spraying flight path, and the line segment connecting each slice-shaped point cloud fitted flight path is the non-spraying flight path;

[0031] S434: Traverse each segment of the flight path and add opening and closing action points of the spray head in the flight path segment that needs to be sprayed.

[0032] Specifically, the specific steps of the UAV flight path planning of the two-dimensional grid pattern to be sprayed in step S4 are as follows:

[0033] S45: converting the two-dimensional grid pattern to be sprayed into a two-dimensional picture;

[0034] S46: realizing pixel-level flight path planning based on the two-dimensional picture in S45 to generate a flight path in a pixel coordinate system;

[0035] S47: post-processing the pixel flight path to convert the flight path in the pixel coordinate system in S46 into a flight path in a three-dimensional coordinate system required by the UAV.

[0036] Further, the specific steps of converting the two-dimensional grid pattern to be sprayed into a two-dimensional picture in step S45 are as follows:

[0037] S451: calculating the size of the two-dimensional picture and preparing a base map: calculating the minimum circumscribed rectangle according to the coordinates of the polygon vertices of the two-dimensional grid pattern to be sprayed;

[0038] S452: determining the picture width and height according to the physical width and height of the calculated minimum circumscribed rectangle and combining the pixel accuracy set in the configuration file, initializing a base map with all 0s according to the picture width and height;

[0039] S453: converting the physical coordinates of the polygon vertices of the two-dimensional grid pattern to be sprayed into pixel coordinates;

[0040] S454: distinguishing by color scale, respectively drawing the normal spraying work area to be sprayed and the area that cannot be sprayed in the spraying work area on the base map.

[0041] Further, the specific steps of realizing pixel-level flight path planning in step S46 to generate a flight path in a pixel coordinate system are as follows:

[0042] S461: expanding the non-spraying area: determining the Kernel of the erosion operation according to the width and height of the spray head spraying shape, and expanding the non-spraying area according to the Kernel erosion picture;

[0043] S462: flight path planning for the original spraying work area:

[0044] calculating the spraying width and spraying height of the spray gun according to the set spraying width and height parameters;

[0045] setting the spraying interval according to the path priority;

[0046] adjusting the part that cannot be divided by the offset amount of the image size and the spraying interval;

[0047] generating the grid coordinates of the flight path points;

[0048] Generate the route index and store the actual coordinates of the waypoints;

[0049] S463: If there is an un-spraying area in the spraying work area, use the un-spraying area in the two-dimensional picture as a mask to mask the initial route, regenerate a new spraying route segment, and generate a map for A* algorithm:

[0050] Create an A* map to mark the passable area;

[0051] Traverse each waypoint to determine whether the waypoint is located in the sprayable area:

[0052] If the path is horizontal axis first, reverse the route path to ensure that the starting point of the spraying route is at a high position on the spraying work area plane;

[0053] Return the filtered obstacle-free waypoints and the map for A* algorithm;

[0054] S464: Use the nearest distance rule to pair the spraying routes and obtain the spraying order of the routes;

[0055] S465: According to the pairing result, use A* algorithm to connect each segment of the spraying route to form a complete pixel route;

[0056] S466: Traverse each segment of the route and add the opening and closing action points of the spray head in the spraying route segment. Protect the un-spraying area to avoid paint spraying in the spraying operation.

[0057] Specifically, the specific steps of S3 step of using 3D model editing tool of three-dimensional visualization platform to draw the three-dimensional grid pattern to be sprayed on the building three-dimensional model are:

[0058] S31: Use the line segment tool of the three-dimensional visualization platform to enclose a closed three-dimensional grid pattern on the surface of the three-dimensional model with line segments;

[0059] S32: Based on the three-dimensional network pattern of step S31, calculate the triangles in the three-dimensional grid pattern respectively located inside the area, outside the area, and on the boundary of the area;

[0060] S33: For the several triangles located on the boundary of the area, cut these triangles with the area boundary, and the triangles on the boundary of the area are divided into two parts inside and outside;

[0061] S34: Combine the part inside the area boundary of the triangle on the area boundary with the triangles originally inside the area to form a new three-dimensional grid pattern, which is the three-dimensional grid pattern to be sprayed;

[0062] The specific steps of S3 step of drawing the specific two-dimensional grid pattern to be sprayed on the three-dimensional model of the building using the 3D model editing tool of the three-dimensional visualization platform are as follows:

[0063] S35: At least six vertices are selected on the building surface of the three-dimensional model using the point selection tool of the three-dimensional visualization platform, and a geometric plane is used to fit the three-dimensional vertices through the least square method and matrix singular value decomposition; wherein the geometric plane is parallel to the to-be-sprayed outer wall of the building model, and is recorded as an auxiliary plane;

[0064] S36: The two-dimensional grid pattern to be sprayed is drawn on the auxiliary plane described in S35 using the drawing tool of the three-dimensional visualization platform; the two-dimensional grid pattern to be sprayed drawn on the auxiliary plane is parallel to the building surface; if the building surface has an unsprayed area, the two-dimensional grid pattern to be sprayed is marked using the unsprayed area marking tool.

[0065] Specifically, the conversion process of the coordinate system data of the building three-dimensional model in S2 step is as follows:

[0066] S21: The matrix composed of 16 floating points in the metadata of the building three-dimensional model is the ENU (East-North-Up) coordinate system, and the three-dimensional coordinates in the building three-dimensional model coordinate system are converted into three-dimensional coordinates in the Earth Rectangular Coordinate System (ERCS) by multiplying the matrix;

[0067] S22: The three-dimensional coordinates in the ERCS are converted into positioning coordinates that can be used by the unmanned aerial vehicle using the conversion method in the cesium open source library, and the positioning coordinates include longitude, latitude and elevation.

[0068] Specifically, the specific process of S1 step is as follows:

[0069] S11: Oblique photography is performed by an oblique photography unmanned aerial vehicle, three-dimensional modeling is performed by using a third-party modeling platform, and an oblique photography three-dimensional model of the sprayed building is obtained;

[0070] S12: Based on the three.js open source library, a three-dimensional visualization platform supporting 3D editing tools is established;

[0071] S13: The oblique photography three-dimensional model is rendered into the three-dimensional visualization platform using the rendering engine of three.js. BRIEF DESCRIPTION OF DRAWINGS

[0072] Figure 1 is the overall flowchart of the present application;

[0073] Figure 2 is the spraying work area drawing flowchart provided by the present application;

[0074] Figure 3This is a flowchart of the flight path planning algorithm for the architectural planar spraying work area provided by the present invention;

[0075] Figure 4 This is a graphical flow diagram of the route planning algorithm for the building planar spraying work area provided by the present invention;

[0076] Figure 5 This is a flowchart of the flight path planning algorithm for the architectural curved surface spraying work area provided by the present invention;

[0077] Figure 6 This is a graphical flow diagram of the route planning algorithm for the architectural curved surface spraying work area provided by the present invention;

[0078] Figure 7 This is a schematic diagram of the flight path planning results for the architectural planar spraying work area provided by the present invention;

[0079] Figure 8 This is a schematic diagram of the route planning results for the architectural curved surface spraying work area provided by the present invention. Detailed Implementation

[0080] First, those skilled in the art should understand that these embodiments are merely used to explain the technical principles of the embodiments of this application and are not intended to limit the scope of protection of the embodiments of this application. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.

[0081] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0082] The spraying flight path planning method is the key to realizing the automated spraying of building facades by spraying drones. The spraying drone performs spraying tasks on the building facades according to the preset flight path. Flight path planning can ensure that the drone maintains a stable spraying distance from the wall during the spraying process, reaches the designated spraying position, and achieves precise spraying results.

[0083] Based on the above objectives, this preferred embodiment provides a method for planning a spraying flight path for automatic spraying of building facades using unmanned aerial vehicles (UAVs). Figure 1 As shown, this spraying route planning method involves drawing the spraying work area and planning the spraying work area route on the 3D building model, specifically including the following steps:

[0084] S1: Obtain the oblique photogrammetry 3D model of the sprayed building object and render the oblique photogrammetry 3D model to a 3D visualization platform.

[0085] The specific process of acquiring and rendering the 3D model is as follows:

[0086] S11: Taking oblique photography of the building object to be sprayed by an oblique photography unmanned aerial vehicle, and obtaining an oblique photography three-dimensional model of the sprayed building by using a third-party modeling platform;

[0087] S12: Based on the three.js open source library, a three-dimensional visualization platform supporting a 3D editing tool is established;

[0088] S13: The oblique photography three-dimensional model is rendered into the three-dimensional visualization platform by using the rendering engine of three.js.

[0089] S2: The building three-dimensional model in the three-dimensional visualization platform is converted into coordinate data for unmanned aerial vehicle positioning, which includes longitude, latitude and elevation.

[0090] The specific process of coordinate system data conversion is as follows:

[0091] S21: The building three-dimensional model has a matrix of 16 floating point numbers in the metadata, and the station-centered coordinate system of the matrix is the northeast celestial coordinate system ENU. The three-dimensional coordinates in the building three-dimensional model coordinate system are multiplied by the matrix to convert them into three-dimensional coordinates in the Earth rectangular coordinate system;

[0092] S22: The three-dimensional coordinates in the Earth rectangular coordinate system are converted into positioning coordinates that can be used by the unmanned aerial vehicle by using the conversion method (Cartographic.fromCartesian) in the cesium open source library, which includes longitude, latitude and elevation (WGS84 coordinate system).

[0093] S3: Based on the 3D model editing tool in the three-dimensional visualization platform, a two-dimensional / three-dimensional grid pattern to be sprayed is drawn on the building three-dimensional model, which is a planar / curved surface spraying work area of the building to be sprayed.

[0094] That is, the building surface spraying work area is divided into a curved surface spraying work area and a planar spraying work area, and the drawing steps for the two spraying work areas are different, as shown in Figure 2

[0095] For the curved surface spraying work area, that is, when the target building surface to be sprayed is a curved surface, the specific process is as follows:

[0096] S31: Using the line segment tool of the three-dimensional visualization platform, a closed three-dimensional grid pattern (curved surface spraying work area) is enclosed on the building surface of the building three-dimensional model by using line segments;

[0097] S32: Based on the three-dimensional network pattern in step S31, the triangles in the three-dimensional grid pattern located inside the region, outside the region and on the boundary of the region are calculated;

[0098] ​S33: For several triangles located at the region boundary, cut the triangles with the region boundary, the triangles on the region boundary are divided into two parts, inside and outside;

[0099] S34: Combine the inside part of the triangles on the region boundary and the triangles originally inside the region into a new 3D mesh, the new 3D mesh is the curved surface spraying work area.

[0100] For the planar spraying work area, that is, the building surface target to be sprayed is a plane, the specific steps are as follows:

[0101] S35: Use the point selection tool of the 3D visualization platform to select at least 6 vertices on the building surface of the building 3D model, and use a geometric plane to fit these 3D vertices by least squares method and singular value decomposition (SVD); the geometric plane is almost completely parallel to the to-be-sprayed outer wall of the building model, and is recorded as an auxiliary plane;

[0102] S36: Use the drawing tool of the 3D visualization platform to draw a 2D mesh on the auxiliary plane described in S35, and the 2D mesh is the planar spraying work area; the spraying work area drawn on the auxiliary plane will be parallelly attached to the building surface; if there is an unspraying area on the building surface, a 2D mesh of the unspraying area can be marked using an unspraying area marking tool, and such area will be avoided in the flight path generation process.

[0103] Further, the 2D mesh is drawn in the 3D visualization platform, and the following functions are supported:

[0104] On the one hand, the planar spraying work area is rectangular by default, and the addition and deletion of vertices, the dragging of the position of an existing vertex, and the modification of the shape of the 2D mesh are supported on the basis of the rectangular 2D mesh, so as to draw various 2D graphs such as triangles, trapezoids, stars, and hole graphs.

[0105] On the one hand, the 2D mesh drawn is subjected to 2D Boolean operation, and the union operation and difference operation between multiple 2D meshes are supported. Two 2D mesh rectangles (rectangular planar spraying areas) are drawn on the same auxiliary plane, when the two 2D mesh rectangles partially overlap, the two 2D mesh rectangles are set to be the same color using the color adjustment tool of the 3D visualization platform, that is, the spraying work area sprays the same color of paint, then the two 2D mesh rectangles will be merged into one rectangular planar spraying work area. When the colors of the two 2D mesh rectangles are different, the covered part of the 2D mesh rectangle drawn first will be cut out from the original area and become part of the 2D mesh rectangle drawn later.

[0106] S4: A pixel-level route planning algorithm is used for the two-dimensional grid pattern to be sprayed drawn in the three-dimensional model of the building; an improved A* algorithm (also referred to as AStar algorithm) for full-coverage path planning is used for the three-dimensional grid pattern to be sprayed drawn in the three-dimensional model of the building; and finally, a spraying route is planned for the unmanned aerial vehicle to perform automatic spraying operation on the building facade.

[0107] Further, the specific process of the path planning algorithm for the grid pattern (spraying work area) in S4 is as follows:

[0108] For the curved surface spraying work area, that is, when the target spraying work area of the building surface is a curved surface, as shown in FIGS. 1, 2, 3, 4, 5, 6, 7 and 8, the specific route planning steps are as follows: Figure 5 、 6 and 8, the specific route planning steps are as follows:

[0109] S41: Obtain the three-dimensional grid pattern of the building wall surface (curved surface spraying work area) and calculate the vertex normal vector. After loading the three-dimensional grid pattern data, pre-process the grid data, convert the grid data to point cloud data, filter the point cloud data to remove noise, and then solve the normal vector according to the grid vertex adjacency relationship.

[0110] The specific steps are as follows:

[0111] S411: Convert the spraying work area grid data drawn in the three-dimensional model of the building to off format, and use the Open3D library to create a triangular mesh object from the grid data;

[0112] S412: Sample the grid data to point cloud data, first calculate the number of resampled point clouds, and use the poisson_disk algorithm in the Open3D library to resample the point clouds;

[0113] S413: Smooth and denoise all resampled point cloud data, use the angle of the normal vector and the distance between the point clouds to cluster, filter out points that do not meet the distance requirement (position outliers) and the normal vector angle requirement (normal vector outliers), and perform region growing filtering on the point cloud to extract point cloud clusters that meet the normal vector angle and distance requirements.

[0114] S42: Position the unmanned aerial vehicle body. For the pre-processed point cloud in S41, translate outward along the direction of the normal vector by a set distance (distance from the building surface grid pattern), which is generally the distance from the spray head to the unmanned aerial vehicle body positioning module plus the optimal spraying distance of the spray head, to obtain the target position of the unmanned aerial vehicle body.

[0115] S43: Optimize the path to achieve full-coverage spraying. Use a full-coverage path planning algorithm to connect the discrete unmanned aerial vehicle poses to generate a complete segmented route.

[0116] The specific steps are as follows:

[0117] S431: Slice the point cloud after the outward movement based on S42 according to the Z-axis using the set spraying interval, and perform voxel downsampling on each slice to generate a single slice-shaped point cloud;

[0118] S432: Perform the following processing on each slice-shaped point cloud,

[0119] S4321: Project the point cloud onto the contour plane to form a two-dimensional point (this step only retains the XY coordinates of the point cloud, which is used for subsequent flight path fitting);

[0120] S4322: Fit the flight path on a series of two-dimensional points using the Douglas-Peucker algorithm (reduce unnecessary waypoints);

[0121] S4323: Convert the two-dimensional flight path back to a three-dimensional flight path (add Z dimension, the Z value of the flight path generated by each slice point cloud is the same as the Z center point of the slice-shaped point cloud);

[0122] S433: Connect the fitted flight path of each slice-shaped point cloud in the order of the three-dimensional flight path Z direction to make it a continuous flight path within the grid pattern, the heading of the flight path is the normal vector of the current flight path facing the wall surface, the fitted flight path of the slice-shaped point cloud is the spraying flight path, and the line segment connecting each slice-shaped point cloud fitted flight path is the non-spraying flight path;

[0123] S434: Traverse each segment of the flight path and add the opening and closing action points of the spray head in the flight path segment that needs to be sprayed to ensure that the spray head is opened and closed within the appropriate distance;

[0124] S4341: If the current flight path segment needs to be sprayed, calculate the spraying direction and distance of the current segment according to the path priority (horizontal axis priority or vertical axis priority);

[0125] S4342: If the spraying distance of the current segment is greater than or equal to the sum of the opening and closing distances of the spray head, add the opening and closing points of the spray head;

[0126] (1) Calculate the position distance of the spray gun opening point from the starting point and set it to the non-spraying state;

[0127] (2) Calculate the position distance of the spray gun closing point from the end point and set it to the spraying state;

[0128] (3) The spraying states of the starting point and the end point are both set to non-spraying, and they are returned to the flight path planning result.

[0129] S44: Perform flight path post-processing to calculate the flight point heading angle and relative heading angle (the angle that the UAV needs to turn after reaching the specified position, the same below).

[0130] For the planar spraying work area, that is, the building surface target to be sprayed is planar, as shown in FIGS. Figure 3 、 4 and 7, the specific steps of route planning are as follows:

[0131] S45: Obtain a two-dimensional grid graph of the building facade (planar spraying work area) and convert the planar spraying work area data into a two-dimensional picture.

[0132] The specific steps are as follows:

[0133] S451: Calculate the size of the two-dimensional picture and prepare a base map: according to the coordinates of the polygon vertices of the two-dimensional grid graph, calculate the minimum circumscribed rectangle;

[0134] S452: According to the physical width and height of the calculated minimum circumscribed rectangle, determine the picture width and height in combination with the pixel accuracy set in the configuration file, and initialize a base map full of 0 according to the picture width and height;

[0135] S453: Convert the physical coordinates of the polygon vertices of the two-dimensional grid graph to be sprayed into pixel coordinates;

[0136] S454: Use color steps to distinguish and draw the normal spraying work area and the area that cannot be sprayed in the spraying work area on the base map.

[0137] S46: Based on the two-dimensional picture described in S45, implement pixel-level route planning and generate a route path in the pixel coordinate system.

[0138] The specific steps are as follows:

[0139] S461: Expand the non-spraying area, determine the Kernel of the erosion operation according to the width and height of the spray shape, expand the non-spraying area according to the Kernel erosion picture, and protect the non-spraying area to avoid paint spraying in the spraying operation;

[0140] S462: Plan a route for the original spraying work area,

[0141] S4621: According to the set spraying width and height parameters, calculate the spraying width and height of the spray gun;

[0142] S4622: Set the spraying interval according to the path priority (vertical axis priority or horizontal axis priority);

[0143] S4623: Calculate the offset of the image size and the spraying interval, adjust the part that cannot be divided evenly, and ensure that the spraying work area is evenly covered by the image;

[0144] S4624: Generate a grid coordinate of the route waypoint,

[0145] (1) If vertical axis priority is used, generate a vertical axis priority route grid;

[0146] (2) If horizontal axis priority is used, generate a horizontal axis priority route grid;

[0147] S4625: Generate route index and store the actual coordinates of waypoints;

[0148] S463: If there are unpaintable areas within the painting work area, use the unpaintable areas in the 2D image as a mask to mask the initial flight path, regenerate a new painting flight path segment, and simultaneously generate the map used by the A* algorithm.

[0149] S4631: Create an A* map to mark passable areas;

[0150] S4632: Traverse each waypoint and determine whether the waypoint is located in a paintable area:

[0151] (1) Determine whether the waypoint is outside the edge of the image (i.e., whether it is a valid waypoint) based on the path priority (horizontal axis priority or vertical axis priority);

[0152] (2) If the waypoint is located within the paintable work area and is not on the edge, mark the point as a feasible area on the A* map;

[0153] S4633: If the path is horizontal axis priority, reverse the flight path to ensure that the starting point of the spraying flight path is at the height of the spraying work area plane.

[0154] S4634: Returns the filtered waypoints and the map used for the A* algorithm;

[0155] S464: Use the nearest distance rule to pair the lines to be painted and obtain the painting order of the lines;

[0156] S465: Based on the pairing results, the A* algorithm is used to connect the segments of the spraying path to form a complete pixel path.

[0157] S466: Traverse each segment of the flight path, adding nozzle on / off action points within the flight path segments requiring spraying, ensuring the nozzles are turned on and off at appropriate distances:

[0158] S4661: If the current route segment needs to be painted, calculate the painting direction and distance of the current segment according to the path priority (horizontal axis priority or vertical axis priority);

[0159] S4662: If the spraying distance of the current segment is greater than or equal to the sum of the nozzle opening and closing distances, add nozzle opening and closing points;

[0160] (1) Calculate the position distance of the spray gun opening point from the starting point, and set it to the non-spraying state;

[0161] (2) Calculate the position distance of the spray gun closing point from the end point, and set it to the spraying state;

[0162] (3) The spraying state of the starting point and the end point is set to non-spraying, and it is returned to the flight path planning result.

[0163] S47: Post-processing of the pixel flight path, converting the flight path in the pixel coordinate system based on S46 to the flight path in the three-dimensional coordinate system required by the unmanned aerial vehicle.

[0164] The spraying flight path planning method enables the spraying unmanned aerial vehicle to autonomously complete the spraying task without manual intervention and assistance, improves the automation level of the unmanned aerial vehicle spraying operation, and can ensure that the unmanned aerial vehicle reaches the specified position in spraying, realizes precise spraying, and avoids the need for manual control of the unmanned aerial vehicle for spraying operation on the building facade in traditional spraying unmanned aerial vehicles.

[0165] Due to the complex shape of the building facade, more accurate flight path planning is needed to ensure the spraying effect. Three-dimensional visualization technology is applied to flight path planning, and the flight path planning scheme described in the method is generated based on the three-dimensional model of the building scene, which can accurately control the spraying range of the building facade to realize high-precision positioning and spraying.

[0166] Based on the three-dimensional visualization platform, the spraying work area is drawn, which can adapt to various complex shapes and structures of building facades, and can also realize the spraying of complex patterns and textures, improving the aesthetic level of the building facade.

[0167] The improved A* algorithm under the full coverage path planning in the method finds a feasible path from the starting point to the end point that covers all key points in the given area, realizes full coverage of the spraying work area, and ensures that the spraying unmanned aerial vehicle can access and cover every area in the environment, avoiding the problem of missed spraying.

[0168] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for planning a spraying flight path for automatic spraying of a building facade by a drone, characterized in that, Specifically comprising the following steps: S1: Obtain the oblique photography three-dimensional model of the sprayed building object, and render the oblique photography three-dimensional model to a three-dimensional visualization platform; S2: Perform coordinate system data conversion on the building three-dimensional model in the three-dimensional visualization platform to obtain coordinate data used for positioning of the unmanned aerial vehicle, the coordinate data containing longitude, latitude and elevation; S3: Draw a two-dimensional / three-dimensional grid pattern to be sprayed on the building three-dimensional model using a 3D model editing tool of the three-dimensional visualization platform, the two-dimensional grid pattern to be sprayed being a planar spraying work area of the building to be sprayed, and the three-dimensional grid pattern to be sprayed being a curved surface spraying work area of the building to be sprayed; S4: Use a pixel-level flight path planning algorithm for the two-dimensional grid pattern to be sprayed, and use an improved A* algorithm for full coverage path planning for the three-dimensional grid pattern to be sprayed, and combine the coordinate data used for positioning of the unmanned aerial vehicle to plan a spraying flight path for the unmanned aerial vehicle to perform automatic spraying operation on the building facade; The specific steps of the unmanned aerial vehicle flight path planning for the three-dimensional grid pattern to be sprayed in step S4 are as follows: S41: Calculate the grid vertex normal vector according to the three-dimensional grid pattern to be sprayed: after loading the three-dimensional grid pattern to be sprayed, perform preprocessing, convert the grid data to point cloud data, filter the point cloud data to remove noise, and then solve the normal vector according to the grid vertex adjacency relationship; S42: Unmanned aerial vehicle body positioning: translate the point cloud after the preprocessing described in S41 outward along the direction of the normal vector by a set distance to obtain the target position of the unmanned aerial vehicle body; wherein the set distance is the distance from the spray head to the unmanned aerial vehicle body positioning module plus the optimal spraying distance of the spray head; S43: Optimize the path to achieve full coverage spraying: use the improved A* algorithm for full coverage path planning to connect the discrete unmanned aerial vehicle poses to generate a complete segmented flight path; S44: Perform flight path post-processing to calculate the heading angle and relative heading angle of the flight point.

2. The method of claim 1, wherein the method comprises: The specific steps of filtering and removing noise from the point cloud data in step S41 are as follows: S411: Convert the three-dimensional grid pattern to be sprayed into off format grid data, and create a triangular mesh object from the grid data using the Open3D library; S412: Sample the grid data into point cloud data, and use the poisson_disk algorithm in the Open3D library to resample the point cloud; S413: Smooth and denoise all resampled point cloud data: use the angle of the normal vector and the distance between the point clouds to cluster and filter out points that do not meet the distance and normal vector angle requirements, perform region growing filtering on the point cloud, and extract point cloud clusters that meet the normal vector angle and distance conditions.

3. The method of claim 2, wherein the method comprises: The specific steps of step S43 are as follows: S431: Slice the point cloud after the outward translation based on S42 according to the Z axis using a set spraying interval, and perform voxel downsampling on each slice to generate a single slice-shaped point cloud; S432: Perform the following operations on each slice-shaped point cloud: Project the point cloud onto an isoplanar surface to form a two-dimensional point in the XY coordinate system; Fit a two-dimensional flight path on a series of two-dimensional points using the Douglas-Peucker algorithm; Adding Z dimension to the two-dimensional route to return to the three-dimensional route; S433: connect the fitting route of each sheet point cloud in sequence according to the three-dimensional route to make it a continuous route in the grid pattern; wherein the heading of the route is the normal vector of the wall surface facing the current route; the fitting route of the sheet point cloud is the spraying route, and the line segment connecting each sheet point cloud fitting route is the non-spraying route; S434: traverse each segment in the route, and add the opening and closing action points of the spray head in the route segment that needs to be sprayed.

4. The method according to any one of claims 1-3, wherein, The specific steps of the UAV route path planning of the two-dimensional grid pattern to be sprayed in step S4 are as follows: S45: convert the two-dimensional grid pattern to be sprayed into a two-dimensional picture; S46: realize pixel-level route planning based on the two-dimensional picture in S45, and generate a route path in a pixel coordinate system; S47: post-process the pixel route, and convert the route in the pixel coordinate system based on S46 into a route in a three-dimensional coordinate system required by the UAV.

5. The method of claim 4, wherein the method comprises: The specific steps of converting the two-dimensional grid pattern to be sprayed into a two-dimensional picture in S45 are as follows: S451: calculate the size of the two-dimensional picture, and prepare a base map: calculate the minimum circumscribed rectangle according to the coordinates of the polygon vertices of the two-dimensional grid pattern to be sprayed; S452: determine the picture width and height according to the physical width and height of the calculated minimum circumscribed rectangle, and combine the pixel accuracy set in the configuration file to initialize a base map with all 0s according to the picture width and height; S453: convert the physical coordinates of the polygon vertices of the two-dimensional grid pattern to be sprayed into pixel coordinates; S454: distinguish by color scale, and draw the normal spraying work area and the non-spraying area in the spraying work area on the base map respectively.

6. The spraying route planning method for realizing automatic spraying of building facades by a UAV according to claim 5, characterized in that, The specific steps of realizing pixel-level route planning and generating a route path in a pixel coordinate system in S46 are as follows: S461: expand the non-spraying area: determine the Kernel of the erosion operation according to the width and height of the spraying shape of the spray head, and expand the non-spraying area according to the Kernel erosion picture; S462: route planning for the original spraying work area: calculate the spraying width and height of the spray gun according to the set spraying width and height parameters; set the spraying interval according to the path priority; adjust the part that cannot be divided by calculating the offset of the image size and the spraying interval; generate the grid coordinates of the route nodes; generate the route index, and store the actual coordinates of the nodes; S463: if there is a non-spraying area in the spraying work area, use the non-spraying area in the two-dimensional picture as a mask to mask the initial route, regenerate a new spraying route segment, and generate a map for A* algorithm: create an A* map for marking the passable area; traverse each node to determine whether the node is located in the sprayable area: if the path is horizontal axis priority, reverse the route path to ensure that the starting point of the spraying route is at the position of the high plane of the spraying work area; return the filtered obstacle-free nodes and the map for A* algorithm; S464: using the nearest distance rule, pairing the to-be-sprayed flight lines to obtain the flight line spraying order; S465: according to the pairing result, connecting each segmented to-be-sprayed flight line using A* algorithm to form a complete pixel flight line; S466: traversing each segment in the flight line, adding the opening and closing action points of the spray head in the to-be-sprayed flight line segment.

7. The building facade spraying flight line planning method according to claim 6, wherein, The specific steps of drawing the three-dimensional grid pattern to be sprayed on the three-dimensional model of the building using the 3D model editing tool of the three-dimensional visualization platform in step S3 are as follows: S31: using the line segment tool of the three-dimensional visualization platform, a closed three-dimensional grid pattern is drawn on the surface of the three-dimensional model using line segments; S32: based on the three-dimensional grid pattern in step S31, the triangles in the three-dimensional grid pattern located inside, outside and on the boundary of the region are calculated; S33: for the triangles located on the boundary of the region, the region boundary is used to cut these triangles, and the triangles on the region boundary are divided into two parts, inside and outside; S34: the part of the triangle on the region boundary inside is combined with the triangle originally inside the region to form a new three-dimensional grid pattern, which is the three-dimensional grid pattern to be sprayed; The specific steps of drawing the two-dimensional grid pattern to be sprayed on the three-dimensional model of the building using the 3D model editing tool of the three-dimensional visualization platform in step S3 are as follows: S35: using the point selection tool of the three-dimensional visualization platform, at least six vertices are selected on the surface of the three-dimensional model, and a geometric plane is used to fit these three-dimensional vertices by least squares method and singular value decomposition; wherein, the geometric plane is parallel to the to-be-sprayed outer wall of the building model, and is recorded as an auxiliary plane; S36: using the drawing tool of the three-dimensional visualization platform, the two-dimensional grid pattern to be sprayed is drawn on the auxiliary plane in S35; the two-dimensional grid pattern to be sprayed drawn on the auxiliary plane is parallel to the building surface; if there is an unspraying area on the building surface, the two-dimensional grid pattern to be sprayed is marked using the unspraying area marking tool.

8. The method of claim 6, wherein the method comprises: The coordinate system data conversion process of the three-dimensional model of the building in step S2 is as follows: S21: the metadata of the three-dimensional model of the building has a matrix composed of 16 floating points, and the station-centered coordinate system of the matrix is the northeast sky coordinate system ENU; the three-dimensional coordinates in the coordinate system of the three-dimensional model of the building are converted into three-dimensional coordinates in the earth rectangular coordinate system by multiplying the matrix; S22: using the conversion method in the cesium open source library, the three-dimensional coordinates in the earth rectangular coordinate system are converted into positioning coordinates that can be used by the unmanned aerial vehicle, which include longitude, latitude and elevation.

9. A method for planning a spraying flight path for automatic spraying of building facades by unmanned aerial vehicles (UAVs) according to claim 6, characterized in that, The specific process of step S1 is as follows: S11: tilt photography is performed by a tilt photography unmanned aerial vehicle, three-dimensional modeling is performed by using a third-party modeling platform, and a tilt photography three-dimensional model of the building to be sprayed is obtained; S12: based on the three.js open source library, a three-dimensional visualization platform supporting 3D editing tools is established; S13: render the oblique photography three-dimensional model into the three-dimensional visualization platform using a rendering engine of three.js.

Images