Three-layer path planning method for fine inspection of unmanned aerial vehicle

Abstract

The invention discloses a three-layer path planning method for fine inspection of an unmanned aerial vehicle, and relates to the technical field of path planning, and the method comprises the steps: determining the nearest detection distance of a three-layer safe region according to an inspection object; an unmanned aerial vehicle carries a photoelectric pod to lift off from a starting point, a main flight path is generated based on a first-layer safety inspection area through an A-star algorithm, first-layer fine inspection is executed, real-time shooting is conducted through the photoelectric pod, a millimeter-level three-dimensional point cloud model is constructed through LiDAR and oblique photography, and a window, a decoration structure and a defect area are recognized in combination with deep learning to serve as a first result; after defects are found, planning and executing a second-layer route by using an A-star algorithm, and comparing a second result with a first result; if yes, returning to the main route, otherwise, entering a third-layer safe area to re-plan the route and obtaining a final result; after completion, returning to the main route to continue the next-cycle inspection; according to the method provided by the invention, the reliability of an inspection result can be improved, and time consumption and potential safety hazards of short-distance flight are reduced.

Description

Technical Field

[0001] The present invention relates to the field of path planning technology, and in particular to a three-layer path planning method for refined inspection by unmanned aerial vehicles (UAVs). Background Art

[0002] Unmanned Aerial Vehicle (UAV) does not require an onboard pilot and has the characteristics of high flexibility and strong maneuverability. It has been widely used in civil and military fields, such as rescue, agriculture, and power transmission line inspection. Inspection tasks often require judging whether the inspection requirements are met. For example, in building quality inspections, if you want to judge whether the surface quality is qualified, you need a drone to take close-up photos and judge in real time whether the detection accuracy requirements have been met. If the requirements are met, you can fly to the next inspection point. If the detection accuracy requirements are not met, you need to get closer to the detection target to improve the shooting accuracy. Previous studies have not designed a planning method specifically for inspections with special needs, such as inspection tasks with identification such as cracks on the surface of buildings. In this regard, the present invention proposes a three-layer path planning method for refined UAV inspections, which generates subsequent UAV tracks based on real-time observation results, optimizes inspection flight plans, and improves inspection efficiency. Summary of the Invention

[0003] The purpose of the present invention is to provide a three-layer path planning method for refined inspection of unmanned aerial vehicles, which realizes the inspection requirements through a three-layer inspection method; specifically, it meets the inspection accuracy requirements; determines whether to execute the subsequent flight path according to the original plan based on the inspection situation; generates a subsequent path based on the inspection results; and ensures the safety of the unmanned aerial vehicle while meeting the inspection requirements.

[0004] To achieve the above objectives, the present invention provides a three-layer path planning method for UAV refined inspection, comprising the following steps:

[0005] Step 1: Determine the minimum detection distance of the three-layer path according to the inspection object, including the minimum detection distance of the first-layer safety inspection area, the minimum detection distance of the second-layer safety inspection area, and the minimum detection distance of the third-layer safety inspection area;

[0006] Step 2: The drone carrying the optoelectronic pod is placed at the starting point. After takeoff, the A-star algorithm generates a main flight mission path based on the first-level safety inspection area, based on the starting point and the inspection object (e.g., an entire building, where cracks may exist on the surface, which requires detailed inspection).

[0007] Step 3: Execute the first-level detailed inspection route, flying along the planned route. During flight, the drone's optoelectronic pod captures real-time images of the inspected objects. Using onboard LiDAR and oblique photography, the drone constructs a millimeter-level precision 3D point cloud model of the building surface. Deep learning (U-Net++ architecture) is used to perform semantic segmentation on the point cloud, identifying windows, decorative structures, and defective areas (such as cracks in the building).

[0008] Step 4: When a defective area is found, its key feature is identified; the key feature is the length of the crack in the building, which is used as the first inspection result;

[0009] Step 5: Use the A-star algorithm to plan the second-level fine inspection route according to the second-level security inspection area. After the route planning is completed, execute the flight route;

[0010] Step 6: After entering the second-level security inspection area, use the photoelectric pod to identify the key features of the defect area and use this as the second inspection result;

[0011] Step 7: Compare the second inspection result with the first inspection result. If the two inspection results are consistent, return to the main flight path; if the two inspection results are inconsistent, prepare to enter the third-level safety inspection area;

[0012] Step 8: Use the A-star algorithm to plan the third detailed inspection route, i.e., the closest safety distance inspection route, based on the third-level safety inspection area. After the route planning is completed, execute the flight route.

[0013] Step 9: Use the photoelectric pod to identify the key features of the defect area and use this as the final inspection result;

[0014] Step 10: Return to the main route for the next inspection cycle.

[0015] Preferably, the minimum detection distance of the first-level security inspection area is 3 times the distance to the inspection object, the minimum detection distance of the second-level security inspection area is 2 times the distance to the inspection object, and the minimum detection distance of the third-level security inspection area is 1 times the distance to the inspection object.

[0016] Preferably, the A-star algorithm includes two lists, one for storing the information of each node. After the node information is stored, the priority of the next search is determined by the cost function. The expression of the cost function is as follows:

[0017] f(n)=g(n)+h(n);

[0018] Where f(n) represents the cost function, g(n) represents the actual cost from the starting point to the current node n (such as the moving distance), and h(n) represents the heuristic estimated cost from the current node n to the end point (the present invention adopts the Manhattan algorithm).

[0019] Preferably, the two lists of the A-star algorithm are: the open list and the closed list; the open list (OpenList) stores the nodes to be explored and is sorted by f(n); the closed list (Closed List) stores the nodes that have been explored to avoid duplicate calculations.

[0020] Preferably, the specific execution process of the A-star algorithm is as follows:

[0021] S1. Initialization: Add the starting point to the open list, g(start)=0, f(start)=h(start), where g(start) represents the cost from the starting point itself to itself, which is 0, f(start) represents the total cost from the starting point to the end point, and h(start) represents the heuristic estimated cost from the starting point to the end point;

[0022] S2. Loop search: Take out the node n with the smallest f(n) from the open list and determine whether n is the end point; if n is the end point, backtrack the path and end; otherwise, move n to the closed list and generate all its neighbor nodes;

[0023] S3. Process neighbor nodes: For each neighbor m, if m is in the closed list or impassable, skip it; calculate the temporary g_temp = g(n) + cost(n→m); if m is not in the open list or g_temp < g(m); then update g(m) = g_temp, f(m) = g(m) + h(m), and record the parent node of m as n; if m is not in the open list, add it; where, g_temp represents the temporary estimated cost from the starting point to node m, cost(n→m) represents the cost of moving from node n to its neighbor m, g(m) represents the actual cumulative cost from the starting point to node m, f(m) represents the total cost from node m to the end point, and h(m) represents the heuristic estimated cost from node m to the end point;

[0024] S4. When the end point is found or the open list is empty (no solution), the algorithm terminates. [[ID=I9]]

[0025] Preferably, the starting point of the second-level fine inspection route is the position of the first inspection result; the starting point of the third-level fine inspection route is the position where the second inspection result is obtained.

[0026] Therefore, the present invention adopts the above-mentioned three-layer path planning method for refined inspection of drones, combined with artificial intelligence image recognition technology, and can use the consistency of the results of two tests to determine whether a reliable result is obtained. If it is not consistent, the highest accuracy detection within a safe range will be provided. If it is consistent with the accuracy, it can quickly enter the next inspection point; the method provided by the present invention reduces the flight time of the nearest inspection distance and avoids the dangers brought to drones by emergencies (such as sudden wind interference, falling objects, etc.).

[0027] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is an overall flow chart of a three-layer path planning method for UAV refined inspection according to the present invention;

[0029] Figure 2 This is a schematic diagram of the three-layer security inspection area according to an embodiment of the present invention;

[0030] Figure 3 Schematic diagram of the three-layer security inspection area corresponding to the closest inspection distance of 5m in an embodiment of the present invention;

[0031] Figure 4 This is a first-level fine inspection roadmap of an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of the first inspection result according to an embodiment of the present invention;

[0033] Figure 6 This is a second-layer fine inspection route map according to an embodiment of the present invention;

[0034] Figure 7 This is a third-layer fine inspection roadmap of an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.

[0036] See also Figure 1-Figure 7 ,A three-layer path planning method for UAV refined inspection includes the following steps:

[0037] Step 1: Determine the minimum detection distance of the three-layer path according to the situation of the inspection object, including the minimum detection distance of the first-layer safety inspection area, the minimum detection distance of the second-layer safety inspection area, and the minimum detection distance of the third-layer safety inspection area; among them, the minimum detection distance of the first-layer safety inspection area is 3 times the distance to the inspection object, the minimum detection distance of the second-layer safety inspection area is 2 times the distance to the inspection object, and the minimum detection distance of the third-layer safety inspection area is 1 times the distance to the inspection object; the three-layer safety inspection area of ​​a rectangular inspection object is as follows: Figure 2 As shown;

[0038] Step 2: The drone carrying the optoelectronic pod is placed at the starting position. After takeoff, the A-star algorithm is used to generate the main flight mission path based on the first-level safety inspection area, based on the starting position and the inspection object to be inspected (such as an entire building, where there may be cracks on the surface, which are points that require detailed inspection). The A-star algorithm includes two lists: an open list and a closed list. The open list (OpenList) stores the nodes to be explored and is sorted by f(n); the closed list (Closed List) stores the nodes that have been explored to avoid repeated calculations. The two lists are used to store the information of each node. After the node information is stored, the priority of the next search is determined by the cost function. The expression of the cost function is as follows:

[0039] f(n)=g(n)+h(n);

[0040] Where f(n) represents the cost function, g(n) represents the actual cost from the starting point to the current node n (such as the moving distance), and h(n) represents the heuristic estimated cost from the current node n to the end point (the present invention adopts the Manhattan algorithm).

[0041] Based on the starting point and the three-layer security inspection zone, a series of path points are generated. The A-Star algorithm is used to plan the path between two adjacent path points. When the inspection path needs to change, new path points are generated and the A-Star algorithm is used again to plan the path between two adjacent path points.

[0042] The specific execution process of the A-star algorithm is as follows:

[0043] S1. Initialization: Add the starting point to the open list, g(start) = 0, f(start) = h(start), where g(start) represents the cost from the starting point to itself, which is 0, f(start) represents the total cost from the starting point to the end point, and h(start) represents the heuristic estimated cost from the starting point to the end point;

[0044] S2. Cyclic Search: Take out the node n with the smallest f(n) from the open list, and determine whether n is the end point; if n is the end point, backtrack the path and end; otherwise, move n to the closed list and generate all its neighbor nodes;

[0045] S3. Process Neighbor Nodes: For each neighbor m, if m is in the closed list or impassable, skip it; calculate the temporary g_temp = g(n) + cost(n→m); if m is not in the open list or g_temp < g(m); then update g(m) = g_temp, f(m) = g(m) + h(m), and record the parent node of m as n; if m is not in the open list, add it; where, g_temp represents the temporary estimated cost from the starting point to node m, cost(n→m) represents the cost of moving from node n to its neighbor m, g(m) represents the actual cumulative cost from the starting point to node m, f(m) represents the total cost from node m to the end point, and h(m) represents the heuristic estimated cost from node m to the end point;

[0046] S4. When the end point is found or the open list is empty (no solution), the algorithm terminates.

[0047] Step 3. Execute the first-layer fine inspection route and fly along the planned flight path; during the flight, use the optoelectronic pod carried by the UAV to take real-time pictures of the inspected object, construct a three-dimensional point cloud model of the building surface with millimeter-level accuracy through airborne LiDAR and oblique photography, and perform semantic segmentation on the point cloud based on deep learning (U-Net++ architecture) to identify windows, decorative structures, and defect areas (such as building cracks), etc.;

[0048] Step 4. When a defect area is found, identify its key features; the key feature is the length of the building crack, which is used as the result of the first inspection;

[0049] Step 5. Use the A-star algorithm to plan the second-layer fine inspection route according to the second-layer safety inspection area, and the starting point of the second-layer fine inspection route is the position of the result of the first inspection; after the route planning is completed, execute this flight route;

[0050] Step 6. After entering the second-layer safety inspection area, use the optoelectronic pod to identify the key features of the defect area, which is used as the result of the second inspection;

[0051] Step 7. Compare the results of the second inspection and the first inspection. If the results of the two inspections are the same, return to the main flight route; if the results of the two inspections are different, prepare to enter the third-layer safety inspection area;

[0052] Step 8: Use the A-star algorithm to plan a third detailed inspection route, i.e., the closest safety distance inspection route, based on the third-layer safety inspection area. The starting point of the third detailed inspection route is the location where the second inspection result was obtained. After the route planning is completed, execute the flight route.

[0053] Step 9: Use the photoelectric pod to identify the key features of the defect area and use this as the final inspection result;

[0054] Step 10: Return to the main route for the next inspection cycle.

[0055] Example

[0056] In the initial state, the drone is located at the starting point. There is a rectangular inspection object in the mission area, and the closest inspection distance is 5m. Based on this situation, a three-layer safety inspection area is obtained, such as Figure 3 shown.

[0057] The drone carries the optoelectronic pod and is placed at the starting position. After taking off, the A-star algorithm is used to generate the main flight mission path, i.e., the first-level fine inspection route, based on the starting position and the objects to be inspected (e.g., a whole building, where cracks may exist on the surface, which are points that require detailed inspection). Figure 4 shown.

[0058] Execute the main flight route and fly along the planned route; during the flight, the optoelectronic pod carried by the drone takes real-time photos of the inspection objects and identifies windows, decorative structures, defective areas (such as building cracks), etc. When a defective area is found, its key features such as the length of the building crack are identified, which is the first inspection result. Figure 5 shown.

[0059] At this time, the A-star algorithm is used to plan the second-level fine inspection route based on the second-level safety inspection area. The starting point of the second-level fine inspection route is the location where the first inspection result was obtained. After the route planning is completed, the flight route is executed. After entering the second-level safety inspection area, the photoelectric pod is used to identify the key features of the defect area, which is the second inspection result. The second inspection result is compared with the first inspection result. If the two inspection results are consistent, the main flight route is returned, such as Figure 6 shown.

[0060] If the two inspection results are inconsistent, then prepare to enter the third-level security inspection area. At this time, the A-star algorithm is used to plan the third-level fine inspection route, that is, the closest safety distance inspection route, based on the third-level security inspection area. The starting point of the third-level fine inspection route is the location where the second inspection result is obtained, such as Figure 7 As shown in the figure, after the route planning is completed, the flight path is executed. The key features of the defective area are identified by the optoelectronic pod, which is the final inspection result.

[0061] Return to the main route to continue the mission, and finally return to the starting point to land and complete all inspections.

[0062] Therefore, the present invention adopts the above-mentioned three-layer path planning method for refined inspection of drones, adds image intelligent recognition during the inspection process, and judges whether the target situation is fully detected by whether the results of the first two inspections are consistent, and then decides whether to conduct a third inspection; through this method, the reliability of the inspection results can be increased, the accuracy of the inspection can be improved, and the time consumption and safety hazards of close-range flights can be reduced.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A three-layer path planning method for drone-based inspection, characterized in that: The following steps are involved: Step 1: Determine the minimum detection distance of the three-layer path based on the inspection object; Step 2: The UAV carrying the optoelectronic pod is placed at the starting position; Step 3: Execute the first-level detailed inspection route and fly along the planned route; Step 4: When a defective area is found, identify its key features; The key feature is the length of the cracks in the building, which is used as the first inspection result; Step 5: Use the A-star algorithm to plan the second-level fine inspection route according to the second-level security inspection area. After the route planning is completed, execute the flight route; Step 6: After entering the second-level security inspection area, use the photoelectric pod to identify the key features of the defect area and use this as the second inspection result; Step 7: Compare the second inspection result with the first inspection result. If the two inspection results are consistent, return to the main flight route. If the two inspection results are inconsistent, prepare to enter the third-level security inspection area; Step 8: Use the A-star algorithm to plan the third detailed inspection route according to the third-level security inspection area. After the route planning is completed, execute the flight route; Step 9: Use the photoelectric pod to identify the key features of the defect area and use this as the final inspection result; Step 10: Return to the main route for the next inspection cycle.

2. A three-layer path planning method for drone refined inspection according to claim 1, characterized in that: Step 1: Determine the minimum detection distance of the three-layer path based on the inspection object; It includes the minimum detection distance of the first-level security inspection area, the minimum detection distance of the second-level security inspection area, and the minimum detection distance of the third-level security inspection area.

3. The three-layer path planning method for UAV refined inspection according to claim 2 is characterized by: Step 2: The drone carrying the optoelectronic pod is placed at the starting position. After taking off, the A-star algorithm is used to generate the main flight mission path based on the first-level safety inspection area according to the starting position and the inspection objects to be inspected.

4. The three-layer path planning method for UAV refined inspection according to claim 3 is characterized by: Step 3: Execute the first-level detailed inspection route and fly along the planned route. During the flight, the optoelectronic pod carried by the drone takes real-time photos of the inspected objects. Airborne LiDAR and oblique photography are used to construct a millimeter-level accurate 3D point cloud model of the building surface. Based on deep learning, the point cloud is semantically segmented to identify windows, decorative structures, and defective areas.

5. The three-layer path planning method for UAV refined inspection according to claim 4 is characterized by: The minimum detection distance of the first-level security inspection area is 3 times the distance to the inspection object, the minimum detection distance of the second-level security inspection area is 2 times the distance to the inspection object, and the minimum detection distance of the third-level security inspection area is 1 times the distance to the inspection object.

6. The three-layer path planning method for UAV refined inspection according to claim 5 is characterized by: The A-star algorithm includes two lists, one for storing the information of each node. After the node information is stored, the priority of the next search is determined by the cost function. The expression of the cost function is as follows: f(n)=g(n)+h(n); Where f(n) represents the cost function, g(n) represents the actual cost from the starting point to the current node n, and h(n) represents the heuristic estimated cost from the current node n to the end point.

7. The three-layer path planning method for UAV refined inspection according to claim 6 is characterized by: The two lists of the A-star algorithm are: open list and closed list; the open list stores the nodes to be explored, sorted by f(n); the closed list stores the nodes that have been explored.

8. A three-layer path planning method for drone refined inspection according to claim 7, characterized in that: The specific execution process of the A-star algorithm is as follows: S1. Initialization: Add the starting point to the open list, g(start) = 0, f(start) = h(start), where g(start) represents the cost from the starting point itself to itself, which is 0, f(start) represents the total cost from the starting point to the end point, and h(start) represents the heuristic estimated cost from the starting point to the end point; S2. Loop search: Take out the node n with the smallest f(n) from the open list, and determine whether n is the end point; if n is the end point, backtrack the path and end; otherwise, move n to the closed list and generate all its neighbor nodes; S3. Process neighbor nodes: For each neighbor m, if m is in the closed list or impassable, skip it; calculate the temporary g_temp = g(n) + cost(n → m); if m is not in the open list or g_temp < g(m); then update g(m) = g_temp, f(m) = g(m) + h(m), and record the parent node of m as n; if m is not in the open list, add it; where, g_temp represents the temporary estimated cost from the starting point to node m, cost(n → m) represents the cost of moving from node n to its neighbor m, g(m) represents the actual cumulative cost from the starting point to node m, f(m) represents the total cost from node m to the end point, and h(m) represents the heuristic estimated cost from node m to the end point; S4. When the end point is found or the open list is empty, the algorithm terminates.

9. The three-layer path planning method for UAV refined inspection according to claim 8 is characterized by: The starting point of the second-level fine inspection route is the position of the first inspection result; The starting point of the third-level fine inspection route is the position where the second inspection result is obtained.

Images