A three-layer path planning method for fine inspection of unmanned aerial vehicles

Abstract

The application discloses a three-layer path planning method for fine inspection of an unmanned aerial vehicle, and relates to the technical field of path planning.The nearest detection distance of a three-layer safety zone is determined according to an inspection object; the unmanned aerial vehicle carries an optoelectronic pod and ascends from a starting point, a main flight path is generated based on the first-layer safety inspection zone by using an A-star algorithm, first-layer fine inspection is performed, a millimeter-level three-dimensional point cloud model is constructed by using the optoelectronic pod to take real-time photos and by using LiDAR and oblique photography, and windows, decorative structures and defect areas are recognized by using deep learning to serve as a first result; after defects are found, a second-layer route is planned by using the A-star algorithm, and a second result is compared with the first result; if the first result is consistent with the second result, the main route is returned, otherwise, a third-layer safety zone is entered to plan a route again and obtain a final result; after completion, the main route is returned to continue the next cycle of inspection; the method provided by the application can increase the reliability of the inspection result, and can reduce the time consumption and safety hazards of close-range flight.

Description

Technical Field

[0001] This invention relates to the field of path planning technology, and in particular to a three-layer path planning method for refined inspection of unmanned aerial vehicles (UAVs). Background Technology

[0002] Unmanned Aerial Vehicles (UAVs) do not require an onboard pilot and are characterized by high flexibility and maneuverability. They are currently widely used in civilian and military fields, such as rescue, agriculture, and power line inspection. Inspection tasks often require determining whether inspection requirements have been met. For example, in building quality inspection, to determine whether the surface quality is up to standard, the UAV needs to take close-up photos and make real-time judgments on whether the required inspection accuracy has been met. If the requirements are met, the UAV can fly to the next inspection point; if the requirements are not met, it needs to move closer to the target to improve the shooting accuracy. Previous research has not specifically designed planning methods for inspection tasks with special needs, such as identifying cracks on building surfaces. To address this, this invention proposes a three-layer path planning method for refined UAV inspection. Based on real-time observation results, it generates subsequent UAV trajectories, optimizes the inspection flight plan, and improves inspection efficiency. Summary of the Invention

[0003] The purpose of this invention is to provide a three-layer path planning method for refined inspection of unmanned aerial vehicles (UAVs). This method achieves inspection requirements through a three-layer inspection approach: meeting inspection accuracy requirements; determining whether to execute the subsequent flight path according to the original plan based on the inspection results; generating the subsequent path based on the inspection results; and ensuring the safety of the UAV while meeting the inspection requirements.

[0004] To achieve the above objectives, this invention provides a three-layer path planning method for refined inspection of unmanned aerial vehicles (UAVs), comprising the following steps:

[0005] Step 1: Determine the shortest detection distance for the three-layer path based on the condition of the inspected object, including the shortest detection distance for the first layer of security inspection area, the shortest detection distance for the second layer of security inspection area, and the shortest detection distance for the third layer of security inspection area;

[0006] Step 2: The UAV carrying the electro-optical pod is placed at the starting position; after takeoff, the main flight mission path is generated based on the starting position and the objects to be inspected (such as an entire building, where there may be cracks on the surface of the building, which are points that need to be inspected in detail) according to the first layer of security inspection area using the A-star algorithm.

[0007] Step 3: Execute the first layer of fine inspection route and fly along the planned route. During the flight, the UAV uses an electro-optical pod to take real-time pictures of the inspected objects. The UAV uses airborne LiDAR and oblique photography to build a 3D point cloud model of the building surface with millimeter-level precision. The point cloud is then semantically segmented based on deep learning (U-Net++ architecture) to identify windows, decorative structures and defect areas (such as building cracks).

[0008] Step 4: When a defective area is found, identify its key features; the key feature is the length of the crack in the building, which will be used as the result of the first inspection.

[0009] Step 5: Use the A-star algorithm to plan the second-layer fine inspection route based on the second-layer security inspection area. After the route planning is completed, execute the flight route.

[0010] Step 6: After entering the second-level security inspection area, use the photoelectric pod to identify the key features of the defect area, and use this as the result of the second inspection.

[0011] Step 7: Compare the results of the second inspection with the results of the first inspection. If the two inspection results are consistent, return to the main flight route; if the two inspection results are inconsistent, prepare to enter the third-level safety inspection area.

[0012] Step 8: Use the A-star algorithm to plan the third fine inspection route, i.e. the shortest safe distance inspection route, based on the third layer of safety inspection zone. After the route planning is completed, execute the flight route.

[0013] Step 9: Use the photoelectric pod to identify the key features of the defective area, and use this as the final inspection result;

[0014] Step 10: Return to the main route for the next cycle of inspection.

[0015] Preferably, the nearest detection distance of the first layer of security inspection area is 3 times the distance to the inspected object, the nearest detection distance of the second layer of security inspection area is 2 times the distance to the inspected object, and the nearest detection distance of the third layer of security inspection area is 1 time the distance to the inspected object.

[0016] Preferably, the A-star algorithm includes two lists, one for storing information about each node. After the node information is stored, the priority of the next search step is determined by a cost function, the expression of which is as follows:

[0017] f(n) = g(n) + h(n);

[0018] In the formula, f(n) represents the cost function, g(n) represents the actual cost (such as the distance traveled) from the starting point to the current node n, and h(n) represents the heuristically estimated cost from the current node n to the destination (this invention uses the Manhattan algorithm).

[0019] Preferably, the two lists of the A-star algorithm are: the open list and the closed list; the open list (OpenList) stores the nodes to be explored and is sorted by f(n); the closed list (Closed List) stores the nodes that have been explored to avoid repeated calculations.

[0020] Preferably, the specific execution process of the A-star algorithm is as follows:

[0021] S1. Initialization: Add the starting point to the open list, g(start)=0, f(start)=h(start), where g(start) represents the cost from the starting point itself to itself, which is 0, f(start) represents the total cost from the starting point to the end point, and h(start) represents the heuristic estimated cost from the starting point to the end point;

[0022] S2. Loop search: Take out the node n with the smallest f(n) from the open list and determine whether n is the end point; if n is the end point, backtrack the path and end; otherwise, move n to the closed list and generate all its neighbor nodes;

[0023] S3. Process neighbor nodes: For each neighbor m, if m is in the closed list or impassable, skip it; calculate the temporary g_temp = g(n) + cost(n→m); if m is not in the open list or g_temp < g(m); then update g(m) = g_temp, f(m) = g(m) + h(m), and record the parent node of m as n; if m is not in the open list, add it; where, g_temp represents the temporary estimated cost from the starting point to node m, cost(n→m) represents the cost of moving from node n to its neighbor m, g(m) represents the actual cumulative cost from the starting point to node m, f(m) represents the total cost from node m to the end point, and h(m) represents the heuristic estimated cost from node m to the end point;

[0024] S4. When the end point is found or the open list is empty (no solution), the algorithm terminates.

[0025] Preferably, the starting point of the second-level fine inspection route is the position of the first inspection result; the starting point of the third-level fine inspection route is the position where the second inspection result is obtained.

[0026] Therefore, this invention employs the aforementioned three-layer path planning method for refined inspection of unmanned aerial vehicles (UAVs), combined with artificial intelligence image recognition technology. It can determine whether a reliable result is obtained by using the consistency of the results of two inspections. If they do not match, it will provide the highest accuracy inspection within a safe range. If they match, it can quickly proceed to the next inspection point. The method provided by this invention reduces the flight time for the nearest inspection distance and avoids the dangers to UAVs caused by sudden situations (such as wind interference, falling objects, etc.).

[0027] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0028] Figure 1 This is an overall flowchart of a three-layer path planning method for refined inspection of unmanned aerial vehicles (UAVs) according to the present invention.

[0029] Figure 2 This is a schematic diagram of the three-layer security inspection area according to an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of the three-layer safety inspection area corresponding to the nearest inspection distance of 5m in an embodiment of the present invention;

[0031] Figure 4 This is the first-layer detailed inspection route diagram of an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of the first inspection result according to an embodiment of the present invention;

[0033] Figure 6 This is the second-layer detailed inspection route diagram according to an embodiment of the present invention;

[0034] Figure 7 This is the third-layer fine inspection route diagram of an embodiment of the present invention. Detailed Implementation

[0035] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0036] Please see Figures 1-7 A three-layer path planning method for refined inspection of unmanned aerial vehicles (UAVs) includes the following steps:

[0037] Step 1: Determine the nearest detection distance for the three-layer path based on the inspected object, including the nearest detection distance for the first-layer security inspection area, the second-layer security inspection area, and the third-layer security inspection area. The nearest detection distance for the first-layer security inspection area is 3 times the distance to the inspected object, the nearest detection distance for the second-layer security inspection area is 2 times the distance to the inspected object, and the nearest detection distance for the third-layer security inspection area is 1 time the distance to the inspected object. The three-layer security inspection areas for a rectangular inspected object are as follows: Figure 2 As shown;

[0038] Step 2: The UAV carrying the electro-optical pod is placed at the starting position. After takeoff, based on the starting position and the objects to be inspected (e.g., an entire building, where cracks on the surface may require detailed inspection), the main flight path is generated using the A-star algorithm based on the first layer of security inspection zones. The A-star algorithm includes two lists: an open list and a closed list. The open list stores nodes to be explored, sorted by f(n); the closed list stores already explored nodes to avoid redundant calculations. Both lists store information for each node. After the node information is stored, the priority of the next search step is determined by a cost function, the expression of which is as follows:

[0039] f(n) = g(n) + h(n);

[0040] In the formula, f(n) represents the cost function, g(n) represents the actual cost (such as the distance traveled) from the starting point to the current node n, and h(n) represents the heuristically estimated cost from the current node n to the destination (this invention uses the Manhattan algorithm).

[0041] Based on the starting point and the three-layer security inspection area, a series of path points are generated. Between adjacent path points, the A-Star algorithm is used for path planning. When the inspection path needs to be changed, new path points are generated, and the A-Star algorithm is used again to plan the path between adjacent path points.

[0042] The specific execution process of the A-star algorithm is as follows:

[0043] S1. Initialization: Add the starting point to the open list, g(start) = 0, f(start) = h(start), where g(start) represents the cost from the starting point itself to itself, which is 0, f(start) represents the total cost from the starting point to the end point, and h(start) represents the heuristic estimated cost from the starting point to the end point.

[0044] S2. Circular Search: Take out the node n with the smallest f(n) from the open list, and determine whether n is the end point. If n is the end point, backtrack the path and end. Otherwise, move n to the closed list and generate all its neighbor nodes.

[0045] S3. Process Neighbor Nodes: For each neighbor m, if m is in the closed list or impassable, skip it. Calculate the temporary g_temp = g(n) + cost(n→m). If m is not in the open list or g_temp < g(m), then update g(m) = g_temp, f(m) = g(m) + h(m), and record the parent node of m as n. If m is not in the open list, add it. Here, g_temp represents the temporary estimated cost from the start point to node m, cost(n→m) represents the cost of moving from node n to its neighbor m, g(m) represents the actual cumulative cost from the start point to node m, f(m) represents the total cost from node m to the end point, and h(m) represents the heuristic estimated cost from node m to the end point.

[0046] S4. When the end point is found or the open list is empty (no solution), the algorithm terminates.

[0047] Step 3. Execute the first-level fine inspection route and fly along the planned flight path. During the flight, use the optoelectronic pod carried by the UAV to capture the situation of the inspection object in real time. Construct a three-dimensional point cloud model of the building surface with millimeter-level accuracy through airborne LiDAR and oblique photography. Based on deep learning (U-Net++ architecture), perform semantic segmentation on the point cloud to identify windows, decorative structures, and defect areas (such as cracks in the building), etc.

[0048] Step 4. When a defect area is found, identify its key features. The key feature is the length of the crack in the building, which is used as the result of the first inspection.

[0049] Step 5. Use the A-star algorithm to plan the second-level fine inspection route according to the second-level safety inspection area. The starting point of the second-level fine inspection route is the position of the result of the first inspection. After the route planning is completed, execute this flight route.

[0050] Step 6. After entering the second-level safety inspection area, use the optoelectronic pod to identify the key features of the defect area, which is used as the result of the second inspection.

[0051] Step 7. Compare the results of the second inspection and the first inspection. If the results of the two inspections are the same, return to the main flight route. If the results of the two inspections are different, prepare to enter the third-level safety inspection area.

[0052] Step 8: Using the A-star algorithm, plan the third fine inspection route, i.e., the shortest safe distance inspection route, based on the third layer of safety inspection zone. The starting point of the third fine inspection route is the location where the second inspection result is obtained. After the route planning is completed, execute the flight route.

[0053] Step 9: Use the photoelectric pod to identify the key features of the defective area, and use this as the final inspection result;

[0054] Step 10: Return to the main route for the next cycle of inspection.

[0055] Example

[0056] Initially, the drone is located at the starting point, and the mission area contains a rectangular inspection object with a minimum inspection distance of 5 meters. Based on this, a three-layer safety inspection zone is derived, as follows: Figure 3 As shown.

[0057] The drone, carrying an electro-optical pod, is placed at the starting point. After takeoff, based on the starting point and the objects to be inspected (e.g., an entire building, where cracks on the surface may require detailed inspection), the A-star algorithm generates the main flight path, i.e., the first-layer detailed inspection route, based on the first-layer safety inspection area. Figure 4 As shown.

[0058] The drone executes the main flight route, flying along a pre-planned path. During flight, it uses an electro-optical pod carried by the drone to capture real-time images of the inspected objects, identifying windows, decorative structures, and defective areas (such as building cracks). When a defective area is detected, its key features, such as the length of the building crack, are identified; this constitutes the initial inspection result. Figure 5 As shown.

[0059] At this point, the A-Star algorithm is used to plan a second-level fine-grained inspection route based on the second-level safety inspection zone. The starting point of the second-level fine-grained inspection route is the location where the first inspection result was obtained. After the route planning is completed, the flight route is executed. After entering the second-level safety inspection zone, the key features of the defect area are identified using an electro-optical pod; this is the second inspection result. The second inspection result is compared with the first inspection result. If the two inspection results are consistent, the main flight route is returned. Figure 6 As shown.

[0060] If the results of the two inspections are inconsistent, then proceed to the third-level security inspection zone. At this point, the A-Star algorithm is used to plan the third-level fine-grained inspection route, i.e., the shortest safe distance inspection route, based on the third-level security inspection zone. The starting point of the third-level fine-grained inspection route is the location where the second inspection result was obtained. Figure 7 As shown; after the route planning is completed, the flight route is executed. The key features of the defective areas are identified using an optoelectronic pod; this is the final inspection result.

[0061] Return to the main route to continue the mission, and finally return to the starting point to land, completing all inspections.

[0062] Therefore, the present invention adopts the above-mentioned three-layer path planning method for refined inspection of UAVs, and incorporates intelligent image recognition during the inspection process. By checking whether the results of the first two inspections are consistent, it is determined whether the target situation has been sufficiently detected, and then it is decided whether to conduct a third inspection. This method can increase the reliability of the inspection results, improve the accuracy of the inspection, and reduce the time consumption and safety hazards of close-range flight.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A three-layer path planning method for refined inspection of unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Step 1: Determine the shortest detection distance for the three-layer path based on the condition of the inspected object; Step 2: The drone carrying the electro-optical pod is placed at the starting point; Step 3: Execute the first layer of detailed inspection route, flying along the planned flight path; Step 4: When a defective area is discovered, identify its key features; The key feature is the length of the building's cracks, which is used as the result of the first inspection. Step 5: Use the A-star algorithm to plan the second-layer fine inspection route based on the second-layer security inspection area. After the route planning is completed, execute the flight route. Step 6: After entering the second-level security inspection area, use the photoelectric pod to identify the key features of the defect area, and use this as the result of the second inspection. Step 7: Compare the results of the second inspection with the results of the first inspection. If the results of the two inspections are consistent, return to the main flight path. If the results of the two inspections are inconsistent, prepare to enter the third-level security inspection area; Step 8: Use the A-star algorithm to plan the third fine inspection route based on the third layer of security inspection area. After the route planning is completed, execute the flight route. Step 9: Use the photoelectric pod to identify the key features of the defective area, and use this as the final inspection result; Step 10: Return to the main route for the next cycle of inspection; The starting point of the second-layer fine inspection route is the location where the results of the first inspection are obtained; The starting point of the third detailed inspection route is the location where the results of the second inspection are obtained.

2. The three-layer path planning method for refined inspection of unmanned aerial vehicles according to claim 1, characterized in that: Step 1: Determine the shortest detection distance for the three-layer path based on the condition of the inspected object; This includes the nearest detection distance for the first layer of security inspection area, the nearest detection distance for the second layer of security inspection area, and the nearest detection distance for the third layer of security inspection area.

3. The three-layer path planning method for refined inspection of unmanned aerial vehicles according to claim 2, characterized in that: Step 2: The UAV carrying the electro-optical pod is placed at the starting position; after takeoff, the main flight mission path is generated based on the starting position and the objects to be inspected, according to the first layer of safety inspection area, using the A-star algorithm.

4. The three-layer path planning method for refined inspection of unmanned aerial vehicles according to claim 3, characterized in that: Step 3: Execute the first layer of fine inspection route and fly along the planned route. During the flight, the UAV uses an electro-optical pod to take real-time pictures of the inspected objects. The UAV uses airborne LiDAR and oblique photography to construct a 3D point cloud model of the building surface with millimeter-level precision. Based on deep learning, the point cloud is semantically segmented to identify windows, decorative structures and defect areas.

5. A three-layer path planning method for refined inspection of unmanned aerial vehicles according to claim 4, characterized in that: The closest detection distance in the first-level security inspection zone is 3 times the distance to the inspected object; the closest detection distance in the second-level security inspection zone is 2 times the distance to the inspected object; and the closest detection distance in the third-level security inspection zone is 1 time the distance to the inspected object.

6. A three-layer path planning method for refined inspection of unmanned aerial vehicles according to claim 5, characterized in that: The A-Star algorithm uses two lists to store information for each node. After the node information is stored, the priority of the next search step is determined by a cost function, the expression of which is as follows: ； In the formula, Represents the cost function, This represents the actual cost from the starting point to the current node n. This represents the heuristically estimated cost from the current node n to the destination.

7. A three-layer path planning method for refined inspection of unmanned aerial vehicles according to claim 6, characterized in that: The A-star algorithm uses two lists: an open list and a closed list. The open list stores nodes to be explored and is sorted by f(n). The closed list stores nodes that have already been explored.

8. A three-layer path planning method for refined inspection of unmanned aerial vehicles according to claim 7, characterized in that, The specific execution process of the A-star algorithm is as follows: S1. Initialization: Add the starting point to the open list. , ,in, This indicates that the cost from the starting point to itself is 0. This represents the total cost from the starting point to the destination. This represents a heuristic estimate of the cost from the starting point to the end point; S2, Loop Search: Retrieved from the open list Find the smallest node n, and determine if n is the endpoint; if n is the endpoint, backtrack the path and end; otherwise, move n to the closed list and generate all its neighbor nodes. S3. Process neighbor nodes: For each neighbor m, skip if m is in the closed list or impassable; calculate temporary... If m is not in the open list or Then update Let m be the parent node of n; if m is not in the open list, then add it; where, This represents the provisional estimated cost from the starting point to node m. This represents the cost of moving from node n to its neighbor m. This represents the actual cumulative cost from the starting point to node m. This represents the total cost from node m to the destination. This represents the heuristic cost estimate from node m to the destination. S4. The algorithm terminates when the endpoint is found or the open list is empty.

Images