Unmanned aerial vehicle inspection method and device

Abstract

The invention discloses an unmanned aerial vehicle inspection method and device, and the method comprises the steps: obtaining environment data, collected by a sensor, of an unmanned aerial vehicle in multiple motion directions, carrying out the three-dimensional reconstruction according to the environment data, and obtaining the three-dimensional environment information of the surrounding environment of the unmanned aerial vehicle; based on the initial position information of the unmanned aerial vehicle, screening a drivable region which contains the initial position information and the termination position information of the unmanned aerial vehicle and does not intersect with any obstacle from the drivable region information; setting a plurality of inspection points in the driving area, and obtaining the position information of each inspection point; and based on the position information of each inspection point, acquiring unmanned aerial vehicle position information and yaw angle information when the unmanned aerial vehicle passes through the corresponding inspection point, and determining a cruise trajectory of the unmanned aerial vehicle according to the unmanned aerial vehicle position information and the yaw angle information to complete an inspection task. The accuracy of the cruise route can be improved, the inspection effect is improved, and it is guaranteed that the unmanned aerial vehicle can efficiently complete inspection work in a complex environment.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to an unmanned aerial vehicle inspection method and device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by inclusion in this section.

[0003] With the rapid development of drone technology, its application in many fields has become more and more extensive, especially in the field of inspection. UAV inspection systems usually include drone platforms, flight control systems, sensor payloads, data transmission systems, and ground control stations. The integration of these systems enables drones to autonomously perform inspection tasks without direct human intervention. However, traditional drone inspections rely on satellite signals for high-precision positioning, but in environments where satellite signals are shielded, such as substations and converter stations with strong electric fields and closed tunnels, this poses a severe challenge to the cruising capabilities of drones. In complex environments, the cruising routes of traditional inspection methods are inaccurate, resulting in unsatisfactory inspection results and the inability to efficiently complete inspection work. Summary of the invention

[0004] The embodiment of the present invention provides a drone inspection method, which is used to improve the accuracy of the cruise route, improve the inspection effect, and ensure that the drone can efficiently complete the inspection work in a complex environment. The method includes:

[0005] Obtain environmental data of multiple movement directions of the drone collected by sensors, perform three-dimensional reconstruction based on the environmental data, and obtain three-dimensional environmental information of the drone's surroundings; the three-dimensional environmental information includes: obstacle location information, drivable area information, and non-drivable area information;

[0006] Based on the initial position information of the UAV, a drivable area that includes the initial position information and the final position information of the UAV and does not intersect with any obstacles is selected from the drivable area information;

[0007] Set up multiple inspection points within the drivable area and obtain the location information of each inspection point;

[0008] Based on the location information of each inspection point, the drone's location information and yaw angle information are collected when the drone passes the corresponding inspection point. The drone's cruising trajectory is determined based on the drone's location information and yaw angle information to complete the inspection task.

[0009] The embodiment of the present invention further provides a drone inspection device, which is used to improve the accuracy of the cruise route, improve the inspection effect, and ensure that the drone can efficiently complete the inspection work in a complex environment. The device includes:

[0010] The three-dimensional environment information acquisition module is used to obtain the environmental data of multiple movement directions of the UAV collected by the sensor, and perform three-dimensional reconstruction based on the environmental data to obtain the three-dimensional environment information of the UAV's surrounding environment; the three-dimensional environment information includes: obstacle location information, drivable area information, and non-drivable area information;

[0011] A drivable domain screening module is used to screen out a drivable domain that includes the initial position information and the final position information of the UAV and does not intersect with any obstacles from the drivable area information based on the initial position information of the UAV;

[0012] The inspection point setting module is used to set multiple inspection points within the drivable domain and obtain the location information of each inspection point;

[0013] The cruise trajectory determination module is used to collect the drone position information and yaw angle information when the drone passes the corresponding inspection point based on the location information of each inspection point, determine the drone's cruise trajectory according to the drone position information and yaw angle information, and complete the inspection task.

[0014] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned drone inspection method when executing the computer program.

[0015] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned drone inspection method is implemented.

[0016] An embodiment of the present invention further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above-mentioned drone inspection method is implemented.

[0017] In the embodiment of the present invention, by acquiring the environmental data of multiple movement directions of the drone collected by the sensor, three-dimensional reconstruction is performed according to the environmental data to obtain the three-dimensional environmental information of the drone's surrounding environment; the three-dimensional environmental information includes: obstacle position information, drivable area information, and non-drivable area information; based on the initial position information of the drone, a drivable domain containing the initial position information and the terminal position information of the drone and not intersecting with any obstacles is screened out from the drivable area information; multiple inspection points are set in the drivable domain to obtain the position information of each inspection point; based on the position information of each inspection point, the drone position information and yaw angle information when the drone passes through the corresponding inspection point are collected, and the cruise trajectory of the drone is determined according to the drone position information and yaw angle information to complete the inspection task. In the above process, the embodiment of the present invention first determines the drivable area information according to the three-dimensional environmental information, and determines the drivable domain of the drone based on the drivable area information on the premise of avoiding intersection with the fault object. On this basis, the cruise trajectory is described according to the three-dimensional environmental information and yaw angle information of the drone at the inspection point, thereby improving the accuracy of the cruise route, improving the inspection effect, and ensuring that the drone can efficiently complete the inspection work in a complex environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0019] Figure 1 This is a flow chart of the UAV inspection method in an embodiment of the present invention;

[0020] Figure 2 A flowchart of determining the position of an obstacle in an embodiment of the present invention;

[0021] Figure 3 A flowchart for determining a drivable domain in an embodiment of the present invention;

[0022] Figure 4 This is a flow chart of UAV cruise trajectory planning based on yaw angle in an embodiment of the present invention;

[0023] Figure 5 This is a rendering of the cruise trajectory planning of a UAV based on the yaw angle in an embodiment of the present invention;

[0024] Figure 6 Schematic diagram of a drone inspection device in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0026] Figure 1 This is a flow chart of a drone inspection method in an embodiment of the present invention, the method comprising:

[0027] Step 101, obtaining environmental data of multiple movement directions of the drone collected by sensors, performing three-dimensional reconstruction based on the environmental data, and obtaining three-dimensional environmental information of the drone's surroundings; the three-dimensional environmental information includes: obstacle location information, drivable area information, and non-drivable area information;

[0028] Step 102, based on the initial position information of the UAV, filter out the drivable area that includes the initial position information and the final position information of the UAV and does not intersect with any obstacles from the drivable area information;

[0029] Step 103, setting a plurality of inspection points within the drivable domain, and obtaining location information of each inspection point;

[0030] Step 104, based on the location information of each inspection point, collect the drone location information and yaw angle information when the drone passes the corresponding inspection point, determine the cruise trajectory of the drone according to the drone location information and yaw angle information, and complete the inspection task.

[0031] Each step is described in detail below.

[0032] In step 101, environmental data of multiple movement directions of the drone collected by sensors are obtained, and three-dimensional reconstruction is performed based on the environmental data to obtain three-dimensional environmental information of the drone's surroundings; the three-dimensional environmental information includes: obstacle location information, drivable area information, and non-drivable area information.

[0033] Figure 2 This is a flow chart for determining the position of an obstacle in an embodiment of the present invention. In one embodiment, environmental data of multiple movement directions of a drone collected by a sensor is obtained, and three-dimensional environmental information of the drone's surrounding environment is obtained by performing three-dimensional reconstruction based on the environmental data, including:

[0034] Step 201, obtaining point cloud data collected by the laser radar, and converting the point cloud data into the drone coordinate system;

[0035] Step 202 , based on the preset threshold height of the obstacle, point cloud data that meets the preset threshold height of the obstacle is screened out from the point cloud data after coordinate conversion as the point cloud data of the obstacle.

[0036] In step 102, based on the initial position information of the drone, a drivable area that includes the initial position information and the final position information of the drone and does not intersect with any obstacles is screened out from the drivable area information.

[0037] In a specific embodiment, the drivable domain of the drone is calculated to obtain a safe area in the surrounding environment where the drone can cruise. Environmental data is collected using sensors such as laser radar and cameras to determine three-dimensional environmental information; obstacles in the environment are represented as point clouds and polygonal geometries, and converted into the union form of convex obstacles.

[0038] Figure 3 The flowchart for determining a drivable domain in an embodiment of the present invention is as follows. In one embodiment, based on the initial position information of the drone, a drivable domain including the initial position information and the final position information of the drone and not intersecting with any obstacle is screened out from the drivable area information, including:

[0039] Step 301, based on the point cloud data of the obstacle, a convex hull algorithm is used to draw a circumscribed convex polygon to obtain a set of convex polygons of the obstacle;

[0040] Step 302: Based on the initial position information of the UAV, a drivable area that includes the initial position information and the final position information of the UAV and does not intersect with the set of convex polygons of obstacles is selected from the drivable area information.

[0041] In a specific embodiment, according to the initial position, final position and flight mission requirements of the UAV, an initial convex feasible domain including the initial position and final position and not intersecting with any obstacle is selected, and a convex feasible domain satisfying driving safety is gradually generated by iteratively optimizing the initial ellipsoid volume.

[0042] In a specific embodiment, first, the present invention uses O to represent a point set of obstacles in the environment. The piecewise linear path P from the starting point to the end point is represented in free space as P = (p0 → p1 → ... → p n ), where p i represents a point in free space, p i →p i+1 Represents a directed line segment in free space. Then, a convex polyhedron is generated around each line segment in P to construct an effective safe feasible region.

[0043] In a specific embodiment, in actual flight, the drone needs to solve the control quantity in real time according to the cruise trajectory. According to the differential flatness characteristics of the drone, the motion trajectory is expressed as a function of a flat output trajectory. The differential flat output is to control and simplify the 12-dimensional full state vector of the drone to find a flat output space, that is, the 3D position x, y, z and attitude angle ψ. The position information and yaw angle information of the drone compared to the world coordinate system are the most intuitive information reflecting the attitude and position of the drone. Therefore, in the differential flat space, the drone position x, y, z, and attitude angle ψ are used as the output vector σ:

[0044] σ=[x,y,z,ψ] T

[0045] After the final adjustment, the trajectory-based timing output is σ(t):

[0046]

[0047] This time series function represents the time from T0 to T M At time , the trajectory of the drone is defined in That is, the position space, and the rotation space of a single yaw angle.

[0048] In step 103, multiple inspection points are set within the drivable domain, and the location information of each inspection point is obtained.

[0049] In a specific embodiment, the drone needs to accurately control the position and yaw angle of the inspection points and plan an optimal cruising trajectory, which must not only cover all important equipment corresponding to the inspection points, but also complete the inspection task within limited time and energy, so that the camera sensor can effectively collect equipment information and complete the shooting task. Figure 4 Flow chart of the UAV cruise trajectory planning based on the yaw angle in the embodiment of the present invention. The specific process of the UAV cruise trajectory planning based on the yaw angle is as follows:

[0050] 1) Preset and enter a series of inspection points to be visited, where each inspection point needs to contain three-dimensional position and yaw angle information. Among them, there are two modes for entering inspection points: manual entry and automatic entry of YAML files. In the manual mode, a series of waypoints containing position information and yaw angle information are set through mouse interaction in the visual interface. In the automatic entry mode of the YAML file, a series of task points are collected by surveying and mapping in advance, and the precise position information and yaw angle information are entered in advance.

[0051] 2) After entering the inspection points, the drone performs point-to-point flights between the inspection points.

[0052] 3) After reaching the target inspection point (target point), the drone performs a yaw angle rotation and uses a camera sensor to capture the equipment and instrument information of the electric field.

[0053] 4) Obtain the yaw angle rotation state and determine whether the yaw angle reaches the preset state. If so, take the next inspection point as the target point, otherwise keep the current inspection point.

[0054] 5) After determining whether the yaw angle has reached the preset state, perform a cruise mission check to confirm mission completion.

[0055] In a specific embodiment, it is determined in real time whether the drone has reached the target point. When the drone reaches the target point, the state machine is switched to enter the differential flat yaw angle rotation mode. The drone enters a uniform rotation state. When the drone reaches the preset yaw angle, it automatically exits the differential flat yaw angle rotation mode and enters the planning mode of the next waypoint in series with the front end. Figure 5 As shown, Figure 5 This is a rendering of the cruise trajectory planning of a UAV based on the yaw angle in an embodiment of the present invention, where the red dots represent inspection points.

[0056] In step 104, based on the location information of each inspection point, the location information and yaw angle information of the drone when the drone passes the corresponding inspection point are collected, and the cruising trajectory of the drone is determined according to the location information and yaw angle information of the drone to complete the inspection task.

[0057] In one embodiment, based on the location information of each inspection point, the location information and yaw angle information of the drone when the drone passes the corresponding inspection point are collected, and the cruising trajectory of the drone is determined according to the location information and yaw angle information of the drone to complete the inspection task, including:

[0058] According to the location information and yaw angle information of the UAV at each inspection point, the UAV cruising point is determined, and the UAV cruising trajectory is generated. The UAV completes the inspection according to the cruising trajectory.

[0059] The present invention also provides a drone inspection device in an embodiment, as described in the following embodiment. Since the principle of solving the problem by the device is similar to that of the drone inspection method, the implementation of the device can refer to the implementation of the drone inspection method, and the repeated parts will not be repeated.

[0060] Figure 6 This is a schematic diagram of a drone inspection device in an embodiment of the present invention, the device comprising:

[0061] The three-dimensional environment information acquisition module 601 is used to acquire the environment data of multiple movement directions of the UAV collected by the sensor, and perform three-dimensional reconstruction based on the environment data to obtain the three-dimensional environment information of the UAV surrounding environment; the three-dimensional environment information includes: obstacle location information, drivable area information, and non-drivable area information;

[0062] The drivable domain screening module 602 is used to screen out the drivable domain that includes the initial position information and the final position information of the drone and does not intersect with any obstacles from the drivable area information based on the initial position information of the drone;

[0063] A checkpoint setting module 603 is used to set multiple checkpoints within the drivable domain and obtain the location information of each checkpoint;

[0064] The cruise trajectory determination module 604 is used to collect the drone position information and yaw angle information when the drone passes the corresponding inspection point based on the position information of each inspection point, determine the cruise trajectory of the drone according to the drone position information and yaw angle information, and complete the inspection task.

[0065] In one embodiment, the three-dimensional environment information acquisition module 601 is specifically used to:

[0066] Obtain the point cloud data collected by the laser radar and convert the point cloud data into the drone coordinate system;

[0067] According to the preset threshold height of the obstacle, point cloud data that meets the preset threshold height of the obstacle is screened out from the point cloud data after coordinate conversion as the point cloud data of the obstacle.

[0068] In one embodiment, the drivable domain screening module 602 is specifically used to:

[0069] Based on the point cloud data of the obstacle, the convex hull algorithm is used to draw the circumscribed convex polygon to obtain the obstacle convex polygon set;

[0070] Based on the initial position information of the UAV, the drivable domain that contains the initial position information and the terminal position information of the UAV and does not intersect with the convex polygon set of obstacles is filtered out from the drivable area information.

[0071] In one embodiment, the cruise trajectory determination module 604 is specifically configured to:

[0072] According to the location information and yaw angle information of the UAV at each inspection point, the UAV cruising point is determined, and the UAV cruising trajectory is generated. The UAV completes the inspection according to the cruising trajectory.

[0073] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned drone inspection method when executing the computer program.

[0074] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned drone inspection method is implemented.

[0075] An embodiment of the present invention further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the above-mentioned drone inspection method is implemented.

[0076] In the embodiment of the present invention, by acquiring the environmental data of multiple movement directions of the drone collected by the sensor, three-dimensional reconstruction is performed according to the environmental data to obtain the three-dimensional environmental information of the drone's surrounding environment; the three-dimensional environmental information includes: obstacle position information, drivable area information, and non-drivable area information; based on the initial position information of the drone, a drivable domain containing the initial position information and the terminal position information of the drone and not intersecting with any obstacles is screened out from the drivable area information; multiple inspection points are set in the drivable domain to obtain the position information of each inspection point; based on the position information of each inspection point, the drone position information and yaw angle information when the drone passes through the corresponding inspection point are collected, and the cruise trajectory of the drone is determined according to the drone position information and yaw angle information to complete the inspection task. In the above process, the embodiment of the present invention first determines the drivable area information according to the three-dimensional environmental information, and determines the drivable domain of the drone based on the drivable area information on the premise of avoiding intersection with the fault object. On this basis, the cruise trajectory is described according to the three-dimensional environmental information and yaw angle information of the drone at the inspection point, thereby improving the accuracy of the cruise route, improving the inspection effect, and ensuring that the drone can efficiently complete the inspection work in a complex environment.

[0077] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0081] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A drone inspection method, characterized in that: include: Obtain environmental data of multiple movement directions of the drone collected by sensors, perform three-dimensional reconstruction based on the environmental data, and obtain three-dimensional environmental information of the drone's surroundings; The three-dimensional environment information includes: obstacle location information, drivable area information, and non-drivable area information; Based on the initial position information of the UAV, a drivable area that includes the initial position information and the final position information of the UAV and does not intersect with any obstacles is selected from the drivable area information; Set up multiple inspection points within the drivable area and obtain the location information of each inspection point; Based on the location information of each inspection point, the drone's location information and yaw angle information are collected when the drone passes the corresponding inspection point. The drone's cruising trajectory is determined based on the drone's location information and yaw angle information to complete the inspection task.

2. The method according to claim 1, characterized in that Obtain environmental data of multiple movement directions of the drone collected by sensors, perform three-dimensional reconstruction based on the environmental data, and obtain three-dimensional environmental information of the drone's surroundings, including: Obtain the point cloud data collected by the laser radar and convert the point cloud data into the drone coordinate system; According to the preset threshold height of the obstacle, point cloud data that meets the preset threshold height of the obstacle is screened out from the point cloud data after coordinate conversion as the point cloud data of the obstacle.

3. The method according to claim 2, characterized in that Based on the initial position information of the UAV, the drivable area that contains the initial position information and the final position information of the UAV and does not intersect with any obstacles is selected from the drivable area information, including: Based on the point cloud data of the obstacle, the convex hull algorithm is used to draw the circumscribed convex polygon to obtain the obstacle convex polygon set; Based on the initial position information of the UAV, the drivable domain that contains the initial position information and the terminal position information of the UAV and does not intersect with the convex polygon set of obstacles is filtered out from the drivable area information.

4. The method according to claim 1, characterized in that Based on the location information of each inspection point, the location information and yaw angle information of the drone when it passes the corresponding inspection point are collected, and the cruising trajectory of the drone is determined according to the location information and yaw angle information of the drone to complete the inspection task, including: According to the location information and yaw angle information of the UAV at each inspection point, the UAV cruising point is determined, and the UAV cruising trajectory is generated. The UAV completes the inspection according to the cruising trajectory.

5. A drone inspection device, characterized in that: include: The three-dimensional environment information acquisition module is used to obtain the environmental data of multiple movement directions of the UAV collected by the sensor, perform three-dimensional reconstruction based on the environmental data, and obtain the three-dimensional environmental information of the UAV's surroundings; The three-dimensional environment information includes: obstacle location information, drivable area information, and non-drivable area information; A drivable domain screening module is used to screen out a drivable domain that includes the initial position information and the final position information of the UAV and does not intersect with any obstacles from the drivable area information based on the initial position information of the UAV; The inspection point setting module is used to set multiple inspection points within the drivable domain and obtain the location information of each inspection point; The cruise trajectory determination module is used to collect the drone position information and yaw angle information when the drone passes the corresponding inspection point based on the location information of each inspection point, determine the drone's cruise trajectory according to the drone position information and yaw angle information, and complete the inspection task.

6. The device according to claim 5, characterized in that The three-dimensional environment information acquisition module is specifically used for: Obtain the point cloud data collected by the laser radar and convert the point cloud data into the drone coordinate system; According to the preset threshold height of the obstacle, point cloud data that meets the preset threshold height of the obstacle is screened out from the point cloud data after coordinate conversion as the point cloud data of the obstacle.

7. The device according to claim 6, characterized in that The drivable domain screening module is specifically used for: Based on the point cloud data of the obstacle, the convex hull algorithm is used to draw the circumscribed convex polygon to obtain the obstacle convex polygon set; Based on the initial position information of the UAV, the drivable domain that contains the initial position information and the terminal position information of the UAV and does not intersect with the convex polygon set of obstacles is filtered out from the drivable area information.

8. The device according to claim 5, characterized in that The cruise trajectory determination module is specifically used for: According to the location information and yaw angle information of the UAV at each inspection point, the UAV cruising point is determined, and the UAV cruising trajectory is generated. The UAV completes the inspection according to the cruising trajectory.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images