Unmanned aerial vehicle automatic inspection method and system, computer storage medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, discloses an unmanned aerial vehicle automatic inspection method and system, a computer readable storage medium and a computer program product, and is used for solving the technical problem that an unmanned aerial vehicle automatic inspection method in the prior art is difficult to automatically control an inspection path in a scene lacking an external navigation signal. The method comprises the following steps: receiving and analyzing an unmanned aerial vehicle automatic inspection instruction to obtain an offline point cloud map, an inspection route and a waypoint task of a scene to be inspected; acquiring real-time image information of a surrounding scene of the unmanned aerial vehicle, and determining real-time pose information of the unmanned aerial vehicle according to the real-time image information and the offline point cloud map; performing routing inspection path planning based on the real-time pose information and the routing inspection route of the unmanned aerial vehicle, and generating a flight control instruction of the unmanned aerial vehicle; and controlling the unmanned aerial vehicle to fly according to the flight control instruction and executing a waypoint task to complete automatic inspection. According to the method, the unmanned aerial vehicle can realize automatic inspection in a complex environment in the absence of external navigation signal support.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for automatic inspection by unmanned aerial vehicles, a computer storage medium, and a computer program product. Background Art

[0002] With the continuous development of unmanned driving technology, drone inspection technology has also been widely used in industrial fields such as power facility inspection, chemical equipment monitoring, and warehouse logistics management. Traditional drones rely primarily on GNSS (Global Navigation Satellite System) for positioning and navigation in outdoor environments to perform automated inspections. However, in typical industrial scenarios such as substation indoor equipment areas, enclosed warehouses, and underground pipeline corridors, building structures severely block GNSS signals, making it impossible for drones to obtain effective positioning information. The complex spatial structures commonly found in these scenarios further exacerbate the difficulty of drone obstacle avoidance and path planning.

[0003] Existing technologies typically use single-sensor fusion or pre-defined environmental feature positioning solutions for drones operating in environments without satellite navigation. However, such solutions rely heavily on prior environmental information or fixed infrastructure, making them difficult to adapt to the complex conditions of industrial scenarios, such as the diverse spatial layouts and dense equipment density. In practice, indoor inspections still require extensive manual control. Therefore, there is an urgent need to develop an automated drone inspection method that can automatically complete inspection tasks in complex environments without the support of external navigation signals. Summary of the Invention

[0004] The main purpose of the present invention is to solve the technical problem that the existing drone automatic inspection method is difficult to automatically control the inspection path in the absence of external navigation signals and cannot realize automated inspection.

[0005] A first aspect of the present invention provides a method for automatic inspection by a drone, comprising:

[0006] Receive and parse UAV automatic inspection instructions to obtain offline point cloud maps, inspection routes, and waypoint tasks of the inspection scene;

[0007] Acquire real-time image information of the scene around the drone, and determine the real-time pose information of the drone based on the real-time image information and the offline point cloud map;

[0008] Performing inspection path planning based on the real-time posture information of the UAV and the inspection route, and generating flight control instructions for the UAV;

[0009] Control the flight of the drone according to the flight control instruction and execute the waypoint task to complete the automatic inspection.

[0010] Optionally, in the first implementation manner of the first aspect of the present invention, the obtaining the real-time image information of the surrounding scene of the drone and determining the real-time pose information of the drone according to the real-time image information and the offline point cloud map includes:

[0011] Obtain the real-time image information of the surrounding scene of the drone through a camera module communicatively connected to the twin autopilot;

[0012] Extract the image features of the real-time image information based on the SLAM algorithm, perform feature matching according to the image features and the offline point cloud map, and calculate the real-time pose information of the drone.

[0013] Optionally, in the second implementation manner of the first aspect of the present invention, the performing the inspection path planning based on the real-time pose information of the drone and the inspection route and generating the flight control instruction of the drone includes:

[0014] Call the Ego Planner path planning algorithm to perform the inspection path planning based on the real-time pose information of the drone and the inspection route of the scene to be inspected, and obtain the pose expected value of the inspection flight path;

[0015] Call the PID control algorithm to generate the flight control instruction of the drone according to the real-time pose information and the pose expected value, and control the drone to adjust the actual flight pose so that the actual flight pose meets the pose expected value.

[0016] Optionally, in the third implementation manner of the first aspect of the present invention, the waypoint task is a photographing task;

[0017] The controlling the flight of the drone according to the flight control instruction and executing the waypoint task includes:

[0018] The drone flies to the target position of the waypoint task in sequence according to the flight control instruction, and obtains the real-time image information of the real-time target position;

[0019] Check whether the real-time image information contains an identifiable waypoint task mark;

[0020] If it contains, take a picture of the target position including the waypoint task mark to complete the waypoint task.

[0021] Optionally, in the fourth implementation manner of the first aspect of the present invention, after the controlling the flight of the drone according to the flight control instruction and executing the waypoint task, it further includes:

[0022] Perform return path planning based on the real-time pose information of the drone and the offline point cloud map, and generate flight control instructions for the return flight.

[0023] Control the drone to fly near the target landing point of the drone nest according to the flight control instructions during the return flight, and perform the drone homing operation.

[0024] Optionally, in the fifth implementation manner of the first aspect of the present invention, it further includes:

[0025] Send real-time control instructions to the twin autopilot according to the actual inspection requirements.

[0026] The twin autopilot controls the drone to perform task startup, termination, suspension, continuation, and return operations according to the received real-time control instructions.

[0027] The second aspect of the present invention provides a drone automatic inspection system, including:

[0028] A drone and a twin autopilot communicatively connected to the drone, and the twin autopilot further includes:

[0029] A message response module for receiving and parsing drone automatic inspection instructions to obtain an offline point cloud map, an inspection route, and a waypoint task of the scene to be inspected.

[0030] A positioning module for obtaining real-time image information of the surrounding scene of the drone, and determining the real-time pose information of the drone according to the real-time image information and the offline point cloud map.

[0031] A path planning module for performing inspection path planning based on the real-time pose information of the drone and the inspection route of the scene to be inspected, and generating flight control instructions for the drone.

[0032] A navigation control module for controlling the drone to fly according to the flight control instructions and performing the waypoint task to complete automatic inspection.

[0033] The third aspect of the present invention provides a computer-readable storage medium, in which instructions are stored, and when they run on a computer, the computer is caused to execute the steps of the above-mentioned drone automatic inspection method.

[0034] The fifth aspect of the present invention provides a computer program product, including computer programs / instructions, characterized in that when the computer programs / instructions are executed by a processor, the steps of the above-mentioned drone automatic inspection method are performed.

[0035] In the technical solution provided by the present invention, an automatic inspection instruction of an unmanned aerial vehicle (UAV) is received and parsed to obtain an offline point cloud map, an inspection route, and a waypoint task of the scene to be inspected; real-time image information of the surrounding scene of the UAV is acquired, and the real-time pose information of the UAV is determined based on the real-time image information and the offline point cloud map; an inspection path is planned based on the real-time pose information of the UAV and the inspection route, and a flight control instruction for the UAV is generated; the UAV is controlled to fly according to the flight control instruction and the waypoint task is executed to complete the automatic inspection. A system, a computer-readable storage medium, and a computer program product provided by the present invention also solve the corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments and descriptions thereof are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0037] Figure 1 is a schematic flowchart of the first embodiment of the UAV automatic inspection method in an embodiment of the present invention;

[0038] Figure 2 is a schematic flowchart of the second embodiment of the UAV automatic inspection method in an embodiment of the present invention;

[0039] Figure 3 is a schematic diagram of the UAV and the twin autopilot in the UAV automatic inspection method in an embodiment of the present invention;

[0040] Figure 4 is a schematic diagram of information transmission in the UAV automatic inspection method in an embodiment of the present invention;

[0041] Figure 5 is a schematic diagram of an embodiment of the UAV automatic inspection system in an embodiment of the present invention;

[0042] Figure 6 is a schematic diagram of another embodiment of the UAV automatic inspection system in an embodiment of the present invention;

[0043] Figure 7 is a schematic diagram of the principle of a computer-readable medium in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, providing these exemplary embodiments enables the present invention to be more comprehensive and complete, and more conveniently conveys the inventive concept to those skilled in the art. Identical reference numerals in the figures represent the same or similar elements, components, or parts, and thus their repeated description will be omitted.

[0045] Under the premise of being consistent with the technical concept of the present invention, the features, structures, characteristics or other details described in a specific embodiment do not exclude that they can be combined in one or more other embodiments in a suitable manner.

[0046] In the description of specific embodiments, the features, structures, characteristics, or other details of the present invention are described to enable those skilled in the art to fully understand the embodiments. However, this does not preclude those skilled in the art from practicing the technical solutions of the present invention without one or more of the specific features, structures, characteristics, or other details.

[0047] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0048] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0049] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0050] See also Figure 1 , the following is a first embodiment of the automatic inspection method of a drone in an embodiment of the present invention.

[0051] S101, receiving and parsing the UAV automatic inspection command, obtaining the offline point cloud map, inspection route and waypoint tasks of the inspection scene;

[0052] It can be understood that the executor of the present invention can be an automatic inspection system for drones, which includes a drone, a twin autopilot connected to the drone, a drone nest, a drone control center and other devices. The devices in the system can cooperate to realize the automatic inspection method for drones.

[0053] In this embodiment, the UAV control center sends an automatic inspection instruction to the corresponding target UAV according to specific inspection requirements. In this embodiment, the UAV inspection plan is mainly implemented based on the twin autopilot linked to the UAV, and the twin autopilot is communicatively connected to the communication module of the UAV. Specifically, the twin autopilot receives the UAV automatic inspection instruction through the communication module of the UAV; by analyzing the instruction of the UAV automatic inspection, the offline point cloud map, inspection route and waypoint task of the scene to be inspected can be obtained.

[0054] In a specific implementation manner, the UAV automatic inspection system for implementing the UAV automatic inspection method further includes a UAV remote controller. The UAV control center sends the UAV automatic inspection instruction to the UAV remote controller. After receiving the UAV automatic inspection instruction, the UAV remote controller controls the UAV nest to start, and then the UAV and the twin autopilot are powered on and initialized; after the initialization is completed, the UAV remote controller sends the UAV automatic inspection instruction to the twin autopilot again. After analyzing the UAV automatic inspection instruction, the twin autopilot obtains the offline point cloud map, inspection route and waypoint task of the scene to be inspected.

[0055] S102. Obtain the real-time image information of the scene around the UAV, and determine the real-time pose information of the UAV according to the real-time image information and the offline point cloud map;

[0056] The twin autopilot calls the camera module to obtain the real-time image information of the scene around the UAV, and corrects the distortion of the obtained real-time image information to obtain the corrected two-dimensional real-time image; performs plane mapping on the offline point cloud map from multiple angles to obtain a two-dimensional point cloud image, and matches the corrected two-dimensional real-time image with the two-dimensional point cloud image to solve the position coordinates of the UAV when the real-time image information of the scene around the UAV is taken, and obtain the real-time pose information of the UAV.

[0057] In a preferred implementation manner, the real-time pose information of the UAV can also be based on the IMU (inertial measurement unit) built in the twin autopilot to determine the specific flight attitude.

[0058] S103. Plan the inspection path based on the real-time pose information of the UAV and the inspection route, and generate a flight control instruction for the UAV;

[0059] Perform path planning for inspection based on the acquired real-time pose information and inspection route of the UAV, so that the UAV can cover the positions where the waypoint tasks on the inspection route are located according to the planned path. Specifically, when planning the path, the expected value of the UAV flight pose is obtained. Based on the obtained expected value of the UAV flight pose and the real-time pose information of the UAV, the flight control command of the UAV is calculated.

[0060] S104. Control the UAV to fly according to the flight control command and execute the waypoint task to complete the automatic inspection.

[0061] Control the UAV to fly to the position where the waypoint task is located based on the calculated flight control command and execute the specific waypoint task.

[0062] In a specific embodiment, the waypoint task is a photographing task; specifically, executing the waypoint task in this step includes: the UAV flies to the target position of the waypoint task in sequence according to the flight control command, and acquires the real-time image information of the real-time target position; checks whether the real-time image information contains recognizable waypoint task marks; if so, takes a target position image containing the waypoint task marks to complete the waypoint task. After all the waypoint tasks are executed, the automatic inspection is completed, and the twin navigator controls the UAV to perform the return operation.

[0063] In the embodiment of the present invention, the position information of the UAV is obtained by using the visual positioning method, and the UAV is controlled to fly along the planned path and automatically execute tasks according to the route and the position information of the UAV, so that the inspection task in a complex environment can be automatically completed even in an environment without external navigation such as indoors.

[0064] Please refer to Figures 2 - 4 , the following is the second embodiment of the UAV automatic inspection method in the embodiment of the present invention.

[0065] The execution subject of the UAV automatic inspection method in this embodiment can be a UAV automatic inspection system. The UAV automatic inspection system includes a UAV, a UAV remote controller, a twin autopilot connected to the UAV, a UAV nest, a UAV control center and other devices. The devices in the system cooperate to run to realize the UAV automatic inspection method.

[0066] Before the UAV receives the automatic inspection command and executes the specific inspection task, the user can configure the specific inspection scenario information and inspection task information at the UAV control center according to the specific task to be executed. Among them, the inspection scenario information may include the offline point cloud map of the scenario. Further, the offline point cloud map may also include the task point annotation information of the task to be executed.

[0067] Please continue to refer toFigure 4 When it is necessary to control the drone to perform an inspection mission, the user triggers a drone inspection execution request based on the drone control center or triggers a drone inspection execution request at regular intervals through a timer. The drone control center sends the drone inspection execution request to the drone remote controller. In addition, during the working process, the drone control center also sends some real-time control instructions to the drone remote controller, such as one-key return instruction, video source switching instruction, drone remote control instruction, gimbal remote control instruction, photo-taking control instruction, and instructions for starting, terminating, pausing, continuing, and returning of the flight path task; the drone remote controller controls the drone and the twin autopilot based on the specific instructions received. Also, the drone remote controller sends the real-time status information of the drone, the twin autopilot, and the drone nest to the drone control center, and uploads data logs, photos, etc. to the drone control center at regular intervals, and uploads alarm information to the drone control center when a failure or special situation occurs. In a specific implementation manner, the drone control center is a space management machine.

[0068] After receiving the issued drone inspection execution request, the drone remote controller controls the drone nest to perform takeoff preparation. Among them, in this embodiment, the drone remote controller can be associated with multiple drones and drone nests, and it can determine the currently required drone nests and corresponding drones according to the specific quantity required during the drone inspection. The specific scheme will not be elaborated here. When the drone remote controller controls the nest to prepare for takeoff, the corresponding main control unit of the drone nest controls the nest to start up based on the takeoff request sent by the drone remote controller, and controls the drone to power on and initialize.

[0069] Please combine Figure 3 and Figure 4In this embodiment, the drone patrol solution is mainly based on the twin autopilot connected to the drone. The twin autopilot can be detachably fixed to the drone by connecting components such as a mounting bracket, and is communicatively connected to the drone's communication module; in a specific embodiment, the communication connection between the twin autopilot and the drone can be achieved based on the drone's development protocol, such as the PDSK (Payload Software Development Kit) and other development protocols. The twin autopilot can communicate with the drone remote control or the drone control center through the drone's communication module, and the twin autopilot can obtain information such as the drone's attitude information and sensor information. The twin autopilot also includes a SLAM positioning module, where SLAM stands for simultaneous localization and mapping. The drone remote control sends a SLAM initialization instruction to the drone's twin autopilot through the drone's communication module, and the twin autopilot performs SLAM initialization based on the SLAM initialization instruction. After the initialization is completed, the twin autopilot sends an initialization completion message to the drone remote control through the drone's communication module.

[0070] After receiving the initialization completion information from the twin autopilot, the drone remote control sends specific drone automatic inspection instructions to the twin autopilot.

[0071] S201: The twin autopilot receives and parses the UAV automatic inspection command to obtain the offline point cloud map, inspection route, and waypoint mission of the inspection scene;

[0072] The twin autopilot receives the drone's automatic inspection instructions through the drone's communication module; by parsing the drone's automatic inspection instructions, an offline point cloud map, inspection route, and waypoint tasks of the scene to be inspected can be obtained.

[0073] S202, obtaining real-time image information of the scene around the drone through a camera module communicating with the twin autopilot;

[0074] S203, extracting image features of real-time image information based on the SLAM algorithm, performing feature matching based on the image features and the offline point cloud map, and calculating the real-time position information of the UAV;

[0075] The twin autopilot calls the camera module that is communicated with the twin autopilot to obtain real-time image information of the scene around the drone, and extracts image features of the real-time image information based on the SLAM algorithm, performs feature matching based on the image features and the offline point cloud map, and solves the position coordinates of the drone when shooting the real-time image information of the scene around the drone to obtain the real-time position information of the drone.

[0076] The twin autopilot can also obtain the real-time motion attitude information of the UAV through the IMU (Inertial Measurement Unit), and calculate the real-time pose information of the UAV based on the real-time position information determined by the real-time image information and the real-time motion attitude information obtained by the inertial measurement unit.

[0077] In a specific implementation manner, the offline point cloud map can be a dense laser point cloud map, and the image features extracted from the real-time image information based on the SLAM algorithm can be the sparse point cloud of the SLAM visual features obtained based on the SLAM algorithm. When performing specific matching, the prior laser point cloud map data can be processed offline to obtain stable three-dimensional point-line features in the point cloud structure; the two-dimensional point-line features in the image are detected in real time and matched with the three-dimensional point-line features to adjust the visual estimated pose.

[0078] Specifically, when performing matching and positioning, the optical flow method and the SLAM direct method can also be combined to estimate the real-time pose of the UAV, obtaining the number of corner points to be optimized and the initialization result; the sub-corner point detection algorithm is used to iterate and improve the accuracy of the initial value; according to the detection result, image marginalization constraints are performed to constrain the problem of out-of-bounds detection of the sub-pixel edges of the corner points of the image, thereby optimizing the corner points and improving the calculation accuracy of the real-time pose information of the UAV.

[0079] S204. Perform inspection path planning based on the real-time pose information of the UAV and the inspection route, and generate flight control instructions for the UAV;

[0080] Perform inspection path planning according to the obtained real-time pose information of the UAV and the inspection route, so that the UAV can cover the positions of the waypoint tasks on the inspection route according to the planned path. Specifically, when planning the path, the expected value of the flight pose of the UAV is obtained. Based on the obtained expected value of the flight pose of the UAV and the real-time pose information of the UAV, flight control instructions for the UAV are calculated.

[0081] In a specific implementation manner, when performing inspection path planning, specifically, the Ego Planner path planning algorithm is called to perform inspection path planning based on the real-time pose information of the UAV and the inspection route of the scene to be inspected, obtaining the expected value of the pose of the inspection flight path; the PID (Proportion Integration Differentiation) control algorithm is called to generate flight control instructions for the UAV according to the real-time pose information and the expected value of the pose, and control the UAV to adjust the actual flight pose so that the actual flight pose meets the expected value of the pose.

[0082] S205. The UAV flies to the target position of the waypoint task in sequence according to the flight control instructions, and obtains the real-time image information of the real-time target position;

[0083] S206. Check whether the real-time image information contains recognizable waypoint task markers. If so, capture an image of the target location containing the waypoint task markers to complete each waypoint task.

[0084] In this embodiment, the waypoint task is a photographing task. Specifically, based on the calculated flight control instructions, the UAV is controlled to fly to the target location where the waypoint task is located, and the camera module of the UAV is called to obtain the real-time image information of the real-time target location. Then check whether the real-time image information contains recognizable waypoint task markers. If so, capture an image of the target location containing the waypoint task markers to complete each waypoint task.

[0085] After completing the photographing tasks for all waypoints, the return path is planned based on the real-time pose information of the UAV and the offline point cloud map, and the flight control instructions for the return flight are generated. According to the flight control instructions for the return flight, the UAV is controlled to fly near the target landing point of the UAV nest. Among them, when generating the flight control instructions for the return flight, it is basically similar to the solution in step S204. That is, based on the Ego Planner path planning algorithm, the PID (Proportion Integration Differentiation) control algorithm generates the UAV flight control instructions according to the real-time pose information and the pose expected value, and controls the UAV to adjust the actual flight pose so that the actual flight pose meets the pose expected value, enabling it to fly near the target landing point of the UAV nest, and perform the landing and homing operations of the UAV according to the preset vision landing algorithm. After the landing is completed, the control module of the nest sends the information of the completion of the landing and the task execution result to the UAV remote controller. After the UAV remote controller obtains the information of the completion of the landing, it sends a cabin closing instruction to the UAV nest. After receiving the cabin closing instruction, the UAV nest closes the UAV switch, charges the UAV and closes the cabin door of the nest.

[0086] In a specific implementation scenario, the UAV automatic inspection method in this embodiment further includes sending control instructions from the UAV remote controller to the twin autopilot. The twin autopilot can determine whether it is necessary to preferentially execute the control instructions sent from the UAV remote controller according to the priority of the received control instructions. When it is necessary to preferentially execute the control instructions sent from the UAV remote controller to the twin autopilot, the twin autopilot pauses the current task and controls the UAV to execute a temporary task based on the received control instructions. Among them, the control instructions can be one-key return instructions, video source switching instructions, UAV remote control instructions, gimbal remote control instructions, photographing control instructions, and instructions for starting, terminating, pausing, continuing, and returning of the route task. Based on these instructions, the twin autopilot can control the UAV to perform one-key return, video source switching, remote real-time control, gimbal remote real-time control, and starting, terminating, pausing, continuing, and returning operations of the route task.

[0087] In the embodiments of the present invention, through the cooperation of the UAV control center, the UAV remote controller, the UAV and the twin autopilot, the automatic inspection of the UAV is realized. Specifically, the twin autopilot obtains the position information of the UAV through visual positioning, and controls the UAV to fly along the planned path and automatically execute tasks according to the flight route and the position information of the UAV. Moreover, it can automatically execute the return flight or perform tasks according to temporary control, enabling it to automatically complete the inspection tasks in complex environments without external navigation.

[0088] The method for automatic inspection of UAVs in the embodiments of the present invention has been described above. Next, the automatic inspection system of UAVs in the embodiments of the present invention will be described. Please refer to Figure 5 An embodiment of the automatic inspection system of UAVs in the embodiments of the present invention includes a UAV 501 and a twin autopilot 502 communicatively connected to the UAV. The twin autopilot 502 further includes:

[0089] A message response module 5021, configured to receive and parse the automatic inspection instruction of the UAV to obtain the offline point cloud map of the scene to be inspected, the inspection flight route, and the waypoint tasks;

[0090] A positioning module 5022, configured to obtain the real-time image information of the surrounding scene of the UAV, and determine the real-time pose information of the UAV according to the real-time image information and the offline point cloud map;

[0091] A path planning module 5023, configured to perform inspection path planning based on the real-time pose information of the UAV and the inspection flight route of the scene to be inspected, and generate a flight control instruction for the UAV;

[0092] A navigation control module 5024, configured to control the flight of the UAV according to the flight control instruction and execute the waypoint tasks to complete the automatic inspection.

[0093] The automatic inspection system of UAVs in the embodiments of the present invention can obtain the position information of the UAV by using visual positioning, and control the UAV to fly along the planned path and automatically execute tasks according to the flight route and the position information of the UAV, and can also automatically complete the inspection tasks in complex environments without external navigation.

[0094] Please refer to Figure 4 and Figure 6 In another embodiment of the present application, the automatic inspection system of UAVs further includes:

[0095] The UAV control center 503, configured to send the automatic inspection instruction of the UAV to the UAV remote controller;

[0096] The drone remote controller 504 is used to send a nest switch instruction to the drone nest and forward the drone automatic inspection instruction and the real-time control instruction to the twin autopilot;

[0097] The nest 505 is used to respond to the nest switch instruction to open or close the warehouse and control the switch of the drone in the cabin.

[0098] In another embodiment of the present application, the positioning module 5022 is specifically used for:

[0099] Obtain the real-time image information of the surrounding scene of the drone through the camera module communicatively connected to the twin autopilot;

[0100] Extract the image features of the real-time image information based on the SLAM algorithm, perform feature matching according to the image features and the offline point cloud map, and calculate the real-time pose information of the drone.

[0101] In another embodiment of the present application, the path planning module 5023 is specifically used for:

[0102] Call the Ego Planner path planning algorithm to perform inspection path planning for the inspection flight path based on the real-time pose information of the drone and the inspection route of the scene to be inspected, and obtain the pose expected value of the inspection flight path;

[0103] Call the PID control algorithm to generate a drone flight control instruction according to the real-time pose information and the pose expected value, and control the drone to adjust the actual flight pose so that the actual flight pose meets the pose expected value.

[0104] In another embodiment of the present application, the waypoint task is a photographing task; the navigation control module 5024 is specifically used for:

[0105] Make the drone fly to the target position of the waypoint task in sequence according to the flight control instruction, and obtain the real-time image information of the real-time target position;

[0106] Check whether the real-time image information contains a recognizable waypoint task mark;

[0107] If it contains, take a picture of the target position image containing the waypoint task mark to complete the waypoint task.

[0108] In another embodiment of the present application, the navigation control module 5024 is specifically further used for:

[0109] Perform return path planning according to the real-time pose information of the drone and the offline point cloud map and generate a flight control instruction for return;

[0110] Control the drone to fly near the target landing point of the drone nest according to the flight control instructions during the return flight, and perform the drone homing operation.

[0111] In another embodiment of the present application, the drone remote controller 504 is further configured to send real-time control instructions to the twin autopilot according to actual inspection requirements;

[0112] The twin autopilot 502 is further configured to control the drone to perform task startup, termination, suspension, continuation, and return flight operations according to the received real-time control instructions.

[0113] In addition, the content during the operation of this drone automatic inspection system can refer to the content in the foregoing method embodiments, so it will not be elaborated here.

[0114] The drone automatic inspection system disclosed in the embodiments of the present invention can realize the automatic inspection of the drone through the cooperation of the drone control center, the drone remote controller, the drone, and the twin autopilot. Specifically, the twin autopilot obtains the position information of the drone through visual positioning, and controls the drone to fly along the planned path and automatically execute tasks according to the route and the position information of the drone, and can automatically execute the return flight or perform work according to temporary control, so that it can automatically complete the inspection tasks in complex environments without external navigation.

[0115] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described in the present invention can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solution according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on the network, including several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the present invention. Therefore, based on the same inventive concept, the embodiments of this specification also provide a computer program and a computer program product for a drone automatic inspection method. When the computer program and the computer program product are executed by a data processing device, the computer-readable medium can implement the above method of the present invention, that is: as Figure 1 Or Figure 2 The method shown.

[0116] Figure 7 It is a schematic diagram of the principle of a computer-readable medium provided by the embodiments of this specification.

[0117] Implement Figure 1A computer program of the method shown in FIG. 1 or 2 can be stored on one or more computer-readable media. The computer-readable media can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0118] The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted by any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.

[0119] The program code for performing the operations of the present invention can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages - such as Java, C++, etc., and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0120] In summary, the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that general-purpose data processing devices such as microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or device program (such as a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0121] In the specific embodiments described above, the purpose, technical solution and beneficial effects of the present invention have been further described in detail. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general-purpose devices can also implement the present invention. The above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0122] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0123] If the technical solution of this application involves personal information, before the product applying the technical solution of this application processes personal information, it has clearly informed the personal information processing rules and obtained the individual's independent consent. If the technical solution of this application involves sensitive personal information, before the product applying the technical solution of this application processes sensitive personal information, it has obtained the individual's separate consent and at the same time meets the requirement of "express consent". For example, at a personal information collection device such as a camera, a clear and prominent sign is set to inform that the personal information collection range has been entered and personal information will be collected. If an individual voluntarily enters the collection range, it is regarded as consenting to the collection of their personal information; or on a personal information processing device, when the personal information processing rules are informed by obvious signs / information, personal authorization is obtained through pop-up information or asking the individual to upload their personal information by themselves; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0124] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. An automatic inspection method for an unmanned aerial vehicle, characterized in that, Including: Receiving and parsing the automatic inspection instructions of the drone to obtain the offline point cloud map, inspection route and waypoint tasks of the scene to be inspected; Obtaining the real-time image information of the surrounding scene of the drone, and determining the real-time pose information of the drone according to the real-time image information and the offline point cloud map; Performing inspection path planning based on the real-time pose information of the drone and the inspection route, and generating the flight control instructions of the drone; Controlling the flight of the drone according to the flight control instructions and executing the waypoint tasks to complete the automatic inspection.

2. The method for automatic inspection of an unmanned aerial vehicle according to claim 1, wherein The obtaining the real-time image information of the surrounding scene of the drone, and determining the real-time pose information of the drone according to the real-time image information and the offline point cloud map includes: Obtaining the real-time image information of the surrounding scene of the drone through the camera module communicatively connected to the twin autopilot; Extracting the image features of the real-time image information based on the SLAM algorithm, performing feature matching according to the image features and the offline point cloud map, and calculating the real-time pose information of the drone.

3. The method for automatic inspection of an unmanned aerial vehicle according to claim 1, wherein The performing inspection path planning based on the real-time pose information of the drone and the inspection route, and generating the flight control instructions of the drone includes: Invoking the Ego Planner path planning algorithm to perform inspection path planning based on the real-time pose information of the drone and the inspection route of the scene to be inspected, and obtaining the pose expected value of the inspection flight path; Invoking the PID control algorithm to generate the flight control instructions of the drone according to the real-time pose information and the pose expected value, and controlling the drone to adjust the actual flight pose so that the actual flight pose meets the pose expected value.

4. The automatic inspection method of an unmanned aerial vehicle according to claim 1, characterized in that, The waypoint task is a photographing task; The controlling the flight of the drone according to the flight control instructions and executing the waypoint tasks includes: The drone flies to the target position of the waypoint task in sequence according to the flight control instructions, and obtains the real-time image information of the real-time target position; Searching whether the real-time image information contains recognizable waypoint task marks; If included, taking a target position image containing the waypoint task mark to complete the waypoint task.

5. The automatic inspection method of an unmanned aerial vehicle according to claim 1, wherein After the controlling the flight of the drone according to the flight control instructions and executing the waypoint tasks, it further includes: Performing return path planning according to the real-time pose information of the drone and the offline point cloud map, and generating the flight control instructions during return; Controlling the flight of the drone to the vicinity of the target landing point of the drone nest according to the flight control instructions during return, and executing the drone homing operation.

6. The method for automatic inspection of an unmanned aerial vehicle according to claim 1, wherein It also includes: Sending real-time control instructions to the twin autopilot according to the actual inspection requirements; The twin autopilot controls the drone to perform task start, termination, pause, continuation and return operations according to the received real-time control instructions.

7. An unmanned aerial vehicle automatic inspection system, characterized in that, Including a drone and a twin autopilot communicatively connected to the drone, and the twin autopilot further includes: A message response module, configured to receive and parse the automatic inspection instructions of the drone to obtain the offline point cloud map, inspection route and waypoint tasks of the scene to be inspected; A positioning module, configured to obtain real-time image information of the surrounding scene of the drone, and determine the real-time pose information of the drone according to the real-time image information and the offline point cloud map; A path planning module, configured to plan a patrol path based on the real-time pose information of the drone and the patrol route of the scene to be inspected, and generate a flight control instruction for the drone; A navigation control module, configured to control the flight of the drone according to the flight control instruction and execute the waypoint task to complete automatic inspection.

8. The drone automatic inspection system according to claim 7, wherein It further includes: A drone control center, configured to send an automatic inspection instruction of the drone to the drone remote controller; A drone remote controller, configured to send a nacelle switch instruction to the drone nacelle, and forward the automatic inspection instruction and the real-time control instruction of the drone to the twin autopilot; A nacelle, configured to respond to the nacelle switch instruction to open or close the compartment, and control the switch of the drone in the compartment; The twin autopilot is further configured to control the drone to perform task start, termination, pause, continuation, and return operations according to the received real-time control instruction.

9. A computer-readable storage medium, on which computer programs / instructions are stored, characterized in that, When the program / instructions are executed by the processor, the steps of the drone automatic inspection method according to any one of claims 1-6 are implemented.

10. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the steps of the drone automatic inspection method according to any one of claims 1-6 are implemented.

Images