Air-ground cooperative autonomous inspection system and method for unmanned aerial vehicle and robot dog

Abstract

The invention discloses an unmanned aerial vehicle and robot dog air-ground cooperative autonomous inspection system and method, and relates to the technical field of inspection, the system comprises an unmanned aerial vehicle, a robot dog and a ground station, the robot dog exerts the characteristics of long endurance and high load and the multi-degree-of-freedom flexibility of mechanical legs, long-distance transportation of the unmanned aerial vehicle and ground observation are achieved, and the unmanned aerial vehicle and robot dog air-ground cooperative autonomous inspection is achieved. The unmanned aerial vehicle provides the flexibility advantage of high-altitude observation, cooperative operation of the unmanned aerial vehicle and the robot dog is achieved, the problem that an existing inspection robot is single in structure and function is solved, and the efficiency and safety of inspection work are improved.

Description

Technical Field

[0001] The present application relates to the field of inspection technology, and in particular to an autonomous inspection system and method for air-to-ground coordinated inspection by a drone and a robot dog. Background Art

[0002] In traditional inspection work, manual inspection faces problems such as low efficiency, high cost, and safety risks. Especially in some areas with complex terrain and harsh environment, the difficulty and danger of manual inspection are further increased. For example, in the maintenance of nuclear power plants, some areas may have excessive radiation, and these areas are not suitable for workers to go there in person for inspection, observation, and instrument inspection; in industrial scenes such as railways, inspection routes are characterized by high repetition and long routes, and in places such as water plants where there are a large number of instruments that need to be read, robots are used to assist workers in field inspections and inspections, which can save physical strength, assist inspection personnel in completing repetitive and tedious reading work, and improve inspection efficiency.

[0003] Currently, inspections are carried out by independent individual robots, but the existing inspection robots have the problem of single structure and function. For example, wheeled robots have fast movement and long battery life, but they have high requirements for the flatness of the ground environment and cannot cross rugged terrain or pass through the ground with obstacles; ground quadruped robots have longer battery life and better terrain adaptability than wheeled robots, but the characteristic of only being able to move on the ground makes it impossible for them to reach higher airspace for detection, and they cannot adapt to terrain with larger slopes. At present, drone inspection operations have problems such as short battery life. They cannot achieve the same long battery life as ground robots while taking into account their smaller size. Large-sized drones have a higher risk factor for indoor flights in factories, and the cost will increase significantly. Summary of the invention

[0004] The purpose of this application is to provide a drone and robot dog air-to-ground collaborative autonomous inspection system and method, which can realize the collaborative operation of drones and robot dogs and solve the problem of single structure and function of existing inspection robots.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a drone and robot dog air-to-ground collaborative autonomous inspection system, including a drone, a robot dog, and a ground station;

[0007] Among them, the drone includes an aerial sensor, an aerial controller, and an aerial actuator;

[0008] The aerial sensor includes an airborne camera module; the airborne camera module is used to collect first image information of the inspection area; the aerial controller is used to control the flight state of the UAV; the aerial driver is used to provide driving force for the UAV;

[0009] The robot dog includes a ground sensor, a ground controller, a ground driver and a take-off and landing platform; the ground sensor includes a ground camera module and a surrounding information acquisition module; the ground camera module is used to acquire second image information of the inspection area; the surrounding information acquisition module is used to acquire environmental information of the inspection area and the position information of the robot dog; the ground controller includes a central processing module; the central processing module is used to perform path planning according to the environmental information of the inspection area, the position information of the robot dog and the target detection point information, and obtain the decision path of the robot dog; the ground driver is used to provide power for the robot dog and the take-off and landing platform; the take-off and landing platform is used to load and place the drone;

[0010] The ground station is used to: issue a detection start instruction; and store the first image information and the second image information.

[0011] Optionally, the central processing module is also used to: control the robot dog to move in the inspection area, and create an environmental map of the inspection area using real-time positioning and map building technology.

[0012] Optionally, the surrounding information collection module includes an infrared sensor; the infrared sensor is used to detect the distance between the robot dog and an obstacle; the distance between the robot dog and the obstacle and the environmental map of the inspection area constitute the environmental information of the inspection area.

[0013] Optionally, the central processing module or the ground station is used to: perform image recognition on the first image information and the second image information to obtain industrial data text information corresponding to the first image information and industrial data text information corresponding to the second image information.

[0014] Optionally, the central processing module or the ground station is used to send a detection start instruction to the drone when there is an area in the inspection area that the robot dog cannot reach.

[0015] Optionally, the aerial sensor further includes an optical flow module; the optical flow module is used to detect speed data of the drone in the x-axis direction and the y-axis direction.

[0016] Optionally, the drone, the robot dog and the ground station are connected via a wireless network communication.

[0017] Optionally, the take-off and landing platform also includes a charging module; the charging module is used to provide power to the drone.

[0018] In the second aspect, the present application provides a method for autonomous air-ground collaborative inspection by a drone and a robot dog, comprising:

[0019] Send a detection start command through the ground station;

[0020] After receiving the detection start command, the robot dog collects environmental information of the inspection area and the location information of the robot dog;

[0021] Through the central processing module, the path planning is carried out according to the environmental information of the inspection area, the location information of the robot dog and the target detection point information to obtain the decision path of the robot dog;

[0022] Based on the decision path, the robot dog is controlled to reach the detection point, and the second image information of the inspection area is collected through the ground camera module;

[0023] When there is an area in the inspection area that the robot dog cannot reach, a detection start instruction is sent to the drone through the central processing module or the ground station to control the drone to reach the inspection point and collect the first image information of the inspection area through the airborne camera module;

[0024] Image recognition is performed on the first image information and the second image information to obtain industrial data text information corresponding to the first image information and industrial data text information corresponding to the second image information.

[0025] Optionally, the method for coordinated autonomous air-to-ground inspection by a drone and a robot dog also includes: controlling the robot dog to move in the inspection area, and creating an environmental map of the inspection area using real-time positioning and map building technology.

[0026] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0027] The present application provides a system and method for the coordinated air-to-ground autonomous inspection of a UAV and a robot dog. The robot dog takes advantage of its long endurance, high load characteristics, and the multi-degree-of-freedom flexibility of its mechanical legs to achieve long-distance transport of the UAV and ground observation. The UAV provides the flexibility advantage of high-altitude observation, and the coordinated operation of the UAV and the robot dog is achieved. The existing inspection robots have the problem of a single structure and function, and the efficiency and safety of the inspection work are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 This is a structural schematic diagram of a drone and robot dog air-to-ground collaborative autonomous inspection system in one embodiment of the present application;

[0030] Figure 2A flowchart of a method for collaborative autonomous air-to-ground inspection by a drone and a robot dog is provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0032] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0033] In an exemplary embodiment, Figure 1 As shown, a UAV and robot dog air-to-ground collaborative autonomous inspection system is provided, including a UAV, a robot dog and a ground station.

[0034] The drone, the robot dog and the ground station are connected via wireless network communication to exchange data. Specifically, the ground station, the robot dog wireless network card and the drone data transmission module are configured so that the three are connected to the same Wi-Fi wireless network, and the IP addresses are configured so that the ground station, the robot dog and the drone can communicate with each other.

[0035] Among them, the drone includes an aerial sensor, an aerial controller and an aerial drive.

[0036] Aerial sensors are responsible for collecting external data information. Aerial sensors include airborne camera modules and optical flow modules. The airborne camera module is used to obtain environmental image information for ground personnel to view and computer image visual recognition. The optical flow module is used to detect the speed data of the drone in the x-axis and y-axis directions, helping the drone to locate and hover indoors or in environments with poor GPS signals.

[0037] The onboard camera module is used to collect the first image information of the inspection area; the air controller has flight control and other chips, and is used to control the flight status of the drone; the air drive is used to provide driving force for the drone, and the air drive is composed of components such as motors and batteries.

[0038] The robot dog includes ground sensors, ground controllers, ground drivers and take-off and landing platforms. The ground sensors, including but not limited to ground camera modules and surrounding information acquisition modules, are responsible for collecting environmental data information and locating the robot dog's own position.

[0039] The ground camera module is used to collect the second image information of the inspection area; the surrounding information collection module includes but is not limited to laser radar (or four-eye fisheye lens) and infrared sensor, etc., and the surrounding information collection module is used to collect environmental information of the inspection area and the position information of the robot dog. The infrared sensor is used to detect the distance between the robot dog and obstacles.

[0040] The ground controller includes a central processing module; the central processing module is responsible for processing the data collected by the ground sensors and making decisions, specifically for: using the path planning algorithm, according to the environmental information of the inspection area, the location information of the robot dog and the target detection point information, to plan the path and obtain the decision path of the robot dog. The detection points that need to be patrolled are set on the ground station, and the robot dog can automatically plan the path to avoid obstacles and reach the specified location (detection point). In some embodiments, the path planning algorithm can adopt algorithms such as the A* algorithm or the R* algorithm.

[0041] The ground drive, which is composed of components such as a motor, is used to provide power for the robot dog and the take-off and landing platform. Specifically, the ground drive provides movement power for the robot dog and power support for the take-off and landing platform.

[0042] The landing platform is used to load and place the drone, and fix and lock the drone when the robot dog moves, so that the drone is in a locked state. The landing platform also includes a charging module; the charging module is used to provide power for the drone.

[0043] The ground station is used to: issue a detection start instruction; store the first image information and the second image information. The ground station is composed of a personnel-operated computer and related antenna equipment. The ground station can be a computer.

[0044] The central processing module or the ground station is used to send a detection start instruction to the drone when there is an area in the inspection area that the robot dog cannot reach.

[0045] The central processing module or the ground station is also used to: use an image processing algorithm to perform image recognition on the first image information and the second image information to obtain industrial data text information corresponding to the first image information and industrial data text information corresponding to the second image information.

[0046] When the robot dog and the drone conduct collaborative inspections in the air and on the ground, first configure the communication connection between the various components of the system, then control the robot dog to map the working environment, set up patrol monitoring points for collaborative inspections by the drone and the robot dog, and finally, read and detect the inspection point data through the ground station or central processing module.

[0047] The central processing module is also used to control the robot dog to move in the inspection area, and to create an environmental map of the inspection area using real-time positioning and map building technology. The specific process of creating the environmental map is as follows:

[0048] Run the mapping command on the central processing module of the robot dog, and then control the robot dog to move in the inspection area. When the central processing module issues a mapping command, it controls the robot dog to move in the inspection area, and uses sensors such as laser radar or four-eye fisheye lens to scan the inspection area to obtain the point cloud data of the inspection area. Use the Simultaneous Localization and Mapping (SLAM) technology to create an environmental map of the inspection area based on the point cloud data of the inspection area. The distance between the robot dog and the obstacle and the environmental map of the inspection area constitute the environmental information of the inspection area. The drone and robot dog air-to-ground collaborative autonomous inspection system can achieve centimeter-level self-positioning accuracy, a mapping area of ​​more than 3,000 square meters, and a mapping speed of more than 100 square meters / min. It can meet the requirements of high positioning accuracy, large mapping area, and fast mapping speed in actual inspection work. The system has the characteristics of rapid deployment and high adaptability. The central processing module can be an airborne computer.

[0049] In another example of the present application, the drone also includes an onboard computer and a PID compensation module. The onboard computer is used to use a path planning algorithm to perform path planning based on the environmental information of the inspection area, the initial position information of the drone, and the position information of the target detection point to obtain the decision path of the drone. The decision path of the drone includes several path planning points. In addition, when the drone is traveling based on the decision path of the drone, each time the drone reaches a path planning point, it also obtains the real-time positioning data of the drone by using the self-positioning algorithm of the four-eye fisheye camera or laser radar hardware, and calculates the current distance error based on the real-time positioning data of the drone and the distance of the current path planning point. If the current distance error is greater than the set error, the onboard computer controls the PID compensation module to compensate the attitude and displacement of the drone based on the current distance error, so that the drone moves to the current path planning point and fine-tunes the position so that it can reach the target detection point more accurately.

[0050] A detection start command program is issued on the ground station, and the robot dog turns on the onboard camera and other detection equipment to read the instrument panel data, recognize the QR code, and obtain the second image information of the inspection area. For areas such as high areas that the robot dog cannot recognize, the ground station or the central processing module of the robot dog can issue a detection start command to command the drone to take off. At this time, the take-off and landing platform is opened to release the locked state of the drone. After the drone takes off, the robot dog or the ground station sends the position coordinates of the target detection point to the drone, and the drone flies autonomously to the vicinity of the target detection point. That is, the onboard computer on the drone plans a decision path for the drone according to the position coordinates of the target detection point, the environmental information of the inspection area, and the initial position information of the drone. According to the decision path of the drone, it moves to the target detection point, automatically identifies the target detection point through the onboard camera module to approach the target, and can fine-tune the position and achieve obstacle avoidance through the autonomous algorithm. Specifically: the onboard camera module is a four-eye fisheye lens, and the real-time point cloud data of the target point or obstacle in the current environment is obtained through the four-eye fisheye lens. The position is fine-tuned according to the real-time point cloud data through the PID compensation module to achieve obstacle avoidance.

[0051] After arriving at the target detection point, the onboard camera module on the drone monitors the data and obtains the first image information of the inspection area, and returns the first image information to the ground station through the image transmission module. After completing the task, the drone can land back to the take-off and landing platform to save power, and can be charged through the wireless charger installed on the take-off and landing platform to increase the flight time of the drone.

[0052] The central processing modules of the ground station and the robot dog can both process data, convert the first image information or the second image information to be detected and read into industrial data text information, and transmit the converted industrial data text information back to the ground station and store it in the database, so as to achieve the purpose of real-time monitoring of industrial data information.

[0053] The collaborative working mode of the robot dog and the drone can complement each other. The drone equipped with an onboard camera can reach the space that the robot dog cannot enter for detection and monitoring. The robot dog can carry more loads such as laser radar, high-computing motherboard, onboard camera, large-capacity battery, etc. to achieve autonomous positioning, navigation and obstacle avoidance, image recognition and other functions. The introduction of the collaborative working mode can avoid the limitations of a single device and reduce the overall cost of the system. At the same time, in terms of structure, the take-off and landing platform on the robot dog can provide support for the drone.

[0054] This application realizes the collaborative operation of drones and robot dogs. The robot dog takes advantage of its long endurance, high load characteristics, and the multi-degree-of-freedom flexibility of its mechanical legs to achieve long-distance transportation of drones and ground observations. The drone provides the flexibility advantage of high-altitude observations. At the same time, SLAM technology and path planning algorithms are applied to achieve autonomous positioning, navigation, and obstacle avoidance of the robot without external positioning, so as to adapt to the needs of complex actual industrial scenarios, improve the efficiency and safety of inspection work, and have broad engineering application value.

[0055] Compared with existing single-robot inspections, institutional collaborative inspections utilize the complementary advantages of air and space collaboration.

[0056] Compared with ground wheeled robots, the collaborative system can send drones to inspect high-altitude targets that ground robots cannot reach. The collaborative inspection of drones and robot dogs can effectively expand the inspection range.

[0057] Compared with aerial robots, drones generally only have an actual battery life of about 10 minutes. As a ground robot, the robot dog has a large payload and a much longer battery life, with the longest battery life reaching several hours. When the drone does not need to work, it can carry the drone and provide charging or battery replacement functions, thereby improving the overall working endurance of the system.

[0058] The coordinated inspection of drones and robot dogs can also reduce the overall cost of the system. If a single robot wants to achieve a large range of activities and a long battery life, it needs to install high-performance motors, high-strength materials, and large-capacity batteries. However, the coordination of mechanisms only requires the ground robot and the aerial robot to work together to complete the tasks they are best at. It does not require extremely advanced individual performance, thereby reducing the overall hardware cost and achieving the effect of one plus one being greater than two.

[0059] Adding SLAM technology to inspection work can achieve self-positioning without external assistance. It can play a good role in relatively fixed and closed indoor work scenarios such as factories. Compared with motion capture technology that can only be used in laboratory environments and UWB positioning technology that has poor positioning accuracy and is affected by external interference, SLAM technology can be better applied to actual engineering projects.

[0060] Adding visual recognition technology to unmanned inspection work can enable the computer to automatically read the information in the images sent back by the onboard camera. Compared with manual image reading or manual field inspection, adding visual recognition technology to unmanned inspection work can better improve inspection efficiency and assist inspection personnel to quickly complete a large amount of tedious data reading, proofreading and inspection work.

[0061] Based on the same inventive concept, the embodiment of the present application also provides a method for realizing the above-mentioned drone and robot dog air-ground collaborative autonomous inspection system. The implementation scheme for solving the problem provided by the method is similar to the implementation scheme recorded in the above-mentioned system, so the specific limitations in the one or more drone and robot dog air-ground collaborative autonomous inspection method embodiments provided below can refer to the above-mentioned limitations on the drone and robot dog air-ground collaborative autonomous inspection system, and will not be repeated here.

[0062] In an exemplary embodiment, Figure 2 As shown, a method for autonomous inspection of air-ground coordinated by a drone and a robot dog is provided, including:

[0063] S1: Send a detection start command through the ground station;

[0064] S2: After receiving the detection start command, the robot dog collects the environmental information of the inspection area and the location information of the robot dog;

[0065] S3: Through the central processing module, path planning is performed according to the environmental information of the inspection area, the location information of the robot dog and the target detection point information to obtain the decision path of the robot dog;

[0066] S4: Control the robot dog to reach the detection point based on the decision path, and collect second image information of the inspection area through the ground camera module;

[0067] S5: When there is an area in the inspection area that the robot dog cannot reach, a detection start instruction is sent to the drone through the central processing module or the ground station to control the drone to reach the inspection point, and the first image information of the inspection area is collected through the airborne camera module;

[0068] S6: Perform image recognition on the first image information and the second image information to obtain industrial data text information corresponding to the first image information and industrial data text information corresponding to the second image information.

[0069] The method for autonomous air-ground inspection by the UAV and robot dog also includes: controlling the robot dog to move in the inspection area, and creating an environmental map of the inspection area using real-time positioning and map building technology.

[0070] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A drone and robot dog air-ground collaborative autonomous inspection system, characterized in that: The drone and robot dog air-to-ground collaborative autonomous inspection system comprises a drone, a robot dog and a ground station; Among them, the drone includes an aerial sensor, an aerial controller, and an aerial actuator; The aerial sensor includes an airborne camera module; the airborne camera module is used to collect first image information of the inspection area; the aerial controller is used to control the flight state of the UAV; the aerial driver is used to provide driving force for the UAV; The robot dog includes a ground sensor, a ground controller, a ground driver and a take-off and landing platform; the ground sensor includes a ground camera module and a surrounding information acquisition module; the ground camera module is used to acquire second image information of the inspection area; the surrounding information acquisition module is used to acquire environmental information of the inspection area and the position information of the robot dog; the ground controller includes a central processing module; the central processing module is used to perform path planning according to the environmental information of the inspection area, the position information of the robot dog and the target detection point information, and obtain the decision path of the robot dog; the ground driver is used to provide power for the robot dog and the take-off and landing platform; the take-off and landing platform is used to load and place the drone; The ground station is used to: issue a detection start instruction; and store the first image information and the second image information.

2. The drone and robot dog air-to-ground collaborative autonomous inspection system according to claim 1 is characterized in that: The central processing module is also used to control the robot dog to move in the inspection area and create an environmental map of the inspection area using real-time positioning and map building technology.

3. The drone and robot dog air-to-ground collaborative autonomous inspection system according to claim 2 is characterized in that: The surrounding information collection module includes an infrared sensor; the infrared sensor is used to detect the distance between the robot dog and an obstacle; the distance between the robot dog and the obstacle and the environmental map of the inspection area constitute the environmental information of the inspection area.

4. The drone and robot dog air-to-ground collaborative autonomous inspection system according to claim 1 is characterized in that: The central processing module or the ground station is used to perform image recognition on the first image information and the second image information to obtain industrial data text information corresponding to the first image information and industrial data text information corresponding to the second image information.

5. The drone and robot dog air-to-ground collaborative autonomous inspection system according to claim 1 is characterized in that: The central processing module or the ground station is used to send a detection start instruction to the drone when there is an area in the inspection area that the robot dog cannot reach.

6. The drone and robot dog air-to-ground collaborative autonomous inspection system according to claim 1 is characterized in that: The aerial sensor also includes an optical flow module; the optical flow module is used to detect the speed data of the drone in the x-axis direction and the y-axis direction.

7. The drone and robot dog air-to-ground collaborative autonomous inspection system according to claim 1 is characterized in that: The drone, the robot dog and the ground station are connected via a wireless network communication.

8. The drone and robot dog air-to-ground collaborative autonomous inspection system according to claim 1 is characterized in that: The take-off and landing platform also includes a charging module; the charging module is used to provide power for the drone.

9. A method for drone and robot dog air-ground collaborative autonomous inspection based on the drone and robot dog air-ground collaborative autonomous inspection system according to any one of claims 1 to 8, characterized in that: The method for autonomous air-ground inspection coordinated by a drone and a robot dog includes: Send a detection start command through the ground station; After receiving the detection start command, the robot dog collects environmental information of the inspection area and the location information of the robot dog; Through the central processing module, the path planning is carried out according to the environmental information of the inspection area, the location information of the robot dog and the target detection point information to obtain the decision path of the robot dog; Based on the decision path, the robot dog is controlled to reach the detection point, and the second image information of the inspection area is collected through the ground camera module; When there is an area in the inspection area that the robot dog cannot reach, a detection start instruction is sent to the drone through the central processing module or the ground station to control the drone to reach the inspection point and collect the first image information of the inspection area through the airborne camera module; Image recognition is performed on the first image information and the second image information to obtain industrial data text information corresponding to the first image information and industrial data text information corresponding to the second image information.

10. The method for autonomous air-ground inspection using a drone and a robot dog according to claim 9, characterized in that: The method for autonomous air-ground inspection coordinated by the drone and the robot dog also includes: Control the robot dog to move in the inspection area, and use real-time positioning and map building technology to create an environmental map of the inspection area.

Images