Air-ground collaborative intelligent inspection robot and inspection method

Abstract

The present invention relates to an air-ground collaborative intelligent inspection robot, including a robot platform and an unmanned aerial vehicle. The robot platform includes a body, wheels and a drive component, a mechanical arm, an environmental perception component, a communicator, a robot controller and a power component. The communicator realizes communication connection between the unmanned aerial vehicle and a base station. The inspection method includes: air-ground collaborative multi-robot positioning and mapping: including perception positioning calculation, map creation, multi-information fusion positioning, air-ground collaborative tracking and control: including unmanned aerial vehicle flight control design, robot platform trajectory tracking control, and unmanned aerial vehicle self-service landing control. The present invention ensures that the robot can perform given navigation and inspection tasks in an all-round and all-weather manner, uses the Internet of Things, artificial intelligence, and other technologies, integrates environmental perception, dynamic decision-making, behavior control and alarm devices, and has the capabilities of autonomous perception, walking, protection, interactive communication, etc., and can complete basic, repetitive, and dangerous security work, reducing security operation costs.

Description

Technical Field

[0001] The present invention belongs to the technical field of robots, and in particular relates to an air-ground collaborative intelligent inspection robot and an inspection method. Background Art

[0002] In recent years, in order to reduce the incidence of safety accidents in the chemical industry, inspection work has become one of the indispensable tasks in the chemical industry. At present, maintenance and inspection work are mainly completed manually. Due to the uneven quality of personnel and other issues, it is difficult to ensure the reliability and accuracy of inspection work. In addition, the chemical industry usually occupies a large area and contains a large number of pressure vessels and nearly 100 kilometers of pressure transmission pipelines. The working environment of inspection work is very complex, and it is difficult to complete all inspection tasks by manual labor alone.

[0003] With the advancement of intelligent operation and maintenance, the demand for robot inspection is growing. The chemical industry has gradually used robots to replace manual inspection tasks, which can not only significantly reduce labor costs, but also ensure the efficiency and reliability of inspections. Although there are many advantages to applying robots to safety inspections in the chemical industry, the current inspection robots are generally single-robot type inspection equipment with low intelligence, including unmanned vehicles and drones.

[0004] At present, inspection robots include unmanned vehicles and drones, which have the following main shortcomings:

[0005] 1. Lack of multi-robot collaboration technology for unmanned vehicles and drones:

[0006] A single unmanned vehicle inspection device is limited by its own working principle and has too high requirements for the smoothness of the road surface. It cannot perform inspections on rough roads, stairs, and high-altitude environments. A single drone inspection device is limited by its cruising capability, and its flight time and load capacity are very limited. It cannot carry multiple sensors and cannot fly long distances.

[0007] 2. Lack of autonomous mobile platforms that can carry drones:

[0008] At present, when inspection drones are performing tasks, operators still need to take them to designated locations by means of transportation. This inspection method still has shortcomings such as low efficiency and waste of manpower and material resources.

[0009] 3. Lack of autonomy:

[0010] At present, when drones perform inspection tasks, operators still need to remotely control the drones on site to collect close-range images of chemical environments and equipment. This inspection method lacks autonomy and has low inspection efficiency.

[0011] 4. Lack of explosion-proof performance:

[0012] At this stage, unmanned vehicles and drones usually do not have explosion-proof properties. High-speed brushless motors and high-energy batteries will generate extremely high currents, which will explode once they come into contact with flammable gases. The potential hazards of fire and explosion are already unimaginable and their consequences are unimaginable.

[0013] 5. Lack of high-level intelligent environmental perception and understanding:

[0014] At this stage, drones have not yet fully realized the intelligent acquisition and recognition of environmental images. When encountering an emergency and initiating the one-key return operation, they are prone to collision or entanglement with obstacles on the return path, which will pose a huge hidden danger to the safety of chemical raw materials, chemical products and inspection drones.

[0015] 6. Lack of synchronous mapping and positioning in dynamic environments:

[0016] At present, most of the robot's SLAM navigation technology is only for indoor environments and can only be tested and operated in relatively controllable environments. Outdoor patrol robots usually operate in more complex environments, which requires extremely high positioning and navigation technology for mobile robots, especially the land-air collaborative system equipped with drones. Due to the changes and uncertainties of the outdoor environment, there is no mature solution for the SLAM technology of outdoor robots.

[0017] 7. Lack of high-speed and high-precision path planning technology:

[0018] The difficulty of path optimization lies in ensuring the search speed while ensuring that the path is as optimal as possible. The current path planning technology usually does not integrate the robot's kinematic constraints into the global path planning, and cannot ensure real-time performance while the robot is actually operating; one of the difficulties in air-ground collaborative path planning is to ensure mutual coordination and constraints on the premise that each path is feasible; the technical difficulty of local path planning is to track the global path as much as possible while completing the robot's obstacle avoidance based on real-time sensor data.

[0019] 8. Lack of accident warning capabilities:

[0020] At present, accident warning technology still has shortcomings. In the algorithms of system fault diagnosis and accident prediction, one of the difficulties is how to predict accidents by integrating heterogeneous data based on discrete event system theory. Summary of the invention

[0021] An object of the present invention is to provide an air-ground collaborative intelligent inspection robot for use in safety inspections in the chemical industry.

[0022] In order to achieve the above object, the technical solution adopted by the present invention is:

[0023] An air-ground collaborative intelligent inspection robot comprises a robot platform and an unmanned aerial vehicle. The robot platform comprises a vehicle body, wheels and a driving component arranged at the bottom of the vehicle body, and a mechanical arm, an environmental sensing component, a communicator, a robot controller and a power supply component arranged on the vehicle body. The communicator realizes communication connection between the unmanned aerial vehicle and a base station.

[0024] Preferably, the robotic arm includes a base arranged on the vehicle body, a joint assembly rotatably connected to the base, a mechanical claw rotatably connected to the joint assembly, and a rotating drive member for driving each component to rotate, and the joint assembly includes one or more connecting joints rotatably connected end to end.

[0025] Further preferably, the joint assembly includes a first connecting joint rotatably connected to the base, a second connecting joint rotatably connected to the first connecting joint, a third connecting joint rotatably connected to the second connecting joint, a fourth connecting joint rotatably connected to the third connecting joint, and a fifth connecting joint rotatably connected to the fourth connecting joint, and the mechanical claw is rotatably connected to the fifth connecting joint.

[0026] Further preferably, the mechanical claw includes a claw body rotatably connected to the joint assembly, a pair of claw hands connected to the claw body, and a claw hand driving assembly that drives the claw hands to perform a grasping action, and the claw hand driving assembly includes a worm gear arranged on the claw body, a turbine connected to one end of the claw hand and cooperating with the worm gear, and a claw hand driving member that drives the worm gear to rotate.

[0027] Preferably, the robot platform further comprises a platform body arranged on the vehicle body for the UAV to take off and land and to charge, and the platform body is connected to the power supply assembly.

[0028] Preferably, the environmental perception component includes a laser radar, a sensor component, and a video and photographic component.

[0029] Further preferably, the sensor assembly includes a gas concentration sensor and a humidity and temperature sensor.

[0030] Further preferably, the video and photographic components include a visible light high-definition camera, an infrared camera, and a monocular camera.

[0031] Preferably, the robot platform further comprises a touch screen display arranged on the vehicle body.

[0032] Further preferably, the touch screen display is integrated with a microphone and a speaker.

[0033] Preferably, the communicator is integrated with an inertial navigation device and a GPS device.

[0034] Preferably, the UAV comprises a fuselage, a UAV control component, and components arranged on the fuselage: a landing gear, a propeller group, a UAV drive and power supply component, and a camera and sensor component.

[0035] Further preferably, the UAV control component includes a flight control module, a data transmission and image transmission module, the flight control module includes a flight control sealed box arranged on the fuselage, a flight controller arranged in the flight control sealed box, a power manager and a parameter adjustment interface, and the flight controller is respectively connected to the power manager and the parameter adjustment interface; the data transmission and image transmission module includes a data transmission and image transmission sealed box arranged on the fuselage, a data transmission and image transmission UAV end arranged in the data transmission and image transmission sealed box, and a data transmission and image transmission ground end connected to the data transmission and image transmission UAV end, and the data transmission and image transmission UAV end is connected to the flight controller.

[0036] Further preferably, the drone drive and power supply assembly includes a power explosion-proof box, a battery arranged in the power explosion-proof box, an electronic speed regulator and a hub, a motor seat, and a motor arranged on the motor seat, the battery is connected to the power manager, the battery is connected to the electronic speed regulator through the hub, the input end of the electronic speed regulator is connected to the parameter adjustment interface, and the output end of the electronic speed regulator is connected to the motor.

[0037] Further preferably, the imaging and sensing assembly includes a camera, a sensor sealing box, a sensor control board disposed in the sensor sealing box, and a gas sensor connected to the sensor control board.

[0038] Further preferably, the propeller groups are provided in four groups, each of the propeller groups is provided with two propellers, and the two propellers are arranged up and down.

[0039] An object of the present invention is to provide an air-ground collaborative intelligent inspection method.

[0040] In order to achieve the above object, the technical solution adopted by the present invention is:

[0041] An air-ground collaborative intelligent inspection method, comprising:

[0042] 1) Air-ground collaborative multi-robot positioning and mapping: including perception positioning calculation, map creation, and multi-information fusion positioning, including:

[0043] Perception positioning computing uses sensor components to collect information about the surrounding environment, and after processing the collected data, it analyzes and processes the effective environmental perception and detection data;

[0044] Map creation includes modeling and scanning the environment, collecting data from sensor components and creating local 3D point cloud maps in their respective reference coordinate systems, initially aligning the motion trajectories of the drone and robot platform, extracting the ground plane parts of the two local maps, optimizing the alignment of the two local maps, and globally optimizing the motion trajectories and local maps obtained by the drone and robot platform;

[0045] Multi-information fusion positioning includes the fusion calculation of relative position information and GPS global coordinates, prior map and current perception data registration and other absolute position information to obtain the robot's position and posture information.

[0046] 2) Air-ground collaborative tracking and control: including UAV flight control design, robot platform trajectory tracking control, and UAV self-landing control, including:

[0047] The design of UAV flight control system includes establishing dynamic model equations for the six degrees of freedom of the UAV space, analyzing the UAV actuator model composed of the motor model and the propeller aerodynamic model, and calculating the UAV-related aerodynamic parameters using the actual measured tension and torque curves; according to the UAV kinematic and dynamic models, the model is divided into two parts: attitude and position. The motion control method is divided into position control and attitude control. A position and a UAV attitude are called a target point. The path control of the UAV is a collection of many target points in space. The UAV needs to reach the planned target points in sequence according to the order of the target points.

[0048] The robot platform trajectory tracking control uses the DDPG algorithm to control the robot platform to track the planned path based on the robot platform status information and environmental feedback information;

[0049] Autonomous landing control of the UAV includes flying to the air above the unmanned vehicle through a planned path, searching for visual signs through the visual guidance system, detecting the visual signs, and starting the automatic guided landing program to achieve autonomous landing of the UAV.

[0050] Preferably, the method also includes detection and early warning of accidents, including establishing a mapping relationship from faults to sensor events and a mapping relationship from accidents to sensor event sequences based on the actual faults and accidents that may occur in the system, and constructing a system diagnostician based on a state tree based on the established mapping relationship. The diagnostician observes system events and performs fault detection and accident prediction online in real time. When a system fault is detected, the diagnostician issues a system warning. The diagnostician calculates the probability of an accident in real time, and issues a warning when the probability exceeds a threshold set by the system.

[0051] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art:

[0052] The present invention builds an all-weather autonomous navigation multi-type robot platform to ensure that the robots can perform given navigation and inspection tasks in an all-round and all-weather manner. It comprehensively uses technologies such as the Internet of Things, artificial intelligence, cloud computing, big data, etc., integrates environmental perception, dynamic decision-making, behavior control and alarm devices, and has the capabilities of autonomous perception, autonomous walking, autonomous protection, interactive communication, etc. It is a multifunctional comprehensive intelligent equipment that can help humans complete basic, repetitive, and dangerous security work, promote the upgrade of security services, and reduce security operating costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Attached Figure 1 It is a structural schematic diagram of this embodiment;

[0054] Attached Figure 2 Schematic diagram of the structure of the robot platform in this embodiment (the platform body is hidden);

[0055] Attached Figure 3 Schematic diagram of the structure of the robotic arm in this embodiment;

[0056] Attached Figure 4 Schematic diagram of the structure of the mechanical claw in this embodiment;

[0057] Attached Figure 5 This is a schematic diagram of the structure of the UAV in this embodiment;

[0058] Attached Figure 6 is a schematic structural block diagram of this embodiment;

[0059] Attached Figure 7 This is a schematic diagram of the inspection system relationship in this embodiment;

[0060] Attached Figure 8 Workflow diagram for image perception;

[0061] Attached Fig. 9 This is a schematic block diagram of air-ground collaborative multi-robot positioning and mapping in this embodiment;

[0062] Attached Fig.10 is a schematic block diagram of perception positioning calculation in this embodiment;

[0063] Attached Fig.11 A schematic block diagram of map creation in this embodiment;

[0064] Attached Fig.12 It is a schematic diagram and block diagram of multi-information fusion positioning in this embodiment;

[0065] Attached Fig.13a ,13b The rotor dynamics model of the UAV in this embodiment;

[0066] Attached Fig.14 This is a schematic block diagram of the UAV tracking control system in this embodiment;

[0067] Attached Fig.15 This is a schematic diagram of the state machine of the drone controller in this embodiment;

[0068] Attached Fig.16 A schematic block diagram of the DDPG trajectory tracking controller design in this embodiment;

[0069] Attached Fig.17 The schematic diagram of the tracking error design in this embodiment is as follows: P is a point at a specified distance L in front of the car, q is the target point of the trajectory, and pq is perpendicular to L;

[0070] Attached Fig.18 This is a schematic diagram of the UAV / unmanned vehicle collaborative inspection process in this embodiment;

[0071] Attached Fig.19 This is a schematic diagram of extracting event information from sensor data in this embodiment;

[0072] Attached Fig. 20 This is a schematic block diagram of the accident prediction related model and architecture in this embodiment;

[0073] Attached Fig.21 This is a schematic diagram of human action behavior video recognition in this embodiment.

[0074] In the above drawings: 1. Robot platform; 10. Vehicle body; 11. Wheels; 12. Robot arm; 120. Base; 121. Robot claw; 1210. Claw body; 1211. Claw hand; 1212. Worm; 1213. Turbine; 122. First connection joint; 123. Second connection joint; 124. Third connection joint; 125. Fourth connection joint; 126. Fifth connection joint; 130. LiDAR; 131. Gas concentration sensor; 132. Humidity and temperature sensor; 133. Visible light high-definition camera; 134. Infrared camera; 135. Rotatable structure; 14. Communicator; 15. Touch screen display; 16. Robot controller; 17. Power supply assembly; 18. Platform body; 2. UAV; 20. Fuselage; 21. Landing gear; 22. Propeller; 23. Camera. DETAILED DESCRIPTION

[0075] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0076] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0077] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0078] like Figure 1 , 2 The air-ground collaborative intelligent inspection robot shown includes a robot platform 1 and an unmanned aerial vehicle 2. The robot platform 1 and the unmanned aerial vehicle 2 are described in detail below.

[0079] The robot platform 1 includes a vehicle body 10, wheels and a driving assembly arranged at the bottom of the vehicle body 10, and a mechanical arm 12, an environment sensing assembly, a communicator 14, a touch screen display 15, a robot controller 16, and a power supply assembly 17 arranged on the vehicle body 1. Among them:

[0080] The wheels and driving components are driven by four motors in cooperation with a reducer to drive the wheels 11 to rotate. The robot controller 16 is used to control the four wheels 11 separately, and steering is achieved through differential speed.

[0081] like Figure 3As shown: the mechanical arm 12 includes a base 120 arranged on the vehicle body 10, a joint assembly rotatably connected to the base 120, a mechanical claw 121 rotatably connected to the joint assembly, and a rotating drive member that drives each component to rotate. The joint assembly includes one or more connecting joints that are rotatably connected head to tail. In this embodiment: the joint assembly includes a first connecting joint 122 rotatably connected to the base 120, a second connecting joint 123 rotatably connected to the first connecting joint 122, a third connecting joint 124 rotatably connected to the second connecting joint 123, a fourth connecting joint 125 rotatably connected to the third connecting joint 124, and a fifth connecting joint 126 rotatably connected to the fourth connecting joint 125, and the mechanical claw 121 is rotatably connected to the fifth connecting joint 126. The rotating drive member adopts a motor, and 6 motors drive 6 rotating joints to realize the movement of 6 degrees of freedom in the mechanical arm 12, and drive the mechanical claw 121 to perform the grasping operation.

[0082] The robotic arm 12 can adjust the landing position of the drone 2 to facilitate wireless charging of the drone 2; the robotic arm 12 can also replace various sensors for the drone 2; the robotic arm 12 can also perform operations such as grasping, rotating, pressing, etc. in dangerous environments, for example, performing valve opening and closing operations.

[0083] like Figure 4 As shown: the mechanical claw 121 includes a claw body 1210 rotatably connected to the fifth connecting joint 126 in the joint assembly, a pair of claw hands 1211 connected to the claw body 1210, and a claw hand driving assembly that drives the claw hand 1211 to perform a grasping action. In this embodiment: the claw hand driving assembly includes a worm 1212 arranged on the claw body 1210, a turbine 1213 connected to one end of the claw hand 1210 and matched with the worm 1212, and a claw hand driving member that drives the worm 1212 to rotate. The claw hand 1211 is provided with saw teeth to help ensure the reliability of grasping. The claw hand driving member can be used, for example, to drive the mechanical claw 121 to realize the grasping function by a motor.

[0084] The environment perception component includes a laser radar 130, a sensor component, and a video and camera component. Among them:

[0085] The laser radar 130 is a radar system that emits laser beams to detect the position, speed and other characteristic quantities of the target. The robot platform 1 uses the laser radar 130 to detect the dynamic environment and collect parameters such as the target distance, direction, height, speed, posture, shape, etc. The laser radar 130 is used to detect the obstacle information around the robot platform 1. After obtaining the raw data of the laser radar 130, the data needs to be preprocessed. Since the initial laser data is relatively messy, including some abnormal data obtained by measurement errors and accidental errors, it is necessary to filter it, and low-pass filtering or Gaussian filtering can usually be used. The amount of raw data is determined by the sensor resolution and is usually large. In order to facilitate application in practice, the data needs to be downsampled. The laser radar data after data filtering and downsampling will be used in related perception functions such as obstacle recognition and object detection.

[0086] The sensor assembly includes a gas concentration sensor 131 and a humidity and temperature sensor 132. The gas concentration sensor 131 can detect the gas composition and concentration in the air to determine whether there is leakage of hazardous gases, such as the concentration of toxic gases (carbon monoxide, vinyl chloride, hydrogen sulfide, etc.) and flammable gases (hydrogen, methane, ethane, etc.). Once the ambient temperature and humidity or the concentration of harmful and flammable gases exceeds the safety threshold, it will be immediately reported to the staff for processing. The humidity and temperature sensor 132 can obtain the temperature and humidity of the actual inspection area environment.

[0087] The video and photographic components include a visible light high-definition camera 133, an infrared camera 134, and a monocular camera. The infrared camera 134 is mainly used for night patrols to capture night video images. In order to be able to visually perceive and understand the chemical production environment, the monocular camera can obtain image information of the working environment and process it. The inspection requirements of the chemical environment usually fall into the following categories: whether the staff are fully wearing protective equipment; whether the appearance of the production equipment is intact; whether the key indicator lights are normal, etc.

[0088] To address these problems, existing methods usually use template matching for recognition, but this method has poor generalization. If the matching target in the original image is rotated or resized, the effect of this method is poor. This embodiment uses a deep learning method to recognize images. First, image samples of objects in the work scene are collected and corresponding labels are set. Then, a deep neural network is built and the samples are trained. Through training, the network can recognize and classify images. Figure 8 shown.

[0089] A rotatable structure 135 is disposed on the vehicle body 10 , and a laser radar 130 , a visible light high-definition camera 133 , and an infrared camera 134 can all be disposed on the rotatable structure 135 .

[0090] The communicator 14 is a wireless communicator, which is mainly responsible for information transmission between the drone 2 and the base station to ensure smooth information transmission. The communicator 14 is integrated with an inertial navigation device and a GPS device.

[0091] The touch screen display 15 provides a human-machine interactive interface for the user to modify control parameters and collect relevant monitoring data. Through the human-machine interactive interface and the highly intelligent robot system, the system operation process is simplified to reduce labor costs. The touch screen display 15 is integrated with a microphone and a speaker, so that the robot platform 1 has the functions of sound data collection and audio playback.

[0092] The power supply assembly 17 includes an explosion-proof box, a battery and a motor arranged in the explosion-proof box. For the chemical industry, explosion-proof capability is essential. The ultra-high current of the robot during operation is prone to generate electric sparks, and once it comes into contact with flammable gas, there will be a risk of explosion. Such hidden dangers will have unimaginable consequences. During the inspection process, if there is a flammable gas leak, it is very likely to ignite the flammable gas. Explosion-proof performance is the most important technical feature of the system, and an explosion-proof box structure is used to ensure its explosion-proof performance.

[0093] In addition, the robot platform 1 also includes a platform body 18 arranged on the vehicle body 10 for the drone 1 to take off and land and charge, and the platform body 18 is connected to the power supply assembly 17. The platform body 18 provides a carrying and take-off and landing platform for the drone 2. The platform body 18 is circular, and the circular platform body 18 is more suitable for the random error of the landing position of the drone 2. At the same time, the platform body 18 also provides a wireless charging function for the drone 2. After the drone 2 completes an inspection task, it must be charged in time. Therefore, after the drone 2 returns to the take-off and landing platform, the platform body 18 uses the wireless charging function to charge it. The high-power wireless power supply technology provided by the platform body 18 does not require any physical connection. The charging operation is completed through non-radiative wireless energy transmission, which can completely free the drone 2 from manual operation when charging, greatly improving the automation level of drone inspection.

[0094] like Figure 5 , 6 As shown: the UAV 2 includes a fuselage 20, a UAV control component, and components arranged on the fuselage 20: a landing gear 21, a propeller group, a UAV drive and power supply component, and a camera and sensor component.

[0095] The drone control component includes a flight control module, a data transmission and image transmission module, the flight control module includes a flight control sealed box arranged on the fuselage 20, a flight controller arranged in the flight control sealed box, a power manager and a parameter adjustment interface, the flight controller is respectively connected to the power manager and the parameter adjustment interface, and the flight controller has a barometer, an inertial navigation system and an attitude stabilization system inside. The data transmission and image transmission module includes a data transmission and image transmission sealed box arranged on the fuselage 20, a data transmission and image transmission drone end arranged in the data transmission and image transmission sealed box, and a data transmission and image transmission ground end connected to the data transmission and image transmission drone end, and the data transmission and image transmission drone end is connected to the flight controller.

[0096] The UAV drive and power supply assembly includes an explosion-proof power box, a battery arranged in the explosion-proof power box, an electronic speed regulator and a hub, a motor seat, and a motor arranged on the motor seat. The battery is connected to the power manager, and the battery is connected to the electronic speed regulator through the hub. The input end of the electronic speed regulator is connected to the parameter adjustment interface, and the output end of the electronic speed regulator is connected to the motor.

[0097] The camera and sensor assembly includes a camera 23, a sensor sealing box, a sensor control board arranged in the sensor sealing box, and a gas sensor connected to the sensor control board.

[0098] There are four propeller groups, each of which is provided with two propellers 22, and the two propellers 22 are arranged up and down. The design of the double propeller structure can provide a stronger flying power for the UAV, so as to carry more sensor components.

[0099] In addition, a robot platform can carry multiple drones, and there is no limitation on the style of the robot platform. For example, changes such as sensor components, installation location of manipulators, increase / decrease in wheels, and similar explosion-proof designs are also within the scope of protection of this application.

[0100] The following is a detailed description of the method of air-ground collaborative intelligent inspection:

[0101] 1. Air-ground collaborative multi-robot positioning and mapping:

[0102] It mainly includes high-precision and low-latency perception and positioning calculation, air-ground collaborative multi-robot map creation in chemical environment, and multi-information fusion positioning in highly dynamic chemical environment. Specifically:

[0103] like Fig.10 As shown: The perception and positioning calculation is implemented by three parts, including the sensor unit, the clock synchronization device and the computer unit.

[0104] The sensor unit uses industrial cameras (color and grayscale), 3D lidar, inertial navigation unit, GPS and other equipment to collect information about the robot's surrounding environment; the raw data streams generated by each sensor are sent to the computer unit after clock synchronization; after the computer unit collects the sensor data, it pre-processes the data, and the processing content includes point cloud noise filtering, normal vector analysis, feature point extraction, feature description operation, etc. The computer unit is installed on the UAV and robot platforms respectively, and the effective environmental perception and chemical detection data are transmitted back to the ground workstation for comprehensive analysis and processing.

[0105] The creation of air-ground collaborative multi-robot maps in a chemical environment mainly solves the problems of map holes and missing perspectives caused by the limited perspective of a single robot, and is the fundamental guarantee for building a high-precision full-coverage environmental map. This embodiment reconstructs the geometric model of the chemical plant environment where the robot is located in three dimensions based on the perception data sent by the computer unit on the drone and robot platform.

[0106] like Fig.11 As shown: The map creation steps include:

[0107] 1. The UAV and robot platform conduct modeling and scanning of the chemical plant environment in a remote control mode. The motion trajectories of the two are in a follow-up state, collecting sensor data from their respective computer units and creating local 3D point cloud maps in their respective reference coordinate systems;

[0108] 2. Use the GPS location information on the drone and robot platform to initially align their motion trajectories;

[0109] 3. Extract the ground plane parts from the two local maps, and then optimize the map alignment of the two local maps based on the surface-to-surface closest point iterative algorithm;

[0110] 4. Based on optimization theory, the motion trajectories and local maps obtained by the UAV and robot platforms are globally optimized to obtain a more accurate chemical plant environment model.

[0111] like Fig.12 As shown in the figure: Multi-information fusion positioning introduces the extended Kalman filter framework, which effectively integrates and calculates relative position information such as inertial navigation integration, wheel odometer, laser odometer and absolute position information such as GPS global coordinates, prior map and current perception data alignment, to obtain high-precision and low-latency robot position and posture information.

[0112] 2. Air-ground collaborative tracking and control:

[0113] It mainly includes the design of UAV flight control system, robot platform trajectory tracking control, and UAV self-landing control. Specifically:

[0114] UAV flight control system design:

[0115] The dynamic model equations of the six degrees of freedom of the UAV space are established, and the UAV actuator model composed of the motor model and the propeller aerodynamic model is analyzed. The actual measured tension and torque curves are used to calculate the relevant aerodynamic parameters of the UAV. The model can calculate the tension and torque received by each actuator at this time based on the input air flow velocity and the speed and angular velocity of the four propeller groups of the UAV, and realize the simulation verification of the model, such as Fig.13a , 13b shown.

[0116] According to the kinematic and dynamic models of the drone, the model can be divided into two parts: attitude and position. Similarly, the motion control method is divided into position control and attitude control. A position and a drone attitude are called a target point. The path control of the drone is a collection of many target points in space. The drone needs to reach the planned target points in sequence, such as Fig.14 shown.

[0117] In the attitude control of the UAV, an anti-disturbance controller is used to control the altitude, yaw, pitch, and roll motion parameters of the UAV. In the position control, a position controller based on the backstepping method is designed so that the UAV can track the target trajectory. In order to enable the UAV to achieve flexible maneuverability in a complex environment, a state machine controller needs to be designed for the controller. The controller can automatically adjust the flight attitude and position of the robot according to the different environmental conditions of the UAV, such as Fig.15 shown.

[0118] Robot platform trajectory tracking control:

[0119] Reinforcement learning is used for trajectory tracking control. Reinforcement learning is a method of learning a controller without knowing the control and mechanical knowledge. Reinforcement learning emphasizes the interaction with the environment and is a dynamic learning process. The Deep Deterministic Policy Gradient Algorithm (DDPG) is a deep reinforcement learning algorithm that inherits the characteristics of the policy gradient algorithm and the actor-critic algorithm. The DDPG algorithm is used to control the wheeled robot to track the planned path based on the wheeled robot state information and the environment feedback information.

[0120] First, the driving error is calculated by the error function between the actual position of the robot and the target point on the planned trajectory, and the error information is passed to the DDPG network. The DDPG network perceives the environment through state description, makes the best decision based on the current environmental state information, and guides its own learning by setting the reward function, and finally achieves high-precision tracking of the planned path. Fig.16The error function can be designed using the lateral error of the robot. The robot obtains its current position through the perception system, and obtains the preset trajectory position through the path planning system. The distance between the assumed point p in front of the robot and the target q is calculated as the value of the error function, and the lateral error is used to continuously guide the robot to walk along the trajectory.

[0121] Before using the DDPG algorithm, the network needs to be pre-trained, and usually the simulation training of the real environment is carried out on the simulator. In the traditional DDPG algorithm, too few training samples will make the training efficiency very low, and the network cannot converge quickly. By improving the strategy of putting samples back into the training experience pool in the algorithm, when there are fewer samples, the network training is not carried out, and the robot continues to explore, filling the number of samples to achieve the purpose of accelerating training; at the same time, in order to solve the complex environment, the cost of robot exploration is too high, and a large number of trial and error processes in the early stage consume a lot of useless work. This embodiment uses the method of transfer learning, first pre-training the DDPG network in a simple environment, and then placing the trained network in a complex environment, gradually increasing the complexity of the environment, so that the network can obtain the ability to generate motion strategies in complex environments.

[0122] Autonomous landing control of drone:

[0123] The robot platform (UGV) serves as the carrier of the UAV, which can realize the automatic takeoff and automatic visual guidance landing of the UAV. In normal inspection tasks, the unmanned vehicle performs inspections according to the preset inspection route. In some scenarios, the unmanned vehicle cannot directly reach the inspection point. At this time, the drone can be used to reach these inspection points, and the drone can be used as an additional eye for the unmanned vehicle to transmit the inspection information to the robot controller of the unmanned vehicle; after completing the information collection task, the drone can automatically land on the platform body of the unmanned vehicle under the guidance of the visual mark and complete the subsequent inspection tasks. Autonomous landing of the drone requires control of the height and attitude adjustment of the drone, which requires the design of directional and certain specifications of visual marks. The drone first flies to the sky above the unmanned vehicle through the planned path of the planning system, and then searches for the visual mark through the visual guidance system. Once the visual mark is detected, the automatic guidance landing program is started to realize the autonomous landing of the drone. The controller that controls the landing process of the drone can use a fuzzy controller to increase the smoothness of the landing process, such as Fig.18 shown.

[0124] 3. Accident detection and early warning:

[0125] This embodiment will establish a discrete event system model for the operation dynamics of the chemical plant and analyze the sensor data with the relevant theories of discrete event systems. For data with different structures in the system, such as personnel actions, temperature, gas concentration, etc., a sensor data dictionary is established based on the data structure and data format, and a sensor data packet parsing method is established. The required feature information in the data is extracted by combining logical judgment, deep learning and other methods, and then a strong mapping relationship between feature information and event information is established in combination with information theory knowledge, such as Fig.19 shown.

[0126] According to the possible faults and accidents in the actual system, the mapping relationship from faults to sensor events and the mapping relationship from accidents to sensor event sequences are established, that is, the system faults are modeled as fault events in the system, and the system accidents are modeled as a combination of a series of event sequences. According to the established mapping relationship, a system diagnostics based on the state tree is constructed. The diagnostics perform fault detection and accident prediction online in real time by observing system events. When a system fault is detected, the diagnostics issue a system warning; for accident prediction, the diagnostics calculate the probability of an accident in real time, and when the probability exceeds the threshold set by the system, a warning is issued, such as Fig. 20 shown.

[0127] In view of the personnel safety issues in the production process of chemical enterprises, video surveillance technology is used to detect and track personnel and identify specific behaviors. When detecting and tracking personnel, the speed and accuracy requirements of human behavior recognition analysis in the microenvironment are comprehensively considered. A pedestrian detection and tracking algorithm combining deep features and artificial features can be used to meet the real-time requirements of the system while achieving the optimal personnel detection accuracy. At the same time, combined with the object detection algorithm based on deep learning, it can automatically identify whether the personnel in the production area are wearing safety helmets, protective goggles, and long-sleeved work clothes. When identifying and analyzing specific behaviors of personnel, a face recognition method based on a neural network is used to identify the tracking target, classify multiple tracking information and fuse them, analyze the relationship between individual behavior and motion trajectory, and the relationship between individual behavior and group behavior, establish an automaton model of individual and group behavior, and use the relevant knowledge of discrete event systems to identify behaviors, such as Fig.21 shown.

[0128] In addition, similar changes to the accident detection and early warning methods in this embodiment also fall within the scope of protection of this application.

[0129] The present invention can achieve the following beneficial effects:

[0130] 1. Collaboration between unmanned vehicles and drones:

[0131] The present invention designs a wheeled robot platform equipped with a drone, which can meet the requirements for the loading, take-off and landing of inspection drones, gets rid of the restriction that staff need to carry drones to the site, and greatly improves the autonomous control capability of the drone; the wheeled robot platform can carry drones and can carry large high-energy batteries to charge the drones when the drones are low on power; the wheeled robot platform can also carry different types of sensor modules, including visible light cameras, infrared cameras, laser radars, and satellite navigation system receivers, to make up for the lack of drone load capacity; the drone can select the required sensor modules on the platform, and use a mechanical arm to replace the sensor modules for the drone, which greatly improves the drone inspection capability and efficiency.

[0132] 2. Good explosion-proof performance:

[0133] The present invention uses an explosion-proof box isolation method to isolate the instantaneous high-voltage current from the outside world, so that the inspection robot has explosion-proof performance.

[0134] 3. Highly intelligent environmental perception and understanding:

[0135] The present invention integrates a variety of sensors to perceive and understand the chemical production environment, and proposes to train a deep neural network by collecting image samples on site to achieve the recognition and detection of people or equipment in the chemical production environment. Compared with traditional recognition methods, the detection results are more reliable and the model generalization is better.

[0136] 4. Synchronous mapping and positioning of dynamic environments:

[0137] In view of the complex and changeable chemical environment, the present invention introduces the air-ground collaborative multi-robot SLAM method, breaks through the full coverage environment modeling technology of large-scale chemical plant areas, overcomes the multi-sensor fusion positioning technology, and realizes the high-precision modeling and safe and reliable positioning of the inspection robot in a mixed chemical environment.

[0138] 5. High-speed and high-precision path planning technology:

[0139] The present invention applies a hybrid algorithm to the solution of path optimization, making the algorithm more efficient, and integrates kinematic constraints into global planning, making the trajectory more reasonable and easy to track. It can complete air-ground collaborative path planning and coordinate movement with each other while ensuring the feasibility of each path.

[0140] 6. Accident warning capability:

[0141] Aiming at the accidents that may occur in the production process of chemical enterprises, the present invention designs a set of heterogeneous data fusion accident prediction algorithms based on discrete event system theory, which can stably operate on the air-ground collaborative intelligent inspection robot platform, enable the inspection robot to complete the work of safety inspection in the chemical operation environment, and supervise the safety issues of accidents in the production process of chemical enterprises, greatly reduce safety accidents, and protect the production property and personnel life safety of chemical enterprises.

[0142] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. An air-ground collaborative intelligent inspection method, Features: include: 1) Air-ground collaborative multi-robot positioning and mapping: including perception positioning calculation, map creation, and multi-information fusion positioning, including: The perception and positioning calculation is realized by three parts, including the sensor unit, the clock synchronization device and the computer unit. The sensor unit uses industrial cameras, three-dimensional laser radars, inertial navigation units and GPS devices to collect information about the robot's surrounding environment. The raw data streams generated by each sensor are sent to the computer unit after clock synchronization. After the computer unit collects the sensor data, it pre-processes the data. The processing content includes point cloud noise filtering, normal vector analysis, feature point extraction, and feature description calculation. The computer unit is installed on the UAV and robot platform respectively. The effective environmental perception and chemical detection data are sent back to the ground workstation for comprehensive analysis and processing. Map creation includes modeling and scanning the chemical plant environment by the UAV and robot platform in a remote control mode. The motion trajectories of the two are in a follow-up state, and the sensor data of their respective computer units are collected to create local three-dimensional point cloud maps in their respective reference coordinate systems. The motion trajectories of the two are initially aligned using the GPS position information on the UAV and robot platforms, and the ground plane parts in the two local maps are extracted. Then, the map alignment of the two local maps is optimized based on the surface-to-surface closest point iteration algorithm. Based on optimization theory, the motion trajectories and local maps obtained by the UAV and robot platforms are globally optimized and adjusted to obtain an accurate chemical plant environment model. Multi-information fusion positioning introduces the extended Kalman filter framework to fuse the relative position information of inertial navigation integration, wheel odometer, laser odometer and GPS global coordinates, prior map and current perception data to align the absolute position information, and obtain the robot's position and posture information. 2) Air-ground collaborative tracking and control: including UAV flight control design, robot platform trajectory tracking control, and UAV self-landing control, including: The design of UAV flight control system includes establishing dynamic model equations for the six degrees of freedom of the UAV space, analyzing the UAV actuator model composed of the motor model and the propeller aerodynamic model, and using the actual measured tension and torque curves to calculate the UAV-related aerodynamic parameters; according to the kinematic and dynamic models of the UAV, the model is divided into two parts: attitude and position. The motion control method is divided into position control and attitude control. A position and a UAV attitude are called a target point. The path control of the UAV is a collection of many target points in space. The UAV needs to arrive at the planned target points in sequence according to the order of the target points. In the attitude control of the UAV, the self-disturbance rejection controller is used to realize the control of the UAV's altitude, yaw, pitch, and roll motion parameters. In the position control, a position controller based on the backstepping method is designed so that the UAV can complete the tracking of the target trajectory; The robot platform trajectory tracking control uses the DDPG algorithm to control the robot platform to track the planned path according to the robot platform state information and the information fed back by the environment. First, the driving error is calculated by the error function from the actual position of the robot and the target point on the planned trajectory, and the error information is passed to the DDPG network. The DDPG network perceives the environment through state description, makes the best decision based on the current environmental state information, and guides its own learning by setting the reward function, and finally achieves high-precision tracking of the planned path. The error function is designed using the lateral error of the robot. The robot obtains the current position through the perception system, and obtains the preset trajectory position through the path planning system. The distance between the assumed point p in front of the robot and the target q is calculated as the value of the error function, and the lateral error is used to continuously guide the robot to walk along the trajectory. Autonomous landing control of the drone includes flying to the sky above the robot platform through a planned path, searching for visual markers through the visual guidance system, detecting the visual markers, starting the automatic guided landing program, and realizing autonomous landing of the drone. In normal inspection tasks, the robot platform performs inspections according to preset inspection routes. In some scenarios, the robot platform cannot reach the inspection points directly. At this time, drones can be used to reach these inspection points and use the drones as additional eyes of the robot platform to transmit inspection information to the robot controller of the robot platform. After completing the information collection task, the drone can automatically land on the platform body of the robot platform under the guidance of visual markers and complete subsequent inspection tasks.

2. The air-ground collaborative intelligent inspection method according to claim 1, Features: The method also includes detection and early warning of accidents, including establishing a mapping relationship from faults to sensor events and a mapping relationship from accidents to sensor event sequences based on the actual faults and accidents that may occur in the system, and building a system diagnostic device based on a state tree based on the established mapping relationship. The diagnostic device observes system events and performs fault detection and accident prediction online in real time. When a system fault is detected, the diagnostic device issues a system warning; the diagnostic device calculates the probability of an accident in real time, and issues a warning when the probability exceeds a threshold set by the system.

3. A robot for implementing the air-ground collaborative intelligent inspection method as described in any one of claims 1 to 2, Features: The invention comprises a robot platform and an unmanned aerial vehicle. The robot platform comprises a vehicle body, wheels and a driving assembly arranged at the bottom of the vehicle body, and a mechanical arm, an environmental sensing assembly, a communicator, a robot controller and a power supply assembly arranged on the vehicle body. The communicator realizes communication connection between the unmanned aerial vehicle and a base station. The robot platform also comprises a platform body arranged on the vehicle body for taking off and landing and charging the unmanned aerial vehicle. The platform body is connected to the power supply assembly. The mechanical arm comprises a base arranged on the vehicle body, a joint assembly rotatably connected to the base, a mechanical claw rotatably connected to the joint assembly, and a rotating driving member driving each component to rotate. The joint assembly comprises one or more connecting joints rotatably connected head to tail in sequence. The mechanical claw comprises a claw body rotatably connected to the joint assembly, a pair of claw hands connected to the claw body, and a claw hand driving assembly driving the claw hand to perform a grasping action. The claw hand driving assembly comprises a worm arranged on the claw body, a worm wheel connected to one end of the claw hand and cooperating with the worm, and a claw hand driving member driving the worm to rotate.

4. The robot according to claim 3, Features: The environmental perception components include laser radar, sensor components, and video and photography components.

5. The robot according to claim 3, Features: The UAV includes a fuselage, a UAV control component, and a landing gear, a propeller group, a UAV drive and power supply component, and a camera and sensor component arranged on the fuselage. The propeller group is provided with four groups, each of which is provided with two propellers, and the two propellers are arranged up and down.

6. The robot according to claim 5, Features: The UAV control component includes a flight control module and a data transmission and image transmission module. The flight control module includes a flight control sealed box arranged on the fuselage, a flight controller arranged in the flight control sealed box, a power manager and a parameter adjustment interface, and the flight controller is respectively connected to the power manager and the parameter adjustment interface; the data transmission and image transmission module includes a data transmission and image transmission sealed box arranged on the fuselage, a data transmission and image transmission UAV end arranged in the data transmission and image transmission sealed box, and a data transmission and image transmission ground end connected to the data transmission and image transmission UAV end, and the data transmission and image transmission UAV end is connected to the flight controller.

7. The robot according to claim 6, Features: The UAV drive and power supply assembly includes a power explosion-proof box, a battery arranged in the power explosion-proof box, an electronic speed regulator and a hub, a motor seat, and a motor arranged on the motor seat. The battery is connected to the power manager, and the battery is connected to the electronic speed regulator through the hub. The input end of the electronic speed regulator is connected to the parameter adjustment interface, and the output end of the electronic speed regulator is connected to the motor.

8. The robot according to claim 5, Features: The camera and sensor assembly comprises a camera, a sensor sealing box, a sensor control board arranged in the sensor sealing box, and a gas sensor connected to the sensor control board.

Images