Fusion obstacle avoidance system for safety inspection robot

Abstract

The invention discloses a fusion obstacle avoidance system for a safety inspection robot, which obtains real-time mobile data in sensors arranged around the safety inspection robot, receives the real-time mobile data from each sensor, carries out fusion processing on the real-time mobile data by using a data fusion algorithm, and carries out real-time obstacle avoidance on the basis of mobile environment information. Constructing a mobile map by using an ORB-SLAM2 algorithm, and determining real-time position information of the robot in the map; the real-time position information is received, a Dijkstra path planning algorithm is used for carrying out inspection path planning on the real-time position information, surrounding moving obstacle information is monitored in real time, when an obstacle is detected, an initial planning path and the real-time position information are combined to calculate an obstacle avoidance path, and the accuracy of obstacle detection and recognition is improved. Stable and reliable positioning information can be provided in different environments, and it is ensured that the robot accurately knows the position of the robot.

Description

Technical Field

[0001] The invention relates to the technical field of robot obstacle avoidance, and in particular to a fusion obstacle avoidance system for a safety inspection robot. Background Art

[0002] The intelligent robot inspection system is based on the inspection robot, combined with the real-time monitoring background, data acquisition server and related accessories, which can realize uninterrupted monitoring of the environment and equipment and emergency disposal. Safety inspection robots are increasingly used in various fields, such as industrial plants, warehouses, public places, etc. However, in complex environments, robots need to accurately perceive the surrounding environment, locate and navigate, and effectively avoid obstacles to ensure their own safe operation and complete inspection tasks. Traditional single sensor systems often have limitations when facing complex environments. For example, visual sensors are easily affected by light, and laser radars have poor reflection effects on certain specific material surfaces. Therefore, how safety inspection robots can avoid obstacles and perform efficient operations is a technical problem that needs to be solved at this stage. Summary of the invention

[0003] The purpose of the present invention is to solve the above problems and to design a fusion obstacle avoidance system for a safety inspection robot.

[0004] To achieve the above-mentioned purpose, the technical solution of the present invention is that, further, in the above-mentioned fusion obstacle avoidance system for a safety inspection robot, the fusion obstacle avoidance system includes a sensor module, a data fusion module, a positioning mapping module, a navigation module, an obstacle avoidance module, and a communication module:

[0005] The sensor module is used to obtain real-time movement data from sensors set around the safety inspection robot, and the sensors include at least SLAM visual sensor, laser radar, IMU inertial measurement unit, Beidou, odometer, and gyroscope;

[0006] A data fusion module is used to receive real-time mobile data from various sensors, and use a data fusion algorithm to fuse the real-time mobile data to obtain mobile environment information;

[0007] A positioning and mapping module is used to construct a mobile map based on the mobile environment information using the ORB-SLAM2 algorithm and determine the real-time position information of the robot in the map;

[0008] A navigation module is used to receive real-time location information, and use the Dijkstra path planning algorithm to perform inspection path planning on the real-time location information to obtain an initial planned path;

[0009] The obstacle avoidance module is used to monitor the obstacle information in the surrounding area in real time. When an obstacle is detected, the obstacle avoidance path is calculated by combining the initial planned path and the real-time position information to obtain the target planned path;

[0010] The communication module is used to transmit the target planning path to the remote control center and obtain real-time feedback data, and adjust the target planning path according to the real-time feedback data.

[0011] Furthermore, in the above-mentioned fusion obstacle avoidance system for a safety inspection robot, the sensor module includes a SLAM visual sensor, a laser radar, an IMU inertial measurement unit, a Beidou, an odometer, a gyroscope, an infrared thermal imaging instrument, and an environment and gas detection sensor:

[0012] The SLAM visual sensor is used to obtain real-time image data around the security inspection robot and extract feature points in the real-time image data, such as corner points and edge points;

[0013] LiDAR is used to scan obstacles around the safety inspection robot, measure the distance between the robot and obstacles, and construct the contour information of surrounding objects;

[0014] IMU inertial measurement unit, used to measure the acceleration of the robot on three coordinate axes and calculate the robot's velocity and displacement;

[0015] Beidou, used to obtain the robot's absolute position coordinates on the earth, including longitude, latitude and altitude;

[0016] Odometer, used to measure the number of rotations and angles of the robot's wheels, and calculate the robot's travel distance and cumulative travel distance;

[0017] A gyroscope is used to measure the rotational angular velocity of the robot around three coordinate axes, and complements the angular velocity data in the IMU inertial measurement unit;

[0018] Infrared thermal imaging instrument, which is used to collect temperature data of plant equipment using its own infrared thermal imaging device, and then analyze the collected data to diagnose heat-causing equipment failures and thermal defects, and trigger corresponding alarms;

[0019] Environmental and gas detection sensors are used to monitor environmental information such as gas, temperature, and humidity in real time.

[0020] Furthermore, in the above-mentioned fusion obstacle avoidance system for a safety inspection robot, the data fusion module includes a data receiving submodule, a position calculation submodule, a feature extraction submodule, and a feature matching submodule:

[0021] A data receiving submodule is used to receive real-time movement data from various sensors, wherein the real-time movement data at least includes real-time image data, laser radar data, robot speed, position coordinates, number of rotations, angle, travel distance, and rotation angular velocity;

[0022] The position calculation submodule is used to fuse the data from the IMU inertial measurement unit and Beidou using the Kalman filter algorithm, and use the acceleration and angular velocity information provided by the IMU as the input of the motion model to predict the state of the robot; the driving distance measured by the odometer and the absolute position information provided by Beidou are used as observation values ​​to correct the predicted state and obtain the robot position and posture estimation;

[0023] The feature extraction submodule is used to extract features from images collected by the SLAM vision sensor to obtain SIFT scale-invariant feature transformation features;

[0024] The feature matching submodule is used to pre-filter and cluster the point cloud data of the lidar. The feature points in the visual image are matched with the corresponding points in the point cloud data through the feature point matching algorithm. The matched visual and lidar data are fused using the point cloud fusion algorithm to obtain the mobile environment information.

[0025] Furthermore, in the above-mentioned fusion obstacle avoidance system for a safety inspection robot, the positioning mapping module includes a map building submodule, a feature matching submodule, a relationship finding submodule, and a position determination submodule:

[0026] A map construction submodule, for constructing a map based on the mobile environment information by using the depth information of the laser radar and the feature information of the visual image through the ORB-SLAM2 algorithm;

[0027] The feature matching submodule is used to match the currently extracted lidar and visual features with the feature points in the constructed map;

[0028] The relationship search submodule is used to determine the correspondence between the robot's current position and the existing positions in the map by finding similar feature patterns or geometric relationships;

[0029] The position determination submodule is used to indicate that the robot has built a map if the match is successful, and to determine the real-time position information of the robot in the map.

[0030] Furthermore, in the above-mentioned fusion obstacle avoidance system for a safety inspection robot, the navigation module includes a modeling submodule, a calculation submodule, and an acquisition submodule:

[0031] The modeling submodule is used to receive real-time position information and target moving position, and use the Dijkstra path planning algorithm to model the environment in which the robot operates, abstracted as a graph structure;

[0032] The computational submodule is used in the graph where nodes represent possible locations for the robot to reach, and edges represent the connections between these locations and their corresponding costs;

[0033] The submodule is used to calculate the path with the minimum cost from the starting node to the target node through the Dijkstra algorithm to obtain the initial planned path.

[0034] Furthermore, in the above-mentioned fusion obstacle avoidance system for a safety inspection robot, the obstacle avoidance module includes:

[0035] The prediction submodule is used to predict the robot's motion trajectory within a certain time range based on the initial planned path and real-time position information using the dynamic window method;

[0036] The evaluation submodule is used to evaluate the collision risk between the robot and obstacles and the cost of reaching the target under different speed and direction combinations on the predicted trajectory;

[0037] The submodule is used to select the speed and direction with the lowest collision risk and the lowest cost to reach the target as the robot's next action instruction, and obtain the target planning path.

[0038] Furthermore, a method for implementing a fusion obstacle avoidance system for a safety inspection robot is implemented, characterized in that the method comprises the following steps:

[0039] Acquire real-time movement data from sensors set around the safety inspection robot, wherein the sensors include at least SLAM visual sensor, laser radar, IMU inertial measurement unit, Beidou, odometer, and gyroscope;

[0040] Receiving real-time mobile data from various sensors, and fusing the real-time mobile data using a data fusion algorithm to obtain mobile environment information;

[0041] Based on the mobile environment information, the ORB-SLAM2 algorithm is used to construct a mobile map and determine the real-time position information of the robot in the map;

[0042] Receiving real-time location information, and performing inspection path planning on the real-time location information using a Dijkstra path planning algorithm to obtain an initial planned path;

[0043] Monitor the obstacle information in the surroundings in real time. When an obstacle is detected, calculate the obstacle avoidance path by combining the initial planned path and real-time position information to obtain the target planned path.

[0044] The target planning path is transmitted to the remote control center, and real-time feedback data is obtained, and the target planning path is adjusted according to the real-time feedback data.

[0045] Furthermore, a method for implementing a fusion obstacle avoidance system for a safety inspection robot is provided, wherein the method comprises the following steps:

[0046] Acquire real-time image data around the safety inspection robot, and extract feature points, such as corner points and edge points, from the real-time image data;

[0047] Scan obstacles around the safety inspection robot, measure the distance between the robot and the obstacles, and construct the contour information of the surrounding objects;

[0048] Measure the acceleration of the robot on three coordinate axes and calculate the robot's velocity and displacement;

[0049] Get the absolute position coordinates of the robot on the earth, including longitude, latitude and altitude;

[0050] Measure the number of rotations and angles of the robot's wheels, and calculate the robot's travel distance and cumulative travel distance;

[0051] The robot's angular velocity around three coordinate axes is measured to complement the angular velocity data in the IMU inertial measurement unit.

[0052] Use the infrared thermal imager it carries to collect temperature data of plant equipment, then analyze the collected data to diagnose heat-causing equipment failures and thermal defects, and trigger corresponding alarms;

[0053] Real-time monitoring of environmental information such as gas, temperature, and humidity.

[0054] Furthermore, a method for implementing a fusion obstacle avoidance system for a safety inspection robot is implemented, characterized in that the method comprises the following steps:

[0055] Receive real-time movement data from various sensors, the real-time movement data including at least real-time image data, laser radar data, robot speed, position coordinates, number of rotations, angle, travel distance, and rotation angular velocity;

[0056] The Kalman filter algorithm is used to fuse the data from the IMU inertial measurement unit and Beidou, and the acceleration and angular velocity information provided by the IMU is used as the input of the motion model to predict the state of the robot. The driving distance measured by the odometer and the absolute position information provided by Beidou are used as observation values ​​to correct the predicted state and obtain the robot's position and posture estimation.

[0057] Perform feature extraction on the images collected by the SLAM vision sensor to obtain SIFT scale-invariant feature transformation features;

[0058] The point cloud data of the lidar is pre-filtered and clustered. The feature points in the visual image are matched with the corresponding points in the point cloud data through the feature point matching algorithm. The matched visual and lidar data are fused using the point cloud fusion algorithm to obtain the mobile environment information.

[0059] Its beneficial effect lies in that, through the sensor module, it is used to obtain real-time mobile data from sensors set around the safety inspection robot, and the sensors include at least SLAM visual sensor, laser radar, IMU inertial measurement unit, Beidou, odometer, gyroscope; data fusion module, used to receive real-time mobile data from various sensors, and use data fusion algorithm to fuse the real-time mobile data to obtain mobile environment information; positioning mapping module, used to build a mobile map based on the mobile environment information, using ORB-SLAM2 algorithm, and determine the real-time position information of the robot in the map; navigation module, used to receive real-time position information, use Dijkstra path planning algorithm to plan the inspection path for the real-time position information, and obtain the initial planned path; obstacle avoidance module, used to monitor the obstacle information in the surrounding movement in real time, when an obstacle is detected, the obstacle avoidance path is calculated by combining the initial planned path and the real-time position information to obtain the target planned path; communication module, used to transmit the target planned path to the remote control center, and obtain real-time feedback data, and adjust the target planned path according to the real-time feedback data. It can improve the inspection progress of staff, realize the transformation of inspection quality from manual management to electronic management, and conduct evaluation to ensure the normal implementation of inspection work. Managers can use this system to scientifically manage the inspection of valves, pressure gauges, pipelines, lines and other facilities. At the same time, relying on big data computing technology, they can predict and analyze abnormal conditions in advance. They can grasp the implementation quality of inspections and the operation status of facilities through various electronic reports without leaving the office. It can fully and accurately perceive the environmental information around the robot, overcome the limitations of a single sensor, and improve the accuracy of obstacle detection and identification. It can provide stable and reliable positioning information in different environments to ensure that the robot knows its position accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0061] Figure 1It is a schematic diagram of a first embodiment of a fusion obstacle avoidance system for a safety inspection robot in an embodiment of the present invention;

[0062] Figure 2 It is a schematic diagram of a second embodiment of a fusion obstacle avoidance system for a safety inspection robot in an embodiment of the present invention;

[0063] Figure 3 Schematic diagram of a third embodiment of a fusion obstacle avoidance system for a safety inspection robot in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0065] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0066] The present invention will be described in detail below in conjunction with the accompanying drawings. Figure 1 As shown, a fusion obstacle avoidance system for a safety inspection robot includes a sensor module, a data fusion module, a positioning and mapping module, a navigation module, an obstacle avoidance module, and a communication module:

[0067] The sensor module is used to obtain real-time movement data from sensors installed around the safety inspection robot. The sensors include at least SLAM visual sensor, laser radar, IMU inertial measurement unit, Beidou, odometer, and gyroscope;

[0068] Specifically, this embodiment also includes:

[0069] SLAM visual sensor, used to obtain real-time image data around the security inspection robot and extract feature points in the real-time image data, such as corner points and edge points;

[0070] LiDAR is used to scan obstacles around the safety inspection robot, measure the distance between the robot and obstacles, and construct the contour information of surrounding objects;

[0071] IMU inertial measurement unit, used to measure the acceleration of the robot on three coordinate axes and calculate the robot's velocity and displacement;

[0072] Beidou, used to obtain the robot's absolute position coordinates on the earth, including longitude, latitude and altitude;

[0073] Odometer, used to measure the number of rotations and angles of the robot's wheels, and calculate the robot's travel distance and cumulative travel distance;

[0074] The gyroscope is used to measure the angular velocity of the robot around the three coordinate axes, which complements the angular velocity data in the IMU inertial measurement unit;

[0075] Infrared thermal imaging instrument, which is used to collect temperature data of plant equipment using its own infrared thermal imaging device, and then analyze the collected data to diagnose heat-causing equipment failures and thermal defects, and trigger corresponding alarms;

[0076] Environmental and gas detection sensors are used to monitor environmental information such as gas, temperature, and humidity in real time.

[0077] The data fusion module is used to receive the real-time mobile data from various sensors, and use the data fusion algorithm to fuse the real-time mobile data to obtain mobile environment information;

[0078] Specifically, this embodiment also includes:

[0079] The data receiving submodule is used to receive real-time mobile data from various sensors, and the real-time mobile data at least includes real-time image data, laser radar data, robot speed, position coordinates, number of rotations, angle, travel distance, and rotation angular velocity;

[0080] The position calculation submodule is used to fuse the data from the IMU inertial measurement unit and Beidou using the Kalman filter algorithm, and use the acceleration and angular velocity information provided by the IMU as the input of the motion model to predict the state of the robot; the driving distance measured by the odometer and the absolute position information provided by Beidou are used as observation values ​​to correct the predicted state and obtain the robot position and posture estimation;

[0081] The feature extraction submodule is used to extract features from images collected by the SLAM vision sensor to obtain SIFT scale-invariant feature transformation features;

[0082] The feature matching submodule is used to pre-filter and cluster the point cloud data of the lidar. The feature points in the visual image are matched with the corresponding points in the point cloud data through the feature point matching algorithm. The matched visual and lidar data are fused using the point cloud fusion algorithm to obtain the mobile environment information.

[0083] The positioning and mapping module is used to build a mobile map based on the mobile environment information using the ORB-SLAM2 algorithm and determine the real-time position information of the robot in the map;

[0084] Specifically, this embodiment also includes:

[0085] The map construction submodule is used to construct a map based on the mobile environment information using the depth information of the lidar and the feature information of the visual image through the ORB-SLAM2 algorithm;

[0086] Specifically, the ORB-SLAM2 algorithm is a powerful open source visual simultaneous localization and mapping (SLAM) algorithm. The following is a detailed introduction to the algorithm:

[0087] Algorithm features: Multiple camera support: ORB-SLAM2 can support monocular, binocular and RGB-D cameras, which makes it applicable to different application scenarios and sensor configurations, with strong versatility and flexibility. For example, in indoor environments, monocular cameras may face certain challenges in the initialization phase due to the lack of depth information, but they have advantages in scenarios where space is limited and equipment cost is sensitive; binocular cameras can obtain depth information through parallax calculations, which is suitable for scenarios with high requirements for depth accuracy; RGB-D cameras directly provide the depth value of each pixel, and excel in real-time and convenience in obtaining depth information.

[0088] The feature matching submodule is used to match the currently extracted lidar and visual features with the feature points in the constructed map;

[0089] The relationship search submodule is used to determine the correspondence between the robot's current position and the existing positions in the map by finding similar feature patterns or geometric relationships;

[0090] The position determination submodule is used to indicate that the robot has built a map if the match is successful, and to determine the real-time position information of the robot in the map.

[0091] The navigation module is used to receive the real-time location information, and use the Dijkstra path planning algorithm to perform inspection path planning on the real-time location information to obtain an initial planning path;

[0092] Specifically, this embodiment also includes:

[0093] The modeling submodule is used to receive real-time position information and target moving position, and use the Dijkstra path planning algorithm to model the environment in which the robot operates, abstracted as a graph structure;

[0094] The computational submodule is used in the graph where nodes represent possible locations for the robot to reach, and edges represent the connections between these locations and their corresponding costs;

[0095] The submodule is used to calculate the path with the minimum cost from the starting node to the target node through the Dijkstra algorithm to obtain the initial planned path.

[0096] The obstacle avoidance module is used to monitor the obstacle information in the surrounding area in real time. When an obstacle is detected, the obstacle avoidance path is calculated by combining the initial planned path and the real-time position information to obtain the target planned path;

[0097] Specifically, this embodiment also includes:

[0098] The prediction submodule is used to predict the robot's motion trajectory within a certain time range based on the initial planned path and real-time position information using the dynamic window method;

[0099] The evaluation submodule is used to evaluate the collision risk between the robot and obstacles and the cost of reaching the target under different speed and direction combinations on the predicted trajectory;

[0100] The submodule is used to select the speed and direction with the lowest collision risk and the lowest cost to reach the target as the robot's next action instruction, and obtain the target planning path.

[0101] Environment modeling and graph construction

[0102] First, the environment in which the robot operates is modeled and abstracted into a graph structure.

[0103] Determine the nodes in the graph, which can be locations that the robot can actually reach, such as different coordinate points, key locations in the room, intersections of corridors, etc. Each node has a corresponding identifier and coordinate information.

[0104] Define the edges in the graph, which represent the connection between nodes. The weight of the edge can be set according to the actual situation, such as the actual distance between two nodes, the time required to walk, the difficulty of passing through, etc. For example, if there is an obstacle between two nodes, the weight of the edge may need to be increased to reflect the difficulty of passing through.

[0105] Create a data structure to store graph information, including a list of nodes and edge connections and weight information.

[0106] initialization:

[0107] Two sets are established, one is the node set with the determined shortest path (denoted as S), which is empty initially; the other is the node set with the undetermined shortest path (denoted as U), which initially contains all the nodes in the graph (except the starting node).

[0108] Set a distance value for each node, which represents the current shortest path estimate from the starting node to that node. The distance value of the starting node is set to 0, and the distance values ​​of other nodes are set to infinity (indicating that the shortest path lengths to these nodes are not known initially).

[0109] Create an array to record the predecessor nodes, which is used to trace back the path after finding the shortest path. The predecessor node of the starting node is set to itself, and the predecessor nodes of other nodes are initialized to empty or a special value indicating that they are undetermined.

[0110] Path Search

[0111] Select the node with the smallest distance value from the node set U of the undetermined shortest path, and add it to the set S of the determined shortest path. This node is the node currently considered to be closest to the starting node.

[0112] For the node just added to the set S (set as the current node), check all its adjacent nodes whose shortest paths have not been determined (ie, neighbor nodes in the set U).

[0113] For each neighbor node, calculate the distance from the start node through the current node to the neighbor node (that is, the distance value of the current node plus the weight of the edge from the current node to the neighbor node).

[0114] If the calculated distance is less than the current distance value of the neighbor node, the distance value of the neighbor node is updated to the newly calculated value, and the predecessor node of the neighbor node is set as the current node. This means that a shorter path is found to reach the neighbor node, passing through the current node.

[0115] Repeat step 3 until the target node is added to the set S of determined shortest paths. At this point, the shortest path from the start node to the target node has been found (under the current graph model and cost definition).

[0116] Path Generation

[0117] Starting from the target node, trace back to the start node in sequence according to the predecessor node array, so as to obtain the shortest path from the start node to the target node.

[0118] The path nodes obtained by backtracking are recorded in order to form a path list, which is the navigation path of the robot.

[0119] Path execution and dynamic adjustment

[0120] The robot starts to move along the generated path. During the movement, it may encounter some situations where the actual situation is different from the modeling, such as the obstacles in the environment have changed or new temporary obstacles have appeared.

[0121] If the robot detects an impassable obstacle on its current path, it can pause and restart the path planning process, recalculating the shortest path from its current location to its goal location using updated information about the environment.

[0122] Continue along the newly planned path and repeat this process until you reach your target location.

[0123] The communication module is used to transmit the target planning path to the remote control center and obtain real-time feedback data, and adjust the target planning path according to the real-time feedback data.

[0124] Specifically, this embodiment also includes:

[0125] Two-way voice intercom

[0126] The two-way voice function of the robot refers to the intercom and broadcast functions. Intercom means that the staff in the station can talk to the inspection robot management background and the robot. This function is realized by turning on the voice intercom of the inspection robot management background, and the front staff needs to turn on the robot's voice button. Broadcasting can be used for the management background to remotely command on-site work or require rapid evacuation in an emergency.

[0127] Task Management

[0128] In the monitoring platform, task management is divided into 7 modules: comprehensive inspection, routine inspection, special inspection, special inspection, custom task, map point selection and task display. Each module is divided into different modules, and different types of tasks can be established according to various situations. In the task system, tasks can be executed manually or assigned according to the schedule to realize automatic inspection function.

[0129] The robot's automatic inspection function enables it to complete indoor inspections autonomously, with highly automated and intelligent features, thus replacing manual inspections. The automatic inspection modes mainly include: scheduled inspections, routine inspections, special inspections, single inspections, and manual control inspections, and various modes support switching between each other. The robot can complete tasks according to pre-set inspection modes and complete inspection activities such as meter readings, power equipment temperature measurements, and environmental monitoring according to established rules.

[0130] ①Automatic inspection

[0131] Automatic inspection is a way for the robot to automatically patrol according to the preset planned path. The main modes of automatic inspection are: routine inspection, special inspection, etc., and various modes support switching. The robot is equipped with a variety of detection equipment such as high-definition cameras, high-sensitivity infrared thermal imagers, and microphones. According to the established rules, it completes the inspection activities of various indoor equipment such as visible light photography, meter reading, infrared temperature measurement, and environmental monitoring. The inspection data is automatically transmitted to the background management system for storage and generates a detection analysis report.

[0132] ②Manual remote inspection

[0133] In addition to automatic inspections, robots can also be remotely controlled in real time through manual remote inspections. This application mode is suitable for operation and maintenance personnel and management units who need to lock and monitor the status of certain types of equipment. In particular, if the robot detects abnormal equipment or environmental status during autonomous inspections and alerts the operation and maintenance personnel, the operation and maintenance personnel can use the manual remote control interface or remote control handle of the management background to control the robot to quickly reach the abnormal equipment location, check the abnormal equipment in time and verify the alarm information, so as to quickly formulate a response strategy.

[0134] Statistics:

[0135] The monitoring platform has the functions of storing, analyzing, counting and retrieving monitoring data. The monitoring platform organizes and sorts data in a standardized manner and supports exporting and saving PDF and Excel reports. Operation and maintenance personnel can search for data in the monitoring platform according to tasks or monitoring points and quickly view monitoring data.

[0136] Report Output

[0137] After completing a task, the track-mounted robot will automatically generate a report, and the operation and maintenance personnel can search and view the report on the monitoring platform according to time, inspection type, task name, etc. The operation and maintenance personnel can search, view, and export various reports according to their needs, including inspection task reports, data reports, alarm record reports, environmental information reports, etc.

[0138] Early warning and linkage

[0139] During the inspection process, the robot automatically warns of abnormal data through numerical analysis, threshold comparison, trend analysis, database and other related technologies. The warning information is presented in two ways: interface alarm and sound and light alarm, which promptly reminds the operation and maintenance personnel to pay attention.

[0140] When the background analysis data shows abnormalities, an alarm information list will appear on the background monitoring software interface, and the operation and maintenance personnel can scroll the alarm information through operations.

[0141] When there is an alarm message in the system, the system will alert the operation and maintenance personnel through the patrol robot alarm audio and flashing lights.

[0142] Robot self-check

[0143] As the main body of the inspection system, the robot is equipped with major detection equipment to collect various data information at the front end. In order to ensure the daily work of the robot, the robot will perform self-inspection before starting the inspection. The self-inspection content includes infrared thermal imager, high-definition camera, motor, pan / tilt, internal storage and various sensors. If any abnormality is found in a component, an abnormal status indication will be given, and the system abnormality information will be uploaded to the monitoring background, so that the operation and maintenance personnel can find faults in time, reduce processing time, and improve the efficiency of troubleshooting.

[0144] Data interface function

[0145] Provide a variety of data interfaces to upload data in a predetermined manner according to the system requirements of the bus. The inspection robot can provide data contacts (switch signals or RS485 communication) for the on-site SIS system; later, it can provide one of the docking methods such as OPC UA and WebApi. During the inspection process, the inspection robot will automatically transmit the inspection data to the system background computer through an encrypted network, and obtain intuitive inspection results after analysis and processing by the background software. After the robot inspection background is connected to the bus system, it can automatically upload the inspection data according to the system's standard format and transmission protocol, and the operation and maintenance personnel can directly view and summarize the equipment information in the bus system.

[0146] Its beneficial effect lies in that, through the sensor module, it is used to obtain real-time mobile data from sensors set around the safety inspection robot, and the sensors include at least SLAM visual sensor, laser radar, IMU inertial measurement unit, Beidou, odometer, gyroscope; data fusion module, used to receive real-time mobile data from various sensors, and use data fusion algorithm to fuse the real-time mobile data to obtain mobile environment information; positioning mapping module, used to build a mobile map based on mobile environment information, using ORB-SLAM2 algorithm, and determine the real-time position information of the robot in the map; navigation module, used to receive real-time position information, use Dijkstra path planning algorithm to plan the inspection path for the real-time position information, and obtain the initial planned path; obstacle avoidance module, used to monitor the obstacle information in the surrounding movement in real time, when an obstacle is detected, the obstacle avoidance path is calculated by combining the initial planned path and the real-time position information to obtain the target planned path; communication module, used to transmit the target planned path to the remote control center, and obtain real-time feedback data, and adjust the target planned path according to the real-time feedback data. It can improve the inspection progress of staff, realize the transformation of inspection quality from manual management to electronic management, and conduct evaluation to ensure the normal implementation of inspection work. Managers can use this system to scientifically manage the inspection of valves, pressure gauges, pipelines, lines and other facilities. At the same time, relying on big data computing technology, they can predict and analyze abnormal conditions in advance. They can grasp the implementation quality of inspections and the operation status of facilities through various electronic reports without leaving the office. It can fully and accurately perceive the environmental information around the robot, overcome the limitations of a single sensor, and improve the accuracy of obstacle detection and identification. It can provide stable and reliable positioning information in different environments to ensure that the robot knows its position accurately.

[0147] See also Figure 2 In a fusion obstacle avoidance system for a safety inspection robot, the positioning and mapping module includes a map building submodule, a feature matching submodule, a relationship finding submodule, and a position determination submodule:

[0148] The map construction submodule is used to construct a map based on the mobile environment information using the depth information of the lidar and the feature information of the visual image through the ORB-SLAM2 algorithm;

[0149] The feature matching submodule is used to match the currently extracted lidar and visual features with the feature points in the constructed map;

[0150] The relationship search submodule is used to determine the correspondence between the robot's current position and the existing positions in the map by finding similar feature patterns or geometric relationships;

[0151] The position determination submodule is used to indicate that the robot has built a map if the match is successful, and to determine the real-time position information of the robot in the map.

[0152] See also Figure 3 In a fusion obstacle avoidance system for a safety inspection robot, the obstacle avoidance module includes a prediction submodule, an evaluation submodule, and an acquisition submodule:

[0153] The prediction submodule is used to predict the robot's motion trajectory within a certain time range based on the initial planned path and real-time position information using the dynamic window method;

[0154] The evaluation submodule is used to evaluate the collision risk between the robot and obstacles and the cost of reaching the target under different speed and direction combinations on the predicted trajectory;

[0155] The submodule is used to select the speed and direction with the lowest collision risk and the lowest cost to reach the target as the robot's next action instruction, and obtain the target planning path.

[0156] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A fusion obstacle avoidance system for a safety inspection robot, characterized in that: The fusion obstacle avoidance system includes a sensor module, a data fusion module, a positioning and mapping module, a navigation module, an obstacle avoidance module, and a communication module: The sensor module is used to obtain real-time movement data from sensors set around the safety inspection robot, and the sensors include at least SLAM visual sensor, laser radar, IMU inertial measurement unit, Beidou, odometer, and gyroscope; A data fusion module is used to receive real-time mobile data from various sensors, and use a data fusion algorithm to fuse the real-time mobile data to obtain mobile environment information; A positioning and mapping module is used to construct a mobile map based on the mobile environment information using the ORB-SLAM2 algorithm and determine the real-time position information of the robot in the map; A navigation module is used to receive real-time location information, and use the Dijkstra path planning algorithm to perform inspection path planning on the real-time location information to obtain an initial planned path; The obstacle avoidance module is used to monitor the obstacle information in the surrounding area in real time. When an obstacle is detected, the obstacle avoidance path is calculated by combining the initial planned path and the real-time position information to obtain the target planned path; The communication module is used to transmit the target planning path to the remote control center and obtain real-time feedback data, and adjust the target planning path according to the real-time feedback data.

2. A fusion obstacle avoidance system for a safety inspection robot as claimed in claim 1, characterized in that: The sensor module includes SLAM visual sensor, laser radar, IMU inertial measurement unit, Beidou, odometer, gyroscope, infrared thermal imaging instrument, environment and gas detection sensor: The SLAM visual sensor is used to obtain real-time image data around the security inspection robot and extract feature points in the real-time image data, such as corner points and edge points; LiDAR is used to scan obstacles around the safety inspection robot, measure the distance between the robot and obstacles, and construct the contour information of surrounding objects; IMU inertial measurement unit, used to measure the acceleration of the robot on three coordinate axes and calculate the robot's velocity and displacement; Beidou, used to obtain the robot's absolute position coordinates on the earth, including longitude, latitude and altitude; Odometer, used to measure the number of rotations and angles of the robot's wheels, and calculate the robot's travel distance and cumulative travel distance; A gyroscope is used to measure the rotational angular velocity of the robot around three coordinate axes, and complements the angular velocity data in the IMU inertial measurement unit; Infrared thermal imaging instrument, which is used to collect temperature data of plant equipment using its own infrared thermal imaging device, and then analyze the collected data to diagnose heat-causing equipment failures and thermal defects, and trigger corresponding alarms; Environmental and gas detection sensors are used to monitor environmental information such as gas, temperature, and humidity in real time.

3. A fusion obstacle avoidance system for a safety inspection robot as claimed in claim 1, characterized in that: The data fusion module includes a data receiving submodule, a position calculation submodule, a feature extraction submodule, and a feature matching submodule: A data receiving submodule is used to receive real-time movement data from various sensors, wherein the real-time movement data at least includes real-time image data, laser radar data, robot speed, position coordinates, number of rotations, angle, travel distance, and rotation angular velocity; The position calculation submodule is used to fuse the data from the IMU inertial measurement unit and Beidou using the Kalman filter algorithm, and use the acceleration and angular velocity information provided by the IMU as the input of the motion model to predict the state of the robot; the driving distance measured by the odometer and the absolute position information provided by Beidou are used as observation values ​​to correct the predicted state and obtain the robot position and posture estimation; The feature extraction submodule is used to extract features from images collected by the SLAM visual sensor to obtain SI FT scale-invariant feature transformation features; The feature matching submodule is used to pre-filter and cluster the point cloud data of the lidar. The feature points in the visual image are matched with the corresponding points in the point cloud data through the feature point matching algorithm. The matched visual and lidar data are fused using the point cloud fusion algorithm to obtain the mobile environment information.

4. A fusion obstacle avoidance system for a safety inspection robot as claimed in claim 1, characterized in that: The positioning and mapping module includes a map construction submodule, a feature matching submodule, a relationship search submodule, and a position determination submodule: A map construction submodule, for constructing a map based on the mobile environment information by using the depth information of the laser radar and the feature information of the visual image through the ORB-SLAM2 algorithm; The feature matching submodule is used to match the currently extracted lidar and visual features with the feature points in the constructed map; The relationship search submodule is used to determine the correspondence between the robot's current position and the existing positions in the map by finding similar feature patterns or geometric relationships; The position determination submodule is used to indicate that the robot has built a map if the match is successful, and to determine the real-time position information of the robot in the map.

5. The fusion obstacle avoidance system for a safety inspection robot according to claim 1, characterized in that: The navigation module includes a modeling submodule, a calculation submodule, and an acquisition submodule: The modeling submodule is used to receive real-time position information and target moving position, and use the Dijkstra path planning algorithm to model the environment in which the robot operates, abstracted as a graph structure; The computational submodule is used in the graph where nodes represent possible locations for the robot to reach, and edges represent the connections between these locations and their corresponding costs; The submodule is used to calculate the path with the minimum cost from the starting node to the target node through the Dijkstra algorithm to obtain the initial planned path.

6. The fusion obstacle avoidance system for a safety inspection robot according to claim 1, characterized in that: The obstacle avoidance module includes a prediction submodule, an evaluation submodule, and an acquisition submodule: The prediction submodule is used to predict the robot's motion trajectory within a certain time range based on the initial planned path and real-time position information using the dynamic window method; The evaluation submodule is used to evaluate the collision risk between the robot and obstacles and the cost of reaching the target under different speed and direction combinations on the predicted trajectory; The submodule is used to select the speed and direction with the lowest collision risk and the lowest cost to reach the target as the robot's next action instruction, and obtain the target planning path.

7. A method for implementing a fusion obstacle avoidance system for a safety inspection robot as claimed in claim 1, characterized in that: The method comprises the following steps: Acquire real-time movement data from sensors set around the safety inspection robot, wherein the sensors include at least SLAM visual sensor, laser radar, IMU inertial measurement unit, Beidou, odometer, and gyroscope; Receiving real-time mobile data from various sensors, and fusing the real-time mobile data using a data fusion algorithm to obtain mobile environment information; Based on the mobile environment information, the ORB-SLAM2 algorithm is used to construct a mobile map and determine the real-time position information of the robot in the map; Receiving real-time location information, and performing inspection path planning on the real-time location information using a Dijkstra path planning algorithm to obtain an initial planned path; Monitor the obstacle information in the surroundings in real time. When an obstacle is detected, calculate the obstacle avoidance path by combining the initial planned path and real-time position information to obtain the target planned path. The target planning path is transmitted to the remote control center, and real-time feedback data is obtained, and the target planning path is adjusted according to the real-time feedback data.

8. A method for implementing a fusion obstacle avoidance system for a safety inspection robot as claimed in claim 1, characterized in that: The method comprises the following steps: Acquire real-time image data around the safety inspection robot, and extract feature points, such as corner points and edge points, from the real-time image data; Scan obstacles around the safety inspection robot, measure the distance between the robot and the obstacles, and construct the contour information of the surrounding objects; Measure the acceleration of the robot on three coordinate axes and calculate the robot's velocity and displacement; Get the robot's absolute position coordinates on the earth, including longitude, latitude and altitude; Measure the number of rotations and angles of the robot's wheels, and calculate the robot's travel distance and cumulative travel distance; The robot's angular velocity around the three coordinate axes is measured, which complements the angular velocity data in the IMU inertial measurement unit. Use the infrared thermal imager it carries to collect temperature data of plant equipment, then analyze the collected data to diagnose heat-causing equipment failures and thermal defects, and trigger corresponding alarms; Real-time monitoring of environmental information such as gas, temperature, and humidity.

9. A method for implementing a fusion obstacle avoidance system for a safety inspection robot as claimed in claim 1, characterized in that: The method comprises the following steps: Receive real-time movement data from various sensors, the real-time movement data including at least real-time image data, laser radar data, robot speed, position coordinates, number of rotations, angle, travel distance, and rotation angular velocity; The Kalman filter algorithm is used to fuse the data from the IMU inertial measurement unit and BeiDou, and the acceleration and angular velocity information provided by the IMU is used as the input of the motion model to predict the state of the robot. The driving distance measured by the odometer and the absolute position information provided by BeiDou are used as observation values ​​to correct the predicted state and obtain the robot position and attitude estimation. Perform feature extraction on the images collected by the SLAM vision sensor to obtain SI FT scale-invariant feature transformation features; The point cloud data of the lidar is pre-filtered and clustered. The feature points in the visual image are matched with the corresponding points in the point cloud data through the feature point matching algorithm. The matched visual and lidar data are fused using the point cloud fusion algorithm to obtain the mobile environment information.

Images