Robot capable of automatically searching personnel in underground coal mine and rescue method

Abstract

The invention relates to an underground coal mine robot capable of automatically searching personnel and a rescue method, and the method comprises the steps that the robot collects environment information of an underground accident site, and the environment information comprises high-frequency IMU data, laser radar point cloud and camera images; and determining the current position of the robot according to the high-frequency IMU data, the laser radar point cloud, the motor encoder pulse data and the camera image. High-precision positioning is achieved through the laser radar, the inertial measurement unit and the camera, detection such as personnel search and gas analysis is carried out on an unknown area, after the trapped personnel are searched, the trapped personnel can be assisted to evacuate rapidly through the manned cabin, the position of the trapped personnel can be sent to rescue personnel outside the scene when the trapped personnel are disabled, and the rescue personnel can rescue the rescue personnel outside the scene. Rescue workers are assisted to enter an accident scene to rescue the wounded. The safety and efficiency of rescue can be greatly improved, powerful technical guarantee can be provided for emergency disposal of coal mine safety accidents, and the system becomes an important tool for guaranteeing the life safety of miners.

Description

Technical Field

[0001] This application relates to the field of underground coal mine rescue, specifically to a robot and rescue method for automatically searching for people underground in coal mines. Background Technology

[0002] Coal mining depths are constantly increasing with advancements in mining technology. Although modern coal mines are equipped with various safety devices and safety management systems are gradually improving, coal mine accidents remain unavoidable. Whenever serious accidents such as collapses, water inrushes, or gas leaks occur, the on-site environment is usually extremely complex, easily triggering secondary disasters and posing a significant threat to the lives of rescue personnel. At the same time, the extremely harsh and unpredictable underground environment, coupled with the numerous dangerous areas, unclear environmental conditions, and blocked rescue routes after an accident, further increases the difficulty of rescue operations. Against this backdrop, traditional rescue methods prove inadequate, especially in complex mine environments where the safety of rescue personnel often cannot be effectively guaranteed. Summary of the Invention

[0003] To overcome at least one deficiency in the prior art, this application provides a robot and rescue method for automatically searching for personnel in underground coal mines.

[0004] Firstly, a robotic rescue method for automatically searching for personnel in underground coal mines is provided, including:

[0005] Step 1: The robot collects environmental information from the downhole accident site, including high-frequency IMU data, lidar point clouds, and camera images.

[0006] Step 2: Determine the robot's current position based on high-frequency IMU data, LiDAR point cloud, motor encoder pulse data, and camera images;

[0007] Step 3: Obtain the gas concentration information at the current location;

[0008] Step 4: Check if there are any trapped personnel at the current location. If yes, proceed to step 5; otherwise, proceed to step 6.

[0009] Step 5: Play the instructions and precautions for using the manned cabin, and wait for the trapped personnel to enter the manned cabin. If the trapped personnel enter the manned cabin within the set time, the robot will carry the manned cabin back to a safe location. If the trapped personnel do not enter the manned cabin within the set time, the robot will send the trapped personnel's location information and the gas concentration information at the current location to the rescue personnel.

[0010] Step 6: The robot performs path planning and continues to move according to the path planning results, then returns to Step 1 until the search of the underground accident site is completed.

[0011] In one embodiment, determining the robot's current position based on high-frequency IMU data, LiDAR point cloud data, motor encoder pulse data, and camera images includes:

[0012] A linear time interpolation method is used to achieve time alignment between lidar point cloud and high-frequency IMU data, resulting in time-aligned lidar point cloud and high-frequency IMU data.

[0013] Based on the calibration extrinsic parameter matrix, cross-modal projection of the time-aligned LiDAR point cloud to the camera is achieved, generating a projection feature map;

[0014] Based on the pixel correspondence between the projected feature map and the camera image, RGB color information is assigned to the time-aligned LiDAR point cloud to generate a colored point cloud.

[0015] The time-aligned high-frequency IMU data is fused with the motor encoder pulse data to obtain the first anti-interference pose estimation, which includes the first covariance matrix and the first pose.

[0016] Based on the colored point cloud at the current time and the colored point cloud at the previous time, a second anti-interference pose estimate is generated. The second anti-interference pose estimate includes a second covariance matrix and a second pose.

[0017] The reliability of the first and second anti-interference pose estimates is determined based on the first and second covariance matrices, and a reliable result is obtained.

[0018] The robot's current position is determined based on the reliable results.

[0019] In one embodiment, determining the robot's current position based on a reliable result includes:

[0020] If the first anti-interference pose estimation is reliable and the second anti-interference pose estimation is unreliable, then the first pose is taken as the robot's current position.

[0021] If the first anti-interference pose estimation is unreliable, but the second anti-interference pose estimation is reliable, then the second pose is taken as the robot's current position.

[0022] If both the first and second anti-interference pose estimates are reliable, then the second pose is taken as the robot's current position.

[0023] Secondly, a robot capable of automatically searching for personnel in underground coal mines is provided, characterized by a robot rescue method for automatically searching for personnel in underground coal mines as described above.

[0024] In one embodiment, the robot includes a lidar, an inertial measurement unit, a motor encoder, and a camera, which are used to acquire lidar point clouds, high-frequency IMU data, motor encoder pulse data, and camera images, respectively.

[0025] In one embodiment, the robot includes a data storage module for storing robot trajectory, colored point cloud, and gas concentration information.

[0026] In one embodiment, the robot includes a life detector for detecting whether there are trapped personnel at the current location.

[0027] Compared to existing technologies, this application offers the following advantages: The robot and rescue method for automatically searching for personnel in coal mines utilize lidar, inertial measurement units (IMUs), and cameras to achieve high-precision positioning and perform personnel searches and gas analysis in unknown areas. After locating trapped personnel, a manned cabin assists in their rapid evacuation. In cases where trapped personnel are incapacitated, their location can be transmitted to off-site rescue personnel to assist in entering the accident site and rescuing the injured. This application not only significantly improves the safety and efficiency of rescue efforts but also provides strong technical support for emergency response to coal mine accidents, becoming an important tool for protecting miners' lives. Attached Figure Description

[0028] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings:

[0029] Figure 1 A flowchart illustrating a robotic rescue method for automatically searching for people underground in a coal mine is shown. Detailed Implementation

[0030] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.

[0031] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0032] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0033] This application provides a robotic rescue method for automatically searching for personnel in underground coal mines. Figure 1 A flowchart illustrating a robotic rescue method for automatically searching for personnel underground in coal mines is shown. (See attached image) Figure 1 The methods mainly include:

[0034] Step 1: The robot collects environmental information from the downhole accident site, including high-frequency IMU data, lidar point clouds, and camera images.

[0035] Here, a lidar, an inertial measurement unit (IMU), and a camera can be used to acquire lidar point clouds, high-frequency IMU data, and camera images, respectively. Lidar measures the distance to points in the environment by emitting a laser beam and receiving its reflection, thus forming a point cloud.

[0036] Step 2: Determine the robot's current position based on high-frequency IMU data, LiDAR point cloud, motor encoder pulse data, and camera images.

[0037] Step 3: Obtain gas concentration information at the current location. This includes the concentration of hazardous gases. After obtaining the gas concentration information at the current location, spatialized gas concentration observations can be generated by aligning IMU data with timestamps, providing real-time hazardous gas monitoring data for environmental modeling.

[0038] Step 4: Check if there are any trapped personnel at the current location. If yes, proceed to step 5; otherwise, proceed to step 6.

[0039] Step 5: Play the instructions and precautions for using the manned cabin, and wait for the trapped personnel to enter the manned cabin. If the trapped personnel enter the manned cabin within the set time, the robot will carry the manned cabin back to a safe location. If the trapped personnel do not enter the manned cabin within the set time, the robot will send the trapped personnel's location information and the gas concentration information at the current location to the rescue personnel.

[0040] Here, the time limit can be set to, for example, 2 minutes. If the trapped personnel do not enter the manned cabin within the set time, it can be assumed that the trapped personnel are incapacitated and unable to enter the manned cabin.

[0041] After receiving the location information of the trapped person, rescuers can estimate the concentration of various gases around the current location based on the gas concentration information, and thus formulate a safe rescue plan to avoid or reduce the harm of harmful gases to the rescuers.

[0042] Step 6: The robot performs path planning and continues to move according to the path planning results, then returns to Step 1 until the search of the underground accident site is completed.

[0043] Here, path planning can be performed using algorithms such as A*, Dijkstra's algorithm, RRT (Rapidly-Exploring Random Tree), and particle swarm optimization.

[0044] In this embodiment, high-precision positioning is achieved using lidar, inertial measurement unit and camera, and detection such as personnel search and gas analysis is performed in unknown areas. After the trapped personnel are found, the manned cabin can assist the trapped personnel to evacuate quickly. In the case of the trapped personnel being incapacitated, the location of the trapped personnel can be sent to the off-site search and rescue personnel to assist the rescue personnel in entering the accident site to rescue the injured.

[0045] In one embodiment, step 2, determining the robot's current position based on high-frequency IMU data, LiDAR point cloud, motor encoder pulse data, and camera images, includes:

[0046] Step 21: Use linear time interpolation to achieve time alignment between lidar point cloud and high-frequency IMU data to obtain time-aligned lidar point cloud and high-frequency IMU data;

[0047] Step 22: Based on the calibration extrinsic parameter matrix, perform cross-modal projection of the time-aligned lidar point cloud to the camera to generate a projection feature map;

[0048] Step 23: Based on the pixel correspondence between the projected feature map and the camera image, assign RGB color information to the time-aligned LiDAR point cloud to generate a colored point cloud; here, the RGB color information of the camera image pixels is assigned to the corresponding pixels of the projected feature map to obtain the colored point cloud.

[0049] Step 24: The time-aligned high-frequency IMU data is fused with the motor encoder pulse data to obtain the first anti-interference pose estimation, which includes the first covariance matrix and the first pose; the pose here includes position and attitude.

[0050] IMUs typically provide high-frequency angular velocity and acceleration data, while motor encoders provide wheel rotation information (pulse counts can be converted into displacement or velocity). Fusion of these two sensors can be complementary: IMUs offer high accuracy over short periods but are prone to drift; motor encoders provide accurate distance information over long periods but may be affected by factors such as ground slippage. Common fusion methods include Kalman filtering and its variants (such as Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF)) as well as complementary filtering.

[0051] Step 25: Generate a second anti-interference pose estimate based on the colored point cloud at the current time and the colored point cloud at the previous time. The second anti-interference pose estimate includes a second covariance matrix and a second pose.

[0052] Here, point cloud registration techniques, such as the IPC algorithm, can be used to estimate the pose against interference. First, the point cloud at the current time step (source point cloud) needs to be correlated with the point cloud at the previous time step (target point cloud), i.e., finding the corresponding point in the target point cloud for each point in the source point cloud. This can be achieved through nearest neighbor search (such as KDTree). Then, for the matched point pairs, their covariance matrix can be calculated. The covariance matrix describes the distribution of the matched point pairs in three-dimensional space and can be used to evaluate the uncertainty of the matching.

[0053] Step 26: Determine whether the first anti-interference pose estimation and the second anti-interference pose estimation are reliable based on the first covariance matrix and the second covariance matrix, and obtain a reliable result.

[0054] In state estimation, the covariance matrix represents the uncertainty of the estimated state. The diagonal elements are the variances of each state component (i.e., the square of the uncertainty), while the off-diagonal elements represent the correlations between the state components. Confidence can be judged from several aspects: a) the trace of the covariance matrix, b) the determinant of the covariance matrix, c) the eigenvalues, and d) the condition number (the ratio of the largest to the smallest eigenvalue).

[0055] For example, trustworthiness can be determined based on traces. Depending on the specific application scenario, a threshold for traces can be set. If a trace exceeds the threshold, it is considered to have too much uncertainty and is therefore untrustworthy.

[0056] Step 27: Determine the robot's current position based on the reliable results.

[0057] In this embodiment, the poses acquired by the lidar and the poses acquired by the IMU are reliably judged, thereby enabling a more accurate determination of the robot's current position.

[0058] In one embodiment, step 27, determining the robot's current position based on the reliable result, includes:

[0059] If the first anti-interference pose estimation is reliable and the second anti-interference pose estimation is unreliable, then the first pose is taken as the robot's current position.

[0060] If the first anti-interference pose estimation is unreliable, but the second anti-interference pose estimation is reliable, then the second pose is taken as the robot's current position.

[0061] If both the first and second anti-interference pose estimates are reliable, then the second pose is taken as the robot's current position.

[0062] It should be noted that if both the first and second anti-interference pose estimates are unreliable, return to step 1 and reacquire the data.

[0063] This application also provides a robot capable of automatically searching for personnel underground in coal mines, used to implement the robot rescue method for automatically searching for personnel underground in coal mines described in the foregoing embodiments.

[0064] Specifically, the robot includes a lidar, an inertial measurement unit, a motor encoder, and a camera, which are used to collect lidar point clouds, high-frequency IMU data, motor encoder pulse data, and camera images, respectively.

[0065] Specifically, the robot includes a data storage module for storing robot trajectory, colored point cloud, and gas concentration information.

[0066] Specifically, the robot includes a life detector to detect whether there are trapped people at the current location.

[0067] The specific implementation method of the robot that can automatically search for people in underground coal mines in this embodiment can be found in the embodiment section of the robot rescue method for automatically searching for people in underground coal mines mentioned above, and its technical effects correspond to the technical effects of the above methods, so it will not be repeated here.

[0068] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A robotic rescue method for automatically searching for personnel in underground coal mines, characterized in that, include: Step 1: The robot collects environmental information at the downhole accident site, including high-frequency IMU data, lidar point clouds, and camera images. Step 2: Determine the robot's current position based on high-frequency IMU data, LiDAR point cloud, motor encoder pulse data, and camera images; Step 3: Obtain the gas concentration information at the current location; Step 4: Check if there are any trapped personnel at the current location. If yes, proceed to step 5; otherwise, proceed to step 6. Step 5: Play the instructions and precautions for using the manned cabin, and wait for the trapped personnel to enter the manned cabin. If the trapped personnel enter the manned cabin within the set time, the robot will carry the manned cabin back to a safe location. If the trapped personnel do not enter the manned cabin within the set time, the robot will send the trapped personnel's location information and the gas concentration information at the current location to the rescue personnel. Step 6: The robot performs path planning and continues to move according to the path planning results, then returns to Step 1 until the search of the underground accident site is completed.

2. The method as described in claim 1, characterized in that, in, Based on the high-frequency IMU data, lidar point cloud, motor encoder pulse data, and camera images, the robot's current position is determined, including: A linear time interpolation method is used to achieve time alignment between lidar point cloud and high-frequency IMU data, resulting in time-aligned lidar point cloud and high-frequency IMU data. Based on the calibration extrinsic parameter matrix, cross-modal projection of the time-aligned LiDAR point cloud to the camera is achieved, generating a projection feature map; Based on the pixel correspondence between the projected feature map and the camera image, RGB color information is assigned to the time-aligned lidar point cloud to generate a colored point cloud; The time-aligned high-frequency IMU data is fused with the motor encoder pulse data to obtain the first anti-interference pose estimation, which includes the first covariance matrix and the first pose. Based on the colored point cloud at the current time and the colored point cloud at the previous time, a second anti-interference pose estimate is generated, which includes a second covariance matrix and a second pose. Based on the first covariance matrix and the second covariance matrix, determine whether the first anti-interference pose estimation and the second anti-interference pose estimation are reliable, and obtain a reliable result; The robot's current position is determined based on the reliable results.

3. The method as described in claim 2, characterized in that, in, Determining the robot's current position based on the reliable result includes: If the first anti-interference pose estimation is reliable and the second anti-interference pose estimation is unreliable, then the first pose is taken as the robot's current position. If the first anti-interference pose estimation is unreliable, but the second anti-interference pose estimation is reliable, then the second pose is taken as the robot's current position. If both the first anti-interference pose estimation and the second anti-interference pose estimation are reliable, then the second pose is taken as the robot's current position.

4. A robot capable of automatically searching for personnel underground in a coal mine, characterized in that, This is used to implement the robotic rescue method for automatically searching for personnel in underground coal mines as described in any one of claims 1-3.

5. The robot as described in claim 4, characterized in that, It includes a lidar, an inertial measurement unit, a motor encoder, and a camera, which are used to acquire lidar point clouds, high-frequency IMU data, motor encoder pulse data, and camera images, respectively.

6. The robot as described in claim 4, characterized in that, It includes a data storage module for storing robot trajectory, colored point cloud, and gas concentration information.

7. The robot as described in claim 4, characterized in that, This includes life detectors, used to detect whether there are any trapped individuals at the current location.

Images