Machine-exploration integrated coal mining robot and autonomous navigation operation method thereof

Abstract

The invention discloses a machine-exploration integrated coal mining robot and an autonomous navigation operation method thereof, and relates to the technical field of coal mining robots, the machine-exploration integrated coal mining robot comprises a coal mining machine body and a remote monitoring terminal, and the remote monitoring terminal is connected with the coal mining machine body. A positioning and attitude determining module, a geological scene integrated sensing module, a self-state sensing module, a decision planning module and a joint control module are arranged on the coal mining machine body. According to the machine-exploration integrated coal mining robot and the autonomous navigation operation method thereof, non-contact autonomous accurate scene and geological modeling in the mining process of the coal mining robot is achieved, the process is simple, manual operation is little, precision is high, efficiency is high, and the method is suitable for popularization and application. And the conversion from automatic memory cutting of the coal mining machine to autonomous perception planning of the coal mining robot is really realized.

Description

Technical Field

[0001] The present invention relates to the technical field of coal mining robots, and in particular to a machine-exploration integrated coal mining robot and an autonomous navigation operation method thereof. Background Art

[0002] From the perspective of autonomous mobile robots, coal mining robots also include three key technologies: "perception, planning, and control". Among them, self-positioning and environmental perception are prerequisites, and planning and control both need to be based on perception capabilities to achieve autonomy. At present, the positioning technology of coal mining machines has been widely studied, and technologies such as automatic north search represented by LASC inertial navigation systems, GIS global positioning, strapdown inertial navigation, encoders combined with total station traverse point measurement methods to achieve absolute positioning have been formed. However, they still face cumbersome initial alignment, zero-speed correction, and station setting processes, and the total station traverse point cannot be used in non-line-of-sight conditions, which reduces the accuracy and reliability of long-term applications.

[0003] In addition, in terms of cutting planning, the current coal mining robots mainly carry out cutting planning based on the prior model obtained by pre-mining geological exploration and modeling, including the use of drilling, seismic, production, coal and rock layer detection data to build a high-precision three-dimensional dynamic geological model and the use of laser scanning equipment, cutting trajectory and top and bottom coal thickness data to correct the initial coal seam three-dimensional model, etc. However, the common defect of these methods is that they can only use the "sub-meter" geological model built before mining to design and control cutting actions, and the geological model lacks an effective dynamic correction mechanism. At present, the on-the-spot perception method of coal mining robots is mainly based on the tactile perception method of three-dimensional seismic monitoring, which specifically refers to the pre-arrangement of seismic sensors and explosive explosion points in the tunnels at both ends of the coal mining face. During the cutting process, the explosion points excite seismic waves, and the seismic sensors collect vibration information. The control background uses the "multi-point measurement + interpolation prediction" method for interpretation. The accuracy and efficiency of this method are still difficult to meet the needs of real-time cutting planning of coal mining robots. Summary of the invention

[0004] The purpose of the present invention is to provide a coal mining robot with integrated exploration and machinery and an autonomous navigation operation method thereof, to realize non-contact autonomous precise scene and geological modeling of the coal mining robot during the mining process, and to propose a coal mining robot map construction method and a control scheme for autonomous face cutting.

[0005] To achieve the above-mentioned purpose, the present invention provides a machine-exploration integrated coal mining robot, comprising a coal mining machine body and a remote monitoring terminal, the remote monitoring terminal communicates with the coal mining robot through an underground wireless network, and the coal mining robot body is provided with a positioning and attitude module, a geological scene integrated perception module, a body motion state perception module, a decision-making and planning module, and a joint control module, and the positioning and attitude module and the geological scene integrated perception module share data from external sensors of the coal mining machine body;

[0006] The positioning and attitude module uses the laser radar and camera installed outside the coal mining machine body and the IMU installed inside the coal mining robot. Based on the multi-sensor fusion SLAM technology, the coal mining robot is accurately positioned along the cutting direction of the coal mining face in the coal mine. At the same time, the natural semantic beacons and artificial beacons deployed in the chute on both sides of the coal mining face are identified to correct the position and attitude of the coal mining robot. The multi-sensor fusion SLAM adopts a geological-scene joint SLAM positioning strategy, and uses the geological features and scene features of the coal mining face to achieve positioning, improve the positioning accuracy in harsh environments, and finally obtain the absolute spatial geographic coordinates in the GIS system;

[0007] The geological scene integrated perception module uses the ground-penetrating radar, laser radar, and high-definition camera installed on the outside of the coal mining machine body to obtain the geological information inside the coal mining face through the ground-penetrating radar, and captures the scene information of the coal mining site such as three-dimensional space information, object color and texture through the laser radar and high-definition camera to construct the geological point cloud model and scene point cloud model;

[0008] The main body motion state perception module includes the coal mining robot main shaft encoder, the coal mining robot front and rear roller shaft encoder, the coal mining robot body and rocker arm connection shaft encoder and the IMU arranged on the rocker arm. The coal mining robot main shaft encoder is used to detect the rotation speed and direction of the main shaft in real time to obtain the traction speed of the coal mining robot; the coal mining robot front and rear roller shaft encoders are used to directly detect the speed of the cutting drum; the coal mining robot body and rocker arm connection shaft encoder is used to measure the swing angle and tilt angle information of the rocker arm and provide absolute angle information; the rocker arm IMU can obtain the relative angle change and tilt angle of the rocker arm; when calculating the rocker arm posture, the initial rocker arm angle is obtained by using the shaft encoder at the connection between the coal mining robot body and the rocker arm as the reference for the posture, and the relative angle change data and tilt angle of the IMU are used for real-time posture update, and the encoder data is used for correction to obtain accurate rocker arm posture information. The main body motion state perception module is used in the subsequent decision-making and planning module to control the current coal mining machine traction speed, cutting drum speed and rocker arm posture according to geological and scene conditions;

[0009] The decision-making planning module controls the speed of the cutting drum of the coal mining machine, the posture of the rocker arm and the traction speed of the coal mining machine body according to the local cost map. The local cost map is a navigation map constructed and visualized by a probabilistic rasterized map generated by a geological point cloud model and a scene point cloud model at a remote monitoring terminal after probabilistic rasterization operations such as two-dimensional plane projection and occupancy probability calculation;

[0010] The joint control module uses the current posture information of the coal mining machine body and the initial position information of the hydraulic support known in the GIS system to jointly control the next position action of the hydraulic support and the advancement of the coal mining machine track in the direction perpendicular to the coal mining working face according to the global cost map. The global cost map is a navigation map constructed and visualized by a probabilistic rasterized map generated by probabilistic rasterization operations such as two-dimensional plane projection and occupancy probability calculation on the geological point cloud model and the scene point cloud model at the remote monitoring terminal.

[0011] An autonomous navigation operation method of a machine-exploration integrated coal mining robot comprises the following steps:

[0012] S1. Importing geological scene prior information: Importing the existing GIS+BIM system into the remote monitoring terminal, including the position information of the top plate and bottom plate of the working face in the geographic coordinate system and the initial position information of the hydraulic support in the geographic coordinate system;

[0013] S2, absolute position and posture estimation of coal mining robot: using the positioning and posture determination module to obtain the absolute spatial geographic coordinates of the coal mining machine body in GIS;

[0014] S3, synchronous perception of geological scene environment: using the geological scene integrated perception module to obtain the geological information inside the coal mining face and the scene information of the coal mining operation site and output it to the decision-making planning module;

[0015] S4. Geological and scene integrated modeling: Combined with the short-term estimation of the robot's motion state provided by inertial sensors, odometers and other information, the original data distortion caused by the movement of the coal mining machine in the geological scene integrated perception module is corrected, the timestamp and position relationship of the lidar and high-definition camera are calibrated to ensure that the data captured by the lidar and high-definition camera are consistent in time and space, and a heterogeneous sensor feature set is constructed. The features in the high-definition camera image are matched with the features in the lidar point cloud to achieve the fusion of the two data. Based on the fused point cloud and image data, a point cloud model of the work site scene is constructed through voxel grids, octrees and other methods. At the same time, the geological information provided by the ground penetrating radar is used to construct a geological point cloud model;

[0016] S5. Coal mining robot planning and control: The decision-making planning module controls the speed of the coal mining machine's cutting drum, the posture of the rocker arm, and the traction speed of the coal mining machine body according to the local cost map constructed by the probabilistic rasterized special navigation map generated by the geological point cloud model and the scene point cloud model after probabilistic rasterization operations such as two-dimensional plane projection and occupancy probability calculation; the joint control module uses the current posture information of the coal mining machine body and the initial position information of the hydraulic support known in the GIS system to jointly control the next position action of the hydraulic support and the advancement of the coal mining machine track in the direction perpendicular to the coal mining working face according to the global cost map constructed by the probabilistic rasterized special navigation map.

[0017] Preferably, in S2, natural features in the chute on both sides of the coal mining face or equipment motors and fire extinguisher cabinets with low movement frequency are used as natural semantic beacons. The point cloud and image data of the natural semantic beacons are collected by lidar and high-definition cameras to make labels, and a natural semantic beacon data set is constructed. The data set is used to train a six-degree-of-freedom pose estimation network. A well-trained six-degree-of-freedom pose estimation network can be used to analyze and calculate the pose of the coal mining robot in the beacon coordinate system.

[0018] Preferably, the absolute position and posture estimation of the coal mining robot in S2 comprises the following steps:

[0019] S21. The positioning and attitude module unifies the data of the laser radar, camera and IMU to the same time and coordinate system through time synchronization and space calibration to ensure data consistency. During the data acquisition stage, the laser radar continuously generates a three-dimensional point cloud, the camera will capture images synchronously, and the IMU will continuously collect the acceleration and angular velocity information of the coal mining robot to reflect its instantaneous motion state. The IMU pre-integration factor technology is used to integrate these data between adjacent key frames to obtain the cumulative posture change between two frames. These data are subjected to steps such as denoising, filtering and distortion correction to ensure the accuracy and reliability of the data. The positioning and attitude module will extract the geometric features in the laser radar point cloud and the texture features in the camera image, and perform multi-sensor data fusion, and realize the relative posture estimation of the coal mining robot through the matching and registration technology of feature points.

[0020] S22, the positioning and attitude module constructs a learning-based ground-penetrating radar sensor model through deep learning methods, maps non-sequential ground-penetrating radar image pairs to relative robot motion, and thus completes the positioning of the coal mining robot using the geological features of the coal mining face. The learning-based ground-penetrating radar sensor model is trained to predict the relative posture transformation between the two sub-images according to the actual situation of the coal mining face, that is, for a well-trained ground-penetrating radar sensor model, when the model input is two ground-penetrating radar images, the output is the posture transformation of the positions corresponding to the two ground-penetrating radar images, and then the factor graph optimization framework is fused with the positioning results of the multi-sensor fusion SLAM in S21 to achieve geological-scene joint SLAM positioning.

[0021] S23. The positioning and attitude determination module uses the point cloud and image information collected by the laser radar and high-definition camera to perform instance-level semantic segmentation and six-degree-of-freedom attitude estimation on the natural semantic beacons and artificial beacons, obtain the coal mining robot's attitude in the beacon coordinate system, and combine the geographic coordinates of the natural semantic beacons and artificial beacons in GIS to solve the absolute spatial geographic coordinates of the coal mining robot in GIS through coordinate transformation.

[0022] S24. The final positioning and attitude determination module integrates the lidar odometer factor, camera odometer factor, IMU pre-integration factor, ground penetrating radar positioning factor, and beacon positioning factor based on the factor graph optimization framework to achieve absolute positioning of the coal mining robot in the GIS coordinate system.

[0023] Preferably, the geological scene environment perception in S3 includes the following steps:

[0024] S31, obtaining the position and attitude external parameters between multiple antennas of the ground penetrating radar by pre-calibration or online estimation, and realizing the clock synchronization of each signal transmitted by each antenna based on hardware synchronization triggering;

[0025] S32, using the robot real-time posture data obtained by the positioning and posture module combined with the antenna external parameter relationship to obtain the spatial position of each data of the antenna, and based on the time synchronization principle, obtain the precise spatial position information of each data on the B-Scan image of each antenna channel;

[0026] S33, performing linear interpolation between two adjacent data in each survey line direction to obtain a B-scan image of each channel, and performing linear interpolation using the data of the channel with the closest distance between the channels to obtain scanning data of any cross section in the direction perpendicular to the survey line;

[0027] S34. Based on the full waveform inversion and anomaly recognition method, identify the coal-rock medium and anomaly type of each channel B-Scan, construct a three-dimensional geological image using the channel B-Scan volume data containing the coal-rock medium and anomaly type information, convert the three-dimensional geological image into a discrete point cloud, reconstruct a geological three-dimensional point cloud model containing medium type information, and use the point cloud intensity channel to characterize the medium type;

[0028] S35. Using the visible light images obtained by the high-definition camera and the infrared images obtained by the infrared camera, the coal-rock interface recognition results of multi-source information fusion are obtained through deep learning image recognition and target detection methods. The medium physical properties of the geological model obtained based on the full waveform inversion and anomaly recognition method based only on the ground penetrating radar information are verified and updated, and the geological model of multi-source information joint inversion is dynamically corrected to ensure the accuracy of the geological model. At the same time, the spatial coordinates of the corresponding coal-rock interface are obtained by using the positioning information of the positioning and attitude module and the observation model of the high-definition camera, lidar and other sensors;

[0029] S36. Use the scene point cloud model and the geological point cloud model to fit the air-coal rock medium surface respectively, cross-validate the accuracy of the two types of models based on the spatial continuity of the boundary area, and use the high-precision elevation model of the scene point cloud model to correct the geological point cloud model to achieve dynamic optimization.

[0030] Preferably, the geological scene integrated perception module in S33 removes the reflected clutter of the gas-coal, gas-rock, and coal-rock interfaces based on the mean filtering method, and suppresses the coal-rock detection clutter signal in the air coupling process of the ground penetrating radar.

[0031] Preferably, the geological scene integrated perception module in S4 performs time matching through a time sliding window to construct an artificial synthetic image combining heterogeneous multi-scale features, and the constructed artificial synthetic image is displayed through a remote monitoring terminal.

[0032] Preferably, the geological and scene point cloud information provided by the geological scene integrated perception module in S4 will participate in the construction of the coal mining robot's probabilistic rasterized navigation map at the remote monitoring terminal. The specific construction steps include:

[0033] A. Construct a new four-channel point cloud data storage format for characterizing geological point cloud models, and redesign the intensity information into data channels for different media types such as gas-coal-rock;

[0034] B. Construct the non-fully mechanized mining face area and non-equipment area in the scenario model as free space, where the shearer track and hydraulic support can perform reasonable advancement. Construct the coal-type medium geological model area as a cutting free space, where the shearer cutting drum can cut freely, and construct the rock and geological anomaly area as an occupied space, where the shearer cutting drum cannot cut;

[0035] C. Provide a low-resolution rough map for the high-level planning of the coal mining robot body, i.e., the adjustment and visualization of the displacement of the coal mining machine track and hydraulic support, and provide a high-resolution fine map for the local planning of the cutting part, i.e., the speed of the coal mining machine cutting drum, the rocker arm posture, and the traction speed adjustment of the coal mining machine body. Then, through cost allocation, a global and local cost map is constructed for the high-level planning and local planning of the coal mining robot. In the global cost map, as the coal mining operation proceeds, the scene model is continuously updated. In the global cost map, the distance between the coal mining machine track and the comprehensive mining face continues to increase due to the shedding of the coal seam. At this time, the coal mining machine track and hydraulic support need to be pushed in a direction perpendicular to the comprehensive mining face, and the pushing amount is affected by the single cutting depth of the coal mining machine. In the local cost map, the geological anomalies and rocks that are not suitable for cutting in the geology that are identified are used as obstacles and expanded with different radii according to different danger levels, so as to design cost maps with different safety levels for the planning of the machine body movement and cutting process.

[0036] D. The local cost map and the global cost map are generated and visualized at the remote monitoring terminal, and the local cost map is sent to the decision planning module and the global cost map is sent to the joint control module.

[0037] Preferably, the decision planning module in S5 includes the following steps:

[0038] S51, when the coal mining robot is cutting in free space, that is, in the coal-type medium area, the coal mining machine cutting drum uses a preset posture to cut freely;

[0039] S52, during the operation of the coal mining robot, the occupied space is identified, the traction speed of the coal mining robot and the cutting speed of the drum are reduced, and the height of the drum rocker arm of the coal mining robot is reduced or increased along the expansion radius direction of the obstacle, so that the drum can safely bypass the occupied space;

[0040] S53, after safely bypassing the occupied space, re-entering the free space for cutting, the decision-making and planning module adjusts the coal mining machine drum back to the preset posture for free cutting.

[0041] Preferably, the S5 joint control module includes the following steps:

[0042] S51. During the coal mining operation, the current cutting drum speed and rocker arm posture information of the coal mining machine are obtained by the self-state perception module, and the spatial geographic coordinates of the coal mining robot at the current time in GIS are obtained by the positioning and posture determination module. Combined with the initial spatial geographic coordinates of the hydraulic support known in GIS, the joint control module controls the front beam guard of the hydraulic support to automatically retract when the cutting drum of the coal mining robot approaches, so as to prevent the front beam guard of the hydraulic support from being damaged during the coal mining robot drum cutting operation;

[0043] S52, when the coal mining robot gradually moves away, in the post-mining area of ​​the coal mining robot on the coal mining face, the joint control module advances the coal mining machine track in a direction perpendicular to the working face in the free space according to the global cost map, and the advancement amount takes into account the cutting depth and maintains a certain safe distance from the working face to ensure the stable operation of the coal mining machine. At the same time, the joint control module controls the hydraulic support in the post-mining area to advance in a direction perpendicular to the working face as a whole and support the front beam guard, and the advancement amount meets the safe distance between the coal mining machine and the hydraulic support;

[0044] S53. During the coal mining operation, the above process is repeated repeatedly.

[0045] Therefore, the present invention adopts the above-mentioned mechanical exploration integrated coal mining robot and its autonomous navigation operation method, which has the following beneficial effects:

[0046] 1. Use the geological characteristics of the coal mining face to optimize the multi-sensor fusion SLAM positioning, improve the positioning accuracy, and realize the geological-scene joint SLAM positioning method. At the same time, combine the natural semantic beacon positioning method to realize the transformation of the coal mining robot from relative position positioning to absolute position positioning in the GIS system.

[0047] 2. Use multiple sensors to achieve synchronous autonomous perception modeling of scenes and geology, and obtain on-site scene information of coal mining operations and geological information of coal seams to be mined during the mining process.

[0048] 3. Based on the local cost map, the coal mining robot can realize autonomous navigation planning and cutting of the coal mining face.

[0049] 4. Based on the global cost map, complete the autonomous navigation and advancement of the coal mining robot track and hydraulic support, and realize the joint autonomous control of the coal mining robot, the coal mining robot track and the hydraulic support.

[0050] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a structural schematic diagram of an embodiment of a mechanical-exploration integrated coal mining robot of the present invention;

[0052] Figure 2 It is the overall flow chart of the present invention;

[0053] Figure 3 This is a flow chart of geological scene environment perception of the present invention;

[0054] Figure 4 Schematic diagram of interpolation processing of bottom penetrating radar data of the present invention, a is a single-channel B-scan obtained by two ground penetrating radar antennas along the survey line direction, b is a schematic diagram of the effect of linear interpolation using the data of the closest channel between the channels, c is a schematic diagram of the effect of linear interpolation between two adjacent data in the survey line direction using the linear interpolation method, and d is the two measured data in a;

[0055] Figure 5 This is a schematic diagram of the motion control of the coal mining robot of the present invention;

[0056] Figure 6 This is a schematic diagram of the action of the hydraulic support of the present invention;

[0057] Figure 7 This is a schematic diagram of the robot operation scene of the present invention;

[0058] Figure 8 This is a schematic diagram of positioning based on geological features. In the figure, a is the coal mining face, b is the significant geological features such as gangue and hard rock, c is the scanning position of the ground penetrating radar at a certain moment, d is the sub-image obtained at that moment, e is the scanning position of the ground penetrating radar at the next moment, f is the sub-image obtained at that moment, h is the ground penetrating radar sensor model based on learning, and g is the pose transformation result;

[0059] Fig. 9 The structure diagram of the learning-based ground penetrating radar sensor model constructed;

[0060] Reference numerals

[0061] 1. Ground penetrating radar; 2. LiDAR; 3. Infrared camera; 4. High-definition camera; 5. Positioning and attitude module; 6. Geological scene integrated perception module; 7. Body motion state perception module; 71. Front drum shaft 72. Rear drum shaft; 73. Encoder; 74. Rear fuselage and rocker arm connecting shaft; 75. Front fuselage and rocker arm connecting shaft 8. Decision-making and planning module; 9. Joint control module; 10. Coal mining machine body; 11. Remote monitoring terminal; 12. Ground penetrating radar antenna; 13. Working face top plate; 14. Working face bottom plate; 15. Expansion radius; 16. Abnormal body; 17. Hydraulic support front beam guard; 18. Hydraulic support; 19. Coal mining machine track; 20. Natural semantic beacon; 21. Artificial beacon. DETAILED DESCRIPTION

[0062] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0063] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0064] Example

[0065] See also Figure 1-9 The present invention provides a machine-exploration integrated coal mining robot, including a coal mining machine body 10 and a remote monitoring terminal 11. The remote monitoring terminal 11 communicates with the coal mining robot 10 through an underground wireless network. The coal mining robot body 10 is provided with a positioning and attitude module 5, a geological scene integrated perception module 6, a body motion state perception module 7, a decision-making and planning module 8 and a joint control module 9. The positioning and attitude module and the geological scene integrated perception module share data from external sensors of the coal mining machine body.

[0066] The positioning and attitude module 5 uses the laser radar 2 and camera 4 installed on the outside of the coal mining machine body and the IMU installed inside the coal mining robot to realize the precise positioning of the coal mining robot along the cutting direction at the coal mining face in the coal mine based on the multi-sensor fusion SLAM technology. At the same time, it identifies the natural semantic beacons 20 and artificial beacons 21 deployed in the chute on both sides of the coal mining face to realize the posture correction of the coal mining robot. The multi-sensor fusion SLAM adopts a geological-scene joint SLAM positioning strategy, and uses the geological characteristics and scene characteristics of the coal mining face to achieve positioning, thereby improving the positioning accuracy in harsh environments and finally obtaining the absolute spatial geographic coordinates in the GIS system.

[0067] The geological scene integrated perception module 6 utilizes the ground penetrating radar 1, the laser radar 2, and the high-definition camera 4 installed on the outside of the coal mining machine body to obtain the geological information inside the coal mining face through the ground penetrating radar, and captures the three-dimensional spatial information, object color and texture and other scene information of the coal mining site through the laser radar and the high-definition camera to construct a geological point cloud model and a scene point cloud model.

[0068] The main body motion state perception module 7 includes a coal mining robot main shaft encoder 73, a coal mining robot front drum shaft encoder 71, a coal mining machine rear drum shaft encoder 72, a coal mining robot body and rocker arm connection shaft encoder 74, and an IMU arranged on the rocker arm. The coal mining robot main shaft encoder 73 is used to detect the rotation speed and direction of the main shaft in real time to obtain the traction speed of the coal mining robot; the coal mining robot front and rear drum shaft encoders are used to directly detect the speed of the cutting drum; the coal mining robot body and rocker arm connection shaft encoder 74 is used to measure the swing angle and tilt angle information of the rocker arm and provide absolute angle information; the rocker arm IMU can obtain the relative angle change and tilt angle of the rocker arm; when calculating the rocker arm posture, the initial rocker arm angle is obtained as the posture reference by using the shaft encoder at the connection between the coal mining robot body and the rocker arm, and the relative angle change data and tilt angle of the IMU are used to update the posture in real time, and the encoder data is used for correction to obtain accurate rocker arm posture information. The main body motion state perception module is used for the subsequent decision-making and planning module to adjust the current coal mining machine traction speed, cutting drum speed and rocker arm posture according to geological and scene conditions.

[0069] The decision-making planning module 8 controls the speed of the coal mining machine's cutting drum, the rocker arm posture and the traction speed of the coal mining robot body 10 according to the local cost map. The local cost map is a navigation map constructed and visualized by a probabilistic rasterized map generated by probabilistic rasterization operations such as two-dimensional plane projection and occupancy probability calculation on the geological point cloud model and the scene point cloud model at the remote monitoring terminal.

[0070] The joint control module 9 uses the current posture information of the coal mining robot body 10 and the initial position information of the hydraulic support 18 known in the GIS system to jointly control the next position action of the hydraulic support 18 and the advancement of the coal mining machine track 19 in the direction perpendicular to the coal mining working face according to the global cost map. The global cost map is a navigation map constructed and visualized by a probabilistic rasterized map generated by probabilistic rasterization operations such as two-dimensional plane projection and occupancy probability calculation on the remote monitoring terminal 11 based on the geological point cloud model and the scene point cloud model.

[0071] An autonomous navigation method for a machine-exploration integrated coal mining robot comprises: importing geological scene prior information, estimating the absolute position and posture of the coal mining robot, synchronously perceiving the geological scene environment, integrating geology and scene modeling, and planning and controlling the coal mining robot.

[0072] S1. Importing prior information of geological scenes; importing the existing GIS+BIM system into the remote monitoring terminal. The existing GIS+BIM system stores the prior three-dimensional geological model of the coal mining face obtained by drilling, seismic and other means, including the position information of the top plate 13 and the bottom plate 14 of the working face in the geographic coordinate system and the initial position information of the hydraulic support 18 in the geographic coordinate system, providing a data benchmark for subsequent planning and control.

[0073] S2, absolute position estimation of the coal mining robot; the positioning and posture module 5 uses the natural semantic beacon 20, the artificial beacon 21 and the existing underground GIS+BIM system to obtain the spatial geographic coordinates of the coal mining machine body in GIS. The arrangement of the natural semantic beacon 20 and the artificial beacon 21 is as follows: Figure 7 As shown in the figure, the laser radar 2 and camera 4 installed on the main body of the coal mining machine are used simultaneously to realize the precise positioning of the coal mining robot in the coal mine based on the multi-sensor fusion SLAM technology. In addition, in order to improve the positioning accuracy of SLAM in the dust and smoke environment, the geological characteristics of the coal mining working face are used to realize positioning, and finally the spatial geographic coordinates in the GIS system based on factor graph optimization are obtained.

[0074] The specific process of absolute pose estimation of coal mining machines includes:

[0075] S21, the positioning and attitude module 5 unifies the data of the laser radar 2, camera 4 and IMU to the same time and coordinate system through time synchronization and space calibration to ensure the consistency of the data. In the data acquisition stage, the laser radar continuously generates a three-dimensional point cloud, the camera will capture images synchronously, and the IMU will continuously collect the acceleration and angular velocity information of the coal mining robot to reflect its instantaneous motion state. The IMU pre-integration factor technology is used to integrate these data between adjacent key frames to obtain the cumulative posture change between two frames. These data are subjected to steps such as denoising, filtering and distortion correction to ensure the accuracy and reliability of the data. The positioning and attitude module will extract the geometric features in the laser radar point cloud and the texture features in the camera image, and perform multi-sensor data fusion, and realize the relative posture estimation of the coal mining robot through the matching and registration technology of feature points.

[0076] S22, the positioning and posture module 5 constructs a learning-based ground-penetrating radar sensor model through a deep learning method, maps non-sequential ground-penetrating radar image pairs to relative robot motion, and thus completes the positioning of the coal mining robot by utilizing the geological features of the coal mining face. The learning-based ground-penetrating radar sensor model is trained to predict the relative posture transformation between the two sub-images according to the actual situation of the coal mining face. That is, for a well-trained ground-penetrating radar sensor model, when the model input is two ground-penetrating radar images, the output is the posture transformation of the positions corresponding to the two ground-penetrating radar images, and then the positioning results of the multi-sensor fusion SLAM in S21 are fused through the factor graph optimization framework to achieve geological-scene joint SLAM positioning. The model structure is as follows: Fig. 9 As shown in Figure 1, it includes ResNet-18 autoencoder, average pooling layer, spatial correlation layer, fully connected layer, and Huber loss function. Figure 8 As shown in d and f, two similar ground penetrating radar images collected by the ground penetrating radar have been preprocessed by removing clutter and artifacts. When collecting the two sub-images d and f, scenes with significant geological features should be selected, such as hard rocks in coal seams that can cause strong ground penetrating radar echo reactions. Figure 8 As shown in , at a certain moment, the ground penetrating radar scans the gangue or hard rock b at position c, and obtains the sub-image d at this moment. When the ground penetrating radar changes its posture to position e, it scans and obtains the sub-image f at this moment. The total station measures the true value of the posture transformation at two moments in this process, thereby obtaining a set of data for training the posture prediction network. According to this method, a data set for training the network can be obtained. For a well-trained learning-based ground penetrating radar model 5, when the model input is two ground penetrating radar images, such as Figure 8 In d and f, the output is the pose transformation of the corresponding positions of the two ground penetrating radar images, such as Figure 8 The constructed learning-based ground penetrating radar model is shown in Figure 6. Fig. 9As shown in the figure, the ResNet-18 autoencoder is first used to obtain the feature activation maps of the two sub-images. These feature activation maps contain relevant features for positioning, and the autoencoder is trained using the L1 loss function. Then, average pooling is performed on the row standard deviation of the sub-image image to check the significant features. If the two sub-images contain valid features, a linear correlation network is used to check whether they share common features. If the sub-images contain significant features and are related, a two-stage method is used to predict the relative pose transformation of the two sub-images. The two-stage method first constructs a set of cost curves by comparing each feature map, and then constructs an argmax element vector from each cost curve, which is provided to a fully connected regression network to identify the relative pose transformation of the two sub-images. Finally, the Huber loss function is used to train the pose change prediction network in combination with the real sub-image pose transformation values ​​collected by the total station. S23, the positioning and posture module 5 uses the natural features in the chute on both sides of the coal mining face or the equipment with low moving frequency as the natural semantic beacon 20, collects the point cloud and image data of the natural semantic beacon 20 through the laser radar 2 and the high-definition camera 4 to make labels, and constructs the natural semantic beacon 20 data set. In addition, based on the characteristic analysis of the laser radar 2 and the high-definition camera 4, artificial beacons 21 of the laser radar 2 reflection target, visual target and other types are arranged, and are staggered and coordinated with the above-mentioned natural semantic beacons 20, so that the natural semantic beacon 20 or artificial beacon 21 can be identified during the mining process and when the coal mining robot reaches the two ends of the chute of the working face. Based on the existing underground GIS+BIM system, the natural semantic beacon 20 and the artificial beacon 21 are added to the BIM model and stored, and the position and posture information of the natural semantic beacon 20 and the artificial beacon 21 in the geographic coordinate system are obtained by the total station measurement method and stored in the GIS+BIM system. After the coal mining robot moves to the two ends of the working face chute, the positioning and posture determination module 5 uses the point cloud and image information collected by the laser radar 2 and the high-definition camera 4 to perform instance-level semantic segmentation and six-degree-of-freedom posture estimation on the natural semantic beacon 20 and the artificial beacon 21, and obtains the coal mining robot posture in the beacon coordinate system. Combined with the geographic coordinates of the natural semantic beacon 20 and the artificial beacon 21 in GIS, the spatial geographic coordinates of the coal mining robot in GIS are solved through coordinate transformation.

[0077] S24. The final positioning and attitude determination module integrates the lidar odometer factor, camera odometer factor, IMU pre-integration factor, ground penetrating radar positioning factor, and beacon positioning factor based on the factor graph optimization framework to achieve absolute positioning of the coal mining robot in the GIS coordinate system.

[0078] S3, geological scene environment perception; the geological scene integrated perception module 6 uses the information provided by sensors such as ground penetrating radar 1, laser radar 2, infrared camera 3, and high-definition camera 4 to perform relevant calculations to obtain the geological information inside the coal mining face and the scene information of the coal mining operation site and output it to the decision-making planning module 8. The overall process of this part is shown as follows Figure 3 shown.

[0079] First, the geological scene integrated perception module 6 removes the reflected clutter of the gas-coal, gas-rock, and coal-rock interfaces based on the mean filtering method, thereby suppressing the coal-rock detection clutter signal in the air coupling process of the ground penetrating radar, and performs linear interpolation between the B-scans of each channel line obtained by the multi-channel ground penetrating radar, such as Figure 4 As shown, by performing linear interpolation between the B-scans of each channel survey line, the geological information of the blank area between the survey lines can be supplemented, the amount of information can be enriched, the position and shape of the abnormal body can be estimated by full waveform inversion, the medium interface information can be obtained, and the type of the abnormal body can be determined by the abnormal body recognition network, so as to realize the recognition of the abnormal body in the typical geological structure of coal and rock. Then, the medium interface information obtained is combined with the point cloud intensity information to characterize the medium type and realize the three-dimensional reconstruction of the geological point cloud model. Secondly, the geological scene integrated perception module 6 obtains the scene laser model, and the initial geological model and initial scene model in the same geographic coordinate system in the GIS system are obtained by combining the coordinates of the coal mining machine in the GIS system output by the positioning and attitude module 5. The infrared camera 3, the high-definition camera 4, and the laser radar 2 are used to obtain the infrared image, visible light image, and laser point cloud intensity image of the exposed surface to construct multi-source information, and the coal-rock interface is identified based on the deep learning method, and the medium physical properties of the above geological point cloud model are verified and updated. At the same time, the geological scene integrated perception module 6 uses the scene laser model to correct the two-way propagation time of each data on the ground penetrating radar 1 image in the air, solving the problem of inaccurate modeling of the geological point cloud model due to the uneven transition surface caused by the mining and stripping process and the different propagation time of the ground penetrating radar 1 in the air medium, and realizing the dynamic correction of the geological point cloud model in the cutting process. The corrected geological point cloud model will update the prior three-dimensional geological model of the coal mining face in the existing GIS+BIM system in real time, facilitating the next round of cutting planning.

[0080] The specific steps of synchronous perception of geological scene environment include:

[0081] S31, obtained by pre-calibration or online estimation Figure 4 The position and attitude external parameters between multiple antennas of the ground penetrating radar are used to realize the clock synchronization of each signal transmitted by each antenna based on hardware synchronization trigger.

[0082] S32: The real-time posture data of the robot obtained by the positioning and posture determination module 5 is combined with the antenna external parameter relationship to obtain Figure 4 The spatial position of each data channel of the antenna is determined, and the precise spatial position information of each data channel on the B-Scan image of each antenna channel is further obtained based on the time synchronization principle.

[0083] S33, based on the spatial position of each data, such as Figure 4 As shown in , the linear interpolation method is used to perform linear interpolation between two adjacent data in the survey line direction to obtain the B-Scan image of each channel, which can supplement the blank area between the two adjacent data and enrich the B-Scan information of a single channel. Similarly, the linear interpolation is performed using the data of the closest distance between channels to supplement the blank area information between two adjacent survey lines, and the scanning data of any cross section in the direction perpendicular to the survey line can be obtained.

[0084] S34. Based on the full waveform inversion and anomaly recognition method, the coal-rock medium and anomaly type of each channel B-Scan are identified, and the channel B-Scan volume data containing the coal-rock medium and anomaly type information is used to construct a three-dimensional geological image. The three-dimensional geological image is converted into a discrete point cloud, and a geological three-dimensional point cloud model containing medium type information is reconstructed. The point cloud intensity channel is used to characterize the medium type, that is, the point cloud intensity channel of the anomaly that is not suitable for cutting in the coal and coal seam is different.

[0085] S35. Using the visible light images and infrared images obtained by the high-definition camera 4 and the infrared camera 3, the coal-rock interface recognition result of multi-source information fusion is obtained through deep learning image recognition and target detection methods. The medium physical properties of the geological model obtained based on the full waveform inversion and anomaly recognition method based only on the ground penetrating radar information are verified and updated, and the dynamic correction of the geological model by the joint inversion of multi-source information is realized to ensure the accuracy of the geological model. At the same time, the positioning information of the positioning and attitude determination module and the observation models of sensors such as high-definition cameras and lidars are used to obtain the spatial coordinates of the corresponding coal-rock interface.

[0086] S36. Use the scene point cloud model and the geological point cloud model to fit the air-coal rock medium surface respectively, cross-validate the accuracy of the two types of models based on the spatial continuity of the boundary area, and use the high-precision elevation model of the scene point cloud model to correct the geological point cloud model to achieve dynamic optimization.

[0087] S4. Integrated modeling of geology and scenes; combining the short-term estimation of the robot's motion state provided by inertial sensors, odometers and other information, correct the distortion of the original data of the geological scene integrated perception module caused by the movement of the coal mining machine, calibrate the timestamp and position relationship of the lidar 2 and the high-definition camera 4, ensure that the data captured by the lidar and the high-definition camera are consistent in time and space, build a heterogeneous sensor feature set, match the features in the high-definition camera image with the features in the lidar point cloud to achieve the fusion of the two data, and build a point cloud model of the work site scene based on the fused point cloud and image data through voxel grids, octrees and other methods. At the same time, use the geological information provided by the ground penetrating radar 1 to build a geological point cloud model.

[0088] For the features obtained by different sensors in the same time period, due to the correlation in time and space, the geological scene integrated perception module 6 can perform time matching through a time sliding window to construct an artificial synthetic image combining heterogeneous multi-scale features, thereby realizing data association of pixel-level alignment of ground penetrating radar 1, lidar 2, visible light and deep infrared vision on the artificial synthetic image. The constructed artificial synthetic image is displayed through a remote monitoring terminal so that the staff can monitor the working status of the coal mining robot in real time and make emergency measures in time.

[0089] The geological and scene point cloud information provided by the geological scene integrated perception module 6 will participate in the construction of the coal mining robot's probabilistic rasterized navigation map and cost map at the remote monitoring terminal 11. The specific construction steps include:

[0090] S41. Construct a new four-channel point cloud data storage format to characterize geological point cloud models. Based on the spatial three-dimensional coordinates and intensity information, redesign the intensity information into data channels for reacting to different media types such as gas, coal and rock, and realize consistent storage and integrated display of geological and scene point cloud models.

[0091] S42. Based on the idea of ​​building a three-dimensional grid map (Octomap) with reference to octrees, the non-comprehensive mining face area and non-equipment area in the scene model are constructed as free space, and the shearer track and hydraulic support can perform reasonable propulsion actions in the free space. The coal-type medium geological model area is constructed as a cutting free space, and the shearer cutting drum can cut freely. The rock and geological anomaly area is constructed as an occupied space, and the shearer cutting drum cannot cut. A three-dimensional grid map is constructed based on a probabilistic representation method, and the navigation map is updated and expanded in a probabilistic manner to achieve efficient storage, strong compression, and fast transmission.

[0092] S43. In order to meet the different requirements of the robot's body planning, cutting planning, and visualization during autonomous navigation, a multi-resolution navigation map is constructed based on a three-dimensional grid map. A low-resolution rough map is provided for the high-level planning of the coal mining robot body, i.e., the adjustment and visualization of the displacement of the coal mining machine track and hydraulic support. A high-resolution fine map is provided for the local planning of the cutting part, i.e., the speed of the coal mining machine cutting drum, the rocker arm posture, and the traction speed adjustment of the coal mining machine body. Then, through cost allocation, global and local cost maps are constructed for the high-level planning and local planning of the coal mining robot. In the global cost map, as the coal mining operation proceeds, the scene model is continuously updated. In the global cost map, the distance between the coal mining machine track and the fully mechanized mining face continues to increase due to the shedding of the coal seam. At this time, the coal mining machine track and hydraulic support need to be pushed in a direction perpendicular to the fully mechanized mining face, and the pushing amount is affected by the single cutting depth of the coal mining machine. In the local cost map, the geological anomalies and rocks that are not suitable for cutting in the geology that are identified are used as obstacles and expanded with different radii according to different danger levels. Cost maps of different safety levels are designed for the planning of the machine movement and cutting process.

[0093] S44, the local cost map and the global cost map are generated and visualized at the remote monitoring terminal, and the local cost map is sent to the decision planning module and the global cost map is sent to the joint control module.

[0094] S5. Coal mining robot planning control refers to the decision-making planning module 8 controlling the speed of the coal mining machine cutting drum, the rocker arm posture and the traction speed of the coal mining machine body 10 according to the local cost map. The joint control module 9 uses the current posture information of the coal mining machine body 10 and the initial position information of the hydraulic support 18 known in the GIS system to jointly control the next position action of the hydraulic support 18 and the advancement amount of the coal mining machine track 19 in the direction perpendicular to the coal mining working face according to the global cost map.

[0095] The specific motion control process of decision-making planning module 8 is as follows:

[0096] S51. When the coal mining robot is cutting in free space, that is, in the coal-type medium area, the coal mining machine cutting drum uses a preset posture to cut freely.

[0097] S52, when the coal mining robot is running, it is identified that the space is occupied, that is, it is identified that there are hard rocks in the geology, etc., the traction speed of the coal mining robot and the cutting speed of the drum are reduced, such as Figure 5 As shown in the figure, the coal mining robot drum lowers or raises the height of the drum rocker arm along the expansion radius 15 of the obstacle 16, so that the drum can safely bypass the occupied space. Different expansion radii are set for abnormal bodies of different danger levels. The higher the danger level, the larger the expansion radius.

[0098] S53, after safely bypassing the occupied space, entering the free space for cutting again, the decision-making planning module 8 adjusts the coal mining machine drum back to the preset posture for free cutting.

[0099] The specific motion control process of the joint control module 9 is as follows:

[0100] S51, during the coal mining operation, the current cutting drum speed and rocker arm posture information of the coal mining machine is obtained by the self-state sensing module 7, and the spatial geographic coordinates of the coal mining robot at the current time in GIS are obtained by the positioning and posture determination module 5. Combined with the initial spatial geographic coordinates of the hydraulic support known in GIS, the joint control module 9 controls the front beam guard of the hydraulic support to automatically retract when the cutting drum of the coal mining robot approaches, so as to prevent the front beam guard of the hydraulic support from being damaged during the coal mining robot drum cutting operation;

[0101] S52, when the coal mining robot gradually moves away, in the post-mining area of ​​the coal mining robot on the coal mining face, the joint control module 9 advances the coal mining machine track in a direction perpendicular to the working face in the free space according to the global cost map, and the advancement amount takes into account the cutting depth and maintains a certain safe distance from the working face to ensure the stable operation of the coal mining machine. At the same time, the joint control module 9 controls the hydraulic support in the post-mining area to advance in a direction perpendicular to the working face as a whole and support the front beam guard, and the advancement amount meets the safe distance between the coal mining machine and the hydraulic support;

[0102] S53. During the coal mining operation, the above process is repeated repeatedly.

[0103] Therefore, the present invention adopts the above-mentioned integrated mechanical exploration coal mining robot and its autonomous navigation method to realize non-contact autonomous precise scene and geological modeling of the coal mining robot during the mining process. The process is simple, with less manual operation, high precision and high efficiency, and truly realizes the transformation from automatic memory cutting coal mining machine to autonomous perception and planning coal mining robot.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A coal mining robot with integrated mechanical exploration, characterized in that: It includes a coal mining robot and a remote monitoring terminal. The remote monitoring terminal communicates with the coal mining robot through an underground wireless network. The coal mining robot body is provided with a positioning and attitude module, a geological scene integrated perception module, a body motion state perception module, a decision-making and planning module, and a joint control module. The positioning and attitude module and the geological scene integrated perception module share data from external sensors of the coal mining machine body. The positioning and attitude module uses the laser radar and camera installed on the outside of the coal mining machine body and the IMU installed inside the coal mining robot. Based on the multi-sensor fusion SLAM technology, it realizes the precise positioning of the coal mining robot along the cutting direction of the coal mining face in the coal mine. At the same time, it recognizes the natural semantic beacons and artificial beacons deployed in the chute on both sides of the coal mining face to realize the position and attitude correction of the coal mining robot. It also uses the geological characteristics of the coal mining face to further improve the positioning accuracy of the coal mining robot and obtain the absolute spatial geographic coordinates of the coal mining robot in the GIS system. The geological scene integrated perception module uses the ground-penetrating radar, laser radar, and high-definition camera installed on the outside of the coal mining machine to obtain the geological information inside the coal mining face and the scene information of the coal mining operation site, so as to construct the geological point cloud model and the scene point cloud model. The main body motion state perception module includes the coal mining robot main axis encoder, the coal mining robot front and rear drum axis encoder, the coal mining robot body and rocker arm connection axis encoder and the IMU arranged on the rocker arm; the main body motion state perception module is used to obtain the coal mining robot traction speed, cutting drum speed and rocker arm posture information, so that the subsequent decision-making planning module can adjust the current coal mining machine traction speed, cutting drum speed and rocker arm posture according to geological and scene conditions; The decision-making planning module controls the speed of the shearer's cutting drum, the rocker arm posture, and the traction speed of the shearer body according to the local cost map. The local cost map is a navigation map constructed and visualized by a probabilistic rasterized map generated by the geological point cloud model and the scene point cloud model through two-dimensional plane projection, occupancy probability calculation and other probabilistic rasterization operations at the remote monitoring terminal; The joint control module uses the current posture information of the coal mining machine body and the initial position information of the hydraulic support known in the GIS system to jointly control the next position action of the hydraulic support and the advancement of the coal mining machine track in the direction perpendicular to the coal mining working face according to the global cost map. The global cost map is a navigation map constructed and visualized by a probabilistic rasterized map generated by probabilistic rasterization operations such as two-dimensional plane projection and occupancy probability calculation on the geological point cloud model and the scene point cloud model at the remote monitoring terminal.

2. An autonomous navigation operation method of a mechanical exploration integrated coal mining robot, using the mechanical exploration integrated coal mining robot according to claim 1, characterized in that: The following steps are involved: S1. Importing geological scene prior information: Importing the existing GIS+BIM system into the remote monitoring terminal, including the position information of the top plate and bottom plate of the working face in the geographic coordinate system and the initial position information of the hydraulic support in the geographic coordinate system; S2, absolute position and posture estimation of coal mining robot: using the positioning and posture module to obtain the absolute spatial geographic coordinates of the coal mining machine body in GIS; S3, synchronous perception of geological scene environment: using the geological scene integrated perception module to obtain the geological information inside the coal mining face and the scene information of the coal mining operation site and output it to the decision-making planning module; S4. Geological and scene integrated modeling: Combined with the short-term estimation of the robot's motion state provided by inertial sensors, odometers and other information, the original data distortion caused by the movement of the coal mining machine in the geological scene integrated perception module is corrected, the timestamp and position relationship of the lidar and high-definition camera are calibrated to ensure that the data captured by the lidar and high-definition camera are consistent in time and space, and a heterogeneous sensor feature set is constructed. The features in the high-definition camera image are matched with the features in the lidar point cloud to achieve the fusion of the two data. Based on the fused point cloud and image data, a point cloud model of the work site scene is constructed through voxel grids, octrees and other methods. At the same time, the geological information provided by the ground penetrating radar is used to construct a geological point cloud model; S5. Coal mining robot planning and control: The decision-making planning module controls the speed of the coal mining machine's cutting drum, the posture of the rocker arm, and the traction speed of the coal mining machine body according to the local cost map constructed by the probabilistic rasterized special navigation map generated by the geological point cloud model and the scene point cloud model after probabilistic rasterization operations such as two-dimensional plane projection and occupancy probability calculation; the joint control module uses the current posture information of the coal mining machine body and the initial position information of the hydraulic support known in the GIS system to jointly control the next position action of the hydraulic support and the advancement of the coal mining machine track in the direction perpendicular to the coal mining working face according to the global cost map constructed by the probabilistic rasterized special navigation map.

3. An autonomous navigation operation method of a mechanical exploration integrated coal mining robot according to claim 2, characterized in that: In S2, natural features in the chute on both sides of the coal mining face or equipment motors and fire cabinets with low movement frequency are used as natural semantic beacons. The point cloud and image data of the natural semantic beacons are collected by lidar and high-definition cameras to make labels, and a natural semantic beacon dataset is constructed. The dataset is used to train a six-degree-of-freedom pose estimation network. A well-trained six-degree-of-freedom pose estimation network is used to analyze and calculate the pose of the coal mining robot in the beacon coordinate system.

4. The autonomous navigation operation method of the integrated mechanical exploration coal mining robot according to claim 3 is characterized in that: The absolute pose estimation of the coal mining robot in S2 includes the following steps: S21. The positioning and attitude module unifies the data of the laser radar, camera and IMU to the same time and coordinate system through time synchronization and space calibration. In the data collection stage, the laser radar continuously generates a three-dimensional point cloud, the camera will capture images synchronously, and the IMU will continuously collect the acceleration and angular velocity information of the coal mining robot. The IMU pre-integration factor technology is used to obtain the cumulative posture change between two frames. The positioning and attitude module extracts the geometric features in the laser radar point cloud and the texture features in the camera image, and performs multi-sensor data fusion. The relative posture estimation of the coal mining robot is achieved through the matching and registration technology of feature points. S22, the positioning and attitude module constructs a learning-based ground penetrating radar sensor model through a deep learning method, maps non-sequential ground penetrating radar image pairs to relative robot motion, thereby completing the positioning of the coal mining robot using the geological features of the coal mining face; S23, the positioning and attitude module uses the point cloud and image information collected by the laser radar and high-definition camera to perform instance-level semantic segmentation and six-degree-of-freedom pose estimation on the natural semantic beacon and the artificial beacon, obtains the coal mining robot pose in the beacon coordinate system, combines the geographic coordinates of the natural semantic beacon and the artificial beacon in GIS, and solves the absolute spatial geographic coordinates of the coal mining robot in GIS through coordinate transformation; S24. The final positioning and attitude determination module integrates the lidar odometer factor, camera odometer factor, IMU pre-integration factor, ground penetrating radar positioning factor, and beacon positioning factor based on the factor graph optimization framework to achieve absolute positioning of the coal mining robot in the GIS coordinate system.

5. An autonomous navigation operation method of a mechanical exploration integrated coal mining robot according to claim 4, characterized in that: The synchronous perception of geological scene environment in S3 includes the following steps: S31, obtaining the position and attitude external parameters between multiple antennas of the ground penetrating radar by pre-calibration or online estimation, and realizing the clock synchronization of each signal transmitted by each antenna based on hardware synchronization triggering; S32, using the robot real-time posture data obtained by the positioning and posture module combined with the antenna external parameter relationship to obtain the spatial position of each data of the antenna, and based on the time synchronization principle, obtain the precise spatial position information of each data on the B-Scan image of each antenna channel; S33, performing linear interpolation between two adjacent data in each survey line direction to obtain a B-scan image of each channel, and performing linear interpolation using the data of the channel with the closest distance between the channels to obtain scanning data of any cross section in the direction perpendicular to the survey line; S34. Based on the full waveform inversion and anomaly recognition method, identify the coal-rock medium and anomaly type of each channel B-Scan, construct a three-dimensional geological image using the channel B-Scan volume data containing the coal-rock medium and anomaly type information, convert the three-dimensional geological image into a discrete point cloud, reconstruct a geological three-dimensional point cloud model containing medium type information, and use the point cloud intensity channel to characterize the medium type; S35. Using the visible light images obtained by the high-definition camera and the infrared images obtained by the infrared camera, the coal-rock interface recognition results of multi-source information fusion are obtained through deep learning image recognition and target detection methods. The medium physical properties of the geological model obtained based on the full waveform inversion and anomaly recognition method based only on the ground penetrating radar information are verified and updated, and the geological model of multi-source information joint inversion is dynamically corrected to ensure the accuracy of the geological model. At the same time, the spatial coordinates of the corresponding coal-rock interface are obtained by using the positioning information of the positioning and attitude module and the observation model of the high-definition camera, lidar and other sensors; S36. Use the scene point cloud model and the geological point cloud model to fit the air-coal rock medium surface respectively, cross-validate the accuracy of the two types of models based on the spatial continuity of the boundary area, and use the high-precision elevation model of the scene point cloud model to correct the geological point cloud model to achieve dynamic optimization.

6. An autonomous navigation operation method of a mechanical-exploration integrated coal mining robot according to claim 5, characterized in that: The integrated geological scene perception module in S33 removes the reflected clutter at the gas-coal, gas-rock, and coal-rock interfaces based on the mean filtering method, and suppresses the coal and rock detection clutter signals in the air coupling process of the ground penetrating radar.

7. An autonomous navigation operation method of a mechanical exploration integrated coal mining robot according to claim 6, characterized in that: The geological scene integrated perception module in S4 performs time matching through a time sliding window to construct an artificial synthetic image that combines heterogeneous multi-scale features. The constructed artificial synthetic image is displayed through a remote monitoring terminal.

8. An autonomous navigation operation method of a mechanical-exploration integrated coal mining robot according to claim 7, characterized in that: The geological and scene point cloud information provided by the geological scene integrated perception module in S4 will be used to build the probabilistic rasterized navigation map for coal mining robots at the remote monitoring terminal. The specific construction steps include: A. Construct a new four-channel point cloud data storage format for characterizing geological point cloud models, and redesign the intensity information into data channels for different media types such as gas-coal-rock; B. Construct the non-fully mechanized mining face area and non-equipment area in the scenario model as free space, where the shearer track and hydraulic support can perform reasonable advancement actions; construct the coal-type medium geological model area as cutting free space, where the shearer cutting drum can cut freely; construct the rock and geological anomaly area as occupied space, where the shearer cutting drum cannot cut; C. Provide a low-resolution rough map for the high-level planning of the coal mining robot body, i.e., the adjustment and visualization of the displacement of the coal mining machine track and hydraulic support, and provide a high-resolution fine map for the local planning of the cutting part, i.e., the speed of the coal mining machine cutting drum, the rocker arm posture, and the traction speed adjustment of the coal mining machine body. Then, through cost allocation, construct global and local cost maps for the high-level planning and local planning of the coal mining robot respectively. In the global cost map, the scene model is continuously updated as the coal mining operation proceeds. In the global cost map, the distance between the coal mining machine track and the comprehensive mining face continues to increase due to the shedding of the coal seam. At this time, the coal mining machine track and the hydraulic support need to be pushed in a direction perpendicular to the comprehensive mining face. The pushing amount is affected by the single cutting depth of the coal mining machine. In the local cost map, the geological anomalies and rocks that are not suitable for cutting in the geology that are identified are used as obstacles and expanded with different radii according to different danger levels. Cost maps with different safety levels are designed for the planning of the machine body movement and cutting process. D. The local cost map and the global cost map are generated and visualized at the remote monitoring terminal, and the local cost map is sent to the decision planning module and the global cost map is sent to the joint control module.

9. An autonomous navigation operation method of a mechanical exploration integrated coal mining robot according to claim 8, characterized in that: The decision planning module in S5 includes the following steps: S51, when the coal mining robot is cutting in free space, that is, in the coal-type medium area, the coal mining machine cutting drum uses a preset posture to cut freely; S52, during the operation of the coal mining robot, the occupied space is identified, the traction speed of the coal mining robot and the cutting speed of the drum are reduced, and the height of the drum rocker arm of the coal mining robot is reduced or increased along the expansion radius direction of the obstacle, so that the drum can safely bypass the occupied space; S53, after safely bypassing the occupied space, re-entering the free space for cutting, the decision-making and planning module adjusts the coal mining machine drum back to the preset posture for free cutting.

10. An autonomous navigation operation method of a mechanical exploration integrated coal mining robot according to claim 9, characterized in that: The S5 joint control module includes the following steps: S51. During the coal mining operation, the current cutting drum speed and rocker arm posture information of the coal mining machine are obtained by the self-state perception module, and the spatial geographic coordinates of the coal mining robot at the current time in GIS are obtained by the positioning and posture determination module. Combined with the initial spatial geographic coordinates of the hydraulic support known in GIS, the joint control module controls the front beam guard of the hydraulic support to automatically retract when the cutting drum of the coal mining robot approaches, so as to prevent the front beam guard of the hydraulic support from being damaged during the coal mining robot drum cutting operation; S52, when the coal mining robot gradually moves away, in the post-mining area of ​​the coal mining robot on the coal mining face, the joint control module advances the coal mining machine track in a direction perpendicular to the working face in the free space according to the global cost map, and the advancement amount takes into account the cutting depth and maintains a certain safe distance from the working face to ensure the stable operation of the coal mining machine. At the same time, the joint control module controls the hydraulic support in the post-mining area to advance in a direction perpendicular to the working face as a whole and support the front beam guard, and the advancement amount meets the safe distance between the coal mining machine and the hydraulic support; S53. During the coal mining operation, the above process is repeated repeatedly.

Images