A machine-probe integrated coal mining robot and an autonomous navigation operation method thereof

Abstract

The application discloses a machine-exploration integrated coal mining robot and an autonomous navigation operation method thereof, relates to the technical field of coal mining robots, and comprises a coal mining machine body and a remote monitoring terminal. The remote monitoring terminal is connected with the coal mining machine body. The coal mining machine body is provided with a positioning and pose setting module, a geological scene integrated perception module, a self-state perception module, a decision planning module and a joint control module. The machine-exploration integrated coal mining robot and the autonomous navigation operation method thereof are adopted, non-contact autonomous accurate scene and geological modeling of the coal mining robot in the mining process are realized, the process is simple, manual operation is less, the precision is high, the efficiency is high, and the transformation from an automatic memory cutting coal mining machine to an autonomous perception planning coal mining robot is truly realized.

Description

Technical Field

[0001] This invention relates to the field of coal mining robot technology, and in particular to an integrated mechanical exploration coal mining robot and its autonomous navigation operation method. Background Technology

[0002] From the perspective of autonomous mobile robots, coal mining robots also involve three key technologies: perception, planning, and control. Among these, self-localization and environmental perception are prerequisites, while planning and control both require perception capabilities to achieve autonomy. Currently, coal mining machine positioning technology has been extensively studied, resulting in technologies such as automatic north-finding and GIS global positioning, represented by LASC inertial navigation systems, and absolute positioning achieved by combining strapdown inertial navigation and encoders with total station traverse point measurements. However, it still faces cumbersome initial alignment, zero-speed correction, and station setup processes. Furthermore, total station traverse points cannot be used when not at line-of-sight, leading to a decrease in accuracy and reliability in long-term applications.

[0003] Furthermore, in terms of cutting planning, current coal mining robots mainly rely on prior models obtained through pre-mining geological exploration and modeling for cutting planning. This includes constructing high-precision three-dimensional dynamic geological models using data from drilling, seismic, production, and coal seam detection, as well as correcting the initial three-dimensional coal seam model using laser scanning equipment, cutting trajectories, and top and bottom coal thickness data. However, a common drawback of these methods is that they can only use the "sub-meter level" geological models constructed before mining for cutting action design and control, and the geological models lack effective dynamic correction mechanisms. Currently, the sensing methods for coal mining robots during mining are mainly based on tactile sensing methods using three-dimensional seismic monitoring. Specifically, this involves pre-positioning seismic sensors and explosive detonation points in the roadways at both ends of the coal face. During the cutting process, the explosion points generate seismic waves, the seismic sensors collect vibration information, and the control backend interprets this information using a "multi-point measurement + interpolation prediction" method. However, the accuracy and efficiency of this method still cannot meet the needs of real-time cutting planning for coal mining robots. Summary of the Invention

[0004] The purpose of this invention is to provide an integrated mechanical exploration coal mining robot and its autonomous navigation operation method, which realizes non-contact autonomous and accurate scene and geological modeling during the coal mining process, and proposes a coal mining robot map construction method and control scheme for autonomous face cutting.

[0005] To achieve the above objectives, the present invention provides an integrated mining robot, comprising a mining machine body and a remote monitoring terminal. The remote monitoring terminal communicates with the mining robot via an underground wireless network. The mining robot body is equipped with a positioning and attitude determination module, a geological scene integrated perception module, a body motion state perception module, a decision planning module, and a joint control module. The positioning and attitude determination module and the geological scene integrated perception module share data from external sensors of the mining machine body.

[0006] The positioning and attitude determination module utilizes a lidar and camera installed on the outside of the coal mining machine and an IMU installed inside the coal mining robot. Based on multi-sensor fusion SLAM technology, it achieves accurate positioning of the coal mining robot along the cutting direction in the underground coal mining face. At the same time, it identifies natural semantic beacons and artificial beacons deployed in the roadways on both sides of the coal mining face to correct the pose of the coal mining robot. The multi-sensor fusion SLAM adopts a geological-scene joint SLAM positioning strategy, and uses the geological features and scene features of the coal mining face to achieve positioning, improving positioning accuracy in harsh environments, and finally obtaining the absolute spatial geographic coordinates in the GIS system.

[0007] The integrated geological scene perception module utilizes ground-penetrating radar, lidar, and high-definition cameras installed on the outside of the coal mining machine. The ground-penetrating radar obtains geological information inside the coal mining face, and the lidar and high-definition cameras capture scene information such as three-dimensional spatial information and object color and texture at the coal mining site, which are used to construct geological point cloud models and scene point cloud models.

[0008] The robot's motion state perception module includes a main spindle encoder, front and rear drum shaft encoders, a shaft encoder at the connection between the robot body and the rocker arm, and an IMU (Integrated Unit) mounted on the rocker arm. The main spindle encoder detects the rotational speed and direction of the main spindle in real time to obtain the robot's traction speed. The front and rear drum shaft encoders directly detect the speed of the cutting drum. The shaft encoder at the connection between the robot body and the rocker arm measures the rocker arm's swing angle and tilt angle, providing absolute angle information. The rocker arm IMU obtains the relative angle change and tilt angle of the rocker arm. During rocker arm posture calculation, the initial rocker arm angle is obtained from the shaft encoder at the connection between the robot body and the rocker arm as a reference for posture. Real-time posture updates are performed using the relative angle change data and tilt angle from the IMU, while encoder data is used for correction to obtain accurate rocker arm posture information. The robot's motion state perception module is used by the subsequent decision-making and planning module to adjust the current traction speed of the coal mining machine, the speed of the cutting drum, and the rocker arm posture based on geological and scene conditions.

[0009] The decision planning module controls the speed of the cutting drum, the rocker arm posture, and the traction speed of the coal mining machine body based on the local cost map. The local cost map is a navigation map constructed and visualized by a probabilistic raster map generated on a remote monitoring terminal after two-dimensional plane projection and probability rasterization operations such as occupation probability calculation of the geological point cloud model and the scene point cloud model.

[0010] The joint control module uses the current pose information of the coal mining machine body and the initial position information of the hydraulic support known in the GIS system to jointly control the next position movement of the hydraulic support and the advance of the coal mining machine track in the direction perpendicular to the coal mining face according to the global cost map. The global cost map is a navigation map constructed and visualized by a probabilistic raster map generated by the geological point cloud model and the scene point cloud model through two-dimensional plane projection, occupancy probability calculation and other probabilistic rasterization operations on the remote monitoring terminal.

[0011] An autonomous navigation operation method for an integrated mechanical and geological coal mining robot includes the following steps:

[0012] S1. Import of geological scene prior information: Import the existing GIS+BIM system into the remote monitoring terminal, including the location information of the working face top plate and working face bottom plate in the geographic coordinate system, as well as the initial location information of the hydraulic support in the geographic coordinate system.

[0013] S2. Absolute pose estimation of coal mining robot: The absolute spatial geographic coordinates of the coal mining robot body in GIS are obtained using the positioning and attitude determination module.

[0014] S3. Synchronous perception of geological scene environment: The integrated geological scene perception module is used to obtain geological information inside the coal mining face and scene information of the coal mining operation site and output it to the decision planning module.

[0015] S4. Integrated Geological and Scene Modeling: Combining short-term estimates of robot motion states provided by inertial sensors, odometers, and other information, the original data distortion caused by the movement of the coal mining machine body in the integrated geological scene perception module is corrected. The timestamps and positional relationships of the lidar and high-definition camera are calibrated to ensure that the data captured by the lidar and high-definition camera are consistent in time and space. A heterogeneous sensor feature set is constructed, and the features in the high-definition camera images are matched with the features in the lidar point cloud to achieve the fusion of the two types of data. Based on the fused point cloud and image data, a point cloud model of the work site scene is constructed using methods such as voxel mesh and octree. At the same time, a geological point cloud model is constructed using geological information provided by ground penetrating radar.

[0016] S5. Coal Mining Robot Planning and Control: The decision-making and planning module controls the speed of the cutting drum, the rocker arm posture, and the traction speed of the coal mining machine body based on the local cost map constructed by the probabilistic rasterized navigation map generated by the geological point cloud model and the scene point cloud model through two-dimensional plane projection, occupation probability calculation, and other probabilistic rasterization operations. The joint control module uses the current pose information of the coal mining machine body and the initial position information of the hydraulic support known in the GIS system to jointly regulate the next position action of the hydraulic support and the advance amount of the coal mining machine track in the direction perpendicular to the coal mining face based on the global cost map constructed by the probabilistic rasterized navigation map.

[0017] Preferably, in S2, natural features or low-frequency moving equipment such as motors and fire cabinets in the roadways on both sides of the coal mining face are used as natural semantic beacons. Point cloud and image data of the natural semantic beacons are collected by LiDAR and high-definition cameras to create labels and construct a natural semantic beacon dataset. The dataset is used to train a six-degree-of-freedom pose estimation network. A well-trained six-degree-of-freedom pose estimation network can be used to analyze and calculate the pose of the coal mining robot in the beacon coordinate system.

[0018] Preferably, the absolute pose estimation of the coal mining robot in S2 includes the following steps:

[0019] S21. The positioning and attitude determination module unifies the data from the LiDAR, camera, and IMU into the same time and coordinate system through time synchronization and spatial calibration to ensure data consistency. During the data acquisition phase, the LiDAR continuously generates 3D point clouds, the camera synchronously captures images, and the IMU continuously acquires the acceleration and angular velocity information of the coal mining robot to reflect its instantaneous motion state. Using IMU pre-integration factor technology, these data are integrated between adjacent keyframes to obtain the cumulative pose change between two frames. These data undergo denoising, filtering, and distortion correction to ensure accuracy and reliability. The positioning and attitude determination module extracts geometric features from the LiDAR point cloud and texture features from the camera images, and performs multi-sensor data fusion. Through feature point matching and registration techniques, the relative pose of the coal mining robot is estimated.

[0020] S22. The localization and attitude determination module constructs a learning-based ground-penetrating radar (GPR) sensor model using deep learning methods. This model maps non-sequential GPR image pairs to relative robot motion, thereby enabling the coal mining robot to locate itself using the geological features of the coal face. The learning-based GPR sensor model is trained to predict the relative pose transformation between two sub-images based on the actual conditions of the coal face. That is, for a well-trained GPR sensor model, when the input is two GPR images, the output is the pose transformation of the corresponding positions in the two GPR images. This transformation is then fused with the localization results of the multi-sensor SLAM fusion in S21 through a factor graph optimization framework to achieve joint SLAM localization of the geological environment and the scene.

[0021] S23. The positioning and attitude determination module uses point cloud and image information collected by LiDAR and high-definition camera to perform instance-level semantic segmentation and six-degree-of-freedom pose estimation on natural semantic beacons and artificial beacons to obtain the pose of the coal mining robot in the beacon coordinate system. Combining the geographic coordinates of natural semantic beacons and artificial beacons in GIS, the absolute spatial geographic coordinates of the coal mining robot in GIS are solved through coordinate transformation.

[0022] S24. The final positioning and attitude determination module integrates the factors of lidar odometer, camera odometer, IMU pre-integration factor, ground-penetrating radar positioning factor, and beacon positioning factor based on the factor graph optimization framework to achieve absolute positioning of the coal mining robot in the GIS coordinate system.

[0023] Preferably, the geological scene environment perception in S3 includes the following steps:

[0024] S31. The position and attitude extrinsic parameters between multiple antennas of the ground penetrating radar are obtained by pre-calibration or online estimation, and the clock synchronization of each signal transmission of each antenna is realized based on hardware synchronization triggering.

[0025] S32. Using the robot's real-time pose data obtained by the positioning and attitude determination module and the relationship of antenna external parameters, the spatial position of each data point of the antenna is obtained. Based on the time synchronization principle, the precise spatial position information of each data point on the B-Scan image of each antenna channel is obtained.

[0026] S33. Perform linear interpolation between two adjacent data points in each survey line direction to obtain the B-scan image of each channel. Use the data point with the closest distance between channels to perform linear interpolation to obtain the scan data of any section perpendicular to the survey line direction.

[0027] S34. Based on the full waveform inversion and anomaly identification method, identify the coal and rock medium and anomaly type of each channel B-Scan, construct a three-dimensional geological image using channel B-Scan volume data containing information on coal and rock medium and anomaly type, convert the three-dimensional geological image into discrete point cloud, reconstruct a geological three-dimensional point cloud model containing medium type information, and use the point cloud intensity channel to characterize the medium type.

[0028] S35. Using visible light images obtained from a high-definition camera and infrared images obtained from an infrared camera, the coal-rock interface identification results are obtained through deep learning image recognition and target detection methods. The geological model obtained solely based on ground-penetrating radar information using full waveform inversion and anomaly identification methods is then verified and updated to achieve dynamic correction of the geological model obtained through joint inversion of multi-source information, ensuring the accuracy of the geological model. Simultaneously, the spatial coordinates of the corresponding coal-rock interface are obtained by using the positioning information of the positioning and attitude determination module and the observation model of sensors such as high-definition cameras and lidar.

[0029] S36. Fit the air-coal-rock medium surface using scene point cloud model and geological point cloud model respectively. Cross-validate the accuracy of the two models based on the spatial continuity of the boundary area. Use the elevation model of the high-precision scene point cloud model to correct the geological point cloud model to achieve dynamic optimization.

[0030] Preferably, the integrated geological scene perception module in S33 uses the mean filtering method to remove reflected clutter from the gas-coal, gas-rock, and coal-rock interfaces, and suppresses coal and rock detection clutter signals in the air coupling process of ground penetrating radar.

[0031] Preferably, the integrated geological scene perception module in S4 performs time matching through a time sliding window to construct an artificially synthesized image that combines heterogeneous multi-scale features, and the constructed artificially synthesized image is displayed through a remote monitoring terminal.

[0032] Preferably, the geological and scene point cloud information provided by the integrated geological scene perception module in S4 will participate in the construction of the probabilistic rasterized dedicated navigation map for the coal mining robot on the remote monitoring terminal. The specific construction steps include:

[0033] A. Construct a new four-channel point cloud data storage format to characterize geological point cloud models, and redesign the intensity information into a data channel that reflects different media types such as gas, coal and rock.

[0034] B. In the scenario model, the non-fully mechanized mining face area and the non-equipment area are constructed as free space, where the coal mining machine track and hydraulic support can perform reasonable advancing actions. The coal-type medium geological model area is constructed as a cutting free space, where the coal mining machine cutting drum can cut freely. The rock and geological anomaly areas are constructed as occupied spaces, where the coal mining machine cutting drum cannot perform cutting operations.

[0035] C. A low-resolution coarse map is provided for the high-level planning of the coal mining robot body, i.e., the adjustment and visualization of the movement of the coal mining machine track and hydraulic supports. A high-resolution fine map is provided for the local planning of the cutting section, i.e., the speed of the coal mining machine cutting drum, the rocker arm posture, and the traction speed adjustment of the coal mining machine body. Then, through cost allocation, global and local cost maps are constructed for the high-level and local planning of the coal mining robot, respectively. In the global cost map, as the coal mining operation progresses and the scene model is continuously updated, the distance between the coal mining machine track and the fully mechanized mining face continuously increases due to coal seam collapse. At this time, the coal mining machine track and hydraulic supports need to be advanced in a direction perpendicular to the fully mechanized mining face. The advancement amount is affected by the single cutting depth of the coal mining machine. In the local cost map, geological anomalies and rocks that are unsuitable for cutting are identified as obstacles and expanded to different radii according to different hazard levels. Cost maps with different safety levels are designed for the planning of the machine movement and cutting process.

[0036] D. The local cost map and the global cost map are generated and visualized on the remote monitoring terminal. The local cost map is sent to the decision planning module and the global cost map is sent to the joint control module.

[0037] Preferably, the decision planning module in S5 includes the following steps:

[0038] S51. When the coal mining robot is cutting in free space, that is, in a coal-type medium area, the cutting drum of the coal mining machine uses a preset posture for free cutting;

[0039] S52. During the operation of the coal mining robot, if an occupied space is detected, the traction speed of the coal mining robot and the cutting speed of the drum are reduced. The height of the drum arm of the coal mining robot is lowered or raised along the expansion radius of the obstacle so that the drum can safely bypass the occupied space.

[0040] S53. After safely bypassing the occupied space, it re-enters the cutting free space, and the decision planning module adjusts the coal mining machine drum back to the preset posture for free cutting.

[0041] Preferably, the S5 control module includes the following steps:

[0042] S51. During the coal mining operation, the current cutting drum speed and rocker arm posture information of the coal mining machine are obtained by the self-state perception module. The spatial geographic coordinates of the coal mining robot at the current moment in the GIS are obtained by the positioning and attitude determination module. Combined with the initial spatial geographic coordinates of the hydraulic support known in the GIS, the joint control module controls the front beam guard of the hydraulic support to automatically retract when the cutting drum of the coal mining robot approaches, so as to prevent the front beam guard of the hydraulic support from being damaged during the cutting operation of the coal mining robot drum.

[0043] S52. As the coal mining robot moves further away, in the post-mining area of ​​the coal mining face, the control module, based on the global cost map, advances the coal mining machine track in free space in a direction perpendicular to the working face. The advancement amount takes into account the cutting depth and maintains a certain safe distance from the working face to ensure the stable operation of the coal mining machine. Simultaneously, the control module controls the hydraulic support in the post-mining area to advance as a whole in a direction perpendicular to the working face and support the front beam sidewall, with the advancement amount meeting the safe distance between the coal mining machine and the hydraulic support.

[0044] S53. The above process shall be repeated during the coal mining operation.

[0045] Therefore, the present invention employs the above-mentioned integrated mechanical and mechanical coal mining robot and its autonomous navigation operation method, which has the following beneficial effects:

[0046] 1. Optimize multi-sensor fusion SLAM positioning by utilizing the geological features of the coal face to improve positioning accuracy and realize the geological-scene joint SLAM positioning method. At the same time, combine it with the natural semantic beacon positioning method to realize the transformation of coal mining robot from relative position positioning to absolute position positioning in the GIS system.

[0047] 2. Utilize multiple sensors to achieve synchronous autonomous perception and modeling of the scene and geology, and obtain information on the coal mining operation site and the geological information of the coal seam to be mined during the mining process.

[0048] 3. Based on local cost maps, realize autonomous navigation planning and cutting of coal mining robots in the coal mining face.

[0049] 4. Based on the global cost map, the autonomous navigation and propulsion of the coal mining robot's track and hydraulic support are completed, realizing the joint autonomous control of the coal mining robot, its track, and hydraulic support.

[0050] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of an embodiment of the integrated mechanical and mechanical coal mining robot of the present invention;

[0052] Figure 2 This is an overall flowchart of the present invention;

[0053] Figure 3 This is a flowchart of the geological scene environment perception process of the present invention;

[0054] Figure 4 This is a schematic diagram of the ground-penetrating radar data interpolation processing of the present invention. a is the single-channel B-scan obtained by the two ground-penetrating radar antennas along the survey line direction, b is a schematic diagram of the effect of linear interpolation using the shortest channel data between channels, c is a schematic diagram of the effect of linear interpolation between two adjacent channels in the survey line direction using the linear interpolation method, and d is the two measured data in a.

[0055] Figure 5 This is a schematic diagram of the motion control of the coal mining robot of the present invention;

[0056] Figure 6 This is a schematic diagram of the operation of the hydraulic support of the present invention;

[0057] Figure 7 This is a schematic diagram of the robot's working scenario according to the present invention;

[0058] Figure 8 This is a schematic diagram of positioning based on geological features. In the diagram, a is the coal mining face, b is significant geological features such as interbedded rock and hard rock, c is the scanning position of the ground penetrating radar at a certain moment, d is the sub-image obtained at that moment, e is the scanning position of the ground penetrating radar at the next moment, f is the sub-image obtained at that moment, h is the learning-based ground penetrating radar sensor model, and g is the pose transformation result.

[0059] Figure 9 The structure diagram of the learning-based ground-penetrating radar sensor model is shown below.

[0060] Figure Labels

[0061] 1. Ground-penetrating radar; 2. LiDAR; 3. Infrared camera; 4. High-definition camera; 5. Positioning and attitude determination module; 6. Integrated geological scene perception module; 7. Body motion state perception module; 71. Front roller shaft; 72. Rear roller shaft; 73. Encoder; 74. Rear body and rocker arm connection shaft; 75. Front body and rocker arm connection shaft; 8. Decision planning module; 9. Joint control module; 10. Coal mining machine body; 11. Remote monitoring terminal; 12. Ground-penetrating radar antenna; 13. Working face roof; 14. Working face floor; 15. Expansion radius; 16. Anomaly body; 17. Hydraulic support front beam side protection; 18. Hydraulic support; 19. Coal mining machine track; 20. Natural semantic beacon; 21. Artificial beacon. Detailed Implementation

[0062] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0063] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0064] Example

[0065] Please see Figure 1-9 This invention provides an integrated coal mining robot, including a coal mining machine body 10 and a remote monitoring terminal 11. The remote monitoring terminal 11 communicates with the coal mining robot 10 through an underground wireless network. The coal mining robot body 10 is equipped with a positioning and attitude determination module 5, a geological scene integrated perception module 6, a body motion state perception module 7, a decision planning module 8, and a joint control module 9. The positioning and attitude determination module and the geological scene integrated perception module share data from external sensors of the coal mining machine body.

[0066] The positioning and attitude determination module 5 utilizes a lidar 2 and a camera 4 installed on the outside of the coal mining machine and an IMU installed inside the coal mining robot. Based on multi-sensor fusion SLAM technology, it achieves precise positioning of the coal mining robot along the cutting direction in the underground coal mining face. At the same time, it identifies natural semantic beacons 20 and artificial beacons 21 deployed in the roadways on both sides of the coal mining face to correct the pose of the coal mining robot. The multi-sensor fusion SLAM adopts a geological-scene joint SLAM positioning strategy, and uses the geological features and scene features of the coal mining face to achieve positioning, improving positioning accuracy in harsh environments, and finally obtaining absolute spatial geographic coordinates in the GIS system.

[0067] The integrated geological scene perception module 6 utilizes ground-penetrating radar 1, lidar 2, and high-definition camera 4 installed on the outside of the coal mining machine. The ground-penetrating radar obtains geological information inside the coal mining face, and the lidar and high-definition camera capture three-dimensional spatial information, object color and texture, and other scene information of the coal mining operation site, in order to construct and obtain geological point cloud models and scene point cloud models.

[0068] The body motion state perception module 7 includes a main spindle encoder 73, a front drum shaft encoder 71, a rear drum shaft encoder 72, a shaft encoder 74 at the connection between the robot body and the rocker arm, and an IMU (Integrated Mutor Unit) located on the rocker arm. The main spindle encoder 73 is used to detect the rotational speed and direction of the main spindle in real time to obtain the traction speed of the robot. The front and rear drum shaft encoders are used to directly detect the speed of the cutting drum. The shaft encoder 74 at the connection between the robot body and the rocker arm measures the swing angle and tilt angle of the rocker arm, providing absolute angle information. The rocker arm IMU can obtain the relative angle change and tilt angle of the rocker arm. During rocker arm posture calculation, the initial rocker arm angle is obtained from the shaft encoder at the connection between the robot body and the rocker arm as a reference for posture. The relative angle change data and tilt angle from the IMU are used for real-time posture updates, and encoder data is used for correction to obtain accurate rocker arm posture information. The body motion state perception module is used by the subsequent decision-making and planning module to adjust the current traction speed of the coal mining machine, the speed of the cutting drum, and the rocker arm posture according to geological and scene conditions.

[0069] The decision planning module 8 controls the speed of the cutting drum, the rocker arm posture, and the traction speed of the coal mining robot body 10 based on the local cost map. The local cost map is a navigation map constructed and visualized by a probabilistic raster map generated on a remote monitoring terminal after two-dimensional plane projection and probability rasterization operations such as occupancy probability calculation of the geological point cloud model and the scene point cloud model.

[0070] The joint control module 9 uses the current pose information of the coal mining robot body 10 and the initial position information of the hydraulic support 18 known in the GIS system to jointly control the next position action of the hydraulic support 18 and the advance amount of the coal mining machine track 19 in the direction perpendicular to the coal mining face according to the global cost map. The global cost map is a navigation map constructed and visualized by a probabilistic raster map generated by the geological point cloud model and the scene point cloud model through two-dimensional plane projection, occupancy probability calculation and other probabilistic rasterization operations on the remote monitoring terminal 11.

[0071] An autonomous navigation method for an integrated geological exploration coal mining robot includes: importing prior information of the geological scene, estimating the absolute pose of the coal mining robot, synchronously perceiving the geological scene environment, integrating geological and scene modeling, and planning and control of the coal mining robot.

[0072] S1. Import of prior geological scene information: Import the existing GIS+BIM system into the remote monitoring terminal. The existing GIS+BIM system stores the prior three-dimensional geological model of the coal mining face obtained by drilling, seismic and other means. Specifically, it includes the position information of the working face roof 13 and working face floor 14 in the geographic coordinate system and the initial position information of the hydraulic support 18 in the geographic coordinate system, which provides a data benchmark for subsequent planning and control.

[0073] S2. Absolute pose estimation of the coal mining robot; the positioning and attitude determination module 5 uses natural semantic beacons 20, artificial beacons 21, and the existing underground GIS+BIM system to obtain the spatial geographic coordinates of the coal mining machine body in the GIS. The arrangement of natural semantic beacons 20 and artificial beacons 21 is as follows: Figure 7 As shown, the precise positioning of the underground coal mining robot is achieved by simultaneously using the lidar 2 and camera 4 installed on the main body of the coal mining machine and the multi-sensor fusion SLAM technology. In addition, in order to improve the positioning accuracy of SLAM in dust and smoke environment, the geological features of the coal mining face are used to achieve positioning, and finally the spatial geographic coordinates in the GIS system are obtained based on factor map optimization.

[0074] The specific process of absolute pose estimation for coal mining machines includes:

[0075] S21. The positioning and attitude determination module 5 unifies the data from the LiDAR 2, camera 4, and IMU into the same time and coordinate system through time synchronization and spatial calibration to ensure data consistency. During the data acquisition phase, the LiDAR continuously generates 3D point clouds, the camera synchronously captures images, and the IMU continuously acquires the acceleration and angular velocity information of the coal mining robot to reflect its instantaneous motion state. Using IMU pre-integration factor technology, these data are integrated between adjacent keyframes to obtain the cumulative pose change between two frames. These data undergo denoising, filtering, and distortion correction steps to ensure accuracy and reliability. The positioning and attitude determination module extracts geometric features from the LiDAR point cloud and texture features from the camera images, and performs multi-sensor data fusion. Through feature point matching and registration techniques, the relative pose estimation of the coal mining robot is achieved.

[0076] S22 and the positioning and attitude determination module 5 construct a learning-based ground-penetrating radar (GPR) sensor model using deep learning methods. This model maps non-sequential GPR image pairs to relative robot motion, thereby achieving robot positioning using the geological features of the coal mining face. The learning-based GPR sensor model is trained to predict the relative pose transformation between two sub-images based on the actual conditions of the coal mining face. Specifically, for a well-trained GPR sensor model, when the input is two GPR images, the output is the pose transformation of the corresponding positions in the two GPR images. This transformation is then fused with the positioning results of the multi-sensor SLAM fusion in S21 through a factor graph optimization framework, achieving joint SLAM positioning of the geological environment and the scene. The model structure is as follows: Figure 9 As shown, it includes a ResNet-18 autoencoder, average pooling layer, spatial correlation layer, fully connected layer, and Huber loss function. Two sub-graphs are shown below. Figure 8 As shown in Figures d and f, these are two similar ground-penetrating radar (GPR) images that have undergone preprocessing operations such as clutter and artifact removal. When acquiring these two sub-images, scenes with significant geological features, such as coal seams containing hard rock, should be selected to elicit strong GPR echo responses. Figure 8 As shown in the diagram, at a certain moment, the ground-penetrating radar scans the rocky area or hard rock at position c, obtaining sub-image d for this moment. When the ground-penetrating radar changes its attitude to position e, it scans again, obtaining sub-image f for this moment. The true values ​​of the pose transformation at these two moments are measured by a total station, thus obtaining a set of data for training the pose prediction network. The dataset for training the network can be obtained using this method. For a well-trained learning-based ground-penetrating radar model 5, when the model input is two ground-penetrating radar images, as shown... Figure 8 The values ​​d and f in the image are used to output the pose transformation of the corresponding positions in the two ground-penetrating radar images, such as... Figure 8 As shown in Figure 6, the constructed learning-based ground-penetrating radar model is as follows. Figure 9As shown, firstly, a ResNet-18 autoencoder is used to obtain feature activation maps of two sub-images. These feature activation maps contain relevant features for localization, and the autoencoder is trained using the L1 loss function. Then, average pooling is performed on the row standard deviation of the sub-images to check for salient features. If the two sub-images contain valid features, a linear correlation network is used to check if they share common features. If the sub-images contain salient features and are correlated, a two-stage method is used to predict the relative pose transformation of the two sub-images. The two-stage method first constructs a set of cost curves by comparing each feature map, and then constructs an argmax element vector from each cost curve, feeding it into a fully connected regression network to identify the relative pose transformation of the two sub-images. Finally, the pose transformation prediction network is trained using the Huber loss function combined with real sub-image pose transformation values ​​acquired by a total station. S23. The positioning and attitude determination module 5 utilizes natural features or devices with low movement frequency within the roadways on both sides of the coal face as natural semantic beacons 20. Point cloud and image data of the natural semantic beacons 20 are collected using LiDAR 2 and HD cameras 4 to create labels and construct a dataset of natural semantic beacons 20. Furthermore, based on the characteristic analysis of LiDAR 2 and HD cameras 4, artificial beacons 21, such as LiDAR 2 reflective targets and visual targets, are deployed in a staggered and coordinated manner with the aforementioned natural semantic beacons 20, ensuring that either natural semantic beacons 20 or artificial beacons 21 can be identified during the mining process and when the coal mining robot reaches both ends of the roadway. Based on the existing underground GIS+BIM system, the natural semantic beacons 20 and artificial beacons 21 are added to the BIM model and stored. Simultaneously, the pose information of the natural semantic beacons 20 and artificial beacons 21 in the geographic coordinate system is obtained using total station measurement methods and stored in the GIS+BIM system. After the coal mining robot moves to both ends of the working face roadway, the positioning and attitude determination module 5 uses the point cloud and image information collected by the lidar 2 and the high-definition camera 4 to perform instance-level semantic segmentation and six-degree-of-freedom pose estimation on the natural semantic beacon 20 and the artificial beacon 21 to obtain the pose of the coal mining robot in the beacon coordinate system. Combining the geographic coordinates of the natural semantic beacon 20 and the artificial beacon 21 in the GIS, the spatial geographic coordinates of the coal mining robot in the GIS are solved through coordinate transformation.

[0077] S24. The final positioning and attitude determination module integrates the factors of lidar odometer, camera odometer, IMU pre-integration factor, ground-penetrating radar positioning factor, and beacon positioning factor based on the factor graph optimization framework to achieve absolute positioning of the coal mining robot in the GIS coordinate system.

[0078] S3. Geological Scene Environment Perception: The integrated geological scene perception module 6 uses information provided by sensors such as ground-penetrating radar 1, lidar 2, infrared camera 3, and high-definition camera 4 to perform relevant calculations, obtains geological information inside the coal mining face and scene information of the coal mining operation site, and outputs it to the decision-making and planning module 8. The overall process of this part is illustrated as follows: Figure 3 As shown.

[0079] First, the integrated geological scene perception module 6 uses mean filtering to remove reflected clutter from gas-coal, gas-rock, and coal-rock interfaces, thereby suppressing coal and rock detection clutter signals during the ground-penetrating radar air coupling process. It also performs linear interpolation between the B-scans of each channel's measurement line obtained by the multi-channel ground-penetrating radar, such as... Figure 4 As shown, linear interpolation between B-scans of each channel survey line can supplement the geological information in the blank areas between survey lines, enriching the information content. The location and shape of anomalies are estimated using full waveform inversion to obtain medium interface information. Anomaly types are determined through an anomaly identification network, thus achieving anomaly identification in typical coal-rock geological structures. Then, combined with the obtained medium interface information, point cloud intensity information is used to characterize the medium type, realizing three-dimensional reconstruction of the geological point cloud model. Secondly, the scene laser model is obtained from the integrated geological scene perception module 6. Combined with the coordinates of the coal mining machine in the GIS system output by the positioning and attitude determination module 5, an initial geological model and an initial scene model under the same geographic coordinate system in the GIS system are obtained. Infrared images, visible light images, and laser point cloud intensity images of the exposed surface are obtained using infrared camera 3, high-definition camera 4, and lidar 2 to construct multi-source information. Coal-rock interfaces are identified based on deep learning methods, and the above geological point cloud model is verified and updated for medium properties. Meanwhile, the integrated geological scene perception module 6 utilizes a scene laser model to correct the two-way propagation time of data from each channel in the ground-penetrating radar 1 image in the air. This solves the problem of inaccurate geological point cloud modeling caused by uneven transition surfaces resulting from the mining and stripping process, which leads to different propagation times of the ground-penetrating radar 1 in the air medium. This enables dynamic correction of the geological point cloud model during the cutting process. The corrected geological point cloud model will update the existing GIS+BIM system's prior three-dimensional geological model of the coal face in real time, facilitating the next round of cutting planning.

[0080] The specific steps for synchronous perception of geological scene environment include:

[0081] S31. Obtained through pre-calibration or online estimation. Figure 4 The position and attitude extrinsic parameters of multiple antennas of the medium-penetration ground radar are synchronized by hardware synchronous triggering to achieve clock synchronization of each signal transmission of each antenna.

[0082] S32. By combining the real-time robot pose data obtained from the positioning and attitude determination module 5 with the antenna extrinsic parameters, we can obtain... Figure 4 The spatial location of each data point from the central antenna is determined, and further, based on the principle of time synchronization, the precise spatial location information of each data point on the B-Scan image of each antenna channel is obtained.

[0083] S33, Based on the spatial location of each data point, such as Figure 4 As shown, linear interpolation is used to interpolate between adjacent data points along the survey line direction to obtain the B-Scan image for each channel. This fills in the blank areas between adjacent data points, enriching the information of a single channel's B-Scan. Similarly, by using the nearest data point between channels for linear interpolation to fill in the blank areas between two adjacent survey lines, scanning data for any cross-section perpendicular to the survey line direction can be obtained.

[0084] S34. Based on the full waveform inversion and anomaly identification method, identify the coal and rock medium and anomaly type of each channel B-Scan, construct a three-dimensional geological image using channel B-Scan volume data containing information on coal and rock medium and anomaly type, convert the three-dimensional geological image into discrete point cloud, reconstruct a geological three-dimensional point cloud model containing medium type information, and use the point cloud intensity channel to characterize the medium type, that is, the point cloud intensity channel of coal and anomalies in coal seams that are not suitable for interception are different.

[0085] S35. Using visible light and infrared images obtained by high-definition camera 4 and infrared camera 3, the coal-rock interface identification results are obtained by multi-source information fusion through deep learning image recognition and target detection methods. The geological model obtained solely based on ground-penetrating radar information using full waveform inversion and anomaly identification methods is verified and updated to achieve dynamic correction of the geological model obtained by multi-source information joint inversion, ensuring the accuracy of the geological model. At the same time, the spatial coordinates of the corresponding coal-rock interface are obtained by using the positioning information of the positioning and attitude determination module and the observation model of high-definition camera, lidar and other sensors.

[0086] S36. Fit the air-coal-rock medium surface using scene point cloud model and geological point cloud model respectively. Cross-validate the accuracy of the two models based on the spatial continuity of the boundary area. Use the elevation model of the high-precision scene point cloud model to correct the geological point cloud model to achieve dynamic optimization.

[0087] S4. Integrated geological and scene modeling: Combining short-term estimations of robot motion states provided by inertial sensors, odometers, and other information, the original data distortion caused by the movement of the coal mining machine body in the integrated geological scene perception module is corrected. The timestamps and positional relationships of LiDAR 2 and HD camera 4 are calibrated to ensure that the data captured by LiDAR and HD camera are consistent in time and space. A heterogeneous sensor feature set is constructed, and the features in the HD camera image are matched with the features in the LiDAR point cloud to achieve the fusion of the two types of data. Based on the fused point cloud and image data, a point cloud model of the work site scene is constructed using methods such as voxel mesh and octree. At the same time, a geological point cloud model is constructed using the geological information provided by ground penetrating radar 1.

[0088] For features obtained by different sensors within the same time period, since there is a correlation in time and space, the integrated geological scene perception module 6 can perform time matching through a time sliding window to construct an artificially synthesized image that combines heterogeneous multi-scale features. This enables pixel-level alignment of data association between ground penetrating radar 1, lidar 2, visible light and depth infrared vision on the artificially synthesized image. The constructed artificially synthesized image is displayed through a remote monitoring terminal so that staff can monitor the working status of the coal mining robot in real time and make timely emergency responses.

[0089] The geological and scene point cloud information provided by the integrated geological scene perception module 6 will be used in the construction of the probabilistic rasterized dedicated navigation map and cost map for the coal mining robot at the remote monitoring terminal 11. The specific construction steps include:

[0090] S41. Construct a new four-channel point cloud data storage format to characterize geological point cloud models. Based on spatial three-dimensional coordinates and intensity information, the intensity information is redesigned into a data channel that reflects different media types such as gas, coal, and rock, so as to achieve consistent storage and integrated display of geological and scene point cloud models.

[0091] S42. Referring to the concept of constructing a 3D grid map using octrees, non-fully mechanized mining face areas and non-equipment areas in the scene model are constructed as free space, within which the coal mining machine track and hydraulic supports can perform reasonable advancement movements. The coal-type geological model area is constructed as a cutting free space, where the coal mining machine cutting drum can freely cut. Rock and geological anomaly areas are constructed as occupied spaces, where the coal mining machine cutting drum cannot perform cutting operations. A 3D grid map is constructed based on a probabilistic representation method, and the navigation map is updated and expanded probabilistically to achieve efficient storage, powerful compression, and fast transmission.

[0092] S43. To meet the diverse needs of robot autonomous navigation, including body planning, cutting planning, and visualization, a multi-resolution navigation map is constructed based on a 3D grid map. This provides a low-resolution coarse map for high-level planning of the coal mining robot body, i.e., adjusting and visualizing the movement of the coal mining machine track and hydraulic supports; and a high-resolution fine map for local planning of the cutting section, i.e., adjusting the speed of the cutting drum, the rocker arm posture, and the traction speed of the coal mining machine body. Furthermore, global and local cost maps are constructed for high-level and local planning of the coal mining robot through cost allocation. In the global cost map, as the scene model is continuously updated during coal mining operations, the distance between the coal mining machine track and the fully mechanized mining face increases due to coal seam collapse. At this point, the coal mining machine track and hydraulic supports need to be advanced perpendicular to the fully mechanized mining face, with the advancement amount affected by the single cutting depth of the coal mining machine. In the local cost map, identified geological anomalies and rocks unsuitable for cutting are treated as obstacles and expanded to different radii according to different hazard levels, designing cost maps with different safety levels for the planning of the robot's movement and cutting process.

[0093] S44, the local cost map and the global cost map are generated and visualized on the remote monitoring terminal. The local cost map is sent to the decision planning module and the global cost map is sent to the joint control module.

[0094] S5. Coal mining robot planning and control refers to the decision-making and planning module 8 controlling the speed of the cutting drum, the rocker arm posture, and the traction speed of the coal mining machine body 10 based on the local cost map. The joint control module 9 utilizes the current pose information of the coal mining machine body 10 and the initial position information of the hydraulic support 18 known in the GIS system, and jointly regulates the next position movement of the hydraulic support 18 and the advance amount of the coal mining machine track 19 in the direction perpendicular to the coal face based on the global cost map.

[0095] The specific motion control process of Decision Planning Module 8 is as follows:

[0096] S51. When the coal mining robot is in the cutting free space, that is, in the coal-type medium area, the coal mining machine cutting drum uses a preset posture for free cutting.

[0097] S52. When the coal mining robot detects occupied space, i.e., hard rock in the geology, during operation, reduce the traction speed and drum cutting speed of the coal mining robot, such as... Figure 5 As shown, the height of the drum arm of the coal mining robot is lowered or raised along the expansion radius 15 of the obstacle 16 so that the drum can safely bypass the occupied space. Different expansion radii are set for abnormal bodies of different danger levels. The higher the danger level, the larger the expansion radius.

[0098] S53. After safely bypassing the occupied space, it re-enters the cutting free space. The decision planning module 8 adjusts the coal mining machine drum back to the preset posture for free cutting.

[0099] The specific motion control process of the joint control module 9 is as follows:

[0100] S51. During the coal mining operation, the current cutting drum speed and rocker arm posture information of the coal mining machine are obtained by the self-state perception module 7, the spatial geographic coordinates of the coal mining robot in the GIS are obtained by the positioning and attitude determination module 5, and combined with the initial spatial geographic coordinates of the hydraulic support known in the GIS, the joint control module 9 controls the front beam guard of the hydraulic support to automatically retract when the cutting drum of the coal mining robot approaches, so as to prevent the front beam guard of the hydraulic support from being damaged during the cutting operation of the coal mining robot drum.

[0101] S52. As the coal mining robot moves away from the face, in the post-mining area, the control module 9, based on the global cost map, advances the coal mining machine track perpendicular to the working face in free space. The advancement amount takes into account the cutting depth and maintains a certain safe distance from the working face to ensure the stable operation of the coal mining machine. Simultaneously, the control module 9 controls the hydraulic support in the post-mining area to advance perpendicular to the working face and support the front beam sidewall, with the advancement amount meeting the safe distance requirement between the coal mining machine and the hydraulic support.

[0102] S53. The above process shall be repeated during the coal mining operation.

[0103] Therefore, the present invention adopts the above-mentioned integrated mechanical exploration coal mining robot and its autonomous navigation method to realize non-contact autonomous accurate scene and geological modeling during the coal mining process. The process is simple, requires less manual operation, has high precision and high efficiency, and truly realizes the transformation from an automatic memory cutting coal mining machine to an autonomous perception and planning coal mining robot.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A coal mining robot integrating mechanical exploration, characterized in that: It includes a coal mining robot and a remote monitoring terminal. The remote monitoring terminal communicates with the coal mining robot through an underground wireless network. The coal mining robot is equipped with a positioning and attitude determination module, a geological scene integrated perception module, a body motion state perception module, a decision planning module, and a joint control module. The positioning and attitude determination module and the geological scene integrated perception module share data from external sensors of the coal mining robot body. The positioning and attitude determination module utilizes a lidar and camera installed on the outside of the coal mining machine and an IMU installed inside the coal mining robot. Based on multi-sensor fusion SLAM technology, it achieves precise positioning of the coal mining robot along the cutting direction in the underground coal mining face. At the same time, it identifies natural semantic beacons and artificial beacons deployed in the roadways on both sides of the coal mining face to correct the pose of the coal mining robot. Furthermore, it utilizes the geological features of the coal mining face to further improve the positioning accuracy of the coal mining robot and obtain the absolute spatial geographic coordinates of the coal mining robot in the GIS system. The integrated geological scene perception module uses ground-penetrating radar, lidar, and high-definition cameras installed on the outside of the coal mining machine to obtain geological information inside the coal mining face and scene information of the coal mining operation site, in order to construct and obtain geological point cloud models and scene point cloud models. The body motion state perception module includes the main shaft encoder of the coal mining robot, the front and rear drum shaft encoders of the coal mining robot, the shaft encoder at the connection between the coal mining robot body and the rocker arm, and the IMU arranged on the rocker arm. The body motion state perception module is used to obtain the traction speed, cutting drum speed and rocker arm posture information of the coal mining robot, so that the subsequent decision planning module can adjust the current traction speed of the coal mining machine, the cutting drum speed and the rocker arm posture according to geological and scene conditions. The decision planning module controls the speed of the cutting drum, the rocker arm posture, and the traction speed of the coal mining machine body based on the local cost map. The local cost map is a navigation map constructed and visualized by a probabilistic raster map generated by the geological point cloud model and the scene point cloud model through two-dimensional plane projection and occupation probability calculation on the remote monitoring terminal. The joint control module uses the current pose information of the coal mining machine body and the initial position information of the hydraulic support known in the GIS system to jointly control the next position movement of the hydraulic support and the advance of the coal mining machine track in the direction perpendicular to the coal mining face according to the global cost map. The global cost map is a navigation map constructed and visualized by a probabilistic raster map generated by the geological point cloud model and the scene point cloud model through two-dimensional plane projection and occupancy probability calculation on the remote monitoring terminal.

2. An autonomous navigation operation method for an integrated mechanical and mechanical coal mining robot, applied to the integrated mechanical and mechanical coal mining robot described in claim 1, characterized in that, Includes the following steps: S1. Import of prior information of geological scene: Import the existing GIS+BIM system into the remote monitoring terminal, including the position information of the working face top plate and working face bottom plate in the geographic coordinate system and the initial position information of the hydraulic support in the geographic coordinate system. S2. Absolute pose estimation of coal mining robot: The absolute spatial geographic coordinates of the coal mining robot body in GIS are obtained using the positioning and attitude determination module. S3. Synchronous perception of geological scene environment: The integrated geological scene perception module is used to obtain geological information inside the coal mining face and scene information of the coal mining operation site and output it to the decision planning module. S4. Integrated Geological and Scene Modeling: Combining short-term estimations of robot motion states provided by inertial sensors and odometry, the distortion of raw data caused by the movement of the coal mining machine body in the integrated geological scene perception module is corrected. The timestamps and positional relationships of the lidar and high-definition camera are calibrated to ensure that the data captured by the lidar and high-definition camera are consistent in time and space. A heterogeneous sensor feature set is constructed, and the features in the high-definition camera image are matched with the features in the lidar point cloud to achieve the fusion of the two types of data. Based on the fused point cloud and image data, a point cloud model of the work site scene is constructed through voxel mesh and octree. At the same time, a geological point cloud model is constructed using geological information provided by ground penetrating radar. S5. Coal Mining Robot Planning and Control: The decision-making and planning module controls the speed of the cutting drum, the rocker arm posture, and the traction speed of the coal mining machine body based on the local cost map constructed by the probabilistic rasterized dedicated navigation map generated by the geological point cloud model and the scene point cloud model through two-dimensional plane projection and occupation probability calculation. The joint control module uses the current pose information of the coal mining machine body and the initial position information of the hydraulic support known in the GIS system to jointly regulate the next position action of the hydraulic support and the advance amount of the coal mining machine track in the direction perpendicular to the coal mining face based on the global cost map constructed by the probabilistic rasterized dedicated navigation map.

3. The autonomous navigation operation method of an integrated mechanical exploration coal mining robot according to claim 2, characterized in that: In S2, natural features or low-frequency moving equipment motors and fire cabinets in the roadways on both sides of the coal mining face are used as natural semantic beacons. Point cloud and image data of the natural semantic beacons are collected by LiDAR and high-definition cameras to create labels and construct a natural semantic beacon dataset. The dataset is used to train a six-degree-of-freedom pose estimation network. A well-trained six-degree-of-freedom pose estimation network is used to analyze and calculate the pose of the coal mining robot in the beacon coordinate system.

4. The autonomous navigation operation method of an integrated mechanical and geological mining robot according to claim 3, characterized in that, The absolute pose estimation of the coal mining robot in S2 includes the following steps: S21. The positioning and attitude determination module unifies the data from the lidar, camera, and IMU into the same time and coordinate system through time synchronization and spatial calibration. During the data acquisition phase, the lidar continuously generates a 3D point cloud, the camera synchronously captures images, and the IMU continuously acquires the acceleration and angular velocity information of the coal mining robot. The cumulative pose change between two frames is obtained using the IMU pre-integration factor technology. The positioning and attitude determination module extracts the geometric features in the lidar point cloud and the texture features in the camera image, and performs multi-sensor data fusion. The relative pose estimation of the coal mining robot is achieved through feature point matching and registration technology. S22. The positioning and attitude determination module constructs a learning-based ground-penetrating radar sensor model through deep learning methods, and maps non-sequential ground-penetrating radar image pairs to relative robot motion, thereby realizing the positioning of the coal mining robot by utilizing the geological features of the coal mining face. S23. The positioning and attitude determination module uses point cloud and image information collected by LiDAR and high-definition camera to perform instance-level semantic segmentation and six-degree-of-freedom pose estimation on natural semantic beacons and artificial beacons to obtain the pose of the coal mining robot in the beacon coordinate system. Combining the geographic coordinates of natural semantic beacons and artificial beacons in GIS, the absolute spatial geographic coordinates of the coal mining robot in GIS are solved through coordinate transformation. S24. The final positioning and attitude determination module integrates the factors of lidar odometer, camera odometer, IMU pre-integration factor, ground-penetrating radar positioning factor, and beacon positioning factor based on the factor graph optimization framework to achieve absolute positioning of the coal mining robot in the GIS coordinate system.

5. The autonomous navigation operation method of an integrated mechanical and geological mining robot according to claim 4, characterized in that, Synchronous perception of geological scene environment in S3 includes the following steps: S31. The position and attitude extrinsic parameters between multiple antennas of the ground penetrating radar are obtained by pre-calibration or online estimation, and the clock synchronization of each signal transmission of each antenna is realized based on hardware synchronization triggering. S32. Using the robot's real-time pose data obtained by the positioning and attitude determination module and the relationship of antenna external parameters, the spatial position of each data point of the antenna is obtained. Based on the time synchronization principle, the precise spatial position information of each data point on the B-Scan image of each antenna channel is obtained. S33. Perform linear interpolation between two adjacent data points in each survey line direction to obtain the B-scan image of each channel. Use the data point with the closest distance between channels to perform linear interpolation to obtain the scan data of any section perpendicular to the survey line direction. S34. Based on the full waveform inversion and anomaly identification method, identify the coal and rock medium and anomaly type of each channel B-Scan, construct a three-dimensional geological image using channel B-Scan volume data containing information on coal and rock medium and anomaly type, convert the three-dimensional geological image into discrete point cloud, reconstruct a geological three-dimensional point cloud model containing medium type information, and use the point cloud intensity channel to characterize the medium type. S35. Using visible light images obtained from a high-definition camera and infrared images obtained from an infrared camera, the coal-rock interface identification results are obtained through deep learning image recognition and target detection methods. The geological model obtained solely based on ground-penetrating radar information using full waveform inversion and anomaly identification methods is then verified and updated to achieve dynamic correction of the geological model obtained through joint inversion of multi-source information, ensuring the accuracy of the geological model. Simultaneously, the spatial coordinates of the corresponding coal-rock interface are obtained by using the positioning information of the positioning and attitude determination module and the observation model of the high-definition camera and lidar sensor. S36. Fit the air-coal-rock medium surface using scene point cloud model and geological point cloud model respectively. Cross-validate the accuracy of the two models based on the spatial continuity of the boundary area. Use the elevation model of the high-precision scene point cloud model to correct the geological point cloud model to achieve dynamic optimization.

6. The autonomous navigation operation method of an integrated mechanical and geological mining robot according to claim 5, characterized in that: The S33 integrated geological scene perception module uses mean filtering to remove reflected clutter from the gas-coal, gas-rock, and coal-rock interfaces, and suppresses coal and rock detection clutter signals during the air coupling process of ground-penetrating radar.

7. The autonomous navigation operation method of an integrated mechanical and geological mining robot according to claim 6, characterized in that: In S4, the integrated geological scene perception module performs time matching through a time sliding window to construct an artificially synthesized image that combines heterogeneous multi-scale features. The constructed artificially synthesized image is displayed through a remote monitoring terminal.

8. The autonomous navigation operation method of an integrated mechanical exploration coal mining robot according to claim 7, characterized in that: The geological and scene point cloud information provided by the integrated geological scene perception module in S4 will participate in the construction of the probabilistic rasterized dedicated navigation map for the coal mining robot on the remote monitoring terminal. The specific construction steps include: A. Construct a new four-channel point cloud data storage format to characterize geological point cloud models, and redesign the intensity information into a data channel that reflects different media types such as gas, coal and rock. B. Construct the non-fully mechanized mining face area and non-equipment area in the scene model as free space, where the coal mining machine track and hydraulic support can perform reasonable advancing actions; construct the coal-type medium geological model area as a cutting free space, where the coal mining machine cutting drum can cut freely; construct the rock and geological anomaly area as occupied space, where the coal mining machine cutting drum cannot perform cutting operations. C. Provide a low-resolution coarse map for the high-level planning of the coal mining robot body, namely the adjustment and visualization of the pushing amount of the coal mining machine track and hydraulic support; provide a high-resolution fine map for the local planning of the cutting part, namely the speed of the coal mining machine cutting drum, the rocker arm posture, and the traction speed adjustment of the coal mining machine body. Then, through cost allocation, construct global and local cost maps for the high-level planning and local planning of the coal mining robot respectively. In the global cost map, as the coal mining operation progresses, the scene model is continuously updated. Due to the coal seam falling off, the distance between the coal mining machine track and the fully mechanized mining face continuously increases. At this time, it is necessary to push the coal mining machine track and hydraulic support in a direction perpendicular to the fully mechanized mining face. The amount of advancement is affected by the single cutting depth of the coal mining machine. In the local cost map, the geological anomalies and rocks that are not suitable for cutting are identified as obstacles and expanded with different radii according to different danger levels. Design cost maps with different safety levels for the planning of the machine movement and cutting process. D. The local cost map and the global cost map are generated and visualized on the remote monitoring terminal. The local cost map is sent to the decision planning module and the global cost map is sent to the joint control module.

9. The autonomous navigation operation method of an integrated mechanical and geological mining robot according to claim 8, characterized in that, The decision planning module in S5 includes the following steps: S51. When the coal mining robot is cutting in free space, that is, in a coal-type medium area, the cutting drum of the coal mining machine uses a preset posture for free cutting; S52. During the operation of the coal mining robot, if an occupied space is detected, the traction speed of the coal mining robot and the cutting speed of the drum are reduced. The height of the drum arm of the coal mining robot is lowered or raised along the expansion radius of the obstacle so that the drum can safely bypass the occupied space. S53. After safely bypassing the occupied space, it re-enters the cutting free space, and the decision planning module adjusts the coal mining machine drum back to the preset posture for free cutting.

10. The autonomous navigation operation method of an integrated mechanical exploration coal mining robot according to claim 9, characterized in that, The S5 central control module includes the following steps: S51. During the coal mining operation, the current cutting drum speed and rocker arm posture information of the coal mining machine are obtained by the self-state perception module. The spatial geographic coordinates of the coal mining robot at the current moment in the GIS are obtained by the positioning and attitude determination module. Combined with the initial spatial geographic coordinates of the hydraulic support known in the GIS, the joint control module controls the front beam guard of the hydraulic support to automatically retract when the cutting drum of the coal mining robot approaches, so as to prevent the front beam guard of the hydraulic support from being damaged during the cutting operation of the coal mining robot drum. S52. As the coal mining robot moves away, in the post-mining area of ​​the coal mining face, the joint control module advances the coal mining machine track in free space perpendicular to the working face according to the global cost map. The advancement amount takes into account the cutting depth and maintains a certain safe distance from the working face to ensure the stable operation of the coal mining machine. At the same time, the joint control module controls the hydraulic support in the post-mining area to advance as a whole in a direction perpendicular to the working face and support the front beam side protection. The advancement amount meets the safe distance between the coal mining machine and the hydraulic support. S53. The above process shall be repeated during the coal mining operation.

Images