A geological scene perception and modeling system and method for coal mine robots

Abstract

The present invention discloses a geological scene perception and modeling system and method for coal mine robots, which relates to the field of geological exploration and experiments. The system includes a working environment simulation module for coal mine robots, an orbital mobile platform, a multi-source perception module, a coal seam geological simulation module, and an experimental analysis module. Different environments are created through the working environment simulation module for coal mine robots and the orbital mobile platform to simulate the operation process of coal mine robots. The multi-source sensor module is used to obtain the scene and the perception information of the coal seam geological simulation module in a complex environment, and construct the perception data sets of each sensor. The experimental analysis module is used to perform comparative analysis and processing on the data, complete the fusion of multi-source perception information, and construct an integrated three-dimensional model of geology and scene. The present invention provides an effective experimental platform and experimental method for studying the signal propagation mechanism of each sensor under complex working conditions, the echo characteristics of electromagnetic waves in different geologies, and the integrated modeling of geological scenes.

Description

Technical Field

[0001] The present invention relates to the field of geological exploration and experiments, and in particular to a geological scene perception modeling system and method for coal mine robots. Background Art

[0002] China is rich in coal resources. However, due to poor coal mine geological conditions and low intelligent level of mining equipment, the mining efficiency is reduced and the incidence of work safety accidents is increased. The fully mechanized coal mining face environment of coal mine robots is complex and harsh. The traditional pre-mining detection and modeling have low efficiency and poor accuracy, and cannot dynamically correct the geological model in real time. Coal mine robots lack the ability of autonomous perception and real-time response. Bad geological structures such as small faults and fracture zones during the construction process increase the tunneling technical difficulty and affect the tunneling speed and safety. Therefore, it is very necessary to develop an experimental system and method that can realize real-time geological scene perception modeling.

[0003] Unmanned intelligent and precise mining is the core of coal mine intelligent construction. The precise mining of coal mine robots requires them to have the ability of autonomous environment perception and real-time adjustment. Due to the harsh underground environment, especially insufficient light, a lot of dust, high humidity and other reasons, different impacts are caused on sensors, resulting in certain defects in the integrated modeling of the geological scene of the fully mechanized coal mining face. At present, geological modeling and scene modeling are carried out using different technologies, and both geological and scene modeling are pre-modeled manually, resulting in problems such as low modeling efficiency, high model coupling complexity, low model accuracy, and difficult update. At present, the geological modeling of ground penetrating radar mainly focuses on the research of propagation characteristics in coal and rock media. The propagation mechanism of sensing signals in the complex geology of the coal mining face is not clear. There is a lack of research on the propagation characteristics of electromagnetic wave echo signals in the gas-coal-rock cross-media during the operation of robots. The multi-sensor fusion perception and information joint interpretation mechanism in complex environments is not clear.

[0004] Therefore, it is necessary to provide a geological scene perception modeling system and method for coal mine robots to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to provide a geological scene perception modeling system and method for coal mine robots, which are used to simulate the operation process and working environment of coal mine robots, facilitate the laboratory simulation test of multi-source fusion perception modeling during the operation process of coal mine robots, collect complex environment scene information, study the perception characteristics of multi-sensors in complex environments and the multi-source information joint interpretation and perception enhancement method, explore the further realization of real-time integrated detection and modeling of geological scenes, and perform real-time modeling of geology and scenes by using devices such as computers. It is convenient to carry out laboratory simulation tests for the identification of coal and rock interfaces by ground penetrating radar, study the cross-media propagation characteristics of electromagnetic waves, construct a typical coal and rock data set, and realize full waveform inversion and anomaly identification of detection.

[0006] To achieve the above object, the present invention provides a geological scene perception and modeling system for coal mine robots, which includes an enclosed box body composed of wood board - iron sheet - thermal insulation layer, a scene feature simulation device arranged on the left side of the enclosed box body, a true value measurement device arranged inside the enclosed box body, and also includes a coal mine robot working environment simulation module arranged inside the enclosed box body, a coal seam geological simulation module arranged inside the enclosed box body, a rail - type mobile platform arranged in front of the coal seam geological simulation module, and a multi - source perception module mounted on the rail - type mobile platform; the multi - source perception module, the true value measurement device, and the coal mine robot working environment simulation module are all connected to the experimental analysis module. The three - dimensional scene information and the pose information of the rail - type mobile platform and the sensors are obtained through the true value measurement device. The distance between the multi - source perception module and the coal seam geological simulation module is changed through the rail - type mobile platform, thereby changing the perception information of the multi - source perception module.

[0007] Preferably, the material of the scene feature simulation device is set as a wooden structure, and its position and posture are adjusted according to the experimental content to simulate different scenes; the true value measurement device includes a laser tracker and a laser scanner. The scene information is obtained by using the laser scanner, and the pose information of the rail - type mobile platform and the sensors is obtained by using the laser tracker and the target; the coal seam geological simulation module includes coal blocks and rock blocks of different shapes. The coal blocks and rock blocks are all cut from complete coal blocks and rock blocks according to requirements, and different simulated geological conditions can be quickly recombined by stacking coal blocks and rock blocks of different shapes.

[0008] Preferably, the coal mine robot working environment simulation module includes a dust generator, a water vapor generator, a lighting device, a dehumidification and dust removal device, a dust concentration detection sensor, an illuminance detection sensor, and an air humidity detection sensor; the dust generator is arranged at the bottom of the coal powder box, the lighting device includes a controllable lighting lamp, and the dehumidification and dust removal device includes an intake fan and an exhaust fan.

[0009] Preferably, the multi - source perception module is jointly deployed by various sensors mounted on the rail - type mobile platform according to the experimental content. The sensors include a ground - penetrating radar, a lidar, a frequency - modulated continuous - wave radar, a visible - light camera, an infrared camera, and an inertial navigation unit.

[0010] Preferably, guide wheels and a driving motor are arranged under the rail - type mobile platform. The rail - type mobile platform includes a transverse rail and a longitudinal rail. The surfaces of both the transverse rail surface and the longitudinal rail are non - smooth surfaces. The transverse rail is arranged at the bottom of the enclosed box body, a guide rail groove is opened under the longitudinal rail, the guide rail groove is in contact with the transverse rail, the guide wheels are in contact with the longitudinal rail, the driving motor is connected to the guide wheels, and the driving motor drives the guide wheels to move. An electric push rod is installed on the left side of the longitudinal rail, and the electric push rod is connected to the longitudinal rail. The longitudinal rail is pushed to move horizontally through the electric push rod.

[0011] Preferably, the experimental analysis module includes a computer and related software. The sensor transmits the collected data to the computer, and the data is analyzed and processed by the computer and related software. According to the experimental equipment and research content, a digital twin simulation engine software development platform is constructed to realize the perception modeling of the operation process of the coal mine robot and the visualization of the digital twin monitoring effect.

[0012] A method for geological scene perception modeling of a coal mine robot specifically includes the following steps:

[0013] S1: Through the coal mine robot working environment simulation module and the rail-mounted mobile platform, change the dust concentration, illuminance, and air humidity in the box environment and simulate the operation process of the coal mine robot;

[0014] S2: Obtain the information of the scene in the complex environment and the information of the coal seam geology simulation module through the multi-source perception module of the coal mine robot, and construct a geological and scene joint data set;

[0015] S3: Use the scene data obtained in step S2 to study the noise modeling and data enhancement method of the scene observation multi-modal heterogeneous sensors;

[0016] S4: Use the geological data obtained in step S2 to construct a radar echo forward and inverse data set, and study the propagation characteristics of the ground penetrating radar echo signal and the abnormal body recognition method in the cross-media process;

[0017] S5: Process the data through the experimental analysis module, study the correlation of geological and scene multi-scale feature data and the joint interpretation and perception enhancement method of multi-source information, and complete multi-source information fusion perception;

[0018] S6: Use the geological and scene multi-source fusion perception information to construct an integrated three-dimensional model of geology and scene, and at the same time use the boundary area characteristics of the geological model and the scene model to correct the integrated three-dimensional model;

[0019] S7: Use the truth measurement device to obtain the truth of the scene three-dimensional model and the pose of the coal mine robot, and compare with the multi-source information fusion perception and three-dimensional model results obtained in steps S5 and S6 to evaluate the experimental effect of the geological scene perception modeling method of the coal mine robot.

[0020] Preferably, in step S1, the coal mine robot working environment simulation module controls the start and stop of the dust generator, water vapor generator, lighting device, dehumidification and dust removal equipment, dust concentration detection sensor, illuminance detection sensor, and air humidity detection sensor through the central controller, and the application situation of the coal mine robot working environment simulation module can be randomly combined and adjusted according to the actual experimental content.

[0021] Preferably, in step S2, a ground penetrating radar and an infrared camera are used to detect the coal seam geological simulation module and collect relevant data; a lidar, a frequency modulated continuous wave radar, and a visible light camera are used to obtain the original information of the scene in a complex environment.

[0022] Preferably, in step S3, based on the control experiment analysis, the forms and influence laws of the interference of the environment and working conditions on the original information of the sensor are mastered, and the noise observation model of each sensor is established.

[0023] Preferably, in step S4, the gprMax simulation software is used to design and establish a simulation coal seam geological simulation module according to the parameters of the constructed coal seam geological physical model, and a forward simulation of the simulation model is carried out.

[0024] Preferably, in step S5, the sensor data is processed by the experimental analysis module, the multi-scale visualization imaging method of the sensor is explored, and the multi-scale data characterization method is constructed, which specifically includes the following steps:

[0025] S51: Through the design of a time sliding window for time matching, a data association method for pixel-level alignment of sensor data on the constructed synthetic image is proposed;

[0026] S52: Using the synthetic image constructed by multi-scale features, a loss function is constructed based on the data association metric;

[0027] S53: Using the data-driven multi-task learning method to infer the damaged types and degrees of multi-scale features, and realizing the joint interpretation of multi-modal information;

[0028] S54: Perform feature prediction on the synthetic image under the condition of feature loss caused by environmental interference, compensate the damaged information, and realize the enhancement of sensor feature information perception under complex conditions.

[0029] Preferably, in step S6, the construction of the three-dimensional geological point cloud model is realized by linear interpolation and point cloud discretization, which specifically includes the following steps:

[0030] S61: Based on the simultaneous localization and mapping technology of multi-sensor fusion, the scene model data is obtained, and the scene point cloud model construction is completed;

[0031] S62: Use the scene point cloud model and the geological point cloud model to respectively fit the air-coal rock medium surface;

[0032] S63: Cross-verify the accuracy of the scene point cloud model and the geological point cloud model based on the spatial continuity of the junction area;

[0033] S64: Use the elevation model of the high-precision scene point cloud model to correct the geological point cloud model to realize dynamic optimization;

[0034] S65: Obtain the coal-rock interface recognition result by using the optimized scene point cloud model;

[0035] S66: Conduct medium physical property verification and update on the geological point cloud model to achieve dynamic correction of the geological model for joint inversion of multi-source information;

[0036] S67: By constructing a new four-channel point cloud data storage data format, based on the spatial three-dimensional coordinates and intensity information, redesign the intensity information into a data channel reflecting different medium types of gas-coal-rock, and achieve consistent storage and integrated display of the geological and scene point cloud models.

[0037] Therefore, the present invention adopts the above-mentioned coal mine robot geological scene perception modeling system and method, and has the following beneficial effects:

[0038] (1) In the coal seam geological simulation module of the present invention, by different stacking methods of coal blocks and rock blocks with various shapes, the coal seam geological simulation module can be changed to create different geological structures, and simulate typical structures such as faults, collapse columns, and cavities in the real underground coal seam.

[0039] (2) Through the working condition simulation system of the coal mine robot and the rail-mounted mobile platform equipped with sensors, the present invention can change the dust concentration, illuminance, air humidity in the box environment and simulate the operation process of the coal mine robot.

[0040] (3) The digital twin simulation engine software development platform constructed by the experimental analysis module of the present invention can realize the visualization of the perception modeling of the coal mine robot operation process and the digital twin monitoring effect.

[0041] (4) The coal mine robot geological scene perception modeling system and method of the present invention are not only applicable to scientific research activities such as geological scene perception modeling in the operation process of coal mine robots, but also can be used for scientific research on geological scene perception modeling in the operations of coal mining robots, rail-mounted inspection robots, tunneling robots, etc.

[0042] The technical solution of the present invention will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0043] Figure 1 is the structural schematic diagram of a coal mine robot geological scene perception modeling system and method of the present invention;

[0044] Figure 2 is the actual application diagram of a coal mine robot geological scene perception modeling system and method of the present invention;

[0045] Figure 3 is the multi-sensor data fusion information weight experiment flow chart of a coal mine robot geological scene perception modeling method of the present invention;

[0046] Figure 4 It is the flow chart of the sensor noise observation experiment of a geological scene perception and modeling method for a coal mine robot in the present invention;

[0047] Figure 5 It is the flow chart of the experiment on the propagation characteristics of the ground penetrating radar echo signal and the identification of abnormal bodies of a geological scene perception and modeling method for a coal mine robot in the present invention;

[0048] Figure 6 It is the flow chart of the integrated and efficient geological scene modeling experiment of a geological scene perception and modeling method for a coal mine robot in the present invention;

[0049] Appendix annotation

[0050] 1. Enclosed box; 2. Intake fan; 3. Pulverized coal box; 4. Dust generator; 5. Water storage tank; 6. Water vapor generator; 7. Lighting device; 8. Dust concentration detection sensor; 9. Illuminance detection sensor; 10. Air humidity detection sensor; 11. Visible light camera; 12. Infrared camera; 13. Coal block; 14. Rock block; 15. Collapse column; 16. Coal seam geological simulation module; 17. Coal seam cavity; 18. Fixed bolt; 19. Rail-mounted mobile platform; 20. Ground penetrating radar; 21. Lidar; 22. Frequency modulated continuous wave radar; 23. Sensor data collector; 24. Driving motor; 25. Horizontal guide rail; 26. Longitudinal guide rail; 27. Guide rail wheel; 28. Electric push rod; 29. True value measuring device; 30. Exhaust fan; 31. Computer; 32. Data acquisition card; 33. Central controller; 34. Scene feature simulation device. Detailed implementation manners

[0051] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0052] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the ordinary meanings understood by those of ordinary skill in the field to which the present invention belongs.

[0053] In the present invention, words such as "including" or "comprising" mean that the elements before this word cover the elements listed after this word, and do not exclude the possibility of also covering other elements. The orientation or positional relationship indicated by terms such as "inside", "outside", "above", "below", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the present invention. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly. In the present invention, unless otherwise clearly specified and limited, terms such as "attached" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be directly connected, or indirectly connected through an intermediate medium, and can be the communication inside two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0054] As Figure 1 shown, the present invention provides a geological scene perception and modeling system for coal mine robots, including a coal mine robot working environment simulation module, an orbital mobile platform, a multi-source perception module, and an experimental analysis module.

[0055] A geological scene perception and modeling system for coal mine robots includes a closed box body 1 composed of wood - iron sheet - thermal insulation layer. The closed box body 1 is composed of wood. The iron sheet is fixed to the inner surface of the closed box body 1 by iron nails, and a layer of thermal insulation material is attached to the inner surface of the iron sheet surface. The closed box body 1 is jointly composed of wood - iron sheet - thermal insulation layer to maintain the temperature stability inside the closed box body 1;

[0056] It further includes a scene feature simulation device 34 arranged inside the closed box body 1, a coal seam geology simulation module 16 arranged inside the closed box body 1, and a true value measurement device 29 arranged inside the closed box body 1, and a coal mine robot working environment simulation module arranged above the closed box body 1; the material of the scene feature simulation device 34 is set as a wooden structure. The coal seam geology simulation module 16 is stacked by coal blocks 13 and rock blocks 14. Both the coal blocks and the rock blocks are cut from complete coal blocks and rock blocks according to requirements, and then the coal blocks 13 and the rock blocks 14 are stacked together and fixed to the front of the orbital mobile platform 19 with fixing bolts 18. Geological structures such as coal seam cavities 17 and subsidence columns 15 are simulated by stacking coal and rock blocks of different shapes to form voids of a preset size.

[0057] There is an orbital mobile platform 19 arranged in front of the coal seam geological simulation module 16. Sensors are combined and deployed according to the experimental content to form a multi-source perception module. The sensors include a ground penetrating radar 20, a lidar 21, a frequency modulated continuous wave radar 22, a visible light camera 11 and an infrared camera 12. A sensor data collector 23, a true value measurement device 29 and a coal mine robot working environment simulation module are all connected to the experimental analysis module. The experimental analysis module includes a computer 31 and related software. The scene information and the pose information of the orbital mobile platform 19 and the sensors are obtained through the true value measurement device 29. The distance between the sensors and the coal seam geological simulation module 16 is changed through the orbital mobile platform 19 to change the information perceived by the multi-source perception module.

[0058] The coal mine robot working environment simulation module controls a dust generator 4, a water vapor generator 6, a lighting device 7, a dehumidification and dust removal device, a dust concentration detection sensor 8, an illuminance detection sensor 9, and an air humidity detection sensor 10 through a central controller 33. The dust generator 4 is arranged at the bottom of a pulverized coal box 3. The lighting device 7 includes a controllable lighting lamp. The dehumidification and dust removal device includes an intake fan 2 and an exhaust fan 30. The data detected by the dust concentration detection sensor 8, the air humidity detection sensor 10, and the illuminance detection sensor 9 will be transmitted to the computer 31 by a data acquisition card 32.

[0059] The pulverized coal in the pulverized coal box 3 is released into the closed box 1 through the dust generator 4 to change the dust concentration in the closed box 1, and the dust concentration in the environment is detected by the dust concentration detection sensor 8. Water vapor is released into the environment through the water vapor generator 6 to change the air humidity in the environment, and the air humidity in the working space is detected by the air humidity detection sensor 10. The environmental illuminance in the working environment is changed by changing the brightness of the lighting device 7, and the environmental illuminance in the box is detected by the illuminance detection sensor 9. When the dust concentration or air humidity in the working passage is too high, the dust concentration or air humidity in the environment is reduced through the intake fan 2 and the exhaust fan 30, and the air pressure in the box 1 is kept balanced.

[0060] There are guide wheels 27 and a drive motor 24 arranged below the orbital mobile platform 19. The orbital mobile platform 19 includes a transverse guide rail 25 and a longitudinal guide rail 26. The surfaces of the transverse guide rail and the longitudinal guide rail are non-smooth surfaces. The transverse guide rail 15 is arranged at the bottom of the closed box. There is a guide rail groove under the longitudinal guide rail 16, and it contacts the transverse guide rail through the guide rail groove. The guide wheel 27 contacts the longitudinal guide rail 26. The drive motor 24 is connected to the guide wheel 27. The drive motor 24 drives the guide wheel 27 to move so that the platform makes a longitudinal movement. An electric push rod 28 is installed on the left side of the longitudinal guide rail 26. The electric push rod 28 is connected to the longitudinal guide rail 26. The longitudinal guide rail 26 is pushed by the electric push rod 28 to make a transverse movement, realizing the movement of the orbital mobile platform 19 in the guide rail plane and simulating the walking operation process of the coal mine robot. Figure 2As shown, the rail-mounted mobile platform can be implemented based on the shearer robot, or replaced by a rail-mounted inspection robot, a tunneling robot, etc.

[0061] The experimental analysis module includes a computer and related software. The sensor transmits the collected data to the computer, and the computer and related software analyze and process the data; according to the experimental equipment and research content, a digital twin simulation engine software development platform is constructed to realize the perception modeling of the operation process of the coal mine robot and the visualization of the digital twin monitoring effect.

[0062] A method for a geological scene perception modeling system of a coal mine robot specifically includes the following steps:

[0063] S1: Through the coal mine robot working environment simulation module and the rail-mounted mobile platform, change the dust concentration, illuminance, and air humidity in the box environment and simulate the operation process of the coal mine robot;

[0064] In step S1, the coal mine robot working environment simulation module controls the start and stop of the dust generator, water vapor generator, lighting device, dehumidification and dust removal equipment, dust concentration detection sensor, illuminance detection sensor, and air humidity detection sensor through the central controller. The application of the coal mine robot working environment simulation module can be randomly combined and adjusted according to the actual experimental content.

[0065] S2: Obtain the information of the scene in the complex environment and the information of the coal seam geological simulation module through the multi-source perception module of the coal mine robot, and construct a geological and scene joint data set; in step S2, use a ground-penetrating radar and an infrared camera to detect the coal seam geological simulation module and collect relevant data; use lidar, frequency-modulated continuous wave radar, and visible light camera blocks to obtain the original information of the scene in the complex environment.

[0066] S3: Use the scene data obtained in step S2 to study the noise modeling and data enhancement methods of the scene observation multi-modal heterogeneous sensors; in step S3, based on the control experiment analysis, master the forms and influence laws of the environment and working conditions on the original information of the sensors, and establish the noise observation models of each sensor.

[0067] S4: Use the geological data obtained in step S2 to construct a radar echo forward and inverse data set, and study the propagation characteristics of the ground-penetrating radar echo signal and the abnormal body recognition method in the cross-media process; in step S4, use the gprMax simulation software to design and establish a simulation coal seam geological simulation module according to the constructed coal seam geological physical model parameters, and perform forward simulation on the simulation model.

[0068] S5: Process the data through the experimental analysis module, study the correlation of geological and scene multi-scale feature data and the joint interpretation and perception enhancement method of multi-source information, and complete the multi-source information fusion perception; in step S5, process the sensor data through the experimental analysis module, explore the multi-scale visualization imaging method of the sensor, and construct a multi-scale data representation method, which specifically includes the following steps:

[0069] S51: Perform time matching by designing a time sliding window, and propose a data association method for pixel-level alignment of sensor data on the constructed synthetic image;

[0070] S52: Use the synthetic image constructed by multi-scale features to construct a loss function based on data association metrics;

[0071] S53: Use a data-driven multi-task learning method to infer the damaged type and degree of multi-scale features, and realize the joint interpretation of multi-modal information;

[0072] S54: Perform feature prediction on the synthetic image under the condition of feature loss caused by environmental interference, compensate for the damaged information, and realize the perception enhancement of sensor feature information under complex conditions.

[0073] S6: Use the geological and scene multi-source fusion perception information to construct an integrated three-dimensional model of geology and scene, and at the same time use the boundary region features of the geological model and the scene model to correct the integrated three-dimensional model. In step S6, use linear interpolation and point cloud discretization to construct a three-dimensional geological point cloud model, which specifically includes the following steps:

[0074] S61: Obtain scene model data based on the simultaneous localization and mapping technology of multi-sensor fusion, and complete the construction of the scene point cloud model;

[0075] S62: Use the scene point cloud model and the geological point cloud model to fit the air-coal rock medium surface respectively;

[0076] S63: Cross-validate the accuracy of the scene point cloud model and the geological point cloud model based on the spatial continuity of the junction area;

[0077] S64: Use the elevation model of the high-precision scene point cloud model to correct the geological point cloud model to achieve dynamic optimization;

[0078] S65: Obtain the coal-rock interface recognition result using the optimized scene point cloud model;

[0079] S66: Verify and update the physical properties of the medium of the geological point cloud model, and realize the dynamic correction of the geological model for multi-source information joint inversion;

[0080] S67: By constructing a new four-channel point cloud data storage data format, based on the three-dimensional spatial coordinates and intensity information, the intensity information is redesigned into a data channel that reflects different medium types of gas-coal-rock, realizing the consistent storage and integrated display of geological and scene point cloud models.

[0081] S7: Use the ground truth measurement device to obtain the ground truth of the scene three-dimensional model and the pose of the coal mine robot, and compare it with the multi-source information fusion perception and three-dimensional model results obtained in steps S5 and S6 to evaluate the experimental effect of the coal mine robot geological scene perception modeling method.

[0082] Embodiment

[0083] A coal mine robot geological scene perception modeling system and method. During the scene modeling experiment, as Figure 3 and Figure 4 shown, create clean environments, single dust environments, single water vapor environments, single lighting environments, dust-water vapor environments, dust-lighting environments, water vapor-lighting environments, dust-water vapor-lighting environments and different dust concentrations, air humidities, light intensities, etc. in the same environment through the coal mine robot working environment simulation module. Drive the motor and electric push rod through program control to make the rail-mounted mobile platform move on the non-smooth guide rail to simulate the vibration conditions during the operation of the coal mine robot. The platform is equipped with a lidar, a visible light camera, an infrared camera, and a frequency-modulated continuous wave radar to collect scene information in various environments, construct an original data set, and conduct comparative analysis and processing on the data collected based on different environments or different parameter information in the same environment through a computer and related software to explore the forms and influence laws of environmental and working conditions on the original information of the sensors, establish a noise observation model for each sensor, and propose methods to improve the information quality according to the experimental results. Through the experimental analysis module, conduct scene modeling and multi-sensor modeling information fusion on the data collected by the sensors, and compare and analyze with the ground truth data of the scene model obtained by the ground truth measurement device to explore the scene modeling accuracy and error of each sensor under different working conditions. According to the accuracy, advantages and disadvantages of each sensor modeling, study the distribution problem of information weights during multi-sensor data fusion, propose a construction method for cross-modal features, evaluate the intensity and direction of constraints provided by different modal features, and use the method of optimization decision-making and learning to realize the automatic adjustment of weights for different environmental working conditions during information fusion. Based on the factor graph optimization framework, realize the real-time fusion of multi-sensor data by constructing an efficient optimization solver. During the geological modeling experiment, as Figure 5, the driving motor and the electric push rod are controlled by a program to make the rail-mounted mobile platform move on an uneven guide rail, simulating the vibration conditions during the operation of a coal mine robot. The platform is equipped with a ground-penetrating radar and an infrared camera to detect and collect raw data from the constructed coal seam geological simulation module. At the same time, the gprMax simulation software is used to design and establish a simulated coal seam geological simulation module based on the physical property parameters of the constructed coal seam geological simulation module, and forward simulation is carried out on the simulated coal seam geological simulation module. Through the experimental analysis module, the forward data and the data truly collected by the ground-penetrating radar are compared, analyzed, and tested to construct a dataset corresponding to the wave field response characteristics of typical geological structures and the true structure parameters. Combining software simulation forward simulation and ground-penetrating radar simulation experiments, the propagation characteristics of the echo signals of the ground-penetrating radar during the gas-coal-rock cross-media process are studied. By changing parameters such as the polarization direction of the ground-penetrating radar antenna and the different physical property parameters of the coal seam geology, the influence mechanism of different conditions on the echo signals of the ground-penetrating radar is explored, and the relationship between factors such as the dielectric constant, electromagnetic characteristics, radar frequency, and antenna deployment in typical structures of coal and rock geology is clarified; through the experimental analysis module, the software simulation and actual simulation experiment data are analyzed and processed to establish a radar echo dataset for different geological structures, realizing the identification of different geological anomalies and providing a basis for the inversion imaging and modeling of the ground-penetrating radar; using methods such as multi-profile and linear interpolation to process the B-scan data collected in the experiment, and determining the spatial position and size of the geological structure anomaly through the radar echo time difference and the propagation speed of electromagnetic waves in the medium, realizing the three-dimensional modeling of the coal seam geology. The three-dimensional geological model established by the simulation experiment is compared and analyzed with the coal seam geological simulation module with known structure sizes to explore the accuracy and error of the three-dimensional geological modeling under different conditions. During the process of the integrated geological scene modeling experiment, as Figure 6 shown, the pose information of sensors such as lidar, visible light camera, infrared camera, and ground-penetrating radar is obtained through a truth measurement device. The data information of the lidar, visible light camera, and frequency-modulated continuous wave radar on the scene and the data information of the ground-penetrating radar and infrared camera on the coal seam geological simulation module are collected simultaneously through the relevant steps in the scene modeling experiment and the relevant steps in the geological modeling experiment. Through the experimental analysis system, three-dimensional modeling of the collected scene and geological information is carried out. Combining multiple constraints such as geological observation information and multi-source observation information of the scene, and using the sensor pose information, the coordinate systems of the two models are unified to achieve integrated geological scene modeling. According to the scene truth and the truth of the geological model obtained by the truth measurement device, the accuracy and error of the integrated geological scene modeling are analyzed, the description method of the integrated geological and scene model is studied, a multi-layer probability grid map construction strategy is proposed, and seamless connection and homogeneous expression of the geological model and the scene model are realized. By designing a model dynamic correction mechanism for mutual verification and joint inversion of the scene model and the geological model, high-efficiency and high-precision modeling of the geology and scene of the fully mechanized coal mining face is realized.

[0084] Therefore, the present invention adopts the above-mentioned geological scene perception and modeling system and method for coal mine robots. Through the coal mine robot working environment simulation module and the rail-mounted mobile platform, it can change the dust concentration, illuminance, air humidity in the box environment and simulate the operation process of the coal mine robot. Through the multi-source perception module, it can obtain complex scene perception information, construct a data set, and explore the noise modeling and data enhancement methods of multi-modal heterogeneous sensors for scene observation; obtain geological data in complex environments, construct a forward and inverse radar echo data set, and explore the propagation characteristics of ground penetrating radar echo signals and the abnormal body recognition method in the cross-media process; process the data through a computer and related software, explore the multi-scale feature data association and the multi-source information joint interpretation and perception enhancement method, complete the multi-source information fusion, and realize the integrated real-time modeling of the geological scene. It can not only carry out scientific research activities such as the integrated modeling of the geological scene in the operation process of coal mine robots, but also be used for geological exploration and inspection of coal mine roadways through different combinations of sensors, providing relevant information for roadway stability maintenance and repair.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that they can still modify or equivalently replace the technical solutions of the present invention, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A coal mine robot geological scene perception modeling system, comprising a closed box composed of a wood board, an iron sheet, and an insulation layer, a scene feature simulation device arranged on the left side of the closed box, and a true value measurement device arranged inside the closed box, characterized in that: It also includes a coal mine robot working environment simulation module arranged in the closed box, a coal seam geology simulation module arranged in the closed box, a track-type mobile platform arranged in front of the coal seam geology simulation module, and a multi-source perception module mounted on the track-type mobile platform; the multi-source perception module, the true value measurement device and the coal mine robot working environment simulation module are all connected to the experimental analysis module. The coal mine robot working environment simulation module includes a dust generator, a water vapor generator, a lighting device, a dehumidification and dust removal equipment, a dust concentration detection sensor, an illumination detection sensor, and an air humidity detection sensor; the dust generator is arranged at the bottom of the coal powder box, the lighting device includes a controllable lighting lamp, and the dehumidification and dust removal equipment includes an intake fan and an exhaust fan; The three-dimensional scene information and the position information of the sensors in the track-type mobile platform and the multi-source perception module are obtained through the true value measurement device, and the distance between the multi-source perception module and the coal seam geological simulation module is changed through the track-type mobile platform to thereby change the perception information of the multi-source perception module; The method of using the system includes the following steps: S1: Through the coal mine robot working environment simulation module and the track-type mobile platform, the dust concentration, illumination and air humidity in the box environment are changed and the operation process of the coal mine robot is simulated; S2: The coal mine robot multi-source perception module obtains scene information in complex environments and coal seam geological simulation module information to build a joint geological and scene data set; S3: Using the scene data obtained in step S2, research is conducted on noise modeling and data enhancement methods for scene observation multi-mode heterogeneous sensors; S4: Using the geological data obtained in step S2, construct a radar echo forward and inversion data set to study the propagation characteristics of ground penetrating radar echo signals in cross-medium processes and the anomaly recognition method; S5: Process the data through the experimental analysis module, study the association of geological and scene multi-scale feature data and the joint interpretation and perception enhancement methods of multi-source information, and complete the fusion perception of multi-source information; S6: Use geological and scene multi-source fusion perception information to build a geological and scene integrated three-dimensional model, and use the boundary area characteristics of the geological model and the scene model to modify the integrated three-dimensional model; S7: Use the truth measurement device to obtain the true value of the scene three-dimensional model and the coal mine robot posture, and compare them with the multi-source information fusion perception and three-dimensional model results obtained in step S5 and step S6 to evaluate the experimental effect of the coal mine robot geological scene perception modeling method.

2. A coal mine robot geological scene perception modeling system according to claim 1, characterized in that: The material of the scene feature simulation device is set to a wooden structure, and the position and posture are adjusted according to the experimental content to simulate different scenes; the true value measurement device includes a laser tracker and a laser scanner, which uses the laser scanner to obtain scene information, and uses the laser tracker and target to obtain the position and posture information of the track-type mobile platform and the sensor; the coal seam geological simulation module includes coal blocks and rock blocks of different shapes, which are cut from complete coal blocks and rock blocks according to requirements, and are quickly recombined into different simulated geological conditions by stacking coal blocks and rock blocks of different shapes.

3. A coal mine robot geological scene perception modeling system according to claim 1, characterized in that: The multi-source perception module is composed of various sensors mounted on a tracked mobile platform and deployed jointly according to the experimental content. The sensors include ground penetrating radar, lidar, frequency-modulated continuous wave radar, visible light camera, infrared camera and inertial navigation unit.

4. A coal mine robot geological scene perception modeling system according to claim 1, characterized in that: Guide wheels and a drive motor are arranged under the track-type mobile platform. The track-type mobile platform includes a transverse guide rail and a longitudinal guide rail. The surfaces of the transverse guide rail and the longitudinal guide rail are both non-smooth surfaces. The transverse guide rail is arranged at the bottom of the closed box. A guide rail groove is opened under the longitudinal guide rail. The guide rail groove is in contact with the transverse guide rail, and the guide wheel is in contact with the longitudinal guide rail. The drive motor is connected to the guide wheel, and the drive motor drives the guide wheel to move. An electric push rod is installed on the left side of the longitudinal guide rail, and the electric push rod is connected to the longitudinal guide rail. The longitudinal guide rail is pushed to make a transverse movement through the electric push rod.

5. A coal mine robot geological scene perception modeling system according to claim 1, characterized in that: The experimental analysis module includes computers and related software. The sensors transmit the collected data to the computer, and the data is analyzed and processed through the computer and related software. According to the experimental equipment and research content, a digital twin simulation engine software development platform is constructed to realize the perception modeling of the coal mine robot operation process and the visualization of the digital twin monitoring effect.

6. A method for a coal mine robot geological scene perception modeling system based on any one of claims 1 to 5, characterized in that The specific steps include: S1: Through the coal mine robot working environment simulation module and the track-type mobile platform, the dust concentration, illumination and air humidity in the box environment are changed and the operation process of the coal mine robot is simulated; S2: The coal mine robot multi-source perception module obtains scene information in complex environments and coal seam geological simulation module information to build a joint geological and scene data set; S3: Using the scene data obtained in step S2, research is conducted on noise modeling and data enhancement methods for scene observation multi-mode heterogeneous sensors; S4: Using the geological data obtained in step S2, construct a radar echo forward and inversion data set to study the propagation characteristics of ground penetrating radar echo signals in cross-medium processes and the anomaly recognition method; S5: Process the data through the experimental analysis module, study the association of geological and scene multi-scale feature data and the joint interpretation and perception enhancement methods of multi-source information, and complete the fusion perception of multi-source information; S6: Use geological and scene multi-source fusion perception information to build a geological and scene integrated three-dimensional model, and use the boundary area characteristics of the geological model and the scene model to modify the integrated three-dimensional model; S7: Use the truth measurement device to obtain the true value of the scene three-dimensional model and the coal mine robot posture, and compare them with the multi-source information fusion perception and three-dimensional model results obtained in step S5 and step S6 to evaluate the experimental effect of the coal mine robot geological scene perception modeling method.

7. The method according to claim 6, characterized in that: In step S1, the coal mine robot working environment simulation module controls the shutdown and operation of the dust generator, water vapor generator, lighting device, dehumidification and dust removal equipment, dust concentration detection sensor, illumination detection sensor and air humidity detection sensor through the central controller. The application of the coal mine robot working environment simulation module can be arbitrarily combined and adjusted according to the actual experimental content.

8. The method according to claim 6, characterized in that: In step S2, ground penetrating radar and infrared camera are used to detect the coal seam geological simulation module and collect relevant data; laser radar, frequency modulated continuous wave radar and visible light camera are used to obtain the original information of the scene in the complex environment.

9. The method according to claim 6, characterized in that: In step S3, based on the control experiment analysis and the form and influence of the environment and working conditions on the original information of the sensor, a noise observation model of each sensor is established.

10. The method according to claim 6, characterized in that: In step S4, the gprMax simulation software is used to design and establish a simulated coal seam geological simulation module according to the constructed coal seam geological physical model parameters, and a forward simulation is performed on the simulation model.

11. The method according to claim 6, characterized in that: In step S5, the sensor data is processed by the experimental analysis module, the multi-scale visualization imaging method of the sensor is explored, and a multi-scale data characterization method is constructed, which specifically includes the following steps: S51: By designing a temporal sliding window for temporal matching, a data association method for pixel-level alignment of sensor data on constructed artificial synthetic images is proposed; S52: Artificially synthesized images constructed using multi-scale features and loss functions based on data association metrics; S53: Use data-driven multi-task learning methods to infer the damage type and degree of multi-scale features and realize the joint interpretation of multimodal information; S54: Perform feature prediction on artificially synthesized images under conditions where features are missing due to environmental interference, perform feature compensation on damaged information, and achieve enhanced sensor feature information perception under complex conditions.

12. The method according to claim 6, characterized in that: In step S6, linear interpolation and point cloud discretization are used to construct a three-dimensional geological point cloud model, which specifically includes the following steps: S61: Acquire scene model data based on multi-sensor fusion simultaneous positioning and mapping technology to complete the construction of scene point cloud model; S62: using the scene point cloud model and the geological point cloud model to fit the air-coal rock medium surface respectively; S63: Cross-validate the accuracy of the scene point cloud model and the geological point cloud model based on the spatial continuity of the boundary area; S64: Use the high-precision elevation model of the scene point cloud model to correct the geological point cloud model to achieve dynamic optimization; S65: Obtaining coal-rock interface recognition results using the optimized scene point cloud model; S66: Verify and update the medium physical properties of the geological point cloud model to achieve dynamic correction of the geological model through joint inversion of multi-source information; S67: By constructing a new four-channel point cloud data storage data format, the intensity information is redesigned into data channels for reacting different media types such as gas-coal-rock on the basis of spatial three-dimensional coordinates and intensity information, so as to achieve consistent storage and integrated display of geological and scene point cloud models.

Images