Ultrathin coal seam working face three-dimensional modeling method and system based on multi-source data fusion

Through multi-source data fusion technology, geological exploration, laser point cloud and image data are integrated, and virtual and real models are established, which solves the accuracy and reliability of extremely thin coal seam modeling, and achieves higher accuracy and comprehensive extremely thin coal seam work. Surface modeling supports intelligent unmanned mining.

CN119027600APending Publication Date: 2024-11-26CHINA UNIV OF MINING & TECH +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410978686.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

There are problems in the mining of extremely thin coal seams, such as data acquisition difficulties, limited model accuracy, lack of perfect verification methods and insufficient special modeling, resulting in limited accuracy and reliability of modeling of extremely thin coal seams comprehensive mining faces.

Method used

Using a multi-source data fusion method, through the integration of geological exploration data, laser point cloud data, image data and sensor data, and using 3D-CVF and convolutional neural network technologies, virtual and real models are established and multi-model data fusion is carried out to generate More accurate and reliable three-dimensional model of extremely thin coal seam working surface.

Benefits of technology

It improves the accuracy and reliability of extremely thin coal seam working surface modeling, enhances the understanding and perception of the mining environment, meets the high-precision and comprehensive modeling needs, and supports the application of intelligent unmanned mining systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027600A_ABST
    Figure CN119027600A_ABST
Patent Text Reader

Abstract

The invention discloses an ultra-thin coal seam working face three-dimensional modeling method and system based on multi-source data fusion. The system comprises a laser point cloud data acquisition subsystem, a camera image acquisition subsystem and an ultra-thin coal seam hydraulic support attitude regulation and control data modeling subsystem. According to the method, geological modeling is firstly carried out on geological exploration data, and then point cloud data and image data are fused, so that the modeling speed is improved, the consistent fusion of the data is ensured, and a more comprehensive and more accurate working face model is obtained; various data such as point cloud data, image data and sensor data are fused, the modeling effect which can be achieved is beneficial to improving the understanding and sensing ability of an intelligent unmanned mining system to the working face environment, and the requirements for the precision and comprehensiveness of a working face model can be better met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for three-dimensional modeling of an ultra-thin coal seam working face, in particular to a method and system for three-dimensional modeling of an ultra-thin coal seam working face based on multi-source data fusion, belonging to the technical field of coal seam mining. Background Art

[0002] In thin coal seam mining, the thickness of the coal seam may be only a few meters or even less, which poses a challenge to the planning, design and management of the mining face. Therefore, the purpose of modeling the mining face of extremely thin coal seams is to better understand the spatial distribution, geological structure, mining conditions, and mining conditions of the coal seams, so as to guide the coal mining process. The modeling of the mining face of extremely thin coal seams usually includes the following: 1. Geological modeling: Establish a geological model of the coal seam and its surrounding strata, including stratum distribution, lithology, fault conditions, etc. The establishment of a geological model is very important for understanding the distribution and structure of the coal seam.

[0003] 2. Geometric modeling of the working face: According to the geological conditions and coal mining technology, the geometric model of the mining working face is established. This includes determining the location, shape, size and other parameters of the mining working face, as well as planning mining tunnels and support structures.

[0004] 3. Coal mining machine trajectory simulation: simulate the movement trajectory of the coal mining machine on the working face to determine the best coal mining route and working method. Considering the thin thickness of the coal seam, special attention needs to be paid to the layout of the coal mining machine and the design of the movement trajectory.

[0005] 4. Support structure modeling: Design and model the support structure of the working face, including pillars, brackets, roof support, etc. Due to the thin thickness of the coal seam, the design of the support structure needs to consider the stability and effectiveness of the support.

[0006] 5. Model optimization and analysis: Optimize and analyze the established working face model, including evaluation of the model's stability, safety, and economy. Guide the planning and management of the mining working face through model analysis. Modeling of the mining working face of extremely thin coal seams is an important part of coal mine production management. It can provide scientific planning and decision-making support for coal mine production and improve mining efficiency and safety.

[0007] With the reduction of thick and medium-thick coal seam reserves and the increasing scarcity of resources, the mining of extremely thin coal seams has attracted great attention from the country. Data collection for extremely thin coal seams is more difficult, and due to the large changes in coal seams, the accuracy of the model obtained by single-modal data modeling is not high, so a more accurate and more precise modeling method is needed for extremely thin coal seams.

[0008] At present, modeling of fully mechanized mining working faces in extremely thin coal seams has the following shortcomings: 1. Difficulty in data acquisition: It is often difficult to accurately obtain data on the geological conditions, occurrence characteristics, and thickness of extremely thin coal seams, because these data usually need to be obtained through drilling, measurement, etc., which is difficult to operate for extremely thin coal seams and the cost of data acquisition is high.

[0009] 2. Limited model accuracy: Since the mining process of extremely thin coal seams is affected by many factors, such as geological conditions, mining site parameters, coal seam deformation characteristics, etc., the interactions between these factors are complex and it is difficult to establish a completely accurate model. Therefore, the current modeling of fully mechanized mining working faces in extremely thin coal seams often has certain accuracy limitations, and the model results may differ from the actual situation.

[0010] 3. Lack of perfect model verification method: There is a lack of sufficient practical verification and verification standards for modeling of fully mechanized mining faces in extremely thin coal seams. Existing verification methods often rely on industry experience and case analysis, and it is difficult to provide quantitative verification results, which limits the credibility and reliability of model results.

[0011] 4. Insufficient modeling for the particularity of extremely thin coal seams: The mining process and deformation characteristics of extremely thin coal seams are significantly different from those of traditional coal seams, and the current fully mechanized mining face modeling method tends to be suitable for the modeling method of traditional coal seam mining, which is insufficient for the particularity of extremely thin coal seams. Summary of the invention

[0012] The purpose of the present invention is to provide a method and system for three-dimensional modeling of ultra-thin coal seam working faces based on multi-source data fusion in order to solve at least one of the above technical problems.

[0013] The present invention achieves the above-mentioned purpose through the following technical scheme: a method for three-dimensional modeling of an ultra-thin coal seam working face based on multi-source data fusion, the method comprising the following steps: S1. Before mining, geological exploration data is obtained by measuring the extremely thin coal seam, and the geological exploration data is processed and stored. Based on the geological exploration data, geological modeling is performed to obtain a geological model M0, thereby obtaining coal seam characteristics. Then, according to the characteristics of the coal seam characteristics, the starting positions of the hydraulic support and the coal mining machine in the coal seam are planned and a three-dimensional space coordinate system is established. The subsequent working face modeling data is integrated based on this three-dimensional space coordinate system; S2, based on the laser point cloud data acquisition system set on the coal mining machine, point cloud data M1 is obtained, and GPS positioning is performed, the point cloud data M1 is transmitted to the ground, and then the point cloud data M1 is processed and stored, and image data M2 is obtained by using a camera image acquisition system fixed on the hydraulic support and moving with the coal mining machine, and GPS positioning is performed, and the image data M2 and coordinate data are transmitted to the ground, and then the image data M2 and coordinate data are processed and stored; S3, extracting the point cloud data M1 and image data M2 of the working surface, unifying the coordinate system and extracting the features of the two, and fusing them using the 3D-CVF method to obtain a comprehensive data set; S4, the geological model M0 is integrated with the comprehensive data set, and three-dimensional modeling is performed based on the application scenario to obtain a new working face model M 012 ; S5, using equipment parameters and attitude control instructions to perform virtual modeling to obtain a virtual model, using the three-machine attitude data extracted by sensors to perform real modeling to obtain a real model, and splicing the two models through a virtual platform, detecting their errors, and performing correction processing to obtain a virtual-real model M3; S6, obtain the geological model M0 data and the virtual-real model M3 data, embed the virtual-real model M3 into the geological model M0, and obtain the virtual-real new model M 03 ; S7, the new working surface model and the virtual-real new model M 03 By fusing multiple model data, we can complete the 3D modeling of the ultra-thin working surface and obtain the comprehensive model M. 0123 .

[0014] As a further solution of the present invention: in S1, the geological exploration data is used for geological modeling, including: S11. Extract the geological exploration data before mining in the storage system, including the scope and location of the coal seam, the coal cover, the rock layer, the fault, and the geological elements of the geological boundary, and clean and pre-process them to remove abnormal data, noise and invalid data; S12. Perform spatial analysis, statistical analysis and correlation analysis on the processed data to identify the spatial distribution patterns and characteristics of coal seam geological data; S13. Establish a preliminary geological model M0 framework, set three-dimensional coordinates in the framework, visualize the coal seam geological model M0, and layout and design the mining face based on the visualization platform.

[0015] As a further solution of the present invention: in S2, the collection of the point cloud data M1 specifically includes: S211, using a three-dimensional laser scanner arranged on the coal mining machine and the mobile support, when the coal mining machine and the conveyor are working, the three-dimensional laser scanner continuously follows up, emits a laser beam in real time and records the time difference and intensity of the returned laser beam; S212, according to the posture change characteristics of the hydraulic support during the mining process, after the coal mining machine cuts coal and feeds, the hydraulic support at the corresponding position will be displaced and change its posture according to the issued command, while the hydraulic support at the non-mining position of the coal mining machine is in a static state, and the support posture does not change, so the scanner on the coal mining machine focuses on scanning the hydraulic support that is moving at this time to obtain the point cloud data M1 of the complete working surface; S213, using the transmission equipment to transmit the collected point cloud data M1 and the location information obtained by GPS positioning to the main control room, and then transmit it to the ground through the 5G communication module; S214, after receiving the data, the ground extracts the coordinate and intensity information, and then performs data segmentation, elimination, alignment, and splicing preprocessing steps on the point cloud data M1 collected by multiple deployed scanners, and stores the processed data.

[0016] As a further solution of the present invention: in S2, the acquisition of the image data M2 specifically includes: S221. In addition to the three-dimensional scanner, a camera is installed every five hydraulic supports to achieve full coverage of the mining face and real-time monitoring; S222. The camera installed on the coal mining machine collects the real-time images of the hydraulic support posture movement and the coal mining machine when it is working, and observes the working status during coal mining. The cameras installed on the conveyor and the hydraulic support collect the real-time images of the working face when it is not mined, and monitor the support status of the hydraulic support, whether there is roof broken, floor weak, and coal seam spalling phenomenon; S223, using the transmission equipment to transmit the collected image data M2 and the location information obtained by GPS positioning to the main control room, and then transmit it to the ground through the 5G communication module; S224. After the data is received on the ground, it is preprocessed, including denoising, image correction, color correction, and coordinate conversion. Due to poor lighting under the coal seam, the naked eye cannot directly observe it. Therefore, when processing the data, it is necessary to use image data M2 training to train the model under the coal seam and adjust the appropriate neural weights. If the roof is broken, the target is directly extracted.

[0017] As a further solution of the present invention: in S3, the specific steps of fusing the point cloud data M1 with the image data M2 using the 3D-CVF (3D cross view fusion) method include: S31, extracting the point cloud data M1 and image data M2 of the working face, reprocessing the data, extracting features from the point cloud data M1 and the image data M2, matching them, using the coordinate information, selecting a common reference point, determining the coordinate zero point, aligning it with the coordinate in the geological model M0, and establishing a corresponding relationship between the point cloud data M1 and the image data M2; S32, extracting features from the point cloud data M1, selecting the point cloud space to be divided on the established spatial coordinate system, sampling the point cloud features generated above in the form of two-dimensional convolution, and forming point cloud bird's-eye view feature maps of different resolutions; S33, upsampling is performed in the form of two-dimensional deconvolution, and the upsampling results are spliced ​​to obtain the final point cloud bird's-eye view features; S34, performing the same operation on the image data M2 in the same manner as the point cloud data M1, to obtain the final image bird's-eye view features; S35, extract the feature information of the bird's-eye view of the two by using the global attention mechanism, and perform channel-by-channel weighting operation on each information channel. Finally, apply the pooling based on the three-dimensional region of interest to fuse the point cloud bird's-eye view features with the image bird's-eye view features to obtain the fused comprehensive dataset M 12 .

[0018] As a further solution of the present invention: in S4, the fusion of the geological model M0 and the comprehensive data set specifically includes: The coordinates of the geological model M0 and the comprehensive dataset are aligned, and the comprehensive dataset is projected to the corresponding area of ​​the geological model M0 in space. The convolutional neural network CNN is also used for grid matching, data interpolation, and data fitting steps to establish the relationship between the comprehensive dataset and the geological model M0. The comprehensive dataset is embedded in the geological model M0, and the working face galaxy and texture information are reconstructed to form a new working face model M0. 012 .

[0019] As a further solution of the present invention: in S5, the splicing and establishment of the virtual-real model M3 specifically includes: S51. Establishing a virtual model S511, collecting working face parameters, including hydraulic support, coal mining machine, conveyor parameters, including the position and orientation of the coal mining machine, and the layout of the hydraulic support, after configuring the working face, determining its coordinates in the coal seam, and aligning the coordinates to obtain the layout and position of the three equipment in the coal seam; S512, recording and transmitting the three-machine control instructions issued by the terminal to the virtual platform, adding interactive effects, that is, the working state of the coal mining machine, the position and movement of the hydraulic support, and the transportation speed of the conveyor obtained by the operation instructions, simulating the actual mining process of the working face, and completing the establishment of the virtual model; S52. Build a real model S521. Collect the data of coal mining machine movement, hydraulic support movement, support extension and extension, and support inclination monitored by sensors, perform preliminary amplification and filtering, transmit the data to the main control room through the 5G communication module and then reach the ground, clean, interpolate, and align the coordinate system to ensure data consistency and accurate coordinates; S522, based on the parameters of the three machines of the working face and the posture data obtained by sensor monitoring, use 3D modeling software to model the three machine elements of the coal mining machine, hydraulic support, and ensure that their positions, orientations, and sizes are consistent with the actual situation, that is, to obtain a real model of the working face; S53, the virtual model is spliced ​​with the real model. The splicing is completed on the virtual platform according to the data of the two models, the error between the virtual model and the real model is detected, and the position correction processing is performed according to the sensor monitoring value to obtain the spliced ​​and corrected virtual and real model M3.

[0020] As a further solution of the present invention: in S6, the fusion of the virtual and real model M3 and the geological model M0 specifically includes: S61, obtaining two model data, aligning the virtual model with the indicators in the geological model M0 based on the coordinates of the geological model M0, and then projecting them into the geological model M0, matching feature points or feature areas, and fusing the two data by interpolation processing according to the matching results; S62, according to the consistency of the shape and characteristics of the actual working face, the working face is fine-tuned as needed to complete the embedding of the virtual-real model M3 into the geological model M0 to obtain the virtual-real new model M 03 .

[0021] As a further solution of the present invention: in S7, the new working surface model and the new virtual-real model M 03 The multi-model data fusion specifically includes: S71, acquiring two model data, and registering and aligning the two sets of data to make their positions and postures consistent in the three-dimensional space; S72. Compare the measured working surface model data and the virtual model data, match the corresponding feature points or feature areas, use the convolutional neural network CNN to match the feature points, fuse the two sets of models, and obtain the final comprehensive model M. 0123 ; S73, the comprehensive model M 0123 Verify and adjust to ensure that the fused model is consistent with the shape and features of the actual working surface, compare and verify with the actual scene, and make fine-tuning and adjustments as needed; S74, the comprehensive model M 0123 Provide a visual display.

[0022] A system used in a method for three-dimensional modeling of an ultra-thin coal seam working face based on multi-source data fusion, the system comprising a laser point cloud data acquisition subsystem, a camera image acquisition subsystem, and an ultra-thin coal seam hydraulic support posture control data modeling subsystem; Wherein, the laser point cloud data acquisition subsystem includes a three-dimensional laser scanner, a first GPS positioning device, a first 5G communication module and a terminal execution module; The camera image acquisition subsystem includes a camera, a second GPS positioning device, a second 5G communication module and a second terminal execution module; The ultra-thin coal seam hydraulic support posture control data modeling subsystem includes a coal mining machine and a conveyor belt, an ultra-thin coal seam hydraulic support, a sensor, a data acquisition device, a third 5G communication module and a third terminal execution module.

[0023] The beneficial effects of the present invention are: 1) In view of the particularity of extremely thin coal seams, the present invention adopts data fusion based on geological models, which facilitates the layout before mining at the mining face and can fully utilize the data collected during the entire modeling process; 2) The present invention aims to fuse point cloud data with image data to obtain real-time information of the mining working face, integrating the advantages of real-time image modeling and accuracy of point cloud data; 3) For data that cannot be collected by 3D laser scanners and cameras, the present invention also integrates the model parameters of the three machines, the data collected by sensors, etc., and performs virtual-real modeling to improve the information of the mining working face; 4) The present invention adopts a method of first performing geological modeling on geological exploration data and then fusing point cloud data and image data, which not only improves the speed of modeling, but also ensures the consistent fusion of data, and obtains a more comprehensive and accurate working face model; 5) The present invention integrates multiple data such as point cloud data, image data, sensor data, etc. The modeling effect it can achieve helps to improve the intelligent unmanned mining system's understanding and perception of the working face environment, and can better meet the requirements for the accuracy and comprehensiveness of the working face model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the overall process structure of the present invention; Figure 2 This is a schematic diagram of the process of fusing laser point cloud data and camera image data in the present invention; Figure 3 It is a schematic diagram of the system framework for implementing the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] Embodiment 1, as Figures 1 to 3 As shown, a method for 3D modeling of an extremely thin coal seam working face based on multi-source data fusion, the method comprises the following steps: S1. The geological exploration data obtained from the actual measurement of the extremely thin coal seam before mining is processed and stored, and geological modeling is performed based on the geological exploration data to obtain the geological model M0, thereby obtaining the coal seam characteristics, and then the starting positions of the hydraulic support and the coal mining machine in the coal seam are planned according to the characteristics of the coal seam and a three-dimensional space coordinate system is established, and the subsequent working face modeling data is integrated based on this coordinate system; S2, obtaining point cloud data M1 through a laser point cloud data acquisition system provided on the coal mining machine, performing GPS positioning based on the obtained point cloud data M1, transmitting the complete point cloud data M1 to the ground, and then processing and storing the data, obtaining image data M2 through a camera image acquisition system fixed on a hydraulic support and moving with the coal mining machine, transmitting the complete image data M2 and coordinate data to the ground through GPS positioning, and then processing and storing the data; S3, extract the point cloud data M1 and image data M2 of the working face, reprocess them, unify the coordinate system and extract the features of the two, and then fuse them based on the geological model M0 using the 3D-CVF (3D cross view fusion) method; S4, the geological model M0 is integrated with the comprehensive data set, and three-dimensional modeling is performed based on the application scenario to obtain a new working face model; S5, using equipment parameters and attitude control instructions to perform virtual modeling to obtain a virtual model, using the attitude data of the three machines extracted by sensors to perform real modeling to obtain a real model, the virtual platform splices the two models (virtual model and real model), detects their errors, performs correction processing, and obtains a virtual-real model M3; S6, obtain geological model M0 data and virtual-real model M3 data, embed the virtual-real model M3 data into geological model M0, and obtain a new virtual-real model M 03 ; S7, the new working surface model and the virtual and real new model M 03 By fusing multiple model data, we can complete the 3D modeling of the ultra-thin working surface and obtain the comprehensive model M. 0123 .

[0027] Embodiment 2: In addition to all the technical features of Embodiment 1, this embodiment also includes: Geological modeling based on geological exploration data includes: S11, extracting geological exploration data before mining from the storage system, the geological exploration data including the range and location of the coal seam, the overlying coal surface, the rock layer, the fault, the geological boundary geological elements, cleaning and preprocessing the data, removing abnormal data, noise and invalid data; S12. Perform spatial analysis, statistical analysis and correlation analysis on the processed data to identify the spatial distribution patterns and characteristics of coal seam geological data; S13. Establish a preliminary geological model M0 framework, set three-dimensional coordinates in the framework, visualize the coal seam geological model M0, and layout and design the mining face based on the visualization platform.

[0028] The collection of point cloud data M1 specifically includes: S211, using a three-dimensional laser scanner installed on the coal mining machine and the mobile support, when the coal mining machine and the conveyor are working, the three-dimensional laser scanner also continues to follow up, emits a laser beam in real time and records the time difference and intensity of the returned laser beam; S212, according to the posture change characteristics of the hydraulic support during the mining process, after the coal mining machine cuts coal and feeds, the hydraulic support at the corresponding position will be displaced and change its posture according to the issued command, while the hydraulic support at the non-mining position of the coal mining machine is generally in a static state, and the support posture basically does not change, so the scanner on the coal mining machine focuses on scanning the hydraulic support that is moving at this time to obtain the point cloud data M1 of the complete working surface; S213, using the transmission equipment to transmit the collected point cloud data M1 and the location information obtained by GPS positioning to the main control room, and then transmit it to the ground through the 5G communication module; S214. After receiving the data on the ground, professional software is used to extract coordinate and intensity information. Then, the point cloud data M1 collected by multiple scanners is subjected to data segmentation, elimination, alignment, and splicing pre-processing steps, and the processed data is stored.

[0029] Embodiment 3: In addition to all the technical features of Embodiment 1, this embodiment also includes: The acquisition of image data M2 specifically includes: S221. Cameras are arranged according to the 3D scanner. A camera is installed every five hydraulic supports to achieve full coverage and real-time monitoring of the mining working face. S222. The camera installed on the coal mining machine collects the real-time images of the hydraulic support posture movement and the coal mining machine when it is working, and observes the working status during coal mining. The cameras installed on the conveyor and the hydraulic support collect the real-time images of the working face when it is not mined, mainly monitoring the support status of the hydraulic support, whether there is roof crushing, floor weakness and coal seam spalling; S223, using the transmission equipment to transmit the collected image data M2 and the location information obtained by GPS positioning to the main control room, and then transmit it to the ground through the 5G communication module; S224. After the data is received on the ground, it is preprocessed, including denoising, image correction, color correction, and coordinate conversion. Due to poor lighting under the coal seam, it is impossible to observe directly with the naked eye. Therefore, when processing the data, it is necessary to use image data M2 training to train the model under the coal seam and adjust the appropriate neural weights. If the roof is broken, the target can be extracted directly.

[0030] Furthermore, in S3, the specific steps of fusing the point cloud data M1 with the image data M2 using the 3D-CVF (3D cross view fusion) method include: S31, extracting the point cloud data M1 and image data M2 of the working face, reprocessing the data, extracting features from the point cloud data M1 and the image data M2, matching them, using the coordinate information, selecting a common reference point, determining the coordinate zero point, aligning it with the coordinate in the geological model M0, and establishing a corresponding relationship between the point cloud data M1 and the image data M2; S32, extracting features from the point cloud data M1, selecting the point cloud space to be divided on the established spatial coordinate system, sampling the point cloud features generated above in the form of two-dimensional convolution, and forming point cloud bird's-eye view feature maps of different resolutions; S33, upsampling is performed in the form of two-dimensional deconvolution, and the upsampling results are spliced ​​to obtain the final point cloud bird's-eye view features; S34, performing the same operation on the image data M2 in the same manner as the point cloud data M1, to obtain the final image bird's-eye view features; S35. A global attention mechanism is used to extract the feature information of the bird's-eye view of the two, and each information channel is weighted channel by channel. Finally, pooling based on the three-dimensional region of interest is applied to fuse the point cloud bird's-eye view features with the image bird's-eye view features to obtain a fused comprehensive data set.

[0031] Furthermore, the fusion of the geological model M0 and the comprehensive dataset specifically includes: aligning the coordinates of the geological model M0 and the comprehensive dataset, projecting the comprehensive dataset to the corresponding area of ​​the geological model M0 in space, and also using the convolutional neural network CNN to perform grid matching, data interpolation, and data fitting steps to establish the relationship between the comprehensive dataset and the geological model M0, embedding the comprehensive dataset into the geological model M0, reconstructing the working face collection galaxy and texture information, and forming a new working face model.

[0032] Furthermore, the splicing and establishment of the virtual-real model M3 specifically includes: S51. Establishing a virtual model S511, collecting working face parameters, including hydraulic support, coal mining machine, conveyor parameters, including the position and orientation of the coal mining machine, and the layout of the hydraulic support, after configuring the working face, determining its coordinates in the coal seam, and aligning the coordinates to obtain the layout and position of the three equipment in the coal seam; S512, recording and transmitting the three-machine control instructions issued by the terminal to the virtual platform, adding interactive effects, that is, the working state of the coal mining machine, the position and movement of the hydraulic support, and the transportation speed of the conveyor obtained by the operation instructions, simulating the actual mining process of the working face, and completing the establishment of the virtual model; S52. Build a real model S521. Collect the data of coal mining machine movement, hydraulic support movement, support extension and extension, and support inclination monitored by sensors, perform preliminary amplification and filtering, transmit the data to the main control room through the 5G communication module and then reach the ground, clean, interpolate, and align the coordinate system to ensure data consistency and accurate coordinates; S522. Based on the parameters of the three machines on the working face and the posture data obtained by sensor monitoring, the three machine elements of the coal mining machine and the hydraulic support are modeled using 3D modeling software; ensuring that their positions, orientations and sizes are consistent with the actual situation, that is, obtaining a real modeling of the working face; In the actual mining process, due to the complex nature of the extremely thin coal seam and the vibration of the three machines, there will be inconsistencies between the operation control instructions and the actual working status of the three machines, that is, there is an error between the terminal instructions and the working status. Based on this, the working status of the three machines monitored by sensors can be used for real modeling; S53, the virtual model is spliced ​​with the real model. The splicing is completed on the virtual platform according to the data of the two models, the error between the virtual model and the real model is detected, and the position correction processing is performed according to the sensor monitoring value to obtain the spliced ​​and corrected virtual and real model M3.

[0033] Furthermore, the integration of the virtual model and the geological model M0 specifically includes: Since the virtual model is a digital representation of the real object or scene, there is not much processing and analysis, and most of the feature areas are composed of three machines, so the fusion is relatively simple and does not require too much algorithm registration and feature extraction; S61, obtaining two model data, aligning the virtual model with the indicators in the geological model M0 based on the coordinates of the geological model M0, and then projecting them into the geological model M0 to match the feature points or feature areas; according to the matching results, using interpolation processing to fuse the two data; S62. Based on the consistency of the shape and characteristics of the actual working face, the working face is fine-tuned as needed, thus completing the embedding of the virtual model into the geological model M0.

[0034] Furthermore, in S7, the new working surface model and the virtual-real new model M 03 The multi-model data fusion specifically includes: S71, obtaining volume model data, and registering and aligning the two sets of data to make their positions and postures consistent in the three-dimensional space; S72. Compare the measured working surface model data and the virtual model data, match the corresponding feature points or feature areas, use the convolutional neural network CNN to match the feature points, fuse the two sets of models, and obtain the final comprehensive model M. 0123 ; S73, the comprehensive model M 0123 Verify and adjust the (fused model) to ensure that the fused model is consistent with the shape and characteristics of the actual working surface, and can be compared and verified with the actual scene, and fine-tuned and adjusted as needed; S74. Visualize the fused model.

[0035] Embodiment 4, a system for three-dimensional modeling of an ultra-thin coal seam working face based on multi-source data fusion, comprising a laser point cloud data acquisition subsystem, a camera image acquisition subsystem and an ultra-thin coal seam hydraulic support posture control data modeling subsystem; Among them, the laser point cloud data acquisition subsystem includes a three-dimensional laser scanner, a GPS positioning device, a 5G communication module and a terminal execution module; the three-dimensional laser scanner uses Chinese to scan the fully mechanized mining face and obtain the point cloud data M1 of the three-dimensional working face, and calculates the distance between the object and the emitted laser beam by the time it takes to reflect back, thereby obtaining the spatial coordinate information of each material on the working face; it is arranged on the upper part of the fully mechanized mining machine (and installed on the following equipment, such as: motorized bracket, transmission machinery, etc.). This placement method can realize all-round scanning of the front and top of the working face, which is suitable for the compact space and complex geological environment of the fully mechanized mining face, such as: extremely thin coal seam; the GPS positioning device is used to determine the real-time changing position and posture information of the three-dimensional laser scanner to realize the accurate positioning of the fully mechanized mining face. The GPS positioning device is arranged at the inlet and outlet of the working face, which can better meet the positioning information of the fully mechanized mining face and accurately obtain the position information of the scanner relative to the working face; The 5G communication module is used to transmit and control the data collected by the laser instrument and the GPS positioning coordinates, so as to realize information interaction and collaborative work between the laser scanner and other equipment, such as ensuring the timely sending and execution of laser scanner control commands and ground remote control parameter adjustment; and the 5G communication module is deployed in the main control room. As a communication center, it can centrally manage and control communication equipment to facilitate system maintenance and management; the terminal execution module stores and manages three-dimensional point cloud data M1 and model data, processes the scanning data, and saves the point cloud data M1 and modeling results obtained by the scanner into a unified database for subsequent data viewing, query and analysis; then the point cloud data M1 is filtered, registered, and spliced ​​to extract the three-dimensional geometric information of coal seams and mineral rocks.

[0036] The camera image acquisition subsystem includes a camera, a GPS positioning device, a 5G communication module, and a terminal execution module; Among them, the camera is used to capture images or videos of the comprehensive mining working face. The camera can be fixed or movable. Different types of cameras can be selected according to needs. The fixed cameras installed on the hydraulic support and the mobile cameras installed on the coal mining machine or conveyor are of different types.

[0037] The camera image acquisition system's GPS positioning device, 5G communication module and terminal execution module use the same equipment as the laser point cloud data acquisition system, which can better adapt to the compact and narrow space of extremely thin coal seams.

[0038] The data modeling system for the posture control of ultra-thin coal seam hydraulic supports includes coal mining machines and conveyor belts, ultra-thin coal seam hydraulic supports, sensors, data acquisition equipment, 5G communication modules, and terminal execution modules; Among them, the coal mining machine, conveyor belt and ultra-thin coal seam hydraulic support are used to model the coal mining working face, and the models and parameters of these main equipment need to be known; Sensors are used to measure the displacement of hydraulic supports in space, changes in relative position, and changes in the posture of the supports. These sensors include displacement sensors, inclination sensors, pressure sensors, inertial measurement units, etc. The corresponding sensors are selected according to the specific quantities to be measured; Data acquisition devices are used to receive and preliminarily process the signals collected by sensors. These units are responsible for converting analog signals into digital signals and may perform preliminary filtering and amplification of the data; The terminal operation device inputs the position movement instruction through the terminal operation device, and the user interface device is usually connected to the control system and can communicate directly with the control system; The 5G communication module of the data modeling subsystem for the attitude control of the ultra-thin coal seam hydraulic support and the laser point cloud data acquisition subsystem use the same equipment, and the relevant processing software is selected according to the specific data requirements.

[0039] Based on the geological exploration data obtained from actual measurements of extremely thin coal seams before mining, the geological characteristic data of the coal seams are processed and stored. The explored geological data are first modeled to obtain the characteristics of the coal seams. Then, the starting positions of the hydraulic supports and the coal mining machine in the coal seams are planned according to their characteristics, and a three-dimensional spatial coordinate system is established. This coordinate system is used as a reference to integrate the subsequent working face modeling data. The laser point cloud data acquisition system composed of three-dimensional scanners and other equipment on the coal mining machine is used to perform GPS positioning based on the obtained point cloud data M1, and the complete point cloud data M1 is transmitted to the ground, and then the data is processed and stored. A camera image acquisition system consisting of cameras fixed on hydraulic supports and moving with the coal mining machine is used. GPS positioning is used to transmit the complete image data M2 and coordinates to the ground. The data is then processed and stored, and the working face point cloud data M1 and image data M2 are extracted and reprocessed to unify the coordinate system and extract the features of the two. Then, based on the geological model M0, the 3D-CVF (3D cross view fusion) method is used to fuse them. The geological model M0 is fused with the comprehensive data set, and three-dimensional modeling is performed based on the application scenario. Virtual modeling is performed using equipment parameters and attitude control instructions, and real modeling is performed using three-dimensional attitude data extracted by sensors and other equipment. The two models are spliced ​​on a virtual platform, their errors are detected, and correction processing is performed to obtain the virtual-real model M3. The geological model M0 and the virtual-real model M3 data are obtained, and the virtual model is embedded in the geological model M0. The modeling models obtained by the three subsystems are subjected to multi-model data fusion to obtain a more accurate new model based on the advantages of each specific model.

[0040] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0041] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for three-dimensional modeling of ultra-thin coal seam working face based on multi-source data fusion, characterized in that: The method comprises the following steps: S1. Before mining, geological exploration data is obtained by measuring the extremely thin coal seam, and the geological exploration data is processed and stored. Based on the geological exploration data, geological modeling is performed to obtain a geological model M0, thereby obtaining coal seam characteristics. Then, according to the characteristics of the coal seam characteristics, the starting positions of the hydraulic support and the coal mining machine in the coal seam are planned and a three-dimensional space coordinate system is established. The subsequent working face modeling data is integrated based on this three-dimensional space coordinate system; S2, based on the laser point cloud data acquisition system set on the coal mining machine, point cloud data M1 is obtained, and GPS positioning is performed, the point cloud data M1 is transmitted to the ground, and then the point cloud data M1 is processed and stored, and image data M2 is obtained by using a camera image acquisition system fixed on the hydraulic support and moving with the coal mining machine, and GPS positioning is performed, and the image data M2 and coordinate data are transmitted to the ground, and then the image data M2 and coordinate data are processed and stored; S3, extracting the point cloud data M1 and image data M2 of the working surface, unifying the coordinate system and extracting the features of the two, and fusing them using the 3D-CVF method to obtain a comprehensive data set; S4, the geological model M0 is integrated with the comprehensive data set, and three-dimensional modeling is performed based on the application scenario to obtain a new working face model M 012 ; S5, using equipment parameters and attitude control instructions to perform virtual modeling to obtain a virtual model, using the attitude data of the three machines extracted by sensors to perform real modeling to obtain a real model, and splicing the two models through a virtual platform, detecting their errors, and performing correction processing to obtain a virtual-real model M3; S6, obtain the geological model M0 data and the virtual-real model M3 data, embed the virtual-real model M3 into the geological model M0, and obtain the virtual-real new model M 03 ; S7, the new working surface model and the virtual-real new model M 03 By fusing multiple model data, we can complete the 3D modeling of the ultra-thin working surface and obtain the comprehensive model M. 0123 .

2. The method according to claim 1, characterized in that: In S1, the geological exploration data is used for geological modeling, including: S11. Extract the geological exploration data before mining in the storage system, including the scope and location of the coal seam, the coal cover, the rock layer, the fault, and the geological elements of the geological boundary, and clean and pre-process them to remove abnormal data, noise and invalid data; S12. Perform spatial analysis, statistical analysis and correlation analysis on the processed data to identify the spatial distribution patterns and characteristics of coal seam geological data; S13. Establish a preliminary geological model M0 framework, set three-dimensional coordinates in the framework, visualize the coal seam geological model M0, and layout and design the mining face based on the visualization platform.

3. The method according to claim 1, characterized in that: In S2, the collection of the point cloud data M1 specifically includes: S211, using a three-dimensional laser scanner installed on the coal mining machine and the mobile support, when the coal mining machine and the conveyor are working, the three-dimensional laser scanner continuously follows up, emits a laser beam in real time and records the time difference and intensity of the returned laser beam; S212, according to the posture change characteristics of the hydraulic support during the mining process, after the coal mining machine cuts coal and feeds, the hydraulic support at the corresponding position will be displaced and change its posture according to the issued command, while the hydraulic support at the non-mining position of the coal mining machine is in a static state, and the support posture does not change, so the scanner on the coal mining machine focuses on scanning the hydraulic support that is moving at this time to obtain the point cloud data M1 of the complete working surface; S213, using the transmission equipment to transmit the collected point cloud data M1 and the location information obtained by GPS positioning to the main control room, and then transmit it to the ground through the 5G communication module; S214, after receiving the data, the ground extracts the coordinate and intensity information, and then performs data segmentation, elimination, alignment, and splicing preprocessing steps on the point cloud data M1 collected by multiple deployed scanners, and stores the processed data.

4. The method according to claim 1, characterized in that: In S2, the acquisition of the image data M2 specifically includes: S221. In addition to the three-dimensional scanner, a camera is installed every five hydraulic supports to achieve full coverage of the mining face and real-time monitoring; S222. The camera installed on the coal mining machine collects the real-time images of the hydraulic support posture movement and the coal mining machine when it is working, and observes the working status during coal mining. The cameras installed on the conveyor and the hydraulic support collect the real-time images of the working face when it is not mined, and monitor the support status of the hydraulic support, whether there is roof broken, floor weak, and coal seam spalling phenomenon; S223, using the transmission equipment to transmit the collected image data M2 and the location information obtained by GPS positioning to the main control room, and then transmit it to the ground through the 5G communication module; S224. After the data is received on the ground, it is preprocessed, including denoising, image correction, color correction, and coordinate conversion. Due to poor lighting under the coal seam, the naked eye cannot directly observe it. Therefore, when processing the data, it is necessary to use image data M2 training to train the model under the coal seam and adjust the appropriate neural weights. If the roof is broken, the target is directly extracted.

5. The method according to claim 4, characterized in that: In S3, the specific steps of fusing the point cloud data M1 with the image data M2 using the 3D-CVF method include: S31, extracting the point cloud data M1 and image data M2 of the working face, reprocessing the data, extracting features from the point cloud data M1 and the image data M2, matching them, using the coordinate information, selecting a common reference point, determining the coordinate zero point, aligning it with the coordinate in the geological model M0, and establishing a corresponding relationship between the point cloud data M1 and the image data M2; S32, extracting features from the point cloud data M1, selecting the point cloud space to be divided on the established spatial coordinate system, sampling the point cloud features generated above in the form of two-dimensional convolution, and forming point cloud bird's-eye view feature maps of different resolutions; S33, upsampling is performed in the form of two-dimensional deconvolution, and the upsampling results are spliced ​​to obtain the final point cloud bird's-eye view features; S34, performing the same operation on the image data M2 in the same manner as the point cloud data M1, to obtain the final image bird's-eye view features; S35, extract the feature information of the bird's-eye view of the two by using the global attention mechanism, and perform channel-by-channel weighting operation on each information channel. Finally, apply the pooling based on the three-dimensional region of interest to fuse the point cloud bird's-eye view features with the image bird's-eye view features to obtain the fused comprehensive dataset M 12 .

6. The method according to claim 5, characterized in that: In S4, the fusion of the geological model M0 and the comprehensive data set specifically includes: The coordinates of the geological model M0 and the comprehensive dataset are aligned, and the comprehensive dataset is projected to the corresponding area of ​​the geological model M0 in space. The convolutional neural network CNN is also used for grid matching, data interpolation, and data fitting steps to establish the relationship between the comprehensive dataset and the geological model M0. The comprehensive dataset is embedded in the geological model M0, and the working face galaxy and texture information are reconstructed to form a new working face model M0. 012 .

7. The method according to claim 6, characterized in that: In S5, the splicing and establishment of the virtual-real model M3 specifically includes: S51. Establishing a virtual model S511, collecting working face parameters, including hydraulic support, coal mining machine, conveyor parameters, including the position and orientation of the coal mining machine, and the layout of the hydraulic support, after configuring the working face, determining its coordinates in the coal seam, and aligning the coordinates to obtain the layout and position of the three equipment in the coal seam; S512, recording and transmitting the three-machine control instructions issued by the terminal to the virtual platform, adding interactive effects, that is, the working state of the coal mining machine, the position and movement of the hydraulic support, and the transportation speed of the conveyor obtained by the operation instructions, simulating the actual mining process of the working face, and completing the establishment of the virtual model; S52. Build a real model S521. Collect the data of coal mining machine movement, hydraulic support movement, support extension and extension, and support inclination monitored by sensors, perform preliminary amplification and filtering, transmit the data to the main control room through the 5G communication module and then reach the ground, clean, interpolate, and align the coordinate system to ensure data consistency and accurate coordinates; S522, based on the parameters of the three machines of the working face and the posture data obtained by sensor monitoring, use 3D modeling software to model the three machine elements of the coal mining machine, hydraulic support, and ensure that their positions, orientations, and sizes are consistent with the actual situation, that is, to obtain a real model of the working face; S53, the virtual model is spliced ​​with the real model. The splicing is completed on the virtual platform according to the data of the two models, the error between the virtual model and the real model is detected, and the position correction processing is performed according to the sensor monitoring value to obtain the spliced ​​and corrected virtual and real model M3.

8. The method according to claim 7, characterized in that: In S6, the fusion of the virtual-real model M3 and the geological model M0 specifically includes: S61, obtaining two model data, aligning the virtual model with the indicators in the geological model M0 based on the coordinates of the geological model M0, and then projecting them into the geological model M0, matching feature points or feature areas, and fusing the two data by interpolation processing according to the matching results; S62, according to the consistency of the shape and characteristics of the actual working face, the working face is fine-tuned as needed to complete the embedding of the virtual-real model M3 into the geological model M0 to obtain the virtual-real new model M 03 .

9. The method according to claim 8, characterized in that: In S7, the new working surface model and the new virtual-real model M are combined. 03 The multi-model data fusion specifically includes: S71, acquiring two model data, and registering and aligning the two sets of data to make their positions and postures consistent in the three-dimensional space; S72. Compare the measured working surface model data and the virtual model data, match the corresponding feature points or feature areas, use the convolutional neural network CNN to match the feature points, fuse the two sets of models, and obtain the final comprehensive model M. 0123 ; S73, the comprehensive model M 0123 Verify and adjust to ensure that the fused model is consistent with the shape and features of the actual working surface, compare and verify with the actual scene, and make fine-tuning and adjustments as needed; S74, the comprehensive model M 0123 Provide a visual display.

10. A system used in the method for three-dimensional modeling of ultra-thin coal seam working face based on multi-source data fusion according to any one of claims 1 to 9, characterized in that: The system includes a laser point cloud data acquisition subsystem, a camera image acquisition subsystem, and an ultra-thin coal seam hydraulic support posture control data modeling subsystem; Wherein, the laser point cloud data acquisition subsystem includes a three-dimensional laser scanner, a first GPS positioning device, a first 5G communication module and a terminal execution module; The camera image acquisition subsystem includes a camera, a second GPS positioning device, a second 5G communication module and a second terminal execution module; The ultra-thin coal seam hydraulic support posture control data modeling subsystem includes a coal mining machine and a conveyor belt, an ultra-thin coal seam hydraulic support, a sensor, a data acquisition device, a third 5G communication module and a third terminal execution module.

Citation Information

Cited By

  • Vision and point cloud data fusion modeling coal face mining auxiliary method

    CN120997631A