Dynamic monitoring image fusion method and system in coal face

Abstract

The invention relates to a dynamic monitoring image fusion method and system in a coal face, and relates to the field of image processing, and the method comprises the steps: constructing a three-dimensional region grid model of the coal face; identifying the spatial distribution density of each grid unit in the three-dimensional area grid model according to a historical acquired image, and clustering each grid unit to output a plurality of sub-areas; linking the plurality of sub-regions to a multi-source dynamic monitoring module, carrying out image source screening according to a quality scoring mechanism, outputting a screened image source corresponding to each sub-region, and obtaining a corresponding region adaptive fusion strategy, and performing image fusion on the real-time monitoring image of the current sub-region according to the corresponding region self-adaptive fusion strategy to obtain a fusion image sequence of each sub-region, thereby solving the problems of poor adaptability of a coal face image fusion environment, insufficient uniformity of image fusion features, limited accuracy and practicability, and high efficiency. And the safety monitoring level is low.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and particularly to a dynamic monitoring image fusion method and system in a coal mining face. Background Art

[0002] During the coal mining process, the safety monitoring of the coal mining face is a crucial link. Due to the uneven distribution of equipment and elements such as supports, shearers, belts, and personnel in the coal mining face, there are significant differences in the target density and occlusion complexity in different regions, which poses great challenges to existing image fusion methods.

[0003] Existing image fusion methods usually adopt a unified fusion strategy and algorithm to fuse the images of the entire coal mining face. However, when facing the differences in target density and occlusion complexity in different regions, this method often fails to take into account the visual characteristics of high-density regions and low-density regions. In high-density regions, due to the large number of targets and occluders, existing fusion methods are prone to loss of image details or fusion distortion, making it impossible to accurately extract key information. In low-density regions, due to the small number of targets and occluders, existing fusion methods may overemphasize contour information and ignore detail features, also affecting the monitoring effect. Summary of the Invention

[0004] Aiming at the technical problems in the prior art that the adaptability of the coal mining face image fusion environment is poor, the uniformity of the image fusion features is insufficient, resulting in limited accuracy and practicability, and a low level of safety monitoring, the present invention provides a dynamic monitoring image fusion method and system in a coal mining face to solve these problems.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: In a first aspect, the present invention provides a dynamic monitoring image fusion method in a coal mining face, the method comprising: constructing a three-dimensional regional grid model of the coal mining face; identifying the spatial distribution density of each grid unit in the three-dimensional regional grid model according to historical acquisition images, clustering each grid unit according to the spatial distribution density to output a plurality of sub-regions, wherein each sub-region corresponds to a spatial distribution density level; linking the plurality of sub-regions to a multi-source dynamic monitoring module to perform image source screening according to a quality scoring mechanism, and outputting a screened image source corresponding to each sub-region; for the screened image source corresponding to each sub-region, obtaining a corresponding regional adaptive fusion strategy, and performing image fusion on the real-time monitoring image of the current sub-region with the corresponding regional adaptive fusion strategy to obtain a fusion image sequence of each sub-region.

[0006] Second aspect, the present invention provides a dynamic monitoring image fusion system in a coal mining face. The system includes: a model construction unit for constructing a three-dimensional regional grid model of the coal mining face; a grid clustering unit for identifying the spatial distribution density of each grid unit in the three-dimensional regional grid model according to historical acquisition images, clustering each grid unit according to the spatial distribution density to output a plurality of sub-regions, where each sub-region corresponds to a spatial distribution density level; an image screening unit for linking the plurality of sub-regions to a multi-source dynamic monitoring module to perform image source screening according to a quality scoring mechanism, and outputting a screened image source corresponding to each sub-region; an image fusion unit for obtaining a corresponding regional adaptive fusion strategy for the screened image source corresponding to each sub-region, and performing image fusion on the real-time monitoring image of the current sub-region with the corresponding regional adaptive fusion strategy to obtain a fusion image sequence for each sub-region.

[0007] The beneficial effects of the present invention are as follows: By constructing a three-dimensional regional grid model of the coal mining face, identifying the spatial distribution density of each grid unit and clustering them into multiple sub-regions, then screening the optimal image source for each sub-region, and adopting a regional adaptive fusion strategy for image fusion, a fusion image sequence for each sub-region is obtained, realizing the adaptive processing of the target density and occlusion complexity in different regions, improving the accuracy and practicability of image fusion, and further improving the safety monitoring level of the coal mining face in the coal mine. Description of the Drawings

[0008] Figure 1 It is a schematic flow chart of a dynamic monitoring image fusion method in a coal mining face provided by the present invention.

[0009] Figure 2 It is a schematic structural diagram of a dynamic monitoring image fusion system in a coal mining face provided by the present invention.

[0010] Description of the reference numerals: Model construction unit 11, grid clustering unit 12, image screening unit 13, image fusion unit 14. Detailed Embodiments

[0011] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0012] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality of" means two or more, unless otherwise specifically defined.

[0013] In the description of the present invention, the term "for example" is used to mean "serving as an example, illustration, or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as more preferred or more advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the present invention. In the following description, details are set forth for purposes of explanation. It should be understood that those of ordinary skill in the art can recognize that the present invention can be implemented without the use of these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0014] Embodiment 1:

[0015] As Figure 1 shown, an embodiment of the present invention provides a dynamic monitoring image fusion method in a coal mining face, and the method includes: S10: Construct a three-dimensional regional grid model of the coal mining face.

[0016] Exemplarily, constructing a three-dimensional regional grid model of a coal mining face requires integrating geological exploration data, mining process parameters, and numerical simulation techniques. Specifically, first, the occurrence characteristics of coal seams, such as strike, dip, and thickness variation, are analyzed through three-dimensional seismic exploration and borehole data, and then combined with the geometric parameters of geological structures (faults, folds) to establish a basic geological model. Next, according to the coal mining method (such as longwall mining, top coal caving technology), the advancing direction of the working face, the support layout, and the mining step distance are determined, and the equipment parameters (hydraulic support size, shearer cutting depth) are converted into spatial constraint conditions. Subsequently, a grid meshing algorithm (such as Delaunay triangulation or hexahedron grid generation) is used to discretize the geological body. Special attention should be paid to the grid encryption of complex areas such as coal seam bifurcation areas and roof fracture zones during this process. For example, a gradually changing size grid is used in the fault zone to balance calculation accuracy and efficiency. Finally, a three-dimensional calculation grid that can simulate stress distribution and deformation failure is formed by coupling rock mass mechanical parameters (elastic modulus, Poisson's ratio) with the dynamic boundary conditions of mining. For example, in the modeling of a thick coal seam top coal caving working face in a certain mine, by refining the grid density in the area where the top coal collapses, the dynamic relationship between the support working resistance and the top coal recovery rate is successfully predicted, providing a quantitative basis for optimizing the mining plan.

[0017] S20: Identify the spatial distribution density of each grid cell in the three-dimensional regional grid model according to the historical acquisition images, and cluster each grid cell according to the spatial distribution density to output multiple sub-regions, where each sub-region corresponds to a spatial distribution density level.

[0018] Preferably, the spatial density analysis and clustering of the three-dimensional regional grid model are further driven by historical collected image data to achieve refined zoning management of the mining area. The spatial distribution density characteristics of grid cells can be extracted based on historical monitoring images of the coal mining face (such as visible light, infrared thermal imaging). For example, the degree of top coal caving, the frequency of coal wall spalling, or the distribution density of support pressure can be calculated through the image pixel intensity gradient, and each grid cell is quantified into a continuous density index, such as a normalized density value within the range of 0-1. Then, a density clustering algorithm (such as DBSCAN or density-based spatial clustering method) is used to perform unsupervised classification on the grid cells. The algorithm automatically identifies the density peak areas and sparse areas, and different levels of sub-regions are divided according to the density threshold. For example, in the grid cells of the roof fracture zone, the density value continuously exceeds 0.7 due to frequent caving and is clustered into a high-density sub-region, while the density value of the stable coal wall area is lower than 0.3 and is classified as a low-density sub-region. Finally, multiple sub-regions with clear spatial distribution density levels are output, and each sub-region corresponds to different mining response characteristics. For example, in the application of an intelligent working face, the high-density stress concentration area (density level > 0.8) identified by clustering is preferentially included in the key monitoring scope of rock burst, while the low-density stable area (density level < 0.4) adopts the conventional support strategy. This regional management based on spatial density grading significantly improves the accuracy of mining dynamic regulation.

[0019] S30: Link the multiple sub-regions to a multi-source dynamic monitoring module for image source screening according to a quality scoring mechanism, and output the screened image sources corresponding to each sub-region.

[0020] Furthermore, through the deep coupling of the multi-source dynamic monitoring module and the spatial characteristics of sub-regions, the quality scoring and precise screening of image sources are realized, and low-value image sources are eliminated. Specifically, multiple sub-regions generated by clustering (such as high-density regions of roof fracture zones, low-density regions of stable coal walls, etc.) are used as spatial constraint units to establish topological associations with the image data streams of the multi-source monitoring system (such as visible light cameras, infrared thermal imagers, lidar point clouds). Each sub-region dynamically binds the multi-source image sources within its spatial range. Subsequently, a quality scoring mechanism is constructed to quantitatively evaluate the effectiveness of image sources through characteristic indicators. For example, the clarity index (based on the gradient amplitude of the Laplacian operator) is used for visible light images, the temperature field entropy value analysis is used for infrared images, and the density distribution uniformity index is used for point cloud data. At the same time, spatio-temporal consistency constraints are introduced, such as the differential entropy of adjacent frame images within the same sub-region. Finally, a multi-dimensional scoring system is formed. Then, the scoring weights are dynamically adjusted according to the mining risk levels of sub-regions. For example, in high-risk sub-regions of rock bursts (density level > 0.8), the weight of the temperature anomaly sensitivity of infrared thermal imaging is increased to 0.6, while in stable support areas (density level < 0.4), more emphasis is placed on the clarity weight of visible light images (such as 0.7). Finally, the optimal image source combination corresponding to each sub-region is output through threshold screening or the Top-K strategy. For example, in a fully mechanized coal face, the image sources selected from the sub-region of the roof fracture zone include high-resolution infrared thermal maps (identifying fissure water seepage temperature anomalies) and dense point cloud data (quantifying the distribution of caving block sizes), while in the coal wall spalling area, high-frame-rate visible light videos are preferentially used (capturing the dynamic process of spalling). This image source screening mechanism based on quality scoring significantly improves the timeliness and reliability of mining state perception.

[0021] S40: For the screened image sources corresponding to each sub-region, obtain the corresponding region adaptive fusion strategy, and perform image fusion on the real-time monitoring images of the current sub-region with the corresponding region adaptive fusion strategy to obtain the fusion image sequence of each sub-region.

[0022] Specifically, the intelligent integration of multi-source image data is finally achieved through a region adaptive fusion strategy to generate a sub-region fusion image sequence with high fidelity. For the selected image sources of each sub-region (such as infrared thermal maps, visible light videos, lidar point clouds), a region adaptive fusion strategy is dynamically generated based on its spatial distribution density level and dynamic monitoring requirements. For example, in the sub-region with high stress concentration (density level > 0.8), an enhanced fusion mode of "infrared thermal map + point cloud structured light" is adopted, and potential impact risks are identified through the coupled analysis of the temperature field gradient and the three-dimensional deformation field. In the sub-region of the broken roof belt, the complementary fusion strategy of "visible light texture + radar depth" is switched, and the contour of the caving block is extracted using an edge detection algorithm, and the three-dimensional shape of the caving body is reconstructed by combining depth information. Subsequently, a feature-level or decision-level fusion algorithm is designed according to the fusion strategy. For example, in the coal wall spalling area, the wavelet transform multi-scale fusion technology is used to perform weighted superposition of the texture details of the visible light image and the temperature anomaly area of the infrared image in the frequency domain, and at the same time, noise interference is suppressed by Kalman filtering. For the equipment-intensive area (such as a hydraulic support group), a semantic segmentation network based on deep learning is introduced to perform pixel-level alignment of the spatial position information of the point cloud data and the visual features of the visible light image, and the joint monitoring of the support attitude and roof separation is realized. Finally, the fusion image sequence of each sub-region is output. For example, in the rock burst early warning scenario, the fusion image sequence simultaneously presents the temperature evolution trend of the coal and rock mass (infrared thermal map), the spatial distribution of microseismic events (point cloud seismic source location), and the acoustic emission signal intensity (visible light video vibration analysis), providing multi-dimensional information support for mining safety decision-making. This region adaptive image fusion mechanism significantly improves the accuracy and robustness of state perception in complex mining environments.

[0023] In a preferred embodiment, the spatial distribution density of each grid cell in the three-dimensional regional grid model is identified according to the historically collected images, including: obtaining the projection of the historically collected images onto the three-dimensional regional grid model to obtain a set of covered grid cells; detecting the density feature vectors of each grid cell in the set of covered grid cells, where the density feature vectors include the number of shooting times, the number of targets appearing, the image texture complexity, and the occlusion probability; and calculating the weights of the density feature vectors to output the spatial distribution density of each grid cell.

[0024] Specifically, through the depth mapping of historical acquisition images and the three-dimensional regional grid model, the accurate quantification of the spatial distribution density of grid cells is achieved. First, historical acquisition images, such as surveillance camera video frames, are projected into the three-dimensional regional grid model through a spatial registration algorithm to identify the set of grid cells within the image coverage. For example, in the longwall face scenario, a single-frame surveillance image can cover dozens of grid cells, thus forming the target area to be analyzed. Then, for each grid cell in the set, multi-dimensional density feature vectors are extracted, including: 1) the number of times photographed, which quantifies the monitoring attention by counting the occurrence frequency of the grid cell in historical images; 2) the number of targets appearing, which uses a target detection algorithm (such as YOLOv5) to count equipment instances such as coal shearers and supports; 3) the image texture complexity, which calculates the entropy value based on the gray-level co-occurrence matrix and reflects the degree of coal wall fragmentation or the development state of roof fissures; 4) the occlusion probability, which predicts the possibility of the grid cell being occluded by coal dust and equipment through a depth estimation network. For example, in the roof water-inrush area, the texture complexity and occlusion probability eigenvalue increase significantly. Subsequently, a dynamic weight allocation mechanism is designed to adjust the feature weights according to the mining stage. For example, in the initial mining period, the emphasis is on the number of times photographed (weight 0.4) and the number of equipment targets (weight 0.3), while in the periodic weighting stage, the texture complexity (weight 0.5) and occlusion probability (weight 0.2) weights are increased; finally, the spatial distribution density value of each grid cell is output through weighted summation. For example, a certain grid cell has a density value calibrated to 0.78 after weight calculation due to being frequently photographed (number of times photographed = 15), continuously having abnormal target of support pressure (number of targets = 8), a texture complexity of 0.82 and an occlusion probability of only 0.1. This density quantification method of multi-feature fusion provides a reliable spatial basis for subsequent regional clustering and dynamic monitoring.

[0025] In a preferred embodiment, each grid cell is clustered according to the spatial distribution density to output multiple sub-regions, including: initially clustering all grids according to the magnitude of the spatial distribution density to output an initial clustering result, the initial clustering result including multiple initial sub-regions; setting a fusion spatial adjacency condition, and performing adjacency analysis on the three-dimensional coordinates of each initial sub-region according to the fusion spatial adjacency condition to obtain three-dimensional coordinates that do not meet the adjacency; re-dividing the three-dimensional coordinates that do not meet the adjacency to output multiple sub-regions.

[0026] Preferably, through the synergistic effect of spatial distribution density and adjacency constraint, high-precision clustering of grid cells and sub-region division are achieved. First, based on the spatial distribution density values of the grid cells, density peak clustering (DPC) or K-Means++ algorithm is used for initial clustering. For example, grid cells with density values higher than 0.7 are preliminarily classified as stress concentration areas, and those with density values lower than 0.3 are classified as stable areas, forming multiple initial sub-regions. Then, the spatial adjacency condition is introduced as a post-clustering processing rule. This condition ensures that physically adjacent grid cells are divided into the same sub-region by setting a three-dimensional space distance threshold (such as 5 meters) and a density attenuation gradient (such as a density decrease of 0.1 per meter). For example, in a longwall face, if the three-dimensional coordinate distance between two high-density grid cells exceeds 10 meters, they will be marked as not meeting the adjacency even if their density values are similar. Subsequently, for the three-dimensional coordinates that do not meet the adjacency, methods such as connected component analysis in graph theory or density-based region growing algorithm are used for re-division. For example, isolated high-density points (caused by equipment occlusion resulting in abnormal density) are merged into adjacent sub-regions with slightly lower density, or "density islands" in low-density areas (caused by sensor noise) are re-classified as background areas. Finally, multiple sub-regions that meet spatial continuity are output. For example, in rockburst monitoring, after correction by adjacency constraint, a long-distance abnormal point that was originally misjudged as an independent high-density area is correctly classified into the roof fracture belt sub-region. This clustering method that combines density features and spatial topology significantly improves the physical rationality of region division and the pertinence of mining response.

[0027] In a preferred embodiment, the multiple sub-regions are linked to a multi-source dynamic monitoring module for image source screening according to a quality scoring mechanism, and the screened image sources corresponding to each sub-region are output, including: obtaining multiple monitoring sources in the multi-source dynamic monitoring module; evaluating the image quality scores corresponding to each sub-region for the multiple monitoring sources respectively according to the quality scoring mechanism, and screening and outputting image sources with scores greater than a preset quality score; wherein, the image quality is obtained through a weighted scoring formula of image sharpness, viewing angle coverage, image frame rate, and feature integrity; the multiple sub-regions are linked to the multi-source dynamic monitoring module, a transmission protocol between each sub-region and the corresponding screened image source is established, and the real-time monitoring images corresponding to the screened image sources are transmitted according to the transmission protocol.

[0028] Furthermore, through the spatial mapping between the multi-source dynamic monitoring module and sub-regions, combined with the quality scoring mechanism, intelligent screening and efficient transmission of image sources are achieved. Specifically, spatial associations are established between the multiple sub-regions generated by clustering and the image sources of the multi-source monitoring system, and each sub-region is dynamically bound to multiple monitoring sources within its coverage. Then, an image quality scoring mechanism is constructed to comprehensively evaluate the monitoring value of image sources through a weighted formula. Among them, the image sharpness is calculated using the Laplacian gradient magnitude; the viewing angle coverage is analyzed based on the spatial intersection of the sub-region three-dimensional model and the camera frustum; the image frame rate dynamically adjusts the weight according to the mining dynamic characteristics (for example, the frame rate weight is increased to 0.3 in rock burst monitoring); and the feature integrity quantifies the recognition rate of key features (such as the attitude of hydraulic supports and coal-rock fractures) through an object detection algorithm (such as SSD). For example, in the sub-region of the roof broken belt, the weight of the temperature field feature integrity of the infrared thermal image is set to 0.4, and the weight of the texture details of the visible light video is 0.3. Subsequently, a quality scoring threshold (such as 0.65) is set, and image sources greater than this threshold are screened as valid inputs. For example, in the coal wall spalling area, visible light image sources with sharpness lower than 0.5 due to coal dust occlusion are excluded, and at the same time, high-frame-rate (50fps) high-speed camera data is retained to capture the dynamic process of spalling. Finally, a customized transmission protocol between the sub-region and the screened image sources is established, and the MQTT protocol is used to achieve low-latency image push. For example, when an early warning is triggered in the stress abnormal sub-region, the transmission priorities of the corresponding infrared thermal images and acoustic emission sensors are automatically increased. This image source screening and transmission mechanism based on quality scoring significantly reduces data redundancy, ensures that each sub-region only receives high-value monitoring data, and provides accurate information support for mining decisions.

[0029] In a preferred embodiment, for the screened image sources corresponding to each sub-region, the corresponding region adaptive fusion strategy is obtained; wherein, the built-in modules of the region adaptive fusion strategy include confidence weighted fusion, main viewing angle priority fusion, and multi-channel partition fusion; taking the type and quantity of the image source samples as inputs and the built-in module corresponding to the optimal solution representing the image fusion effect as the output, a strategy template mapping library is established, and the region adaptive fusion strategy corresponding to the screened image sources is obtained according to the strategy template mapping library.

[0030] In a specific embodiment, an adaptive fusion strategy matching for sub-region screening image sources is achieved by constructing a policy template mapping library. Specifically, first, for the screening image sources of each sub-region (such as visible light, infrared, lidar, etc.), according to their spatial distribution density levels (high density, medium density, low density) and image source characteristics (type, quantity), a region adaptive fusion strategy is dynamically generated. In high-density sub-regions (such as stress concentration areas), due to complex monitoring requirements and high data redundancy, the policy template mapping library recommends using a combination mode of "confidence-weighted fusion + multi-channel partition fusion". For example, pixel-level confidence weighting is performed on the infrared thermal map and visible light video (based on the probability of temperature anomaly and texture clarity), and at the same time, multi-channel partition fusion technology is used to complementarily fuse image sources from different perspectives (such as top view and side view) in the feature space to comprehensively capture the multi-physical field coupling information of coal and rock masses. In medium-density sub-regions (such as roof stable areas), the policy library preferentially selects the "main view first fusion" mode. For example, the front view of the support (clarity > 0.8) is selected from multiple visible light image sources as the benchmark, supplemented by auxiliary view images with low weights for feature enhancement, which not only ensures the monitoring efficiency but also reduces the computational overhead. In low-density sub-regions (such as equipment idle areas), a fusion strategy of "maximum coverage angle + strong synchrony" is adopted. For example, the image source with the largest coverage angle taken by a wide-angle lens is selected, and the multi-source image sequences are aligned through a timestamp synchronization mechanism to ensure low-cost and high-efficiency monitoring in areas with less equipment interference and slow dynamic changes. Among them, the construction of the policy template mapping library is based on historical fusion effect data, and the mapping relationship between different image source combinations and fusion strategies is learned through a deep learning model (such as ResNet-50). For example, in the scenario of rock burst early warning, the "multi-modal complementary fusion" strategy in high-density sub-regions has successfully improved the temperature field anomaly detection accuracy to 92%, while the "wide-angle synchronous fusion" strategy in low-density sub-regions has reduced the data transmission volume by 60%. This density-based adaptive fusion strategy significantly improves the utilization efficiency of multi-source image data in complex mining environments.

[0031] Specific policy fusion details can be understood based on the following density-based image fusion policy table:

[0032] Specifically, the table content formulates a differentiated image fusion scheme around the density levels of different areas of the coal mining face, aiming to improve the quality of image fusion and meet the monitoring requirements of different areas. In high-density areas, usually the target distribution is dense and the occlusion situation is complex, and the number of image sources needs to be 3 or more. The weighted average fusion and redundancy removal are combined. Among them, the weighted average fusion synthesizes the information of multiple image sources, and the redundancy removal can avoid the interference of duplicate information and highlight the key details. In the weighting strategy, the clarity weight and occlusion ratio penalty are used. For example, in the area near the shearer, where equipment and personnel are concentrated, the images are prone to blurring and occlusion. Assigning weights according to the image clarity can make the clear parts more prominent in the fused image; while giving a penalty to the occluded areas can reduce their contribution in the fusion, thus effectively avoiding information redundancy and improving the clarity of boundary information such as the edges of equipment and operation details, enabling the staff to accurately grasp the situation of this area. For medium-density areas, the number of image sources is generally 2-3, and the fusion mode is perspective selection and sequence fusion. Perspective selection can select the most representative and complementary perspective images, and sequence fusion integrates these images in chronological order to show the dynamic changes in the area. The weighting strategy emphasizes stability and field-of-view complementarity first. For example, in areas such as along the scraper conveyor, where the equipment and personnel are moderately distributed, selecting stable and field-of-view complementary images for fusion can comprehensively present the operating status of the equipment and the activity trajectories of the personnel, providing a reliable basis for safe production. For low-density areas, due to fewer targets and less occlusion, 1-2 image sources can meet the requirements. The single-source + time-series extension mode is adopted. Based on a single high-quality image source, the number of image frames is increased through time-series extension to enhance the image coherence. There is no need for a complex weighting strategy in this area. For example, in the low-density area at the edge of the working face, using a single-source to collect images and then performing time-series extension can enhance the quality of the single-source images and present the situation of this area with continuous pictures, ensuring the comprehensiveness of monitoring.

[0033] In a preferred embodiment, image fusion is performed on the screened image sources for real-time monitoring of the current sub-region with the corresponding region adaptive fusion strategy. If the region adaptive fusion strategy corresponding to the screened image sources is main perspective priority fusion, it includes: identifying the sizes of the image quality scores corresponding to the screened image sources respectively, selecting the first image source as the main image source, and the remaining image sources as the auxiliary image sources of the main image source; fusing the auxiliary image sources based on the main image source to obtain a sequence of fused images for each sub-region.

[0034] Exemplarily, if the "main perspective first fusion" mode in the region adaptive fusion strategy is adopted to achieve efficient image integration. First, the system identifies and screens the image quality scores of each image source, which are calculated based on multiple dimensions such as clarity, perspective coverage, frame rate, and feature integrity. For example, visible light images obtain a relatively high comprehensive score due to high clarity (score 0.85) and wide perspective coverage (score 0.78). Then, the image source with the highest score is automatically selected as the main image source (such as visible light images), and the remaining image sources are used as auxiliary image sources. During the main perspective first fusion process, with the main image source as the reference framework, the key information (such as temperature anomaly areas) of the auxiliary image source (such as an infrared thermal map for supplementing details) is accurately superimposed onto the main image through feature matching and color correction techniques. At the same time, a fade-in and fade-out algorithm is used to smoothly transition the edges to avoid information mutations. For example, in the roof monitoring scenario, the main image source provides the overall structural texture, and the auxiliary image source enhances the fracture development characteristics. Finally, a fused image sequence is generated, which not only ensures the monitoring efficiency but also improves the information richness of the key areas, providing intuitive and comprehensive visual support for mining safety decision-making.

[0035] In a preferred embodiment, image fusion is performed on the screened image sources for real-time monitoring of the current sub-region with the corresponding region adaptive fusion strategy. If the region adaptive fusion strategy corresponding to the screened image source is confidence weighted fusion, it includes: calculating the confidence index of the screened image source, and performing confidence weighted fusion on the screened image source according to the confidence index to obtain a fused image sequence for each sub-region.

[0036] Optionally, intelligent integration of multi-source image data is achieved through a region adaptive fusion strategy to generate a high-fidelity sub-region fusion image sequence. Specifically, image sources applicable to the current sub-region are screened according to real-time monitoring requirements. For example, visible light images and infrared thermal maps are selected in the roof fracture zone to simultaneously capture the texture details and temperature anomalies of coal and rock masses. Then, the region adaptive fusion strategy is executed for the screened image sources. If the strategy is "confidence weighted fusion", the confidence calculation link is entered, that is, the confidence index of the image sources is calculated through feature extraction algorithms (such as SIFT key point matching, HOG feature description). This index comprehensively considers image clarity (based on the Laplacian gradient amplitude), target recognition rate (such as the accuracy of hydraulic support posture detection), and time synchronization (inter-frame time difference <50ms). For example, in the scenario of rock burst early warning, the confidence weight of the temperature field of the infrared thermal map is set to 0.6, and the texture confidence weight of the visible light video is 0.4. Subsequently, the screened image sources are weighted and fused according to the confidence index, and a pixel-level weighted superposition algorithm is used to generate a fusion image. For example, in the equipment-intensive area, the high-confidence visible light equipment contour and the infrared temperature distribution map are seamlessly fused, significantly improving the accuracy of equipment status monitoring. Finally, a fusion image sequence for each sub-region is output, providing multi-dimensional information support for mining safety decision-making. This confidence-weighted fusion mechanism effectively improves the accuracy and robustness of state perception in complex mining environments.

[0037] In a preferred embodiment, a three-dimensional regional grid model of the coal mining face is constructed, including: obtaining the three-dimensional point cloud data of the coal mining face, and determining a three-dimensional regional model according to the three-dimensional point cloud data; obtaining the equipment distribution information of the coal mining face, performing grid scale fitness analysis according to the equipment distribution information, and determining the grid size step of the three-dimensional space region; performing grid processing on the three-dimensional regional model based on the grid size step to obtain the three-dimensional regional grid model.

[0038] Specifically, a high-precision three-dimensional regional grid model of the coal mining face is constructed through multi-source data fusion and adaptive grid division technology. First, three-dimensional point cloud data of the coal mining face is collected using lidar or depth cameras, and a high-quality three-dimensional regional model is generated through point cloud registration and denoising algorithms (such as the ICP algorithm and statistical filtering). This model accurately restores the spatial forms of physical structures such as coal walls, roof, and supports. Then, the equipment distribution information of the working face (such as the positions of hydraulic supports and the trajectories of shearers) is obtained, and grid scale fitness analysis is carried out based on equipment density and spatial distribution characteristics. For example, small-sized grids (step size 0.5 meters) are used in areas with dense supports to capture the details of equipment postures, while large-sized grids (step size 2 meters) are used in open areas to reduce computational complexity. Subsequently, according to the grid size step determined by the analysis, the three-dimensional regional model is meshed using octree or Delaunay triangulation algorithms to generate a three-dimensional regional grid model with spatial topological relationships. For example, in a longwall working face, grids with a step size of 0.8 meters are used in the coal wall spalling area to accurately monitor crack propagation, and grids with a step size of 1.5 meters are used in the transportation roadway to balance accuracy and efficiency. This adaptive grid division method based on equipment distribution significantly improves the model's response ability to mining dynamics and the analysis accuracy of multi-source data fusion.

[0039] A dynamic monitoring image fusion method in a coal mining face provided by an embodiment of the present invention has at least the following technical effects: 1. Through the construction of a three-dimensional regional grid model and the analysis of the spatial distribution density of historical collected images, adaptive clustering of grid cells is achieved. For example, small-sized grids are used for fine division in high-density areas (such as stress concentration areas), while large-sized grids are used in low-density areas (such as idle areas) to reduce computational complexity. This dynamic clustering method significantly improves the physical rationality and computational efficiency of regional division, enabling monitoring resources to focus on key areas.

[0040] 2. A quality scoring mechanism is introduced. Through the weighted evaluation of image clarity, view coverage, frame rate, and feature integrity, high-value image sources are screened out. At the same time, a regional adaptive fusion strategy template mapping library is established, and the optimal fusion strategy (such as confidence-weighted fusion, main view priority fusion) is automatically matched according to the type and quantity of image sources. For example, the main view priority fusion strategy is automatically selected in areas with dense equipment to ensure high-precision monitoring of the states of key equipment, effectively solving the problems of redundancy of multi-source image data and poor fusion effect.

[0041] 3. By setting spatial adjacency conditions, it is ensured that the clustering results conform to the physical space continuity. For example, the isolated grid cells in the initial clustering due to noise or occlusion are re-divided so that they belong to adjacent sub-regions. This dynamic correction algorithm significantly improves the robustness of the region division, avoids misjudgment caused by spatial discontinuity, and provides a more reliable regional benchmark for subsequent image fusion and mining decision-making.

[0042] Embodiment 2:

[0043] As Figure 2 shown, based on the same inventive concept as the dynamic monitoring image fusion method in a coal mining face provided in Embodiment 1, the embodiment of the present invention further provides a dynamic monitoring image fusion system in a coal mining face, and the system includes: A model construction unit 11 for constructing a three-dimensional regional grid model of the coal mining face.

[0044] A grid clustering unit 12 for identifying the spatial distribution density of each grid cell in the three-dimensional regional grid model according to the historical acquisition images, and clustering each grid cell according to the spatial distribution density to output a plurality of sub-regions, wherein each sub-region corresponds to a spatial distribution density level.

[0045] An image screening unit 13 for linking the plurality of sub-regions to a multi-source dynamic monitoring module to perform image source screening according to a quality scoring mechanism, and outputting the screened image sources corresponding to each sub-region.

[0046] An image fusion unit 14 for obtaining a corresponding region adaptive fusion strategy for the screened image source corresponding to each sub-region, and performing image fusion on the real-time monitoring images of the current sub-region with the corresponding region adaptive fusion strategy to obtain a fusion image sequence for each sub-region.

[0047] Furthermore, the grid clustering unit 12 is further configured to perform the following steps: Project the historical acquisition images onto the three-dimensional regional grid model to obtain a set of covered grid cells; detect the density feature vectors of each grid cell in the set of covered grid cells, where the density feature vectors include the number of shooting times, the number of targets appearing, the image texture complexity, and the occlusion probability; calculate the weights of the density feature vectors to output the spatial distribution density of each grid cell.

[0048] Furthermore, the grid clustering unit 12 is further configured to perform the following steps: Perform initial clustering on all grids according to the magnitude of the spatial distribution density, and output the initial clustering result, where the initial clustering result includes multiple initial sub-regions; set the fusion spatial adjacency condition, perform adjacency analysis on the three-dimensional coordinates of each initial sub-region according to the fusion spatial adjacency condition, and obtain the three-dimensional coordinates that do not satisfy the adjacency; re-partition the three-dimensional coordinates that do not satisfy the adjacency, and output multiple sub-regions.

[0049] Furthermore, the image screening unit 13 is also used to perform the following steps: Obtain multiple monitoring sources in the multi-source dynamic monitoring module; evaluate the image quality scores corresponding to each sub-region with respect to the multiple monitoring sources according to the quality scoring mechanism, and screen and output the image sources with scores greater than the preset quality score; wherein, the image quality is obtained through a weighted scoring formula of image clarity, viewing angle coverage, image frame rate, and feature integrity; link the multiple sub-regions to the multi-source dynamic monitoring module, establish a transmission protocol between each sub-region and the corresponding screened image source, and transmit the real-time monitoring images corresponding to the screened image source according to the transmission protocol.

[0050] Furthermore, the image fusion unit 14 further includes: Among them, the built-in modules of the region adaptive fusion strategy include confidence weighted fusion, main view priority fusion, and multi-channel partition fusion; use the type and quantity of image source samples as input, and use the built-in module corresponding to the optimal solution of the image fusion effect as output to establish a strategy template mapping library, and obtain the region adaptive fusion strategy corresponding to the screened image source according to the strategy template mapping library.

[0051] Furthermore, the image fusion unit 14 is also used to perform the following steps: Identify the magnitudes of the image quality scores corresponding to the screened image sources respectively, select the first image source as the main image source, and the remaining image sources as the auxiliary image sources of the main image source; fuse the auxiliary image sources based on the main image source to obtain a fused image sequence for each sub-region.

[0052] Furthermore, the image fusion unit 14 is also used to perform the following steps: Calculate the confidence index of the screened image sources, and perform confidence weighted fusion on the screened image sources according to the confidence index to obtain a fused image sequence for each sub-region.

[0053] Furthermore, the model construction unit 11 is also used to perform the following steps: Obtain the three-dimensional point cloud data of the coal mining face, and determine the three-dimensional regional model according to the three-dimensional point cloud data; obtain the equipment distribution information of the coal mining face, perform grid scale fitness analysis according to the equipment distribution information, and determine the grid size step of the three-dimensional space region; perform grid processing on the three-dimensional regional model based on the grid size step to obtain the three-dimensional regional grid model.

[0054] Through the foregoing detailed description of a dynamic monitoring image fusion method in a coal mining face in this specification, those skilled in the art can clearly know a dynamic monitoring image fusion system in a coal mining face in this embodiment. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0055] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dynamic monitoring image fusion method in a coal mining face, characterized in that: The method comprises: Construct a three-dimensional regional grid model of the coal mining face; Identify the spatial distribution density of each grid unit in the three-dimensional regional grid model according to the historically collected images, cluster the grid units according to the spatial distribution density and output a plurality of sub-regions, wherein each sub-region corresponds to a spatial distribution density level; Linking the multiple sub-regions to a multi-source dynamic monitoring module to perform image source screening according to a quality scoring mechanism, and outputting the screened image source corresponding to each sub-region; For the screening image source corresponding to each sub-region, the corresponding regional adaptive fusion strategy is obtained, and the real-time monitoring image of the current sub-region is fused with the corresponding regional adaptive fusion strategy to obtain a fused image sequence of each sub-region.

2. The method according to claim 1, characterized in that Identifying the spatial distribution density of each grid cell in the three-dimensional regional grid model according to the historically collected images includes: Obtain the historically collected image and project it into the three-dimensional regional grid model to obtain a set of covered grid units; Detecting a density feature vector of each grid unit in the set of covered grid units, wherein the density feature vector includes the number of shots, the number of targets that appear, the image texture complexity, and the occlusion probability; By performing weight calculation on the density feature vector, the spatial distribution density of each grid unit is output.

3. The method according to claim 2, characterized in that Cluster each grid unit according to the spatial distribution density to output multiple sub-areas, including: Performing initial clustering on all grids according to the size of the spatial distribution density, and outputting an initial clustering result, wherein the initial clustering result includes a plurality of initial sub-regions; Setting a fusion space adjacency condition, performing adjacency analysis on the three-dimensional coordinates of each initial sub-region according to the fusion space adjacency condition, and obtaining three-dimensional coordinates that do not meet the adjacency condition; The three-dimensional coordinates that do not meet the adjacency requirement are re-divided and multiple sub-regions are output.

4. The method according to claim 1, characterized in that Linking the multiple sub-regions to the multi-source dynamic monitoring module to perform image source screening according to the quality scoring mechanism, and outputting the screened image source corresponding to each sub-region, including: Acquire multiple monitoring sources in the multi-source dynamic monitoring module; Evaluate the image quality scores of each sub-region corresponding to the multiple monitoring sources according to the quality scoring mechanism, and select image source outputs with a quality score greater than a preset quality score; The image quality is obtained by a weighted scoring formula of image clarity, viewing angle coverage, image frame rate and feature completeness; The multiple sub-areas are linked to a multi-source dynamic monitoring module, a transmission protocol is established between each sub-area and a corresponding screening image source, and the real-time monitoring image corresponding to the screening image source is transmitted according to the transmission protocol.

5. The method according to claim 4, characterized in that For each sub-region corresponding to the filtered image source, obtain the corresponding regional adaptive fusion strategy; Among them, the built-in modules of the regional adaptive fusion strategy include confidence weighted fusion, main view priority fusion and multi-channel partition fusion; The type and quantity of image source samples are taken as input, and the built-in module corresponding to the optimal solution of image fusion effect is taken as output, a strategy template mapping library is established, and the regional adaptive fusion strategy corresponding to the screened image source is obtained according to the strategy template mapping library.

6. The method according to claim 5, characterized in that The image fusion is performed on the filtered image source monitored in real time in the current sub-region using the corresponding regional adaptive fusion strategy. If the regional adaptive fusion strategy corresponding to the filtered image source is the main perspective priority fusion, it includes: Identify the image quality scores corresponding to the screened image sources, select the first image source as the main image source, and the remaining image sources as auxiliary image sources of the main image source; The auxiliary image sources are fused based on the main image source to obtain a fused image sequence for each sub-region.

7. The method according to claim 5, characterized in that Performing image fusion on the filtered image source monitored in real time in the current sub-region using the corresponding regional adaptive fusion strategy, if the regional adaptive fusion strategy corresponding to the filtered image source is confidence weighted fusion, including: The confidence index of the screened image source is calculated, and the screened image source is confidence-weighted fused according to the confidence index to obtain a fused image sequence for each sub-region.

8. The method according to claim 1, characterized in that Construct a three-dimensional regional grid model of the coal mining face, including: Acquire three-dimensional point cloud data of the coal mining working face, and determine a three-dimensional regional model according to the three-dimensional point cloud data; Acquire equipment distribution information of the coal mining face, perform grid scale fitness analysis based on the equipment distribution information, and determine the grid size step length of the three-dimensional space area; The three-dimensional regional model is meshed based on the mesh size step to obtain the three-dimensional regional mesh model.

9. A dynamic monitoring image fusion system in a coal mining face, characterized in that: The system is used to implement a method for dynamic monitoring image fusion in a coal mining face according to any one of claims 1 to 8, and comprises: A model building unit, used for building a three-dimensional regional grid model of the coal mining working face; A grid clustering unit, used to identify the spatial distribution density of each grid unit in the three-dimensional regional grid model according to the historically collected images, and cluster each grid unit according to the spatial distribution density to output a plurality of sub-regions, wherein each sub-region corresponds to a spatial distribution density level; An image screening unit, used for linking the plurality of sub-regions to a multi-source dynamic monitoring module to screen image sources according to a quality scoring mechanism, and outputting a screened image source corresponding to each sub-region; The image fusion unit is used to obtain the corresponding regional adaptive fusion strategy for the screening image source corresponding to each sub-region, and perform image fusion on the real-time monitoring image of the current sub-region with the corresponding regional adaptive fusion strategy to obtain a fused image sequence of each sub-region.

Images