A dynamic monitoring image fusion method and system in a coal mining face

Abstract

The present invention relates to a dynamic monitoring image fusion method and system in a coal mining face, belonging to the field of image processing. A three-dimensional regional grid model of the coal mining face is constructed; the spatial distribution density of each grid unit in the three-dimensional regional grid model is identified according to historical acquisition images, and each grid unit is clustered to output multiple sub-regions; the multiple sub-regions are linked to a multi-source dynamic monitoring module to perform image source screening according to a quality scoring mechanism, and the screened image sources corresponding to each sub-region are output, and the corresponding regional adaptive fusion strategy is obtained. The real-time monitoring images of the current sub-region are subjected to image fusion with the corresponding regional adaptive fusion strategy to obtain a fusion image sequence for each sub-region, solving the technical problems of poor environmental adaptability of coal mining face image fusion, insufficient uniformity of image fusion features, resulting in limited accuracy and practicability, and low safety monitoring level.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and particularly to a dynamic monitoring image fusion method and system for a coal mining face. Background Art

[0002] During the coal mining process, the safety monitoring of the coal mining face is a crucial link. Due to the uneven distribution of equipment and elements such as supports, shearers, belts, and personnel in the coal mining face, there are significant differences in the target density and occlusion complexity in different regions, which pose great challenges to existing image fusion methods.

[0003] Existing image fusion methods usually adopt a unified fusion strategy and algorithm to perform fusion processing on the images of the entire coal mining face. However, when facing the differences in target density and occlusion complexity in different regions, this method often fails to take into account the visual characteristics of both high-density regions and low-density regions. In high-density regions, due to the large number of targets and occluders, existing fusion methods are prone to loss of image details or fusion distortion, making it impossible to accurately extract key information. In low-density regions, due to the small number of targets and occluders, existing fusion methods may overemphasize contour information and ignore detail features, also affecting the monitoring effect. Summary of the Invention

[0004] In view of the technical problems in the prior art that the adaptability of the coal mining face image fusion environment is poor, the uniformity of the image fusion features is insufficient, resulting in limited accuracy and practicality, and the safety monitoring level is low, the present invention provides a dynamic monitoring image fusion method and system for a coal mining face to solve these problems.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] In a first aspect, the present invention provides a dynamic monitoring image fusion method for a coal mining face, the method comprising: constructing a three-dimensional regional grid model of the coal mining face; identifying the spatial distribution density of each grid unit in the three-dimensional regional grid model according to historical acquisition images, clustering each grid unit according to the spatial distribution density to output a plurality of sub-regions, wherein each sub-region corresponds to a spatial distribution density level; linking the plurality of sub-regions to a multi-source dynamic monitoring module to perform image source screening according to a quality scoring mechanism, and outputting a screened image source corresponding to each sub-region; for the screened image source corresponding to each sub-region, obtaining a corresponding region adaptive fusion strategy, and performing image fusion on the real-time monitoring image of the current sub-region with the corresponding region adaptive fusion strategy to obtain a fusion image sequence for each sub-region.

[0007] Second aspect, the present invention provides a dynamic monitoring image fusion system in a coal mining face, the system comprising: a model construction unit for constructing a three-dimensional regional grid model of the coal mining face; a grid clustering unit for identifying the spatial distribution density of each grid unit in the three-dimensional regional grid model according to historical collected images, clustering each grid unit according to the spatial distribution density to output a plurality of sub-regions, wherein each sub-region corresponds to a spatial distribution density level; an image screening unit for linking the plurality of sub-regions to a multi-source dynamic monitoring module to perform image source screening according to a quality scoring mechanism, and outputting a screened image source corresponding to each sub-region; and an image fusion unit for obtaining a corresponding region adaptive fusion strategy for the screened image source corresponding to each sub-region, and performing image fusion on the real-time monitoring image of the current sub-region with the corresponding region adaptive fusion strategy to obtain a fusion image sequence for each sub-region.

[0008] The beneficial effects of the present invention are as follows: By constructing a three-dimensional regional grid model of the coal mining face, identifying the spatial distribution density of each grid unit and clustering them into a plurality of sub-regions, then screening the optimal image source for each sub-region, and adopting a region adaptive fusion strategy for image fusion, so as to obtain a fusion image sequence for each sub-region, realizing the adaptive processing of the target density and occlusion complexity of different regions, improving the accuracy and practicability of image fusion, and further improving the safety monitoring level of the coal mining face in coal mines. Description of the Drawings

[0009] Figure 1 It is a schematic flow chart of a dynamic monitoring image fusion method in a coal mining face provided by the present invention.

[0010] Figure 2 It is a schematic structural diagram of a dynamic monitoring image fusion system in a coal mining face provided by the present invention.

[0011] Description of the reference numerals: model construction unit 11, grid clustering unit 12, image screening unit 13, image fusion unit 14. Detailed Embodiments

[0012] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of the present invention.

[0013] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present invention, "a plurality of" means two or more unless otherwise specifically defined.

[0014] In the description of the present invention, the term "for example" is used to mean "serving as an example, illustration, or explanation". Any embodiment described as "for example" in the present invention is not necessarily construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the present invention. In the following description, details are set forth for purposes of explanation. It should be understood that those of ordinary skill in the art can recognize that the present invention can be implemented without the use of these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0015] Embodiment 1:

[0016] As Figure 1 shown, an embodiment of the present invention provides a dynamic monitoring image fusion method in a coal mining face, and the method includes:

[0017] S10: Construct a three-dimensional regional grid model of the coal mining face.

[0018] Exemplarily, constructing a three-dimensional regional grid model of a coal mining face requires integrating geological exploration data, mining process parameters, and numerical simulation techniques. Specifically, first, the occurrence characteristics of coal seams, such as strike, dip, and thickness variation, are analyzed through three-dimensional seismic exploration and borehole data, and then combined with the geometric parameters of geological structures (faults, folds) to establish a basic geological model. Next, according to the coal mining method (such as longwall mining, top coal caving technology), the advancing direction of the working face, the support layout, and the mining step distance are determined, and the equipment parameters (hydraulic support size, shearer cutting depth) are converted into spatial constraint conditions. Subsequently, a grid meshing algorithm (such as Delaunay triangulation or hexahedron mesh generation) is used to discretize the geological body. Special attention should be paid to the grid encryption of complex areas such as coal seam bifurcation areas and roof broken zones during this process. For example, a gradually changing size grid is used in the fault zone to balance calculation accuracy and efficiency. Finally, a three-dimensional calculation grid that can simulate stress distribution and deformation failure is formed by coupling rock mass mechanical parameters (elastic modulus, Poisson's ratio) with the dynamic boundary conditions of mining. For example, in the modeling of a thick coal seam top coal caving working face in a certain mine, by refining the grid density in the top coal caving area, the dynamic relationship between the support working resistance and the top coal recovery rate is successfully predicted, providing a quantitative basis for optimizing the mining plan.

[0019] S20: Identify the spatial distribution density of each grid cell in the three-dimensional regional grid model according to the historical acquisition images, and cluster each grid cell according to the spatial distribution density to output multiple sub-regions, where each sub-region corresponds to a spatial distribution density level.

[0020] Preferably, the spatial density analysis and clustering of the three-dimensional regional grid model are further driven by historical collected image data to achieve refined zoning management of the mining area. The spatial distribution density characteristics of grid cells can be extracted based on historical monitoring images of the coal mining face (such as visible light and infrared thermal imaging). For example, the caving degree of the top coal, the rib spalling frequency or the support pressure distribution density can be calculated through the image pixel intensity gradient, and each grid cell is quantified into a continuous density index, such as a normalized density value in the range of 0-1. Then, a density clustering algorithm (such as DBSCAN or density-based spatial clustering method) is used to perform unsupervised classification on the grid cells. The algorithm automatically identifies the density peak areas and sparse areas, and divides sub-regions of different grades according to the density threshold. For example, in the grid cells of the roof fracture zone, the density value continuously exceeds 0.7 due to frequent caving and is clustered into a high-density sub-region, while the density value of the stable rib area is less than 0.3 and is classified as a low-density sub-region. Finally, multiple sub-regions with clear spatial distribution density levels are output, and each sub-region corresponds to different mining response characteristics. For example, in the application of an intelligent working face, the high-density stress concentration area (density level > 0.8) identified by clustering is preferentially included in the key monitoring scope of rock burst, while the low-density stable area (density level < 0.4) adopts the conventional support strategy. This regional management based on spatial density grading significantly improves the accuracy of mining dynamic regulation.

[0021] S30: Link the multiple sub-regions to a multi-source dynamic monitoring module to perform image source screening according to a quality scoring mechanism, and output the screened image sources corresponding to each sub-region.

[0022] Furthermore, through the deep coupling of the multi-source dynamic monitoring module and the spatial characteristics of sub-regions, the quality scoring and precise screening of image sources are realized, and low-value image sources are eliminated. Specifically, multiple sub-regions generated by clustering (such as high-density regions of roof fracture zones, low-density regions of stable coal walls, etc.) are used as spatial constraint units to establish topological associations with the image data streams of the multi-source monitoring system (such as visible light cameras, infrared thermal imagers, lidar point clouds). Each sub-region dynamically binds the multi-source image sources within its spatial range. Subsequently, a quality scoring mechanism is constructed to quantitatively evaluate the effectiveness of image sources through characteristic indicators. For example, the clarity index (based on the gradient magnitude of the Laplacian operator) is used for visible light images, the temperature field entropy value analysis is used for infrared images, and the density distribution uniformity index is used for point cloud data. At the same time, spatio-temporal consistency constraints are introduced, such as the differential entropy of adjacent frame images within the same sub-region, and finally a multi-dimensional scoring system is formed. Then, the scoring weights are dynamically adjusted according to the mining risk levels of sub-regions. For example, in high-risk sub-regions of rock bursts (density level > 0.8), the weight of the temperature anomaly sensitivity of infrared thermal imaging is increased to 0.6, while in stable support areas (density level < 0.4), more emphasis is placed on the clarity weight of visible light images (such as 0.7). Finally, the optimal image source combination corresponding to each sub-region is output through threshold screening or the Top-K strategy. For example, in a fully mechanized coal mining face, the image sources screened out in the sub-region of the roof fracture zone include high-resolution infrared thermal maps (identifying temperature anomalies of fissure water seepage) and dense point cloud data (quantifying the distribution of caving block sizes), while in the coal wall spalling area, high-frame-rate visible light videos are preferred (capturing the dynamic process of spalling). This image source screening mechanism based on quality scoring significantly improves the timeliness and reliability of mining state perception.

[0023] S40: For the screened image sources corresponding to each sub-region, obtain the corresponding region adaptive fusion strategy, and perform image fusion on the real-time monitoring images of the current sub-region with the corresponding region adaptive fusion strategy to obtain the fused image sequence of each sub-region.

[0024] Specifically, the intelligent integration of multi-source image data is finally achieved through the region adaptive fusion strategy to generate a high-fidelity sub-region fusion image sequence. For the selected image sources of each sub-region (such as infrared thermal maps, visible light videos, lidar point clouds), a region adaptive fusion strategy is dynamically generated based on its spatial distribution density level and dynamic monitoring requirements. For example, in the sub-region with high stress concentration (density level > 0.8), an enhanced fusion mode of "infrared thermal map + point cloud structured light" is adopted, and potential impact risks are identified through the coupled analysis of the temperature field gradient and the three-dimensional deformation field. In the sub-region of the broken roof belt, it is switched to a complementary fusion strategy of "visible light texture + radar depth". The edge detection algorithm is used to extract the contour of the caving blocks, and the three-dimensional shape of the caved body is reconstructed by combining the depth information. Subsequently, a feature-level or decision-level fusion algorithm is designed according to the fusion strategy. For example, in the coal wall spalling area, the wavelet transform multi-scale fusion technology is used to perform weighted superposition of the texture details of the visible light image and the temperature anomaly area of the infrared image in the frequency domain, and at the same time, the Kalman filter is used to suppress noise interference. For the equipment-intensive area (such as a hydraulic support group), a semantic segmentation network based on deep learning is introduced to perform pixel-level alignment of the spatial position information of the point cloud data and the visual features of the visible light image, and the joint monitoring of the support attitude and roof separation is realized. Finally, a fusion image sequence of each sub-region is output. For example, in the rock burst warning scenario, the fusion image sequence simultaneously presents the temperature evolution trend of the coal and rock mass (infrared thermal map), the spatial distribution of microseismic events (point cloud seismic source location), and the acoustic emission signal intensity (visible light video vibration analysis), providing multi-dimensional information support for mining safety decisions. This region adaptive image fusion mechanism significantly improves the accuracy and robustness of state perception in complex mining environments.

[0025] In a preferred embodiment, the spatial distribution density of each grid cell in the three-dimensional regional grid model is identified according to the historically collected images, including: obtaining the projection of the historically collected images onto the three-dimensional regional grid model to obtain a set of covered grid cells; detecting the density feature vectors of each grid cell in the set of covered grid cells, where the density feature vectors include the number of times of being photographed, the number of targets appearing, the complexity of the image texture, and the occlusion probability; and calculating the weights of the density feature vectors to output the spatial distribution density of each grid cell.

[0026] Specifically, through the depth mapping of historical acquisition images and the three-dimensional regional grid model, the accurate quantification of the spatial distribution density of grid cells is achieved. First, historical acquisition images, such as surveillance camera video frames, are projected into the three-dimensional regional grid model through a spatial registration algorithm to identify the set of grid cells within the image coverage. For example, in the longwall face scenario, a single-frame surveillance image can cover dozens of grid cells, thus forming the target area to be analyzed. Then, for each grid cell in the set, multi-dimensional density feature vectors are extracted, including: 1) The number of times of being photographed quantifies the monitoring attention by statistically counting the occurrence frequency of the grid cell in historical images; 2) The number of targets appearing uses a target detection algorithm (such as YOLOv5) to count equipment instances such as coal shearers and supports; 3) The image texture complexity calculates the entropy value based on the gray-level co-occurrence matrix, reflecting the degree of coal wall fragmentation or the development state of roof fissures; 4) The occlusion probability predicts the possibility of the grid cell being occluded by coal dust and equipment through a depth estimation network. For example, in the roof water inrush area, the texture complexity and occlusion probability eigenvalue increase significantly. Subsequently, a dynamic weight allocation mechanism is designed to adjust the feature weights according to the mining stage. For example, in the initial mining period, the number of times of being photographed (weight 0.4) and the number of equipment targets (weight 0.3) are emphasized, while in the periodic weighting stage, the texture complexity (weight 0.5) and the occlusion probability (weight 0.2) are given more weight; finally, the spatial distribution density value of each grid cell is output through weighted summation. For example, a certain grid cell has a density value of 0.78 after weight calculation because it is frequently photographed (number of times of being photographed = 15), continuously has abnormal target of support pressure (number of targets = 8), the texture complexity reaches 0.82 and the occlusion probability is only 0.1. This density quantification method integrating multiple features provides a reliable spatial basis for subsequent regional clustering and dynamic monitoring.

[0027] In a preferred embodiment, each grid cell is clustered according to the spatial distribution density to output multiple sub-regions, including: initially clustering all grids according to the size of the spatial distribution density to output an initial clustering result, the initial clustering result including multiple initial sub-regions; setting a fusion spatial adjacency condition, analyzing the adjacency of the three-dimensional coordinates of each initial sub-region according to the fusion spatial adjacency condition to obtain the three-dimensional coordinates that do not meet the adjacency; re-dividing the three-dimensional coordinates that do not meet the adjacency to output multiple sub-regions.

[0028] Preferably, through the synergistic effect of spatial distribution density and adjacency constraint, high-precision clustering of grid cells and sub-region division are achieved. First, based on the spatial distribution density values of the grid cells, density peak clustering (DPC) or K-Means++ algorithm is used for initial clustering. For example, grid cells with density values higher than 0.7 are preliminarily classified as stress concentration areas, and those with density values lower than 0.3 are classified as stable areas, forming multiple initial sub-regions. Then, the spatial adjacency condition is introduced as a post-clustering processing rule. By setting a three-dimensional space distance threshold (such as 5 meters) and a density decay gradient (such as a density decrease of 0.1 per meter), it is ensured that physically adjacent grid cells are divided into the same sub-region. For example, in a longwall face, if the three-dimensional coordinate distance between two high-density grid cells exceeds 10 meters, they will be marked as not meeting the adjacency even if their density values are similar. Subsequently, for three-dimensional coordinates that do not meet the adjacency, methods such as connected component analysis in graph theory or density-based region growing algorithm are used for re-division. For example, isolated high-density points (caused by equipment occlusion resulting in abnormal density) are merged into adjacent sub-regions with slightly lower density, or "density islands" in low-density areas (caused by sensor noise) are re-classified as background areas. Finally, multiple sub-regions that meet spatial continuity are output. For example, in rock burst monitoring, after correction by adjacency constraint, a long-distance abnormal point that was originally misjudged as an independent high-density area is correctly classified into the roof fracture belt sub-region. This clustering method that combines density features and spatial topology significantly improves the physical rationality of region division and the pertinence of mining response.

[0029] In a preferred embodiment, the multiple sub-regions are linked to a multi-source dynamic monitoring module to screen image sources according to a quality scoring mechanism, and the screened image sources corresponding to each sub-region are output, including: obtaining multiple monitoring sources in the multi-source dynamic monitoring module; evaluating the image quality scores corresponding to each sub-region for the multiple monitoring sources respectively according to the quality scoring mechanism, and screening and outputting image sources with scores greater than a preset quality score; wherein, the image quality is obtained through a weighted scoring formula of image sharpness, view coverage, image frame rate, and feature integrity; the multiple sub-regions are linked to the multi-source dynamic monitoring module, a transmission protocol between each sub-region and the corresponding screened image source is established, and the real-time monitoring images corresponding to the screened image sources are transmitted according to the transmission protocol.

[0030] Furthermore, through the spatial mapping of the multi-source dynamic monitoring module and sub-regions, combined with the quality scoring mechanism, intelligent screening and efficient transmission of image sources are realized. Specifically, spatial associations are established between the multiple sub-regions generated by clustering and the image sources of the multi-source monitoring system, and each sub-region is dynamically bound to multiple monitoring sources within its coverage. Then, an image quality scoring mechanism is constructed to comprehensively evaluate the monitoring value of image sources through a weighted formula. Among them, the image sharpness is calculated using the Laplacian gradient magnitude; the viewing angle coverage is analyzed based on the spatial intersection of the three-dimensional model of the sub-region and the camera frustum; the image frame rate dynamically adjusts the weight according to the mining dynamic characteristics (for example, the frame rate weight is increased to 0.3 in rock burst monitoring); and the feature integrity quantifies the recognition rate of key features (such as the attitude of hydraulic supports and coal-rock fractures) through an object detection algorithm (such as SSD). For example, in the sub-region of the roof broken belt, the weight of the temperature field feature integrity of the infrared thermal image is set to 0.4, and the weight of the texture details of the visible light video is 0.3. Subsequently, a quality scoring threshold (such as 0.65) is set, and image sources greater than this threshold are screened as valid inputs. For example, in the coal wall rib spalling area, visible light image sources with sharpness lower than 0.5 due to coal dust occlusion are excluded, and at the same time, high-frame-rate (50fps) high-speed camera data is retained to capture the dynamic process of rib spalling. Finally, a customized transmission protocol for sub-regions and screened image sources is established, and the MQTT protocol is used to achieve low-latency image push. For example, when a warning is triggered in the stress anomaly sub-region, the transmission priorities of the corresponding infrared thermal images and acoustic emission sensors are automatically increased. This image source screening and transmission mechanism based on quality scoring significantly reduces data redundancy, ensures that each sub-region only receives high-value monitoring data, and provides accurate information support for mining decisions.

[0031] In a preferred embodiment, for the screened image sources corresponding to each sub-region, the corresponding region adaptive fusion strategy is obtained; wherein, the built-in modules of the region adaptive fusion strategy include confidence weighted fusion, main view priority fusion, and multi-channel partition fusion; taking the type and quantity of image source samples as inputs and the built-in module corresponding to the optimal solution representing the image fusion effect as the output, a strategy template mapping library is established, and the region adaptive fusion strategy corresponding to the screened image sources is obtained according to the strategy template mapping library.

[0032] In a specific embodiment, an adaptive fusion strategy matching for screening image sources in sub-regions is achieved by constructing a policy template mapping library. Specifically, first, for the screening image sources (such as visible light, infrared, lidar, etc.) in each sub-region, according to their spatial distribution density levels (high density, medium density, low density) and image source characteristics (type, quantity), a region adaptive fusion strategy is dynamically generated. In high-density sub-regions (such as stress concentration areas), due to complex monitoring requirements and high data redundancy, the policy template mapping library recommends using a combination mode of "confidence-weighted fusion + multi-channel partition fusion". For example, pixel-level confidence weighting is performed on infrared thermal images and visible light videos (based on the probability of temperature anomalies and texture clarity), and at the same time, multi-channel partition fusion technology is used to complementarily fuse image sources from different perspectives (such as top view and side view) in the feature space to comprehensively capture the multi-physical field coupling information of coal and rock masses. In medium-density sub-regions (such as roof stable areas), the policy library preferentially selects the "main perspective first fusion" mode. For example, the front view of the support (clarity > 0.8) is selected as the benchmark from multiple visible light image sources, supplemented by auxiliary perspective images with low weights for feature enhancement, which not only ensures the monitoring efficiency but also reduces the computational overhead. In low-density sub-regions (such as equipment idle areas), a fusion strategy of "maximum coverage angle + strong synchrony" is adopted. For example, the image source with the largest coverage angle captured by a wide-angle lens is selected, and the multi-source image sequences are aligned through a timestamp synchronization mechanism to ensure low-cost and high-efficiency monitoring in areas with less equipment interference and slow dynamic changes. Among them, the construction of the policy template mapping library is based on historical fusion effect data, and the mapping relationship between different image source combinations and fusion strategies is learned through a deep learning model (such as ResNet-50). For example, in the scenario of rock burst early warning, the "multi-modal complementary fusion" strategy in high-density sub-regions has successfully increased the temperature field anomaly detection accuracy to 92%, while the "wide-angle synchronous fusion" strategy in low-density sub-regions has reduced the data transmission volume by 60%. This density-based adaptive fusion strategy significantly improves the utilization efficiency of multi-source image data in complex mining environments.

[0033] Specific policy fusion details can be understood based on the following density-based image fusion strategy table:

[0034]

[0035] Specifically, the table content formulates a differentiated image fusion scheme around the density levels of different areas of the coal mining face, aiming to improve the quality of image fusion and meet the monitoring requirements of different areas. In high-density areas, usually the target distribution is dense and the occlusion situation is complex, and the number of image sources needs to be 3 or more. The mode of combining weighted average fusion and redundancy removal is adopted. Among them, weighted average fusion synthesizes the information of multiple image sources, and redundancy removal can avoid the interference of duplicate information and highlight key details. In the weighting strategy, clarity weight and occlusion ratio penalty are used. For example, in the area near the shearer, equipment and personnel are concentrated, the image is easy to be blurred and there are occlusions. Assigning weights according to the image clarity can make the clear part more prominent in the fused image; while giving a penalty to the occluded area can reduce its contribution in the fusion, thus effectively avoiding information redundancy and improving the clarity of boundary information such as equipment edges and operation details, enabling the staff to accurately grasp the situation of this area. For medium-density areas, the number of image sources is generally 2-3, and the fusion mode is perspective selection and sequence fusion. Perspective selection can select the most representative and complementary perspective images, and sequence fusion integrates these images in chronological order to show the dynamic changes of the area. The weighting strategy emphasizes stability and field-of-view complementarity first. For example, in areas such as along the scraper conveyor, the equipment and personnel are moderately distributed. Selecting stable and field-of-view complementary images for fusion can comprehensively present the operating state of the equipment and the activity trajectories of the personnel, providing a reliable basis for safe production. For low-density areas, due to fewer targets and fewer occlusions, 1-2 image sources can meet the requirements. The single-source + time-series extension mode is adopted. Based on a single high-quality image source, the number of image frames is increased through time-series extension to enhance the image coherence. There is no need for a complex weighting strategy in this area. For example, in the low-density area at the edge of the working face, using a single-source to collect images and then performing time-series extension can enhance the quality of the single-source image and present the situation of this area with continuous pictures, ensuring the comprehensiveness of monitoring.

[0036] In a preferred embodiment, image fusion is performed on the filtered image sources for real-time monitoring of the current sub-region with the corresponding region adaptive fusion strategy. If the region adaptive fusion strategy corresponding to the filtered image sources is main perspective priority fusion, it includes: identifying the sizes of the image quality scores corresponding to the filtered image sources respectively, selecting the first image source as the main image source, and the remaining image sources as the auxiliary image sources of the main image source; performing fusion on the auxiliary image sources based on the main image source to obtain a fused image sequence for each sub-region.

[0037] Exemplarily, if the "main perspective first fusion" mode in the regional adaptive fusion strategy is adopted to achieve efficient image integration. First, the system identifies and screens the image quality scores of each image source. This score is calculated based on multiple dimensions such as clarity, perspective coverage, frame rate, and feature integrity. For example, visible light images obtain a relatively high comprehensive score due to high clarity (score 0.85) and wide perspective coverage (score 0.78). Then, the image source with the highest score is automatically selected as the main image source (such as visible light images), and the remaining image sources are used as auxiliary image sources. During the main perspective first fusion process, with the main image source as the reference framework, the key information (such as temperature anomaly areas) of the auxiliary image source (such as an infrared thermal map for supplementing details) is accurately superimposed onto the main image through feature matching and color correction techniques. At the same time, a fade-in and fade-out algorithm is used to smoothly transition the edges to avoid information mutations. For example, in the roof monitoring scenario, the main image source provides the overall structural texture, and the auxiliary image source enhances the fracture development features. Finally, a fused image sequence is generated, which not only ensures the monitoring efficiency but also improves the information richness of key areas, providing intuitive and comprehensive visual support for mining safety decisions.

[0038] In a preferred embodiment, image fusion is performed on the screened image sources for real-time monitoring of the current sub-region with the corresponding regional adaptive fusion strategy. If the regional adaptive fusion strategy corresponding to the screened image source is confidence-weighted fusion, it includes: calculating the confidence index of the screened image source, and performing confidence-weighted fusion on the screened image source according to the confidence index to obtain a fused image sequence for each sub-region.

[0039] Optionally, intelligent integration of multi-source image data is achieved through a region adaptive fusion strategy to generate a high-fidelity sub-region fusion image sequence. Specifically, image sources applicable to the current sub-region are screened according to real-time monitoring requirements. For example, visible light images and infrared thermal maps are selected in the roof fracture zone area to simultaneously capture the texture details and temperature anomalies of coal and rock masses. Then, the region adaptive fusion strategy is executed for the screened image sources. If the strategy is "confidence weighted fusion", the confidence calculation link is entered, that is, the confidence index of the image sources is calculated through feature extraction algorithms (such as SIFT key point matching, HOG feature description). This index comprehensively considers image clarity (based on Laplacian gradient amplitude), target recognition rate (such as the accuracy of hydraulic support attitude detection), and time synchronization (inter-frame time difference <50 ms). For example, in the rock burst warning scenario, the confidence weight of the temperature field of the infrared thermal map is set to 0.6, and the texture confidence weight of the visible light video is 0.4. Subsequently, the screened image sources are weighted and fused according to the confidence index, and a pixel-level weighted superposition algorithm is used to generate a fusion image. For example, in the equipment-intensive area, the high-confidence visible light equipment contour and the infrared temperature distribution map are seamlessly fused, significantly improving the accuracy of equipment status monitoring. Finally, the fusion image sequence of each sub-region is output, providing multi-dimensional information support for mining safety decision-making. This confidence-weighted fusion mechanism effectively improves the accuracy and robustness of state perception in complex mining environments.

[0040] In a preferred embodiment, a three-dimensional regional grid model of the coal mining face is constructed, including: obtaining the three-dimensional point cloud data of the coal mining face, and determining a three-dimensional regional model according to the three-dimensional point cloud data; obtaining the equipment distribution information of the coal mining face, and performing grid scale fitness analysis according to the equipment distribution information to determine the grid size step of the three-dimensional space region; performing grid processing on the three-dimensional regional model based on the grid size step to obtain the three-dimensional regional grid model.

[0041] Specifically, a high-precision three-dimensional regional grid model of the coal mining face is constructed through multi-source data fusion and adaptive grid division technology. First, three-dimensional point cloud data of the coal mining face is collected using lidar or depth cameras, and a high-quality three-dimensional regional model is generated through point cloud registration and denoising algorithms (such as ICP algorithm and statistical filtering). This model accurately restores the spatial forms of physical structures such as coal walls, roof, and supports. Then, the equipment distribution information of the working face (such as the positions of hydraulic supports and the trajectories of shearers) is obtained, and grid scale fitness analysis is carried out based on equipment density and spatial distribution characteristics. For example, small-sized grids (step size 0.5 m) are used in areas with dense supports to capture the details of equipment postures, while large-sized grids (step size 2 m) are used in open areas to reduce computational complexity. Subsequently, according to the grid size step determined by the analysis, the three-dimensional regional model is meshed using octree or Delaunay triangulation algorithms to generate a three-dimensional regional grid model with spatial topological relationships. For example, in a longwall working face, grids with a step size of 0.8 m are used in the coal wall spalling area to accurately monitor crack propagation, and grids with a step size of 1.5 m are used in the transportation roadway to balance accuracy and efficiency. This adaptive grid division method based on equipment distribution significantly improves the model's response ability to mining dynamics and the analysis accuracy of multi-source data fusion.

[0042] A dynamic monitoring image fusion method in a coal mining face provided by an embodiment of the present invention has at least the following technical effects:

[0043] 1. Through the construction of a three-dimensional regional grid model and the analysis of the spatial distribution density of historical collected images, adaptive clustering of grid cells is achieved. For example, small-sized grids are used for fine division in high-density areas (such as stress concentration areas), while large-sized grids are used in low-density areas (such as idle areas) to reduce computational complexity. This dynamic clustering method significantly improves the physical rationality and computational efficiency of regional division, enabling monitoring resources to focus on key areas.

[0044] 2. A quality scoring mechanism is introduced. Through the weighted evaluation of image clarity, viewing angle coverage, frame rate, and feature integrity, high-value image sources are screened out. At the same time, a regional adaptive fusion strategy template mapping library is established, and the optimal fusion strategy (such as confidence-weighted fusion, main viewing angle priority fusion) is automatically matched according to the type and quantity of image sources. For example, the main viewing angle priority fusion strategy is automatically selected in areas with dense equipment to ensure high-precision monitoring of the states of key equipment, effectively solving the problems of multi-source image data redundancy and poor fusion effect.

[0045] 3. By setting spatial adjacency conditions, it is ensured that the clustering results conform to the physical space continuity. For example, re-partition the isolated grid cells in the initial clustering due to noise or occlusion, so that they belong to adjacent sub-regions. This dynamic correction algorithm significantly improves the robustness of the region division, avoids misjudgment caused by spatial discontinuity, and provides a more reliable regional benchmark for subsequent image fusion and mining decision-making.

[0046] Embodiment 2:

[0047] As Figure 2 shown, based on the same inventive concept as the dynamic monitoring image fusion method in a coal mining face provided in Embodiment 1, the embodiment of the present invention further provides a dynamic monitoring image fusion system in a coal mining face, and the system includes:

[0048] A model construction unit 11 for constructing a three-dimensional regional grid model of a coal mining face.

[0049] A grid clustering unit 12 for identifying the spatial distribution density of each grid cell in the three-dimensional regional grid model according to historical acquisition images, and clustering each grid cell according to the spatial distribution density to output a plurality of sub-regions, wherein each sub-region corresponds to a spatial distribution density level.

[0050] An image screening unit 13 for linking the plurality of sub-regions to a multi-source dynamic monitoring module to perform image source screening according to a quality scoring mechanism, and outputting a screened image source corresponding to each sub-region.

[0051] An image fusion unit 14 for obtaining a corresponding region adaptive fusion strategy for the screened image source corresponding to each sub-region, and performing image fusion on the real-time monitoring image of the current sub-region with the corresponding region adaptive fusion strategy to obtain a fusion image sequence for each sub-region.

[0052] Furthermore, the grid clustering unit 12 is further configured to perform the following steps:

[0053] Project the historical acquisition images onto the three-dimensional regional grid model to obtain a set of covered grid cells; detect the density feature vectors of each grid cell in the set of covered grid cells, where the density feature vectors include the number of times of being photographed, the number of targets appearing, the image texture complexity, and the occlusion probability; calculate the weights of the density feature vectors to output the spatial distribution density of each grid cell.

[0054] Furthermore, the grid clustering unit 12 is further configured to perform the following steps:

[0055] Perform initial clustering on all grids according to the magnitude of the spatial distribution density, and output the initial clustering result, where the initial clustering result includes multiple initial sub-regions; set the fusion spatial adjacency condition, and perform adjacency analysis on the three-dimensional coordinates of each initial sub-region according to the fusion spatial adjacency condition to obtain the three-dimensional coordinates that do not satisfy the adjacency; re-partition the three-dimensional coordinates that do not satisfy the adjacency, and output multiple sub-regions.

[0056] Furthermore, the image screening unit 13 is further configured to perform the following steps:

[0057] Obtain multiple monitoring sources in the multi-source dynamic monitoring module; evaluate the image quality scores corresponding to each sub-region with respect to the multiple monitoring sources respectively according to the quality scoring mechanism, and screen and output the image sources with scores greater than the preset quality score; wherein, the image quality is obtained through a weighted scoring formula of image sharpness, viewing angle coverage, image frame rate, and feature integrity; link the multiple sub-regions to the multi-source dynamic monitoring module, establish a transmission protocol between each sub-region and the corresponding screened image source, and transmit the real-time monitoring images corresponding to the screened image source according to the transmission protocol.

[0058] Furthermore, the image fusion unit 14 further includes:

[0059] Among them, the built-in modules of the region adaptive fusion strategy include confidence weighted fusion, main view priority fusion, and multi-channel partition fusion; use the type and quantity of the image source samples as the input, and use the built-in module corresponding to the optimal solution of the image fusion effect as the output to establish a strategy template mapping library, and obtain the region adaptive fusion strategy corresponding to the screened image source according to the strategy template mapping library.

[0060] Furthermore, the image fusion unit 14 is further configured to perform the following steps:

[0061] Identify the magnitudes of the image quality scores corresponding to the screened image sources respectively, select the first image source as the main image source, and the remaining image sources as the auxiliary image sources of the main image source; fuse the auxiliary image sources based on the main image source to obtain the fused image sequence of each sub-region.

[0062] Furthermore, the image fusion unit 14 is further configured to perform the following steps:

[0063] Calculate the confidence index of the screened image sources, and perform confidence weighted fusion on the screened image sources according to the confidence index to obtain the fused image sequence of each sub-region.

[0064] Furthermore, the model construction unit 11 is further configured to perform the following steps:

[0065] Obtain the three-dimensional point cloud data of the coal mining face, and determine a three-dimensional regional model according to the three-dimensional point cloud data; obtain the equipment distribution information of the coal mining face, perform grid scale fitness analysis according to the equipment distribution information, and determine the grid size step of the three-dimensional space region; perform grid processing on the three-dimensional regional model based on the grid size step to obtain the three-dimensional regional grid model.

[0066] Through the foregoing detailed description of a dynamic monitoring image fusion method in a coal mining face in this specification, those skilled in the art can clearly know a dynamic monitoring image fusion system in a coal mining face in this embodiment. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For related parts, refer to the description in the method part.

[0067] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A dynamic monitoring image fusion method in a coal mining face, characterized in that, The method includes: Constructing a three-dimensional regional grid model of the coal mining face; Identifying the spatial distribution density of each grid cell in the three-dimensional regional grid model according to the historically collected images, clustering each grid cell according to the spatial distribution density, and outputting multiple sub-regions, where each sub-region corresponds to a spatial distribution density level; Linking the multiple sub-regions to a multi-source dynamic monitoring module to screen image sources according to a quality scoring mechanism, and outputting the screened image sources corresponding to each sub-region; For the screened image sources corresponding to each sub-region, obtaining a corresponding regional adaptive fusion strategy, and performing image fusion on the real-time monitoring images of the current sub-region with the corresponding regional adaptive fusion strategy to obtain a fusion image sequence for each sub-region; Among them, linking the multiple sub-regions to a multi-source dynamic monitoring module to screen image sources according to a quality scoring mechanism, and outputting the screened image sources corresponding to each sub-region, includes: Obtaining multiple monitoring sources in the multi-source dynamic monitoring module; Evaluating the image quality scores corresponding to each sub-region for the multiple monitoring sources respectively according to the quality scoring mechanism, and screening and outputting the image sources with a quality score greater than a preset quality score; Among them, the image quality is obtained through a weighted scoring formula of image clarity, viewing angle coverage, image frame rate, and feature integrity; Linking the multiple sub-regions to a multi-source dynamic monitoring module, establishing a transmission protocol between each sub-region and the corresponding screened image source, and transmitting the real-time monitoring images corresponding to the screened image source according to the transmission protocol; For the screened image sources corresponding to each sub-region, obtaining a corresponding regional adaptive fusion strategy; Among them, the built-in modules of the regional adaptive fusion strategy include confidence weighted fusion, main viewing angle priority fusion, and multi-channel partition fusion; Taking the type and quantity of image source samples as input and the built-in module corresponding to the optimal solution representing the image fusion effect as output, establishing a strategy template mapping library, and obtaining the regional adaptive fusion strategy corresponding to the screened image source according to the strategy template mapping library.

2. The method according to claim 1, characterized in that, Identifying the spatial distribution density of each grid cell in the three-dimensional regional grid model according to the historically collected images, includes: Projecting the historically collected images onto the three-dimensional regional grid model to obtain a set of covered grid cells; Detecting the density feature vectors of each grid cell in the set of covered grid cells, where the density feature vectors include the number of shooting times, the number of targets appearing, the complexity of image texture, and the occlusion probability; Outputting the spatial distribution density of each grid cell by performing weight calculation on the density feature vectors.

3. The method according to claim 2, wherein Clustering each grid cell according to the spatial distribution density to output multiple sub-regions, includes: Performing initial clustering on all grids according to the magnitude of the spatial distribution density, and outputting an initial clustering result, where the initial clustering result includes multiple initial sub-regions; Setting a fusion spatial adjacency condition, and performing adjacency analysis on the three-dimensional coordinates of each initial sub-region according to the fusion spatial adjacency condition to obtain three-dimensional coordinates that do not meet the adjacency; Re-dividing the three-dimensional coordinates that do not meet the adjacency, and outputting multiple sub-regions.

4. The method according to claim 1, wherein Perform image fusion on the screened image sources for real-time monitoring of the current sub-region with the corresponding region adaptive fusion strategy. If the region adaptive fusion strategy corresponding to the screened image source is the main perspective priority fusion, it includes: Identify the image quality score sizes corresponding to the screened image sources respectively, select the first image source as the main image source, and the remaining image sources as the auxiliary image sources of the main image source; Fuse the auxiliary image sources based on the main image source to obtain the fused image sequence for each sub-region.

5. The method according to claim 1, characterized in that Perform image fusion on the screened image sources for real-time monitoring of the current sub-region with the corresponding region adaptive fusion strategy. If the region adaptive fusion strategy corresponding to the screened image source is the confidence weighted fusion, it includes: Calculate the confidence index of the screened image sources, and perform confidence weighted fusion on the screened image sources according to the confidence index to obtain the fused image sequence for each sub-region.

6. The method according to claim 1, wherein Construct a three-dimensional regional grid model of the coal mining face, including: Obtain the three-dimensional point cloud data of the coal mining face, and determine the three-dimensional regional model according to the three-dimensional point cloud data; Obtain the equipment distribution information of the coal mining face, perform grid scale fitness analysis according to the equipment distribution information, and determine the grid size step of the three-dimensional space region; Perform grid processing on the three-dimensional regional model based on the grid size step to obtain the three-dimensional regional grid model.

7. A dynamic monitoring image fusion system in a coal mining face, characterized in that A system for implementing the dynamic monitoring image fusion method in a coal mining face according to any one of claims 1-6, the system includes: A model construction unit for constructing a three-dimensional regional grid model of the coal mining face; A grid clustering unit for identifying the spatial distribution density of each grid unit in the three-dimensional regional grid model according to the historical collected images, clustering each grid unit according to the spatial distribution density, and outputting multiple sub-regions, where each sub-region corresponds to a spatial distribution density level; An image screening unit for linking the multiple sub-regions to a multi-source dynamic monitoring module to screen image sources according to a quality scoring mechanism, and outputting the screened image sources corresponding to each sub-region; An image fusion unit for, for the screened image sources corresponding to each sub-region, obtaining the corresponding region adaptive fusion strategy, and performing image fusion on the real-time monitoring images of the current sub-region with the corresponding region adaptive fusion strategy to obtain the fused image sequence for each sub-region.

Images