A coal bunker blocking monitoring and early warning method based on instance segmentation

Abstract

This invention relates to a coal bunker blockage monitoring and early warning method based on instance segmentation, belonging to the field of coal bunker safety monitoring technology. It solves the problems of existing technologies such as delayed detection, low accuracy, and inability to provide early warnings. The method includes: first, collecting multi-condition monitoring images of the upper and lower entrances of the coal bunker and labeling them to construct a training set; then, improving the YOLOv12-seg algorithm to construct a coal block instance segmentation model, which is optimized through training with a multi-task joint loss function; inputting real-time monitoring images into the model, combining the detection results with pixel-level segmentation masks to quantitatively identify large-volume coal blocks; constructing a first-in-first-out (FIFO) queue of large-volume coal blocks to simulate the dynamic flow of coal blocks and statistically analyze the amount of coal in the bunker; finally, implementing a three-level graded early warning system based on the queue's inventory, configuring gradient-based audible and visual alarms, multi-terminal push notifications, and emergency response logic. This method achieves high-precision detection and segmentation of coal blocks and full-process tracking of large-volume coal blocks, enabling early warning of blockage risks, improving the real-time performance and accuracy of monitoring and early warning, and adapting to the needs of intelligent coal mine production.

Description

Technical Field

[0001] This invention relates to the field of coal bunker safety monitoring technology, and in particular to a method for monitoring and early warning of coal bunker blockage based on instance segmentation. Background Technology

[0002] In the coal production, processing, and storage processes, coal bunkers are crucial core equipment for ensuring continuous production. Their core function is to temporarily store coal and enable continuous feeding. During actual operation, coal bunkers are highly susceptible to blockages due to factors such as uneven coal particle size, moisture fluctuations, and impact compression during feeding. Blockages not only cause production system interruptions and reduced efficiency but can also lead to sudden collapse of blocked material, causing impact loads that damage the bunker structure and even trigger safety accidents. Therefore, accurate monitoring and early warning systems for coal bunker blockages are urgently needed across the industry.

[0003] Currently, the mainstream methods for detecting coal bunker blockages in the industry mainly fall into two categories: manual inspection and contact sensor detection. Manual inspection relies on regular on-site patrols by staff, which suffers from problems such as detection lag, strong subjectivity, high labor intensity, and high on-site operation risks, making it unsuitable for the continuous and unmanned production needs of coal mines. Contact detection methods, such as pressure and vibration sensors, are susceptible to coal impact, abrasion, and corrosion from high-dust environments, resulting in a continuous decrease in detection accuracy over time. Furthermore, sensor installation and maintenance require production shutdowns, leading to high costs. More importantly, these methods can only identify abnormal parameters after a blockage has occurred, providing only post-event alarms and failing to predict and warn of potential blockage risks in advance.

[0004] With the development of computer vision technology, non-contact detection methods based on image recognition have gradually attracted attention. Instance segmentation algorithms can accurately identify targets and complete pixel-level contour segmentation, providing technical support for coal block identification in coal bunkers. However, existing solutions mostly focus on monitoring a single location in the coal bunker, still limited to post-event detection, and lack a full-process coal block tracking mechanism from the top of the bunker to the bottom, making it impossible to quantify and predict the risk of blockage. The timeliness and accuracy of early warnings are insufficient. Currently, there is no mature solution that combines instance segmentation with coal block queue management to achieve full-process tracking and early warning of blockages. Summary of the Invention

[0005] Based on the above analysis, the embodiments of the present invention aim to provide a coal bunker blockage monitoring and early warning method based on instance segmentation, in order to solve the problems of detection lag, low identification accuracy and lack of early warning in the existing technology.

[0006] This invention provides a method for monitoring and early warning of coal bunker blockage based on instance segmentation, including the following steps: acquiring monitoring images of the upper and lower openings of the coal bunker under different conditions, labeling coal blocks, and constructing a training set; A coal block instance segmentation model was constructed based on YOLOv12-seg. The coal block instance segmentation model was trained based on the training set and the model parameters were optimized through the loss function to obtain the trained coal block instance segmentation model. The real-time acquired monitoring images of the upper and lower openings of the coal bunker are input into the trained coal block instance segmentation model, and the coal block recognition result and the pixel-level segmentation mask of the coal block are output. Based on the coal block recognition result and the pixel-level segmentation mask of the coal block, large-volume coal blocks in the image are identified. A large-volume coal block queue is constructed. When the coal block instance segmentation model identifies m large-volume coal blocks in the monitoring image at the top of the coal bunker, the information of the m large-volume coal blocks is stored sequentially at the tail of the large-volume coal block queue. When the coal block instance segmentation model identifies n large-volume coal blocks in the monitoring image at the bottom of the coal bunker, the information of n large-volume coal blocks is taken out from the head of the large-volume coal block queue to obtain the number of large-volume coal blocks stored in the large-volume coal block queue in real time. Early warning is issued based on the number of large-volume coal blocks stored in the large-volume coal block queue in real time.

[0007] Based on further improvements to the above method, the coal block instance segmentation model is derived from YOLOv12-seg and consists of a backbone network, a Neck layer, a detection head, and a segmentation head. The preprocessed monitoring images of the upper and lower openings of the coal bunker are input into the backbone network. The output of the backbone network is connected to the input of the Neck layer, and the output of the Neck layer is connected to the input of the detection head and the input of the segmentation head, respectively. The modules sequentially complete the unidirectional transfer of feature tensors, and the detection head and the segmentation head perform inference operations in parallel based on the feature tensors output by the Neck layer.

[0008] Based on further improvements to the above method, the backbone network of the model consists of a 3×3 convolutional layer and a C2f-ELAN module connected in sequence. The 3×3 convolutional layer receives pre-processed monitoring images of the upper and lower openings of the coal bunker. After downsampling and channel upscaling of the pre-processed monitoring images of the upper and lower openings of the coal bunker, a shallow feature tensor is obtained, which is then transmitted to the C2f-ELAN module. The C2f-ELAN module performs branch splitting, residual connection, and feature splicing on the shallow feature tensor to achieve multi-scale feature aggregation, obtain mid-level feature tensors of different levels, and output them to the Neck layer.

[0009] Based on the further improvement of the above method, the Neck layer of the model is a PAN-FPN structure that integrates a CBAM-Lite lightweight hybrid attention module. The Neck layer first performs top-down feature upsampling and bottom-up feature downsampling operations on the mid-level feature tensor output by the backbone network through the PAN-FPN structure to complete the fusion of deep semantic features and shallow detail features across layers. Then, the CBAM-Lite lightweight hybrid attention module performs channel and spatial attention weighting processing on the fused feature tensor to enhance the coal block region features and suppress background interference, and finally outputs a multi-scale fused feature tensor. The detection head is a decoupled structure and integrates an SPPF spatial pyramid pooling module. It receives the multi-scale fused feature tensor and performs pooling processing through multiple convolutions and the SPPF spatial pyramid pooling module to output the coal block recognition result. The segmentation head receives the multi-scale fused feature tensor, improves the feature resolution through upsampling, and outputs the pixel-level segmentation mask corresponding to each coal block through the Sigmoid activation function.

[0010] Based on further improvements to the above method, the identification results include: the bounding box of the coal block and the binary classification results and confidence scores of the coal block and non-coal block; based on the bounding box of the coal block output by the detection head, the binary classification results and confidence scores of the coal block and non-coal block, the coal block targets in the image are screened out, and then according to the pixel-level segmentation mask corresponding to each coal block target output by the segmentation head, the number of pixels in the region corresponding to each coal block target in the image is counted; coal block targets with a number of pixels greater than a threshold are identified as large-volume coal blocks.

[0011] Based on a further improvement to the above method, the loss function is a total loss comprising detection loss, classification loss, segmentation loss, and attention guidance loss, as shown in the following formula.

[0012] in, For the total loss, To detect the loss, For classifying losses, To divide the loss, To guide attention loss, This is the loss weight.

[0013] Based on a further improvement of the above method, the detection loss formula is as follows:

[0014] in, To detect the loss, IoU is the intersection-union ratio between the model's predicted bounding box and the ground truth bounding box. To predict the distance between the center point of the bounding box and the ground truth bounding box for the model, b is the diagonal length of the bounding box, and b is the coordinate of the center point of the bounding box predicted by the model. gt These are the coordinates of the center point of the true bounding box learned by the model. The aspect ratio difference between the model's predicted bounding box and the ground truth bounding box is represented by α, which is a weighting coefficient.

[0015] Based on a further improvement of the above method, the classification loss formula is as follows:

[0016] in, For classification loss, Pt is the model's predicted probability for the target class, and a t γ is the balancing factor, where t represents the true target category and γ is the adjustment coefficient for easy and difficult samples.

[0017] Based on a further improvement of the above method, the segmentation loss formula is as follows:

[0018] in, To divide the loss, A pixel-level mask for real coal blocks. To predict the pixel-level mask for coal blocks.

[0019] Based on a further improvement of the above method, the attention-guided loss formula is as follows:

[0020] in, The loss is the attention-guided loss, where N is the number of samples. For CBAM-Lite attention maps, This is a true saliency diagram of the coal block.

[0021] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects: 1. By customizing and improving the YOLOv12-seg algorithm for coal bunker scenarios, a coal block instance segmentation model was designed, consisting of a backbone network composed of 3×3 convolutional layers and C2f-ELAN modules, a Neck layer with PAN-FPN fused with CBAM-Lite attention enhancement, a decoupled detection head with built-in SPPF, and a pixel-level segmentation head. A multi-task joint loss function including detection, classification, segmentation, and attention guidance was introduced to optimize the model parameters. At the same time, standardized preprocessing such as color gamut correction and size scaling was performed on the monitoring images for the high dust environment of coal mines. This achieved high-precision real-time detection and pixel-level segmentation of coal blocks in coal bunkers, effectively suppressing background interference such as bunker walls and dust. It solved the problems of poor robustness of general instance segmentation models in coal bunker scene feature extraction and low positioning accuracy of large coal blocks. Moreover, the non-contact image detection method avoids the disadvantages of contact sensors such as easy wear, accuracy decrease with use, and high installation and maintenance costs. 2. By statistically analyzing the number of independent pixels in coal blocks and quantitatively identifying large-volume coal blocks based on the pixel-level segmentation mask output by the segmentation head, and combining real-time image monitoring of the upper inlet and lower outlet areas of the coal bunker, a first-in-first-out (FIFO) large-volume coal block queue is constructed to simulate the actual flow process of coal blocks. This achieves dynamic and accurate statistics of the inventory of large-volume coal blocks in the bunker, solving the problems of existing technologies that only monitor a single location in the coal bunker, cannot establish a full-process tracking mechanism for coal blocks, and are difficult to quantitatively predict the risk of blockage. This avoids the lag in blockage detection from the source. 3. By setting a three-level gradient early warning threshold based on the real-time inventory of large-volume coal block queues, an audible and visual alarm with progressively increasing intensity and a multi-terminal early warning information push mechanism are designed. Automatic and manual cancellation logic is configured for different early warning levels. At the same time, in the event of an emergency severe early warning, the coal bunker conveyor belt control system is linked to achieve emergency shutdown. This realizes advanced graded early warning and emergency linkage response to the risk of coal bunker blockage. It solves the problems of strong subjectivity and high operational risk of traditional manual inspection, and the problem that contact sensors can only provide post-event alarms after the blockage failure has occurred. This minimizes the probability of production interruption and safety accidents caused by blockage. 4. By optimizing the model structure to achieve a lightweight design, a frame-slicing frequency of 10-30 frames / second is adopted to balance the real-time detection and computing efficiency. The entire monitoring and early warning method is automatically executed by the data processing terminal. Early warning information can be simultaneously pushed to the production monitoring platform, staff mobile terminals and production scheduling center, realizing the automation and intelligence of coal bunker blockage monitoring and early warning, adapting to the continuous and unmanned production development needs of coal mines, and significantly reducing the labor intensity and on-site operation risks of manual inspection.

[0022] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0023] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0024] Figure 1 This is a flowchart of the coal bunker blockage monitoring and early warning method based on instance segmentation in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the coal block instance segmentation model structure in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the C2f-ELAN structure of the coal block instance segmentation model in Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the PAN-FPN structure of the coal block instance segmentation model in Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the CBAM-Lite lightweight hybrid attention module structure of the coal block instance segmentation model in Embodiment 1 of the present invention; Figure 6 This is a schematic diagram of the SPPF structure of the coal block instance segmentation model in Embodiment 1 of the present invention. Detailed Implementation

[0025] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0026] Example 1, A specific embodiment of the present invention discloses a method for monitoring and early warning of coal bunker blockage based on instance segmentation, such as... Figure 1 As shown, it includes the following steps: S1: Obtain monitoring images of the upper and lower openings of the coal bunker under different conditions, label the coal blocks, and construct a training set.

[0027] It should be noted that high-definition industrial cameras are installed at the upper inlet and lower outlet of the coal bunker, respectively. The camera's shooting angle, focal length, and exposure parameters are adjusted to ensure that the camera's field of view can completely cover the entire upper inlet and lower outlet areas of the coal bunker, ensuring that the captured images can clearly show the shape, stacking status, and other characteristics of the coal blocks. The high-definition industrial cameras are used to collect real-time video streams of the upper inlet and lower outlet processes of the coal bunker, and the collected upper and lower outlet video streams are synchronously transmitted to the data processing terminal.

[0028] The data processing terminal performs independent frame-slicing processing on the received video streams from the upper and lower ends of the coal bunker. By setting a frame-slicing frequency, the continuous video stream is converted into discrete static image frames. The frame-slicing frequency can be flexibly adjusted according to the actual detection needs at the coal bunker site, with a preferred frequency of 10-30 frames per second. This ensures real-time coal block detection while also considering the computational efficiency of the data processing terminal, avoiding computational overload due to excessively high frame rates or missed coal block features due to excessively low frame rates. Through the above frame-slicing processing, images of coal blocks at the upper and lower ends of the coal bunker are obtained, covering different lighting conditions, different coal block particle size distributions, and different coal block stacking states. This covers typical working conditions at the coal bunker site, ensuring the diversity of working conditions in the training set.

[0029] All coal block images at the top and bottom of the coal bunker obtained from frame segmentation were manually labeled. The labeling process followed the labeling specifications of computer vision instance segmentation models. For coal block targets in the images, the bounding box position information and pixel-level segmentation mask information of the coal blocks were labeled to complete the accurate labeling of the coal block targets. All coal block images with complete labeling information and comprehensive working conditions were organized to construct the training set of the coal block instance segmentation model, providing data support for subsequent model training, parameter optimization and generalization ability improvement.

[0030] S2: Construct a coal block instance segmentation model based on YOLOv12-seg, train the coal block instance segmentation model based on the training set, and optimize the model parameters through the loss function to obtain the trained coal block instance segmentation model.

[0031] It should be noted that the well-known YOLOv12-seg model is a general-purpose instance segmentation architecture, consisting of a backbone network, a PAN-FPN structured Neck layer, a decoupled detection head, and a segmentation head. It is lightweight and has a fast inference speed, making it suitable for object detection and pixel-level segmentation tasks in general scenarios. However, when directly applied to coal block identification in coal mine bunker blockage monitoring, the native model exhibits significant adaptability defects due to the special conditions present in the coal bunker environment, such as high dust levels, low light, coal block accumulation and occlusion, and complex background textures on the bunker walls. It lacks feature enhancement and background interference suppression designs specifically for the coal bunker scenario, leading to difficulties in coal block feature extraction. The original solution lacked robustness, had low accuracy in feature capture and bounding box regression for large coal blocks, and was prone to misidentifying non-coal block areas such as silo walls and dust accumulations. It could not meet the practical application requirements for real-time and accurate coal block identification in coal silos. Therefore, this solution addresses the above deficiencies by customizing and improving the YOLOv12-seg algorithm for coal silo scenarios. By integrating a lightweight attention module, optimizing the feature fusion structure, and embedding a pooling module in the detection head, the solution enhances the feature extraction capability of coal block areas, suppresses on-site background interference, and improves the detection accuracy of multi-scale coal blocks. This allows the improved model to adapt to the complex working conditions of coal silos and meet the actual needs of coal block identification.

[0032] Specifically, the coal block instance segmentation model is an improvement based on YOLOv12-seg, such as... Figure 2 As shown, the system consists of a backbone network, a Neck layer, a detection head, and a segmentation head. The preprocessed monitoring images of the upper and lower openings of the coal bunker are input into the backbone network. The output of the backbone network is connected to the input of the Neck layer. The output of the Neck layer is connected to the input of the detection head and the input of the segmentation head, respectively. The modules sequentially complete the unidirectional transfer of feature tensors. The detection head and the segmentation head perform inference operations in parallel based on the feature tensors output by the Neck layer.

[0033] It should be noted that the samples need to be preprocessed before being input into the model. The preprocessing operations specifically include size scaling, normalization, color gamut correction, and batch normalization. First, the monitoring images of the upper and lower openings of the coal bunker are uniformly adjusted to a fixed size suitable for the model, which can be 640×640 or 1280×1280. Color gamut correction is used to correct the color cast caused by the high dust environment in the coal mine. Then, the image pixel values ​​are normalized and batch normalization is performed. Finally, the normalized feature tensor [B,C,H,W] is output, where B is the batch number, C is the number of image channels and is fixed at 3, and H and W are the height and width of the image, respectively. This preprocessing operation can effectively eliminate the interference of differences in size, brightness, and color gamut between the images collected on-site in the coal bunker, providing a unified input format for subsequent model feature extraction and adapting to the complex image acquisition conditions in the coal mine. Meanwhile, the sequential unidirectional transfer design of feature tensors between the modules of the model can ensure the orderly extraction and enhancement of coal block features from shallow to deep layers. The design of transmitting fused feature tensors to the detection head and the segmentation head at the same time, so that the detection head and the segmentation head can perform inference operations in parallel, can improve the overall inference efficiency of the model while ensuring the accuracy of coal block target detection and pixel-level segmentation, and adapt to the real-time requirements of coal bunker blockage monitoring.

[0034] Specifically, the backbone network of the model consists of sequentially connected 3×3 convolutional layers and C2f-ELAN modules. The 3×3 convolutional layers receive pre-processed monitoring images of the upper and lower openings of the coal bunker. After downsampling and channel upscaling of the pre-processed images, shallow feature tensors are obtained. These shallow feature tensors are then transmitted to the C2f-ELAN module, whose structure is as follows: Figure 3 The C2f-ELAN module performs branch splitting, residual connection and feature splicing on shallow feature tensors to achieve multi-scale feature aggregation, obtain mid-level feature tensors of different levels and output them to the Neck layer.

[0035] It should be noted that the backbone network adopts a design of 3×3 convolutional layers and C2f-ELAN modules cascaded sequentially in a unidirectional manner. This design retains the advantages of the original YOLOv12-seg architecture, such as lightweight design and high inference efficiency, while optimizing the refined processing of feature extraction for detection conditions involving high dust levels, irregular coal block shapes, and large size differences. This enables the orderly extraction and aggregation of coal block features from shallow basic features to mid-level semantic features, and the feature processing operations performed by each module are adapted to the actual needs of coal mine bunker detection. The three convolutional layers perform a 3×3 convolution operation with a stride of 2 on the preprocessed feature tensor [B,C,H,W]. Combined with BatchNorm batch normalization and SiLU activation function, downsampling and channel upscaling are completed to obtain a shallow feature tensor [B,C1,H1,W1], where H1=H / 2, W1=W / 2, and C1 is the number of channels after upscaling. C1 can be 64 or 128. While compressing the spatial dimension of the image and reducing the amount of computation, the layers effectively extract shallow basic texture features such as the edges and contours of the coal block.

[0036] The C2f-ELAN module first performs channel compression on shallow feature tensors using 1×1 convolutions (CBS, S=1, K=1) to reduce computation. Here, CBS is a fixed abbreviation for Conv convolution, referring to the standard convolutional structure of Conv convolution + BatchNorm batch normalization + SiLU activation function. K=1 indicates a 1×1 kernel size, and S=1 indicates a stride of 1. The kernel slides across the feature map with a stride of 1, without changing the spatial dimensions of the feature map. Next, a Split operation splits the features into multiple branches. Some branches are used for deep feature extraction via 3×3 convolutions (CBS, S=1, K=3) and a BottleNeck layer, while another branch retains the original features and performs residual connections. Finally, Concat is used to fuse all branch features, and a 1×1 convolution is used to adjust the number of channels. This processing method achieves multi-scale feature aggregation and capture... While capturing the features of coal blocks of different sizes and stacking morphologies, the model avoids the gradient vanishing problem during deep training by using residual connections. In the process, multi-scale mid-level feature tensors [B,C2,H1,W1] are naturally generated, where C2>C1. The size of the mid-level feature tensors remains unchanged compared to the shallow feature tensors, but the feature representation ability is enhanced. The generated mid-level feature tensors of 76×76, 38×38, and 19×19 are adapted to the feature extraction needs of small, medium, and large coal blocks in the coal bunker, respectively, providing a multi-scale feature foundation for cross-level feature fusion in the subsequent Neck layer.

[0037] Specifically, the Neck layer of the model is a PAN-FPN structure that incorporates a CBAM-Lite lightweight hybrid attention module (Convolutional Block Attention Module Lite), as shown in the PAN-FPN structure. Figure 4 CBAM-Lite lightweight hybrid attention module, such as Figure 5 The Neck layer first performs top-down and bottom-up feature upsampling on the mid-level feature tensor output by the backbone network using a PAN-FPN structure, completing the fusion of deep semantic features and shallow detail features across layers. Then, the CBAM-Lite lightweight hybrid attention module performs channel and spatial attention weighting on the fused feature tensor to enhance coal block region features and suppress background interference, finally outputting a multi-scale fused feature tensor. The detection head is a decoupled structure and integrates an SPPF spatial pyramid pooling module, such as... Figure 6 The segmentation head receives the multi-scale fusion feature tensor, performs pooling processing through multiple layers of 3×3+1×1 convolution and SPPF spatial pyramid pooling module, and outputs the identification result of coal blocks; the segmentation head receives the multi-scale fusion feature tensor, improves the feature resolution through upsampling operation, and outputs the pixel-level segmentation mask corresponding to each coal block through Sigmoid activation function.

[0038] It should be noted that the PAN-FPN structure achieves cross-level interaction of features at different levels through bidirectional operations of top-down upsampling and bottom-up downsampling, fusing the deep semantic information of large-scale features with the shallow detail information of small-scale features to compensate for the information loss of single-scale features. The CBAM-Lite module is a lightweighted hybrid attention module. It first performs channel attention processing on the feature tensor after PAN-FPN fusion, obtains feature information through global average pooling and global max pooling, generates channel attention weights through a shared MLP multilayer perceptron and sigmoid activation, and then performs spatial attention processing. After performing average pooling and max pooling along the channel dimension and concatenating the results, spatial attention weights are generated through 7×7 convolution and Sigmoid activation. This bidirectional attention weighting achieves precise enhancement of coal block region features while suppressing feature interference from irrelevant backgrounds such as bin wall textures and dust accumulation. Furthermore, the lightweight design of this module does not increase the computational power consumption of the model. The multi-scale fusion feature tensor after fusion and enhancement takes into account both the detailed and semantic features of the coal block, adapting to the coal block target detection requirements of different scales in the coal bin. This provides a high-quality feature foundation for subsequent inference by the detection head and segmentation head, ensuring the model's ability to capture features of both small and large coal blocks.

[0039] The Neck layer performs top-down and bottom-up feature upsampling operations via the PAN-FPN structure. This cross-layer fusion of feature tensors at three scales (76×76, 38×38, and 19×19) transfers deep semantic information from large-scale features to small-scale features, while transferring shallow details from small-scale features to large-scale features. The final output is a multi-scale fused feature tensor of 76×76, 38×38, and 19×19, consistent with the output scale of the backbone network. This ensures that the fused features at each scale contain both the localization details and category semantic information of the coal block target, providing a high-quality feature foundation for the parallel inference of the subsequent detection and segmentation heads.

[0040] The decoupled structure of the detection head separates bounding box regression and category classification into two independent convolutional branches. The SPPF spatial pyramid pooling module is integrated after the regression branch, performing multi-scale max pooling and feature concatenation on the features after multi-layer 3×3+1×1 convolution processing to expand the feature receptive field and optimize the regression accuracy of coal block bounding boxes. It is especially suitable for the bounding box localization requirements in scenarios with large coal block accumulation and occlusion. Finally, it synchronously outputs the coordinates of the coal block bounding box, the binary classification results of coal block / non-coal block, and the confidence values ​​corresponding to each result. The segmentation head uses transposed convolution combined with upsampling operation to gradually restore the resolution of the multi-scale fused feature tensor to the same size as the preprocessed coal bunker monitoring image. Then, the probability value of the pixel belonging to the coal block is mapped to the 0~1 interval by the Sigmoid activation function, and outputs an independent pixel-level segmentation mask for each coal block. This provides an accurate pixel range basis for subsequent statistics on the number of pixels in the coal block area and the determination of large coal blocks.

[0041] Specifically, the loss function is the total loss that includes detection loss, classification loss, segmentation loss, and attention guidance loss, as shown in the following formula.

[0042] in, For the total loss, To detect the loss, For classifying losses, To divide the loss, To guide attention loss, This is the loss weight.

[0043] It should be noted that the total loss is the multi-task joint loss function of the coal block instance segmentation model. By integrating the losses of four sub-tasks—detection and localization, category classification, pixel segmentation, and attention guidance—the model parameters are synergistically optimized. These represent the loss weights for detection loss, classification loss, segmentation loss, and attention-guided loss, respectively. Each loss weight coefficient is used to adjust the proportion of loss contribution for different sub-tasks, ensuring that the model simultaneously considers the accuracy of coal block bounding box regression, binary classification accuracy, pixel-level segmentation integrity, and the effectiveness of attention feature focusing during training. This adapts to the multi-task requirements of accurate coal block identification and segmentation under complex working conditions in coal bunkers. All are positive numbers and satisfy the following conditions: In this invention, coal bunker blockage monitoring mainly relies on the accuracy of coal block bounding box positioning and pixel-level segmentation. The weights of detection loss and segmentation loss should be higher than those of classification loss and attention-guided loss. The weight relationship is set as follows: The specific value can be fine-tuned based on the convergence of the validation set loss, provided that the above conditions are met. For example, Take 0.35, Take 0.20, Take 0.35, Take 0.10.

[0044] Specifically, the detection loss formula is as follows:

[0045] in, To detect the loss, IoU is the intersection-union ratio between the model's predicted bounding box and the ground truth bounding box. To predict the distance between the center point of the bounding box and the ground truth bounding box for the model, b is the diagonal length of the bounding box, and b is the coordinate of the center point of the bounding box predicted by the model. gt These are the coordinates of the center point of the true bounding box learned by the model. The aspect ratio difference between the model's predicted bounding box and the ground truth bounding box is represented by α, which is a weighting coefficient.

[0046] It should be noted that the detection loss is the CIoU loss function, and the bounding box refers to the smallest bounding rectangle between the predicted bounding box and the true bounding box. It is a dimensionless coefficient specifically used to measure the difference in aspect ratio between the predicted bounding box and the actual bounding box. =4 / π²×(arctan( / )-arctan(w / h))², where w and h are the width and height of the bounding box predicted by the model, respectively. , These are the width and height of the actual bounding box, respectively. The core function is to allow the loss function to directly perceive the shape matching degree between two boxes, solving the problem that traditional IoU only focuses on the overlapping area and ignores the difference in box shape; the weight coefficient α is based on the formula α= / (1-IoU+ (Settings to adjust aspect ratio differences) The impact of detection loss; this detection loss takes into account the consistency of crossover ratio, center point distance, and aspect ratio, which can significantly improve the bounding box regression accuracy of irregularly shaped and piled-up coal blocks in coal bunkers.

[0047] Specifically, the classification loss formula is as follows:

[0048] in, For classification loss, Pt is the model's predicted probability for the target class, and a t γ is the balancing factor, where t represents the true target category and γ is the adjustment coefficient for easy and difficult samples.

[0049] It should be noted that the classification loss is Focal Loss, specifically designed to address the problem of imbalanced positive and negative samples between coal block targets and background areas, and the large difference between easily classified and difficult samples in coal bunker monitoring images; the balance factor α t The subscript 't' represents the true target class of the sample. 't=1' corresponds to the positive class of coal blocks, and 't=0' corresponds to the negative class of background samples, satisfying α1+α0=1. This factor is primarily used to adjust the weight of the loss impact of positive and negative samples, assigning a larger α value to the class with fewer samples. t Based on the principle of adapting values, and considering the characteristic that the proportion of positive coal block samples in the coal bunker scenario is much lower than that of negative background samples such as bunker walls and dust, the easy and difficult sample adjustment coefficient γ is used to reduce the loss weight of simple background samples and strengthen the loss weight of difficult-to-distinguish coal block samples. For difficult-to-distinguish samples such as dust and bunker wall textures that are similar to coal blocks in the coal bunker, the values ​​are adapted based on the known Focal Loss value specification. For example, γ is set to 2. The appropriate selection of γ can effectively suppress the misclassification caused by background such as bunker walls and dust, and improve the binary classification accuracy of coal blocks and non-coal blocks.

[0050] Specifically, the segmentation loss formula is as follows:

[0051] in, To divide the loss, A pixel-level mask for real coal blocks. To predict the pixel-level mask for coal blocks.

[0052] It should be noted that the segmentation loss is the Dice Loss function, which is used to measure the pixel overlap between the predicted coal block segmentation mask and the real coal block segmentation mask. The smaller the value, the higher the overlap and the better the segmentation effect. This loss function can effectively solve the problems of uneven pixel proportion in coal block areas in coal bunker images and the easy omission of segmentation of small-volume loose coal, ensuring the integrity and accuracy of the model's pixel-level segmentation of coal blocks of different scales.

[0053] Specifically, the formula for attention-guided loss is as follows:

[0054] in, The loss is the attention-guided loss, where N is the number of samples. For CBAM-Lite attention maps, This is a true saliency diagram of the coal block.

[0055] It should be noted that the attention guidance loss uses the L2 norm to calculate the error between the attention map generated by the CBAM-Lite module and the coal block true saliency map. By constraining the minimization of error, the CBAM-Lite lightweight hybrid attention module of the model is guided to accurately focus on the coal block target area and suppress background interference. The coal block true saliency map is obtained by converting pixel-level labeled image segmentation labels into code to generate a binary grayscale image with the same size as the attention map. The attention map is the feature map generated after the image is processed by the CBAM-Lite module in the Neck layer during model forward inference. The weight parameters of the CBAM-Lite module are automatically learned and optimized by the model during training based on gradient descent and backpropagation algorithms.

[0056] S3: Input the real-time acquired monitoring images of the upper and lower openings of the coal bunker into the trained coal block instance segmentation model, and output the coal block recognition result and the pixel-level segmentation mask of the coal block. Based on the coal block recognition result and the pixel-level segmentation mask of the coal block, identify large-volume coal blocks in the image.

[0057] Specifically, the identification results include: the bounding box of the coal block and the binary classification results and confidence scores of the coal block and non-coal block; based on the bounding box of the coal block output by the detection head, the binary classification results of the coal block and non-coal block and the confidence scores, the coal block targets in the image are screened out, and then according to the pixel-level segmentation mask corresponding to each coal block target output by the segmentation head, the number of pixels in the region corresponding to each coal block target in the image is counted; coal block targets with a number of pixels greater than a threshold are identified as large-volume coal blocks.

[0058] It should be noted that the confidence screening operation, by setting a preset confidence threshold for coal block targets, eliminates low-confidence and non-coal block interference targets predicted by the model, avoiding misidentification caused by background areas such as bin walls, dust, and equipment, and ensuring the accuracy of coal block target screening; the number of pixels in the region is the total number of effective pixels independently occupied by a single coal block target based on pixel-level segmentation mask statistics, accurately representing the actual area and volume of the coal block in the monitoring image, effectively avoiding the problem of volume misjudgment caused by coal block stacking and overlapping; the pixel number threshold is based on the actual bin blockage judgment criteria, monitoring image resolution (640×640 / 1280×1280), and camera capture. The distance is pre-calibrated and can be adaptively adjusted according to changes in coal bunker conditions. For example, the pixel number threshold is 50,000 pixels. Coal blocks with a pixel number greater than 50,000 pixels are identified as large coal blocks and are subject to subsequent queuing operations. Coal blocks with a pixel number less than or equal to 50,000 pixels are identified as non-large coal blocks and are not subject to subsequent queuing operations. By determining the number of pixels to identify large coal blocks, the volume of coal blocks can be quantitatively identified, accurately capturing large coal blocks that are prone to causing coal bunker blockage. This provides core quantitative basis for subsequent coal bunker blockage risk warning and is also compatible with the coal block volume identification needs of both the upper inlet and lower outlet areas of the coal bunker.

[0059] S4: Construct a large-volume coal block queue. When the coal block instance segmentation model identifies m large-volume coal blocks in the monitoring image at the top of the coal bunker, the information of the m large-volume coal blocks is stored sequentially at the tail of the large-volume coal block queue. When the coal block instance segmentation model identifies n large-volume coal blocks in the monitoring image at the bottom of the coal bunker, the information of the n large-volume coal blocks is taken out from the head of the large-volume coal block queue to obtain the number of large-volume coal blocks stored in the large-volume coal block queue in real time.

[0060] It should be noted that the large-volume coal block queue adopts a first-in, first-out (FIFO) storage mechanism. Its core design purpose is to simulate the actual flow process of coal blocks from the top of the coal bunker to the bottom, so as to realize the dynamic statistics of the real-time inventory of large-volume coal blocks in the bunker and provide accurate inventory data support for early warning of coal bunker blockage. Here, m and n are both non-negative integers, which correspond to the number of large-volume coal blocks identified in real time at the top of the coal bunker and the number of large-volume coal blocks identified in real time at the bottom of the coal bunker, respectively. The two are linked synchronously and dynamically update the queue status.

[0061] The information on large coal blocks stored at the tail of the queue must include at least the bounding box coordinates, pixel-level segmentation mask, identification timestamp, and number of pixels to ensure that each large coal block can be uniquely identified and to avoid confusion between different coal block information. The operation of retrieving information on n large coal blocks from the head of the queue strictly follows the actual coal flow logic of first-in, first-out, and is consistent with the flow order of coal blocks in the warehouse to ensure the accuracy of inventory statistics.

[0062] When the number of n large-volume coal blocks identified at the bottom of the coal bunker exceeds the number of large-volume coal blocks currently stored in the queue, all existing large-volume coal block information in the queue is retrieved. At this time, the real-time storage quantity in the queue is 0 to avoid the logical loophole of negative storage. The real-time storage quantity of large-volume coal blocks in the queue is dynamically calculated using the formula "current queue quantity = previous round quantity + mn", which reflects the accumulation of large-volume coal blocks in the coal bunker in real time.

[0063] This queue design can effectively solve the problem of difficulty in real-time tracking of large coal blocks in the coal bunker. Through the dynamic entry and exit linkage mechanism, it can accurately match the actual working conditions of material entering from the top and material exiting from the bottom. At the same time, it provides direct quantitative basis for subsequent judgment on whether large coal blocks are accumulating in the bunker or whether the blockage threshold has been reached, thus meeting the actual needs of continuous monitoring and dynamic early warning of the coal bunker.

[0064] S5: Issue an early warning based on the number of large-volume coal blocks stored in the large-volume coal block queue in real time.

[0065] It should be noted that the warning is divided into three levels, which increase in urgency: general warning, general severe warning, and emergency severe warning. When the number of large coal blocks stored in the large coal block queue reaches the first warning threshold but is less than the second warning threshold, the general warning mechanism is triggered: the data processing terminal controls the audible and visual alarm device to emit a yellow light prompt and an intermittent buzzer signal, and at the same time pushes the general warning information, including the warning time, the number of large coal blocks stored in the large coal block queue, and the monitoring and capture images of the large coal blocks entering the queue, to the production monitoring platform to remind the staff to pay close attention to the coal bunker's operating status and make preparations for unblocking. When the number of large coal blocks stored in the large coal block queue reaches the second warning threshold but is less than the third warning threshold, the general severe warning mechanism is triggered: the data processing terminal controls the audible and visual alarm device to switch to flashing red lights and continuous buzzer alarm, and at the same time pushes the general severe warning information, namely the warning time, the number of large coal blocks stored in the large coal block queue, and the warning level, to the production monitoring platform and the mobile terminals of relevant personnel, clearly indicating that dredging measures need to be taken immediately; When the number of large coal blocks stored in the large coal block queue reaches the third warning threshold, the emergency severe warning mechanism is triggered: the data processing terminal immediately controls the audible and visual alarm device to emit a red high-frequency flashing light and a high-decibel continuous buzzing alarm, and at the same time executes linkage control commands, including sending an emergency stop signal to the coal mine coal bunker conveyor belt control system, and simultaneously pushing emergency severe warning information, including the warning time, the number of large coal blocks stored in the large coal block queue, real-time upper and lower monitoring images, and the belt stop status, to the production monitoring platform, staff mobile terminals, and the coal mine production dispatch center, requiring the dispatch center to immediately activate the emergency response plan and organize professional personnel to rush to the scene to carry out emergency dredging operations.

[0066] The deactivation logic of the three-level early warning mechanism is as follows: when the number of large coal blocks in the queue drops below the first early warning threshold, the general early warning signal is automatically deactivated; when a general severe early warning is triggered, staff must complete the clearing operation and confirm safety before manually resetting and deactivating the general severe early warning signal, or wait until the number of coal blocks in the queue drops below the first early warning threshold and remains stable for 5 minutes before automatically deactivating the general severe early warning signal; when an emergency severe early warning is triggered, staff must complete the clearing operation, confirm the safe operation of the coal bunker and conveyor belt, and manually reset and deactivate the emergency severe early warning signal, while simultaneously sending a restart permission signal to the conveyor belt control system. Only after confirmation by the dispatch center can the conveyor belt operation be restored.

[0067] It should be noted that the first, second, and third warning thresholds are comprehensively determined based on the actual effective volume of the coal bunker, the coal flow rate at the upper and lower ends of the coal bunker, the historical pattern of bunker blockage, the average equivalent volume of a single large coal block, and the on-site safety production control standards of the coal mine. They can also be adaptively adjusted according to changes in the working conditions of the coal bunker and production scheduling requirements. For example, in this embodiment, the first warning threshold is 6, the second warning threshold is 10, and the third warning threshold is 16.

[0068] The three-level early warning mechanism can implement tiered management based on the degree of danger of the number of large coal blocks piled up in the coal bunker. The intensity of early warning and emergency response can be increased at each level. This avoids frequent alarms that disrupt normal production in low-risk situations, while also being able to quickly trigger emergency response measures when there is a high risk of accumulation, thus minimizing the probability of bunker blockage accidents.

[0069] The warning cancellation logic combines automatic cancellation with manual reset. For general warnings, a purely automatic cancellation method is used to improve the level of monitoring automation while ensuring safety. For general severe warnings and emergency severe warnings, manual reset or delayed automatic reset conditions are set. In particular, after the emergency severe warning is linked to the belt stop, it must be reset only after on-site safety confirmation. This can effectively prevent the warning signal from being mistakenly cancelled when the coal bunker is not fully cleared or the status has not returned to safety, thus taking into account both the automation of monitoring and the safety and standardization of coal mine production operations.

[0070] Those skilled in the art will understand that all or part of the processes implementing the methods of the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0071] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for monitoring and early warning of coal bunker blockage based on instance segmentation, characterized in that, Includes the following steps: Acquire monitoring images of the upper and lower openings of the coal bunker under different conditions, label the coal blocks, and construct a training set; A coal block instance segmentation model is constructed based on YOLOv12-seg. The coal block instance segmentation model consists of a backbone network, a Neck layer, a detection head, and a segmentation head. The CBAM-Lite lightweight hybrid attention module is integrated into the Neck layer. The coal block instance segmentation model is trained based on the training set and the model parameters are optimized by the loss function to obtain the trained coal block instance segmentation model. The real-time acquired monitoring images of the upper and lower openings of the coal bunker are input into the trained coal block instance segmentation model, and the coal block recognition result and the pixel-level segmentation mask of the coal block are output. Based on the coal block recognition result and the pixel-level segmentation mask of the coal block, large-volume coal blocks in the image are identified. A large-volume coal block queue is constructed. When the coal block instance segmentation model identifies m large-volume coal blocks in the monitoring image at the top of the coal bunker, the information of the m large-volume coal blocks is stored sequentially at the tail of the large-volume coal block queue. When the coal block instance segmentation model identifies n large-volume coal blocks in the monitoring image at the bottom of the coal bunker, the information of n large-volume coal blocks is taken out from the head of the large-volume coal block queue to obtain the number of large-volume coal blocks stored in the large-volume coal block queue in real time. Early warning is issued based on the number of large-volume coal blocks stored in the large-volume coal block queue in real time.

2. The method according to claim 1, characterized in that, The coal block instance segmentation model is an improvement based on YOLOv12-seg and consists of a backbone network, a Neck layer, a detection head, and a segmentation head. The preprocessed monitoring images of the upper and lower openings of the coal bunker are input into the backbone network. The output of the backbone network is connected to the input of the Neck layer. The output of the Neck layer is connected to the input of the detection head and the input of the segmentation head, respectively. The modules sequentially complete the unidirectional transfer of feature tensors. The detection head and the segmentation head perform inference operations in parallel based on the feature tensors output by the Neck layer.

3. The method according to claim 2, characterized in that, The backbone network of the model consists of a 3×3 convolutional layer and a C2f-ELAN module connected in sequence. The 3×3 convolutional layer receives pre-processed monitoring images of the upper and lower openings of the coal bunker. After downsampling and channel upscaling of the pre-processed monitoring images, a shallow feature tensor is obtained, which is then transmitted to the C2f-ELAN module. The C2f-ELAN module performs branch splitting, residual connection, and feature splicing on the shallow feature tensor to achieve multi-scale feature aggregation, obtain mid-level feature tensors of different levels, and output them to the Neck layer.

4. The method according to claim 2, characterized in that, The model's Neck layer is a PAN-FPN structure incorporating a CBAM-Lite lightweight hybrid attention module. The Neck layer first performs top-down and bottom-up feature upsampling on the mid-level feature tensors output from the backbone network using the PAN-FPN structure, fusing deep semantic features with shallow detail features across layers. Then, the CBAM-Lite lightweight hybrid attention module performs channel and spatial attention weighting on the fused feature tensors, enhancing coal block region features and suppressing background interference, ultimately outputting a multi-scale fused feature tensor. The detection head is a decoupled structure integrating an SPPF spatial pyramid pooling module. It receives the multi-scale fused feature tensor, performs pooling processing through multiple convolutions and the SPPF spatial pyramid pooling module, and outputs the coal block recognition result. The segmentation head receives the multi-scale fused feature tensor, improves feature resolution through upsampling, and outputs a pixel-level segmentation mask for each coal block using a Sigmoid activation function.

5. The method according to claim 1, characterized in that, The identification results include: the bounding box of the coal block and the binary classification results and confidence scores of coal blocks and non-coal blocks; based on the bounding box of the coal block output by the detection head, the binary classification results and confidence scores of coal blocks and non-coal blocks are used to filter out coal block targets in the image, and then based on the pixel-level segmentation mask corresponding to each coal block target output by the segmentation head, the number of pixels in the region corresponding to each coal block target in the image is counted; coal block targets with a number of pixels greater than a threshold are identified as large-volume coal blocks.

6. The method according to claim 1, characterized in that, The loss function is the total loss that includes detection loss, classification loss, segmentation loss, and attention guidance loss, and the formula is as follows: in, For the total loss, To detect the loss, For classifying losses, To divide the loss, To guide attention loss, This is the loss weight.

7. The method according to claim 6, characterized in that, The detection loss formula is as follows: in, To detect the loss, IoU is the intersection-union ratio between the model's predicted bounding box and the ground truth bounding box. To predict the distance between the center point of the bounding box and the ground truth bounding box for the model, b is the diagonal length of the bounding box, and b is the coordinate of the center point of the bounding box predicted by the model. gt These are the coordinates of the center point of the true bounding box learned by the model. The aspect ratio difference between the model's predicted bounding box and the ground truth bounding box is represented by α, which is a weighting coefficient.

8. The method according to claim 6, characterized in that, The classification loss formula is as follows: in, For classification loss, Pt is the model's predicted probability for the target class, and a t γ is the balancing factor, where t represents the true target category and γ is the adjustment coefficient for easy and difficult samples.

9. The method according to claim 6, characterized in that, The segmentation loss formula is as follows: in, To divide the loss, A pixel-level mask for real coal blocks. To predict the pixel-level mask for coal blocks.

10. The method according to claim 6, characterized in that, The formula for attention-guided loss is as follows: in, The loss is the attention-guided loss, where N is the number of samples. For CBAM-Lite attention maps, This is a true saliency diagram of the coal block.

Images