Method and system for monitoring abnormal state of coal bunker based on multivariate data fusion

Abstract

The invention discloses a multivariate data fusion coal bunker abnormal state monitoring method and system, and the method comprises the steps: collecting the continuous N-frame image information of a coal piling detection region in an image, obtaining the static pixel area of the coal piling region, calculating the ratio of the static pixel area of the coal piling region to the total area of the whole region, and calculating the area of the coal piling region; if the proportion in the continuous N frames of images exceeds a set threshold value, it is judged that coal piling abnormity occurs; image information of a coal falling opening in the image is collected, and whether coal blockage abnormity occurs or not is judged in combination with a coal level meter signal and the running state of the coal feeder; the method comprises the following steps: acquiring continuous N frames of image information of a lower opening and a coal falling opening of a coal bunker in an image, presetting a bunker collapse detection area in the image, calculating a chromatic aberration diffusion rate and a coal level descending rate in the bunker collapse detection area, and judging that bunker collapse is abnormal if the chromatic aberration diffusion rate and the coal level descending rate in the continuous N frames of image continuously exceed corresponding threshold values. Under the condition that existing equipment does not need to be transformed and replaced, recognition of various states of the coal bunker is achieved at low cost.

Description

Technical Field

[0001] This invention relates to the field of coal bunker detection and protection technology, and in particular to a method and system for monitoring abnormal states of coal bunkers using multi-source data fusion. Background Technology

[0002] In coal mining operations, underground coal bunkers are used to temporarily store coal transported from the working face and then transport it out of the mine or to other processing stages via a conveying system. However, coal caking due to excessive moisture, uneven particle size, and excessive coal discharge from the bunker's lower opening can easily lead to blockages and coal pile-ups. Furthermore, situations such as a sudden, large outflow of coal (i.e., bunker collapse) can occur when there is a large amount of water and coal inside the bunker, or when the bunker structure is damaged due to prolonged heavy loads, internal pressure changes, structural design flaws, or external geological activity. These issues not only affect coal transportation efficiency but can also cause serious safety hazards, such as personnel injuries, equipment damage, and mine shutdowns. Moreover, the complex underground environment, limited ventilation, confined space, and high humidity present numerous challenges to coal bunker management and maintenance.

[0003] With the increasing occurrence of various accidents in coal bunkers, coal mines are paying more and more attention to coal bunker monitoring. Simultaneously, with the continuous development of sensor technology and automated control systems, real-time monitoring of coal bunkers using advanced detection methods has become possible. For example, CN202010514214 uses a virtual line to determine whether coal accumulation has occurred; when the coal flow obscures the virtual line, a coal accumulation alarm is triggered. However, this method is easily affected by the scene and camera perspective, and cannot correctly distinguish between continuous coal flow and accumulated coal flow, thus easily leading to false alarms. CN118419520B uses grating projection technology to measure and process image information of coal accumulation to obtain image signal data, and analyzes the image signal data based on a target recognition model to determine whether coal accumulation has occurred. This method is difficult to operate and has a small monitoring area. CN117533807A proposes a method, device, and system for preventing coal bunker collapse, but it requires the installation of an impeller at the feeder belt, and the impeller must be in contact with the raw coal, placing high demands on installation. Current solutions can only monitor single types of anomalies or require modifications to existing equipment, making them unsuitable for comprehensive monitoring of coal bunkers in already producing coal mines.

[0004] Therefore, there is an urgent need for an intelligent monitoring system for underground coal bunkers. This system should be able to accurately detect various abnormal states within the coal bunker, such as coal accumulation, coal blockage, and bunker collapse. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies, which can only monitor single abnormal conditions or require modification of existing equipment, making them difficult to apply to comprehensive monitoring of coal bunkers in operating coal mines. This results in low monitoring efficiency and reduced safety of the coal bunkers. The invention provides a multi-data fusion method and system for monitoring abnormal states in coal bunkers. By setting up an image capture module to capture images of the coal bunker, a coal pile identification module determines coal pile abnormalities based on pre-processed images, a coal blockage identification module determines coal blockage abnormalities by combining pre-processed images, coal level signals, and the operating status of the coal feeder, and a bunker collapse identification module determines bunker collapse abnormalities by combining pre-processed images and coal level signals. Finally, an abnormality warning module generates corresponding alarms based on the abnormality type, achieving the goal of monitoring multiple abnormal states in the coal bunker, including coal pile-up, coal blockage, and bunker collapse, and alerting relevant personnel.

[0006] The objective of this invention is achieved through the following technical solution: A method for monitoring abnormal conditions in coal bunkers based on multi-source data fusion includes the following steps: The image capture device captures images of the coal bunker and preprocesses the image data; The system collects image information from N consecutive frames of the coal blockage detection area, obtains the pixel area of ​​the coal blockage area, calculates the ratio of the pixel area of ​​the coal blockage area to the total area of ​​the overall area, and determines that a coal accumulation anomaly has occurred if the ratio in the N consecutive frames of the image exceeds the set threshold. The image information of the coal drop outlet in the acquired image is combined with the coal level gauge signal and the operating status of the coal feeder to determine whether a coal blockage abnormality has occurred. Collect N consecutive frames of image information of the coal bunker lower opening and coal drop opening in the image, preset a bunker collapse detection area in the image, calculate the color difference diffusion rate and coal level drop rate in the bunker collapse detection area, and if the color difference diffusion rate and coal level drop rate in the N consecutive frames of the image both continuously exceed the corresponding threshold, it is judged as a bunker collapse anomaly. Based on the type of abnormality in the coal bunker, corresponding alarms will be triggered to alert relevant personnel.

[0007] Preferably, the image data preprocessing specifically includes: Image data is filtered to eliminate noise from coal falling from above the coal bunker or sensor errors; a semantic segmentation model is used to perform semantic segmentation on the image data to obtain images of coal flow, conveyor belts, and the environment, as well as corresponding feature mask information.

[0008] Preferably, the method involves acquiring image information from N consecutive frames of the coal pile detection area in the acquired image, pre-setting the coal pile detection area in the image, and obtaining the pixel area of ​​the coal pile area, specifically as follows: Step 1a: Extract the mask of the coal flow detection area to eliminate interference from the conveyor belt and the environment; Step 1b: Divide the coal flow detection area into grid sub-blocks, and calculate the color standard deviation and the histogram similarity of N consecutive frames for each grid sub-block; Step 1c: If the color standard deviation of the grid sub-block is less than the set color standard deviation threshold, and the similarity with the histogram of the previous frame is greater than the set histogram similarity threshold, then the grid sub-block is determined to be a stationary coal body. Step 1d: Accumulate the area of ​​the grid sub-blocks corresponding to all stationary coal bodies, which is the pixel area of ​​the coal pile area.

[0009] Preferably, the coal drop hole image information in the acquired image is combined with the coal level gauge signal and the coal feeder operating status to determine whether a coal blockage abnormality has occurred. Specifically: Step 2a: Collect coal level gauge signal at the top of the coal bunker, coal feeder operating status, and coal drop port image data; Step 2b: When the coal feeder start signal is triggered, the coal flow mask is extracted from the conveyor belt image based on the semantic segmentation model, and the data is processed at time intervals. Calculate the coal flow filling rate in the coal drop area; Step 2c: If the coal flow filling rate within the time duration T is less than the set threshold, then proceed to step 2d. Step 2d: When the coal level gauge meets the condition that the real-time coal level height is greater than the safety threshold and the coal level descent rate is less than the preset threshold, it is determined that a coal blockage anomaly has occurred.

[0010] Preferably, the acquired images include N consecutive frames of images of the coal bunker's lower opening and the coal drop outlet. A breach detection area is preset in the images, and the color difference diffusion rate and coal level descent rate are calculated within this area. If both the color difference diffusion rate and the coal level descent rate continuously exceed their corresponding thresholds in the N consecutive frames, a breach anomaly is determined. Specifically: Step 3a: Collect N consecutive frames of image information of the coal bunker's lower opening and coal drop opening in the image, and preset the bunker collapse detection area in the image; Step 3b: Convert the image of the cavity detection area to the CIE-Lab color space and calculate the standard color difference value of each sub-region in the color space; Step 3c: Based on the vector field changes corresponding to the standard color difference values ​​of adjacent frames, the proportion of pixels with the same diffusion direction is statistically analyzed, and the color difference diffusion rate is calculated in combination with the frame rate. Step 3d: Input coal level sensor data and calculate the coal level descent rate in real time; Step 3e: When the color difference diffusion rate continues to exceed the threshold and the coal level drop rate continues to exceed the threshold for a preset time, it is judged as a collapse anomaly.

[0011] Preferably, the method of issuing corresponding alarms to relevant personnel based on the type of anomaly in the coal bunker specifically includes: A three-tiered alarm mechanism is established. When there is an abnormality of coal blockage or coal accumulation, a yellow alert is issued, which requires manual confirmation. When there is an abnormality of coal bin collapse, an orange alert is issued, and the machine is shut down for inspection. When a coal bin collapse is confirmed to have occurred, a red alert is issued, and emergency avoidance measures are implemented.

[0012] A coal bunker abnormality monitoring system integrating multi-source data includes: The image capture module is used to capture image data of the coal bunker. A data preprocessing module, electrically connected to the image capture module, is used to receive image data output by the image capture module and perform preprocessing. The coal level signal acquisition module is used to acquire the coal level gauge signal inside the coal bunker. The coal feeder status acquisition module is used to collect the operating status signals of the coal feeder; The coal pile identification module is electrically connected to the data preprocessing module. It is used to extract N consecutive frames of image information of the coal blockage detection area from the preprocessed image, calculate the ratio of the area of ​​coal blockage-related pixels in the area to the total area of ​​the detection area, and determine that a coal pile abnormality has occurred if the ratio of the N consecutive frames exceeds a set threshold. The coal blockage identification module is electrically connected to the data preprocessing module, the coal level signal acquisition module and the coal feeder status acquisition module, respectively. It is used to receive the preprocessed coal drop image information, coal level gauge signal and coal feeder operating status signal, and combine the three to determine whether a coal blockage abnormality has occurred. The coal bunker collapse detection module is electrically connected to the data preprocessing module and the coal level signal acquisition module, respectively. It is used to extract N consecutive frames of image information of the coal bunker lower opening and coal drop opening from the preprocessed image, preset a collapse detection area in the image, calculate the color difference diffusion rate in the area, and calculate the coal level drop rate in combination with the coal level data output by the coal level signal acquisition module. If the color difference diffusion rate and the coal level drop rate both exceed the corresponding threshold in N consecutive frames, the collapse anomaly is determined to have occurred. The anomaly warning module is electrically connected to the coal pile identification module, the coal blockage identification module, and the silo collapse identification module, respectively. It is used to generate corresponding alarm information to remind relevant personnel based on the anomaly type determined by each identification module.

[0013] Preferably, the data preprocessing module includes an image filtering unit and a semantic segmentation unit; The image filtering unit is used to filter the raw image data output by the image capture module to mitigate the noise effects caused by coal falling from above the coal bunker or sensor errors. The semantic segmentation unit is used to segment the filtered image data using a pre-trained semantic segmentation model, and output feature masks of coal flow, conveyor belt and environmental background to separate the coal flow area from the interference area.

[0014] Preferably, the pre-trained model used by the semantic segmentation unit is the U-net model or the lightweight YOLO semantic segmentation model, and the model training dataset contains coal bunker images under different lighting conditions, different coal types, and different dust concentrations.

[0015] As a preferred option, the coal bunker abnormal status monitoring system that integrates multiple data sources also includes a visual interface for simultaneously displaying coal level curves, real-time video streams, and color difference heat maps, as well as displaying alarm information.

[0016] The beneficial effects of this invention are: by combining machine vision technology with coal level information, coal feeder status, etc., this invention can achieve low-cost identification of multiple states of the coal bunker without modifying or replacing existing equipment. This feature makes this invention particularly suitable for monitoring coal bunkers in coal mines already in production, saving costs and avoiding production interruptions caused by equipment upgrades; Compared to other solutions based on physical sensors, machine vision-based solutions are more intuitive and allow for remote monitoring to check for false alarms. This not only reduces the need for manual downhole confirmation and inspection, thus lowering operational risks, but also improves the speed and accuracy of fault diagnosis, further ensuring production safety. Attached Figure Description

[0017] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0018] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0019] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0020] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0021] Example: A multi-source data fusion method for monitoring abnormal conditions in coal bunkers, such as... Figure 1 As shown, it includes the following steps: The image capture device captures images of the coal bunker and preprocesses the image data; The system collects image information from N consecutive frames of the coal blockage detection area, obtains the pixel area of ​​the coal blockage area, calculates the ratio of the pixel area of ​​the coal blockage area to the total area of ​​the overall area, and determines that a coal accumulation anomaly has occurred if the ratio in the N consecutive frames of the image exceeds the set threshold. The image information of the coal drop outlet in the acquired image is combined with the coal level gauge signal and the operating status of the coal feeder to determine whether a coal blockage abnormality has occurred. Collect N consecutive frames of image information of the coal bunker lower opening and coal drop opening in the image, preset a bunker collapse detection area in the image, calculate the color difference diffusion rate and coal level drop rate in the bunker collapse detection area, and if the color difference diffusion rate and coal level drop rate in the N consecutive frames of the image both continuously exceed the corresponding threshold, it is judged as a bunker collapse anomaly. Based on the type of abnormality in the coal bunker, corresponding alarms will be triggered to alert relevant personnel.

[0022] The aforementioned preprocessing of image data specifically includes: Image data is filtered to eliminate noise from coal falling from above the coal bunker or sensor errors; a semantic segmentation model is used to perform semantic segmentation on the image data to obtain images of coal flow, conveyor belts, and the environment, as well as corresponding feature mask information.

[0023] The acquisition of N consecutive frames of image information from the coal blockage detection area in the acquired image, the pre-defined coal accumulation detection area in the image, and the acquisition of the pixel area of ​​the coal blockage area are specifically as follows: Step 1a: Extract the mask of the coal flow detection area to eliminate interference from the conveyor belt and the environment; Step 1b: Divide the coal flow detection area into grid sub-blocks, and calculate the color standard deviation and the histogram similarity of N consecutive frames for each grid sub-block; Step 1c: If the color standard deviation of the grid sub-block is less than the set color standard deviation threshold, and the similarity with the histogram of the previous frame is greater than the set histogram similarity threshold, then the grid sub-block is determined to be a stationary coal body. Step 1d: Accumulate the area of ​​the grid sub-blocks corresponding to all stationary coal bodies, which is the pixel area of ​​the coal blockage area.

[0024] The image information of the coal drop outlet in the acquired images, combined with the coal level gauge signal and the operating status of the coal feeder, is used to determine whether a coal blockage abnormality has occurred. Specifically: Step 2a: Collect coal level gauge signal at the top of the coal bunker, coal feeder operating status, and coal drop port image data; Step 2b: When the coal feeder start signal is triggered, the coal flow mask is extracted from the conveyor belt image based on the semantic segmentation model, and the data is processed at time intervals. Calculate the coal flow filling rate in the coal drop area; Step 2c: If the coal flow filling rate within the time duration T is less than the set threshold, then proceed to step 2d. Step 2d: When the coal level gauge meets the condition that the real-time coal level height is greater than the safety threshold and the coal level descent rate is less than the preset threshold, it is determined that a coal blockage anomaly has occurred.

[0025] The acquired images include N consecutive frames of images of the coal bunker's lower opening and coal drop outlet. A breach detection area is preset in the images. The color difference diffusion rate and coal level descent rate are calculated in the breach detection area. If both the color difference diffusion rate and the coal level descent rate continuously exceed the corresponding thresholds in N consecutive frames, it is judged as a breach anomaly. Specifically: Step 3a: Collect N consecutive frames of image information of the coal bunker's lower opening and coal drop opening in the image, and preset the bunker collapse detection area in the image; Step 3b: Convert the image of the cavity detection area to the CIE-Lab color space and calculate the standard color difference value of each sub-region in the color space; Step 3c: Based on the vector field changes corresponding to the standard color difference values ​​of adjacent frames, the proportion of pixels with the same diffusion direction is statistically analyzed, and the color difference diffusion rate is calculated in combination with the frame rate. Step 3d: Input coal level sensor data and calculate the coal level descent rate in real time; Step 3e: When the color difference diffusion rate continues to exceed the threshold and the coal level drop rate continues to exceed the threshold for a preset time, it is judged as a collapse anomaly.

[0026] The aforementioned method of issuing corresponding alarms to relevant personnel based on the type of anomaly in the coal bunker specifically includes: A three-tiered alarm mechanism is established. When there is an abnormality of coal blockage or coal accumulation, a yellow alert is issued, which requires manual confirmation. When there is an abnormality of coal bin collapse, an orange alert is issued, and the machine is shut down for inspection. When a coal bin collapse is confirmed to have occurred, a red alert is issued, and emergency avoidance measures are implemented.

[0027] A coal bunker abnormality monitoring system integrating multi-source data includes: The image capture module is used to capture image data of the coal bunker. A data preprocessing module, electrically connected to the image capture module, is used to receive image data output by the image capture module and perform preprocessing. The coal level signal acquisition module is used to acquire the coal level gauge signal inside the coal bunker. The coal feeder status acquisition module is used to collect the operating status signals of the coal feeder; The coal pile identification module is electrically connected to the data preprocessing module. It is used to extract N consecutive frames of image information of the coal blockage detection area from the preprocessed image, calculate the ratio of the area of ​​coal blockage-related pixels in the area to the total area of ​​the detection area, and determine that a coal pile abnormality has occurred if the ratio of the N consecutive frames exceeds a set threshold. The coal blockage identification module is electrically connected to the data preprocessing module, the coal level signal acquisition module and the coal feeder status acquisition module, respectively. It is used to receive the preprocessed coal drop image information, coal level gauge signal and coal feeder operating status signal, and combine the three to determine whether a coal blockage abnormality has occurred. The coal bunker collapse detection module is electrically connected to the data preprocessing module and the coal level signal acquisition module, respectively. It is used to extract N consecutive frames of image information of the coal bunker lower opening and coal drop opening from the preprocessed image, preset a collapse detection area in the image, calculate the color difference diffusion rate in the area, and calculate the coal level drop rate in combination with the coal level data output by the coal level signal acquisition module. If the color difference diffusion rate and the coal level drop rate both exceed the corresponding threshold in N consecutive frames, the collapse anomaly is determined to have occurred. The anomaly warning module is electrically connected to the coal pile identification module, the coal blockage identification module, and the silo collapse identification module, respectively. It is used to generate corresponding alarm information to remind relevant personnel based on the anomaly type determined by each identification module.

[0028] The data preprocessing module includes an image filtering unit and a semantic segmentation unit; The image filtering unit is used to filter the raw image data output by the image capture module to mitigate the noise effects caused by coal falling from above the coal bunker or sensor errors. The semantic segmentation unit is used to segment the filtered image data using a pre-trained semantic segmentation model, and output feature masks of coal flow, conveyor belt and environmental background to separate the coal flow area from the interference area.

[0029] The semantic segmentation unit uses a U-net model or a lightweight YOLO semantic segmentation model as its pre-trained model. The model training dataset contains coal bunker images under different lighting conditions, coal types, and dust concentrations.

[0030] The coal bunker abnormality monitoring system, which integrates multiple data sources, also includes a visual interface for simultaneously displaying coal level curves, real-time video streams, and color difference heat maps, as well as alarm information.

[0031] Specifically, the implementation method of this solution is as follows: 1) Coal Flow Segmentation: Images acquired by industrial cameras are processed by segmentation models (such as U-net, YOLO, and other deep learning models) to segment the coal flow, conveyor belt, and background masks. Subsequently, the coal flow area fill rate within the defined coal blockage detection area is calculated. The training dataset for the semantic segmentation model should cover different light intensities, coal accumulation patterns, and dust interference scenarios to ensure generalization ability under complex environments.

[0032] 2) Coal level signal filtering: Noise in the coal level data affects the calculation of the coal level change rate. Therefore, the coal level data can be filtered (e.g., primary filtering, Kalman filtering) before calculation. Taking Kalman filtering as an example, the original data from the radar coal level gauge is used as input to establish a state vector. and (Height and rate of change), and define the process noise covariance. Observation noise The coal level estimate is dynamically corrected using a recursive formula, and the smoothed height sequence and instantaneous velocity are output. This filtering algorithm can suppress random fluctuations caused by mechanical vibration and data drift, providing a stable data base for judging coal pile-up, coal blockage, and silo collapse.

[0033] After completing the above data processing, coal accumulation, blockage, and silo failure were identified at the coal feeder's coal drop point. 1) Coal pile identification module After coal flow segmentation, the system dynamically defines the coal blockage detection area (e.g., a 300×400 pixel area at the center of the coal inlet). N consecutively acquired images are processed, and the pixel area of ​​the coal blockage detection area identified as "stationary coal" is calculated. The area of ​​stationary coal within the current frame's coal blockage detection area is divided by the total area of ​​the coal blockage detection area to obtain the percentage of stationary coal area in that frame. When continuous N The proportion of stationary coal area in the frame exceeds a set threshold. And the duration reached At a certain time, it is determined to be a coal accumulation event. The method for calculating the "stationary coal body" is as follows: (a) Extract the coal flow area mask in the coal blockage detection area to eliminate interference from the conveyor belt and background; (b) Divide the coal flow region into grid sub-blocks and calculate the color standard deviation (HSV space) and continuous frame histogram similarity (e.g., Bach coefficient) for each sub-block. (c) If the color fluctuation of the sub-block is low (standard deviation < threshold) And its color distribution is highly similar to that of the previous frame (similarity > threshold). If the sub-block is a stationary coal body, then it is determined that the sub-block is a stationary coal body. (d) Accumulate the areas of all stationary sub-blocks to obtain the total area of ​​stationary coal bodies within the coal blockage detection area.

[0034] As an optimization, the overall difference between coal-piling areas in adjacent frames can be quantified using the Structural Similarity Index (SSIM). When the SSIM value exceeds 0.85 for 10 consecutive seconds and the coal coverage is >85%, it is determined to be a coal-piling event. In addition, the ROI area can be filtered using methods such as sliding window mean filtering, which can effectively filter out the influence of instantaneous coal flow fluctuations or slight changes in light.

[0035] 2) Coal blockage identification module Upon triggering the coal feeder start signal, the system immediately activates the collaborative acquisition of the radar coal level gauge and the industrial camera. At this time, the industrial camera facing the coal inlet initiates high frame rate capture (25fps), and the video stream is input to a pre-trained lightweight coal flow segmentation model to segment the coal flow and conveyor belt mask in real time. (L is the actual length of the coal flow detection area along the belt direction) Calculate the coal flow area within the ROI (Region of Interest) for coal flow detection at the coal drop point based on the time interval (for belt conveyor speed). and belt area This allows us to obtain the effective coal flow area ratio (i.e., coal flow filling rate). If the fill rate sequence detected for 30 consecutive seconds... Average coal flow filling rate ( , N For sequence If the length is less than 5% and the morphological continuity is broken, visual verification confirms that the coal flow is interrupted. The system aligns the coal level information with the timestamp of the visual analysis results through hardware-level synchronization signals. When the coal quantity in the coal bunker is normal and the coal level descent rate is calculated and smoothed in real time by Kalman filtering, if the rate is detected to be lower than a preset threshold (such as 0.1 m / min) for 30 consecutive seconds, it is determined that there is a risk of coal blockage and a coal blockage alarm is triggered.

[0036] 3) Collapse Detection Module: Collapse detection converts the video stream to the Lab color space in real time, utilizing the luminance separation of the L channel and the color difference sensitivity of the ab channels (ΔE>15 indicates a significant color change). A detection area is defined, divided into a 16×12 grid. The color difference change rate is independently calculated for each grid, and the diffusion speed of abnormal areas is tracked using optical flow. When the color difference of three adjacent grids changes abruptly and the diffusion rate is >2 grids / second, a primary warning is triggered, and an orange alarm is issued for manual confirmation. Coal level sensor data is simultaneously integrated to calculate the coal level descent rate in real time; if the color difference diffusion rate continuously exceeds a threshold... Furthermore, the rate of coal level decline continues to exceed the threshold. When the preset time Δt is reached (e.g., a drop of more than 2 meters within 5 minutes), it is determined to be a warehouse collapse event; a red alarm is triggered and multimodal evidence is pushed to the monitoring terminal to support manual confirmation.

[0037] 4) Anomaly warning module, specifically including: (a) Alarm Triggering and Response: Yellow Alarm: When the coal blockage identification module or coal pile identification module is activated, a yellow alarm is triggered and a manual confirmation window pops up on the operation interface; Orange Alarm: If the collapse identification module detects a color difference diffusion rate > 1200 pixels / second or a coal level drop rate > 0.6 m / second, the belt operation is immediately stopped and the equipment is locked. At the same time, an encrypted alarm work order is sent to the safety supervisor; Red Emergency Stop: Within 0.5 seconds after the collapse event is confirmed, the power supply to the coal feeder is cut off and the spray dust suppression system is started. (b) Visualization and self-testing: The control panel integrates three screens: the left screen plays thermal imaging video stream in real time (with color difference diffusion vectors marked), the middle screen dynamically refreshes the coal level curve (sampling rate 1Hz), and the right screen overlays semantic segmentation results and equipment status tree; sensor diagnosis is automatically performed at 02:00 every day: the degree of contamination is judged by lens transmittance detection (threshold < 85% triggers cleaning alarm), and the echo intensity of the coal level gauge is compared with the reference value (calibration is performed when the deviation > 3dB); (c) Data Management: Use lossy compression (H.265) to store the video stream of the most recent 72 hours (compression rate ≥60%), and record all operating parameters to the edge database (retention period 30 days); support the backtracking of abnormal events by timestamp, and automatically generate fault analysis reports containing key frame markers.

[0038] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0039] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for monitoring abnormal conditions in coal bunkers based on multi-source data fusion, characterized by: Includes the following steps: The image capture device captures images of the coal bunker and preprocesses the image data; Collect image information of N consecutive frames of coal pile detection area in the image, preset coal pile detection area in the image, obtain the pixel area of ​​stationary coal in the coal pile area, calculate the ratio of the pixel area of ​​stationary coal in the coal pile area to the total area of ​​the whole area, and judge that a coal pile abnormality has occurred if the ratio of stationary coal in the N consecutive frames of the image exceeds the set threshold. The image information of the coal drop outlet in the acquired image is combined with the coal level gauge signal and the operating status of the coal feeder to determine whether a coal blockage abnormality has occurred. Collect N consecutive frames of image information of the coal bunker lower opening and coal drop opening in the image, preset a bunker collapse detection area in the image, calculate the color difference diffusion rate and coal level drop rate in the bunker collapse detection area, and if the color difference diffusion rate and coal level drop rate in the N consecutive frames of the image both continuously exceed the corresponding threshold, it is judged as a bunker collapse anomaly. Based on the type of abnormality in the coal bunker, corresponding alarms will be triggered to alert relevant personnel.

2. The coal bunker abnormality monitoring method based on multi-source data fusion according to claim 1, characterized in that, The aforementioned preprocessing of image data specifically includes: Image data is filtered to eliminate noise from coal falling from above the coal bunker or sensor errors; a semantic segmentation model is used to perform semantic segmentation on the image data to obtain images of coal flow, conveyor belts, and the environment, as well as corresponding feature mask information.

3. The coal bunker abnormality monitoring method based on multi-source data fusion according to claim 2, characterized in that, The method involves acquiring N consecutive frames of image information from the coal blockage detection area in the acquired images, pre-setting a coal pile detection area in the images, and obtaining the pixel area of ​​the stationary coal body in the coal pile area. Specifically: Step 1a: Extract the mask of the coal flow detection area to eliminate interference from the conveyor belt and the environment; Step 1b: Divide the coal flow detection area into grid sub-blocks, and calculate the color standard deviation and the similarity of the histograms of N consecutive frames for each grid sub-block; Step 1c: If the color standard deviation of the grid sub-block is less than the set color standard deviation threshold, and the similarity with the histogram of the previous frame is greater than the set histogram similarity threshold, then the grid sub-block is determined to be a stationary coal body. Step 1d: Accumulate the area of ​​the grid sub-blocks corresponding to all stationary coal bodies, which is the pixel area of ​​the coal pile area.

4. The coal bunker abnormality monitoring method based on multi-source data fusion according to claim 2, characterized in that, The image information of the coal drop outlet in the acquired images, combined with the coal level gauge signal and the operating status of the coal feeder, is used to determine whether a coal blockage abnormality has occurred. Specifically: Step 2a: Collect coal level gauge signal at the top of the coal bunker, coal feeder operating status, and coal drop port image data; Step 2b: When the coal feeder start signal is triggered, the coal flow mask is extracted from the conveyor belt image based on the semantic segmentation model, and the time interval is calculated. Calculate the coal flow filling rate in the coal drop area; Step 2c: If the coal flow filling rate within the time duration T is less than the set threshold, then proceed to step 2d. Step 2d: When the coal level gauge meets the condition that the real-time coal level height is greater than the safety threshold and the coal level descent rate is less than the preset threshold, it is determined that a coal blockage anomaly has occurred.

5. The method for monitoring abnormal states of coal bunkers based on multi-source data fusion according to claim 1 or 2, characterized in that, The acquired images include N consecutive frames of images of the coal bunker's lower opening and coal drop outlet. A breach detection area is preset in the images. The color difference diffusion rate and coal level descent rate are calculated in the breach detection area. If both the color difference diffusion rate and the coal level descent rate continuously exceed the corresponding thresholds in N consecutive frames, it is judged as a breach anomaly. Specifically: Step 3a: Collect N consecutive frames of image information of the coal bunker's lower opening and coal drop opening in the image, and preset the bunker collapse detection area in the image; Step 3b: Convert the image of the cavity detection area to the CIE-Lab color space and calculate the standard color difference value of each sub-region in the color space; Step 3c: Based on the vector field changes corresponding to the standard color difference values ​​of adjacent frames, the proportion of pixels with the same diffusion direction is statistically analyzed, and the color difference diffusion rate is calculated in combination with the frame rate. Step 3d: Input coal level sensor data and calculate the coal level descent rate in real time; Step 3e: When the color difference diffusion rate continues to exceed the threshold and the coal level drop rate continues to exceed the threshold for a preset time, it is judged as a collapse anomaly.

6. The coal bunker abnormal state monitoring method based on multi-source data fusion according to claim 1, characterized in that, The aforementioned method of issuing corresponding alarms to relevant personnel based on the type of anomaly in the coal bunker specifically includes: A three-tiered alarm mechanism is established. When there is an abnormality of coal blockage or coal accumulation, a yellow alert is issued, which requires manual confirmation. When there is an abnormality of coal bin collapse, an orange alert is issued, and the machine is shut down for inspection. When a coal bin collapse is confirmed to have occurred, a red alert is issued, and emergency avoidance measures are implemented.

7. A coal bunker abnormal state monitoring system based on multi-source data fusion, applicable to the coal bunker abnormal state monitoring method based on multi-source data fusion as described in any one of claims 1-6, characterized in that, include: The image capture module is used to capture image data of the coal bunker. A data preprocessing module, electrically connected to the image capture module, is used to receive image data output by the image capture module and perform preprocessing. The coal level signal acquisition module is used to acquire coal level gauge signals inside the coal bunker. The coal feeder status acquisition module is used to collect the operating status signals of the coal feeder; The coal pile identification module is electrically connected to the data preprocessing module. It is used to extract N consecutive frames of image information of the coal blockage detection area from the preprocessed image, calculate the ratio of the area of ​​coal blockage-related pixels in the area to the total area of ​​the detection area, and determine that a coal pile abnormality has occurred if the ratio of the N consecutive frames exceeds a set threshold. The coal blockage identification module is electrically connected to the data preprocessing module, the coal level signal acquisition module and the coal feeder status acquisition module, respectively. It is used to receive the preprocessed coal drop image information, coal level gauge signal and coal feeder operating status signal, and combine the three to determine whether a coal blockage abnormality has occurred. The coal bunker collapse detection module is electrically connected to the data preprocessing module and the coal level signal acquisition module, respectively. It is used to extract N consecutive frames of image information from the lower opening of the coal bunker and the coal drop opening from the preprocessed image, preset a collapse detection area in the image, calculate the color difference diffusion rate in the area, and calculate the coal level descent rate by combining the coal level data output by the coal level signal acquisition module. If the color difference diffusion rate and the coal level descent rate both exceed the corresponding threshold in N consecutive frames, the collapse anomaly is determined to have occurred. The anomaly warning module is electrically connected to the coal pile identification module, the coal blockage identification module, and the silo collapse identification module, respectively. It is used to generate corresponding alarm information to remind relevant personnel based on the anomaly type determined by each identification module.

8. The coal bunker abnormal state monitoring system based on multi-source data fusion according to claim 7, characterized in that, The data preprocessing module includes an image filtering unit and a semantic segmentation unit; The image filtering unit is used to filter the raw image data output by the image capture module to mitigate the noise effects caused by coal falling from above the coal bunker or sensor errors. The semantic segmentation unit is used to segment the filtered image data using a pre-trained semantic segmentation model, and output feature masks of coal flow, conveyor belt and environmental background to separate the coal flow area from the interference area.

9. The coal bunker abnormal state monitoring system based on multi-source data fusion according to claim 7, characterized in that, The semantic segmentation unit uses a U-net model or a lightweight YOLO semantic segmentation model as its pre-trained model. The model training dataset contains coal bunker images under different lighting conditions, coal types, and dust concentrations.

10. The coal bunker abnormal state monitoring system based on multi-source data fusion according to claim 7, characterized in that, It also includes a visualization interface for synchronously displaying coal level curves, real-time video streams, and color difference thermal maps, as well as displaying alarm information.

Images