Coal flow image detection method, device, medium, program product
By improving the YOLOv11 neural network model, the problem of inaccurate coal flow status perception in dust control of coal mine belt conveyors was solved, achieving high-precision detection in underground environments and improving the effectiveness of dust control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 华电江苏能源有限公司
- Filing Date
- 2026-03-24
- Publication Date
- 2026-06-09
Smart Images
Figure CN122176406A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of machine vision technology, specifically relating to a coal flow image detection method, device, medium, and program product. Background Technology
[0002] In coal mining and transportation, belt conveyors are the core equipment for coal transfer. The large amounts of dust generated during their operation not only pollute the underground working environment but also pose serious risks to the respiratory system of workers, and can even trigger safety accidents such as coal dust explosions. Therefore, dust control is one of the key aspects of safe coal mine production. Currently, dust control for coal mine belt conveyors mainly relies on traditional spray dust suppression systems. These systems often employ fixed spray patterns or trigger-based spraying based on simple sensors, which have significant limitations.
[0003] Traditional systems lack the ability to accurately sense the state of coal flow, and cannot dynamically identify the presence, flow rate, and transfer location of coal. This often leads to problems such as "no-coal spraying" causing water waste or "coal-containing but not timely spraying" causing dust diffusion. Furthermore, due to environmental factors such as underground humidity, dust obstruction, and changes in light, existing image recognition-based detection methods suffer from low detection accuracy and weak anti-interference capabilities, and are also unable to provide reliable triggering information for the spraying system. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a coal flow image detection method, device, medium, and program product to overcome environmental interference such as dust and light in underground mines and achieve accurate perception of coal flow status.
[0005] The present invention achieves the above-mentioned technical objectives through the following technical means.
[0006] A coal flow image detection method employs the following image detection model, which is an improvement upon YOLOv11 with the following modifications:
[0007] The convolutional layers in the original network are replaced with windmill convolutions, and an AMPFM module is embedded before each windmill convolution; in the AMPFM module:
[0008] For the input feature map, three feature maps of different scales are extracted through three parallel branches;
[0009] The feature maps at the three scales are weighted and fused. The feature map with the same coal flow feature scale as the input feature map is given a larger weight, while the feature maps at the other two scales are given smaller weights.
[0010] The weighted fused feature maps are then batch normalized and ReLU activated sequentially before being output.
[0011] Furthermore, in the AMPFM module, the three parallel branches are as follows:
[0012] The first branch extracts small-scale feature maps using a 3×3 convolution with a stride of s=1.
[0013] The second branch extracts the mesoscale feature map using a 5×5 dilated convolution with a stride of s=1 and a dilation rate of r=2.
[0014] The third branch extracts large-scale feature maps using a 7×7 dilated convolution with a stride of s=2 and a dilation rate of r=3.
[0015] Furthermore, in the AMPFM module, the input feature map is first processed by global max pooling to obtain the feature scale feature vector, and then processed by a fully connected layer to obtain the scale classification of coal flow features.
[0016] Furthermore, in the AMPFM module, the feature vector of the feature scale is sequentially passed through the first fully connected layer and ReLU activation, the second fully connected layer and Softmax normalization to obtain the probability that the coal flow feature belongs to the three different scales, and the scale classification of the coal flow feature is determined according to the probability magnitude.
[0017] Furthermore, the DFF module is added to the original C3k2 module, where:
[0018] The feature maps output by Split and the last Botteleneck in the C3k2 module are used as the two input features of the DFF module. and ;
[0019] feature and After adjusting the number of channels through 1×1×1 convolutions, the results are summed, and then spatial weights are generated through Sigmoid activation. ;
[0020] feature and The feature maps are concatenated along the channel dimension, and then sequentially processed by global average pooling, convolution, and sigmoid activation to generate channel weights. ;
[0021] Channel weight The concatenated feature map is multiplied element-wise, and then the number of channels is adjusted by a 1×1×1 convolution to obtain the intermediate feature. ;
[0022] Spatial weight With intermediate features Element-wise multiplication yields the characteristics of fusion. And output it.
[0023] Furthermore, a CFPM module is added to the neck network of YOLOv11, set after each feature concatenation and each upsampling; in the CFPM module:
[0024] Based on the grayscale difference between the coal flow image region and the surrounding background image region, global grayscale statistics are performed on the input feature map to generate a grayscale mask matrix. The matrix In the image, points whose grayscale values exceed the grayscale range of the coal flow image are set to 0;
[0025] Grayscale mask matrix The initial screening feature map is obtained by multiplying it element-wise with the original input feature map. ;
[0026] Initial screening feature map First, texture information is extracted using a 3×3 depthwise separable convolution. Then, the texture gradient value is calculated using a Laplacian edge detection operator to generate a texture mask matrix. The matrix In the middle, for points whose texture gradient values are lower than a preset threshold, set them to 0;
[0027] Texture mask matrix Compared with the initial screening feature map Element-wise multiplication yields the refined screening feature map. ;
[0028] Fine screening feature map Compared with the initial screening feature map The residual connection is first multiplied by a weak feature compensation coefficient, and then a 1×1 depthwise separable convolution is used for channel compression and fusion to adjust the number of channels to match the original input features. Figure 1 Output after completion.
[0029] Furthermore, in the windmill convolution:
[0030] Four sets of parallel convolutions are performed on the input feature map. Each set uses asymmetric padding and asymmetric convolution kernels in different directions to capture features in different directions of coal flow.
[0031] Then the outputs of the four convolutions are concatenated along the channel dimension;
[0032] Finally, a 2×2 convolution is used to fuse the spliced features and adjust the number of channels.
[0033] A computer device, including a memory and a processor;
[0034] The memory is used to store computer programs;
[0035] The processor is used to execute the computer program and implement the above-described coal flow image detection method when executing the computer program.
[0036] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the above-described coal flow image detection method.
[0037] A computer program product includes a computer program that, when executed by a processor, implements the above-described coal flow image detection method.
[0038] The beneficial effects of this invention are as follows:
[0039] (1) This invention provides a coal flow image detection method, which uses a network model based on YOLOv11 for image detection. The traditional convolution is replaced by windmill convolution, and a newly designed AMRFM module is embedded, which can achieve efficient and accurate extraction of the features of coal flow targets, overcome the interference of environmental factors such as underground humidity, dust obstruction, and light changes, and improve detection accuracy.
[0040] (2) The present invention incorporates the CFPM module to optimize the neck network of YOLOv11, which is used to target the differences in grayscale and texture between the coal flow target and the background. First, the coal flow and background features are separated by multi-threshold feature screening. Then, the core features such as coal flow edge, texture, and spatial distribution are enhanced by feature residual purification. Finally, the module is lightweight by channel compression fusion, thereby improving the model detection performance.
[0041] (3) The present invention incorporates the DFF module into the C3k2 module, enabling it to adaptively fuse multi-scale local features and global information, while preserving local feature details and enhancing the mining and utilization of global information. Attached Figure Description
[0042] Figure 1 This is a diagram of the improved YOLOv11 network structure of the present invention;
[0043] Figure 2 This is the original YOLOv11 network structure diagram;
[0044] Figure 3 Here is a diagram of the AMPFM module structure;
[0045] Figure 4 Here is the structure diagram of the C3k2 module;
[0046] Figure 5 This is a diagram of the DFF module structure.
[0047] Figure 6 This is a diagram of the CFPM module structure. Detailed Implementation
[0048] Embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0049] I. Technical Solution
[0050] For coal flow detection tasks, this invention improves upon the YOLOv11 neural network to obtain a relevant coal flow image detection model for coal flow image detection. Figure 1 The diagram shown is a network structure diagram of the improved version of this invention. Figure 2 The diagram shown is the original YOLOv11 network architecture. (Summary) Figure 1 and Figure 2 As shown, the present invention makes the following improvements based on YOLOv11:
[0051] 1. Windmill Convolution
[0052] We replace the traditional convolutions in the original YOLOv11 with windmill convolutions (PSConv). This results in a larger receptive field and feature extraction that better matches the distribution of small objects with only a small increase in parameters.
[0053] The core of PSConv is to simulate the Gaussian spatial distribution of small targets:
[0054] First, four sets of parallel convolutions are performed on the input feature map. Each set uses asymmetric padding and asymmetric convolution kernels in different directions to capture features in different directions of coal flow. For example, the padding quantity of a certain set is P(1, 0, 0, 3) (1 padding on the left, 0 padding on the right, 0 padding on the top, and 3 padding on the bottom), which is paired with a 1×3 convolution kernel to specifically extract features in the vertical direction.
[0055] Then, the outputs of the four directional convolutions are concatenated along the channel dimension to integrate feature information from different directions, forming a richer feature tensor.
[0056] Finally, a 2×2 convolution is used to fuse the concatenated features, while adjusting the number of channels to match the output size of a traditional convolution.
[0057] Compared to the traditional 3×3 convolution with a receptive field of 9, PSConv (k=3) achieves a receptive field of 25 (a 177% improvement), while PSConv (k=4) boasts an even greater 444% improvement. Through asymmetric padding and oriented convolution, the convolution kernel expands outwards, covering a larger contextual region with minimal parameter increases, making it more suitable for capturing the edges and background information of coal flows. PSConv achieves a larger receptive field with a small increase in parameters; furthermore, since small targets typically exhibit a Gaussian distribution with a bright center and dark edges, the effectiveness of PSConv's receptive field decreases from the center outwards (similar to a Gaussian distribution), highlighting the central features of the coal flow while suppressing background interference. This significantly enhances the contrast between the target and the background, improving the target detection rate.
[0058] 2. Adaptive multi-scale receptive field adjustment module
[0059] Before each layer of windmill convolution, an adaptive multi-scale receptive field adjustment module (AMPFM) is embedded. This module is based on "feature scale perception + dynamic receptive field switching + multi-branch feature fusion". The feature scale detection unit determines the large, medium and small scale type of the input coal flow features in real time. Then, relying on the multi-branch receptive field unit which includes large, medium and small receptive field branches, the corresponding branches are dynamically activated and irrelevant branches are suppressed. Finally, the feature map adapted to the coal flow scale is output by weighted fusion of branch features.
[0060] like Figure 3 In the AMPFM module shown:
[0061] On one hand, for the input feature map (of this module), a feature vector of the feature scale is first obtained through global max pooling; then, a fully connected layer is used to classify the feature vector to obtain the scale category label of the coal flow feature. Specifically, the feature vector is first subjected to dimensionality reduction by the first fully connected layer to compress the number of channels, then activated by ReLU, and then passed through the second fully connected layer and Softmax normalization in sequence to obtain the probability that the coal flow feature belongs to the three categories of "small scale", "medium scale" and "large scale", and the corresponding scale label is determined according to the probability.
[0062] On the other hand, through parallel branches with three different receptive fields (small, medium, and large), feature maps of different scales are extracted from the input feature map, where:
[0063] 1) The small-scale branch uses 3×3 convolution (Conv) with a stride s=1 to achieve detail extraction with a receptive field of 9, and obtain the corresponding small-scale feature map;
[0064] 2) The mesoscale branch uses 5×5 dilated convolution (DConv) with stride s=1 and dilation r=2 to achieve balanced extraction with a receptive field of 25, and obtains the corresponding mesoscale feature map.
[0065] 3) The large-scale branch uses 7×7 dilated convolution (DConv) with stride s=2 and dilation rate r=3 to achieve global extraction with a receptive field of 49, and obtains large-scale feature maps accordingly.
[0066] The three scale feature maps are then weighted and fused. The weight of each scale feature map is determined based on the scale label: the feature map with the same scale label category is given a larger weight for enhancement, while the other two feature maps with different scale label categories are given a smaller weight for weak feature compensation.
[0067] Finally, the weighted fusion feature map is then batch normalized and ReLU activated sequentially to output a feature map adapted to the coal flow scale. This expands the receptive field adaptability while avoiding exponential growth in the number of parameters, thus adapting to the detection scenarios with varying coal flow scales in the mine.
[0068] 3. Dynamic Feature Fusion Module
[0069] A Dynamic Feature Fusion (DFF) module is added to the original C3k2 module. This adaptively fuses multi-scale local features and global information, enhancing the mining and utilization of global information while preserving local feature details.
[0070] Figure 4 The diagram shown is a structural diagram of the C3k2 module. Figure 5 The diagram shown is a structural diagram of the DFF module. Figure 4 and Figure 5 As shown:
[0071] 1) Use the feature map output from Split in the C3k2 module and the feature map output from the last Bottleneck module as the two input features of the DFF module. and .
[0072] 2) Features and First, the number of channels is reduced / adjusted by 1×1×1 convolutions, then summed and activated by the Sigmoid activation function to generate spatial weights. .
[0073] 3.1) Features and Concat the data according to the channel dimension.
[0074] 3.2) The concatenated feature maps are then sequentially processed by global average pooling (AVGPool), convolution (Conv), and sigmoid activation to generate channel weights. ;
[0075] 4.1) Utilizing channel weights The concatenated feature maps are multiplied element-wise to highlight important feature channels; then, a 1×1×1 convolution is performed to adjust the number of channels to obtain intermediate features. ;
[0076] 4.2) Utilizing spatial weights For the intermediate features Element-wise multiplication is performed to weight the spatial dimensions, ultimately yielding the fused features. (i.e., the output of this module) achieves adaptive fusion of local feature detail preservation and global information enhancement, enabling the network to efficiently integrate multi-scale information in a larger receptive field.
[0077] 4. Lightweight coal flow characteristic purification module
[0078] A lightweight coal flow feature purification module (CFPM) is added to the YOLOv11 neck network, positioned after each feature concatenation and upsampling. This module focuses on "background suppression + core feature enhancement + lightweight computation." Addressing the differences in grayscale and texture between the coal flow target and the underground background, it first separates coal flow and background features through multi-threshold feature filtering. Then, it enhances core features such as coal flow edges, texture, and spatial distribution through feature residual purification. Finally, channel compression fusion ensures the module's lightweight nature, adapting to the computational demands of underground embedded devices. This multi-dimensional approach improves the model's detection accuracy and robustness through feature extraction, scale adaptation, and feature purification.
[0079] like Figure 6 The CFPM module shown:
[0080] 1.1) First, based on the grayscale difference between the coal flow image region and the surrounding underground background image region, perform global grayscale statistics on the input feature map to generate a grayscale mask matrix. The matrix In the image, points whose grayscale values exceed the grayscale range of the coal flow image are set to 0.
[0081] 1.2) Using a grayscale mask matrix The feature map is multiplied element-wise with the original input feature map to perform preliminary screening of background features and obtain a preliminary screening feature map. .
[0082] 2.1) To address the difference between the granular texture of the coal flow and the smooth texture of the background, a 3×3 depth-separable convolution was used to extract the initial screening feature map. The texture information is then used to calculate the texture gradient values using the Laplacian edge detection operator, thereby generating a texture mask matrix. The matrix In the process, texture gradient values below a preset threshold are identified as fine-grained background noise, and the value of that point is set to 0.
[0083] 2.2) Using texture mask matrix Compared with the initial screening feature map By multiplying element by element, a fine screening of coal flow characteristics is completed, resulting in a fine screening characteristic map. .
[0084] 2.3) Fine screening feature map Compared with the initial screening feature map Perform residual join (i.e., add the two together).
[0085] 3) For the feature map after residual connection, first multiply by the weak feature compensation coefficient to recover weak features such as small coal streams and coal stream edges lost during the screening process, thereby strengthening the core feature signal of the coal stream; then use 1×1 depthwise separable convolution to perform channel compression and fusion, adjusting the number of channels to match the original input features. Figure 1 This approach ensures that the core characteristics of the coal flow (edges, textures, spatial distribution) are fully preserved while achieving a lightweight design of the module to meet the computing power requirements of downhole embedded devices.
[0086] II. Devices, storage media, and software products
[0087] 1. Based on the same inventive concept as the above-described coal flow image detection method, this application also provides an electronic device, which includes a processor and a memory, wherein computer-readable code is stored in the memory, and when the computer-readable code is executed by the processor, the coal flow image detection method of the present invention is implemented.
[0088] The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium can store the operating system and computer-readable code. The computer-readable code includes program instructions that, when executed, cause the processor to perform the coal flow image detection method. The processor provides computational and control capabilities to support the operation of the entire electronic device. The memory provides an environment for the execution of the computer-readable code in the non-volatile storage medium, which, when executed by the processor, causes the processor to perform the coal flow image detection method.
[0089] It should be understood that a processor can be a central processing unit, other general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, transistor logic devices, discrete hardware components, etc. Among them, a general-purpose processor can be a microprocessor or any conventional processor.
[0090] 2. This application also provides a readable storage medium, which may be an internal storage unit of the electronic device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, smart memory card, or secure digital card equipped on the electronic device.
[0091] 3. This application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the coal flow image detection method of the present invention.
[0092] This invention is not limited to the above-described embodiments. Any obvious improvements, substitutions, or modifications that can be made by those skilled in the art without departing from the essence of this invention are within the scope of protection of this invention.
Claims
1. A method for detecting coal flow images, characterized in that: The following image detection model is adopted, which is an improvement on YOLOv11 with the following modifications: The convolutional layers in the original network are replaced with windmill convolutions, and an AMPFM module is embedded before each windmill convolution; in the AMPFM module: For the input feature map, three feature maps of different scales are extracted through three parallel branches; The feature maps at the three scales are weighted and fused. The feature map with the same coal flow feature scale as the input feature map is given a larger weight, while the feature maps at the other two scales are given smaller weights. The weighted fused feature maps are then batch normalized and ReLU activated sequentially before being output.
2. The coal flow image detection method according to claim 1, characterized in that: In the AMPFM module, the three parallel branches are as follows: The first branch extracts small-scale feature maps using a 3×3 convolution with a stride of s=1. The second branch extracts the mesoscale feature map using a 5×5 dilated convolution with a stride of s=1 and a dilation rate of r=2. The third branch extracts large-scale feature maps using a 7×7 dilated convolution with a stride of s=2 and a dilation rate of r=3.
3. The coal flow image detection method according to claim 1, characterized in that: In the AMPFM module, the input feature map is first processed by global max pooling to obtain the feature scale feature vector, and then processed by a fully connected layer to obtain the scale classification of coal flow features.
4. The coal flow image detection method according to claim 3, characterized in that: In the AMPFM module, the feature vector of the feature scale is sequentially passed through the first fully connected layer and ReLU activation, the second fully connected layer and Softmax normalization to obtain the probability that the coal flow feature belongs to the three different scales, and the scale classification of the coal flow feature is determined according to the probability.
5. The coal flow image detection method according to claim 1, characterized in that: Add the DFF module to the original C3k2 module, where: The feature maps output by Split and the last Botteleneck in the C3k2 module are used as the two input features of the DFF module. and ; feature and After adjusting the number of channels through 1×1×1 convolutions, the results are summed, and then spatial weights are generated through Sigmoid activation. ; feature and The feature maps are concatenated along the channel dimension, and then sequentially processed by global average pooling, convolution, and sigmoid activation to generate channel weights. ; Channel weight The concatenated feature map is multiplied element-wise, and then the number of channels is adjusted by a 1×1×1 convolution to obtain the intermediate feature. ; Spatial weight With intermediate features Element-wise multiplication yields the characteristics of fusion. And output it.
6. The coal flow image detection method according to claim 1, characterized in that: Add a CFPM module to the neck network of YOLOv11, setting it after each feature concatenation and each upsampling; the CFPM module includes: Based on the grayscale difference between the coal flow image region and the surrounding background image region, global grayscale statistics are performed on the input feature map to generate a grayscale mask matrix. The matrix In the image, points whose grayscale values exceed the grayscale range of the coal flow image are set to 0; Grayscale mask matrix The initial screening feature map is obtained by multiplying it element-wise with the original input feature map. ; Initial screening feature map First, texture information is extracted using a 3×3 depthwise separable convolution. Then, the texture gradient value is calculated using a Laplacian edge detection operator to generate a texture mask matrix. The matrix In the middle, for points whose texture gradient values are lower than a preset threshold, set them to 0; Texture mask matrix Compared with the initial screening feature map Element-wise multiplication yields the refined screening feature map. ; Fine screening feature map Compared with the initial screening feature map The residual connection is first multiplied by a weak feature compensation coefficient, and then a 1×1 depthwise separable convolution is used for channel compression and fusion. The number of channels is adjusted to be consistent with the original input feature map before output.
7. The coal flow image detection method according to claim 1, characterized in that: In the windmill convolution: Four sets of parallel convolutions are performed on the input feature map. Each set uses asymmetric padding and asymmetric convolution kernels in different directions to capture features in different directions of coal flow. Then the outputs of the four convolutions are concatenated along the channel dimension; Finally, a 2×2 convolution is used to fuse the spliced features and adjust the number of channels.
8. A computer device, characterized in that: Including memory and processor; The memory is used to store computer programs; The processor is used to execute the computer program and, in executing the computer program, implement the coal flow image detection method as described in any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that: The device contains a computer program that, when executed by a processor, causes the processor to perform the coal flow image detection method as described in any one of claims 1 to 7.
10. A computer program product, characterized in that: The method includes a computer program that, when executed by a processor, implements the coal flow image detection method as described in any one of claims 1 to 7.