Foreign matter detection method for raw coal belt conveyor

Abstract

The invention discloses a raw coal belt conveyor foreign matter detection method. The method specifically comprises the steps that S1, a real-time foreign matter detection model is established; s2, belt surface foreign matter images are collected, and a belt foreign matter data set is constructed; s3, performing multi-scale feature extraction on the surface foreign matter image; s4, performing intra-scale feature interaction on the surface foreign matter features, and performing fusion after interaction to generate a foreign matter image feature sequence; and S5, using query selection perceived by IoU, selecting a fixed number of features from the foreign matter image feature sequence as an initial target query, iteratively optimizing the target query through an auxiliary prediction head, generating a bounding box and a confidence score, completing foreign matter detection and obtaining a prediction result. According to the raw coal belt conveyor foreign matter detection method provided by the invention, the problem of false detection or missing detection during foreign matter detection in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a foreign matter detection method for a raw coal belt conveyor. BACKGROUND

[0002] The belt conveyor in a coal mine is applied in actual production, including production, transfer and processing of the coal mine, and plays an important role in underground coal mine transportation and is a key equipment in coal mine transportation. In a long-term and high-load use process, large pieces of coal, large pieces of coal gangue and anchor rods and other foreign matters are easily mixed in the conveying belt for coal flow. If the large pieces of coal gangue and other foreign matters are accumulated in the coal drop port, coal stacking and coal blocking problems may occur, and the anchor rods and other foreign matters may be hooked with the components of the conveying belt, easily stuck between the groove body, the carrier roller and the belt body, causing the conveying belt to be ground through, so that the belt body appears tearing phenomenon, which seriously affects the normal coal flow transportation, not only causes unpredictable economic losses, but also may bring safety risks to the safe operation of other equipment in the working system. At present, the common foreign matter detection method for the underground conveying belt of the coal mine can be divided into two categories according to the use of the conveying belt foreign matter detection method based on deep learning according to the model design logic and the operation process: two-stage and single-stage network. The two-stage network realizes target detection through two stages, the first stage uses a candidate region generation algorithm to generate a large number of candidate regions, and the second stage applies a convolutional neural network to perform feature extraction, target classification, coordinate regression and other operations on the candidate regions to obtain the final result. Typical algorithms include R-CNN, Fast R-CNN, Faster R-CNN, Cascade R-CNN and the like. The single-stage network skips the candidate region generation stage and directly outputs the target position and target category, and the main algorithms include SSD, YOLO series and the like. Whether it is a two-stage or a single-stage algorithm, most of them rely on the non-maximum suppression (NMS) technology to remove redundant boxes. However, NMS is low in efficiency and difficult to handle in parallel, which reduces the inference speed, and at the same time, the threshold needs to be adjusted according to different scenes to avoid target loss, which increases the application complexity. After that, the DETR algorithm simplifies this process, but faces the challenge of large number of parameters. In order to optimize the parameters, various DETR variants appear. Compared with mainstream models such as the YOLO series, RT-DETR shows significant advantages in detection accuracy and speed. However, its high computational load and large number of parameters limit the deployment capability on the resource-limited equipment in the coal mine. In addition, due to the complex underground environment, there are problems such as uneven light, serious coal dust interference and image blurring caused by high-speed movement of the conveying belt, and RT-DETR still has the phenomena of missed detection and false detection in foreign matter detection, which affects the stability and accuracy of the foreign matter detection of the conveyor.

[0003] In summary, the existing conveyor belt foreign matter detection algorithm has low foreign matter recognition accuracy in complex environments, cannot meet real-time detection requirements, and cannot meet deployment requirements of mobile platforms under limited underground resources, and still has a large improvement space. SUMMARY

[0004] The purpose of the present application is to provide a raw coal belt conveyor foreign matter detection method, which solves the problem of false detection or missed detection in the prior art.

[0005] The technical solution adopted by the present application is a raw coal belt conveyor foreign matter detection method, which specifically includes the following steps: S1, establishing a real-time foreign matter detection model for a coal mine underground belt conveyor based on RT-DETR; S2, collecting foreign matter images on the belt surface of the underground belt conveyor and constructing a belt foreign matter dataset; S3, using the foreign matter detection model constructed in S1 to perform multi-scale feature extraction on the foreign matter images on the belt surface; S4, performing scale-in feature interaction on the foreign matter features extracted in S3, and generating a foreign matter image feature sequence after fusion; S5, using IoU-aware query selection to select a fixed number of features as initial target queries from the foreign matter image feature sequence generated in S4, iteratively optimizing the target queries through an auxiliary prediction head, generating a bounding box and a confidence score, completing foreign matter detection and obtaining a prediction result.

[0006] The present application also has the following characteristics: In S1, a multi-branch adaptive feature enhancement aggregation network is designed for the RT-DETR-R18 model backbone network to obtain an MSC-DETR model; the multi-branch adaptive feature enhancement aggregation network includes an efficient GPU network module, a cross-stage multi-core convolution attention module, and a multi-scale adaptive spatial attention module.

[0007] In S2, the foreign matter images include large foreign matter images and anchor rod foreign matter images.

[0008] The efficient GPU network module is composed of standard convolution and lightweight convolution, and a depth separable convolution is used to replace the traditional convolution at the front end of the efficient GPU network module.

[0009] S3 specifically performs multi-scale feature extraction using the original backbone network of the MSC-DETR model and the multi-branch adaptive feature enhancement aggregation network; The cross-stage multi-core convolution attention module includes n PKI blocks, wherein each PKA module is composed of a PKI module and a CAA module, and the input and output of the nth PKA module are and ; In the extraction process, due to the existence of different sizes of foreign matter in the coal mine underground belt conveyor, the foreign matter detection needs to locate and identify different size targets in a single image, and capture multi-scale texture features through the PKI module; the PKI module in the nth PKA block is specifically represented as follows: (1) (2) In the formula, is the local feature extracted by depth separable convolution, is the context feature extracted by the mth depth separable convolution; Then, the local and context features are fused by a convolution with a size of to represent the mutual relationship between various channels, and the specific representation is as follows: (3) In the formula, represents the output feature, convolution is used as a channel fusion mechanism to integrate features with different receptive field sizes; In order to capture long-range context information, the CAA module is further integrated into the PKA block, and the CAA module in the nth PKA block is adopted by average pooling, and then convolution is performed to obtain local region features, which is represented as: (4) In the formula, represents the average pooling operation; After obtaining the region features, two depth strip convolutions are used as an approximation of the standard large kernel depth convolution, which is represented as: (5) In the formula, the kernel size is set to ; Finally, the CAA module generates attention weights through convolution, which is represented as: (6) (7) In the formula, function ensures that the attention map is within the range (0, 1), represents element-wise multiplication, represents element-wise summation, to enhance the feature; the output of the nth PKA block is obtained by the following formula: (9) wherein, denotes the enhanced feature; After the high-efficiency GPU network module and the cross-stage multi-core convolution attention module learn the features of the input data respectively, a multi-scale adaptive spatial attention module is used for feature fusion.

[0010] S4 is specifically: performing scale-in feature interaction on the P5 layer in the MSC-DETR model backbone network, introducing a spectrum enhancement feedforward network for feature fusion after the interaction, inputting the features fused by the spectrum enhancement feedforward network into a multi-scale adaptive spatial attention module to generate an image feature sequence.

[0011] The feature fusion process of the spectrum enhancement feedforward network is as follows: (8) (9) (10) (11) (12) wherein, is a 3*3 hollow depth separable convolution, denotes an interpolated feature map of the spectrum domain weight, is a bias feature map, is a SiLU activation.

[0012] The cross-scale feature fusion network includes a rectangular self-calibration module, a pyramid context extraction module, a dynamic interpolation fusion module, a multi-path feature fusion module and an intensity enhancement module.

[0013] The rectangular self-calibration module is specifically: by horizontal and vertical pooling operations, the global context information of the input image with foreign matter in the horizontal and vertical directions is extracted, such as the input feature map is After horizontal and vertical pooling, the axial feature vector is obtained, that is, , , two axial vectors are fused by broadcast addition to form a rectangular attention region; then two large kernel convolutions are used to adjust in horizontal and vertical directions respectively; first, convolution is performed in the horizontal direction, then ReLU nonlinear activation is performed, and convolution is performed in the vertical direction, then ReLU nonlinear activation is performed to form a self-calibrated foreground region, as shown in formula (13): (13) wherein, represents a large kernel strip convolution, represents a kernel size of strip convolution, represents batch normalization after a ReLU function, represents a function; Finally, the self-calibrated rectangular attention feature is fused with the input band-face foreign feature map to enhance the feature expression of the foreground region, as shown in formula (14): (14) wherein, represents a deep convolution with a kernel of 3x3, is an attention feature, represents element-wise multiplication.

[0014] The pyramid context extraction module is specifically: first, the input multi-scale band-face foreign feature map is down-sampled to construct a pyramid structure; the multi-scale band-face foreign feature map comes from different levels of the feature extraction network, and is represented as , , ; after completing the construction of the pyramid structure, further multi-scale context information is extracted by stacking multiple rectangular self-calibration modules; the RCM module captures global context and performs spatial reconstruction on each layer of element map, ensuring that elements at each scale in the pyramid structure can benefit from global information, and the process is specifically represented as: (15) wherein, represents a feature average pooling with a factor ; The dynamic interpolation fusion module is specifically: the alignment of scales and channels is realized by interpolation and convolution operation, and then feature fusion is completed by element-wise addition; during forward propagation, for the input band-face foreign feature map , first, the low-resolution feature map is adjusted to the same spatial resolution as by bilinear interpolation; then, the interpolated is adjusted to the same spatial resolution as Having the same number of channels to obtain As shown in formula (16): (16) in, and This represents feature maps of surfaced foreign objects at two different scales. Interpolate is a bilinear interpolation algorithm. It's a 1×1 convolution; the result is added element by element. The fused feature map is obtained as shown in formula (17): (17) The multi-path feature fusion module specifically works as follows: for the input feature mapping First, depthwise separable convolution is used, and then the low-resolution feature map after convolution is normalized using the activation function h_sigmoid to control the range of the signal, as shown in formula (18): (18) in, It is a low-resolution feature map. It is a high-resolution feature map, and then bilinear interpolation is applied to... Adjust to The resolution allows the two to be multiplied element-wise, specifically: (19).

[0015] Compared with the prior art, the beneficial effects of the present invention are: (1) The foreign object detection method for raw coal belt conveyors provided by this invention has high accuracy and is adaptable to complex environments. This invention adopts a detection model based on RT-DETR, combined with a multi-branch adaptive feature enhancement aggregation network, which can effectively solve the problems of uneven lighting, severe coal dust interference, and image blurring caused by high-speed movement of the conveyor belt in coal mines. Through multi-scale feature extraction and cross-stage multi-kernel convolutional attention modules, the model can accurately capture the local and global features of foreign objects of different sizes, significantly reduce missed detections and false detections, and improve the accuracy and stability of foreign object detection.

[0016] (2) The foreign object detection method for raw coal belt conveyors provided by this invention has strong real-time performance and meets industrial needs. Through the design of efficient GPU network modules and depthwise separable convolution, the computational load and number of parameters of the model are greatly reduced, the detection speed is improved, and the algorithm can run efficiently on resource-constrained downhole equipment.

[0017] (3) The foreign object detection method for raw coal belt conveyors provided by this invention has strong multi-scale feature fusion capability. By introducing a spectrum enhancement feedforward network and a cross-scale feature fusion network, and through multi-path feature fusion and dynamic interpolation technology, feature information of different scales is effectively integrated. Compared with the single-scale feature extraction in traditional methods, this invention can more comprehensively capture the complex relationship between foreign objects and the background, and is especially suitable for detecting foreign objects with large morphological differences such as large pieces of coal gangue and anchor bolts.

[0018] (4) The foreign object detection method for raw coal belt conveyors provided by this invention has excellent generalization performance. The model uses diverse datasets during training to cover foreign object images under different light source conditions, which enhances the algorithm's adaptability to complex underground environments. In addition, the introduction of multi-branch design and attention mechanism reduces the risk of overfitting, enabling the model to maintain high detection performance in unknown scenarios.

[0019] (5) The foreign object detection method for raw coal belt conveyors provided by this invention is flexible in deployment and low in cost. Through lightweight design and modular structure, this invention reduces hardware resource requirements and is suitable for deployment on edge computing devices. Compared with radiation detection methods that rely on high-cost detectors or are harmful to the human body, this invention is vision-based, requires no additional hardware support, and significantly reduces implementation and maintenance costs. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the foreign object detection method for a raw coal belt conveyor according to the present invention. Figure 2 This is a schematic diagram of the structure of the MSC-DETR of the present invention; Figure 3 This is a schematic diagram of the structure of the multi-branch adaptive feature enhancement aggregation network of the present invention; Figure 4 This is a schematic diagram of the structure of the high-efficiency GPU network module of the present invention; Figure 5 This is a schematic diagram of the structure of the multi-scale adaptive spatial attention module of the present invention; Figure 6 This is a schematic diagram of the spectrum enhancement feedforward module of the present invention; Figure 7 This is a schematic diagram of the feature-enhanced aggregation pyramid network of the present invention; Figure 8 This is a structural schematic diagram of the strength enhancement module IELC3 of the present invention.

[0021] Figure 9 This is a visual comparison chart of the algorithms in Comparative Example 1 of this invention. Detailed Implementation

[0022] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The described embodiments are only some embodiments of the present invention, and not all embodiments.

[0023] Example 1 This invention provides a method for detecting foreign objects in a raw coal belt conveyor, such as... Figure 1 As shown, the specific steps include: S1. Establish a real-time foreign object detection model for underground belt conveyors in coal mines based on RT-DETR; S2, Collect images of foreign objects on the surface of the underground belt conveyor and construct a belt foreign object dataset; S3 utilizes the foreign object detection model constructed in S1 to extract multi-scale features from surface foreign object images; S4. The features extracted in S3 are interacted within the scale and then fused to generate an image feature sequence. S5 uses IoU-aware query selection to select a fixed number of features from the image feature sequence generated in S4 as the initial target query. The target query is then iteratively optimized by an auxiliary prediction head to generate bounding boxes and confidence scores, thereby completing foreign object detection and obtaining prediction results.

[0024] Example 2 Based on Example 1, the foreign object detection model in S1 adopts the RT-DETR-R18 model; the backbone network of the RT-DETR-R18 model is designed with a multi-branch adaptive feature enhancement aggregation network (MAFEA); the multi-branch adaptive feature enhancement aggregation network includes an efficient GPU network module (HGBlock), a cross-stage multi-kernel convolutional attention module (CPKA), and a multi-scale adaptive spatial attention module (MASAG), resulting in the MSC-DETR model.

[0025] In this embodiment, a multi-branch adaptive feature enhancement aggregation network (MAFEA) is designed to improve the backbone network structure. As shown, a dual-backbone design combines two different backbone systems to process the input data together. Dynamic alignment and fusion address the information gap problem between different feature sources, enabling the model to capture the features of the input foreign object image more comprehensively. MAFEA includes three key strategies: a high-performance GPU network module (High Performance GPU Network V2, HGBlock), a cross-stage multi-kernel convolutional attention module (CPKA), and a multi-scale adaptive spatial attention gate (MASAG). These strategies aim to enhance the ability to model global dependencies during the feature extraction stage and reduce computational resource consumption. HGBlock extracts information at different levels step by step through hierarchical graph convolutional layers, progressively enhancing feature representation from local to global. This hierarchical structure effectively captures the complex relationships between different regions in foreign object images, enhancing the network's ability to perceive features at different scales. CPKA utilizes undilated Inception-style Depth-wise convolutions to extract multi-scale texture features across different receptive fields, improving the adaptive ability of feature extraction and the ability to learn context. MASAG uses spatial selection to dynamically adjust the receptive field, prioritizing basic features. At the same time, it enriches feature maps with local details and global background using spatial interaction and cross-modulation to enhance feature extraction performance.

[0026] Example 3 Building upon Example 2, the dataset in S2 was collected and released by the Detection and Pattern Recognition Research Center team at China University of Mining and Technology. This benchmark dataset was collected from the transportation environment of conveyor belts in underground mines, containing 6000 images of foreign objects under different lighting conditions. The foreign objects were divided into three categories: large block images, anchor bolt images, and normal images, with 2000 images in each category. This included 1600 training images and 400 test images. During the experiment, the dataset was divided into training and validation sets in an 8:2 ratio for training and testing.

[0027] Example 4 In this embodiment, S3 specifically refers to: the high-efficiency GPU network module consists of standard convolution and lightweight convolution (LightConvolution, LightConv). The structure of HGBlock is as follows: Figure 4As shown, HGBlock comprises multiple convolutional layers with varying filter sizes, primarily designed to capture diverse features from the input foreign object image. Replacing traditional convolutions with depthwise separable convolutions (DWConv) at the front end of HGBlock reduces computational load, improves inference speed, and allows for independent feature learning on each channel, effectively reducing the number of parameters and mitigating the risk of overfitting, thus enhancing generalization ability. It is particularly suitable for deployment in resource-constrained environments, such as edge computing scenarios in underground coal mines.

[0028] In the CPKA module, there are n PKI blocks. Each PKA module consists of one PKI module and one CAA module. It integrates rich gradient information from different levels through a Cross-stage Partial (CSP) strategy, enabling the extraction of various features, such as... Figure 3 As shown in (b) and (c), the input and output of the nth PKA module are respectively... and .

[0029] Because coal mine underground belt conveyors contain foreign objects of varying sizes, the goal of foreign object detection is to locate and identify targets of different sizes within a single image. To address the challenge of large object scale variations, we introduce a PKI module to capture multi-scale texture features of the image. For example... Figure 3 As shown in (d), the PKI module is a receptive module, consisting of a small kernel convolution to capture local information, followed by a set of parallel depthwise convolutions to capture contextual information at multiple scales. The PKI module within the nth PKA block can be mathematically represented as follows: (1) (2) In the formula, Through Local features extracted by depthwise separable convolution. Through the mth Contextual features are extracted using depthwise separable convolutions; furthermore, this PKI module does not use dilated convolutions, thus preventing the extraction of overly sparse feature representations.

[0030] Then, through a size of The convolutional fusion of local and contextual features represents the interrelationships between various channels, specifically as follows: (3) In the formula, Indicates output features, Convolution is used as a channel fusion mechanism to integrate features with different receptive field sizes; Convolution is used as a channel fusion mechanism to integrate features with different receptive field sizes. In this way, the PKI module can capture extensive contextual information without compromising the integrity of local texture features.

[0031] As mentioned above, the initial PKI module in the PKA block focuses on extracting multi-scale local contextual information from foreign object images. To capture long-range contextual information, we further integrate a Context Anchor Attention (CAA) module into the PKA block. The purpose of CAA is to grasp the interdependencies between contexts while expanding the central functionality. A diagram of CAA is shown below. Figure 3 As shown in (e). Taking the CAA module in the nth PKA block as an example, we use average pooling, and then perform... Convolution yields local region features: (4) In the formula, This indicates an average pooling operation; After obtaining the region features, two depthwise strip convolutions are used as an approximation of the standard large kernel depthwise convolution, expressed as: (5) Depth striping convolution was chosen based on two main considerations. First, depth striping convolution is lightweight. Compared to traditional... Compared to two-dimensional depth-direction convolution, a similar effect can be achieved using a one-to-one one-dimensional depth-direction kernel, while reducing the number of parameters. Secondly, depthwise strip convolution can conveniently identify and extract features from objects with slender shapes, such as anchor bolts. As the CAA module delves deeper into the PKA module, in order to increase the receptive field of the CAA module, settings are... That is: to reduce the size of the nucleus Calculated as PKA block depth This design enhances CPKA's ability to establish relationships between long-distance pixels, and due to the strip depth design, it does not significantly increase computational cost.

[0032] Finally, the CAA module generates an attention weight. This is further used to enhance the output of the PKA module, and is represented as... (6) (7) In the formula, Function ensures attention mapping Within the range (0, 1), This represents element-wise multiplication. This indicates element-wise summation. To enhance the features, the output of the nth PKA block is obtained using the following formula: (8) In the formula, Indicates enhanced features; The multi-scale adaptive spatial attention module MASAG performs feature fusion in stages p3, p4, and p5 after CPKA and HGBlock learn the features of the input data respectively, in order to reduce the information differences between different feature sources.

[0033] MASAG comprises four stages: multi-scale fusion, spatial selection, spatial interaction and cross-modulation, and recalibration. Multi-scale fusion aggregates the outputs of HGBlock and CPKA, Feature1 and Feature2, in a semantically similar manner, preparing the fused feature maps for subsequent processing. Spatial selection dynamically adjusts the receptive field to prioritize fundamental features of the foreign object image, while spatial interaction and cross-modulation enrich the feature maps with local details and global background. In the recalibration stage, dynamically generated selective attention maps from MASAG are used to refine the initial foreign object feature maps, ensuring accurate and reliable feature fusion. Figure 5 As shown.

[0034] Example 5 In this embodiment, S4 specifically involves: performing intra-scale feature interaction on the P5 layer of the RT-DETR-R18 model backbone network, inputting the foreign object features into the feature enhancement aggregation pyramid network, introducing a spectral enhancement feedforward network for feature fusion after interaction, and inputting the features fused by the spectral enhancement feedforward network into the multi-scale adaptive spatial attention module to generate an image feature sequence.

[0035] Among them, the AIFI (Attention-based Intra-scale Feature Interaction) module in the RT-DETR model effectively enhances the model's expressive power by fusing multi-level foreign object features. However, the feedforward network (FFN) part in the AIFI module still has certain limitations. Specifically, the FFN in the traditional AIFI module usually adopts single-scale or single-range convolution operations. Although it can enhance local contextual information to a certain extent, it is difficult to fully capture the diverse features of foreign objects and background at different scales and frequencies for foreign object detection tasks in the complex environment of underground coal mines. In particular, the surface of belt conveyors has a rich variety of foreign objects with large morphological differences and a complex background environment. The single-scale FFN structure often ignores the complex spatial and spectral relationships between foreign objects and background, resulting in limited detection performance.

[0036] To address the aforementioned shortcomings, this embodiment proposes a spectrum-enhanced feed-forward module, AIFI-SEFFN (SEFFN), to enhance feature representation capabilities and achieve efficient identification of foreign objects on belt conveyors, such as... Figure 6 As shown.

[0037] The AIFI-SEFFN module integrates two different multi-scale depthwise separable convolutional paths to fully capture spatial information across a wide range. Furthermore, the module leverages the property that spatial domain convolution is equivalent to element-wise multiplication in the spectral domain, thus generating the concept of the spectral enhancement feedforward module AIFI-SEFFN. In this module, a pair of weights and biases is resized and broadcast, acting as a biased filter on spectral domain features. In this way, each part representing a frequency component can be adaptively enhanced or filtered, effectively separating background noise from foreign features.

[0038] Given input tensor First, pointwise convolution is used to expand the number of channels. The expanded tensor is fed into two parallel branches, employing 3×3 depthwise separable convolutions and 3×3 dilated depthwise separable convolutions to enhance the extraction of multi-distance information throughout the feature transformation process. Subsequently, the features from both branches are transformed into the spectral domain and enhanced through a pair of learnable weights and bias variables, thereby improving the expressive power of spectral domain information. These weights have a predefined size. However, the size can be adjusted by interpolation to suit the shape of the input foreign object feature, and it acts as an adaptive frequency pass filter to effectively separate background noise from the foreign object feature. After the gating mechanism, the activation output of one branch with a longer receptive field acts as the gating unit for another branch. Therefore, the complete feature fusion process within the AIFI-SEFFN module can be expressed as the following formula: (9) (10) (11) (12) in, It is a 3×3 hole-depth separable convolution. The interpolation feature map represents the weights in the spectral domain. It is a bias feature map. It is activated by SiLU.

[0039] Example 6 Building upon Example 5, this example presents a cross-scale feature fusion network. In the original RT-DETR model, the Cross-Scale Feature Fusion Module (CCFF) is responsible for fusing multi-scale features, aiming to integrate foreign object features from different scales. However, current CCFF feature fusion methods mainly rely on simple feature stacking or layer-by-layer fusion, which struggles to effectively capture multi-scale contextual information, especially when the target size varies significantly. To address these issues, this example designs a Context-guided Feature Enhancement Aggregation Pyramid Network (CEAFPN) to improve the model's multi-scale feature representation capabilities and enhance its ability to identify targets in complex backgrounds. The CEAFPN module includes the following core modules: Rectangular Self-calibration Module (RCM), Pyramid Context Extraction Module (PCE), Dynamic Interpolation Fusion (DIF), Multi-Path Feature Fusion (MFF), and Intensity Enhancement Module (IELC3). The specific structure of CEAFPN is as follows: Figure 7 As shown. The following will describe each structure in detail.

[0040] Specifically, the Rectangular Self-Calibration Module (RCM) extracts global contextual information from the input feature map in the horizontal and vertical directions through horizontal and vertical pooling operations, respectively. This operation effectively models the key features of foreign objects within a rectangular region. If the input feature map is... After horizontal pooling and vertical pooling, the axial eigenvectors are obtained, i.e. , The two axial vectors are merged by broadcast addition to form a rectangular region of interest.

[0041] Then, to make the generated rectangular region of interest closer to the foreground features, RCM uses two large-kernel convolutions, adjusted in the horizontal and vertical directions respectively. First, a convolution is performed in the horizontal direction, followed by ReLU non-linear activation, and the same operation is applied in the vertical direction to form a self-calibrated foreground region, as shown in the following equation: (13) in, This represents large-kernel strip convolution. This indicates the kernel size of the strip convolution. This indicates the batch normalization following the ReLU function. express function.

[0042] Finally, the self-calibrated rectangular attention features are fused with the input feature map (element-wise multiplication) to enhance the feature representation of the foreground region, as shown in the following equation: (14) in, This indicates a depthwise convolution with a 3×3 kernel. These are the attention features obtained in the previous step. This represents element-wise multiplication.

[0043] The Pyramid Context Extraction (PCE) module works by first downsampling the input multi-scale feature maps to construct a pyramid structure. In RT-DETR, the input surface-mount feature maps come from different levels of the feature extraction network, represented as follows: , , After constructing the pyramid structure, the PCE module further extracts multi-scale contextual information by stacking multiple RCM modules. The RCM modules capture the global context and perform spatial reconstruction on the feature map at each level, ensuring that features at every scale within the pyramid structure benefit from the global information. The application of multi-layer RCM improves the expressive power of features at various scales and enhances the model's context-awareness. These features contain rich spatial and semantic contextual information, providing multi-scale support for subsequent detection and segmentation tasks. This process can be represented by the following equation: (15) in, Representation of features With factors The average collection.

[0044] Dynamic Interpolation Fusion (DIF) works as follows: The DIF module is primarily used to integrate feature maps with different numbers of channels and spatial resolutions. This module achieves scale and channel alignment through interpolation and convolution operations, and then completes feature fusion by element-wise addition. During forward propagation, for the input feature map... First, the low-resolution feature map is transformed using bilinear interpolation. Adjusted to be with Same spatial resolution. Then, the interpolation is performed using a 1×1 convolution. Adjusted to be with Having the same number of channels to obtain As shown in the following equation: (16) in, and Representing feature maps of surfaced foreign objects at two different scales, Interpolate is a bilinear interpolation algorithm. It's a 1×1 convolution. The result is added element by element. To obtain the fused feature map, as shown in the following formula: (17) Multi-path Feature Fusion (MFF) works by using an adaptive activation signal from a low-resolution feature map to guide a high-resolution feature map, effectively fusing details and context, and is suitable for cross-stage adaptive fusion. For the input feature map... First, depthwise separable convolutions are performed, and then the low-resolution feature map after convolution is normalized using the activation function h_sigmoid to control the range of the signal, according to the following equation: (18) in, It is a low-resolution feature map. It is a high-resolution feature map. Then bilinear interpolation is applied to... Adjust to The resolution allows for element-wise multiplication. The overall formula can be summarized as follows: (19) The Intensity Enhancement Module (IELC3) works as follows: In RT-DETR, the original RepC3 module is used to further extract features from the obtained foreign object feature map. While the reparameterization of the RepC3 module speeds up its inference, it primarily relies on standard convolutional kernels for feature extraction. This limits its ability to handle spatially complex data and hinders the model's performance in capturing long-term dependencies. These limitations become particularly pronounced when processing high-resolution images or video streams, as these scenarios require the model to have a stronger ability to learn multi-scale features.

[0045] To overcome these limitations, this invention proposes the IELC3 module, which enhances feature fusion performance by introducing an Intensity Enhancement Layer (IEL). The IELC3 module includes two 1 × 1 convolutions to adjust the number of channels and uses three sequential IEL modules for feature fusion. Subsequently, the outputs of the two paths are merged through element-wise addition, as follows: Figure 8 As shown.

[0046] The IELC3 module uses 1x1 convolutions to adjust the number of input channels, mapping the original features to a higher spatial dimension and providing richer features for subsequent convolution operations. Secondly, the input features are divided into two parts by depthwise separable convolutions, and each part is processed separately. This helps to separate local and global information in the foreign object feature map, improving detail extraction capabilities. The two output foreign object feature maps are then processed again by two depthwise separable convolutions, with the Tanh activation function applied to both parts to enhance non-linear feature representation. Next, residual connections are used to ensure the flow of important information and avoid gradient vanishing. The two processed feature maps are then multiplied element-wise for feature fusion. Through the interaction of local and global information, the feature representation capability is further enhanced, helping the model better identify targets of different sizes and shapes. Finally, the fused features are adjusted to the same dimensions as the original input features, generating the final output features.

[0047] The flexibility and efficiency of the IELC3 module demonstrate significant advantages in processing multi-scale data, especially in complex vision tasks that require consideration of both global and local features. By introducing the IELC3 module, the MSC-DETR model achieves more accurate feature fusion while maintaining efficient inference, thereby improving the performance of foreign object detection tasks in underground coal mines.

[0048] Comparative Example 1 In complex environments, underground coal mines are affected by various factors, such as low illumination, uneven lighting, water vapor and dust interference, and overlapping and obstruction of objects. This comparative study evaluates the model's performance in this scenario. The test results show that the present invention can accurately identify various foreign objects in the real underground coal mine belt conveyor production environment, demonstrating good robustness. A visual comparison of the various algorithms is shown below. Figure 9 As shown.

[0049] This paper compares the present invention with several existing advanced target detection algorithms (YOLOv5m, YOLOv8m, YOLO12m, RT-DETR-1, RT-DETR-R18, etc.) in the production environment of a coal mine belt conveyor, using foreign object images for target detection, reflecting the detection performance of each algorithm in these images. The present invention can accurately identify and select small and blurry foreign objects, and comprehensively detect large blocks and anchor rods in the original image. Especially when there is a lot of noise and complex background in the image, it can clearly distinguish the target object. In contrast, the existing YOLO series and RT-DETR series algorithms have missed detections and false detections in some experiments. For example, YOLOv5m and YOLOv8m failed to accurately identify small foreign objects in the image, resulting in large deviations in the target bounding box or complete omission of the target. Meanwhile, compared with YOLOv5m and YOLO12m, the target bounding box localization accuracy of the method of this invention is significantly improved, especially when foreign objects are closely arranged or partially obscured. For example, the target bounding boxes of YOLOv5m and YOLO12m sometimes deviate significantly from the actual target boundary, while MSC-DETR can locate each target more accurately, and the bounding box selection results are tighter and more accurate. Furthermore, although the method of this invention has a significant improvement in foreign object detection accuracy, it also demonstrates good computational efficiency in terms of computational resource consumption, ensuring high detection performance while significantly reducing computational overhead. This makes it particularly suitable for practical applications in the special and resource-constrained environment of underground coal mines.

[0050] In summary, the improved method proposed in this invention surpasses existing advanced algorithms in several aspects, not only improving the accuracy and precision of foreign object detection on underground belt conveyors in coal mines, but also demonstrating stronger robustness in complex environments. Therefore, this invention has high practical application value in foreign object detection tasks in underground coal mines.

Claims

1. A method for detecting foreign objects in a raw coal belt conveyor, characterized in that, Specifically, the following steps are included: S1. Establish a real-time foreign object detection model for underground belt conveyors in coal mines based on RT-DETR; S2, Collect images of foreign objects on the surface of the underground belt conveyor and construct a belt foreign object dataset; S3 utilizes the foreign object detection model constructed in S1 to extract multi-scale features from surface foreign object images; S4. The features extracted in S3 are interacted within the scale and then fused to generate an image feature sequence. S5 uses IoU-aware query selection to select a fixed number of features from the image feature sequence generated in S4 as the initial target query. The target query is then iteratively optimized by an auxiliary prediction head to generate bounding boxes and confidence scores, thereby completing foreign object detection and obtaining prediction results.

2. The method for detecting foreign objects in a raw coal belt conveyor according to claim 1, characterized in that, A multi-branch adaptive feature enhancement aggregation network is designed for the backbone network of the RT-DETR-R18 model in S1 to obtain the MSC-DETR model; the multi-branch adaptive feature enhancement aggregation network includes an efficient GPU network module, a cross-stage multi-kernel convolutional attention module, and a multi-scale adaptive spatial attention module.

3. The method for detecting foreign objects in a raw coal belt conveyor according to claim 2, characterized in that, The foreign object images mentioned in S2 include images of large foreign objects and images of anchor bolt foreign objects.

4. The method for detecting foreign objects in a raw coal belt conveyor according to claim 2, characterized in that, The high-efficiency GPU network module consists of standard convolution and lightweight convolution, with depthwise separable convolution replacing traditional convolution at the very front of the high-efficiency GPU network module.

5. The method for detecting foreign objects in a raw coal belt conveyor according to claim 3, characterized in that, S3 specifically involves: using the MSC-DETR model backbone network and a multi-branch adaptive feature enhancement aggregation network for multi-scale feature extraction; The cross-stage multi-kernel convolutional attention module includes n PKI blocks, where each PKA module consists of one PKI module and one CAA module. The input and output of the nth PKA module are respectively... and ; During the extraction process, due to the presence of foreign objects of varying sizes in the underground coal mine conveyor belt, foreign object detection requires locating and identifying targets of different sizes within a single image. This is achieved by capturing multi-scale texture features using a PKI module. The PKI module within the nth PKA block is specifically represented as follows: （1） （2） In the formula, Through Local features extracted by depthwise separable convolution. Through the mth Contextual features extracted by depthwise separable convolution; Then, through a size of The convolutional fusion of local and contextual features represents the interrelationships between various channels, specifically as follows: （3） In the formula, Indicates output features, Convolution is used as a channel fusion mechanism to integrate features with different receptive field sizes; To capture remote context information, the CAA module is further integrated into the PKA block. The CAA module in the nth PKA block is then subjected to average pooling, and then... Convolution yields local region features, specifically: （4） In the formula, This indicates an average pooling operation; The local region features are obtained by using two depthwise strip convolutions as an approximation of the standard large kernel depthwise convolution, expressed as: （5） In the formula, the kernel size is set to ; Finally, the CAA module passes through Convolution generates attention weights , is represented as: （6） （7） In the formula, Function ensures attention mapping Within the range (0, 1), This represents element-wise multiplication. This indicates element-wise summation. To enhance the features, the output of the nth PKA block is obtained using the following formula: （8） In the formula, Indicates enhanced features; After the high-efficiency GPU network module and the cross-stage multi-kernel convolutional attention module learn the features of the input data respectively, a multi-scale adaptive spatial attention module is used for feature fusion.

6. The method for detecting foreign objects in a raw coal belt conveyor according to claim 3, characterized in that, Specifically, S4 involves performing intra-scale feature interaction on the P5 layer of the MSC-DETR model backbone network, inputting the features into the feature enhancement aggregation pyramid network, introducing the spectral enhancement feedforward network for feature fusion after interaction, and inputting the fused features from the spectral enhancement feedforward network into the multi-scale adaptive spatial attention module to generate an image feature sequence.

7. The method for detecting foreign objects in a raw coal belt conveyor according to claim 3, characterized in that, The feature fusion process of the spectrum-enhanced feedforward network is as follows: （9） （10） （11） （12） In the formula, It is a 3×3 hole-depth separable convolution. The interpolation feature map represents the weights in the spectral domain. It is a bias feature map. It is activated by SiLU.

8. The method for detecting foreign objects in a raw coal belt conveyor according to claim 3, characterized in that, The cross-scale feature fusion network includes a rectangular self-calibration module, a pyramid context extraction module, a dynamic interpolation fusion module, a multi-path feature fusion module, and an intensity enhancement module; The rectangular self-calibration module specifically extracts global contextual information of the input image with a planar foreign object in the horizontal and vertical directions through horizontal and vertical pooling operations, such as the input feature mapping as follows: After horizontal pooling and vertical pooling, the axial eigenvectors are obtained, i.e. , The two axial vectors are fused by broadcast addition to form a rectangular region of interest; then two large kernel convolutions are used to adjust the region in the horizontal and vertical directions respectively; first, convolution is performed in the horizontal direction and then non-linearly activated by ReLU, and convolution is performed in the vertical direction and then non-linearly activated by ReLU to form a self-calibrated foreground region, as shown in Equation (13): （13） in, This represents large-kernel strip convolution. This indicates the kernel size of the strip convolution. This indicates the batch normalization following the ReLU function. express function; Finally, the self-calibrated rectangular attention features are fused with the input feature map of the surface foreign object to enhance the feature representation of the foreground region, as shown in Equation (14): （14） in, This indicates a depthwise convolution with a 3×3 kernel. It is an attention feature. This represents element-wise multiplication.

9. The method for detecting foreign objects in a raw coal belt conveyor according to claim 8, characterized in that, The pyramid context extraction module specifically involves: firstly, downsampling the input multi-scale surface foreign object feature map to construct a pyramid structure; the multi-scale surface foreign object feature map comes from different levels of the feature extraction network, represented as follows: , , After constructing the pyramid structure, multiple rectangular self-calibration modules are stacked to extract multi-scale contextual information. The RCM module captures the global context and performs spatial reconstruction on the feature map at each level, ensuring that features at each scale in the pyramid structure can benefit from the global information. This process is specifically represented as follows: （15） in, Representation of features With factors The average collection; The dynamic interpolation fusion module specifically works by: aligning scale and channels through interpolation and convolution operations, and then completing feature fusion through element-wise addition; during forward propagation, for the input feature map of the surface foreign object... First, the low-resolution feature map is transformed using bilinear interpolation. Adjusted to be with Same spatial resolution; then, the interpolation is performed using a 1×1 convolution. Adjusted to be with Having the same number of channels to obtain As shown in formula (16): （16） in, and This represents feature maps of surfaced foreign objects at two different scales. Interpolate is a bilinear interpolation algorithm. It's a 1×1 convolution; the result is added element by element. The fused feature map is obtained as shown in formula (17): （17） The multi-path feature fusion module specifically performs the following: for the input feature mapping... First, depthwise separable convolution is used, and then the low-resolution feature map after convolution is normalized using the activation function h_sigmoid to control the range of the signal, as shown in formula (18): （18） in, It is a low-resolution feature map. It is a high-resolution feature map, and then bilinear interpolation is applied to... Adjust to The resolution allows the two to be multiplied element-wise, specifically: （19）。

Images