Lightweight real-time detection method and system for multi-scale defects on surface of coal mine conveying belt

Abstract

This invention provides a lightweight real-time detection method and system for multi-scale defects on the surface of coal mine conveyor belts, relating to the fields of industrial vision inspection and artificial intelligence technology. The method includes real-time acquisition of image data of the coal mine conveyor belt surface; inputting the real-time image data into a pre-trained defect detection model for real-time multi-scale defect detection, and obtaining the defect detection results. The defect detection model is based on the YOLOv11 architecture and includes a backbone network with embedded dynamic multi-scale lightweight convolutional modules. These modules dynamically generate multi-scale convolutional kernels to extract multi-scale features. A neck network is also included, embedding dual-gated partial convolutional modules for selective feature fusion and enhancement based on channel grouping and pixel-level confidence. An improved bounding box regression loss function is used for optimization during the defect detection model training process. This method ensures high accuracy while achieving lightweight real-time detection of multi-scale defects.

Description

Technical Field

[0001] This invention relates to the fields of industrial visual inspection and artificial intelligence technology, and in particular to a lightweight real-time detection method and system for multi-scale defects on the surface of coal mine conveyor belts. Background Technology

[0002] Coal mine conveyor belts are core equipment for coal mining and long-distance transportation, directly bearing heavy loads and dynamic impacts. During coal transportation, conveyor belts are easily scratched by sharp foreign objects such as anchor bolts and wires, and the continuous impact of materials exacerbates the expansion of tears. With the development of conveyor systems towards higher speeds and larger scales, the probability of defects such as tears, scratches, aging, and breakage on the surface of conveyor belts has increased significantly. If these defects are not detected and addressed in a timely manner, they will not only significantly reduce the transportation efficiency of the conveyor belt and increase energy consumption, but may also lead to catastrophic accidents such as belt breakage and fires, directly threatening personnel safety and the stability of the production system.

[0003] In existing technologies, deep learning-based target detection methods (such as Faster R-CNN and YOLOv5) have been applied to industrial defect detection. However, for the specific scenario of coal mine conveyor belts, existing methods still have significant shortcomings: First, the morphology of conveyor belt defects is highly heterogeneous, ranging from minor scratches to large spallings, with a very large scale span; second, the actual working conditions are complex, and interference factors such as coal dust adhesion, non-uniform lighting, and material deformation seriously affect the robustness of detection; finally, existing high-precision models have huge computational requirements and cannot meet the real-time requirements under the dynamic operation of conveyor belts, while lightweight models are not accurate enough when dealing with multi-scale and hidden damage. Summary of the Invention

[0004] This invention provides a lightweight real-time detection method and system for multi-scale defects on the surface of coal mine conveyor belts, in order to solve the technical problems of insufficient defect detection accuracy and poor real-time performance in the prior art.

[0005] On the one hand, the present invention provides a lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts, comprising: Real-time image data of the surface of coal mine conveyor belts; The real-time image data is input into a pre-trained defect detection model to perform multi-scale real-time defect detection and obtain defect detection results. The defect detection model is based on the YOLOv11 architecture and trained using historical image data of the coal mine conveyor belt surface. The defect detection model includes: The backbone network is embedded with a dynamic multi-scale lightweight convolution module, which is used to dynamically generate multi-scale convolution kernels to extract multi-scale features. The neck network is embedded with a dual-gated partial convolutional module, which is used to selectively perform feature fusion and enhancement based on channel grouping and pixel-level confidence. The defect detection model is optimized during training using an improved bounding box regression loss function; Based on the defect detection results, the corresponding risk level is determined.

[0006] On the other hand, the present invention also provides a lightweight real-time detection system for multi-scale defects on the surface of coal mine conveyor belts, comprising: The image acquisition module is used to acquire real-time image data of the surface of the coal mine conveyor belt; The edge computing module is used to input the real-time image data into a pre-trained defect detection model to perform multi-scale real-time defect detection and obtain defect detection results. The defect detection model is based on the YOLOv11 architecture and trained using historical image data of the coal mine conveyor belt surface. The defect detection model includes: The backbone network is embedded with a dynamic multi-scale lightweight convolution module, which is used to dynamically generate multi-scale convolution kernels to extract multi-scale features. The neck network is embedded with a dual-gated partial convolutional module, which is used to selectively perform feature fusion and enhancement based on channel grouping and pixel-level confidence. The defect detection model is optimized during training using an improved bounding box regression loss function; The risk assessment module is used to determine the corresponding risk level based on the defect detection results.

[0007] This invention provides a lightweight real-time detection method and system for multi-scale defects on the surface of coal mine conveyor belts. Real-time image data is input into a defect detection model trained on a YOLOv11-based architecture. The defect detection model embeds a dynamic multi-scale lightweight convolutional module in the backbone network for dynamically generating multi-scale convolutional kernels to extract multi-scale features, and a dual-gated partial convolutional module in the neck network for selective feature fusion and enhancement based on channel grouping and pixel-level confidence. Furthermore, an improved bounding box regression loss function is used for optimization during training. This avoids the problems of high defect morphological heterogeneity, severe environmental interference, and the difficulty in balancing detection accuracy and efficiency under complex working conditions. It can perform lightweight real-time detection of multi-scale defects while maintaining high accuracy, exhibiting strong timeliness. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0009] Figure 1 This is one of the flowcharts of a lightweight real-time detection method for multi-scale defects on the surface of a coal mine conveyor belt provided in an embodiment of the present invention; Figure 2 This is the second flowchart of the lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts provided in this embodiment of the invention. Figure 3 This is a schematic diagram of the overall architecture of the DMDG-YOLOv11 model provided in this embodiment of the invention; Figure 4 This is a schematic diagram of the principle structure of the DG-PConv2 module provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the α*-MPDIoU module provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the layout of the lightweight real-time detection system provided in an embodiment of the present invention. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0011] Figure 1 This is a flowchart illustrating a lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts provided in an embodiment of the present invention.

[0012] See Figure 1 A lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts may include the following steps.

[0013] Step 110: Acquire real-time image data of the surface of the coal mine conveyor belt.

[0014] In this step, real-time image data of the surface of the coal mine conveyor belt can be acquired using an industrial camera. Typically, this step involves acquiring real-time image data of the lower surface of the coal mine conveyor belt.

[0015] Step 120: Input the real-time image data into a pre-trained defect detection model for multi-scale real-time defect detection to obtain the defect detection result.

[0016] Among them, the defect detection model is based on YOLOv11 and is trained using historical image data of the coal mine conveyor belt surface. The defect detection model includes a backbone network and a neck network.

[0017] The backbone network is embedded with a dynamic multi-scale lightweight convolution module (abbreviated as GSKW-Conv). The dynamic multi-scale lightweight convolution module is used to dynamically generate multi-scale convolution kernels to extract multi-scale features. The dynamic multi-scale lightweight convolution module (abbreviated as GSKW-Conv) replaces the original standard convolution layer.

[0018] The neck network is embedded with a double-gated partial convolution module (abbreviated as DG-PConv2). The double-gated partial convolution module is used to selectively perform feature fusion and enhancement based on channel grouping and pixel-level confidence. The double-gated partial convolution module (DG-PConv2) replaces the C3K2 module.

[0019] The defect detection model is optimized using an improved bounding box regression loss function during the training process. The improved bounding box regression loss function replaces the original bounding box regression loss function of the YOLOv11 model.

[0020] In this step, the historical image data can be enhanced through image enhancement techniques, and the enhanced historical image data is divided into a training set, a validation set, and a test set according to a preset ratio. The training set is used to train the defect detection model. The validation set and the test set are used to validate and test the defect detection model. On the NVIDIA GeForce RTX4090 GPU, the SGD optimizer is used to train the DMDG-YOLOv11 model. After training is completed, performance evaluation is carried out on the test set. The evaluation metrics can include at least one of precision (P), recall (R), mean average precision (mAP), number of parameters (Params), and computational complexity (FLOPs). When each evaluation metric reaches its corresponding preset standard threshold, the final defect detection model (DMDG-YOLOv11) is determined to be qualified.

[0021] During training, images of conveyor belt surface defects, including damage, tear, and scratch, were collected. Up to 10,000 defect images were acquired, with a resolution of 1920×1080 pixels. Enhancement operations were performed on the defect images, including scaling, center flipping, adding Gaussian and Poisson mixed noise to simulate coal dust, illumination correction based on Retinex theory, and background texture suppression filtering, to improve model robustness. The Labelme tool was used to label the defect locations and categories on the processed images, generating COCO format annotation files. The datasets were then divided into training, validation, and test sets in a 7:2:1 ratio. The defect detection model was optimized during training using an improved bounding box regression loss function, resulting in the final defect detection model (DMDG-YOLOv11).

[0022] Step 130: Determine the corresponding risk level based on the defect detection results.

[0023] In this embodiment, real-time image data is input into a defect detection model trained on the YOLOv11 architecture for detection. The defect detection model embeds a dynamic multi-scale lightweight convolutional module in the backbone network for dynamically generating multi-scale convolutional kernels to extract multi-scale features, and a dual-gated partial convolutional module in the neck network for selective feature fusion and enhancement based on channel grouping and pixel-level confidence. Furthermore, an improved bounding box regression loss function is used for optimization during training, avoiding the problems of high defect morphological heterogeneity, severe environmental interference, and difficulty in balancing detection accuracy and efficiency in complex working conditions. It can perform lightweight real-time detection of multi-scale defects while ensuring high accuracy, and has strong timeliness.

[0024] In one embodiment of this specification, the dynamic multi-scale lightweight convolution module is constructed by integrating the dynamic kernel repository mechanism of KWConv into the GSConvV2 module, and may include: kernel repository unit, feature compression unit, attention generation unit, kernel assembly unit, and convolution execution unit.

[0025] The kernel repository unit is used to store multiple sets of convolutional kernel primitives; Specifically, the nuclear warehouse unit W = {W1, W2, ..., W...} M It contains M learnable convolutional kernel primitives, each primitive having a size of k×k×C. in ×C base k is the basic core size, which can be 3; C is the basic number of channels. baseThe value of is 4-16, preferably 8; M is the kernel repository capacity, ranging from 16-64, preferably 32, and can be adjusted according to the model size. All primitives are initialized using a He normal distribution and are optimized together with other network parameters during training. The kernel repository is stored in global shared memory and is available for use by the kernel assembler of all branches. W1 to W M C represents the first to the Mth convolutional kernel primitives. in The number of channels in the input feature map.

[0026] The feature compression unit is used to perform global average pooling on the input feature map to generate a global feature vector.

[0027] The attention generation unit is used to generate dynamic attention weights based on the global feature vector and the current operating parameters; Specifically, the attention generation unit introduces a working condition perception factor and dynamically adjusts the attention weight distribution based on the real-time collected coal dust concentration and light intensity, as shown in the following formula (1): (1); in, ; These are the normalized dynamic attention weights; Branch number; The core element number; The number of nuclear primitives; For the first The branch, the first The original attention score of each kernel primitive; For operating condition sensing factors; The coal dust concentration collected in real time is normalized. The normalized value is used to collect the light intensity in real time; and These are preset hyperparameters, with values ​​ranging from 0.1 to 0.5. For example, =0.3, =0.2; For the first The branch, the first The original attention score of each kernel primitive. The attention generation unit processes the global feature vector through two fully connected layers (FC) and the ReLU activation function to obtain the original attention score a∈R. 4×M The score of each branch is Softmax normalized to obtain the attention weight vector α. i ∈R M (i=1,2,3,4). Where a is the original attention score matrix, R... 4×M It is a 4xM real matrix. α i Let be the attention weight vector for the i-th branch.

[0028] The kernel assembly unit is used to linearly combine convolutional kernel primitives from the kernel repository unit according to the dynamic attention weights to generate convolutional kernels for each branch; Specifically, the core assembly unit is mainly equipped with an independent core assembler for each branch, based on the corresponding attention weight α. i The convolution kernel for this branch is generated by linearly combining primitives from the kernel repository, as shown in formula (2) below: (2); For the first Each core element; This is the basic convolution kernel. Each branch requires a different kernel size, necessitating size adaptation of the basic kernel: Branch 1 (standard 3×3 convolution): The number of channels is increased from C using pointwise convolution (1×1 convolution). base Extend to target output channel C out We obtain K1∈R 3×3×C in ×C out Branches 2-4 (depthwise convolutions 9×9, 13×13, 17×17): Bilinear interpolation is used to enlarge the basic kernel space size from 3×3 to the target size s×s, resulting in K. i ∈R s×s×C in ×1 , where s∈{9,13,17}. K i is the large kernel depth convolution kernel adapted for the i-th branch. s is the target size.

[0029] The convolution execution unit uses the assembled convolution kernel to perform convolution operations on the input feature map; Specifically, the convolution execution unit simultaneously feeds the input feature map X into four branches: branch 1 performs standard convolution using K1; branches 2-4 perform depthwise convolution using K2, K3, and K4 respectively. Each branch convolution is followed by batch normalization and ReLU activation, and the number of channels in the output feature map is C.

[0030] The input features of the dynamic multi-scale lightweight convolution module are simultaneously fed into a KWConv branch and three large-kernel depth convolution branches. The outputs of each branch are fused after channel shuffling.

[0031] Specifically, the feature maps output from the four branches are concatenated along the channel dimension to obtain a feature map with 4C channels. To promote the full integration of standard convolutional features and deep convolutional features, a channel shuffling mechanism is adopted. The feature map is divided into four groups along the channel dimension, and cross-group information is mixed through transposition before being reconstructed back to the original dimension. The final output feature map Y∈R is obtained after channel shuffling. H×W×4CH represents the height of the output feature, and W represents the width of the output feature. Replacing all standard convolutional layers in the YOLOv11 backbone network with GSKW-Conv modules allows the detection model to dynamically adjust its convolutional kernels according to different operating conditions, improving its adaptability and robustness to various environments.

[0032] In this embodiment, a kernel repository unit is set up in the dynamic multi-scale lightweight convolution module to store multiple sets of convolution kernel primitives. The feature compression unit performs global average pooling on the input feature map to capture global context information. The attention generation unit generates dynamic attention weights based on the global feature vector and the current working condition parameters. The kernel assembly unit linearly combines the convolution kernel primitives from the kernel repository according to the weights to dynamically generate suitable convolution kernels for each branch. Finally, the convolution execution unit uses the assembled convolution kernels to perform convolution operations. At the same time, the input features are simultaneously fed into a KWConv branch and three large kernel depth convolution branches and fused through channel shuffling for output. Under the premise of reducing the number of model parameters, the dynamic adaptive generation of convolution kernels based on input features and working conditions is realized, expanding the receptive field to capture multi-scale context information and avoiding the problem that static convolution kernels are difficult to adapt to the high heterogeneity of defect morphology and large scale span.

[0033] In one embodiment of this specification, the input feature map of the dual-gated partial convolutional module is divided into two subsets along the channel dimension; The first branch performs dynamic gated convolution on one of the subsets, divides the subset into multiple channel groups, generates gated vectors through global average pooling and a gate generator, and performs weighted convolution on each channel group; The second branch performs partial convolution on another subset, generates a pixel-level confidence map through the confidence prediction subnetwork, obtains a mask by thresholding and binarization, and performs convolution only in the valid area marked by the mask. The outputs of the first branch and the second branch are concatenated along the channel dimension. The concatenated feature map is then shuffled along the channel to obtain the final output feature map.

[0034] In this embodiment, an improved dual-gated partial convolutional module (DG-PConv2) is embedded after each layer of the backbone network, with the input feature map X∈R. C×H×W Divide into the first subset X1∈R along the channel dimension. C / 2×H×W The second subset X2∈R C / 2×H×W The first subset X1 enters the dynamically gated convolution branch; the second subset X2 enters the partial convolution branch.

[0035] In the dynamically gated convolution branch, the first subset X1 is divided into 4 channel groups, denoted as X. 11 X 12 X 13 X14 Each group has C / 8 channels. Global average pooling is performed on the first subset X1 to obtain a one-dimensional vector z1∈R. C / 2 The one-dimensional vector is input into a lightweight gated generator (consisting of a fully connected layer and a sigmoid activation function), which outputs a four-dimensional gated vector g=[g1,g2,g3,g4]∈(0,1). 4 , amount g i2 This represents the importance weight of the i2th channel group. Then, a 3×3 convolution operation is performed on each channel group, and then multiplied by the corresponding gate weight. The corresponding mathematical expression is shown in the following formula (3), which is 4 groups of soft fusion: (3); Output the concatenated Y1∈R C / 2×H×W . The first in the first group Output feature maps of each branch. The kernel size is The convolution operation. The first in the first group Input feature maps for each branch.

[0036] In the partial convolutional branch, the second subset X2 is first processed by a lightweight confidence prediction sub-network to generate a confidence map. This confidence prediction sub-network consists of two 3×3 convolutional layers and a sigmoid activation function. The first layer compresses the number of input channels from C / 2 to C / 4, and the second layer outputs a single-channel feature map, which is then activated by a sigmoid function to obtain a pixel-level confidence map R∈[0,1]. H×W The confidence map R is compared with a threshold τ=0.5 and then converted into a binary mask. If the binary mask of a pixel is 1, the pixel is considered reliable; if the binary mask of a pixel is 0, the pixel is considered unreliable (e.g., areas obscured by coal dust or highly reflective areas). A partial convolution operation is performed on the second subset X2, performing standard 3×3 convolution calculations only within the valid regions (values ​​of 1) marked by the binary mask. This suppresses interference from coal dust-covered or highly reflective areas; invalid regions are set to zero and do not participate in the calculation. The output of the partial convolution, Y2∈R, is... C / 2×H×W .

[0037] The output Y1 of the first group and the output Y2 of the second group are concatenated along the channel dimension to obtain a C×H×W feature map. This is then subjected to channel shuffling (group number = 2) to promote information interaction, resulting in the final feature map. The DG-PConv2 module replaces the C3K2 module in the YOLOv11 neck network to further refine features and enhance robustness against coal dust interference.

[0038] In this embodiment, efficient and accurate feature fusion and enhancement are achieved by evenly dividing the input feature map along the channel dimension and employing two complementary lightweight processing mechanisms: the first branch uses dynamic gated convolution to weight each channel group, enabling the network to adaptively emphasize important feature channels based on global information; the second branch generates pixel-level confidence masks through a confidence prediction sub-network, performing partial convolution only in effective regions with high confidence, reducing redundant computation. After combining the two and performing channel shuffling, the computational and parameter counts of the model are reduced while improving the neck network's ability to extract key defect features and its fusion efficiency. Lightweight deployment is achieved while ensuring detection accuracy, making it particularly suitable for resource-constrained edge computing scenarios in underground coal mines.

[0039] In one embodiment of this specification, the confidence prediction subnetwork consists of two 3×3 convolutional layers; The first convolutional layer compresses the input channels; The second convolutional layer outputs a single-channel feature map, which is then activated by a sigmoid function to obtain a confidence map.

[0040] In one embodiment of this specification, the improved bounding box regression loss function is the α*-MPDIoU loss function, as shown in the following formulas (4) and (5): (4); (5); For prediction boxes; For true bounding boxes; This represents the minimum horizontal boundary distance between the predicted bounding box and the ground truth bounding box. This represents the minimum boundary distance between the predicted bounding box and the ground truth bounding box in the vertical direction. c Let be the diagonal length of the smallest bounding rectangle that encloses the predicted bounding box and the ground truth bounding box. This is a hyperparameter.

[0041] In this embodiment, the original MPDIoU is improved by incorporating the hyperparameter α*. This enhances the bounding box regression accuracy and improves the model's generalization ability without introducing additional parameters or increasing computational overhead. For well-aligned, high-quality predicted boxes, the loss function is more sensitive to optimizing bounding box accuracy, further improving localization precision. For poorly aligned, low-quality predicted boxes, the loss function appropriately reduces their optimization weights, thereby suppressing gradient perturbations caused by sample imbalance or annotation noise. This enhances the convergence speed and stability of bounding box regression, making it particularly suitable for scenarios involving large differences in the scale and irregular shapes of various defects, as well as a large number of difficult and easy samples, in the detection of surface defects on coal mine conveyor belts.

[0042] Specifically, when α*=1, the loss degenerates to the standard MPDIoU loss; when α*>1, the loss function is significantly amplified in the region of 1-MPDIoU<1 (i.e., high overlap), causing the model to focus more on sub-pixel-level boundary alignment in the later stages of training. By determining the hyperparameter α*=2.5 through grid search, the model achieves optimal localization accuracy and convergence speed. Finally, the complete DMDG-YOLOv11 detection model is obtained.

[0043] In one embodiment of this specification, real-time image data of the surface of a coal mine conveyor belt is acquired, including: Real-time image data of the surface of the coal mine conveyor belt is acquired through an image acquisition platform; The image acquisition platform includes an industrial camera, an LED light source, and a dustproof device. The industrial camera and LED light source are installed below the conveyor belt, and the industrial camera and LED light source are triggered synchronously. The lens axis of the industrial camera has a first angle with the normal to the lower surface of the conveyor belt, and the beam axis of the LED light source has a second angle with the lower surface of the conveyor belt.

[0044] In this embodiment, the first included angle can be 30° to 40°. The second included angle can be 15° to 25°. The wavelength of the LED light source is generally 850nm. The lens axis forms a first included angle with the normal to the lower surface of the conveyor belt, and the light source beam axis forms a second included angle with the lower surface of the conveyor belt, which can suppress specular reflection and highlight the morphological features of defects such as scratches and cracks. The dustproof device can mainly include a dust cover, and may also include an air curtain dust removal device. The industrial camera and LED light source are encapsulated inside the dust cover. An air curtain dust removal device is provided at the front end of the dust cover corresponding to the lens window. This air curtain dust removal device introduces clean compressed air at 0.4MPa and intermittently blows at a frequency of 2Hz, forming a continuous air curtain or air screen in front of the lens window, thereby effectively preventing coal dust from adhering to the lens surface and keeping the optical window clean.

[0045] In one embodiment of this specification, determining the corresponding risk level based on the defect detection results includes: The risk level is determined based on the defect detection results and a pre-set risk mapping table. The defect detection results include: physical dimensions, category, and spatial location; The risk mapping table contains the defect results corresponding to each risk level.

[0046] In this embodiment, if the defect is located within the conveyor belt joint area or edge area, the risk level is automatically increased by one level. Generally, the joint area can refer to a range of 5 meters before and after the joint. The edge area can refer to the range within 50mm of the edge.

[0047] In one embodiment of this specification, after the defect detection model training is completed, the method further includes: The trained defect detection model was FP16 quantized using TensorRT and deployed on an edge computing device.

[0048] Based on the same concept, the present invention also provides a lightweight real-time detection system for multi-scale defects on the surface of coal mine conveyor belts, the system including an image acquisition module, an edge computing module, and a risk assessment module.

[0049] The image acquisition module is used to acquire real-time image data of the surface of the coal mine conveyor belt; The edge computing module is used to input real-time image data into a pre-trained defect detection model to perform multi-scale real-time defect detection and obtain defect detection results. The defect detection model is based on the YOLOv11 architecture and trained using historical image data of the coal mine conveyor belt surface. The defect detection model includes: The backbone network is embedded with a dynamic multi-scale lightweight convolution module, which is used to dynamically generate multi-scale convolution kernels to extract multi-scale features. The neck network is embedded with a dual-gated partial convolutional module, which is used to selectively perform feature fusion and enhancement based on channel grouping and pixel-level confidence. The defect detection model is optimized during training using an improved bounding box regression loss function; The risk assessment module is used to determine the corresponding risk level based on the defect detection results.

[0050] In this embodiment, the lightweight real-time detection system uses the lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts described in any of the above embodiments.

[0051] To verify the comprehensive advantages of the DMDG-YOLOv11 model and system proposed in this invention, comparative and ablation experiments were conducted on the self-built CBD-2024 dataset.

[0052] The DMDG-YOLOv11 model was trained using a training set to obtain a coal conveyor belt damage detection model. The model was then tested using a test set to evaluate its performance. The training and testing of the model were conducted in the same environment to minimize interference from external factors. This embodiment's experiment was based on a 64-bit Windows 11 operating system, with PyCram Community Edition 2023.1.3 as the integrated compiler, Python 3.9.7 as the deep learning framework, and CUDA version 12.2. The computer used in the experiment had an AMD Ryzen 9 7945HX CPU with Radeon Graphics, an NVIDIA GeForce RTX4060 GPU, and 8GB of RAM.

[0053] To verify the effectiveness and stability of the DMDG-YOLOv11 model, fixed hyperparameter settings were used during both training and testing. During training, the SGD optimizer was used for optimization, with 300 epochs (at which convergence was achieved); 8 workers; a batch size of 8 to ensure the GPU could run at full power; an input image size of 640 pixels to scale and unify image dimensions, improving speed; a random seed of 1 to fix the random number and avoid errors affecting the ablation experiment; data augmentation of 1 to enable data augmentation, using gradient descent to adjust the learning rate; an initial learning rate of 0.01 and a minimum learning rate of 0.0001, with the learning rate continuously decreasing as training progressed. A larger initial learning rate saved training time, while a smaller minimum learning rate prevented the network from getting trapped in local minima.

[0054] The DMDG-YOLOv11 model was fully trained using the training and validation sets to obtain a detection model for coal conveyor belt targets. Subsequently, the detection model for damaged parts of the coal conveyor belt was tested using the test set.

[0055] The performance of the coal conveyor belt damage target detection model was evaluated. In this embodiment, in order to verify the effectiveness of the DMDG-YOLOv11 model in coal conveyor belt damage target detection, the present invention introduced evaluation indicators such as precision (P), recall (R), mean average precision (mAP), number of parameters (Params), floating point operations (FLOPs), and detection speed (Frames Per Second, FPS). The evaluation indicators are shown in the following formulas (6) to (10): (6); (7); (8); (9); (10); P is precision; TP is the number of true positive samples correctly identified by the model; FP is the number of true negative samples incorrectly predicted as positive by the model. R is recall. FN is the number of true positive samples incorrectly predicted as negative by the model. AP is the area under the P and R curves; mAP is the average AP value for all classes; N is the number of all classes. cls i3 represents the number of categories, where i3 represents a certain type of defect, and AP i3 t represents the detection accuracy for type i3 defects. preprocess This represents the image preprocessing time, including the time spent on image resizing, channel conversion, and dimensional expansion. inference The inference time of the model refers to the time it takes for the model to receive preprocessed image data as input and for the neural network to calculate and output the original prediction result. postprocess This represents the time cost of filtering and optimizing the raw predictions output by the model. This process mainly includes non-maximum suppression (NMS) and confidence filtering. k represents the k-th predicted bounding box after sorting by confidence. For prediction boxes with an index greater than k. For the first The accuracy corresponding to each prediction box. This represents the change in recall corresponding to the k-th prediction box. The curve is plotted with recall R on the x-axis and precision P on the y-axis.

[0056] To further verify the damage detection performance of the DMDG-YOLOv11 model in complex environments, this invention conducted experiments on a self-built complex scenario coal conveyor belt damage dataset and compared the experimental results with those of models such as Faster R-CNN, EfficientDet, RT-DETR, YOLOv3-tiny, and YOLOv5s. The experimental results are shown in Table 1.

[0057] Table 1 As shown in Table 1, the DMDG-YOLOv11 of this invention outperforms other comparative models in terms of accuracy (97.3%), mAP50 (97.9%), and mAP50:95 (80.1%), indicating that it has the highest accuracy in detecting conveyor belt defects under complex working conditions and maintains strong robustness even under stricter IoU thresholds. The two-stage model Faster-RCNN has an mAP50 of only 92.9% and a large number of parameters (136.7M) and a computational cost of up to 370.6 GFLOPs; the single-stage model EfficientDet has an mAP50 of only 92.1%; although the Transformer-based RT-DETR has a faster detection speed (57.5FPS), its mAP50 is only 93.5%, indicating insufficient accuracy.

[0058] The YOLO series performs well overall. YOLOv6 achieves a good balance between lightweight design (4.24M parameters) and speed (65.4 FPS). YOLOv5m achieves a recall of 96.2% and mAP50 of 95.9%, but its large parameter count (21.07M) and computational cost (49.6 GFLOPs) and inference speed of only 30.1 FPS result in insufficient real-time performance. In contrast, DMDG-YOLOv11 achieves a good balance between accuracy, lightweight design, and speed: only 2.62M parameters (12.4% of YOLOv5m); only 5.8 GFLOPs of computation (11.7% of YOLOv5m); mAP50 of 97.9%, an improvement of 1.4 and 4.6 percentage points compared to YOLOv6 and YOLOv11s, respectively; and an mAP50:95 of 80.1%, the highest among all models. Its inference speed of 66.4 FPS meets the requirements for real-time detection. In the comparison of lightweight models, DMDG-YOLOv11 reduces computational cost by 0.7 GFLOPs compared to YOLOv12n, while improving mAP50 by 4.7 percentage points with a comparable number of parameters. The model size is only 4.6 MB, making it suitable for edge deployment. In summary, DMDG-YOLOv11 achieves both high accuracy and lightweight, real-time performance, validating the effectiveness of each improved module.

[0059] To verify the effectiveness of each core design module in the DMDG-YOLOv11 model, a series of ablation experiments were conducted using a test set. In these experiments, the contribution of each module to the model performance was systematically evaluated by progressively adding the GSKW-Conv module, the DG-PConv2 module, and the α*-MPDIoU loss function. The experimental results are shown in Table 2.

[0060] Table 2 In Table 2, √ indicates that the model was used, and / indicates that it was not used. Table 2 shows that the baseline model (YOLOv11s) has 9.43M parameters, 22.1 GFLOPs of computation, an mAP50 of 93.3%, an mAP50:95 of 67.9%, and a detection speed of 48.2 FPS. After introducing the GSKW-Conv module (Experiment B), the number of parameters decreased to 2.46M (a reduction of 73.9%), the mAP50 improved by 2.4 percentage points to 95.7%, and the FPS improved by 13.3% to 54.6, verifying the lightweight and efficient nature of this module. After introducing the DG-PConv2 module (Experiment C), the mAP50 improved by 3.2 percentage points to 96.5%, and the mAP50:95 improved by 3.2 percentage points to 71.1%, indicating that this module enhances the model's robustness to complex backgrounds. After introducing the α*-MPDIoU loss function (Experiment D), mAP50:95 improved by 8.6 percentage points to 76.5%, significantly improving bounding box localization accuracy. When GSKW-Conv and DG-PConv2 were used together (Experiment E), mAP50 reached 97.2%, an improvement of 3.9 percentage points compared to the baseline, and FPS increased to 56.4, indicating that the two modules worked effectively together. The complete DMDG-YOLOv11 model (Experiment H) integrated three improved modules and achieved the best performance on the test set: P, R, mAP50, and mAP50:95 reached 97.3%, 95.8%, 97.9%, and 80.1%, respectively, an improvement of 6.4, 4.5, 4.6, and 12.2 percentage points compared to the baseline; the number of parameters was 2.62M, a reduction of 72.2% compared to the baseline; the computational cost was 5.8 GFLOPs, a reduction of 73.8% compared to the baseline; and the detection speed was 66.4 FPS, an improvement of 37.8% compared to the baseline.

[0061] In summary, each module contributes positively to the model's performance, and the complete model achieves the optimal balance between accuracy, lightweight design, and real-time performance. The above analysis demonstrates that the DMDG-YOLOv11 model proposed in this invention improves both real-time performance and detection accuracy, meeting the requirements for efficient, accurate, and stable conveyor belt damage detection in complex scenarios, thus providing a guarantee for safe underground operations.

[0062] Figure 2This is the second flowchart illustrating the lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts provided in this embodiment of the invention. Belt damage detection data acquisition: An image acquisition device for the conveyor belt surface is set up, and images are acquired using an industrial camera. A ResNet50-based classification algorithm is used to filter out low-quality images such as those with motion blur and missing targets, thereby constructing a conveyor belt damage dataset. Then, a data preprocessing stage is performed. After preprocessing, the dataset is labeled. After labeling, the data is partitioned, which may include: data augmentation (merging the original dataset with the augmented dataset) and partitioning into training, validation, and test sets at a ratio of 70%, 20%, and 10%, respectively. In the model construction part, a dynamic multi-scale lightweight convolutional module (GSKW-Conv) is embedded in the backbone network, and an improved dual-gated partial convolutional module (DG-PConv2) is embedded in the neck network, with an α*-MPDIoU loss function introduced for optimization. After training and evaluation, the model is used to detect the location of three types of damage: damage, tear, and scratch, obtaining the detection results. The test results are input into the risk level assessment model, which outputs four levels: safe, low risk, medium risk, and high risk. It also triggers real-time risk warnings and graded handling strategies, forming a complete closed-loop process of detection, assessment, warning, and handling.

[0063] Figure 3 This is a schematic diagram of the overall architecture of the DMDG-YOLOv11 model provided in this embodiment of the invention. It mainly consists of three parts: the backbone, the neck, and the head. In the backbone, the input image first passes through multiple GSKW-Conv modules for feature extraction. These modules replace the original standard convolutional layers of YOLOv11 and can dynamically generate multi-scale convolutional kernels to adapt to defect features at different scales. The input image first passes through the first layer of the GSKW-Conv module for feature extraction. Then, GSKW-Conv, C3K2, GSKW-Conv, C3K2, GSKW-Conv, C3K2, GSKW-Conv, C3K2, GSKW-Conv, C3K2 modules are connected sequentially to form a deep feature extraction path. At the end of the backbone, the SPPF module and the C2PSA module are connected sequentially to enhance the receptive field and feature representation capability.

[0064] In the neck network, a structure combining feature pyramids and path aggregation is employed for multi-scale feature fusion. Specifically, deep features are upsampled and then concatenated with shallow features, followed by feature refinement through the DG-PConv2 module. The neck network also includes an Upconv upsampling convolutional module and cascaded processing of multiple DG-PConv2 modules. The DG-PConv2 module replaces the original C3K2 module in YOLOv11, and is used for selective feature fusion and enhancement based on channel grouping and pixel-level confidence.

[0065] In the detection head, the network is set up with three detection layers, each corresponding to a feature map output at a different scale. Each detection layer is connected to a DG-PConv2 module for final feature optimization. Then, the defect detection results are output through the 11Detect convolutional layer, which enables accurate localization and recognition of various defects such as breakage, tearing, and scratches.

[0066] Figure 4 This is a schematic diagram of the DG-PConv2 module principle structure provided in this embodiment of the invention. The input feature map X has a size of 256×80×80, and is divided into a first subset X1 and a second subset X2 along the channel dimension, with each subset having 128 channels. The left side is a partial convolution branch, processing the second subset X2. The second subset X2 is input to the confidence prediction subnetwork, which consists of two 3×3 convolutional layers and a sigmoid activation function, outputting a pixel-level confidence map R with a size of 80×80 and values ​​between 0 and 1. The confidence map is converted into a binary mask after thresholding and binarization, also with a size of 80×80. Subsequently, partial convolution operations are performed on the second subset X2, performing 3×3 convolution calculations only within the effective area marked by the mask, outputting a feature map with a size of 128×80×80. The right side is a dynamically gated convolution branch, processing the first subset X1. The first subset X1 is obtained as a one-dimensional vector Z with a dimension of 128 through global average pooling. This vector is input to a lightweight gating generator, which outputs four gating weights g1 to g4. The first subset X1 is further divided into four channel groups X. 11 To X 14Each channel group has 32 channels. After performing a 3×3 convolution on each channel group, it is multiplied by the corresponding gating weights and then soft-fused with the original input. The outputs of the four branches are concatenated along the channel dimension to obtain Y1, with dimensions of 128×80×80. Finally, the outputs of some convolutional branches and the outputs of the dynamically gated convolutional branches are concatenated along the channel dimension to obtain a feature map with 256 channels. Channel shuffling is then performed to promote information interaction between the two sets of features, ultimately outputting a refined feature map. This module employs a dual-branch parallel processing mechanism. Some convolutional branches use confidence prediction to select effective regions for computation, while the dynamically gated convolutional branch adaptively adjusts the channel weights through a gating mechanism. Together, they achieve efficient feature fusion and enhancement, effectively improving the model's robustness under complex conditions such as coal dust obstruction and uneven lighting.

[0067] Figure 5 This is a schematic diagram of the α*-MPDIoU module provided in an embodiment of the present invention. The diagram contains two rectangles: the solid-lined rectangle represents the actual bounding box, and its upper left corner coordinates are (x1...). gt ,y1 gt The coordinates of the lower right corner are (x2) gt ,y2 gt The dashed box represents the prediction box, with its upper left corner at coordinates (x1...). pred ,y1 pred The coordinates of the lower right corner are (x2) pred ,y2 pred The figure highlights two key distance parameters: d1 represents the Euclidean distance between the top-left corner of the predicted bounding box and the top-left corner of the ground truth bounding box, and d2 represents the Euclidean distance between the bottom-right corner of the predicted bounding box and the bottom-right corner of the ground truth bounding box. The loss function is calculated based on the distance metric between corresponding vertices of the two bounding boxes, optimizing the alignment between the predicted and ground truth bounding boxes by minimizing d1 and d2. w represents the image width, and h represents the image height.

[0068] Figure 6This is a schematic diagram of the layout of the lightweight real-time detection system provided in this embodiment of the invention. In the conveyor aisle, a location with ample space and easy maintenance is selected below the conveyor belt 710. An industrial camera 720 (a Basler acA2000-165um NIR global shutter camera) is mounted on a rigid bracket 770, with its lens center approximately 400mm from the lower surface of the conveyor belt 710, and the lens axis forming a 35° angle with the belt surface normal, ensuring the field of view covers the entire belt width. Two high-brightness linear LED light sources 730 (wavelength 850nm) are symmetrically installed on both sides of the camera. The light source installation angle is adjusted to 20° to form low-angle grazing illumination, highlighting fine surface scratches and defects. The entire camera is encapsulated within an IP67-rated dust cover 740. An air curtain dust removal device is installed at the lens window at the front of the dust cover, introducing clean compressed air at 0.4MPa and intermittently blowing at a frequency of 2Hz to keep the window clean.

[0069] Edge computing and data transmission: Edge computing device 750, employing an NVIDIA Jetson AGX Orin processor, is installed in a nearby explosion-proof electrical control box. It connects to the camera via a gigabit Ethernet cable 760. The trained DMDG-YOLOv11 model was optimized for FP16 quantization and inference using TensorRT 8.6 and deployed on this device. The optimized model achieved an average inference latency of 14ms on the Jetson AGX Orin. The device connects to the mine's ring network via an industrial switch, communicating with the server in the central monitoring room on the ground.

[0070] Risk assessment and early warning are linked. After processing each frame of image, the DMDG-YOLOv11 model outputs the defect category, confidence level, bounding box, and pixel size. The system converts the pixel size into physical dimensions (length, area) based on the calibrated pixel equivalent (0.2mm / pixel). The system has a built-in risk mapping table (see Table 3) to initially determine the risk level (low, medium, high) based on the physical size and category of the defect. At the same time, combined with the defect's location coordinates in the image, if it is determined to be within 5 meters before or after the conveyor belt joint or within 50mm of its edge, its risk level is increased by one level.

[0071] Table 3 The final risk level and defect information are uploaded to the central monitoring system. The monitoring software interface displays the area image, defect annotation box, risk level, and location in real time. Simultaneously, the system executes preset strategies based on the risk level: Low risk: Only logs are recorded and annotated on the monitoring interface. Medium risk: Triggers on-site audible and visual alarms to alert inspection personnel and sends an alarm SMS to maintenance personnel's mobile phones. High risk: Immediately sends an emergency stop command to the conveyor belt PLC control system via the OPC UA protocol and triggers the highest level audible and visual alarm. After the emergency stop signal is issued, the system records the precise location of the defect at the time of the emergency stop (relative to the zero point of the conveyor belt encoder), providing location information for subsequent maintenance.

[0072] Through the above deployment and application scheme, this invention not only achieves high-precision, low-latency conveyor belt damage detection, but also constructs a complete industrial intelligent closed loop encompassing perception, decision-making, execution, and feedback. Compared to existing solutions that rely on high-performance servers or offline analysis, this system maintains 97.9% mAP50 detection accuracy while keeping end-to-end latency below 50ms and power consumption below 50W, truly meeting the core requirements of real-time, reliable, low-power, and easy-to-maintain systems in underground coal mines.

[0073] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units.

[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight real-time detection method for multi-scale defects on the surface of a coal mine conveyor belt, characterized in that, include: Real-time image data of the surface of coal mine conveyor belts; The real-time image data is input into a pre-trained defect detection model to perform multi-scale real-time defect detection and obtain defect detection results. The defect detection model is based on the YOLOv11 architecture and trained using historical image data of the coal mine conveyor belt surface. The defect detection model includes: The backbone network is embedded with a dynamic multi-scale lightweight convolution module, which is used to dynamically generate multi-scale convolution kernels to extract multi-scale features. The neck network is embedded with a dual-gated partial convolutional module, which is used to selectively perform feature fusion and enhancement based on channel grouping and pixel-level confidence. The defect detection model is optimized during training using an improved bounding box regression loss function; Based on the defect detection results, the corresponding risk level is determined.

2. The lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts according to claim 1, characterized in that, The dynamic multi-scale lightweight convolution module is built by integrating KWConv's dynamic kernel repository mechanism into the GSConvV2 module, and includes: The kernel repository unit is used to store multiple sets of convolutional kernel primitives; The feature compression unit is used to perform global average pooling on the input feature map to generate a global feature vector; An attention generation unit is used to generate dynamic attention weights based on the global feature vector and the current operating parameters; A kernel assembly unit is used to linearly combine convolutional kernel primitives from the kernel repository unit according to the dynamic attention weights to generate convolutional kernels for each branch; The convolution execution unit performs convolution operations on the input feature map using the assembled convolution kernel; The input features of the dynamic multi-scale lightweight convolution module are simultaneously fed into a KWConv branch and three large kernel depth convolution branches, and the outputs of each branch are fused after channel shuffling.

3. The lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts according to claim 2, characterized in that, The attention generation unit incorporates a working condition sensing factor, dynamically adjusting the attention weight distribution based on real-time collected coal dust concentration and light intensity, as shown in the following formula: ； in, ; These are the normalized dynamic attention weights; Branch number; The core element number; The number of nuclear primitives; For the first The branch, the first The original attention score of each kernel primitive; For operating condition sensing factors; This refers to the real-time concentration of coal dust. The light intensity is collected in real time; and These are the preset hyperparameters; For the first The branch, the first The raw attention score of each kernel primitive.

4. The lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts according to claim 1, characterized in that, The input feature map of the dual-gated convolutional module is divided into two subsets along the channel dimension; The first branch performs dynamic gated convolution on one of the subsets, divides the subset into multiple channel groups, generates gated vectors through global average pooling and a gate generator, and performs weighted convolution on each channel group; The second branch performs partial convolution on another subset, generates a pixel-level confidence map through the confidence prediction subnetwork, obtains a mask through threshold binarization, and performs convolution only within the valid area marked by the mask. The outputs of the first branch and the second branch are concatenated along the channel dimension. The concatenated feature map is then shuffled along the channel to obtain the final output feature map.

5. The lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts according to claim 4, characterized in that, The credibility prediction subnetwork consists of two 3×3 convolutional layers; The first convolutional layer compresses the input channels; The second convolutional layer outputs a single-channel feature map, which is then activated by a sigmoid function to obtain a confidence map.

6. The lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts according to claim 1, characterized in that, The improved bounding box regression loss function is the α*-MPDIoU loss function, and its formula is as follows: ； ； in, For prediction boxes; For true bounding boxes; This represents the minimum horizontal boundary distance between the predicted bounding box and the ground truth bounding box. This represents the minimum boundary distance between the predicted bounding box and the ground truth bounding box in the vertical direction. c Let be the diagonal length of the smallest bounding rectangle that encloses the predicted bounding box and the ground truth bounding box. This is a hyperparameter.

7. The lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts according to claim 1, characterized in that, The real-time acquisition of image data of the coal mine conveyor belt surface includes: Real-time image data of the surface of the coal mine conveyor belt is acquired through an image acquisition platform; The image acquisition platform includes an industrial camera and an LED light source; The industrial camera and the LED light source are installed below the conveyor belt, and the industrial camera and the LED light source are triggered synchronously. The industrial camera has a first angle between its lens axis and the normal to the lower surface of the conveyor belt, and the LED light source has a second angle between its beam axis and the lower surface of the conveyor belt.

8. The lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts according to claim 1, characterized in that, The step of determining the corresponding risk level based on the defect detection results includes: The risk level is determined based on the defect detection results and a preset risk mapping table. The defect detection results include: physical size, category, and spatial location; The risk mapping table contains the defect results corresponding to each risk level.

9. The lightweight real-time detection method for multi-scale defects on the surface of coal mine conveyor belts according to claim 1, characterized in that, After the defect detection model has been trained, the following is also included: The trained defect detection model was FP16 quantized using TensorRT and deployed on an edge computing device.

10. A lightweight real-time detection system for multi-scale defects on the surface of a coal mine conveyor belt, characterized in that, include: The image acquisition module is used to acquire real-time image data of the surface of the coal mine conveyor belt; The edge computing module is used to input the real-time image data into a pre-trained defect detection model to perform multi-scale real-time defect detection and obtain defect detection results. The defect detection model is based on the YOLOv11 architecture and trained using historical image data of the coal mine conveyor belt surface. The defect detection model includes: The backbone network is embedded with a dynamic multi-scale lightweight convolution module, which is used to dynamically generate multi-scale convolution kernels to extract multi-scale features. The neck network is embedded with a dual-gated partial convolutional module, which is used to selectively perform feature fusion and enhancement based on channel grouping and pixel-level confidence. The defect detection model is optimized during training using an improved bounding box regression loss function; The risk assessment module is used to determine the corresponding risk level based on the defect detection results.

Images