Methods and systems for detecting coal gangue on coal conveyor belts

Abstract

This invention relates to a method and system for detecting coal gangue on coal conveyor belts. The coal gangue detection method includes the following steps: using a MobileNetV2 model as a pre-trained model for coal gangue detection, and pre-training it using an open-source material conveying image set to obtain a pre-trained coal gangue model; acquiring a set of material conveying images from the coal conveyor belt, dividing the set into a training set and a test set, training the pre-trained model using the training set to obtain a coal gangue detection model, and verifying the effectiveness of the coal gangue detection model using the test set; and using the coal gangue detection model to detect coal gangue in real-time material conveying images of the coal conveyor belt. This disclosure improves the model's feature extraction performance and its ability to focus on important information, suppresses useless information, and improves the stability of model training.

Description

Technical Field

[0001] This invention relates to the field of coal mine transportation detection technology, specifically to a method and system for detecting coal gangue on coal conveyor belts. Background Technology

[0002] Coal is a crucial component of modern energy, but raw coal contains a significant amount of gangue after mining. This gangue reduces the combustion rate of coal, thus lowering energy efficiency and causing substantial environmental pollution. In coal mine production, conveyor belts serve as the primary coal transport channel, and ensuring their safe operation is paramount, as it directly impacts the efficient mining and transportation of coal. Gangue on high-speed conveyor belts can cause scratches and blockages. Therefore, accurate identification and classification of large pieces of gangue in raw coal transport on conveyor belts are essential.

[0003] Currently, most enterprises use manual sorting to screen coal gangue. This not only results in a harsh working environment, but also makes people prone to fatigue due to the high intensity and repetitive nature of the work. Relying on workers' subjective judgment can easily lead to problems such as missed selections and incorrect selections, and it is also difficult to guarantee work efficiency.

[0004] Existing technologies also include coal gangue detection techniques based on video surveillance image processing, mainly comprising target detection and image classification methods. Target detection methods first accurately locate foreign objects in the raw coal and then identify them; however, this requires labeling a large number of images, making the process cumbersome and computationally expensive. Image classification methods still suffer from difficulties in sample collection (limited data volume), poor robustness, and susceptibility to lighting and environmental factors. These factors can lead to image quality degradation and affect classification accuracy.

[0005] Therefore, it is particularly important to further research and develop detection methods for coal gangue from belt conveyors that are adapted to the special environment of underground coal mines. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for detecting coal gangue on a coal conveyor belt, thereby solving or at least alleviating one or more of the above-mentioned problems and other problems existing in the prior art.

[0007] To achieve the aforementioned objectives, a first aspect of the present invention provides a method for detecting coal gangue on a coal conveyor belt based on the MobileNetV2 model, wherein the MobileNetV2 model includes an inverted residual module, and a permutation attention mechanism is introduced into the inverted residual module. The coal gangue detection method includes the following steps: The MobileNetV2 model was used as the pre-trained model for coal gangue detection, and the pre-trained model for coal gangue detection was pre-trained using an open-source material transport image set to obtain the pre-trained model for coal gangue detection. A set of material conveying images of the coal conveying belt conveyor is obtained. This set includes images of material conveying with coal gangue and images of material conveying with normal coal flow. The set is divided into a training set and a test set, where the number of images N1 in the training set is greater than the number of images N2 in the test set. A coal gangue detection model is trained using the training set to obtain a coal gangue detection model. The coal gangue detection model is then tested using the test set to verify its effectiveness. The coal gangue detection model of the coal conveyor belt is used to detect coal gangue in the real-time material conveying image of the coal conveyor belt.

[0008] In the coal gangue detection method described above, optionally, the inverse residual module has sequential 1 x 1 up-dimensional convolution, k x k Dwise convolution, and 1 x 1 linear down-dimensional convolution. The permutation attention mechanism is implemented through the permutation attention module, which is introduced into the last layer of the 1 x 1 linear down-dimensional convolution of the inverse residual module. The permutation attention module groups the input material conveying image feature maps along the channel dimension of the material conveying image, rearranges each group of material conveying image feature maps into channel attention branches and spatial attention branches, processes them to obtain the channel dependency coefficients and spatial dependency coefficients of the material conveying image feature maps, and then integrates the material conveying image feature maps to realize the fusion of the channel attention mechanism and the spatial attention mechanism.

[0009] In the coal gangue detection method described above, optionally, after the channel attention mechanism and the spatial attention mechanism are fused, the channel attention output and the spatial attention output are weighted using a Hard-Sigmoid activation function, wherein... The input to the Hard-Sigmoid function is defined as x = F(.) = wu + b, where u is the channel statistics generated after permutation attention, w is the scaling operation on u, and b is the shift operation on u. The output of the Hard-Sigmoid function is defined as follows: .

[0010] In the coal gangue detection method as described above, optionally, the MobileNetV2 model comprises a sequential first-stage 3×3 convolution, inverted residual blocks from the second stage to the thirteenth stage, and a fourteenth stage composed of three layers of 1×1 convolution and a Softmax block.

[0011] In the coal gangue detection method as described above, optionally, the inverted residual blocks comprise a first inverted residual block SA-BottleNeck-s1 that short-circuits connects an input and an output, and a second inverted residual block SA-BottleNeck-s2 that does not short-circuit connect the input and the output, wherein the first inverted residual block SA-BottleNeck-s1 is adopted in the second stage, the eighth stage to the tenth stage, and the second inverted residual block SA-BottleNeck-s2 is adopted in the third stage to the seventh stage, and the eleventh stage to the thirteenth stage.

[0012] In the coal gangue detection method as described above, optionally, the open-source material conveying image set is an ImageNet data set, during the pre-training, an image size in the ImageNet data set is adjusted to 224 × 224, and a batch-size is set to 32.

[0013] In the coal gangue detection method as described above, optionally, N1:N2 is equal to 8:2.

[0014] In the coal gangue detection method as described above, optionally, after acquiring the material conveying image set of the coal conveying belt conveyor, an image preprocessing step is provided, the image preprocessing comprises performing enhancement processing on the material conveying image set, and the data enhancement processing comprises the following steps: defining P1 image enhancement operations; selecting P2 image enhancement operations from the P1 image enhancement operations, wherein P2<P1, randomly generating a probability and an amplitude for using the P2 image enhancement operations, which is referred to as one strategy, and generating P2 strategies in total; performing training and verification on the strategies in a verification data set, and selecting an image enhancement scheme with the highest accuracy.

[0015] To achieve the foregoing object, a second aspect of the present invention provides a coal gangue detection system for a coal conveying belt conveyor based on a MobileNetV2 model, wherein the coal gangue detection system comprises: a material conveying image acquisition module of a coal conveying belt conveyor, wherein the material conveying image acquisition module is configured to acquire a material conveying image during transportation of the coal conveying belt conveyor; The data processing device includes a trained coal gangue detection model for a coal conveyor belt. The coal gangue detection model is a MobileNetV2 model, which includes an inverted residual module and incorporates a permutation attention mechanism. The data processing device detects material conveying images acquired by the material conveying image acquisition module based on the coal gangue detection model.

[0016] In the coal gangue detection system described above, optionally, the inverse residual module has sequential 1 x 1 up-dimensional convolution, k x k Dwise convolution, and 1 x 1 linear down-dimensional convolution. The permutation attention mechanism is implemented through the permutation attention module, which is introduced into the last layer of the 1 x 1 linear down-dimensional convolution of the inverse residual module. The permutation attention module groups the input material conveying image feature maps along the channel dimension of the material conveying image, rearranges each group of material conveying image feature maps into channel attention branches and spatial attention branches, processes them to obtain the channel dependency coefficients and spatial dependency coefficients of the material conveying image feature maps, and then integrates the material conveying image feature maps to realize the fusion of the channel attention mechanism and the spatial attention mechanism. After the channel attention mechanism and the spatial attention mechanism are fused, the channel attention output and the spatial attention output are weighted using a Hard-Sigmoid activation function, wherein... The input to the Hard-Sigmoid function is defined as x = F(.) = wu + b, where u is the channel statistics generated after permutation attention, w is the scaling operation on u, and b is the shift operation on u. The output of the Hard-Sigmoid function is defined as follows: ; The MobileNetV2 model includes a first-level 3 x 3 convolution, second to thirteenth-level inverse residual modules, and a fourteenth-level module consisting of three 1 x 1 convolutions and a Softmax module. The inverted residual module includes a first inverted residual module SA-BottleNeck-s1 that short-circuit the input and output and a second inverted residual module SA-BottleNeck-s2 that does not short-circuit the input and output. The second stage and the eighth to tenth stages use the first inverted residual module SA-BottleNeck-s1, and the third to seventh stages and the eleventh to thirteenth stages use the second inverted residual module SA-BottleNeck-s2. The coal gangue detection system comprises an image preprocessing module. After acquiring the material conveying image set of the coal conveying belt conveyor, the image preprocessing module performs enhancement processing on the material conveying image set, and the data enhancement processing comprises the following steps: defining P1 image enhancement operations, selecting P2 image enhancement operations from the P1 image enhancement operations, wherein P2<P1, randomly generating the probability and amplitude for applying the P2 image enhancement operations, which is referred to as one strategy, generating P2 strategies in total, performing training verification on the strategies in a verification dataset, and selecting the image enhancement scheme with the highest accuracy.

[0017] The coal gangue detection method for coal conveying belt conveyor and the coal gangue detection system of the present invention adopt an improved MobileNetV2 structure, an improved permutation attention mechanism is introduced into the inverted residual block, and the Hard-Sigmoid activation function is adopted in a further technical solution. The above improvements enable the model to pay more attention to important information and suppress useless information, thereby improving learning efficiency and generalization ability, and effectively enhancing the performance of coal gangue detection in complex backgrounds of underground coal mines. Description of Drawings

[0018] The disclosure of the present invention will become more apparent with reference to the accompanying drawings. It should be understood that these drawings are only for illustrative purposes, and are not intended to limit the protection scope of the present invention. In the drawings: Figure 1 is a schematic flow chart of an embodiment of the coal gangue detection method of the present disclosure; Figure 2 is a specific schematic flow chart of an embodiment of the coal gangue detection method of the present disclosure, and the structural schematic diagrams of the SA-BottleNeck-s1 module and the SA-BottleNeck-s2 module are also shown in the figure; Figure 3 is a structural schematic diagram of the improved permutation attention module SA in the present disclosure; Figure 4 is Figure 2 is a schematic diagram of a learning rate change curve during training in ; Figure 5 is Figure 2 is a schematic diagram of training loss and test accuracy curves during training in ; and Figure 6 is a comparison diagram of visualization results between an embodiment of the coal gangue detection method of the present disclosure and the prior art MobileNetV2. Detailed Description of the Embodiments

[0019] Referring to the accompanying drawings and specific embodiments, the structure, composition, features, and advantages of the coal gangue detection method and coal gangue detection system for coal conveyor belts of the present invention will be described below by way of example. However, all descriptions should not be construed as limiting the present invention in any way.

[0020] Furthermore, for any single technical feature described or implied in the embodiments mentioned herein, or any single technical feature shown or implied in the various figures, the present invention still allows for any combination or deletion of these technical features (or their equivalents) without any technical obstacle, and thus these further embodiments according to the present invention should also be considered within the scope of this description.

[0021] Figure 1 This is a schematic flowchart of one embodiment of the coal gangue detection method disclosed herein.

[0022] from Figure 1 As can be seen, this detection method may include the following steps: Step S1: Use the MobileNetV2 model as the pre-trained model for coal gangue detection, and pre-train the pre-trained model for coal gangue detection using an open-source material transport image set to obtain the pre-trained model for coal gangue detection.

[0023] Step S2: Obtain the material conveying image set of the coal conveying belt conveyor. The material conveying image set includes material conveying images with coal gangue and material conveying images with normal coal flow. Divide the material conveying image set into a training set and a test set. The number of images N1 in the training set is greater than the number of images N2 in the test set. Train the coal gangue pre-training model using the training set to obtain a coal gangue detection model. Test the coal gangue detection model using the test set to verify the effectiveness of the coal gangue detection model.

[0024] Step S3: Use the coal gangue detection model of the coal conveyor belt to detect coal gangue in the real-time material conveying image of the coal conveyor belt.

[0025] In step S1 of this embodiment, the present invention detects coal gangue on a coal conveyor belt based on the MobileNetV2 model. The MobileNetV2 model can be used as a pre-training model for coal gangue detection in this invention. An open-source material conveying image set can be used to pre-train the coal gangue detection pre-training model, thereby obtaining the coal gangue pre-training model.

[0026] In an optional embodiment, the open-source material handling image set can be the ImageNet dataset. In other optional embodiments, it can also be obtained through other suggested methods based on the type of the target being detected.

[0027] In an alternative embodiment, to unify the output standard of material conveying images, the size of images in the ImageNet dataset can be uniformly adjusted to 224 × 224, and the batch-size is set to 32.

[0028] In step S2 of this embodiment, the material conveying image set of the coal conveyor belt is acquired. The material conveying image set of the coal conveyor belt may include material conveying images with coal gangue and material conveying images with normal coal flow. The acquisition can be performed by arranging a visual device during the material conveying process of the coal conveyor belt to collect the material conveying images, and the visual device can be, but is not limited to, a camera.

[0029] Meanwhile, in order to make the material conveying images clearer and easier for feature extraction, an image preprocessing step is provided after acquiring the material conveying image set of the coal conveyor belt. Image preprocessing may include, but is not limited to, graying, image enhancement, image smoothing or sharpening, denoising processing, image augmentation, etc. Image augmentation may include image transposition, horizontal mirroring, image rotation, image translation, scaling transformation, etc.

[0030] In an alternative embodiment, the image preprocessing may include performing enhancement processing on the material conveying image set, and the data enhancement processing includes the following steps: defining P1 image enhancement operations; selecting P2 image enhancement operations from the P1 image enhancement operations, where P2<P1, randomly generating the probability and amplitude of using the P2 image enhancement operations, which is referred to as one strategy, and generating P2 strategies in total; performing training and verification on the strategies in a validation dataset, and selecting the image enhancement scheme with the highest accuracy.

[0031] After image preprocessing, the material conveying images collected by the visual device can be divided, specifically into a training set and a test set, and the division ratio can be 8:2 in an alternative embodiment. A person skilled in the art can select and adjust an appropriate ratio according to the actual situation of the dataset and work requirements. The training set is used to train the coal gangue pre-training model described in step S1 to obtain a coal gangue detection model, and the test set is used to test the coal gangue detection model to verify the effectiveness of the coal gangue detection model. It should be noted that the number of images N1 in the training set should be greater than the number of images N2 in the test set, so that the training set can obtain sufficient computing power support during the training process of the coal gangue pre-training model, thereby reducing the test time and ensuring that the obtained coal gangue detection model is accurate and effective.

[0032] In step S3 of this embodiment, the coal gangue detection model of the coal conveyor belt in step S2 of this embodiment is introduced, and the coal gangue detection model is used to detect coal gangue in the real-time material transportation image of the conveyor belt, thereby solving the problem of coal gangue identification during the coal transportation process of the conveyor belt.

[0033] In summary, the pre-training used in this invention aims to improve the performance of the MobileNetV2 model, specifically the pre-trained coal gangue detection model. This is a transfer learning method. Because the number of real-time material conveying images of coal conveyor belts is insufficient, a large number of open-source material conveying image sets are first used to pre-train the coal gangue detection model to obtain a coal gangue pre-trained model. Then, a set of material conveying images of coal conveyor belts is acquired using a vision device. This set of images can be divided into training and testing sets, and the coal gangue pre-trained model is gradually trained and tested to obtain the coal gangue detection model for coal conveyor belts. Finally, this coal gangue detection model is used to detect coal gangue in real-time material conveying images.

[0034] Figure 2 This is a schematic flowchart of an embodiment of the coal gangue detection method disclosed herein, and the figure also shows the structural schematic diagrams of the SA-BottleNeck-s1 module and the SA-BottleNeck-s2 module.

[0035] The original MobileNetV2 model is a lightweight convolutional neural network model with the characteristics of small number of parameters, low complexity and fast inference speed. However, when faced with the complex background data of coal transportation in underground coal mines, the original MobileNetV2 model has problems such as insufficient feature extraction and easy neglect of detailed features, which will lead to a decrease in detection accuracy.

[0036] from Figure 2 As can be seen, this invention employs an improved MobileNetV2 model. This improved MobileNetV2 model solves the aforementioned problems, possesses powerful feature extraction capabilities in complex environments, and addresses the issue of the original MobileNetV2 model easily neglecting detailed features, making the preprocessing process of the coal gangue field dataset more stable and efficient. As shown in the figure, this MobileNetV2 model consists of sequential 3x3 convolutions, an inverse residual module, three layers of 1x1 convolutions, and a Softmax module.

[0037] In an optional embodiment, the open-source material conveying image set and the material conveying image set of the coal conveying belt are input into the MobileNetV2 model to pre-train and train the MobileNetV2 model.

[0038] The aforementioned 3x3 convolution is used to extract features from the open-source material conveying image set and the coal conveyor belt image set, thereby forming a feature map that can be used by the MobileNetV2 model. Next, the feature map can be input into the inverse residual module, through which the MobileNetV2 model can undergo deep learning and training. Then, the trained feature map is subjected to 1x1 convolution for feature extraction, and the extracted features are transmitted to the Softmax module. Finally, the Softmax module outputs the data after feature extraction.

[0039] The aforementioned inverse residual module can have sequential 1x1 up-dimensional convolutions, kxk Dwise convolutions (depthwise convolutions), and 1x1 linear down-dimensional convolutions. In the illustrated example, K=3. The permutation attention mechanism can be implemented through a permutation attention module, which is introduced into the last layer of the inverse residual module's 1x1 linear down-dimensional convolution. The permutation attention module groups the input material conveying image feature maps along the channel dimension of the material conveying image, rearranges each group of material conveying image feature maps (by splitting them through permutation units) into channel attention branches and spatial attention branches, processes them to obtain the channel dependency coefficients and spatial dependency coefficients of the material conveying image feature maps, and then integrates the material conveying image feature maps to achieve the fusion of the channel attention mechanism and the spatial attention mechanism.

[0040] exist Figure 2 In an optional embodiment, the inverse residual module enables the shallow network of the MobileNetV2 model to obtain gradients during training, thereby allowing the parameters of the shallow network to be trained, increasing feature representation capabilities, and effectively avoiding the gradient vanishing phenomenon; the 1 x 1 convolution makes the feature extraction network structure of the constructed MobileNetV2 model more suitable for coal gangue detection, while also reducing the number of model parameters and effectively reducing the computational loss of deep networks; the Softmax module is used to convert the input features into values ​​between 0 and 1, which can be interpreted as probabilities. When the input features are small, the softmax module transforms them into low probabilities, and when the input features are large, the softmax module transforms them into high probabilities.

[0041] Specifically, the inverted residual module includes SA-BottleNeck-s1 and SA-BottleNeck-s2. As shown in the figure, the input and output of SA-BottleNeck-s1 are short-circuited and the step size is set to 1, while the input and output of SA-BottleNeck-s2 are not short-circuited and the step size is set to 2.

[0042] exist Figure 2 In one embodiment, the distribution of the MobileNetV2 model specifically consists of a first-level 3x3 convolution, an inverse residual module (second-level SA-BottleNeck-s1, third to seventh-level SA-BottleNeck-s2, eighth to tenth-level SA-BottleNeck-s1, eleventh to thirteenth-level SA-BottleNeck-s2), a fourteenth-level 1x1 convolution, and a Softmax module. In other embodiments of the invention, the specific number and distribution of SA-BottleNeck-s1 and SA-BottleNeck-s2 and 1x1 convolutions in the inverse residual module can be adjusted by the amount of test data verified by those skilled in the art.

[0043] from Figure 2 It can also be seen that the inverse residual module has sequential 1x1 up-dimensional convolutions, kxk Dwise convolutions, and 1x1 linear down-dimensional convolutions. The 1x1 up-dimensional convolutional layer is used to up-dimensionalize the input feature map, thereby achieving the purpose of dilating the convolution. The kxk Dwise convolution is used to extract features from the feature map. The 1x1 linear down-dimensional convolution is used to reduce the number of channels in the feature map and output the feature map. In the illustrated example, k=3.

[0044] Furthermore, the MobileNetV2 model also includes a permutation attention module, which is introduced into the last layer of the inverse residual module's 1x1 linear dimensionality-reduced convolution. The introduction of the permutation attention module allows the MobileNetV2 model to focus more on important information and suppress useless information, thereby improving the model's learning efficiency and generalization ability. It also addresses the problem of total feature loss during training, providing a more comprehensive focus on feature information in both spatial and channel dimensions.

[0045] In optional embodiments, the permutation attention module is primarily based on a permutation attention mechanism, which includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism can model the input feature map, determine the importance of each channel in the feature map, and further enhance or suppress different channels for the detection target. Since this disclosure requires the detection of coal gangue in coal, the channel attention mechanism can enhance the channels for coal gangue in the feature map and suppress other channels in the feature map that are unrelated to coal gangue.

[0046] Simultaneously, the spatial attention mechanism can identify and process the most important parts in the feature map. Since this invention requires the detection of coal gangue in raw coal, the spatial attention mechanism can locate the coal gangue in the feature map and proceed with further processing. In summary, the channel attention mechanism enhances the channels for coal gangue in the feature map, while the spatial attention mechanism locates the coal gangue within the feature map.

[0047] Figure 3 This is a schematic diagram of the improved permutation attention module SA in this disclosure.

[0048] from Figure 3 As can be seen, C, H, and W are set as the number, height, and width of the image channels of the displacement attention module, respectively. Therefore, the material conveying image feature map Y input to the image channels of the displacement attention module can be represented as: , First, the displacement attention module groups the input material transport image feature map Y along the channel dimension of the material transport image, specifically dividing it into D groups. The resulting material transport image feature map Y is represented as follows: , Where D is a positive integer. The permutation attention module uses the sub-features formed after dividing the feature map Y to... The sub-features are represented as follows: It can be represented as: , Secondly, the permutation attention module SA contains permutation units, and the sub-features of feature map Y. The sub-features can be replaced by the substitution unit. Divide the attention into channel attention branch and spatial attention branch along the channel dimension, and apply the attention to each sub-feature. The dependencies in the spatial and channel dimensions are described.

[0049] Each sub-feature The input corresponds to a channel attention branch, which generates a channel attention image. This image is used to train the pre-trained coal gangue model and the coal gangue detection model, enabling them to better capture channel attention features. Simultaneously, each sub-feature... The input corresponds to a spatial attention branch, which can generate a spatial attention image to train the coal gangue pre-trained model and the coal gangue detection model so that they can better capture spatial attention features.

[0050] Specifically, sub-features Input the channel attention branch and spatial attention branch respectively. The channel attention branch first processes the sub-features at the input. The channel attention features are averaged and then averaged. The function enhances the channel attention features, and finally uses an activation function. The generation of channel attention weight coefficients activates channel attention features, thereby enabling better capture of these features when training pre-trained models and coal gangue detection models. Simultaneously, the spatial attention branch first adjusts the sub-features at the input end. Processing yields sub-features Spatial attention features, and then using The function enhances spatial attention features, and finally an activation function is used. By generating weight coefficients for spatial attention and activating spatial attention features, the spatial attention features can be better captured when training pre-trained models and coal gangue detection models.

[0051] In an optional embodiment, during the channel attention branch step, average pooling can extract the largest feature value in the window to reduce useless information. The channel statistics generated after average pooling can be expressed as: , Where H represents the height of the sub-feature, W represents the width of the sub-feature, a represents the change in H and a∈[1,H], and b represents the change in W and b∈[1,W].

[0052] In an optional embodiment, during the steps of channel attention branching and spatial attention branching, The function is represented as follows: , in, , Here are the learnable parameters. u is the channel statistic after average pooling, W is the scaling operation on u, and b is a constant representing the shift operation on u.

[0053] In an optional embodiment, the activation function in the channel attention branch and spatial attention branch steps is the Hard-Sigmoid activation function, the formula of which is as follows: , Among them, the Hard-Sigmoid activation function contains " The input value for " is as described above. The output value of the function is the activation unit of the Hard-Sigmoid activation function. This Hard-Sigmoid activation function is a linear activation function, which can effectively obtain cross-channel information. The Hard-Sigmoid activation function formula is specifically divided into three line segment formulas, each of which does not contain power or exponentiation operations. It features low computational cost and non-smoothness, exhibiting good performance when processing cross-channel information. Furthermore, compared to the ordinary Sigmoid activation function, the Hard-Sigmoid activation function can stably output 0 or 1 feature signals, effectively preventing the gradient vanishing phenomenon and allowing the feature map to complete the training process more effectively.

[0054] Specifically, the output features of the channel attention branch and the spatial attention branch, after being activated by the Hard-Sigmoid activation function, can yield the final channel attention output as follows: , in, It is a constant. and Used for Perform initialization and translation operations separately. The final spatial attention output is obtained as follows: , in, Used to obtain spatial channel statistics. and Used for statistical information of the space channel Scale and displacement operations are performed separately. Therefore, the channel attention output and the spatial attention output can be fused again through the permutation unit to obtain the output features of each sub-feature after training. The output feature of this sub-feature is .

[0055] Finally, the output features of each of the aforementioned sub-features are aggregated to form the trained feature map. Furthermore, component feature communication is performed through channel permutation operations, using the feature map trained by the permutation attention module. Produce the output. It should be noted here that the feature map Y input to the permutation attention module and the feature map after training by the permutation attention module SA... They are the same size.

[0056] In summary, the MobileNetV2 model consists of 3 x 3 convolutions, multiple bottleneck blocks, 1 x 1 convolutions, and a Softmax module. Open-source material handling image sets and material handling image sets from coal conveyor belts are input into the MobileNetV2 model. Feature maps are extracted through the 3 x 3 convolutions, and these feature maps are then input into the inverse residual module. After training in the inverse residual structure, the trained features are output through 1 x 1 convolutions. Finally, the Softmax module is used to obtain the trained coal gangue detection model, which can be used to detect coal gangue during the coal conveying process of coal mine conveyor belts.

[0057] In an optional embodiment, after obtaining the coal gangue detection model, it is necessary to use a test set to test the coal gangue detection model in order to obtain specific data of the trained coal gangue detection model.

[0058] First, the effectiveness of the coal gangue detection model needs to be verified. Specific verification methods can be implemented using evaluation metrics, including: accuracy, precision, recall, and F1 score. The formulas for calculating these metrics are as follows: Wherein, TP represents a true positive, indicating that the model correctly identifies a positive sample; FP represents a false positive, indicating that the model incorrectly identifies a negative sample as positive; FN represents a false negative, indicating that the model incorrectly classifies a positive sample as negative; and TN represents a true negative, indicating that the model correctly identifies a negative sample. Specifically, the positive sample is the image of the large coal gangue, and the negative sample is the image of the normal coal flow.

[0059] Then, after the coal gangue detection model has been verified, the actual material conveying image of the coal conveyor belt can be input into the coal gangue detection model to test the coal gangue detection model.

[0060] Figure 4 for Figure 2 A schematic diagram of the learning rate change curve during the training process.

[0061] In the pre-training and training process disclosed herein, the number of training iterations can be set to 100, the channel configuration to 1280, and the batch-size (the number of data samples captured in one training session) to 32. Before training, the image size of the coal gangue dataset captured by the vision device during the coal gangue transportation process is uniformly scaled to 224. Size: 224.

[0062] In other optional embodiments, those skilled in the art can also set the number of iterations to 200 or 300. If the convergence is fast during training, the number can be set smaller, and if the convergence is slow, it can be set larger. The batch size can be set to 8, 16, 32, or other numbers that are powers of 2, mainly depending on the performance of the computer. A batch size with good performance can be set larger. Finally, the optimal value is determined through multiple training sessions. These values ​​can be set according to experience and actual work needs.

[0063] In this embodiment, the initial learning rate can be set to 0.045, the weight decay can be set to 0.00005, the momentum coefficient can be set to 0.9, and the optimizer can be the SGD optimizer. It should also be noted that because a higher learning rate means a greater impact of output error on the parameters, faster parameter updates, and greater susceptibility to outlier data, the learning rate can be gradually reduced as the number of iterations increases. In other alternative embodiments, those skilled in the art can adjust the learning rate appropriately according to the specific circumstances.

[0064] exist Figure 4 In an optional embodiment, the number of iterations is set to 100, and the learning rate is updated every 3 iterations, with a decay factor of 0.92. The horizontal axis in the graph represents the number of iterations (Epoch), and the vertical axis represents the learning rate. As can be seen from the curve in the graph, after continuous training, the learning rate gradually decreases and eventually stabilizes. Therefore... Figure 4 This indicates that the accuracy of the coal gangue detection model gradually stabilizes after multiple training sessions.

[0065] Figure 5 for Figure 2 A schematic diagram of training loss and test accuracy curves during the training process.

[0066] exist Figure 5In an optional embodiment, the number of iterations is set to 100. The horizontal axis in the graph represents the number of iterations, the vertical axis on the side with 0 iterations represents the loss value during training, and the vertical axis on the side with 1 iteration represents the detection accuracy of the coal gangue detection model. As can be seen from the graph, the MobileNetV2 model gradually converges after continuous learning. Specifically, the graph shows that as the number of iterations increases, the curve of the loss value during training decreases sharply in the initial stage, and then gradually stabilizes after reaching a certain stage. This indicates that the gap between the coal gangue detection model trained on the coal gangue field dataset and the actual value gradually narrows; simultaneously, from... Figure 5 It can also be seen that the detection accuracy of the trained coal gangue detection model gradually improves with the increase of the number of iterations.

[0067] Figure 6 This is a visualization comparison of an embodiment of the coal gangue detection method disclosed herein with the prior art MobileNetV2.

[0068] from Figure 6 As can be seen, group (a) contains images of the coal gangue dataset captured by the vision device during the conveyor belt transport process; group (b) contains images of the original MobileNetV2 model detecting coal gangue in the images of group (a); and group (c) contains images of the MobileNetV2 model of this invention detecting coal gangue in the images of group (a) after training. It is evident that group (c) has a smaller labeled range for coal gangue detection and yields more detailed results, indicating that this invention has higher accuracy in detecting coal gangue in underground coal mines.

[0069] To achieve the above objectives, this disclosure also proposes a coal gangue detection system, which includes a material conveying image acquisition module for a coal conveyor belt and a data processing device.

[0070] In an optional embodiment, the material conveying image acquisition module may include a vision device, such as a camera, for acquiring images of material conveying during coal conveyor belt transportation. The data processing device is equipped with a trained coal gangue detection model for the coal conveyor belt, which is an improved MobileNetV2 model. This MobileNetV2 model includes an inverted residual module, and a permutation attention mechanism is introduced into the inverted residual module. The data processing device detects the material conveying images acquired by the material conveying image acquisition module based on the coal gangue detection model. During the detection of materials transported by the coal conveyor belt in the coal gangue detection system, the vision device in the material conveying image acquisition module can acquire a set of material conveying images of the coal conveyor belt in real time and input them into the data processing device. The data processing device can process the input set of material conveying images of the coal conveyor belt according to the trained MobileNetV2 model, thereby automatically marking the coal gangue during coal transportation on an external display device.

[0071] Based on the above, it can be understood that the technical solution of this invention proposes an improved MobileNetV2 structure for detecting coal gangue on coal conveyor belts. This effectively solves the problems of insufficient feature extraction and easy neglect of detailed features leading to recognition errors in existing MobileNetV2 technologies when facing complex background data in coal mines. This invention selects MobileNetV2 as the basic network skeleton and introduces an improved permutational attention (SA) mechanism in the inverse residual module.

[0072] In some technical solutions, the Hard-Sigmoid activation function is used for the channel attention output and spatial attention output of the SA module in this invention. This attention mechanism weights the feature maps under limited computational power, enabling the model to focus more on important information and suppress useless information, thereby improving learning efficiency and generalization ability. This improvement makes the model's feature extraction process more focused on spatial and positional information.

[0073] Furthermore, some technical solutions of this invention modify the feature extraction network of MobileNetV2 by replacing the last three BottleNeck layers with 1×1 regular convolutions. This adjustment not only reduces the number of model parameters but also lowers the computational loss of deep networks.

[0074] In summary, by introducing an improved SA attention mechanism and optimizing the feature extraction network structure, this invention yields a model that pays more attention to details, spatial and location information, effectively improving the performance of coal gangue detection in complex underground coal mine backgrounds. Specifically, some technical solutions of this invention address the following technical problems: the original MobileNetV2 suffers from insufficient feature extraction in complex underground coal mine background data, which may lead to recognition errors; the original MobileNetV2 may ignore detailed features when processing complex scenes, reducing the model's accuracy. Furthermore, the following beneficial technical effects were achieved: an improved SA attention mechanism was introduced into the inverse residual module of MobileNetv2, which uses the Hard-Sigmoid activation function to weight channel and spatial attention, thereby improving the model's focus on important information and suppressing useless information, thus improving learning efficiency and generalization ability; the MobileNetV2 feature extraction network was modified by replacing the BottleNeck in the last three layers with 1×1 regular convolutions, reducing the number of model parameters and the computational loss of deep networks, thereby improving the feature extraction process; by improving SA and modifying the structure of the last layer, the model pays more attention to spatial and positional information, effectively improving detection performance in complex backgrounds in coal mines.

[0075] The technical scope of this invention is not limited to the contents of the above specification. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the scope of this invention.

Claims

1. A method for detecting coal gangue on a coal conveyor belt based on the MobileNetV2 model, characterized in that, The MobileNetV2 model includes an inverted residual module, and a permutation attention mechanism is introduced in the inverted residual module; The inverse residual module has sequential 1x1 up-dimensional convolutions, kxk Dwise convolutions, and 1x1 linear down-dimensional convolutions. The permutation attention mechanism is implemented through the permutation attention module, which is introduced into the last layer of the inverse residual module's 1x1 linear down-dimensional convolution. The permutation attention module groups the input material conveying image feature maps along the channel dimension of the material conveying image, rearranges each group of material conveying image feature maps into channel attention branches and spatial attention branches, and processes them to obtain the channel dependency coefficients and spatial dependency coefficients of the material conveying image feature maps. Then, the material conveying image feature maps are integrated to achieve the fusion of the channel attention mechanism and the spatial attention mechanism. After the fusion of the channel attention mechanism and the spatial attention mechanism, the channel attention output and the spatial attention output are weighted using the Hard-Sigmoid activation function, where the input of the Hard-Sigmoid activation function is defined as x=F(.)=wu+b, where u is the channel statistics generated after permutation attention, w is the scaling operation on u, and b is the displacement operation on u. The output of the Hard-Sigmoid activation function is defined as follows: ； The coal gangue detection method includes the following steps: The MobileNetV2 model was used as the pre-trained model for coal gangue detection, and the pre-trained model for coal gangue detection was pre-trained using an open-source material transport image set to obtain the pre-trained model for coal gangue detection. A set of material conveying images of the coal conveying belt conveyor is obtained. This set includes images of material conveying with coal gangue and images of material conveying with normal coal flow. The set is divided into a training set and a test set, where the number of images N1 in the training set is greater than the number of images N2 in the test set. A coal gangue detection model is trained using the training set to obtain a coal gangue detection model. The coal gangue detection model is then tested using the test set to verify its effectiveness. The coal gangue detection model is used to detect coal gangue in real-time material conveying images of the coal conveyor belt.

2. The method for detecting coal gangue as described in claim 1, characterized in that, The MobileNetV2 model includes a first-level 3x3 convolution, second to thirteenth-level inverse residual modules, and a fourteenth-level module consisting of three 1x1 convolutions and a Softmax module.

3. The coal gangue detection method as described in claim 2, characterized in that, The inverted residual module comprises a first inverted residual module SA-BottleNeck-s1 that performs short-circuit connection between input and output, and a second inverted residual module SA-BottleNeck-s2 that does not perform short-circuit connection between input and output, wherein the second stage, the eighth stage to the tenth stage adopt the first inverted residual module SA-BottleNeck-s1, and the third stage to the seventh stage, the eleventh stage to the thirteenth stage adopt the second inverted residual module SA-BottleNeck-s2.

4. The method for detecting coal gangue as described in claim 1, characterized in that, The open-source material conveying image set is an ImageNet data set, and during the pre-training, the image size in the ImageNet data set is adjusted to 224 × 224, and the batch-size is set to 32.

5. The method for detecting coal gangue as described in claim 1, characterized in that, N1:N2 is equal to 8:

2.

6. The method for detecting coal gangue as described in claim 1, characterized in that, After obtaining the material conveying image set of the coal conveyer belt, an image preprocessing step is provided, the image preprocessing comprises performing enhancement processing on the material conveying image set, and the enhancement processing comprises the following steps: defining P1 image enhancement operations; selecting P2 image enhancement operations from the P1 image enhancement operations, wherein P2<P1, randomly generating the probability and amplitude of using the P2 image enhancement operations, which is called one strategy, and generating P2 strategies in total; performing training verification on the strategies in a verification data set, and selecting the image enhancement scheme with the highest accuracy.

7. A coal gangue detection system for a coal conveyor belt based on the MobileNetV2 model, using the coal gangue detection method as described in claim 1, characterized in that, The coal gangue detection system comprises: a material conveying image acquisition module of a coal conveyer belt, wherein the material conveying image acquisition module is configured to acquire material conveying images during transportation by the coal conveyer belt; a data processing device, wherein a trained coal gangue detection model for the coal conveyer belt is arranged in the data processing device, the coal gangue detection model for the coal conveyer belt is a MobileNetV2 model, the MobileNetV2 model comprises an inverted residual module, a permutation attention mechanism is introduced into the inverted residual module, and the data processing device detects the material conveying images acquired by the material conveying image acquisition module based on the coal gangue detection model for the coal conveyer belt.

8. The coal gangue detection system according to claim 7, characterized in that, the inverted residual module has sequential 1×1 dimensionality-increasing convolution, k×k Dwise convolution and 1×1 linear dimensionality-reducing convolution, the permutation attention mechanism is implemented by a permutation attention module, the permutation attention module is introduced into the last 1×1 linear dimensionality-reducing convolution of the inverted residual module, the permutation attention module groups an input material conveying image feature map along a channel dimension of the material conveying image, rearranges each group of the material conveying image feature map into a channel attention branch and a spatial attention branch, processes to obtain a channel dependence coefficient and a spatial dependence coefficient of the material conveying image feature map, then integrates the material conveying image feature map, so as to realize fusion of a channel attention mechanism and a spatial attention mechanism; After the fusion of the channel attention mechanism and the spatial attention mechanism, the output of the channel attention and the output of the spatial attention are weighted by using the Hard-Sigmoid activation function, wherein the input of the Hard-Sigmoid activation function is defined as x=F(.)=wu+b, wherein u is a channel statistic generated after permutation attention, w is a scale operation performed on u, and b is a displacement operation performed on u, the output of the Hard-Sigmoid activation function is defined as: ； the MobileNetV2 model comprises a sequential first-stage 3×3 convolution, inverted residual modules from the second stage to the thirteenth stage, and a fourteenth stage composed of three layers of 1×1 convolution and a Softmax module; the inverted residual modules comprise a first inverted residual module SA-BottleNeck-s1 that short-circuits connects an input and an output, and a second inverted residual module SA-BottleNeck-s2 that does not short-circuit connects an input and an output, the first inverted residual module SA-BottleNeck-s1 is adopted in the second stage, the eighth stage to the tenth stage, and the second inverted residual module SA-BottleNeck-s2 is adopted in the third stage to the seventh stage, and the eleventh stage to the thirteenth stage; the coal-gangue detection system is provided with an image preprocessing module, after acquiring a material conveying image set of the coal conveyer, the image preprocessing module performs enhancement processing on the material conveying image set, and the enhancement processing comprises the following steps: defining P1 image enhancement operations, selecting P2 image enhancement operations from the P1 image enhancement operations, wherein P2<P1, randomly generating the probability and amplitude of using the P2 image enhancement operations, which is called a strategy, generating P2 strategies in total, performing training verification on the strategies in a verification data set, and selecting an image enhancement scheme with the highest accuracy.

Images